Is exploratory testing only for senior testers?
A while ago I answered the following question on SearchSoftwareQuality.com’s Ask The Software Quality Expert: Questions & Answers:
It’s been my experience that only the senior testers I work with do exploratory testing well. Is experience a requirement for exploratory testing?
Here is a clip from my answer:
Experience isn’t a requirement for exploratory testing, but certainly, as with anything, experience can help. An "experienced" Java programmer (whatever that means) will (in theory) write better code then a Java programmer out of college. That doesn’t mean the Java programmer out of college can’t write code — he can. He just doesn’t have the same experience to draw on.
The same is true with exploratory testing. Experience can be good, but it isn’t required. Notice I said can be, because the wrong experience can be bad.
You can find the full posting here.
Part of that posting lists the skills and tactics used by exploratory testers. It’s also worth noting that version 1.6 of the list is out and posted on the Satisfice website. It has some stylistic changes to it.
Hi Mike, nice post. I agree with all of the skills listed above, although I might not be able to remember all the big words. ;)
There’s one more important skill that I have found to be critical in helping someone become a good exploratory tester — the ability to *recognize* a bug when it’s staring you in the face.
The list of skills you list above are all key in good exploration and investigation. For exploratory *Testing* to be effective, the person needs to know how to identify (important) bugs so that they can report them.
I have found that many ‘experienced’ testers don’t always recognize their own biases and assumptions when testing and therefore miss a lot of important bugs. In fact, many ‘experienced’ testers take offense at the question when I ask them if they know how to recognize a bug. Sigh. Denial of the existence of personal biases, or lack of awareness of issues like inattentional blindness, is a sure way to ensure that you won’t be good at ET.
Another key element in making someone successful in ET is good feedback. For example, I interviewed a smart candidate out of college once *with actual ET work experience*(!) but I turned them down. Their past experience had a ‘test lead’ that provided negative feedback that discouraged effective investigative skills and bug reporting due to office politics. The end result was to take a smart kid and make them dumb. What a waste.
A good test lead/mentor/coach/senior tester will understand all of the skills you listed above (and more) and help a newbie fast-forward their professional development by providing good, effective, timely feedback.
Take for example one of the youngest members of my test team right now. He now knows that when I ask him a question while reviewing one of his (ET) session reports, he’s not expected to answer the question right there. The point of asking questions is to spark more questions and new ideas. .. To encourage him to think and expand his understanding of the system and the problem.
There are an infinite number of questions that could be asked on any test project. Are you prepared to address/investigate any of them? A good exploratory tester will be.. regardless of past ‘experience.’
Embrace the unknown! =)
Cheers!
Thank you Mike and Paul for sharing your tips on exploratory testing!
I think that if the tester possesses the required discipline of mind, exploratory testing can be quite fruitful.
Thanks again!
Inder
Paul is totally right. At our shop we hire co-op students and after running, maybe, a cycle or two of manual tests, we openly allow them to start in on exploratory testing. We routinely find that each discovers bugs differently and we actually use that as a means of refreshing our own knowledge.
If I routinely lost my keys, for instance, it might take me an hour to find them the first time, but the next time I go searching for them, it’s likely that I will immediately check the place I found them last time. As time goes on, I may have accumulated 6 or 7 or 8 places where I have lost my keys. So as long as they’re in one of those places then I am pretty safe. If they’re not then chances are I will spend an hour looking for my keys again. Testing is no different, it’s very much a psychological exercise. So experience can be a good thing or it can be a bad thing.
Exploratory testing is designed to be most effective when you try new things. It’s the whole reason for not structuring steps into the charters. However, when we are on a deadline, we will tend to adhere to what has worked before, much like biases that Paul mentions above.
We actually conduct daily meetings which I unofficially call “brag sessions” which allows our senior testers, junior testers and co-op students alike share information in an open forum. If you found a particularly stubborn bug, then brag about how you tracked it down.
I think that communication and continunous learning are the key factors to making ET work. If you have, even a senior, who isn’t willing to open themselves up to new methods of doing something or new paths to arrive at a common result, then they won’t be effective for you. If you have a junior who only wants to perform monkey testing, because they’re afraid of what they’re missing if they don’t stick to a plan, then they probably won’t be right for it.
I think that the biggest thing to wrap your head around is that what’s on paper, the manual test cases that are written down, are a single path through the software. A very linear one at that. So whether you’re experienced or not, following a manual test suite will serve only to prove whether or not the person who wrote the test cases fully understood the application. Nothing more. If you can understand that there are, seemingly, infinite paths through an application, then I think that experience in actual testing is a learned skill that can be acquired very easily. Whereas, if this simple concept can’t be grasped, you are ultimately doomed to fail if you attempt to tackle ET.
Brent, Thanks for the comment. I like the idea of the brag sessions. Seems like a cool way to do a group debrief.