Exploratory testing is a testing approach and mindset. In this post I compare exploratory testing with two different testing approaches:
Scripted testing
Try to break it testing
Scripted testing
Scripted testing provides a series of checking points for a specific part of a feature. For example, a script might tell you to select the print button and expect print dialogue opens. You might go on to check the default number of pages to print and expect the default is set to "All".
Scripted testing captures a pass or fail for each test:
Pass: It met the expected result
Fail: It did not meet the expected result
But what happens when a specific test does something beyond the boundaries of the expected result? Do you pass or fail it? An item under test isn’t so black and white and in the grey lies problems and ideas waiting to be discovered.
Scripted testing takes time to plan and document. Much of this planning and documentation doesn’t provide immediate feedback about the item under test. It’s only during a test execution phase where scripts are put to use and something is discovered. So why not immediately explore, even without a script? What’s stopping you? Surely you’ll find something interesting even without a script telling you what to do and expect! Go evaluate what’s actually in front of you instead of what a document says it’s supposed to do. You could use try to break it testing.
Try to break it testing
Discard the scripts and use your judgement to explore with the intention of breaking the item under test. Discovery is an alternative way of thinking about breaking. You might discover part of a feature is broken but you didn’t actually break the feature. With a discovery approach you follow your instincts to learn what happens. You discover a bug with the orientation feature of the print dialogue box, something the script wouldn’t have asked you to check. You let the developer know immediately. Fast feedback and hopefully a fast fix for retesting!
But how do you know you’re testing the most important thing? Where do you choose to start? How do you evaluate how much testing is enough? When do you decide to stop exploring? Scripts give you that sense of organisation and priority. They also give you a set of activities with a stopping point. Try to break it testing (or ad-hoc testing) has no explicit prioritisation or focus.
Exploratory testing
Exploratory testing provides the organised feel of scripted testing along with the rapid feedback of try to break it testing. Identifying what to test next is where exploratory testing comes into its own – by using structure and fast feedback.
Risk areas help focus exploratory testing efforts. Risk areas are things that might be troublesome to the desired outcomes of customers, users and business stakeholders. Risks-to-explore are assumptions and these assumptions might present themselves as actual risks but only if we take the time to run focused exploration to find out. Use risks to guide where and what to explore and you'll provide your team with an ongoing evaluation of test coverage.
So how do you get started? Define a test exploration goal (also referred to as a “charter”) using risks as your trigger. Don’t stop at one goal, come up with many so you and your team can decide which goal to explore first. Choose an amount of time you'd like to spend exploring each test exploration goal. Use a timebox to provide focus and a clear stopping point. Any timebox from 30 to 90 minutes works best.
đź’ˇTry this
Grab a physical notepad and pen or open up an application that allows you to type notes, such as Google Docs, Word or Evernote.
Open up this simple little Parking Calculator app.
Write down a simple goal for a 10 minute exploration of the date picker e.g. Explore the date picker of the Parking Calculator application to learn about it.
Start a 10 minute timer and start exploring!
Write down everything you observe, specifically focusing on the date picker. It’s ok if you wonder onto other parts of the app, make a note and come back to the date picker.
Continue to write down everything you observe even if it feels a little odd doing so. What are you thinking? What do you see? Have you noticed anything that might be a problem? What test ideas are triggered? What makes you do this and that?
Stop at 10 minutes and resist the urge to continue.
Review your notes. How was it? What was it like to explore in this way? What surprised you? How would you use your notes to share your discoveries with your team?
Discover more!
A test exploration goal and timebox is an excellent alternative to following a test script or attempting to break an item under test by playing with it. The balance of structure and freedom is liberating! The output of every goal-driven timeboxed exploratory testing session provides a team with a useful starting point for a conversation. And these conversations help a team answer one of their most important and ongoing questions: What do we need to work on next?
Or better still: What shall we learn next?
