Exploratory testing provides both flexibility and speed, characteristics that have become increasingly important, especially with the quick pace of short agile iterations. But, how do you retain traceability in exploratory testing without losing your creativity? How do you, as a manager, actually manage testing that is unscripted and improvised? One answer is to use a combination of session-based test management (SBTM) and thread-based test management (TBTM) called xBTM.
Session-based Test Management (SBTM)
SBTM (ref: Jonathan Bach, 2000, http://www.satisfice.com) is the reply to the common misconception that exploratory testing is, by its very nature, always unplanned, undocumented and unmeasurable. Using SBTM, exploratory testing is done in time-boxed sessions. Each session has a mission that is specified in the test charter, for example, the sessions are actually planned ahead (but not scripted). A session results in a session report that provides a reviewable record of the testing that was done. Based on the session report, the test manager can derive various metrics to track the progress of the testing. SBTM doesn’t have much in common with ”ad hoc” testing, in fact – SBTM is quite strict, sometimes even too strict.
Thread-based Test Management (TBTM)
There are environments too chaotic or hectic for SBTM to work properly, and this led to the introduction of TBTM (ref: James Bach, 2010, http://www.satisfice.com). TBTM is a generalization of SBTM that allows for more freedom and flexibility. The core of TBTM is threads and, similarly to sessions, threads are test activities intended to reach a certain test goal. The big difference is that whereas a session is a commitment to complete a specific task in a given amount of time, threads come with no such commitments. A thread can be interrupted, resumed, canceled or go on indefinitely. SBTM defines both where to start and where to end, but TBTM only defines where to start. In TBTM, activities are allowed to change over time. TBTM is the free-spirited sister of SBTM.
SBTM -> TBTM -> xBTM
SBTM and TBTM both have their strengths and weaknesses, and rather than having to choose one over the other, xBTM was created (ref: http://www.addq.se/utforskande-testmetodik-xbtm). xBTM unites the two exploratory techniques to get the full advantage of both, focusing on increased test efficiency and creation only of artifacts that actually add value. The name xBTM highlights that it is combination of SBTM and TBTM (x = S or T).
The xBTM workflow is centered around a mind map:
1. List all test ideas and activities – these are your threads.
2. Arrange all threads in a mind map grouped by function area or test technique. This constitutes the test plan.
3. Prioritize the threads.
4. Use SBTM when possible. Group threads to create suitable charters and create session reports once the charters are executed.
5. Use TBTM when SBTM is not an option. Follow the threads where they take you and add new threads when needed.
6. Update the mind map continuously, showing the progress and state of the threads. The mind map is the status report.
Using a mind map, rather than a traditional text document, makes conveying the test plan easier, and it helps communication — not only within the test team but across the whole project team. Figure 1 shows a simplified example of what the xBTM mind map for testing an e-shop might look like.
Figure 1: Example of an xBTM mind map. Here, an e-shop is being tested and the mind map shows some of the threads. For example, “Create user account” is being tested in a session, whereas “Add item to shopping cart” is being tested as a thread.
Want more?
Want to know more about xBTM? A detailed walk-through of applying xBTM on a real project is given here: Follow-up on xBTM.
Also, look for our workshop ”Exploratory Testing on Agile Projects” at STAREAST in 2013.
Links
There are many different mind map tools to choose from and, in most cases, the core features are very similar. Which one you end up picking will mainly be a question of personal taste. The author has had very good experiences using the following three tools:
- XMind – http://www.xmind.net: A powerful tool with a good selection of icons and markers. The base version is free and there is a pro version available for purchase. XMind was used to create the example in this article.
- Mindmeister – http://www.mindmeister.com: Collaborative tool that stores mind maps online and allows multiple users to edit the same mind map simultaneously. The free version limits the number of mind maps you can create and store.
- FreeMind – http://freemind.sourceforge.net/wiki/index.php/Main_Page: The simpler of the mind mapping tools, but still very useful and completely free.
Additionally there are a number of tools that help you manage your session-based testing. Most of these tools focus on recording the session reports. Three tools which the author has used in the past are:
- Rapid Reporter – http://testing.gershon.info/reporter: A note-taking tool for exploratory testing sessions.
- Session Tester – http://sessiontester.openqa.org: A tool for recording and managing exploratory testing session.
- SBTExecute – http://www.addq.se/utforskande-testmetodik-xbtm (scroll down to bottom of page for English): A tool that produces summary reports and calculates metrics from an Excel session report template.
