Going To The Extreme – xBTM

Now that the project has finished it is time to sum up my experiences of adapting Thread-Based Test Management (TBTM). Since I generally do not believe in rigorously adhering to a protocol, I ended up not using TBTM strictly, but instead embraced a hybrid of Session-Based Test Management (SBTM) and TBTM. Naturally this hybrid will be denoted xBTM.

The project
In order for this text to make sense, a few words on the project are needed. The customer runs an application that generates output which is stored daily in a single XML file, and later used by the customer’s own applications to derive vital business statistics. There were defects that needed correction, and the customer also asked for new features regarding how log on/log off was recorded. The corrections as well as the new requirements should be implemented in a new file that was supposed to be generated in parallel with the old file for a transition period.

The team
It was a small project. I acted as project manager, test manager and tester. Later in the project I was joined by a second tester. There was one single developer.

TBTM
Most of the time my working environment in general is simply too hectic and borderline chaotic for SBTM to be suitable. There tends to be a lot on interruptions and distractions, and it is rarely possible to sit down and focus on a single test task for a given time period. Therefore I decided to give TBTM a try when a new project started.

My first step was to make a mind map containing all my test ideas as threads. Since I am very fond of open source products I used the software FreeMind. In the mind map below, e.g. File Name is a thread. XML file is the actual product. ID denotes a defect, and CR denotes a new change request. The test threads are grouped in two ways. Most groups represent key areas, e.g. Generation which means file generation. Other groups are formed based on the type of testing, e.g. Stress testing. In some cases I felt short notes were needed to explain the thread, and in those cases I attached text files – knots – to the thread. These files are shown as red arrows in the mind map. You click on the arrow to open the text file.

Test PlanTest Plan

I tried using colours and icons to make the mind map easier to read. The warning signs mark key areas that I judged to be high risk and especially important. The stop light marks a thread that was tied off. That particular change request was retracted by the customer just before the project started. The initial mind map as it was before I started testing made up my test plan and was sent to the customer.

During the test period I would constantly be updating the mind map and it would always give me an accurate picture of the current status of the testing. As soon as I started working on a thread I would mark it with a smiley, see image below. Threads where I found defects were marked with a red cross, and threads where I felt sufficient testing for delivery had been done (not the same thing as claiming to be done testing!) were checked off in green.

TestSnapshot of mind map in the middle of the test period

Initially my goal was to write daily status reports containing a few short notes on what I had done. In reality I only kept this up for four days. With my constantly updated mind map and the occasional session report (see below) I really did not feel a need for it.

When there was no more time for testing, I took the current status of my mind map and used it as my test report, see below. This test report was sent to the customer.

Test ReportTest Report

SBTM
I do still like SBTM and I find the session reports as well as the time-boxing very useful, so when circumstances would allow, I created test charters and ran time-boxed test sessions. Typically a couple of threads would make one session, but in come cases one thread deserved a session of its own. For example, looking at the test plan the two threads File name and File header belonging to the key area File generation would be tested in the same session, whereas Install from scratch and Install upgrade belonging to the key area Installation were tested in separate sessions.

In most cases I would draw a simple activity diagram – pattern – for each test charter, see example below. I prefer using yEd for my diagrams.

Log on/offTest Charter

I wrote short session reports according to a new AddQ session report template, see image below.

SBTM ReportSBTM session report template

Using the AddQ tool SBTExecute I could then derive metrics such as Total session time (for all sessions), Number of test charters, Number of test charters per key area, and so on from my session reports. However, since I was still experimenting in this project, as well as working on the template and the tool as I went along, I could not obtain any useful statistics. Also note than any metrics derived would only be valid for my SBTM sessions – and a considerable part of the testing was done according to TBTM.

SBT ExecuteSummary report generated by SBTExecute

Summary and Conclusions
What is the main difference to previous projects I have worked on? Well…the first thing that comes to mind is that this time I actually used the test plan! And I have read and come back to the test report. Keeping the mind map up to date has been much easier than updating any other kind of status document, which means that it actually has been updated. As it turned out, even the developers liked it – they preferred looking at the mind map over using our bug tracking tool!

I immediately liked the combination of TBTM and SBTM – it is a simple matter of applying the methodology that is best suited for the task at hand. Some threads were fairly isolated and straight forward to test, making them suitable for SBTM. Other threads were very convoluted and had to be explored together with the developer, making them better suited for TBTM.

Christin Wiedemann

About Christin Wiedemann

Christin Wiedemann (@c_wiedemann) is the Co-CEO and Chief Scientist of PQA Testing. After finishing her PhD in Physics at Stockholm University, Christin Wiedemann started working as a software developer for the Swedish consulting company HiQ. Christin soon discovered that software testing was more interesting and challenging than development and subsequently joined the Swedish test company AddQ Consulting. At AddQ, she worked as a tester, test lead and trainer, giving courses on agile testing, test design and exploratory testing throughout Europe. Christin developed a course on exploratory testing, and is a co-creator of the exploratory testing approach xBTM. Christin currently lives in Vancouver, where she joined Professional Quality Assurance (PQA) Ltd. in 2011. In her current role as Chief Scientist, she drives PQA’s research and method development work. She continues to use her scientific background and pedagogic abilities to develop her own skills and those of others. LinkedIn Profile
This entry was posted in  All, Agile Testing, Automation & Tools, Test Planning & Strategy and tagged , , , , . Bookmark the permalink.