What is a Test Strategy?
A test strategy is the top-level artifact that gives visibility to how testing will be approached for a given project, as agreed to by the stakeholders, typically addressing:
- Purpose / scope
- Quality criteria
- Assumptions & constraints
- Test approach
- Inclusions / exclusions
- Types of testing
- Issues & risks
Quality Criteria and Acceptance
Quality criteria are those things that must be true for acceptance to be possible. They can be derived from risks, scope, and organization standards (“quality bar”)
How do we know, or have confidence, that the quality criteria have been met for a given release? And, is 80% confident “good enough”? 90%? 95%? 99.99999%?
Testing Objective/Motivation/Priorities
As testing is virtually always constrained with not enough time, not enough people, not enough infrastructure, etc it is vital that the testing effort is able to prioritize what is critical, and what is not, so the scarce resources can be applied in the most valuable manner possible.
One of the crucial components to being “smart” in a constrained situation is to be able to correctly decide or select which things are “must-haves”, which are “should-haves”, and which are “nice-to-haves”.
Having a mission statement, such as the following, can help keep your focus on what is important and reveal what is less-so, and give visibility of the same to the rest of the organization:
“The mission of the project’s Test Team is to undertake such test-related activities within the constraints provided to them by the organization to maximize mitigation of the likelihood of a failure of the software system from occurring after deployment that would impact the business.” *
* This example is for an embedded test team on an enterprise scale project. Tailor your own mission statement to fit your team’s scope of responsibility for quality within your project and organization.
Finding Your Best Option
In developing your test strategy, you might use a table like the following to compare the different flavours or options that you identify:
| Comparison Criteria | Option A | Option B | … |
| Cost (during project) | |||
| Coverage (breadth & depth of testing) | |||
| Schedule required (elapsed time) | |||
| Resources required (avg / max burn rate) | |||
| Future re-use of test assets |
Tailor the above list of comparison criteria to your needs, perhaps including others like: test auditability, repeatability of test steps, early involvement/feedback, manual vs. automated tests ratio, test preparation vs. test execution effort ratio, Total Cost of Quality (slide 5), etc.
Note: In creating your options, you are really trying to find the “best” option. Therefore, any initial option can be “looted” for pieces to be merged with others on the way to creating that final, agreed approach.
Test Planning Framework
Consider the following steps in your test planning process:
- Review the project scope, delivery schedule, and constraints
- Determine if/how testing activities are able to assess and/or mitigate the risks (to quality, the technical solution, and the project itself)
- Capture the needs of the stakeholders in the testing effort
- Confirm the test artifacts required and the level of formality
- Identify available team members, tools, environments, etc.
- Outline reasonable options for the project’s test strategy
- Support the options with test effort estimates and test project schedule/resourcing data
- Present test strategy options for review and input by project team and stakeholders
- Iterate on feedback and converge to the selected/agreed approach
Gaining and maintaining this agreement between the stakeholders for the duration of the project (as things progress and change) will enable testing activities to be appropriately prioritized so as to successfully test the right things at the right time within the project constraints.
Summary
The Test Strategy, the Test Estimate, and the Test Project Plan are all co-dependent on each other which causes a cyclical dependency between the three artifacts. ie: a change in any one of the three artifacts should result in a review and possible update of the others.
And, when these three artifacts are driven by risk, the activities captured will be strategically prioritized, making it straightforward to decide what tasks could be most confidently dropped if there was a sudden need to do so.
For related reading, check out these articles:
