A Test Strategy is a documented approach to testing where the test effort, test domain, test configurations, and test tools employed to verify and validate a set of functionality are defined. It also includes information on schedules, resource allocations, and staff utilization. This information is crucial to allow the test team (Test) to be as organized and effective as possible.
A Test Strategy is not the same as a Test Plan, which is a document that collects and organizes test cases by functional areas and/or types of testing in a form that can be presented to the other teams and/or customer.
Both are important pieces of the Quality Assurance process since they help communicate the test approach scope and ensure test coverage while improving the efficiency of the testing effort.
What is in the Test Strategy?
The following is a list of some of the sections that are typically included in the Test Strategy document.
- Introduction – contains an overview of the project, lists related documents and references, document conventions, and assumptions made in the course of creating the strategy.
- Scope – describes the scope of the test team’s involvement in the project; describes the test areas Test is responsible for and why. It also defines the areas for which Test is not responsible.
- Resources & Schedule for Test Activities – describes the resources available and their roles. Includes a schedule overview for the project, making sure the estimated time for the testing activities and milestone dates are present. The build schedule can also be included if available.
- Acceptance Criteria – defines the minimum criteria that a product must achieve before it is shipped.
- Test environment – describes the hardware and software platforms that are used for testing, including Client/Server configuration, Network, etc…and what will be tested in each platform.
- Tools – describes the tools used for test case management, defect reporting and test automation.
- Test Priorities – describes the priorities of the test effort during the test planning, test automation, test data creation, and test execution phases.
- Test Planning – describes such activities as requirements review and test analysis to determine a list of suitable tests required for verification of the product. It also describes how the tests are expanded into full test cases, complete with descriptions, reproduction steps, and expected results.
- Executing a Test Pass – describes how the test pass execution is performed, and when the testing is executed, in accordance with the types of testing to be performed. For example, test cases that are critical are tested first to ensure the build has the minimum functionality required before further testing.
- Types of testing to be performed – defines the different types of testing to be performed, and the extent to which Test will be carrying out each type of testing. The most common types of testing types are:
- Build Verification Tests
- Functionality Testing
- User Interface Testing
- Usability Testing
- Error Handling
- System Platform
- Stress Testing
- Performance Testing
- Installation Testing
- Print Testing
- Localization Testing
- Regression testing
- Risks and Issues – lists of outstanding risks and issues related to the test effort.
Benefiting from the Test Strategy
The main groups that benefit from having a Test Strategy are the testing team, development and project management, but other groups such as user ed and marketing can also benefit from the information contained in the Test Strategy.
- Test Team: The Test team will follow the Test Strategy and make sure testing is performed in accordance with the plan. They will also analyze the results and make recommendations on the quality of the functionality. The Test Strategy document should help the Test Team answer the questions below:
- Do I have all documentation I need to start the test planning?
- Is the time scheduled for testing planning adequate?
- Do I have the tools to develop the test cases? To log defects?
- Who is going to review the test analysis/ test planning and when?
- Do I have all I need to start testing (equipment/tools)?
- Do I have all the data/files I need to start testing?
- Do I know the functionality I will test on each build?
- Is all functionality being covered during all phases?
- What are the procedures if I find a serious defect?
- Development: Development will understand the functionality being tested and the conditions under which these tests are to be performed. The questions that the Test Strategy document should answer are:
- What is the overall approach to testing on this project?
- Who is responsible for the different types of testing, particularly Unit and Integration testing?
- Do I have time scheduled for reviewing the test plans for depth of coverage?
- Is the test environment adequate relative to the intended production environment?
- Project Management: Project Management will understand the information regarding configurations (hardware and software) under which the product was verified and validated, and the procedure for assessing the quality of the product based on the type of testing being performed. They are also informed about the testing schedule and its impact on the deadlines. The Test Strategy should help with the following questions:
- Do I need to hire more people during the test planning or testing phase?
- Do we have all the hardware and software required for testing?
- Do we have the tools required for test planning and defect reporting?
- If a new tool is required, is the time needed for training the testing team scheduled?
- Are all types of testing defined as required?
- Are all the testing tasks well defined?
- Are the testing priorities clear for each phase of the project?
- Are there enough test execution passes for each phase of the project?
- What are the issues and risks identified by Test that are still outstanding?
There is another important document whose purpose is very often confused with the Test Strategy or Test Plan; and that is the QA Plan. The QA Plan is intended to be a high level document that clearly outlines the boundaries of QA’s responsibilities on the project relative to the rest of the project personnel, including any clients, sub-contractors, and co-contractors.
The QA Plan includes descriptions of methodologies, practices, standards, quality requirements, metrics, reviews and audits, code control, media control, etc., in addition to outlining the basics of the responsibilities of the Test Team.
The Test Strategy draws upon this parent document and its information, if available, and further details the responsibilities of the Test Team and its approach to testing.
For related reading, check out these articles:
