In a previous article, I discussed managing risk with quality gates (“None Shall Pass…unless? Managing Risk with Quality Gates”). Such gates and their expectations facilitate tracing issue root cause and examining which preventative measures failed or need to be enhanced so as to continue to lower costs associated with poor quality.
But that last gate is there to enforce a level of standard, or ‘quality bar’, for what is ultimately to be published, deployed, or otherwise seen outside of the project team itself. To guard that gate, we need “release criteria”.
Assuming that you won’t be checking against these criteria unless you have passed all prior gates, release criteria are the last few things that need to be ‘just so’ to make your release a success. Or more bluntly, meeting these last criteria is required to allow you to make a release at all.
Go/No-Go – A Rare Binary Decision
What is important at the point of making a release? As a team, we need to be sure that both the technical and business risks associated with the value we intend to provide have been mitigated to acceptable levels given the purpose or intended usage of the release.
How do we evaluate whether something has been ‘mitigated’? As a team, we need to describe a set of criteria in a quantifiable way such that they can be confirmed or not by a simple ‘yes’ or ‘no’.
Black, White, and a dab of Grey
To keep the focus clear and create the best chance for a release to make it out the door without compromising our standards, we can separate release criteria into two categories; what MUST be achieved before a release can happen and what is EXPECTED to be achieved.
The following are the minimum criteria that MUST be met prior to a given build being released. If these criteria are not met, the build will not be approved for release and corrective action must be undertaken and given the highest priority.
| EXAMPLE: Minimum Release Criteria |
| All unit tests or peer reviews of code changes must be documented and pass |
All “Fix Today” priority issues raised during testing of the series of builds leading up to the release or found during the release process are confirmed resolved (i.e. none are outstanding) via testing.Here a “Fix Today” priority issue is defined as an issue that meets one or more of the following conditions:
|
| Every testable defect fix has been confirmed by testing to be implemented successfully |
| All acceptance tests must pass |
| The release notes will document if a particular change request or new feature is included in the build but has known open issues against it at the time of release |
The following are the additional criteria that are EXPECTED, but not absolutely required to be met for a given build prior to it being approved for release. If any of these criteria are not met, corrective action will be undertaken following a risk/impact assessment of the unmet criteria (eg: non-conformances will be logged as issues of varying priority).
This provides the project team the ability to react to other situations should they be determined to have a higher priority at the time. However, it would be expected that corrective action would be undertaken in time for the following release.
| EXAMPLE: Expected Release Criteria |
| Features that Product Managers/Customers have been promised are implemented |
| Every testable change request or new feature that has been implemented has been confirmed to be implemented successfully and confirmed as such through verification of documented expected capability and behaviours |
| No spelling mistakes in the screens within screens or documentation modified by change requests, new features, or fixed defects (e.g. typos or inconsistencies found in any other screen can be automatically added as a “Fix Later” issue) |
| All change requests, new features, or fixed defects implemented in the build are included in the release notes |
Going Back for More
If we return to the idea of quality gates throughout the project lifecycle, we can add another dimension to release criteria.
Suppose we consider the promotion of a build from one environment to another a ‘release’ and, as such, establish criteria for this maturing of the system or code-base towards the final gate.
The following diagram provides an example of what this might look like at the highest level:
Conclusion
If the project team has a ‘quality bar’ based on criteria that are practical and quantifiable, you will benefit from having a repeatable standard for quality in your releases. Then challenge yourself to be even better. Your customers will love it.
