I Say “Quality”! You Say…?

Quality is one of those words that embodies so much and can mean such different things to different people.

high quality guarantee
Image credit: Pixelors.com

For instance:

  • Why do you have the car that you have?
  • Would you change it if you could (money is no object)?  Why?
  • What if you had to buy a car for a friend?
  • What if you had to choose a car for 100 people and you would be evaluated on their happiness or satisfaction with the car after 1 week?  Or after 1 year?  5 years?  15 years?
  • What if you manufactured cars for the mass market?  What would you ensure that your cars had so that potential buyers would perceive your cars to be “of quality”?  What would your company have to be like?

! If we were talking about movies instead of cars, would your answer to “what is quality” change?

What is Quality for Software Systems?

Strawberry is the answer!
Maybe.  Is it a gelato scenario?
What?  No, we are talking about frozen yogurt!
We have to go with blueberry then.

Discussions about what is quality in relation to software have been going on for a long time.  Some well-referenced definitions include:

  • “…conformance to requirements: meeting customer expectations, both stated and unstated.” – Philip Crosby, 1979
  • “…the degree to which a set of inherent characteristics fulfill requirements.” – PMI Project Management Body of Knowledge, 2008
  • “Fitness-for-use.” – Joseph Juran, 1974

However, quality has many aspects or dimensions to it.  For example: When we start to think about software and quality, we might think about concepts like capability/functionality, performance, usability, compatibility, maintainability, testability, etc.

These quality attributes/factors, or ‘ilities, are used to describe the different aspects of quality for a given software system.

From a customer satisfaction point of view:

“Satisfaction with the overall quality of the product and its specific dimensions is usually obtained through various methods of customer surveys.  For example, the specific parameters of customer satisfaction in software monitored by IBM include the CUPRIMDSO categories (capability, functionality, usability, performance, reliability, installability, maintainability, documentation / information, service, and overall); for Hewlett-Packard they are FURPS (functionality, usability, reliability, performance, and service).” – Stephen H. Kan, “Metrics and Models in Software Quality Engineering”, 2nd Edition.

Of course, for them to be useful we would like to be able to assess/measure these quality attributes/factors; to track and to compare.

Evaluating Software Quality

We need to improve quality.
Why do you say that?
Isn’t it obvious?

In order to rationally take steps to improve quality, we need to first understand which quality attributes are important to us.  And then, we need to be able to associate one or more ways to measure each quality attribute so as to be able to quantify it in an agreeable manner.

A first step is to take each quality attribute or quality factor, that is of interest to us, and describe them in terms of reasonable sub-factors.  The following diagram illustrates this decomposition for the quality model defined in ISO/IE25010:2011:

Product Quality Model ISO/IEC 25010
Image credit: “System Quality Requirement and Evaluation”, Kazuhiro Esaki

Next, we can define appropriate metrics (with their associated measures, indicators and thresholds) to evaluate/assess the degree to which our software system possesses each quality sub-factor (or quality factor in the case of direct metrics).

Quality Factors To Metrics Tree

This will provide us a model from which we can make more objective comparisons and judgements about the degree to which our system possesses a given quality attribute and, when the attributes are considered as a group, to what degree our system is “of quality”. (paradoxical caveat: at least insofar as we have defined quality through our selection of metrics)

Who Cares About Quality?

Hey I found a bug!
Oh?  Show me.
Yeah…don’t worry about that.  It’s not important.
To who?

Before we set off to write up “What is Quality” and define the quality model for our organization or project, we should really involve some other people in our team (in the broadest sense).  Recall, quality can mean different things to different people, and that is largely because they hold different aspects of quality to be more or less important than another, given their context.

Consider: it would be a limited view indeed to simply consider the testers on the team as the only stakeholders in defining quality.  Likewise, when considering only the customer or end-user.

“Quality is everyone’s responsibility.” W. E. Deming

Stakeholder input should be gathered from:

  • Customer representatives
  • Technical “Peers” (architecture, business analysts, developers, testers, operations, support)
  • Management (project management, departmental management)
  • Business Organization (strategic planning, corporate management)

For each stakeholder identified from the groups above, it is important to understand their contextual interest in quality:

  • What are the needs of their role, the pressures or demands they face, and from whom?
  • What are their motivations or goals for being involved in the project?
  • How will they participate in the project?
  • How can understanding what is agreed to be quality help them?

It Just Got Real

Fast! Easy to use! Reliable!
How do I turn that into code?

We can then take our understanding of our stakeholders’ quality needs and purposefully build it into our software.  The following diagram illustrates translating the stakeholders’ needs for different aspects of quality into actual (eg: testable) functional and non-functional requirements for the software system.

Quality Requirement Definition Analysis ISO/IEC 25030
Image adapted from: “System Quality Requirement and Evaluation”, Kazuhiro Esaki

Turning It Around

We can’t be perfect.
We can improve.
But it costs too much.
What will it cost if we don’t?

Just as poor quality in a released product hurts market share, the lack of quality within each phase of the development lifecycle costs the company hard, quantifiable, dollars.

Quality In Product Lifecycle ISO/IEC 9126
Image credit: “ISO/IEC SQuaRE. The second generation of standards for software product quality”, Witold Suryn, Alain Abran

Identifying what can lead to poor quality downstream, lets us see what we need to change upstream in our processes to mitigate this risk, lower the Total Cost of Quality, and realize those savings and revenues.


“What is quality?” is a key part of the conversation regarding “what is success” on a given project, for a particular system, in a specific organization, for those users.

Having this question answered lets us plan and prioritize on our project at the beginning, throughout, and on-going for the life of the software system.

Make sure you have it.


About Trevor Atkins

Trevor Atkins has been involved in 100’s of software projects over the last 20+ years and has a demonstrated track record of achieving rapid ROI for his customers and their business. Experienced in all project roles, Trevor’s primary focus has been on planning and execution of projects and improvement of the same, so as to optimize quality versus constraints for the business. LinkedIn Profile
This entry was posted in  All, Planning for Quality and tagged , , , . Bookmark the permalink.