“Bad User” Testing

How can you be sure that an application will behave properly when users perform actions or combinations of actions that were not considered during the development of the functionality? During the testing phase, you have to plan for what is sometimes called “bad user” testing, or negative testing.

Boris Beizer’s definition of negative testing in “Software Testing Techniques” is: “Testing aimed at showing software does not work”.

In his paper, “A Positive View of the Negative Testing”, James Lyndsay stated the objectives of bad user testing to be:

  • Discovery of faults that result in significant failures; crashes, corruption and security breaches
  • Exposure of software weakness and potential for exploitation
  • Observation and measurement of a system’s response to external problems

Why Is Bad User Testing Important?

During negative or bad user testing, the tester seeks to abuse the functionality of the product in an effort to create odd program states by exercising functionality that deals with state management, input validation, boundary conditions, fault recovery, and more.

Bad user testing is generally performed as part of integration or system testing and does not have a distinct phase of its own. The basic principle to follow is if the tester can perform bad user tests without error, there is a significantly lower chance that the users will inevitably find such defects later on.

As Lyndsay points out in is his paper, negative testing can find significant failures and also can produce invaluable strategic information about the risk model underlying testing, and allow overall confidence in the quality of the system.

Where To Start?

It is possible to design a bad user test plan starting from the specification documentation. The most important thing to keep in consideration when designing a bad user test plan is to not test what is described, but what is not. The tester should look at the specification documentation as a guide to what the boundaries of the software and then look beyond the boundary to the extreme edges of the software’s functionality. Employ your creative destructiveness to take you beyond these boundaries and perform actions and tasks that you are sure will fail, or should be impossible. These are the areas where the most interesting defects can be found.

Of course, each product is specific and all tests cannot be applied in all circumstances. The few brief examples below can be used as a guide when starting a bad user test plan. Use your creativity to add to and expand this list.

Generic Bad User Test Scenarios

What is expected behaviour when you:

  • Manually shut down, or reboot, the computer while the application is running
  • Manually restart the computer by pushing the reset button while the application is running
  • Restart the computer while the application is running using the start menu
  • Log off while the application is running

Boundary Test Scenarios

What is expected behaviour when you:

  • Attempt to go below the minimum input limits
  • Exceed the maximum input limits

Stress Test Scenarios

What is expected behaviour when you:

  • Run the program concurrently with many other programs
  • Set memory to a stressed state (low Virtual Memory, low RAM)
  • Run on slower or older machines
  • Load large volumes of data
  • Create large numbers of concurrent connections or open files

Performance Test Scenarios

What is expected behaviour when you:

  • Test on a system lower than the minimum configuration requirements
  • Open extremely large files
  • Attempt to import a corrupt file
  • Create a situation requiring an error message

Install/Uninstall Test Scenarios

What is expected behaviour when you:

  • Run the installation script again while the application is running
  • Cancel the installation midway through the install using the task manager
  • Install the application onto a hard drive without enough free space
  • Install the application onto a network drive and disconnect the drive partway through
  • Uninstall the application when the application is still running
  • Change or remove the registry settings before trying to uninstall the application

Summary

Bad user testing is performed by the tester to find weaknesses in the design or the code by attempting actions that are likely to occur after deployment. Although it is hard to make predictions about real-world use, real users will find all ways of using the system including those that were considered unreasonable or improbable.

Including this type of testing as part of your project will result in a more robust and user-friendly application and will of course save effort and costs after the product is released.

Avatar

About Trevor Atkins

Trevor Atkins (@thinktesting) has been involved in 100’s of software projects over the last 20+ years and has a demonstrated track record of achieving rapid ROI for his customers and their business. Experienced in all project roles, Trevor’s primary focus has been on planning and execution of projects and improvement of the same, so as to optimize quality versus constraints for the business. LinkedIn Profile
This entry was posted in  All, Test Planning & Strategy and tagged , . Bookmark the permalink.