Fundamentals of Testing


What is Testing?

The process used to assess the quality of computer software to provide stakeholders with information about the quality of the application, product or system, with respect to the context in which it is intended to operate.


Why is Testing Necessary?

- Reduce the risk of problems occurring during operation (production)

- To give confidence in the quality of software

- Meet contractual or legal requirements

- Provide stakeholders with information about the quality of the application, product or system, with respect to the context in which it is intended to operate


Principles of Testing

Principle 1 - Testing shows presence of defects


Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness.


Principle 2 - Exhaustive testing is impossible


Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, risk analysis and priorities should be used to focus testing efforts.


Principle 3 - Early testing


To find defects early, testing activities shall be started as early as possible in the software or system development life cycle, and shall be focused on defined objectives.


Principle 4 - Defect clustering


Testing effort shall be focused proportionally to the expected and later observed defect density of modules. A small number of modules usually contains most of the defects discovered during prerelease testing, or is responsible for most of the operational failures.


Principle 5 - Pesticide paradox


If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new defects. To overcome this 'pesticide paradox', test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to find potentially more defects.


Principle 6 - Testing is context dependent


Testing is done differently in different contexts. For example, safety-critical software is tested differently from an e-commerce site.


Principle 7 - Absence-of-errors fallacy


Finding and fixing defects does not help if the system built is unusable and does not fulfill the users needs and expectations.


Independent Testing


The effectiveness of finding defects by testing and reviews can be improved by using independent testers. For large, complex or safety critical projects, it is usually best to have multiple levels of testing, with some or all of the levels performed by independent testers.


The benefits of independence include:


  • Independent testers tend to find more and different types of defects
  • Independent testers are unbiased and have no hidden agendas
  • Independent testers can verify assumptions people made during the specification and implementation of the system