REQUIREMENT TESTABILITY

To help discover untestable requirements, ask yourself how you would prove a requirement has been met. Either rewrite untestable requirements or find proxies for them. See examples of proxies or read the definition.

Planning tests during requirements helps improve the requirements, which reduces overall effort and cost by minimizing rework. In addition, getting testers' input will greatly improve your requirements skills. The following can make a requirement untestable.

Most Adjectives

Adjectives can make meaning open to interpretation.

Adjectives in requirements transfer work to other project phases. For example, "all" is one of the most abused adjectives in requirements. "All" does not protect a client or ensure completeness: using "all" means the analytical work is incomplete. The unfinished analytical work is transferred to subsequent phases of the SDLC. How would you plan for, implement, or test "the system must calculate all payments" without knowing what specific payments are involved? Later in the SDLC, others must complete the analysis outside of the analysis process and hope they’re right or annoy clients with additional questions. The client feels cheated. In fact, complete, traceable requirements are what give insight into and control over the contents of the system and support accurate planning and cost estimating for subsequent phases—not vague adjectives like "all."

Some other adjectives that make a requirement untestable are: robust, safe, accurate, timely, effective, efficient, correct, expandable, flexible, interoperable, maintainable, modular, portable, reliable, user friendly, adequate, better, and state-of-the-art. When these appear, usually in a statement of work, proxies or rewriting are needed to define what the client actually wants.

Problem requirement: The system shall be written in the approved language.

Discussion: What language? Who approves it?

Correction: The system shall be written in C++. This is a restrictive requirement, and why it is needed determines the content of the requirement. For example, if this requirement exists to ensure compatibility with another system, then it should address compatibility instead of requiring a specific language. On the other hand, for a statement of work, specificity in a requirement might be needed so contractors can create proposals and bid qualified personnel.

Most Adverbs

Adverbs and adverbial phrases can make meaning open to interpretation. Examples are: quickly, safely. Some are impossible to prove: always, never.

Problem requirement: The system shall produce the ABC report in a timely manner.

Discussion: "Timely" needs to be defined in accordance with the needs of the organization.

Problem requirement: The system shall require users to select only one of the following...

Discussion: The placement of adverbs can cause problems. In this requirement, "only" can mean the one action available to users is "select," or it can mean users are limited to one item from the list. The correction is not adding more words ("one and only one"). The correction is deleting "only," which is an extraneous word.
Problem requirement: The system shall automatically do X."

Discussion: If there is an event that triggers this system action, it may need to be stated. Otherwise, automated systems are, by nature, automated, so the adverb is unnecessary.

Next page

Page updated 5/22/2005
Ludwig Consulting Services, LLC