Virtually any book dealing with software testing mentions that user acceptance testing (UAT) is an ultimate validation activity, often quoting Boehms informal definition: “Validation: Are building the right product?”
But how come that Boehm himself considers UAT a verification?
GUIDELINES FOR VERIFYING AND VALIDATING SOFTWARE REQUIREMENTS AND DESIGN SPECIFICATIONS.
B.Boehm:
From the V-chart, it is clear that the key artifact idstinguishes
verification activities from validation activities is the software
requirements baseline. This refers to the requirements specification
which is developed and validated during the Plans and Requirements
Phase, accepted by the customer and developer at the Plans and
Requirements Review as the basis for the software development
contract, and formally change-controlled thereafter.By definition,verification involves the comparison between the requirements baseline
and the successive refinements descending from it — the product
design, detailed design, code, data base, and documentation — in order
to keep these refinements consistent with the requirements baseline.
Thus, verification activities begins in the Product Design Phase and
conclude with the Acceptance Test. They do not lead changes in the
requirements baseline; only to changes in refinements descending from
it.On the other hand, validation identifies problems which must be
resolved by a change of the requirements specification. Thus, there
are validation activities which occur throughout the software
life-cycle, including the development phase. For example, a simulation
of the product design may establish not only that the design cannot
meet the baseline performance requirements (verification), but also
that the performance requirements are too stringent for any
cost-effective product designs, and therefore need to be changed
(validations).
The only explanation I can think of would be that Boehm did not consider Acceptance test as a USER acceptance test but as some kind of final check against the specification.
In addition, I believe this original article every book refers to is much clearer than those artificial definitions of V&V (that are, in addition, wrong from this point of view). According to Boehm, when I check the products against the requirements, it is a verification. When I test the product for its use, I validate it as I might find something that leads to changes in the requirements.
Of course Boehm’s original idea contradicts some standards, such as ISO 12207, in which validation is to confirm that the requirements for a specific intended use of the software work product are fulfilled. Which means checking against the spec.
Most books says that verification is only static and the code is not run. I believe that according to Boehm, it is not true. E.g.:
Software Engineering And Quality Assurance, A.A.Puntambekar: The verification activities fall into the category of static testing.
Effective Methods for Software Testing: Includes Complete
Guidelines,William E. Perr: Verification testing – testing in a static
mode – ….. Validation testing (e.g. unit, integration …)
I would conclude that despite many authors’ opinions, I believe VERIFICATION includes testing which is done in order to check the correspondence of the specification and the program. Validation includes testing as well but it would not be based on the specifications.
14
In the opening sections of the paper that you quote, Boehm states:
There is very little agreement in the software field on the
definitions of the terms “verification” and “validation.”
Boehm defines the terms verification and validation as he intends to use them in the rest of the paper, in order to be able to clearly present the rest of the writing without ambiguity:
Verification – to establish the truth of the correspondence between a
software product and its specification. (Note: This definition is
derived from the Latin word for “truth,” veritas. Note also that data
bases and documentation are fit subjects for verification, as well as
programs.)Validation – to establish the fitness or worth of a software product
for its operational mission. (Note: This definition is derived from
the Latin word for “to be worthy,” valere.)
I think that the Guide to the Software Engineering Body of Knowledge can provide some insight into this as well:
Classical treatments suggest that “verification” deals with static
evaluation methods and that “testing” deals with dynamic evaluation
methods. Recent treatments, including ISO/IEC draft 29119, are
blurring this distinction, though, so testing standards are mentioned
here.
What all of this says is that, at the time Boehm wrote his paper, there wasn’t a widely accepted singular definition for what is “verification” and what is “validation”. Even today, that’s still true.
Now that we know that there is some ambiguity in the terms “verification” and “validation” (and how “testing” fits into these), we can look at the diagram that Boehm presented.
Something that Boehm didn’t discuss, but I’ve seen in multiple sources (including ISO/IEC 12207), validation specifically requires that the validation activities occur in the operational environment and can achieve its intended purposes, from the user’s perspective. This may be implied by Boehm’s definitions, but it’s not clearly stated.
Boehm also used the term “acceptance testing”. You are interpreting this as “user acceptance testing”. There are other kinds of acceptance testing, such as factory acceptance testing (FAT). In my experience, a FAT is performed by the developing organization at the developing organization’s site, with the procedures reviewed and accepted by the customer. This is a check to ensure the complete system (all software and hardware) are fully functioning, meet the specification, and a final inspection prior to delivery to the customer’s facility. Based on Boehm’s definitions, this can be a verification activity the developing organization can not fully exercise the product within the operational context.
What you are considering “user acceptance testing” is likely “OT&E” – Operational Test and Evaluation. This is indeed a validation activity since the customer and user are able to see the product within the operational context and say if it does or does not meet their needs.
3
It might be useful to examine the testing artifacts that ISO 12207 demands under Verification and Validation:
Verification
7.2.4.3.2.3 Code verification.
The code shall be verified considering the criteria listed below:
a) The code is traceable to design and requirements, testable,
correct, and compliant with requirements and coding standards.b)
The code implements proper event sequence, consistent interfaces,
correct data and control flow, completeness, appropriate allocation
timing and sizing budgets, and error definition, isolation, and
recovery.
This describes Unit Testing and Static Analysis, among other things. Unit Testing and static analysis are below the red requirements baseline; they are verification activities.
7.2.4.3.2.4 Integration verification.
The integration shall be verified considering the criteria listed below:
a) The software components and units of each software item
have been completely and correctly integrated into the software item.b) The hardware items, software items, and manual operations of the
system have been completely and correctly integrated into the system.
That’s Integration Testing, still part of the Verification process.
Validation
7.2.5.3.2.3 [Testing Includes]:
a) Testing with stress, boundary, and singular inputs;
b) Testing the software product for its ability
to isolate and minimize the effect of errors; that is, graceful
degradation upon failure, request for operator assistance upon stress,
boundary, and singular conditions;c) Testing that representative
users can successfully achieve their intended tasks using the software
product.
Testing of this kind is seldom embodied in the original requirements specification. It normally arises as a result of user testing and feedback, among other things. It rightly belongs above the Requirements Baseline, and is part of the Validation process.
ISO 12207 does not appear to make a distinction between “static” and “dynamic” testing.
As to Acceptance Testing, my view is that if a requirement is a valid requirement, it is testable. Acceptance testing is, therefore, the process of verifying the requirements via tests, and belongs below the Requirements Baseline as part of the Verification process.
Accordingly, changes to the software requirements that arise as a result of user studies belong above the line, as part of the Validation process.