Yesterday my question How come verification does not include actual testing? created a lot of controversy, yet did not reveal the answer for related and very important question: does black box functional testing done by testers belong to verification or validation?
- ISO 12207:12208 here mentiones testing explicitly only as a validation activity, however, it speaks about validation of requirements of the intended use. For me its more high level, like UAT test cases written by business users
- ISO mentioned above does not mention any specific verification (7.2.4.3.2)except for Requirement verification, Design verification, Document and Code & Integration verification. The last two can be probably thought as unit and integrated testing. But where is then the regular testing done by testers at the end of the phase?
The book I mentioned in the original question mentiones that verification is done by static techniques, yet on the V model graph it describes System testing against high level description as a verification, mentioning it includes all kinds of testing like functional, load etc.
In the IEEE standard for V&V, you can read this: Even though the tests and
evaluations are not part of the V&V processes, the techniques described in this standard may be useful in
performing them. So that is different than in ISO, where validation mentiones testing as the activity.
Not to mention a lot of contradicting information on the net. I would really appreciate a reference to e.g. a standard in the answer or explanation of what I missed in the ISO. For me, I am unable to tell where the testers work belong.
1
Part of the challenge here is that I think you’re misreading the ISO specification.
7.2.4 Software Verification Process
7.2.4.1 Purpose
The purpose of the Software Verification Process is to confirm that each software work product and/or service of a process or project properly reflects the specified requirements.
As has been answered in some of your other questions – Verification is making sure the requirements are defined correctly based upon the business’ needs.
7.2.5 Software Validation Process
7.2.5.1 Purpose
The purpose of the Software Validation Process is to confirm that the requirements for a specific intended use of the software work product are fulfilled.
Validation is being defined here as making sure the software actually does what the requirements say they should, i.e. the software’s correct operation fulfills the requirements. This is the colloquial Testing phase.
So your comment of: however, it speaks about validation of requirements of the intended use. For me its more high level, like UAT test cases written by business users is a misunderstanding of what the ISO doc is specifying. Validation can be cursory high-level acceptance tests, but it can also be lower level, deep testing of the product.
Section 7.2.5.3.1.2 explicitly says to figure out how much testing is needed. That’s the meaning behind: a validation process shall be established
Section 7.2.5.3.1.4 hints at what should be tested but is really just saying to document what is to be tested as part of the validation plan.
IMO, The ISO doc is pretty light on the particulars of testing, so I’ll refer to SWEBOK for some additional details:
- SWEBOK Software Testing Breakdown of Topics The graph here does a decent job at showing the scope of testing.
- SWEBOK Test Levels Goes into more details about unit | integration | system testing, the objectives, and techniques.
One thing to note is that SWEBOK appears to use the terms validation and verification interchangeably, which is precisely what you’re trying to avoid in this question. SWEBOK wasn’t written with the degree of precision that an ISO standard is. I would argue that it’s a forgivable lapse of precision, but I understand you may not feel that way at this point. Semantically, validate and verify are very similar terms. Using the context of the development phase is appropriate to determine the correct, precise term.
The answer to your ultimate question of where the testers work belong[s?] is that their effort belongs in the Validation portion of the development. The ISO doc could have been more clear in their meaning of requirements ... are fulfilled. But that’s the answer you are looking for to your question — Testers and testing fall into section 7.2.5 Software Validation.
2
Note: Whilst the question asks about ISO 12207, and calls for Authoritative Sources, the ISO standard is copyright, and whilst there are copies available on the ‘net, I have deliberately not linked to one…
Terminology time:
- Verification can be summed up by “Are we building the software correctly”
- Validation can be summed up by “Are we building the right software
Within ISO 12207 these are two distinct processes (para 7.2.4 and 7.2.5). These paragraphs focussing on two distinct activities – proving the process, and proving the product. As such (IMHO) “testing” (within the normally accepted meaning) is a validation only activity.
Within the specifics of your question, and note my emphasis:
7.2.4.3.2.3 Code verification. The code shall be verified considering the criteria
listed below:
a) The code is traceable to design and requirements, testable, correct, and compliant
with requirements and coding standards.
Note: testable, not tested
Verification ensures that the correct processes and standards have been followed to generate the requirements, do the design, write the code and released.
Whereas, as part of the Validation activity:
7.2.5.3.2.3 Conduct the tests in subclauses 7.2.5.3.2.1 and 7.2.5.3.2.2 {…}
Testing is a validation activity, to prove that the software is correct and does what it is supposed to do, both in terms that the requirements are met, but also that the requirements reflect what is really required.
In my opinion.
3