I am writing about testing and I am confused about the following:
What exactly is Black-Box testing?
Are functional testing and Usability testing two different types of testing and are they types of Black Box testing?
I have to perform two different types of testing procedures. Is it ok if I do functional and Usability testing?
1
Black-box testing means that the tester is not familiar with the inner workings of the system, while white-box testing is performed by someone who knows exactly how the program works. Both has advantages and disadvantages.
Functional testing is testing whether or not the software product fulfills functional requirements, like “when loading an incorrectly formated file the program must output an error message” or “when clicking on the X, the program must close”. The results of such tests are a binary “works” or “does not work”.
Usability, however, is difficult to describe with functional requirements. It is hard to tell in advance if a certain user-interface would be easy to use or not. You have to build it, let people test it, and listen to their subjective feedback. What might be easy to use for one tester might be hard to use for another. That means a simple “is usable” or “is not usable” distinction is not useful in this case. The test results will usually be a lot more free-form and open to interpretation.
Also, usability testing is not useful to systematically find defects like functional testing does, because usability testers are encouraged to use the product how they would use it in the real-world and not intentionally try to perform irrational actions trying to break it. It’s not like usability testers are discouraged from filing bug reports when they stumble upon any obvious bugs (they do affect usability, after all), but that’s not their primary concern. They are also not expected to file bugs when the product does not behave according to the functional requirements, but the result is actually preferable.
Black-box testing as opposed to white-box testing is a broad category of testing that includes tests where you do not know, and do not need to know, how the implementation works, you just want to test the outcome.
For example: I click the buy button on a web site and the item I selected is now in my shopping cart. I don’t care how the programmer made this happen, just that it did happen and the correct item was put in the shopping cart.
In white box testing you want to peer into the internals of the code and probe implementation details. This is more common at the lower level, unit testing.
Functional testing could be either black box or white box testing, depending on exactly what type of tests you want to perform. The two categorisations are not mutually exclusive, they refer to different aspects of the tests. There are lots of ways of categorising tests and everyone has their own interpretation of which is best, or most appropriate in a given situation.
I would say usability testing is almost certainly going to be black box testing. There are many different kinds of usability testing where you should perform the ones that you think the product would most benefit from. You might be critiquing the user interface design, verifying that a likely customer end to end scenario works, or testing that accessibility statutory requirements are met. I would consider all of these usability testing, although, as I said before about people having different opinions on what to call different types of testing, others may disagree.
It is ok to do both types of testing if you think you are adding value by doing both. Sometimes you are adding value by supporting your team in find bugs, sometimes by critiquing the design with a fresh pair of eyes. Sometimes by creating automated regression tests so that the team don’t make common mistakes in the future and can verify easily that they haven’t broken anything with a change. If you think you are testing the same thing again you probably aren’t adding much value, but you can be adding value without every test you carry out ending in filling a bug report.
White-box and black-box testing are kinds of automated tests, that can be run by a computer and produce a pass or fail result. White-box means tests that require you to “see inside the box” and know how it works (I believe regression tests and boundary tests often count), while black-box means you are testing the external interface without knowing any of the internal details.
Usability testing is NOT automated in any way, since it involves finding a human to try using your program, and watching what happens. This sort of testing usually outputs results like “the user didn’t realize this icon meant…” or “only expert users realized that…”. You could call it “black-box” in the sense that your test subject doesn’t know the code, but that’s a little like arguing C++ is a “dynamic” language because it has polymorphism; that’s just not the way that word is normally used in my experience.
Functional testing is may or may not be automated, since it’s a much vaguer term which simply means testing “Does the software do X when I do Y?” If it’s done with automated tests, those automated tests are probably of the black-box variety. If not, it’s probably done by a QA team going off a “test script” written in English (which, again, you could call “black-box” testing, but you normally don’t).
As is often the case with testing, you want to do at least some of all of these kinds. If for some bizarre reason you really do have to choose exactly two kinds, I would go with usability testing and any kind of automated testing.
1