Sometimes we have code like this:
private static final List<Goat> INEDIBLE_GOATS = [GOAT_A, GOAT_B, GOAT_C]
void eatGoat(Goat goat) {
if(INEDIBLE_GOATS.contains(goat)) {
throw new CantEatGoatException();
}
...
}
Should it best tested (along with whatever tests for the actual eating) as:
for(Goat goat: INEDIBLE_GOATS) {
// forgive abuse of syntax
assertThrows(CantEatGoatException, eatGoat(goat));
}
Or more like?
assertThrows(CantEatGoatException, eatGoat(GOAT_A));
Should tests verify each and every sort of inedible goat? Or should they test a representative goat from the list and trust that it works for the other goats?
4
A major reason to have tests is to build your confidence in your code. Tested code can be proved to have a specific functionality because the tests say they do. Confidence grows from the proof. The question is, by looping through all the INEDIBLE_GOATS, do you gain any more confidence in your code? Either way, with both positive and negative tests, you cover the relevant code. With the loop, though, you are basically exercising these lines over and over. Essentially, you are testing that List<Goat>#contains() does what it says. Do you gain anything from that? If you trust that a List works as it should, then no, you don’t get anything additional. If, on the other hand, you are not confident that a List works as it should, then you should have a separate test of List#contains() to give you that confidence.
You also have to balance the number of tests with the performance of your test suite. Looping through all the INEDIBLE_GOATS may give you a little more confidence, but if there are, say, 1.8 million of them, and running the tests takes 30 seconds longer, does that tiny additional confidence outweigh the lost time (and ensuing lost productivity).?
Finally, while INEDIBLE_GOATS may only have a small number of elements now, theoretically, the number of inedible Goats may actually be infinite. It is impossible to test every potential Goat. While in this simplified example, you may say that there won’t really be infinite Goats, in general, we must account for infinite and near-infinite possible inputs and accept that we cannot practically test everything. We must also realize that this is a good thing. Not all inputs are equal, and trying to test everything wouldn’t necessarily get us any more confidence. It is optimizing tests for confidence that makes them practical and profitable.
1
The second test seems more accurate to me. If you put a Goat within your List, you expect that Goat to be a foobarable Goat (that is, you expect to get true from your method).
2
Seems like you’re just pushing the logic around. How about:
assertTrue(FOOBARABLE_GOATS.contains(goat));
5
Your question is an important one. To restate more generically: should you test all possible inputs or just selected ones? As @cbojar points out, the former is typically either (a) not practical or (b) not useful.
The question then becomes how to whittle the list of all possible test inputs into a set of optimal test inputs. To do this, divide the set of all possible inputs into groups based on expected outcomes. Any input from one group should yield equivalent (though not identical) results. Such groups are called equivalence classes. and this process of identifying such classes is equivalence partitioning.
As a simple example, to test a square root function that takes an integer input, a good starting point would be these equivalence classes:
1 -- all positive integers
-1 -- all negative integers
0 -- zero
You might refine that list further:
1 -- all positive integers that are not perfect squares
4 -- all positive integers that are perfect squares
-1 -- all negative integers
0 -- zero
And if you are a real stickler, you might add maxint and minint for your integer type. But with any of these you have infinitely less test cases than the set of all possible inputs 🙂 — a pretty good savings!
(For more, see my answer to a related question, What should you test with unit tests?)