From Martin Fowler’s Mocks Aren’t Stubs he states that Stubs provide canned answers to calls made during the test while Mocks objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
In addition several sources state that Stubs shouldn’t fail tests, but Mock could.
So what about this:
$validator= $this->getMock('ValidatorService');
// Set up the expectation for the isValid() method
// to be called only once and with the string 'textToBeValidated'
// as its parameter and returns true or false.
$validator->expects($this->once())
->method('isValid')
->with($this->equalTo('textToBeValidated'))
->will($this->returnValue(true));
This came up in a code review in a project I’m working on and we called it a Stock (St ub – M ock). Why you ask?!
- It set an expectation for
isValid()to be called only once and with certain input like a Mock. - It returns a canned answer like a Stub.
I get a feeling that this is wrong, but what if we used a pure Stub only:
$validator->expects($this->any()) //<-- can't fail the test now
->method('isValid')
->will($this->returnValue(true)); //<-- canned answer
- I can’t verify that the method
isValidwas called only once with the right parameters. - If the test passed I’m not sure that it passed for the right reason. Maybe
isValid()was called twice and that caused it to pass.
So let’s use a pure Mock instead:
$validator->expects($this->once())
->method('isValid')
->with($this->equalTo('textToBeValidated'))
- I can’t verify that my function works correctly, because I only verified that call to
isValid()was successful. Maybe there is a bug in the rest of my code using theValidatorService.
I know some people will talk about state vs behavior verification, but the way I see it the only way to be sure that the test passes for the correct reason is to to use a Stock (i.e. verify the behavior and final state).
Of course the other option is to write 2 test for every test criteria. One using a Mock and an identical test using a Stub. However, that will result in a TestOverflow (a huge number of nearly identical tests).
What is the correct way to have a concrete test of a function using a dependency?
10
Mocks will return stub answers if that is required by the contract of the class/interface they’re mocking. That’s just necessary. Your object isn’t anything special, it’s just a mock.
Nevertheless, prefer stubs to mocks when you can get away with it. Mocks generally lead to less flexible tests compared to tests without mocks. For example, do you have to validate that isValid was called exactly once? Can’t you instead have a test case that stubs isValid to return true, another that stubs isValid to return false, and simply observe the results of the two tests? (The answer may be ‘no’ if you have to make sure the correct text is passed. It’s just something to think about.)
5
Well, what you are describing as “Stock” is actually the definition of a Mock, at least in the book xUnit Patterns, which describes all the main types of test doubles.
The logic for the types of test doubles is that each inherits from the previous.
Consider a Dummy as the baseline, a NullObject implementation for any given interface.
The next type is the Stub, which returns some values.
Then there is the Spy, which returns values just as a Stub, but also records the details of its interactions.
Then comes the Mock, which by inheriting from the Spy, can decide whether to fail a test if the interaction is not valid for the given expectations.
By this logic, a Mock is a Spy, which is a Stub, which is a Dummy, which is a Test Double (the completely abstract concept).
There is also the concept of fakes, but they follow a different hierarchy.
You can see the details on the definition of each type here.
OK I did some research and finally found what I was looking for in the post Mocks for Commands, Stubs for Queries by Mark Seemann.
To make a long story short Stocks are bad. They are a code smell or better a unit testing smell indeed.
Having experienced the pain of over specifying interactions between classes in my code I find “J. B. Rainsberger” post to reflect this ugly truth:
When one uses a mock in place of a stub, one tends to over-specify the
collaboration between a Client and its Suppliers, and that leads to
brittle tests. These tests tend to make hyperactive assertions,
meaning assertions that fail even when the underlying production code
works as expected. Such hyperactive assertions play the role of the
boy who cried “wolf!”, distracting us with needless concern. Even when
these superfluous mocks don’t over-specify a collaboration, they often
duplicate specifications in other tests.
So next time you find yourself writing “expects” and “returns” in the same fake object think again.