Consider this piece of code from the PHPUnit manual(I’m using PHP only as an example):
class StubTest extends PHPUnit_Framework_TestCase
{
public function testStub()
{
// Create a stub for the SomeClass class that could be later injected into another class, but removed here for simplicity.
$stub = $this->getMock('SomeClass');
// Configure the stub.
$stub->expects($this->any())
->method('doSomething')
->will($this->returnValue('foo'));
// Calling $stub->doSomething() will now return 'foo'.
$this->assertEquals('foo', $stub->doSomething());
}
}
Now I just stubbed the method doSomething() to return foo.
What if some time later in the future I decide to implement SomeClass like this:
class SomeClass{
function doSomething(){
return 'bar';
}
}
Of course i have tests for SomeClass that test for bar, but I totally forgot to test for foo.
Now I have a lying test. How does TDD handle this situation?
1
// Calling $stub->doSomething() will now return 'foo'.
$this->assertEquals('foo', $stub->doSomething());
This right here is actually the key part of the code that represents the following two problems:
- The way you are implementing your tests, you are testing your tests, not testing your code,
- If you need that level of specificity, you are testing implementation, not interface.
Asserting that the output of a stub is what you said it should be exercises the functionality of the testing framework’s stubbing functionality rather than anything in your code. I doubt you have any actual lines like the above in your tests, but, subtly, you probably have tests that amount to the same if you are asking the question you are asking.
Let’s say we have an example test, which I will use to explain:
// A Doubler takes the result of SomeClass#doSomething and doubles it.
class DoublerTest extends PHPUnit_Framework_TestCase
{
public function testToString()
{
// Create a stub for the SomeClass
$stub = $this->getMock('SomeClass');
// Configure the stub.
$stub->expects($this->any())
->method('doSomething')
->will($this->returnValue('foo'));
$doubler = new Doubler($stub);
// Calling $doubler->toString() will now return 'foofoo'.
$this->assertEquals('foofoo', $doubler->toString());
}
}
A Doubler has a simple functionality: double the return value of the doSomething method of the injected SomeClass. What’s important to note there is that I did not specify what SomeClass#doSomething needed to returns, as it is irrelevant to the algorithm. Whether it returns "foo" or "bar", the method works the same. The test still gives useful and valid information.
Now, you may contend that, in your case, the return value does change the outcome of the algorithm:
// Doubler also doubles 'r's in the output string
// ... setup the stub, return "bar" ...
$this->assertEquals('barrbarr', $doubler->toString());
// If Doubler sees a 'q', party time! (return "PartyParty")
// ... setup the stub, return "qux" ...
$this->assertEquals('PartyParty', $doubler->toString());
This should be apparent in the specification of the Doubler class (which you use to guide building your tests in TDD), and should already be tested for, since it represents a code path in that class. A code coverage tool will make spotting code paths easy if you miss any (and there’s one built into PHPUnit). Therefore, changing the implementation of SomeClass#doSomething is still irrelevant to the test.
If you change the interface of SomeClass in a way that doesn’t follow a pre-existing code path, you will find you need to write new code in the dependent classes anyway, which is the point where you will alter the tests (including stubs) for that code, bringing it back in alignment with your interface.
Unit tests are meant to test interfaces rather than implementations. If your code depends on subtle implementation details like the exact return value of a method beyond just its type (or beyond other general factors), writing and running integration tests will most likely find those types of problems. Ideally, your code won’t depend on implementation details and you could use OO design patterns and principles to isolate those details, but we don’t live in an ideal world.