I’m really struggling to write effective unit tests for a large Django project. I have reasonably good test coverage, but I’ve come to realize that the tests I’ve been writing are definitely integration/acceptance tests, not unit tests at all, and I have critical portions of my application that are not being tested effectively. I want to fix this ASAP.
Here’s my problem. My schema is deeply relational, and heavily time-oriented, giving my model object high internal coupling and lots of state. Many of my model methods query based on time intervals, and I’ve got a lot of auto_now_add going on in timestamped fields. So take a method that looks like this for example:
def summary(self, startTime=None, endTime=None):
# ... logic to assign a proper start and end time
# if none was provided, probably using datetime.now()
objects = self.related_model_set.manager_method.filter(...)
return sum(object.key_method(startTime, endTime) for object in objects)
How does one approach testing something like this?
Here’s where I am so far. It occurs to me that the unit testing objective should be given some mocked behavior by key_method on its arguments, is summary correctly filtering/aggregating to produce a correct result?
Mocking datetime.now() is straightforward enough, but how can I mock out the rest of the behavior?
- I could use fixtures, but I’ve heard pros and cons of using fixtures for building my data (poor maintainability being a con that hits home for me).
- I could also setup my data through the ORM, but that can be limiting, because then I have to create related objects as well. And the ORM doesn’t let you mess with
auto_now_addfields manually. - Mocking the ORM is another option, but not only is it tricky to mock deeply nested ORM methods, but the logic in the ORM code gets mocked out of the test, and mocking seems to make the test really dependent on the internals and dependencies of the function-under-test.
The toughest nuts to crack seem to be the functions like this, that sit on a few layers of models and lower-level functions and are very dependent on the time, even though these functions may not be super complicated. My overall problem is that no matter how I seem to slice it, my tests are looking way more complex than the functions they are testing.
5
I’m going to go ahead and register an answer for what I’ve come up with so far.
My hypothesis is that for a function with deep coupling and state, the reality is that it’s simply going to take a lot of lines to control for its outside context.
Here’s what my test case looks like roughly, relying on the standard Mock library:
- I use the standard ORM to set up the sequence of events
- I create my own start
datetimeand subvert theauto_now_addtimes to fit a fixed timeline of my design. I thought that the ORM didn’t allow this, but it works fine. - I make sure the function-under-test uses
from datetime import datetimeso that I can patchdatetime.now()in just that function (if I mock the entiredatetimeclass, the ORM pitches a fit). - I create my own replacement for
object.key_method(), with simple but well defined functionality that depends on the arguments. I want it to depend on the arguments, because otherwise I might not know if the logic of the function-under-test is working. In my case, it simply returns the number of seconds betweenstartTimeandendTime. I patch it in by wrapping it in a lambda and patching directly on toobject.key_method()using thenew_callablekwarg ofpatch. - Finally, I run a series of asserts on various calls of
summarywith different arguments to check equality with expected hand-calculated results accounting for the given behavior of the mockkey_method
Needless to say, this is significantly longer and more complicated than the function itself. It depends on the DB, and doesn’t really feel like a unit test. But it is also fairly decoupled from the internals of the function–just its signature and dependencies. So I think it might actually be a unit test, still.
In my app, the function is quite pivotal, and subject to refactoring to optimize its performance. So I think the trouble is worth it, complexity notwithstanding. But I’m still open to better ideas on how to approach this. All part of my long journey toward a more test-driven style of development…