On logic in tests

There are cases where we could include some logic in the source code in order to help us performing some functional tests or end-to-end tests

Unit testing is an area of software engineering where people seem to become very dogmatic. “Do it this way or it isn’t unit testing” – this can get in the way of actually doing tests. People have argued it each way and will continue to argue about it for quite some time.

One of my favorite documents on testing is The Way of Testivus – its a good (and humorous) read.

The thing is to write the test that needs to be written. If this means putting logic in there, then there’s logic in there.

The danger of the this is who tests the logic in the tests (Test-Driven Hypocrisy? Who tests the test?). The issue being if your test logic and your business logic are both broke, you may miss an actual failed test.

On code with test logic

Another aspect to this question is the “we stuck some code in the build that goes to QA (and presumably to production – because you don’t switch out the build after giving it to QA…) to make their job easier.

There’s a real danger here.

When you put test code into the build it is possible that it will get tickled in production. “But that will never happen” has happened far too often. The code allows you to set the time stamp, or get into a certain state that allows you to do set some data without going through some other process.

These are bad things. Don’t do them.

One example could be that we have some test code around date/time logic so that if the testers make server and client time different, this code could help them to perform some tests.

Don’t do that. And here I’m being rather dogmatic. There’s a difference between having the unit test code which never gets bundled into a production build where its important to do the testing… and code that is going to production.

Having test code in there that lets you bypass the normal flow of the code that a user would need to do has several issues with it:

• The full process isn’t being tested. The QA tester is jumping around the code in a way a normal user can’t. This means that the actual workflow isn’t being tested. You’ve stuck data in some place through an admin/test interface that initializes the structure before using it… but your normal code doesn’t and you’ve got a bug that QA won’t find.
• There’s a hook in there that someone else can use. We know all the users are nice and don’t go poking into places they shouldn’t be looking. Sure you’ve got it hidden needing to hold down the shift-control-alt and ‘cat’ at the same time… but you’ve never had a cat actually step on your keyboard to do this in a way that only a cat can. And cats aren’t the most nefarious of the keyboard users – people with less than honorable intentions will find this hook too.

If you need an environment that makes a particular thing easier to test, set up that environment. You want the testers to have the ability to do some test with the client and server out of sync? Create a VM that recreates this environment easily rather than inserting some code that makes the client interpret its timestamps as 30 seconds in the past (or future).

Don’t put test code in the production build. Make sure you test the production build. This may necessitate having longer testing processes, but the alternatives can be very bad (missing a bug or letting someone use the software improperly).

Code that is added to, say, a class, for testing purposes, is symptomatic of code that hasn’t been written to be testable.

I’ll give you an example. In one of my current applications, there is a parser which reads a data file containing Key/Value pairs, and creates a representative recursive data structure. The specification is here, if you’re at all interested.

To speed up the testing process, I overrode the ToString() method in the Node class (a recursive data structure containing the current data node, and a list of descendant nodes), and put some code in there that recursively walks the nodes and builds a string containing a human-readable representation of the content of that part of the tree. I then parse a bit of the data file, examine the output I get from ToString(), and if it looks good, I paste the output into a unit test and assert against it.

You can build a test suite relatively quickly this way. The problem, of course, is that if you change anything, especially the ToString() method, it breaks all of your tests. Also, because the tests don’t test specific functionalities (preferring to cast a wide net instead, and hoping that adequate code coverage is achieved), they are essentially “go, no-go” tests. If the test fails, you have to analyze the output to determine why (a process made easier with the use of diff methods).

So the tradeoff is that such tests are relatively easy to fabricate and save a ton of time, but the downside is that the degree of code coverage is uncertain, the tests are brittle, and a failed test doesn’t tell you anything without additional analysis.

An ugly test is better than no test

When the code is ugly, the tests may be ugly.
You don’t like to write ugly tests, but ugly code needs testing the most.
Don’t let
ugly code stop you from writing tests, but let ugly code stop you
from writing more of it.

— The Way of Testivus

The question seems a little vague so I will try to answer it the best that I can. If the answer does not help feel free to add more info and I can add an edit to focus on the new info.

Typically it is a good idea to test data that is passed into a function or method. One of the more popular tests would be a null or none test. These types of tests can ensure that the data passed in will not break the function/method. Additionally the function/method has the option of handling or passing back an error to the user without having a null pointer exception, or similar, being thrown causing the program to crash.

Additionally it is a good idea to check for other things too. If you are supposed to get a datetime but only receive a date this could be a problem. You may want to throw an exception or simply add in a default time so that code further down the function/method will not break.

I would not want to supply code that preforms the same type of tests that a unit test would provide. Meaning that you should not have code that checks to see if a function/method can handle a certain type of data. This can be pulled out into a test that is run in a non-production environment.

It sounds like you are trying to add in code to check a passed in parameter. For example, if a call to an API returns the time of the server and the client machine checks to see if that time matches or is close to its own internal time. Assuming the same time zone or both machines are comparing using UTC (its good to have a common base to compare to) you would almost certainly want to check that time sensitive data has a correct timestamp in your code. From there you could either let the user know that their data may be out of date or that their system time is off.

Your problem calls for the IOC Pattern. Where ever you need a DateTime you will use the DateTimeProvider that you injected into your class as a constructor parameter.

Now when you are testing your class you can use your own mocked/dummy/whatever DateTimeProvider and do whatever your want.

Now your classes will not contain any test code. It may seem like extra boiler plate code, but in that case I recommend using an IOC container.