The only advantage I can think of for inline tests would be reducing the number of files to be written. With modern IDEs this really isn’t that big a deal.

There are, however, a number of obvious drawbacks to inline testing:

• It violates separation of concerns. This may be debatable, but to me testing functionality is a different responsibility than implementing it.
• You’d either have to introduce new language features to distinguish between tests/implementation, or you’d risk blurring the line between the two.
• Larger source files are harder to work with: harder to read, harder to understand, you’re more likely to have to deal with source control conflicts.
• I think it would make it harder to put your “tester” hat on, so to speak. If you’re looking at the implementation details, you’ll be more tempted to skip implementing certain tests.

I can think of some:

• Readability. Interspersing “real” code and tests will make it harder to read the real code.

• Code bloat. Mixing “real” code and test code into the same files / classes / whatever is likely to result in larger compiled files, etc. This is particularly important for languages with late binding.

• You may not want your customers / clients to see your test code. (I don’t like this reason … but if you are working on a closed source project, the test code is unlikely to help the customer anyway.)

Now there are possible workarounds for each of these issue. But IMO, it is simpler not to go there in the first place.

It is worth observing that in the early days, Java programmers used to do this kind of thing; e.g. including a main(...) method in a class to facilitate testing. This idea has almost completely disappeared. It is industry practice to implement tests separately using a test framework of some kind.

It is also worth observing that Literate Programming (as conceived by Knuth) has never caught on in the software engineering industry.

For many of the same reasons that you try to avoid tight coupling between classes in your code, it’s also a good idea to avoid unnecessary coupling between tests and code.

Creation: Tests and code may be written at different times, by different people.

Control: If tests are used to specify requirements, you’d certainly want them to be subject to different rules about who can change them and when than the actual code is.

Reusability: If you put the tests inline, you can’t use them with another piece of code.

Imagine that you’ve got a chunk of code that does the job correctly, but leaves a lot to be desired in terms of performance, maintainability, whatever. You decide to replace that code with new and improved code. Using the same set of tests can help you verify that the new code produces the same results as the old code.

Selectability: Keeping the tests separate from the code makes it easier to choose which tests you want to run.

For example, you might have a small suite of tests that relate just to the code you’re currently working on, and a larger suite that tests the entire project.

Actually, you can think of Design By Contract as doing this. The problem is most programming languages don’t let you write code like this 🙁 It’s very easy to test for preconditions by hand, but the post conditions are a real challenge without changing the way you write code (a huge negative IMO).

Michael Feathers has a presentation about this and this is one of the many ways he mentions you can improve code quality.

Here are some additional reasons I can think of:

• having tests in a separate library makes it easier to link only that library against your testing framework, and not your production code (this could be avoided by some preprocessor, but why to build such a thing when the easier solution is to write the tests in a separate place)

• tests of a function, a class, a library are typically written from a “users” point of view (a user of that function/class/library). Such “using code” is typically written in a separate file or library, and a test may be clearer or “more realistic” if it mimics that situation.

This is in response to a large number of comments suggesting that inline tests are not done because it’s difficult to impossible to remove the test code from release builds. This is untrue. Almost all compilers and assemblers support this already, with compiled languages, such as C, C++, C#, this is done with what are called compiler directives.

In the case of c# ( I believe c++ as well, the syntax might’ve slightly different depending on what compiler you are using) this is how you can do it.

#define DEBUG //  = true if c++ code
#define TEST /* can also be defined in the make file for c++ or project file for c# and applies to all associated .cs/.cpp files */

//somewhere in your code
#if DEBUG
// debug only code
#elif TEST
// test only code
#endif

Because this uses compiler directives, the code will not exist in the executable files that are built if the flags are not set. This is also how you make “write once , compile twice” programs for multiple platforms/hardware.

This idea simply amounts to a “Self_Test” method within the context of an object-based or object-oriented design. If using a compiled object-based language like Ada, all the self-test code will be marked by the compiler as unused (never invoked) during the production compilation, and therefore it will all be optimized away – none of it will appear in the resulting executable.

Using a “Self_Test” method is an extremely good idea, and if programmers were really concerned with quality they would all be be doing it. One important issue, though, is that the “Self_Test” method needs to have intense discipline, in that it cannot access any of the implementation details and must instead rely only on all the other published methods within the object’s specification. Obviously, if the self test fails, the implementation will need to change. The self test should be rigorously testing all the published properties of the object’s methods, but never ever relying in any way upon any details of any specific implementation.

Object-based and object-oriented languages frequently do provide exactly that type of discipline with respect to methods external to the tested object (they enforce the object’s specification, preventing any access to its implementation details and raising a compilation error if any such attempt is detected). But the object’s own internal methods are all given complete access to every implementation detail. So the self test method is in a unique situation: it needs to be an internal method because of its nature (self test is obviously a method of the object being tested), yet it needs to receive all the compiler discipline of an external method (it has to be independent of the object’s implementation details). Few if any programming languages provide the capability to discipline an object’s internal method as if it were an external method. So this is an important programming language design issue.

In the absence of proper programming language support, the best way to do it is to create a companion object. In other words, for every object you code (let’s call it “Big_Object”), you also create a second, companion object whose name consists of a standard suffix concatenated with the name of the “real” object (in this case, “Big_Object_Self_Test”), and whose specification consists of a single method (“Big_Object_Self_Test.Self_Test (This_Big_Object : Big_Object) return Boolean;”). The companion object will then depend upon the specification of the main object, and the compiler will fully enforce all of the discipline of that specification against the companion object’s implementation.

If tests were inline, it would be necessary to remove the code you need for testing when you ship the product to your customer. So an extra place where you store your tests simply separates between the code you need and the code your customer needs.

We use inline tests with our Perl code. There’s a module, Test::Inline, that generates test files from the inline code.

I’m not particularly good at organizing my tests, and have found them easier and more likely to be maintained when inlined.

Responding to a couple of the concerns raised:

• The inlined tests are written in POD sections, so they’re not part of the actual code. They are ignored by the interpreter, so there’s no code bloat.
• We use Vim folding to hide the test sections. The only thing you see is a single line above each method being tested like +-- 33 lines: #test----. When you want to work with the test, you just expand it.
• The Test::Inline module “compiles” the tests to normal TAP-compatible files, so they can co-exist with traditional tests.

For reference:

• POD
• TAP

Erlang 2 does actually support inline tests. Any boolean expression in the code which is not used (e.g. assigned to a variable or passed) is automatically treated as a test and evaluated by the compiler; if the expression is false, the code doesn’t compile.

Another reason for separating tests is that you often use additional or even different libraries for testing than for the actual implementation. If you mix tests and implementation, accidential usage of test libraries in the implementation cannot be caught by the compiler.

Also, tests tend to have way more code lines than the implementation parts they test, so you’ll have trouble finding the implementation between all the tests. 🙂

This isn’t true. It’s much better to place your unit-tests along side the production code when the production code especially when the production routine is pure.

If you’re developing under .NET for instance, you could put your test code in the production assembly, and then use Scalpel to remove them before you ship.