When doing the Red, Green & Refactor cycle we should always write the minimum code to pass the test. This is the way I have been taught about TDD and the way almost all books describe the process.
But what about the logging?
Honestly I have rarely used logging in an application unless there was something really complicated that was happening, however, I have seen numerous posts that talk about the importance of proper logging.
So other than logging an exception I couldn’t justify the real importance of logging in a proper tested application (unit/integration/acceptance tests).
So my questions are:
- Do we need to log if we are doing TDD? won’t a failing test reveal
what wrong with the application? - Should we add test for the logging process in each method in each class?
- If some log levels are disabled in the production environment for example, won’t that introduce a dependency between the tests and enviroment?
- People talk about how logs ease debugging, but one of the main advantages about TDD is that I always know what’s wrong due to a failing test.
Is there something I am missing out there?
2
1) Do we need to log if we are doing TDD? won’t a failing test reveal what wrong with the application?
That assumes you have every possible test your application needs, which is rarely true. Logs help you track down bugs you did not write tests for yet.
2) Should we add test for the logging process in each method in each class?
If the logger itself is tested, it would not need to be retested in each class, similar to other dependencies.
3) If some log levels are disabled in the production environment for example, won’t that introduce a dependency between the tests and environment?
Humans (and log aggregators) depend on the logs, the tests should not depend on them. Typically there are several log levels, and some are used in production, and some additional levels are used in development, similar to:
“Rails log level is info in production mode and debug in development and test” – http://guides.rubyonrails.org/debugging_rails_applications.html
Other applications use a similar approach.
4) People talk about how logs ease debugging, but one of the main advantages about TDD is that I always know what’s wrong due to a failing test.
Production bugs will have passed all the tests, so you may need some other reference to investigate those issues.
1
Logging is useful to explain non-exceptional behavior of an application:
Event logs record events taking place in the execution of a system in order to provide an audit trail that can be used to understand the activity of the system and to diagnose problems. They are essential to understand the activities of complex systems, particularly in the case of applications with little user interaction (such as server applications)…
No matter how application was tested and no matter how well exceptions are logged, its users may ask,
output of your program is 0 while we expected it to be 1, why is that?
You need logging to verify what was application config, parameters and other runtime details to explain its (non-exceptional) behavior.
From above perspective, logging is more oriented on support than on development. After the application goes live, it is desirable to let someone else handle questions from users, to allow programmers focus on further development.
Logging what application does allows that someone else understand program behavior without digging into the code and without distracting developers by requests to explain what’s going on.
3
Most answers here focus on the aspect of correctness. But logging serves a different purpose, too: Logging can be a way to gather performanc relevant data. So even when the system works without error a log can tell why it is slow. Even with full test coverage of all aspects a test suite won’t tell.
Of course a performance critical system can/should provide key performance metrics to some operational dashboard, but “classic” logging can provide a different level of detail.
5
-
Unless you have a 100% test cover, which is usually not the case, you can’t know that your software will never crash (EDIT:and — as said in the comments — even if it does, something independent of your software might cause a crash); it’s the same as thinking you can do a software that has absolutely no bugs (not even the NASA can do this).
So at the very least, you need to log possible failures in case your program crashes so you can know why. -
The logging should be done by an external library or intern framework depending on the technology you’re using. What I mean by that is that it should be something that has already been tested before and that you don’t need to test yourself. It’s overkill to test that every methods logs the things it’s supposed to.
-
The logs are not meant for tests, there should be no dependency whatsoever. That said, you don’t need to disable logging for tests if it feels like a constraint to you, though the logs should be kept in a file corresponding to the environement (you should have a different file for the testing, development and production environment at the very least).
-
An error might be very unclear and it’s not always obvious what’s gone wrong when one TDD test failed. Logs should be more precise. For instance, if you’re doing a sorting algorithm and the entire test case fails, you should have logs for every single test of the algorithm which help you spot where the problem actually lies.
6
The short answer to your main question is: as a general rule, bugs in your code will NOT be exposed by TDD. Some might, ideally many, but the absence of failing tests does not imply the absence of bugs. This is a very important maxim in software testing.
Since you cannot know whether you will have incorrect behavior in your system, maybe under rare conditions, logging is a useful tool that might help understand what’s wrong when things inevitably do go wrong.
Logging and TDD address different concerns.
Yes, in the general case you need logging.
Logging is not about debugging. Well, OK, a part of logging is sometimes about debugging, and you can skip that part if you don’t need it during development.
But the more important part of logging is about maintainability. Well-designed logging might answer the following questions:
- Is the application still alive and kicking? (By logging a heartbeat every 1000 seconds.)
- Did the performance of the application change over the last 10 months? (By logging statistics of the performance of use cases.)
- Did the load of the appliation change over the last 10 months? (By logging the number of requests per request types.)
- Did any of our interfaces change their performance characteristics?
- Does the new release cause a different usage characteristic towards some of the subsystems?
- Are we under a DoS attack?
- What kinds of errors are happening?
All this can be achieved by logging. And yes, it should be planned and designed and tested, preferable automatic.
Logging is a feature that deservers treatment, just as other features.
TL;DR: Logging and TDD are orthogonal. Having one has no bearing on needing the other
Do we need to log if we are doing TDD? won’t a failing test reveal what wrong with the application?
By and large most logging that I’ve implemented, and that I’ve seen implemented has been for operational troubleshooting, not for development debugging (although it may help). The primary audience for this logging are the admins and the ops who run your servers, support people who have logs sent to them for analysis, and customers who want to look at the logs and try and figure out what’s going on.
These logs are there to help troubleshoot issues largely do to integration points. This may network services (database, soap, etc), local resources (disk, memory, etc), bad data (customer input, bad/corrupt data sources, etc), etc. Capturing exceptions, logging faults, and even doing informative logging (settings, configurations, etc) can all help with troubleshooting.
Should we add test for the logging process in each method in each class?
Add testing where ever you need to in order to test logging. If you have ad hoc calls to log out information then they should be tested. Although if you implement logging and log testing using Aspect Oriented Programming or meta programming, that may reduce the burden of testing.
If some log levels are disabled in the production environment for example, won’t that introduce a dependency between the tests and enviroment?
If you are writing your code using IoC and you make use of mocks, you should be able to effectively test all your logging without relying on a particular environmental setup.
TDD generally helps reduce coding bugs. It helps a lot less with bugs with the specification or just misunderstandings about how things work.
“Oh? You can get a data message before you get confirmation the logon succeeded? I never knew that, well it won’t handle that!”… That kind of thing. Logging is very useful for telling you what the software tried to do so you can spot the thing you did wrong.
In my experience great level of logging is added to the application when we don’t do TDD. Then the level of uncertainty becomes high hence we add logging to see what is going on.
Whereas when doing TDD (or maybe test-whenever) I find myself adding a lot less log statements. This in turn means less LOC and can (not always) affect performance.
But we do add entering-exiting logs for functions in a semi-automatic way at my company in most cases regardless of development method. As I know this was considered mandatory for production issue analysis.
Example could be methods of an EJB service bean which are present on the public interface. Another might be a case when a function does complex calculations. It can help a lot to have figures entering the method (for example you can write a unit test to get back to the general topic at hand).
2