Please note that “best practice” questions are discouraged, because they are too easily answered as if there were a single way of doing something that is superior to all others. Almost always, there isn’t such a way.

When deciding whether to use one log stream or different ones simultaneously, you have to weigh what the cost and benefit is in the expected use case. Writing everything to one stream is easy when programming, but makes for more effort when you have to retrieve or analyse logs. Pre-sorting all log output takes more effort when programing every log message but saves some effort when reading logs.

In the situations I’m familiar with, log messages are programmed more often than they are reported by customers or searched by support staff, so it makes sense to make the common case (developing) easy at the cost of more effort in the rarer case (troubleshooting). Analysing a log involves searching anyway, and searching a longer file is not much more effort than searching one of several shorter files because the computer does the main work, not your brain.

But that is just my project and my worldview. If (lucky you!) you have millions of customers and a huge traffic in support requests, then logs will be analysed much more often than they are written. In that case it makes sense to make reporting, analysing and correlating logs as easy as possible at the cost of making it more difficult to program a new log message. Then you should put more effort into making the reporting and analysis workflow efficient. (For instance, if customers have trouble gathering all relevant logs, it’s a good idea to program log retrieval into your software so that the admin user need only press a button to send everything relevant.)

Lets assume you are talking about log files with the purpose of reproducing error scenarios happening at your customers site (which you did not describe). Lets assume the log files are to be analysed by you, not by the customer himself. And lets assume the log files don’t grow to a size of several hundreds MB. Then your team lead is correct, one log file is probably easier to handle than many.

The situation changes if there are many completely different kinds of logs, and different users of your customer (with different roles) have to look directly into the log files by themselves, and each user needs to see different kind of informations. Then it would make more sense to split the logs thematically. If log file size matters, it may be a good idea to split the logs to time intervals (for example, one log for each day, week or month).

Whatever you do, it is always a good idea to make the log information filterable. That means, add machine-readable tags to each log entry, for example, which application or subsystem produced the entry, was it a warning, a severe error message or just a status information, and of course add the exact time stamp. That will help you to find the information you need afterwards much more easily even when you have them all in one file.

And its often unnecessary to develop your own log analyser – there are great free tools like the MS log parser which can query and filter your log files for you. For some scenarios you don’t need much more than a decent spreadsheet.

I think I would even prefer a logfile with everything in it.

My experience so far is that logfiles are especially useful for bugs where there is a problem in the communication between your subsystems. If they all log to the same file, it makes the problem easier to diagnose, because you can see the timeline of what is happening.

Also for customer specific performance issues, it is often helpful to be able to correlate what is happening in all subsystems at the same time of the problem.

Putting on my sysadmin hat, the best practice is that the developers put plenty of logging calls in their apps and then expose the logging level and destination configuration for me — logging to file might work in one scenario but I might really want to send stuff to SNMP in another or to email in another. Please don’t make my decisions for me.