Hi I have fairly complex program that is doing computations in a quite large loop. I want to log some basic statistic about the run to be able to analyze its performance over time and vs loop number. For now I am outputting a set of numbers, about 5 or 6, each loop.
The problem is the number of loops is quite large so the log files are big. Also putting 10^6 of measurement points on a plot is kind of pointless.
I thought of “filtering” the results by for example printing every 1k’th loop but I would lose some points-of-interest. I also cannot log only POIs, because I would lose track of general progress.
I decided to switch to an event based logging. Primarily the events I have in mind are combination of the two aforementioned things: improvement of current solution and some progress over the loops.
The Implementation I have in mind is to write a simple StatLogger that would be responsible for outputting the data to the log stream. The main program would just issue a log() method whenever an event occurs. The events can occur simultaneously and it does not make sense to log the state twice, so the class would be responsible for not outputting the stats twice.
My questions are:
- Is this a good or common approach and pattern?
- Is it better to pass a handler to the number cruncher object, so the logger can get all the interesting stats like
StatLogger::log(const RunEnviroment *this), or is better to make another structure to hold all the interesting stats and make aStatLogger::log(const Stats& stats)method. - If my approach is bad, what are common solutions in similar cases?
2
I would avoid something general like a StatLogger to be dependent on something special like a RunEnviroment, whatever that looks like. Keeping StatLogger independent will most probably enhance the testability and reusability of that class.
If you need, however, a simple method like
StatLogger::log( EnumEventType evenTtype, const std::string &logtext)
or something like
StatLogger::log(const Stats& stats)
(where I guess Statsrepresents one log entry), will mainly depend on what information you are going to log and how complex that information is.
If you want to decouple your main program from a specific logger, you could also introduce an abstract base class as an interface (like IStatLogger), and let your main program only access the logger through that interface. That will allow you to replace the real logger easily by a mock logger for testing purposes.
3
One solution would be to introduce the logging levels. Each level would correspond to importance (or maybe priority) of log information. Such strategy would enable a log information to be grouped and therefore more easily filtered out. These groups could be for example:
- TRACE; DEBUG; INFO; WARN; ERROR; FATAL
One such implementation can be found in Linux kernel.
Also, I’m sure you will find more information by googling various logger implementations.