I am working with a trading application (reading data from the exchange) which generates a bucket load of data on a per second basis. We have different “log-levels” but even the minimal log-level generates so much data ! This process of log creation is quite I/O intensive.
Since I can write assembly using __asm__ , I was wondering if the operation of writing to the files is coded in assembly, will there be a speedup ? The rest of the code is C++.
Also, what are the caveats of doing such a thing ?
The performance overhead of writing lots of data to disk isn’t the execution speed of your code, but rather the physical limitations of the actual hard drive. Doing it assembly won’t give you a noticeable performance increase.
Your best bet is to either log less data (recommended, if you’re logging that much stuff how useful can it be) or change the drives so they can write quicker! Consider higher RPM’s, SSD’s, RAID (zero I think for parallel writing) or at least a dedicated disk with a dedicated controller. But really, unless you really need all that logging data, you’re wasting your time. Fix your log levels so that WARN and ERROR only log stuff when those things actually happen. (and if they are happening all the time… fix your code).
3