I hope this is not off topic.
I have to save output from a command to file, but only if length of this output is positive. I’ve thought about two solutions:
- save output to a python variable, check length, if positive save to destination file;
- save output to a temporary file, check length, if positive rename to destination file.
Is there any best practice for this little problem, or which one usually performs better between my proposed solutions?
Typical configuration:
- file length is usually 1KB
- we are not using ssds, usually 7200rpm hard disk
- average RAM is usually 10-20 GB
- I don’t know if filesystem is important, anyway ext3
I know this is a bit unclear, but I was hoping there was some best practice to apply in this case.
Thanks
5
For Little problems I would suggest storing in memory because memory is cheap and fast and storing in Disk is slow and a costly operation.
32 GB of RAM memory are more common nowadays than solid state drives.
1
I think the answer to this is: it depends.
If it is important that you do not lose this data, then my advice would be to get it onto the disk as soon as possible. Bear in mind that if something bad happens on the computer, anything in memory will be lost.
However if it is more important that your program runs in as performant a way as possible, then writing to memory is bound to be quicker than writing to disk.
For problems of this size (few kilobytes to several megabytes), I would say store it in memory, check the length condition, and if it’s successful, save it to the disk as a permanent file. Unless the output data retrieved from the command line is very important, it’s better to verify the data on the memory. In fact, I would also say that unless the operation performed is “risky” (high chance of data corruption or similar) or there’s a noticeable chance of data or power failure, to use the memory first, and then flush it to the disk. RAM is inexpensive and fast but, temporary. Whereas, the hard drive is slow, expensive, but, (more) permanent.
Overall, it’s a matter of how big your data is, and how many risks are involved in the process (and you are willing to take).