I’m running some graph algorithms that take a long time to run, so I need to set a timelimit to ensure it doesn’t run for weeks. I have a 12 core CPU and 24 graphs to run the algorithm, so I set the timelimit for 24h for each graph instance, so i thought that using 11 cores to run this algorithms would give me at most 72h of wait. What happened is that 19 graphs ran until they hit the 24h limit and the others terminated earlier, I measured the CPU time and the results were (in seconds)
79.424548756
86400.000046456
297.74826937499995
86400.000043983
3088.426709061
86400.000042841
16472.334290547
86400.00005754901
32793.841275341
86400.000042971
86400.00004230901
86400.000038542
86400.000032061
86400.000058278
86400.000056663
86400.00004854199
86400.000037238
86400.00003733199
86400.00003613501
86400.000047777
86400.000049447
86400.000048982
86400.00003390199
86400.00004551001
Now, it took about a day (in wall time) from the moment I ran the script to the moment the files were written, which I find odd because as 19 instances took 24h, I expected it to take about 2 days to finish, because each core would solve 2 24h instance in average. Is my reasoning wrong or the script actually run faster than it should?
In case it helps I did write a very wacky script. It runs the algorithm in Python, but I could not make it work in parallel (natively) by any means, so I simply wrote 24 .py files and ran them in parallel in bash using & (like python3 script1.py & python3 script2.py &...
), can this be the reason of the (supposed) time inconsistency?
nogrodrigo is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.