Maybe I am not understanding how multiprocessing works, but please help with below issue.
My program runs some heavy process at startup that consolidates data for the multiprocessing function.
The heavy process should only ever run once at startup, but upon the multiprocessing code kicking in, each spawn starts executing from the top of the script agin.
This means that each spawn executes the heavy process again instead instead of just accessing the consolidated data.
The actual code results in the given behavior whether compiled using pyinstaller or been run directly from the script. The example code below exhibits the same behavior:
import multiprocessing
print("I am a heavy process that should only run once. ")
listing = [[1,2,3,4], [5,6,7,8], [9,10,11,12]]
def multiply_it(listed):
print([x*2 for x in listed])
if __name__ == "__main__":
multiprocessing.freeze_support()
for listed in listing:
multiprocessing.Process(target=multiply_it, args=(listed,)).start()
results in this
I am a heavy process that should only run once
I am a heavy process that should only run once
[2, 4, 6, 8]
I am a heavy process that should only run once
[10, 12, 14, 16]
I am a heavy process that should only run once
[18, 20, 22, 24]
while I am expecting this
I am a heavy process that should only run once
[2, 4, 6, 8]
[10, 12, 14, 16]
[18, 20, 22, 24]
Any help will be appreciated