I am using multi processes in python, each process executes the same program that reads from some pkl files, which store dictionaries, analyses new data based on these pkl and updates the same pkl files with new results.
Multiprocessing is just used to speed up the execution, ideally they should work as if it was a single process executing a program. That is, these pkl files are updated at every iteration of the loop of running this program.
In practice, they might update these pkl files together, or if process A is reading pkl files while process B is writing into it, resulting A reading corrupted values.
Can anyone help me understand how to solve this proved?