Is there any way to be able to start two parallel processes in two different cells in Jupyter Notebook, so that each process can run independently of the other, and so that each process can print below the cell?
I am trying to pass functions as arguments into a parallel processing pool in Python (Jupyter Notebook), but fail for some reason. When I run the code below there is no error, but however the parallel processes dont seem to run at all, because nothing inside the process is printed. I tried to work from the assumption that passing functions into parallel processes might not work in Python, but I couldn’t find any resource online that said that. So I am just wondering how I can achieve this objective of passing functions and data as argument.
To test this proposition, I pass a list of tuples into the parallel pool. The first element in the tuple is a list of numbers, and the second element is a function that will sum that list. Two different functions are passed with their own data, supposedly to run in parallel, to see if the set-up works. Can any modification fix the problem? Or is there some other clever way to achieve this?
import concurrent.futures
import time
def sum_data_1(data):
print("function 1 sums array 1", sum(data))
def sum_data_2(data):
print("function 2 sums array 2", sum(data))
def sum_in_parallel(data_and_function):
data, function = data_and_function
loop_counter = 0
while loop_counter < 30:
loop_counter += 1
if (loop_counter % 20 == 0):
print("in the loop")
print(data)
time.sleep(1)
function(data)
print("the summing of this array", data, "is done")
if __name__ == '__main__':
data_1 = [1, 2, 3]
data_2 = [4, 5, 6]
inputs = [(data_1, sum_data_1), (data_2, sum_data_2)]
with concurrent.futures.ProcessPoolExecutor() as executor:
executor.map(sum_in_parallel, inputs)