Problem

I’m trying to use a ProcessPoolExecutor with recursive calls, but it doesn’t work. I created a minimal example below

from concurrent.futures import ProcessPoolExecutor
from time import sleep

executor = ProcessPoolExecutor()
i = 3

def test():
    global i
    print("top")
    if i == 0:
        print("bar")
        return
    else:
        print("foo")
        i -= 1
        p = executor.submit(test)
        print(p.result())
        print(i)

test()

What I would expect to happen is that i is decremented, a recursive call is made in the process pool, and then we wait for the result. This happens recursively until i hits 0, at which point the calls cascade back. However, this does not happen. I see the following output

top
foo
top
foo

at which point it hangs

Possible Issues

Race Condition

I figured it could be a race condition and i was not getting decremented every time, but this shouldn’t matter, because the function should get called however many times

Global State

I removed the dependence on Global state by making the recursive calls happen with some random probability. It still hangs after two calls.