I am exectuting a python script as a subprocess. Doing this from a regular module ineherits the virtual environment but when I do it inside a Jupyter notebook, the subprocess is using the system python. Why is this and how can I fix it in a platform independent way? (I do not know what virtual environments will be used by other users.)
A minimal example:
# kernel_printer.py
import sys
import os
def print_kernel():
kernel_name = os.path.basename(sys.executable.replace("/bin/python", ""))
print('Kernel: ', kernel_name)
if __name__ == '__main__':
print_kernel()
Code to be exectued either as a script or as a notebook cell:
import subprocess
print(subprocess.run(['python', 'kernel_printer.py'], capture_output=True))
# as module: subprocess output contains print of activated virtualenv
# as notebook cell: ... systemwide python is printed as current kernel
Note that the jupyter notebook indeed is running the same virtualenv as the module. That is, the following code prints the name of the virtual env regardless if run in a module or a notebook cell.
import kernel_printer
kernel_printer.print_kernel()
# always prints name of virtual env
I still don’t know the mechanics of Jupyter and thus not why this happens. But I did find a simple solution. This only works if you can change the code that is being called. In that case, you can use the current python executable stored in sys.executable, as in
import sys
import subprocess
print(subprocess.run([sys.executable, 'kernel_printer.py'], capture_output=True))
# outputs the virtual env as kernel regardless if run as module or notebook cell