I have a python program which is separated into three files, dielectric_functions.py, routines.py and cartmoments.py, organized as follows —

1. dielectric_functions.py is the entry point to the program and more or less just provides structure and order to the calculation by calling in multiple functions written in routines.py.
2. routines.py contains most functionality, including a function that is throwing up issues with multiprocessing which will discuss now.
3. cartmoments.py implements class objects and certain functions on those class objects (which are parallelized).

The codes are
dielectric_functions.py:

import routines as routines

def initialize_cell() -> tuple[routines.pbcgto.cell.Cell, dict]:
    cell = routines.build_cell_from_input()                                    # Build cell object
    primgauss = routines.gen_all_1D_prim_gauss(cell)                           # Get all primitive gaussian objects
    primindices = routines.gen_prim_gauss_indices(primgauss)                   # Get all main indices for primitive gaussian objects.
    all_ao = routines.gen_all_atomic_orbitals(cell, primgauss)                 # Get all atomic orbitals
    G_vectors = routines.gen_G_vectors(cell)                                   # Get all relevant G vectors
    R_vectors = routines.construct_R_vectors(cell)
    dark_objects = {
        'primitive_gaussians': primgauss,
        'all_ao': all_ao,
        'G_vectors': G_vectors,
        'primindices': primindices[0],
        'atom_locs': primindices[1],
        'R_vectors': R_vectors
    }
    return cell, dark_objects

def dielectric_RPA(dark_objects: dict) -> None:
    routines.primgauss_1D_overlaps(dark_objects)
    return dark_objects

def main():
    cell, dark_objects = initialize_cell()
    #dark_objects = electronic_structure(cell, dark_objects)
    dark_objects = dielectric_RPA(dark_objects)
    return

if __name__ == '__main__':
    main()

routines.py (only relevant part):

from multiprocessing import Pool
import cartmoments
from functools import partial
import numpy as np

def time_wrapper(func):
    """
    Wrapper for printing execution time to logger.
    """
    def wrap(*args, **kwargs):
        logging.info('Entering function {}'.format(func.__name__))
        start = time.time()
        val = func(*args, **kwargs)
        end = time.time()
        logging.info('Exiting function {}. Time taken = {:.2f} s.n'.format(func.__name__, end - start))
        return val

    return wrap

@time_wrapper
def primgauss_1D_overlaps(dark_objects: dict):
    """
    Store all 1D primitive gaussians in files. 
    NOTE: Currently returns one numpy array for all overlaps. This is not good and will not work for anisotropic systems.
    NOTE: Implement file stores with name parmt.store + '/1d_overlaps/{}_{}.npy'.format(d, q) 
    Inputs:
        dark_objects:   dict: equivalent to a class object, except not self-referential.
    """
    primindices = dark_objects['primindices']
    atom_locs = dark_objects['atom_locs']
    q, G = np.load(parmt.store + '/unique_q.npy'), dark_objects['G_vectors']
    Rv = dark_objects['R_vectors']
    makedir(parmt.store + '/primgauss_1d_integrals/')
    for d in range(3):
        qu, Gu = get_all_unique_nums_in_array(q[:,d], round_to=10), get_all_unique_nums_in_array(G[:,d], round_to=10)
        qG = (qu[:, None] + Gu[None, :]).reshape((-1))
        qG = get_all_unique_nums_in_array(qG, round_to=10)
        qG = qG[np.abs(qG) <= parmt.q_max]
        Ru = get_all_unique_nums_in_array(Rv[:,d], round_to=10)
        with Pool(10) as p:
            res = p.map(partial(cartmoments.primgauss_1D_overlaps_uR, primindices = primindices, q = qG, atom_locs = atom_locs[:,d]), Ru)
            p.close()
            p.join()
    logging.info("Generated overlaps of 1D primitive gaussians.")
    return

The issue that I am having is if I run the commands in ipython as

from dielectric_functions import *
main()

it works as expected.

However, if I attempt to run python3 dielectric_functions.py it corrupts my log, attempts to load routines.py tens of times. Note that the code actually still produces the correct results, but there is a bug there that needs to be fixed so logging can continue to exist.

cartmoments.py does not import routines.py and the entirety of function is within cartmoments.py.

I am on python 3.12.4 (main, Jun 6 2024, 18:26:44) [Clang 15.0.0 (clang-1500.3.9.4)] using MacOS in a virtual environment (M3 Pro, 11 cores).

I am completely lost, the fact that running the code on ipython with two separate lines and running it as python3 dielectric_functions.py are not equivalent is surprising.