I am required to run a particular AI model in a particular Azure ML Studio notebooks environment so it can run through my work’s model review process.

My model runs from a custom module, and this module code loads a saved pickle file to load the AI model (think of it as like a learned decision tree model).

In a regular Python environment, I set up the model just fine as follows:

1. import my_module

2. sys.modules['my_module'] = my_module # So that pickle can load the saved model properly

3. my_inst = my_module.new_inst()

4. my_inst.load('path_to_my_model')

(The need for step 2 is explained here: /a/2121918/3860846)

I can set up my module code in Azure with the following structure:

Users
-> My.Name
   -> my_module
      -> mod_file1.py
      -> mod_file2.py 
      -> (etc)
   -> models
      -> my_model.pkl
   -> scripts
      -> test_model.py

Problem 1

I can’t seem to import custom modules inside test_model.py in the Azure notebook environment (neither in a .py script nor a .ipynb notebook). Relative imports give the “no known parent package” message. Appending the module’s parent dir to sys.path doesn’t seem to help either (even when using pathlib.Path("..").resolve() to get the absolute path, which from this answer would seem to have worked if this was in an Azure Functions environment.). I’m very inexperienced with Azure though, and am not sure how its environments are set up.

The only way around this that I’ve found is to manually squish all my module’s files into a single file and then load that file in Azure using

with open("my_merged_module.py", 'r') as f:
    file_string = f.read()
exec(file_string)

But while that lets me access the functions in my module, it creates…

Problem 2

If I load my module as a single script rather than as a module in order to accomplish step 1, then I can’t do step 2 properly, since it’s not a module anymore. That means step 4 (pickle load) fails.

I am wondering:

Is there a way to import a custom module in Azure so that I can solve problem 1 without having to face problem 2?

Or is there a way around problem 2 while using my hack to avoid problem 1?

Thanks!