Question regarding the Python import system and the importer (finder, loader) protocol:
In this example on Real Python, a custom importer for importing CSV files is implemented by specializing find_spec
and exec_module
. What puzzles me is the use of exec_module
here, according to my understanding
- create_module is responsible for creating and returning a module object
- exec_module is responsible for executing the module’s code in its namespace.
Since the goal here is to hook into module creation and not module code execution, to me create_module
would be the logical candidate for implementing this logic.
I tried to create_module
for a toy JSON importer and it works just fine:
from importlib.machinery import ModuleSpec
import json
import pathlib
import sys
class JSONImporter:
def __init__(self, json_path):
self.json_path = json_path
@classmethod
def find_spec(cls, name, path, target=None):
*_, module_name = name.rpartition(".")
json_file_name = f"{module_name}.json"
directories = sys.path if path is None else path
for directory in directories:
json_path = pathlib.Path(directory) / json_file_name
if json_path.exists():
return ModuleSpec(name, cls(json_path))
def create_module(self, spec):
try:
with open(self.json_path) as f:
json_object = json.load(f)
except json.JSONDecodeError:
return None
else:
return json_object
def exec_module(self, module):
pass
sys.meta_path.append(JSONImporter)
Obviously the implementation in the Real Python article also does the job, but it seems to me that their implementation using exec_module
is a bit awkward. Am I missing something?