I’m working on an AWS project where a file is uploaded to a specific S3 bucket, which then triggers an AWS Lambda function. This function reads the uploaded JSON file, asks GPT to summarize the text, and then writes the summary to a new JSON file in a separate S3 bucket.

The setup involves two Lambda layers:

pydantic_layer for pydantic_core
openai_layer for OpenAI dependencies

Here is my process:

Created a directory structure for the layers: python/lib/python3.8/site-packages/
Used pip to install the dependencies into the respective directories
Zipped the directories and uploaded them as Lambda layers
Added the layers to the Lambda function, which runs on Python 3.8 with architecture x86_64

However, I keep encountering the following error when I try to test on Lambda Code Source:

No module named ‘pydantic_core._pydantic_core’

Steps I’ve taken:

Created pydantic_layer:

mkdir -p python/lib/python3.8/site-packages
pip install –target python/lib/python3.8/site-packages pydantic-core
zip -r pydantic_layer.zip python

Created openai_layer:

mkdir -p python/lib/python3.8/site-packages
pip install –target python/lib/python3.8/site-packages openai
zip -r openai_layer.zip python

Uploaded the ZIP files as layers in the AWS Lambda console

Added both layers to the Lambda function

Despite this, I still get the No module named ‘pydantic_core._pydantic_core’ error. Any help resolving this would be greatly appreciated.