I’m running a Python 3.8 app that is using some proto generated from a private repo. This integration was working until the generated code used the 5.27.0 version.

Working version vs error version

The error I’m getting is something like this:

Traceback (most recent call last):
  File "src/manage.py", line 54, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate
    app_config.import_models()
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/django/apps/config.py", line 211, in import_models
    self.models_module = import_module(models_module_name)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/runner/work/MY_PROJECT/MY_PROJECT/src/MY_PROJECT/models/__init__.py", line 8, in <module>
    from .user_profile import UserProfile
  File "/home/runner/work/MY_PROJECT/MY_PROJECT/src/MY_PROJECT/models/user_profile.py", line 11, in <module>
    from MY_PROJECT.lib.SOME_FILE import AdapultApiWrapper
  File "/home/runner/work/MY_PROJECT/MY_PROJECT/src/MY_PROJECT/lib/SOME_FILE.py", line 8, in <module>
    from MY_PROTOS.common.event.v1 import job_pb2
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/MY_PROTOS/common/event/v1/job_pb2.py", line 9, in <module>
    from google.protobuf import runtime_version as _runtime_version
ImportError: cannot import name 'runtime_version' from 'google.protobuf' (/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/google/protobuf/__init__.py)
Error: Process completed with exit code 1.

I already checked the files in the pod and all the generated code from the protos is there.

It seems the problem can be related to the “_” before “runtime_version”. I’m not sure how to fix this.

Here are the versions I’m using:

• Python 3.8
• pip 24.0
• Buf 1.28.1 (to generate the Python code from .proto)
• Protobuf 4.25.1 but I also tried 5.27.0
• grpcio 1.60.0
• grpcio-tools 1.60.0

Here is the buf.gen.yaml config I use to generate the final code:

version: v1
managed:
  enabled: true
  go_package_prefix:
    default: github.com/PRIVATE_REPO
    except:
      - buf.build/googleapis/googleapis
      - buf.build/grpc-ecosystem/grpc-gateway
plugins:
  - plugin: buf.build/protocolbuffers/go:v1.30.0
    out: gen/go
    opt: paths=source_relative
  - plugin: buf.build/grpc/go:v1.3.0
    out: gen/go
    opt: paths=source_relative,require_unimplemented_servers=false
  - plugin: buf.build/grpc/python:v1.56.1
    out: gen/python
  - plugin: buf.build/protocolbuffers/python
    out: gen/python
  - plugin: buf.build/grpc-ecosystem/gateway:v2.15.2
    out: gen/go
    opt: paths=source_relative
  - plugin: buf.build/grpc-ecosystem/openapiv2:v2.15.2
    out: third_party/OpenAPI

And finally, this is the pyproject.toml file used to generate the requirements.txt file using poetry:

[tool.poetry]
name = "project"
version = "0.1.14"
description = "..."
authors = ["..."]

[tool.poetry.dependencies]
python = "^3.8"
bleach = "^4.1.0"
coverage = "^5.5.0"
django = "^2.2"
django-categories = "^1.8.0"
django-cors-headers = "^3.8.0"
django-filter = "^2.4.0"
django-fsm = "^2.7.0"
django-fsm-log = "^2.0.0"
django-imagekit = "^4.0.2"
django-log-request-id = "^1.6.0"
django-oauth-toolkit = "^1.5.0"
django-permission = "^1.0.4"         # consider removing, not longer mantained.
django-rest-swagger = "^2.1.2"       # consider removing, unmantained
django-simple-history = "^3.0.0"
django-storages = "^1.11.1"
django-taggit = "^1.5.1"
django-taggit-serializer = "^0.1.7"  # consider removing, last update 4 years ago
djangorestframework = "^3.12.4"
google-api-python-client = "^2.20.0"
google-auth = "^2.0.2"
google-cloud-storage = "^1.42.1"
oauth2client = "^4.1.3"
# Limited by django 2.2.x: https://github.com/psycopg/psycopg2/issues/1293                                                                                     # consider migrating, deprecated
psycopg2 = "2.8.6"
pycryptodome = "^3.10.1"
pytest-django = "^4.4.0"
python-gmaps = "^0.3.1"
pyyaml = "^5.4.1"                                                                                                                      # consider migrating, last update 2016                                                                                         # consider removing or moving to official library
smtpapi = "^0.3.1"                                                                                                                     # consider removing or updating
stripe = "^2.60.0"
unittest-xml-reporting = "^3.0.4"
grpcio = "~1.51.3"
grpcio-tools = "~1.51.3"
filestack-python = "^3.4.0"
django-session-header = { git = "https://github.com/kball/django-session-header.git", branch = "master" }                              # consider removing
django-access-tokens = { git = "https://github.com/sixfeetup/django-access-tokens.git", branch = "master" }                            # consider removing
pillow = "^8.3.2"
google-cloud-error-reporting = "^1.2.2"
django-elasticsearch-dsl-drf = "^0.22.2"
django-elasticsearch-dsl = "^7.2.0"
django-safedelete = "^1.0.0"
django-prometheus = "^2.1.0"
private-apis-python = { git = "https://github.com/PRIVATE_REPO", branch = "alternative-package-name", subdirectory = "gen/python" } # names changed for privacy
responses = "^0.14.0"
kafka-python = "^2.0.2"
sonyflake-py = "^1.3.0"

## Not supported yet: 
# grpcio-testing = "^1.46.3"
gunicorn = "^20.1.0"

[tool.poetry.dev-dependencies]
flake8 = "^3.9.2"
black = { version = "^21.8b0", allow-prereleases = true }
rope = "^0.19.0"

[build-system]
requires = ["poetry>=1.0.0"]
build-backend = "poetry.masonry.api"

Here are the final version in the requirements.txt file:

grpcio-status==1.51.3 ; python_version >= "3.8" and python_version < "4.0"
grpcio-tools==1.51.3 ; python_version >= "3.8" and python_version < "4.0"
grpcio==1.51.3 ; python_version >= "3.8" and python_version < "4.0"
googleapis-common-protos==1.60.0 ; python_version >= "3.8" and python_version < "4.0"
googleapis-common-protos[grpc]==1.60.0 ; python_version >= "3.8" and python_version < "4.0"
proto-plus==1.22.3 ; python_version >= "3.8" and python_version < "4.0"
protobuf==4.24.0 ; python_version >= "3.8" and python_version < "4.0"

The app was working in the 4.26.1 version

I tried to manually modify the versions both in the proto generated files and also in the python app.
I’m expecting to be able to run my python app with the generated proto code.