I am trying to install the dependencies for a project which uses Poetry, inside a docker container.

If I remove the line /root/.local/bin/poetry config virtualenvs.create false Poetry installs, but then then project is installed inside a venv. Because this is inside a docker container, I don’t need or want the packages installed in a venv.

The reason for this is I want other apps and tools inside the container to have access to the installed dependencies. If the deps installed by poetry are in a venv, one has to run a program using poetry run foo to access those deps, but if foo forks new threads or subprocess, they aren’t inside the poetry shell environment and can’t won’t have access to the deps, so a multi-threaded or multi-process application started with poetry run foo doesn’t work.

Therefore I just want to install the poetry deps globally (which is fine, because this is contained in a container).

This is the Dockerfile:

FROM ubuntu:24.04
SHELL ["/bin/bash", "-c"] # Poetry doesn't work with sh

RUN apt-get update && 
apt-get install -y ca-certificates build-essential python3-dev pipx gpg libenchant-2-2

RUN pipx install poetry && pipx ensurepath

COPY ./pyproject.toml /pyproject.toml
COPY ./poetry.lock /poetry.lock

RUN /root/.local/bin/poetry config virtualenvs.create false && 
/root/.local/bin/poetry install -vv --no-interaction

The docker-compose file is very basic:

services:
  app:
    build:
      context: ../
      dockerfile: docker/Dockerfile

Build with: docker compose build

The is the error I get at build time:

...
2.506   - Installing sphinxcontrib-serializinghtml (1.1.5)
2.532   - Installing wrapt (1.16.0)
2.561   - Installing yarl (1.11.1)
3.539 
3.539   Stack trace:
3.540 
3.540   2  ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:338 in _run
3.540        output = subprocess.check_output(
3.540 
3.540   1  /usr/lib/python3.12/subprocess.py:466 in check_output
3.540        return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
3.540 
3.540   CalledProcessError
3.540 
3.540   Command '['/usr/bin/python3.12', '/root/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/virtualenv/seed/wheels/embed/pip-24.2-py3-none-any.whl/pip', 'uninstall', 'packaging', '-y']' returned non-zero exit status 1.
3.540 
3.540   at /usr/lib/python3.12/subprocess.py:571 in run
3.564        567│             # We don't call process.wait() as .__exit__ does that for us.
3.564        568│             raise
3.564        569│         retcode = process.poll()
3.564        570│         if check and retcode:
3.565     →  571│             raise CalledProcessError(retcode, process.args,
3.565        572│                                      output=stdout, stderr=stderr)
3.565        573│     return CompletedProcess(process.args, retcode, stdout, stderr)
3.565        574│ 
3.565        575│ 
3.565 
3.565 The following error occurred when trying to handle this error:
3.565 
3.565 
3.565   Stack trace:
3.565 
3.565   9  ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:285 in _execute_operation
3.565        result = self._do_execute_operation(operation)
3.565 
3.565   8  ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:395 in _do_execute_operation
3.566        result: int = getattr(self, f"_execute_{method}")(operation)
3.566 
3.566   7  ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:527 in _execute_update
3.566        status_code = self._update(operation)
3.566 
3.566   6  ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:586 in _update
3.566        return self._install(operation)
3.566 
3.566   5  ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:576 in _install
3.566        self._remove(operation.initial_package)
3.566 
3.566   4  ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:596 in _remove
3.566        return self.run_pip("uninstall", package.name, "-y")
3.566 
3.566   3  ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:418 in run_pip
3.567        self._env.run_pip(*args, **kwargs)
3.567 
3.567   2  ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:310 in run_pip
3.567        return self._run(cmd, **kwargs)
3.567 
3.567   1  ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/generic_env.py:99 in _run
3.567        return super(VirtualEnv, self)._run(cmd, **kwargs)
3.567 
3.567   EnvCommandError
3.567 
3.567   Command ['/usr/bin/python3.12', '/root/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/virtualenv/seed/wheels/embed/pip-24.2-py3-none-any.whl/pip', 'uninstall', 'packaging', '-y'] errored with the following return code 1
3.567   
3.567   Output:
3.567   error: externally-managed-environment
3.567   
3.567   × This environment is externally managed
3.567   ╰─> To install Python packages system-wide, try apt install
3.567       python3-xyz, where xyz is the package you are trying to
3.567       install.
3.567       
3.567       If you wish to install a non-Debian-packaged Python package,
3.567       create a virtual environment using python3 -m venv path/to/venv.
3.567       Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
3.567       sure you have python3-full installed.
3.567       
3.567       If you wish to install a non-Debian packaged Python application,
3.567       it may be easiest to use pipx install xyz, which will manage a
3.567       virtual environment for you. Make sure you have pipx installed.
3.567       
3.567       See /usr/share/doc/python3.12/README.venv for more information.
3.567   
3.567   note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
3.567   hint: See PEP 668 for the detailed specification.
3.567   
3.567 
3.567   at ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:342 in _run
3.572       338│                 output = subprocess.check_output(
3.572       339│                     cmd, stderr=stderr, env=env, text=True, **kwargs
3.572       340│                 )
3.572       341│         except CalledProcessError as e:
3.572     → 342│             raise EnvCommandError(e)
3.572       343│ 
3.572       344│         return output
3.572       345│ 
3.572       346│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
3.572 
3.572 Cannot install packaging.

What is interesting about this, is if I start a blank Ubuntu container with docker run -it --rm ubuntu:22.04 and copy and paste all the commands in the Dockerfile into the shell, everything works. So maybe this is some Docker issue?

From the error I guess that poetry is using pip or pipx behind the scenes and there is no way that I can find to pass the --break-system-packages option.