I am attempting to provide an S3 bucket to an application, using localstack (https://github.com/localstack/localstack) Helm chart as the implementation. The application has a Helm chart that I use to configure and deploy to the cluster. I added localstack as a chart dependency.

I first tested deploying to a single-node cluster provided by my locally installed Docker Desktop. I worked through various issues until the charts deployed, and as far as I am able to tell, have successfully connected and are running without errors.

My next step was to try to deploy all of this to one of our multi-node Kubernetes clusters, in this case, using Kubeapps to manage the charts deployment. A few differences in the charts values YAML is the only (obvious) difference from the prior testing.

When I do this, localstack throws multiple exceptions, that seem to be centered around a problem with Docker. It seems odd that according to these errors, it would seem that Docker is “missing”.

Here is a sample log from the localstack pod:

LocalStack version: 3.4.1.dev
LocalStack Docker container name: localstack-main
Failed to inspect docker container: Docker not available Traceback (most recent call last):
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.11/http/client.py", line 996, in send
    self.connect()
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/docker/transport/unixconn.py", line 26, in connect
    sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/requests/adapters.py", line 589, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/urllib3/util/retry.py", line 470, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/urllib3/util/util.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.11/http/client.py", line 996, in send
    self.connect()
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/docker/transport/unixconn.py", line 26, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/docker/api/client.py", line 223, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
                        ^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/docker/utils/decorators.py", line 44, in inner
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/docker/api/client.py", line 246, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/requests/adapters.py", line 604, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/code/localstack/localstack/utils/container_utils/docker_sdk_client.py", line 78, in _create_client
    return docker.from_env(timeout=DOCKER_SDK_DEFAULT_TIMEOUT_SECONDS)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/docker/client.py", line 94, in from_env
    return cls(
           ^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/docker/client.py", line 45, in __init__
    self.api = APIClient(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/docker/api/client.py", line 207, in __init__
    self._version = self._retrieve_server_version()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.11/site-packages/docker/api/client.py", line 230, in _retrieve_server_version
    raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/code/localstack/localstack/services/infra.py", line 208, in print_runtime_information
    inspect_result = DOCKER_CLIENT.inspect_container(container_name)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/localstack/utils/container_utils/docker_sdk_client.py", line 374, in inspect_container
    return self.client().containers.get(container_name_or_id).attrs
           ^^^^^^^^^^^^^
  File "/opt/code/localstack/localstack/utils/container_utils/docker_sdk_client.py", line 69, in client
    self.docker_client = self._create_client()
                         ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/localstack/utils/container_utils/docker_sdk_client.py", line 91, in _create_client
    raise DockerNotAvailable("Docker not available") from e
localstack.utils.container_utils.container_client.DockerNotAvailable: Docker not available

LocalStack build date: 2024-05-24
LocalStack build git hash: 0a410d0c8

Ready.
make_bucket failed: s3://dex1 Could not connect to the endpoint URL: "http://localhost:4566/dex1"

Hopefully, someone can make sense of these errors. It seems very unlikely (to me anyway) that Docker would not be available.