I am attempting to use Delve to debug an application running remotely, on Kubernetes. The application is an HTTP server (using Gin), so it should start before the remote debugger connects.

Locally, using Docker, everything seems to work fine – the container logs on startup look like this:

API server listening at: [::]:2345
2024-05-27T06:56:35Z warning layer=rpc Listening for remote connections (connections are not authenticated nor encrypted)
2024-05-27T06:56:35Z info layer=debugger launching process with args: [/usr/src/app/bin/app]
2024-05-27T06:56:35Z debug layer=debugger Adding target 11 "/usr/src/app/bin/app"
2024-05-27T06:56:35Z debug layer=debugger continuing
2024-05-27T06:56:35Z debug layer=debugger ContinueOnce
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
...
[GIN-debug] Listening and serving HTTP on :8080

So the debugger starts correctly, and starts the application as well.

However, the same container image started on Kubernetes results in this container log:

API server listening at: [::]:2345
2024-05-27T06:49:17Z warning layer=rpc Listening for remote connections (connections are not authenticated nor encrypted)
2024-05-27T06:49:17Z info layer=debugger launching process with args: [/usr/src/app/bin/app]
2024-05-27T06:49:18Z debug layer=debugger Adding target 12 "/usr/src/app/bin/app"
2024-05-27T06:49:18Z debug layer=debugger continuing
2024-05-27T06:49:18Z debug layer=debugger ContinueOnce
2024-05-27T06:49:18Z debug layer=debugger callInjection protocol on:
2024-05-27T06:49:18Z debug layer=debugger       17 PC=0x4783b7
2024-05-27T06:49:18Z debug layer=debugger       18 PC=0x478983
2024-05-27T06:49:18Z debug layer=debugger       20 PC=0x478983
2024-05-27T06:49:18Z debug layer=debugger       12 PC=0x440944
2024-05-27T06:49:18Z error layer=rpc writing response:write tcp [::1]:2345->[::1]:33826: use of closed network connection

The actual (Gin) application never starts, and the pod exits and is automatically restarted.

Can anyone point me in the right direction for solving this?

K8s version 1.29.3 (vanilla k8s), containerd://1.7.13, calico networking, on Ubuntu 22.04.3. Container based on golang:1.22-alpine, dockerfile below:

FROM golang:1.22-alpine

WORKDIR /usr/src/app

RUN go install github.com/go-delve/delve/cmd/dlv@latest

COPY . .

WORKDIR /usr/src/app/src

RUN --mount=type=cache,target=/go/pkg/mod 
  go mod download && go mod verify

RUN 
  --mount=type=cache,target=/go/pkg/mod 
  --mount=type=cache,target=/root/.cache/go-build 
  go build -v -gcflags="all=-N -l" -o ../bin/app 

EXPOSE 8080
EXPOSE 2345

CMD ["dlv", "--listen=:2345", "--api-version=2", "--headless=true", "--accept-multiclient", "--continue", "--log", "exec", "/usr/src/app/bin/app"]