I’m running a Tempo container for distributed tracing, and I’m encountering an issue where the container fails due to an out-of-memory (OOM) error. This happens when someone searches for a trace in Grafana, and if the parent trace is not received(Root span not received yet), the Tempo container stops and returns the following error:

Failed to get trace with ID: 4e6db4e64ba01c2f192b044b3a540cad. Status:
502 Bad Gateway. Body: 502 Bad

Tempo version: 2.4

current memory allocated for tempo is 4GB

Below is my tempo config file

server:
  http_listen_port: 3200

distributor:
  receivers:
      otlp:
        protocols:
          http:
          grpc:

compactor:
  compaction:
    block_retention: 48h

storage:
  trace:
    backend: s3
    s3:
      region: ${AWS_REGION}
      bucket: ${AWS_BUCKET_NAME}
      endpoint: ${AWS_S3_ENDPOINT}
    wal:
      path: /opt/tempo/wal
    local:
      path: /opt/tempo/blocks
      
overrides:
  defaults:
    ingestion:
      max_traces_per_user: 0
      max_global_traces_per_user: 0