I’m encountering an issue with distributed tracing and baggage propagation in my Spring Boot application (version 3.2) utilizing the Actuator and Micrometer tracing with Brave.

My use-case:

We have 3 microservices:

• Service 1: A REST API written in C# also functioning as an Apache Kafka producer. It sends event information to a Kafka topic, including baggage information in the Kafka message headers (e.g., X-USER-ID), which needs to be propagated through the microservices chain.
• Service 2: A Spring Boot application consuming messages from the Kafka topic, performing modifications, and sending HTTP POST requests to Service 3.
• Service 3: A third-party REST API executing specific functionality.

Service 1 (REST API and kafka producer) -> Service 2 (Kafka consumer) -> Service 3 (REST API)

My primary objective is to implement service 2 and propagate information from the Kafka message headers produced in Service 1 to Service 3. However, I encounter an issue when the header keys are not in lowercase.

The problem

I’ve observed that when I produce a message into Kafka with headers in lowercase (e.g., x-user-id), the application behaves as expected. The value from this header appears in the MDC context variable (X-USER-ID) and is also propagated to Service 3 as an HTTP header (x-user-id).

However, when I produce a message into Kafka with headers in uppercase (e.g., X-USER-ID), the application doesn’t parse the header’s value into the MDC context and also fails to propagate it to Service 3 as an HTTP header.

I expected that baggage propagation in Kafka headers would be case-insensitive, similar to HTTP headers. However, it seems that Spring baggage propagation only works with baggage whose keys are in lowercase. Is this statement correct? Or am I overlooking something?

Is there a way to achieve the desired behavior without modifying Service 1?

I know that I can create BaggageField beans manually, takes the values from kafka message headers and updates their’s values before processing the messages, but this is not very universal approach since I expect that the list of headers which will be propagated will contains more values in the future.

Thank you in advance for your assistance.

Current Kafka Consumer Listener Configuration:

Below is a simplified version of my consumer application configuration (Service 2):

spring:
  kafka:
    listener:
      type: single
      observation-enabled: true
management:
  tracing:
    enabled: true
    propagation:
      type: b3
    baggage:
      enabled: true
      correlation:
        enabled: true
        fields: X-USER-ID
      remote-fields: X-USER-ID

I’ve also attempted to bypass Spring Boot’s BaggageField autoconfiguration by manually creating instances of BaggagePropagationCustomizer and CurrentTraceContext.ScopeDecorator, as it offers more options (such as adding multiple names). However, this didn’t resolve the issue.