I’m encountering issues when trying to send data from Vector.dev to OpenMeter using the HTTP sink. Both services are hosted locally in separate Docker containers. The data is processed and transformed by Vector, but when sent to OpenMeter, it seems to fail or not be processed as expected. The data is sent successfully when using a curl command directly, but not through Vector.

I tried the example present in the openmeter/examples, but it seems outdated as the api path has been changed. I tried changing the content-type as per the example, but I’m getting a generic error

2024-08-02T12:54:41.914289Z ERROR sink{component_kind="sink" component_id=openmeter_sink component_type=http}:request{request_id=1}: vector::sinks::util::retries: Not retriable; dropping the request. reason="Http status: 400 Bad Request" internal_log_rate_limit=true

From the vector.dev container, if I try making a curl request and it works fine, and is received by openmeter

curl -X POST http://openmeter:8888/api/v1/events -H "Content-Type: application/cloudevents+json" -d '{"data":{"duration_ms":0},"id":"1234","source":"/my-source","specversion":"1.0","subject":"test","time":"2023-08-02T10:58:11Z","type":"request"}'

the same data when being sent though sink gives error

{"data":{"duration_ms":0},"id":"1234","source":"/my-source","specversion":"1.0","subject":"test","time":"2023-08-02T10:58:11Z","type":"request"}

Vector Configuration (vector.yaml):

transforms:
  om_two:
    type: remap
    inputs:
      - kong_gateway_logs_full
    source: |-
    
      . = if .encoded != null { parse_json!(.encoded) } else { { "data": { "duration_ms": "0" } } }

      # Ensure that the required fields are set
      .specversion = "1.0"
      .subject = "test"
      .type = "request"
      .source = "/my-source"
      .id = "1234"
      .time = "2023-08-02T10:58:11Z"

      .data.duration_ms, err = to_int(.data.duration_ms)
      if err != null {
        .data.duration_ms = 0
        log("Failed to convert duration_ms to integer: " + err)
      }

      . = {
        "specversion": .specversion,
        "subject": .subject,
        "type": .type,
        "source": .source,
        "id": .id,
        "time": .time,
        "data": {
          "duration_ms": .data.duration_ms
        }
      }
sinks:
  openmeter_sink:
    type: http
    inputs:
      - om_two
    uri: "http://openmeter:8888/api/v1/events"
    method: "post"
    encoding:
      codec: "json"
    headers:
      Content-Type: "application/cloudevents+json"
    healthcheck:
      enabled: false
    

There are two other sink present console and clickhouse which work fine