I’m new to sumologic and otel. I have been trying to export some application logs to sumologic using sumologic otel collector with java agent automatic instrumentation. But I see only partial logs i.e part of the log lines are not present
this is what actual raw log looks like
2024-04-30 11:16:15.029 DEBUG 8 --- [nio-8080-exec-2] m.m.a.RequestResponseBodyMethodProcessor : Writing ["Docker Demo - Hello Service logging sample"]
But I don’t see any info like RequestResponseBodyMethodProcessor
or DEBUG
anywhere in sumologic. This is what sumologic log look like this.
{"aws_batch":"3a7469f7-3009-4d2b-92b4-3415afd6dfd4","batch":"jservice","log":"Writing ["Docker Demo - Hello Service logging sample"]"}
Apart from this I see Message Time and Receipt Time along with some metadata.
This is my sumo otel config. Interested only in logs so metrics and traces config can be ignored
extensions:
sumologic:
installation_token: ${SUMOLOGIC_INSTALLATION_TOKEN}
collector_name: "jservice"
collector_category: "jservice_cat"
health_check:
endpoint: localhost:13133
opamp:
remote_configuration_directory: /etc/otelcol-sumo/opamp.d
endpoint: wss://opamp-events.sumologic.com/v1/opamp
receivers:
otlp:
protocols:
grpc:
http:
cors:
allowed_origins:
- "http://*"
- "https://*"
httpcheck/frontendproxy:
targets:
- endpoint: http://x.x.x.x:8080
hostmetrics:
collection_interval: 2s
scrapers:
load:
cpu:
memory:
process:
awsecscontainermetrics:
collection_interval: 2s
exporters:
debug:
sumologic:
auth:
authenticator: sumologic
client: "jservice"
timeout: 4s
otlp:
endpoint: "x.x.x.x:32782"
tls:
insecure: true
otlp/logs:
endpoint: "x.x.x.x:21892"
tls:
insecure_skip_verify: true
otlphttp/prometheus:
endpoint: "http://x.x.x.x:9090/api/v1/otlp"
tls:
insecure_skip_verify: true
processors:
memory_limiter:
check_interval: 5s
limit_percentage: 75
spike_limit_percentage: 20
batch:
send_batch_size: 50
timeout: 5s
filter/ottl:
error_mode: ignore
metrics:
metric:
- 'name == "rpc.server.duration"'
transform:
metric_statements:
- context: metric
statements:
- set(description, "") where name == "queueSize"
- set(description, "") where name == "http.client.duration"
transform/logs:
log_statements:
- context: log
statements:
- set(attributes["batch"], "jservice")
- set(attributes["aws_batch"], "${AWS_BATCH_JOB_ID}")
sumologic:
resourcedetection/ecs:
detectors: [env, ecs]
resourcedetection/system:
detectors: ["system"]
timeout: 2s
override: false
connectors:
spanmetrics:
service:
extensions:
- sumologic
- health_check
telemetry:
metrics:
address: localhost:8888
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [otlp, debug, spanmetrics]
metrics:
receivers: [httpcheck/frontendproxy, otlp, spanmetrics]
processors: [filter/ottl, transform, batch]
exporters: [otlphttp/prometheus, debug]
logs:
receivers: [otlp, awsecscontainermetrics, hostmetrics]
processors: [batch, memory_limiter, sumologic, transform/logs, resourcedetection/ecs, resourcedetection/system]
exporters: [sumologic]
I tried log transforms and parse expression but nothing of use since the DEBUG info is not exported in the first place. It must be something basic but without expertise in otel I could not figure out what? Is it because of log not in otel format? Is there a way around it?
raghu.pyr is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.