Given a simple fluent bit config, I am trying to achieve, that the timestamp of the log entry should the parsed from the JSON. Obviously I am missing something very basic.
[SERVICE]
Parsers_File parsers.conf
Log_Level debug
[INPUT]
Name dummy
Dummy {"foo":"bar", "time":"2024-08-29T08:51:25.0000Z"}
[FILTER]
Name parser
Match *
Key_Name message
Parser vault
[OUTPUT]
Name stdout
and a parsers.conf:
[PARSER]
Name vault
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%LZ
Running this gives the following output, which does not seem to parse the timestamp…
Fluent Bit v3.1.6
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
______ _ _ ______ _ _ _____ __
| ___| | | | | ___ (_) | |____ |/ |
| |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __ / /`| |
| _| | | | | |/ _ '_ | __| | ___ | __| / / | |
| | | | |_| | __/ | | | |_ | |_/ / | |_ V /.___/ /_| |_
_| |_|__,_|___|_| |_|__| ____/|_|__| _/ ____(_)___/
[2024/08/29 15:27:24] [ info] Configuration:
[2024/08/29 15:27:24] [ info] flush time | 1.000000 seconds
[2024/08/29 15:27:24] [ info] grace | 5 seconds
[2024/08/29 15:27:24] [ info] daemon | 0
[2024/08/29 15:27:24] [ info] ___________
[2024/08/29 15:27:24] [ info] inputs:
[2024/08/29 15:27:24] [ info] dummy
[2024/08/29 15:27:24] [ info] ___________
[2024/08/29 15:27:24] [ info] filters:
[2024/08/29 15:27:24] [ info] parser.0
[2024/08/29 15:27:24] [ info] ___________
[2024/08/29 15:27:24] [ info] outputs:
[2024/08/29 15:27:24] [ info] stdout.0
[2024/08/29 15:27:24] [ info] ___________
[2024/08/29 15:27:24] [ info] collectors:
[2024/08/29 15:27:24] [ info] [fluent bit] version=3.1.6, commit=, pid=46896
[2024/08/29 15:27:24] [debug] [engine] coroutine stack size: 36864 bytes (36.0K)
[2024/08/29 15:27:24] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/08/29 15:27:24] [ info] [cmetrics] version=0.9.4
[2024/08/29 15:27:24] [ info] [ctraces ] version=0.5.5
[2024/08/29 15:27:24] [ info] [input:dummy:dummy.0] initializing
[2024/08/29 15:27:24] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2024/08/29 15:27:24] [debug] [dummy:dummy.0] created event channels: read=21 write=22
[2024/08/29 15:27:24] [debug] [stdout:stdout.0] created event channels: read=23 write=24
[2024/08/29 15:27:24] [ info] [output:stdout:stdout.0] worker #0 started
[2024/08/29 15:27:24] [ info] [sp] stream processor started
[2024/08/29 15:27:26] [debug] [task] created task=0x600003b74000 id=0 OK
[2024/08/29 15:27:26] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[0] dummy.0: [[1724938045.737938000, {}], {"foo"=>"bar", "time"=>"2024-08-29T08:51:25.0000Z"}]
[2024/08/29 15:27:26] [debug] [out flush] cb_destroy coro_id=0
[2024/08/29 15:27:26] [debug] [task] destroy task=0x600003b74000 (task_id=0)
[2024/08/29 15:27:27] [debug] [task] created task=0x600003b74000 id=0 OK
[2024/08/29 15:27:27] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[0] dummy.0: [[1724938046.738413000, {}], {"foo"=>"bar", "time"=>"2024-08-29T08:51:25.0000Z"}]
[2024/08/29 15:27:27] [debug] [out flush] cb_destroy coro_id=1
[2024/08/29 15:27:27] [debug] [task] destroy task=0x600003b74000 (task_id=0)
[2024/08/29 15:27:28] [debug] [task] created task=0x600003b78000 id=0 OK
[2024/08/29 15:27:28] [debug[0] dummy.0: [[1724938047.737795000, {}], {"foo"=>"bar", "time"=>"2024-08-29T08:51:25.0000Z"}]
] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2024/08/29 15:27:28] [debug] [out flush] cb_destroy coro_id=2
[2024/08/29 15:27:28] [debug] [task] destroy task=0x600003b78000 (task_id=0)
^C[2024/08/29 15:27:28] [engine] caught signal (SIGINT)
[2024/08/29 15:27:28] [ info] [input] pausing dummy.0
[2024/08/29 15:27:28] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2024/08/29 15:27:28] [ info] [output:stdout:stdout.0] thread worker #0 stopped