I’m encountering an issue while ingesting JSON structured logs from Fluent Bit into Azure Data Explorer

I create Kusto table named LogsTable with specific columns like timestamp, message, container_name, log, level, etc.

But when fluent-bit send data to my ADX cluster, all information go only in 1 column, “log”
Here the content of my column log

{"container_id":"f92fb57e79d50cfa6e4ba89fc1e7feee50d669dea5879722f436108a84bd502f","log":"06/26/2024 18:08:33 - [INFO] Client connected","date":"06/26/2024","container_name":"/pandapache3_pandapache3_1","time":"18:08:33","level":"INFO","message":"Client connected","source":"stdout"}

The result expected, would be to have the container_id go in the column named container_id, etc…

I create a JSON ingestion mapping

.create-or-alter table LogsTable ingestion json mapping 'LogsMapping' ```[{
    "column": "timestamp",
    "path": "$.timestamp",
    "datatype": "datetime"
}, {
    "column": "message",
    "path": "$.message",
    "datatype": "string"
}, {
    "column": "container_name",
    "path": "$.container_name",
    "datatype": "string"
}, {
    "column": "log",
    "path": "$.log",
    "datatype": "string"
}, {
    "column": "source",
    "path": "$.source",
    "datatype": "string"
}, {
    "column": "level",
    "path": "$.level",
    "datatype": "string"
}, {
    "column": "date",
    "path": "$.date",
    "datatype": "string"
}, {
    "column": "time",
    "path": "$.time",
    "datatype": "string"
}, {
    "column": "container_id",
    "path": "$.container_id",
    "datatype": "string"
}]```

but even like that it didn’t change the output.

I’m blocking on that since days know, trying everything to make it work without success, it’s always go in the log column and that all.

The log is the original record of fluent-bit, but to have other column I wrote a lua script that extract from this initial record other column

    -- Example regex to extract date, time, level, and message
    local log = record["log"]
    local date, time, level, message = string.match(log, "(%d%d/%d%d/%d%d%d%d) (%d%d:%d%d:%d%d) %- %[(.-)%] (.*)")

    print("Record:", record)
    print("Log:", log)
    print("Date:", date)
    print("Time:", time)
    print("Level:", level)
    print("Message:", message)

    record["date"] = date
    record["time"] = time
    record["level"] = level
    record["message"] = message

    return 1, timestamp, record
end

I think the issue could be somewhere in my configuration file for fluent-bit, something I do wrong, but I didn’t succeed to solve it until now

[SERVICE]
    Daemon Off
    Flush 1
    Log_Level trace
    HTTP_Server On
    HTTP_Listen 0.0.0.0
    HTTP_Port 8080
    Health_Check On
    Parsers_File parsers.conf

[INPUT]
    Name forward
    Listen 0.0.0.0
    Port 24224

[FILTER]
    Name    lua
    Match   *
    Script  /fluent-bit/etc/script.lua
    Call    cb_filter

Any help will be appreciate !