I’m trying to filter logs to make it cleaner and more readable before forwarding it to ElasticSearch using this fluent bit configuration:

This is one of the example raw logs i’m receiving:

Jun 25 13:27:34 host3 fluent-bit[745468]: [0] cpu_usage: [[1719322045.744281337, {}], {"cpu_p"=>3.250000, "user_p"=>2.750000, "system_p"=>1.60000, "cpu0.p_cpu"=>3.000000, "cpu0.p_user"=>2.000000, "cpu0.p_system"=>1.000000, "cpu1.p_cpu"=>5.600000, "cpu1.p_user"=>4.8000000, "cpu1.p_system"=>0.800000, "cpu2.p_cpu"=>3.000000, "cpu2.p_user"=>2.000000, "cpu2.p_system"=>1.000000, "cpu3.p_cpu"=>5.600000, "cpu3.p_user"=>4.8000000, "cpu3.p_system"=>0.800000}]

fluent-bit.conf:

[SERVICE]
    Daemon Off
    Flush 1
    Log_Level info
    Parsers_File parsers.conf
    HTTP_Server On
    HTTP_Listen 0.0.0.0
    HTTP_Port 2020
    Health_Check On

[INPUT]
    Name cpu
    Tag cpu_usage
    Interval_Sec 5

[FILTER]
    Name parser
    Match cpu_usage*
    Key_Name filtered_cpu
    Parser custom-log-filter

[OUTPUT]
    Name Stdout
    Match *

I’m using Regex to filter out the strings that I want to remove. In my parsers.conf this is my configuration

parsers.conf:

[PARSER]
     Name   custom-log-filter
     Format regex
     Regex  ^w{3} d{2} d{2}:d{2}:d{2} .*?: [0] |[.+], |]$

Using https://regexr.com/ I was able to select the strings that I want to remove. How do I “reverse” the selection such that I return everything else other than the one selected. Please let me know if there’s a better way of doing this. Thanks in advance

Expected Output:
host3 cpu_usage: {"cpu_p"=>3.250000, "user_p"=>2.750000, "system_p"=>1.60000, "cpu0.p_cpu"=>3.000000, "cpu0.p_user"=>2.000000, "cpu0.p_system"=>1.000000, "cpu1.p_cpu"=>5.600000, "cpu1.p_user"=>4.8000000, "cpu1.p_system"=>0.800000, "cpu2.p_cpu"=>3.000000, "cpu2.p_user"=>2.000000, "cpu2.p_system"=>1.000000, "cpu3.p_cpu"=>5.600000, "cpu3.p_user"=>4.8000000, "cpu3.p_system"=>0.800000}