I need to add a custom field to all log entries that contain string “myapp.database – Statement returned”. All other logs that do not contain this string, should not be discarded/filtered out.

I managed to add the custom field by applying grep, parser and record_transformer filters, but at the cost of all other logs being filtered out or the log entries containing string “myapp.database – Statement returned” being duplicated. Neither of these results is acceptable – as i said, the log entries containing the string should get the new field and the remaining log entries should not be filtered out.

Eventually i tried to apply label routing but it resulted in log entries being filtered out and custom field not being added at all. Below is my fluentd config where i tried to apply labels.

<source>
  @type tail
  path C:dmskrnl.log
  pos_file C:dmskrnl.log.pos
  tag dmskrnl.raw  
  <parse>
    @type none
  </parse>
</source>

<filter dmskrnl.raw>
  @type grep
  <regexp>
    key message
    pattern /myapp.database - Statement returned/
  </regexp>
  @label @ENRICH
</filter>
 
<label @ENRICH>
  <filter dmskrnl.raw>
    @type parser
    key_name message
    reserve_data true
    tag dmskrnl.enriched  
    <parse>
      @type regexp
      expression /myapp.database - Statement returned (?<returned_bytes>d+) bytes/
    </parse>
</filter>

<filter dmskrnl.enriched>
  @type record_transformer  
  <record>    
    returned_bytes ${record["returned_bytes"]}
  </record>
</filter>

<match dmskrnl.enriched>
  @type stdout
</match>
</label>
  
<match dmskrnl.raw>
  @type stdout
</match>