I have a Flink pipeline with RMQ source, some filters, enrichers ,keyby, aggregator ,Tumbling window(of 2sec,no watermark strategy is used. Processing time is used for windows trigger) and then RMQ sink. Increametal checkpoints is enabled with rocks DB as state backend.
Over the period of time, i see that my checkpoint size is increasing(>50MB). Even if compaction runs only significant amount of checkpoint size is reducing ie just 3 to 5MB(checkpoint size is still greater than 45MB).
I observed from flink webUI that its my tumbling window which keeps lot of check pointed data. After the window ends, i want my window operator to clean up the state and check pointed size in UI to return to 0(I understand that tumbling windows clears the state automatically after the window ends ). But that is not happening. Why?
I am not able to configure stateTTLs as it is not a rich Fuction. Now how can i clear the checkpoint size of the window operator.
I am kind of stuck, any help will really be appreciated.