Working on a Spring boot application where it listens to kafka topic, process and inserts to DB. Currently we are manually acknowledging the messages and using batchlistener as “true”. We are getting a batch of 500 messages once to process, using “for loop” to process each message and put into DB and then acknowledge the message to kafka.
I have used “ExponentialBackOff” for re-tries, like if DB is down, the current message out of 500 messages, should try for “N” no.of times and finally go to “CommonErrorHandler” and log the message.
The problem here is,
If there is any error in the 500th event, again the whole batch of 500 trying to re-process, which is not acceptable.

I have a spring boot + kafka application, which consumes the messages from kafka, process and update DB. We have set “manual auto commit” and the max poll records as “500” and max poll interval is 1000ms and concurrency set as “100”.

I upgraded spring from 2.x to 3.x and spring-kafka version is also changed. Due to this change ‘ListenableFuture’ is deprecated and I need to use ‘CompletableFuture’. How to change below method?