Working on a Spring boot application where it listens to kafka topic, process and inserts to DB. Currently we are manually acknowledging the messages and using batchlistener as “true”. We are getting a batch of 500 messages once to process, using “for loop” to process each message and put into DB and then acknowledge the message to kafka.
I have used “ExponentialBackOff” for re-tries, like if DB is down, the current message out of 500 messages, should try for “N” no.of times and finally go to “CommonErrorHandler” and log the message.
The problem here is,
If there is any error in the 500th event, again the whole batch of 500 trying to re-process, which is not acceptable.

To resolve it, started throwing “BatchListenerFailedException” with index, instead of Just “Exception”. But it introduces other problem that, its not trying for “N” no.of times, as soon as control goes to catch block with “BatchListenerFailedException”, its going to common error handler and skipping the event.

To resolve it, we need to keep the track of how many re-tries so far, so at the final attempt we can throw “BatchListenerFailedException”. But not sure how to get the retry count in batch listener as @RetryableTopic is not supported with batch listeners.
How can I solve the problem, any suggestions?

for(int i=0;i<list.size();i++){
try{
    acknowledgement.acknowledge();

}catch(Exception e){

throw e; // Option 1 - If we throw direct exception, its trying while batch again
throw new BatchListenerFailedException("Failed", i); // Option 2 - If we throw it, its not trying for "N" no.of times, directly skipping and going to next record.
   }
}