Expected behavior: after calling kafkaListenerEndpointRegistry.stop(), all consumers should be stopped and exit the consumer group
Observed behavior: after calling kafkaListenerEndpointRegistry.stop(), consumers are stopped, but they don’t leave consumer group. The time it takes for them to leave consumer group varies, but it is usually session.timeout.ms, max.poll.interval.ms or up to about 15 minutes in the tests we’ve done
Changed configuration:
properties.put("max.poll.records", 1000);
properties.put("auto.offset.reset", "earliest");
properties.put("session.timeout.ms", 300000); //set to avoid unwanted rebalance
properties.put("max.poll.interval.ms", 300000); //set to avoid unwanted rebalance
properties.put("internal.leave.on.close", "true"); //consumers should leave consumer group on close
some other info:
- using spring boot 3, kafka 3.4.0, java 21, spring-kafka 3.0.10
- using dynamic consumer with unique
group.instance.idvalues - all other operations of the service are functional, but this is a problem when we want to reset offset of consumer group
- setting
internal.leave.on.closeto true or false doesn’t effect this - no errors or warnings are logged at this time
Why would this be happening? Can I get some guidance on what I should look into?