Assume a KAFKA PRODUCER application (C++, librdkafka, with enable.idempotence=true) which produce msgs 1,2,3,4 and 5 in order for a single topic partition.
Is there a possiblity of kafka writing (i.e. for a Consumer when reading), these msgs with a gap ?
e.g. is there a possiblity of kafka writing 1,2,5 (i.e. when consumer reads, they get 1,2,5) – 3 and 4 missing.
If there is such a possiblity, what can a producer application do to avoid it ?
with enable.idempotence=true the gaps should not occur.
However check for broker failures
check for producer throttling.
ack=all
increase retries if needed
max.in.flight.requests.per.connection=1
refer https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html?utm_medium=sem&utm_source=google&utm_campaign=ch.sem_br.nonbrand_tp.prs_tgt.dsa_mt.dsa_rgn.namer_lng.eng_dv.all_con.docs&utm_term=&creative=&device=c&placement=&gad_source=1&gclid=Cj0KCQiA0–6BhCBARIsADYqyL-OSCgnA5c4cYO4J2Y4_FXTJ9lKzNWNwwHNpGGdQVCMw9o5_Ce7FacaAr6pEALw_wcB to configure producers and consumers accordingly