I’m using the Confluent SDK from an existing application (C#), and moving over to the Azure Event Hub Kafka endpoint. I noticed that the Consumer Group, as defined within the constructor parameters, does not get created in Azure Event Hubs; only the Event Hub (or Topic) is created with a $Default consumer group. (I consider this a behaviour change between a native Kafka endpoint, and the Azure Event Hub endpoint).

Using latest as an auto offset policy, I’m finding messages are not consumed directly after a Topic has been created (using the IAdminClient, CreateTopicsAsync method).

I’m new to Kafka, and I know messages are in the Topic, but it seems the offset is not resolved to allow the latest messages to be “consumed”? I also don’t understand the mis-match between a Confluent SDK defined Consumer Group, and the $Default in Azure. I’ve tried to match them up, and still having issues with the Latest policy.

I think I’m missing something basic here when a “Topic” is created and how to initialize it correctly. The only documentation I have found on this issue is here

Thanks in advance for anyone using the Kafa endpoint for Azure Eventhub, and helps me out here.