We recently started migrating our test clusters from Zookeeper to KRaft, all running Confluent Kafka on prem. In KRaft, cluster’s metadata is stored in __cluster_metadata topic (see: https://developer.confluent.io/courses/architecture/control-plane/#kraft-cluster-metadata) . What I noticed after the first cluster was migrated is that I am not able to describe this topic using kafka-topics command, it just does not exist:

Error while executing topic command : Topic ‘__cluster_metadata’ does not exist as expected

Yet, the topic’s directory is found in the broker:

sh-4.4$ cd /mnt/data/data0/logs
sh-4.4$ ls -dl __cluster_metadata-0
drwxr-sr-x. 2 1002150000 1002150000 4096 Jul 25 08:36 __cluster_metadata-0

Confluent’s support replied to this issue saying “it is not an actual internal kafka topic“.

If you run Kafka with KRaft, can you actually check if __cluster_metadata topic can be described with kafka-topics command? Not that I would need this on a daily basis, I just want to check what __cluster_metadata really is.