I have a kafka topic t1 and a topic t2. A consumer group g1 and a consumer group g2 are consuming from t1 and t2 respectively as shown in the figure below. This sequential architecture is deployed on a Kubernetes cluster.

I also have a controller process that is simultaneously (concurrently using different threads) scaling g1 and g2 to 2 replicas. The aim is that g1 and g2 are launched concurrently and eventually undergo concurrent (pseudo-parallel) rebalancing so that to achieve almost ~ one rebalancing overhead. However, I notice that the two groups are undergoing rather sequential not concurrent/parallel rebalancing.

Can someone provide me a hint on what’s wrong with my methodology/setup/thinking, for example is that the kube API server that is not launching the replicas concurrently or Kafka can not handle concurrent rebalance of different consumer group (I doubt).

Thank you.