i have following docker-compose:
version: '2'
services:
zookeeper-1:
image: confluentinc/cp-zookeeper:7.4.4
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ports:
- 22181:2181
volumes:
- stream-zookeeper1-log:/var/lib/zookeeper/log
- stream-zookeeper1-secrets:/etc/zookeeper/secrets
- stream-zookeeper1-data:/var/lib/zookeeper/data
zookeeper-2:
image: confluentinc/cp-zookeeper:7.4.4
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ports:
- 32181:2181
volumes:
- stream-zookeeper2-log:/var/lib/zookeeper/log
- stream-zookeeper2-secrets:/etc/zookeeper/secrets
- stream-zookeeper2-data:/var/lib/zookeeper/data
kafka-1:
image: confluentinc/cp-kafka:7.4.4
depends_on:
- zookeeper-1
- zookeeper-2
ports:
- 29092:29092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-1:9092,PLAINTEXT_HOST://10.55.0.1:29092
KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,PLAINTEXT_HOST://0.0.0.0:29092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
volumes:
- stream-kafka1-data:/var/lib/kafka/data
- stream-kafka1-secrets:/etc/kafka/secrets
kafka-2:
image: confluentinc/cp-kafka:7.4.4
depends_on:
- zookeeper-1
- zookeeper-2
ports:
- 39092:39092
environment:
KAFKA_BROKER_ID: 2
KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-2:9092,PLAINTEXT_HOST://10.55.0.1:39092
KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,PLAINTEXT_HOST://0.0.0.0:39092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
volumes:
- stream-kafka2-data:/var/lib/kafka/data
- stream-kafka2-secrets:/etc/kafka/secrets
volumes:
stream-zookeeper1-log:
stream-zookeeper2-log:
stream-zookeeper1-secrets:
stream-zookeeper2-secrets:
stream-zookeeper1-data:
stream-zookeeper2-data:
stream-kafka1-data:
stream-kafka2-data:
stream-kafka1-secrets:
stream-kafka2-secrets:
it starts successfully and works pretty nice. but if the stack is restarted or the server it self. the following error occurs:
[2024-08-01 19:05:46,947] INFO [ZooKeeperClient Kafka server] Connected. (kafka.zookeeper.ZooKeeperClient)
[2024-08-01 19:05:47,210] INFO Cluster ID = S2ymLgAIQq-Lzgem0bGBsg (kafka.server.KafkaServer)
[2024-08-01 19:05:47,215] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentClusterIdException: The Cluster ID S2ymLgAIQq-Lzgem0bGBsg doesn't match stored clusterId Some(kjs0fZu2Rb2td4bUnHELQQ) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.
at kafka.server.KafkaServer.startup(KafkaServer.scala:235)
at kafka.Kafka$.main(Kafka.scala:115)
at kafka.Kafka.main(Kafka.scala)
[2024-08-01 19:05:47,217] INFO shutting down (kafka.server.KafkaServer)
[2024-08-01 19:05:47,219] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2024-08-01 19:05:47,324] INFO Session: 0x101116b6e3f0001 closed (org.apache.zookeeper.ZooKeeper)
[2024-08-01 19:05:47,324] INFO EventThread shut down for session: 0x101116b6e3f0001 (org.apache.zookeeper.ClientCnxn)
[2024-08-01 19:05:47,325] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2024-08-01 19:05:47,329] INFO App info kafka.server for 1 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2024-08-01 19:05:47,329] INFO shut down completed (kafka.server.KafkaServer)
[2024-08-01 19:05:47,329] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$)
why is this happening? config/state is saved in volumes.
what can i do to prevent this error?
i check following post out: Kafka Broker doesn’t find cluster id and creates new one after docker restart
but nothing works to fix this issue.