I am trying to run a single node Kafka with Docker Compose, but I keep getting the following error when I start the containers:
[2024-05-27 12:19:31,223] WARN [RequestSendThread controllerId=1] Controller 1's connection to broker kafka:9092 (id: 1 rack: null) was unsuccessful (kafka.controller.RequestSendThread)
kafka | java.io.IOException: Connection to kafka:9092 (id: 1 rack: null) failed.
kafka | at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
kafka | at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:298)
kafka | at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
kafka | at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:130)
kafka | [2024-05-27 12:19:31,224] INFO [Controller id=1, targetBrokerId=1] Client requested connection close from node 1 (org.apache.kafka.clients.NetworkClient)
kafka | [2024-05-27 12:19:31,225] INFO [Controller id=1] Partitions undergoing preferred replica election: (kafka.controller.KafkaController)
kafka | [2024-05-27 12:19:31,225] INFO [Controller id=1] Partitions that completed preferred replica election: (kafka.controller.KafkaController)
kafka | [2024-05-27 12:19:31,225] INFO [Controller id=1] Skipping preferred replica election for partitions due to topic deletion: (kafka.controller.KafkaController)
kafka | [2024-05-27 12:19:31,225] INFO [Controller id=1] Resuming preferred replica election for partitions: (kafka.controller.KafkaController)
kafka | [2024-05-27 12:19:31,225] INFO [Controller id=1] Starting replica leader election (PREFERRED) for partitions triggered by ZkTriggered (kafka.controller.KafkaController)
kafka | [2024-05-27 12:19:31,228] INFO [ExpirationReaper-1-AlterAcls]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
kafka | [2024-05-27 12:19:31,233] INFO [Controller id=1] Starting the controller scheduler (kafka.controller.KafkaController)
kafka | [2024-05-27 12:19:31,243] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
kafka | [2024-05-27 12:19:31,249] INFO [SocketServer listenerType=ZK_BROKER, nodeId=1] Enabling request processing. (kafka.network.SocketServer)
kafka | [2024-05-27 12:19:31,250] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor)
kafka | [2024-05-27 12:19:31,253] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.DataPlaneAcceptor)
kafka | [2024-05-27 12:19:31,257] INFO Kafka version: 7.6.1-ccs (org.apache.kafka.common.utils.AppInfoParser)
kafka | [2024-05-27 12:19:31,257] INFO Kafka commitId: 11e81ad2a49db00b1d2b8c731409cd09e563de67 (org.apache.kafka.common.utils.AppInfoParser)
kafka | [2024-05-27 12:19:31,257] INFO Kafka startTimeMs: 1716812371255 (org.apache.kafka.common.utils.AppInfoParser)
kafka | [2024-05-27 12:19:31,258] INFO [KafkaServer id=1] started (kafka.server.KafkaServer)
kafka | [2024-05-27 12:19:31,329] INFO [RequestSendThread controllerId=1] Controller 1 connected to kafka:9092 (id: 1 rack: null) for sending state change requests (kafka.controller.RequestSendThread)
kafka | [2024-05-27 12:19:31,350] TRACE [Controller id=1 epoch=11] Received response UpdateMetadataResponseData(errorCode=0) for request UPDATE_METADATA with correlation id 0 sent to broker kafka:9092 (id: 1 rack: null) (state.change.logger)
kafka | [2024-05-27 12:19:31,356] INFO [zk-broker-1-to-controller-forwarding-channel-manager]: Recorded new controller, from now on will use node kafka:9092 (id: 1 rack: null) (kafka.server.BrokerToControllerRequestThread)
kafka | [2024-05-27 12:19:31,416] INFO [zk-broker-1-to-controller-alter-partition-channel-manager]: Recorded new controller, from now on will use node kafka:9092 (id: 1 rack: null) (kafka.server.BrokerToControllerRequestThread)
Docker Compose file that I am using is like this:
version: '3'
services:
zookeeper:
container_name: zookeeper
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ports:
- 22181:2181
networks:
- mtn
kafka:
container_name: kafka
image: confluentinc/cp-kafka:latest
depends_on:
- zookeeper
ports:
- 29092:29092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_NUM_PARTITIONS: 1
networks:
- mtn
I am trying to expose port 29092 from the host, this is because the application running on the host tries to connect to that port.
TIA.