I’m new to debezium and having trouble getting it setup in minikube (kubernetes later). I followed the tutorial to get it running with docker compose, but get errors in minikube. I am missing something in the configuration for kafka, but am unsure what it is. Any help?
Tutorial: https://debezium.io/documentation/reference/stable/tutorial.html
Error:
WARNING: Using default NODE_ID=1, which is valid only for non-clustered installations.
Starting in ZooKeeper mode using NODE_ID=1.
Using ZOOKEEPER_CONNECT=zookeeper:2181
Using configuration config/server.properties.
Using KAFKA_LISTENERS=PLAINTEXT://10.244.0.45:tcp://10.109.182.14:9092 and KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://10.244.0.45:9092
2024-09-18 01:49:59,213 - INFO [main:Log4jControllerRegistration$@31] - Registered kafka:type=kafka.Log4jController MBean
2024-09-18 01:49:59,698 - INFO [main:X509Util@78] - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2024-09-18 01:49:59,704 - ERROR [main:MarkerIgnoringBase@159] - Exiting Kafka due to fatal exception
java.lang.IllegalArgumentException: Error creating broker listeners from 'PLAINTEXT://10.244.0.45:tcp://10.109.182.14:9092': No security protocol defined for listener PLAINTEXT://10.244.0.45:TCP
at kafka.utils.CoreUtils$.listenerListToEndPoints(CoreUtils.scala:243)
at kafka.server.KafkaConfig.listeners(KafkaConfig.scala:2177)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1845)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1639)
at kafka.Kafka$.buildServer(Kafka.scala:71)
at kafka.Kafka$.main(Kafka.scala:90)
at kafka.Kafka.main(Kafka.scala)
Caused by: java.lang.IllegalArgumentException: No security protocol defined for listener PLAINTEXT://10.244.0.45:TCP
at kafka.cluster.EndPoint$.$anonfun$createEndPoint$2(EndPoint.scala:49)
at scala.collection.immutable.Map$Map4.getOrElse(Map.scala:548)
at kafka.cluster.EndPoint$.securityProtocol$1(EndPoint.scala:49)
at kafka.cluster.EndPoint$.createEndPoint(EndPoint.scala:57)
at kafka.utils.CoreUtils$.$anonfun$listenerListToEndPoints$10(CoreUtils.scala:240)
at scala.collection.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:100)
at scala.collection.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
at scala.collection.mutable.ArraySeq.map(ArraySeq.scala:35)
at kafka.utils.CoreUtils$.listenerListToEndPoints(CoreUtils.scala:240)
... 6 more
apiVersion: v1
kind: Service
metadata:
name: zookeeper
spec:
ports:
- name: client
port: 2181
targetPort: 2181
- name: follower
port: 2888
targetPort: 2888
- name: election
port: 3888
targetPort: 3888
selector:
app: zookeeper
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: zookeeper
spec:
replicas: 1
selector:
matchLabels:
app: zookeeper
template:
metadata:
labels:
app: zookeeper
spec:
containers:
- name: zookeeper
image: quay.io/debezium/zookeeper:2.7.2.Final
ports:
- containerPort: 2181
- containerPort: 2888
- containerPort: 3888
---
apiVersion: v1
kind: Service
metadata:
name: kafka
spec:
ports:
- port: 9092
targetPort: 9092
selector:
app: kafka
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: kafka
spec:
replicas: 1
selector:
matchLabels:
app: kafka
template:
metadata:
labels:
app: kafka
spec:
containers:
- name: kafka
image: quay.io/debezium/kafka:2.7.2.Final
ports:
- containerPort: 9092
env:
- name: ZOOKEEPER_CONNECT
value: zookeeper:2181
---
apiVersion: v1
kind: Service
metadata:
name: postgres
spec:
ports:
- port: 5432
targetPort: 5432
selector:
app: postgres
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
spec:
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: quay.io/debezium/example-postgres:2.7.2.Final
ports:
- containerPort: 5432
env:
- name: POSTGRES_USER
value: postgres
- name: POSTGRES_PASSWORD
value: postgres
---
apiVersion: v1
kind: Service
metadata:
name: connect
spec:
ports:
- port: 8083
targetPort: 8083
selector:
app: connect
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: connect
spec:
replicas: 1
selector:
matchLabels:
app: connect
template:
metadata:
labels:
app: connect
spec:
containers:
- name: connect
image: quay.io/debezium/connect:2.7.2.Final
ports:
- containerPort: 8083
env:
- name: BOOTSTRAP_SERVERS
value: kafka:9092
- name: GROUP_ID
value: "1"
- name: CONFIG_STORAGE_TOPIC
value: my_connect_configs
- name: OFFSET_STORAGE_TOPIC
value: my_connect_offsets
- name: STATUS_STORAGE_TOPIC
value: my_connect_statuses
2
Check your listener.security.protocol.map config, for example
listener.security.protocol.map=PLAINTEXT:PLAINTEXT
1