I want to use debezium to CDC, my source is SQL Server. I got problem my connect is work and i have topic but it is empty. This is my docker-compose.yml:
version: '3.8'
services:
zookeeper:
image: confluentinc/cp-zookeeper:7.7.0
hostname: zookeeper
container_name: zookeeper
ports:
- '2181:2181'
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
healthcheck:
test: echo srvr | nc zookeeper 2181 || exit 1
start_period: 10s
retries: 20
interval: 10s
broker:
image: confluentinc/cp-kafka:7.7.0
hostname: broker
container_name: broker
ports:
- '29092:29092'
- '9092:9092'
- '9101:9101'
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
KAFKA_JMX_PORT: 9101
KAFKA_JMX_HOSTNAME: localhost
healthcheck:
test: nc -z localhost 9092 || exit -1
start_period: 15s
interval: 5s
timeout: 10s
retries: 10
control-center:
image: confluentinc/cp-enterprise-control-center:7.7.0
hostname: control-center
container_name: control-center
depends_on:
broker:
condition: service_healthy
ports:
- "9021:9021"
environment:
CONTROL_CENTER_BOOTSTRAP_SERVERS: 'broker:29092'
CONTROL_CENTER_REPLICATION_FACTOR: 1
CONTROL_CENTER_INTERNAL_TOPICS_PARTITIONS: 1
CONTROL_CENTER_MONITORING_INTERCEPTOR_TOPIC_PARTITIONS: 1
CONFLUENT_METRICS_TOPIC_REPLICATION: 1
CONFLIENT_METRICS_ENABLE: 'false'
PORT: 9021
healthcheck:
test: [ "CMD", "curl", "-f", "http://localhost:9021/health" ]
interval: 30s
timeout: 10s
retries: 5
debezium:
image: quay.io/debezium/connect:2.7
container_name: debezium
hostname: debezium
depends_on:
- sql_server
- broker
ports:
- '8093:8083'
environment:
BOOTSTRAP_SERVERS: broker:29092
CONNECT_REST_ADVERTISED_HOST_NAME: debezium
GROUP_ID: 1
CONFIG_STORAGE_TOPIC: connect_configs
STATUS_STORAGE_TOPIC: connect_statuses
OFFSET_STORAGE_TOPIC: connect_offsets
KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter
VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter
ENABLE_DEBEZIUM_SCRIPTING: 'true'
healthcheck:
test:
[ 'CMD', 'curl', '--silent', '--fail', '-X', 'GET', 'http://localhost:8083/connectors', ]
start_period: 10s
interval: 10s
timeout: 5s
retries: 5
debezium-ui:
image: debezium/debezium-ui:latest
container_name: debezium-ui
hostname: debezium-ui
depends_on:
debezium:
condition: service_healthy
ports:
- '8080:8080'
environment:
KAFKA_CONNECT_URIS: http://debezium:8083
sql_server:
build:
context: .
dockerfile: Dockerfile
hostname: sql_server
healthcheck:
test: ["CMD", "/opt/mssql-tools/bin/sqlcmd", "-Usa", "-PStrongPassw0rd", "-Q", "select 1"]
start_period: 10s
interval: 10s
timeout: 5s
retries: 3
ports:
- 1433:1433
```. And this my connector config:
curl -i -X POST -H “Accept:application/json” -H “Content-Type:application/json” localhost:8083/connectors/ -d ‘{
“name”: “sale-connector”,
“config”: {
“connector.class”: “io.debezium.connector.sqlserver.SqlServerConnector”,
“tasks.max” : “1”,
“topic.prefix”: “order”,
“database.hostname”: “sql_server”,
“database.port”: “1433”,
“database.user”: “sa”,
“database.password”: “StrongPassw0rd”,
“database.names”: “AdventureWorks2022”,
“database.applicationIntent”: “ReadOnly”,
“snapshot.isolation.mode”: “snapshot”,
“schema.history.internal.kafka.bootstrap.servers”: “broker:29092”,
“schema.history.internal.kafka.topic”: “schema-changes.order”,
“database.encrypt”: “false”
}
}’
I tried using only one port in kafka but it is not work
Vu Ngoc Tran is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.