I’m encountering an issue while attempting to produce messages to a Kafka topic using the kafka-console-producer command. Despite configuring the connection details correctly, I face the following error:

[2024-05-13 13:55:31,148] WARN [Producer clientId=console-producer] Bootstrap broker kafka.confluent.svc.cluster.local:443 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)

I followed the steps mentioned in this readme file which can be found here.

To my knowledge I followed the steps mentioned in it and created the certificates and the secrets that are necessary as mentioned in here but the I’m not being able to get the same end output as the guide.

It is said to use the kubernetes cluster domain as the domain in the kafka configuration, I’m running the kubernetes cluster on the docker desktop for mac and I used this as the domain for the kafka configuration and as well in the config of the ingress resource

confluent.svc.cluster.local

Is this the correct domain or have I used an incorrect one?

From the attempts I have done to resolve I think the issue is due to this domain configuration.
It would be a great help if somebody can shed some light on the issue I’m facing now and I’m very new kubernetes and exposing services using ingress controllers.

This is the kafka config for the host based routing

externalAccess:
        type: staticForHostBasedRouting
        staticForHostBasedRouting:
          domain: confluent.svc.cluster.local
          brokerPrefix: 
          port: 443
      tls:
        enabled: true

and the configuration of the ingress resource

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-with-sni
  annotations:
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - kafka.confluent.svc.cluster.local
        - b0.confluent.svc.cluster.local
        - b1.confluent.svc.cluster.local
        - b2.confluent.svc.cluster.local
        - controlcenter.confluent.svc.cluster.local
        - ksqldb.confluent.svc.cluster.local
        - connect.confluent.svc.cluster.local
  rules:
    - host: kafka.confluent.svc.cluster.local
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kafka-bootstrap
                port: 
                  number: 9092

These are the attempts I took to resolve the issue:

• Confirmed the availability of the kafka cluster.
• Ensured the correctness of the bootstrap-server and the port configuration
  (not sure whether the kubernetes cluster domain I used is correct).
• Investigated potential networking and firewall restrictions but found none.

Thank you for your help in advance