I’ve been struggling with exposing a Kubernetes service and running the confluent login
command for the MDS (Metadata Service) in a local Kubernetes cluster on Docker Desktop. Despite multiple attempts and following various troubleshooting steps, I haven’t been able to resolve the issue. Here’s a detailed breakdown of the problem and what I’ve tried so far:
Context
I’m running a local Kubernetes cluster using Docker Desktop. I’m trying to expose a Confluent Kafka Metadata Service (MDS) and access it using the confluent login
command. I’m also trying to expose and access other related services like ksqldb
.
Steps Followed
-
Hosts Configuration:
-
Added entries to the
/etc/hosts
file for various services, includingkafka.kubernetes.docker.internal
,ksqldb.kubernetes.docker.internal
, andmds.kubernetes.docker.internal
.<code>127.0.0.1 kafka.kubernetes.docker.internal127.0.0.1 b0.kubernetes.docker.internal127.0.0.1 controlcenter.kubernetes.docker.internal127.0.0.1 connect.kubernetes.docker.internal127.0.0.1 ksqldb.kubernetes.docker.internal127.0.0.1 mds.kubernetes.docker.internal</code><code>127.0.0.1 kafka.kubernetes.docker.internal 127.0.0.1 b0.kubernetes.docker.internal 127.0.0.1 controlcenter.kubernetes.docker.internal 127.0.0.1 connect.kubernetes.docker.internal 127.0.0.1 ksqldb.kubernetes.docker.internal 127.0.0.1 mds.kubernetes.docker.internal </code>127.0.0.1 kafka.kubernetes.docker.internal 127.0.0.1 b0.kubernetes.docker.internal 127.0.0.1 controlcenter.kubernetes.docker.internal 127.0.0.1 connect.kubernetes.docker.internal 127.0.0.1 ksqldb.kubernetes.docker.internal 127.0.0.1 mds.kubernetes.docker.internal
-
-
Port Forwarding:
- For the Ingress controller:
<code>kubectl port-forward svc/ingress-nginx-controller 8443:443 -n ingress-nginx</code><code>kubectl port-forward svc/ingress-nginx-controller 8443:443 -n ingress-nginx </code>kubectl port-forward svc/ingress-nginx-controller 8443:443 -n ingress-nginx
- For MDS service:
<code>kubectl port-forward svc/mds-bootstrap 8090:8090 -n confluent</code><code>kubectl port-forward svc/mds-bootstrap 8090:8090 -n confluent </code>kubectl port-forward svc/mds-bootstrap 8090:8090 -n confluent
-
Testing Connectivity:
- Running the
curl
command to test connectivity toksqldb
:
<code>curl -k <https://ksqldb.kubernetes.docker.internal:8443> --cacert $TUTORIAL_HOME/externalCacerts.pem --key $TUTORIAL_HOME/kafka-server-key.pem --cert $TUTORIAL_HOME/kafka-server.pem</code><code>curl -k <https://ksqldb.kubernetes.docker.internal:8443> --cacert $TUTORIAL_HOME/externalCacerts.pem --key $TUTORIAL_HOME/kafka-server-key.pem --cert $TUTORIAL_HOME/kafka-server.pem </code>curl -k <https://ksqldb.kubernetes.docker.internal:8443> --cacert $TUTORIAL_HOME/externalCacerts.pem --key $TUTORIAL_HOME/kafka-server-key.pem --cert $TUTORIAL_HOME/kafka-server.pem
- Running the
confluent login
command to authenticate with MDS:
<code>confluent login --url <https://mds.kubernetes.docker.internal> --ca-cert-path $TUTORIAL_HOME/externalCacerts.pem</code><code>confluent login --url <https://mds.kubernetes.docker.internal> --ca-cert-path $TUTORIAL_HOME/externalCacerts.pem </code>confluent login --url <https://mds.kubernetes.docker.internal> --ca-cert-path $TUTORIAL_HOME/externalCacerts.pem
- Running the
Issues Encountered
-
Curl Command:
- The
curl
command toksqldb
returns no output, indicating no successful connection or response from the service.
- The
-
Confluent Login:
- The
confluent login
command returns an error related to certificate verification:
<code>Error: Get "<https://mds.kubernetes.docker.internal:8090/security/1.0/authenticate>": tls: failed to verify certificate: x509: certificate is valid for kafka, kafka.confluent, kafka.confluent.svc, kafka.confluent.svc.cluster.local, *.kafka.confluent, *.kafka.confluent.svc.cluster.local, *.confluent.svc.cluster.local, not mds.kubernetes.docker.internal</code><code>Error: Get "<https://mds.kubernetes.docker.internal:8090/security/1.0/authenticate>": tls: failed to verify certificate: x509: certificate is valid for kafka, kafka.confluent, kafka.confluent.svc, kafka.confluent.svc.cluster.local, *.kafka.confluent, *.kafka.confluent.svc.cluster.local, *.confluent.svc.cluster.local, not mds.kubernetes.docker.internal </code>Error: Get "<https://mds.kubernetes.docker.internal:8090/security/1.0/authenticate>": tls: failed to verify certificate: x509: certificate is valid for kafka, kafka.confluent, kafka.confluent.svc, kafka.confluent.svc.cluster.local, *.kafka.confluent, *.kafka.confluent.svc.cluster.local, *.confluent.svc.cluster.local, not mds.kubernetes.docker.internal
- The
Debugging Steps Taken
-
Certificate Verification:
- I verified the certificate details using
openssl x509 -in $TUTORIAL_HOME/kafka-server.pem -text -noout
. The output shows the certificate SANs (Subject Alternative Names) include only.kubernetes.docker.internal
.
<code>X509v3 Subject Alternative Name:DNS:*.kubernetes.docker.internal</code><code>X509v3 Subject Alternative Name: DNS:*.kubernetes.docker.internal </code>X509v3 Subject Alternative Name: DNS:*.kubernetes.docker.internal
- I verified the certificate details using
Conclusion
Despite following the Confluent documentation guide and ensuring the necessary setup steps, I am unable to get the services exposed and accessible as intended. The primary issue seems to be related to certificate SANs not matching the required domain names.
Request for Help
Could you provide guidance on how to properly configure the certificates or any other troubleshooting steps that might help in resolving the connectivity and authentication issues?