I am managing a Kubernetes cluster with ArgoCD, which includes an ingress-nginx. We are trying to obtain a certificate for HTTPS on a domain using cert-manager, but we are encountering a series of errors during the process.

The challenge fails with the following error:

Failed 10s cert-manager-challenges Accepting challenge authorization failed: acme: authorization error for example.org: 403 urn:ietf:params:acme:error:unauthorized: 2001:8d8:100f:f000::200: Invalid response from http://example.org/.well-known/acme-challenge/ZzU4jDSzvVCHwPHwPMsUleJDwf-K3URomZwuhQgNZOo: 204. 

The challenge generates a cm-acme-http-solver with the following logs:

I0712 10:37:33.303185       1 solver.go:39] "cert-manager/acmesolver: starting listener" expected_domain="example.org" expected_token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" expected_key="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY.2DhKIbjC5D1b_jnc9Katl9vWzWPu9HPi-bGtAm8wLnw" listen_port=8089 

I0712 10:37:42.095947       1 solver.go:64] "cert-manager/acmesolver: validating request" host="example.org" path="/.well-known/acme-challenge/NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" base_path="/.well-known/acme-challenge" token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" 

I0712 10:37:42.095987       1 solver.go:72] "cert-manager/acmesolver: comparing host" host="example.org" path="/.well-known/acme-challenge/NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" base_path="/.well-known/acme-challenge" token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" expected_host="example.org" 

I0712 10:37:42.096005       1 solver.go:79] "cert-manager/acmesolver: comparing token" host="example.org" path="/.well-known/acme-challenge/NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" base_path="/.well-known/acme-challenge" token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" expected_token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" 

I0712 10:37:42.096030       1 solver.go:87] "cert-manager/acmesolver: got successful challenge request, writing key" host="example.org" path="/.well-known/acme-challenge/NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" base_path="/.well-known/acme-challenge" token="NF6dyNYknCNUVSRzun5JuUzN8cNF86aflve-WInfvKY" 

... 

Error: http: Server closed 

Usage: 

  acmesolver [flags] 

  

Flags: 

      --domain string     the domain name to verify 

  -h, --help              help for acmesolver 

      --key string        the challenge key to respond with 

      --listen-port int   the port number to listen on for connections (default 8089) 

      --token string      the challenge token to verify against 

  

E0712 10:37:53.692260       1 main.go:39] "cert-manager: error executing command" err="http: Server closed" 

Additionally, the cert-manager-webhook logs show these errors:

W0712 11:33:35.571985       1 client_config.go:618] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work. 

I0712 11:33:35.657545       1 webhook.go:128] "cert-manager: using dynamic certificate generating using CA stored in Secret resource" secret_namespace="cert-manager" secret_name="cert-manager-webhook-ca" 

I0712 11:33:35.657833       1 server.go:133] "cert-manager/webhook: listening for insecure healthz connections" address=":6080" 

I0712 11:33:35.657899       1 server.go:197] "cert-manager/webhook: listening for secure connections" address=":10250" 

I0712 11:33:36.662043       1 dynamic_source.go:255] "cert-manager/webhook: Updated cert-manager webhook TLS certificate" DNSNames=["cert-manager-webhook","cert-manager-webhook.cert-manager","cert-manager-webhook.cert-manager.svc"] 

I0712 11:33:54.925304       1 logs.go:59] http: TLS handshake error from 10.221.113.195:52358: remote error: tls: bad certificate 

I0712 11:33:59.817534       1 logs.go:59] http: TLS handshake error from 10.216.164.3:46654: EOF 

... 

Our ClusterIssuer configuration is as follows:

apiVersion: cert-manager.io/v1 

kind: ClusterIssuer 

metadata: 

  name: letsencrypt-example 

spec: 

  acme: 

    email: [email protected] 

    preferredChain: "" 

    privateKeySecretRef: 

      name: lets-encrypt-ionos-issuer-account-key 

    server: https://acme-v02.api.letsencrypt.org/directory 

    solvers: 

    - selector: 

        dnsZones: 

          - example.org 

      http01: 

        ingress: 

          ingressClassName: nginx 

And the Ingress configuration:

apiVersion: networking.k8s.io/v1 

kind: Ingress 

metadata: 

  annotations: 

    cert-manager.io/cluster-issuer: letsencrypt-example 

  name: example 

  namespace: example 

spec: 

  ingressClassName: nginx 

  rules: 

    - host: example.org 

      http: 

        paths: 

          - backend: 

              service: 

                name: example 

                port: 

                  number: 8080 

            path: / 

            pathType: Prefix 

  tls: 

    - hosts: 

        - example.org 

      secretName: example-dev-tls 

Has anyone experienced similar issues or have any insights on how to resolve these errors? Any help would be greatly appreciated.

I tried to add some ssl-redirect=false annotations and edit the dnskube config on kube-system.