I deployed ingress into k8s using helm:
helm install nginx-ingress-mtls ingress-nginx/ingress-nginx
--namespace nginx-ns
--create-namespace
--set controller.ingressClass="nginx-mtls"
--set controller.ingressClassResource.name="nginx-mtls"
--set controller.ingressClassResource.controllerValue="k8s.io/ingress-nginx"
--set controller.ingressClassResource.enabled=true
--set controller.extraArgs.default-ssl-certificate="server-tls"
--set controller.extraArgs.enable-ssl-passthrough=true
--set "controller.extraVolumes[0].name=ca-cert"
--set "controller.extraVolumes[0].secret.secretName=root-ca"
--set "controller.extraVolumes[0].secret.defaultMode=400"
--set "controller.extraVolumeMounts[0].name=ca-cert"
--set "controller.extraVolumeMounts[0].mountPath=/etc/nginx/ssl/ca"
--set "controller.extraVolumeMounts[0].readOnly=true"
--set controller.admissionWebhooks.enabled=true
server-tls secret in the same namespace contains .crt for api server
root-ca secret contains root CA certificate with interim cert (cert chain)
This is my deployment:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: mtls-ingress
namespace: nginx-ns
annotations:
nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
nginx.ingress.kubernetes.io/auth-tls-secret: "root-ca"
nginx.ingress.kubernetes.io/auth-tls-verify-depth: "2"
nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
spec:
ingressClassName: nginx-mtls
tls:
- hosts:
- "admin.myapi.com"
secretName: "server-tls"
rules:
- host: "admin.myapi.com"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-api-proxy
port:
number: 443
my-api-proxy is external service, because I have api in different namespace
I installed client certificate to browser and when I hit the pod via ip directly, I was requested for client cert, however when I hit it via ingress I receive 403 immediately.
curl shows only tls handshake
curl -v -k https://admin.myapi.com --key client.key --cert client.crt
* Trying 34.128.122.16:443...
* Connected to admin.myapi.com (34.128.122.16) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384
* ALPN: server accepted h2
* Server certificate:
* subject: O=myCorp; OU=team; CN=admin.myapi.com
* start date: Feb 1 14:07:50 2024 GMT
* expire date: Feb 1 14:07:50 2026 GMT
* issuer: CN=myIssuer
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multiplexing
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* h2h3 [:method: GET]
* h2h3 [:path: /]
* h2h3 [:scheme: https]
* h2h3 [:authority: admin.myapi.com]
* h2h3 [user-agent: curl/7.84.0]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x140017400)
> GET / HTTP/2
> Host: admin.myapi.com
> user-agent: curl/7.84.0
> accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 403
< date: Fri, 03 May 2024 04:07:44 GMT
< content-type: text/html
< content-length: 146
<
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host admin.myapi.com left intact