Requirement:

I’m working on the following requirement. I have two devices that act like server and a client communicating over a LAN cable. They exchange some data and I’m trying to secure the data using TLS.

What I’ve done and the Issue I’m facing:

1. Got a CA private key and CA cert from a colleague of mine who worked on a similar requirement few years back

2. Using the CA key and cert, I created certs and keys for my server and client using OpenSSL in the following way

Private Keys:

openssl genrsa -out privateserver.key.pem 2048
openssl genrsa -out privateclient.key.pem 2048

Server CSR and Cert:

openssl req -config openssl.cnf -key privateserver.key.pem -new -sha256 -subj "/CN=Server" -out csrserver.csr.pem
openssl ca -config openssl.cnf -batch -extensions server_cert -days 730 -notext -md sha256 -in csrserver.csr.pem -out certsserver.cert.pem -passin pass:myPassword

Client CSR and Cert:

openssl req -config openssl.cnf -key privateclient.key.pem -new -sha256 -subj "/CN=Client" -out csrclient.csr.pem
openssl ca -config openssl.cnf -batch -extensions iot_cert -days 730 -notext -md sha256 -in csrclient.csr.pem -out certsclient.cert.pem -passin pass:myPassword

3. Once the certs and keys are generated, I append the CA cert to the server and client certs. These will be the certificates of my server and client. I have a Root CA certificate as well. Then I loaded the server cert, server private key, root CA cert on to the Server and Client cert, Client Private key, root CA cert on to the Client

4. I use wolfSSL in my application to do the handshake and connection. I had written code for that already and when I loaded the certs and rebooted my devices, the connection was successful and they exchanged the data

5. I noticed the issue when I tested the following negative scenario: I removed the certs and keys from Client. I created a new batch of them using the following commands,

openssl -x509 -newkey rsa:2048 -days 365 -keyout caKey.pem -out caCert.pem
openssl req -newkey rsa:2048 -nodes -keyout clientKey.pem -out clientCsr.pem
openssl x509 -req -in clientCsr.pem -days 365 -CA caCert.pem -CAkey caKey.pem -CAcreateserial -out clientCert.pem

I loaded these on to the client and tried initiating connection. I was expecting them to not connect since they are signed by different CA. But, they ended up connecting. This is the issue that I’m facing

Debugging:

To test whether the problem is my application or the certificates, I used OpenSSL to run a server and client using the following commands

openssl s_server -cert serverCert.pem -key serverKey.pem -CAfile rootCA.pem -accept 4433
openssl s_client -connect localhost:4433 -cert WrongClientCert.pem -key WrongClientKey.pem -CAfile caCert.pem

It connected here as well. This confirmed to me that the problem might not be with my application and maybe with the certs and/ or the way OpenSSL and WolfSSL is configured. I checked online and it looks like the commands I used to generate the files seem to be correct. I’m stuck here as I’m not very familiar with the configuration settings of OpenSSL/ WolfSSL.

To help anyone answering this further, here are some information:

Extensions definition in my openssl.cnf:

[ iot_cert ]
basicConstraints = CA:FALSE,pathlen:0
nsCertType = client 
nsComment = "OpenSSL Generated Client Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
keyUsage = digitalSignature
extendedKeyUsage = clientAuth, emailProtection
 
[ server_cert ]
basicConstraints = CA:FALSE
nsCertType = server
nsComment = "OpenSSL Generated Server Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer:always
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth

My application code server side:

//I'm doing proper error checking, not shown here in this snippet
wolfSSL_Init();
WOLFSSL_CTX* pTmpsslCtx = wolfSSL_CTX_new(wolfTLS_server_method());
wolfSSL_CTX_set_verify(pTmpWolfsslCtx, WOLFSSL_VERIFY_NONE, NULL);
wolfSSL_CTX_set_verify_depth(pTmpWolfsslCtx, 9);
wolfSSL_CTX_load_verify_locations(pTmpWolfsslCtx, "pathToRootCA", NULL);
wolfSSL_CTX_use_PrivateKey_file(pTmpWolfsslCtx, "pathToPrivateKeyServer", WOLFSSL_FILETYPE_PEM);
wolfSSL_CTX_use_certificate_chain_file(pTmpWolfsslCtx, "pathToServerCert");
WOLFSSL* pTmpWolfssl = wolfSSL_new(pTmpWolfsslCtx);

My application code client side:

//I'm doing proper error checking, not shown here in this snippet
wolfSSL_Init();
WOLFSSL_CTX* pTmpsslCtx = wolfSSL_CTX_new(wolfTLS_client_method());
wolfSSL_CTX_set_verify(pTmpWolfsslCtx, WOLFSSL_VERIFY_NONE, NULL);
wolfSSL_CTX_set_verify_depth(pTmpWolfsslCtx, 9);
wolfSSL_CTX_load_verify_locations(pTmpWolfsslCtx, "pathToRootCA", NULL);
wolfSSL_CTX_use_PrivateKey_file(pTmpWolfsslCtx, "pathToPrivateKeyClient", WOLFSSL_FILETYPE_PEM);
wolfSSL_CTX_use_certificate_chain_file(pTmpWolfsslCtx, "pathToClientCert");
WOLFSSL* pTmpWolfssl = wolfSSL_new(pTmpWolfsslCtx);
wolfSSL_connect(pTmpWolfssl);

I’m stuck here and any inputs on why this connection is successful despite CA not matching would be helpful. Let me know if any of what I’ve done is incorrect. Thank You for your time!