I’m developing a TLS 1.3 client in C#, targeting .NET 8, and I’m stuck in the server certificate verification step. The problem is that all other things being equal (as far as I can tell), verification succeeds when using RSA signature algorithms, but fails when using ECDSA signature algorithms.

Here’s the relevant snippet from my code, it runs when the “Certificate Verify” message is received from the server:

//receivedRecord is the CertificateVerify message,
//without the record header and without the handshake type and length header.
var algorithm = (SignatureAlgorithm)receivedRecord.ExtractBigEndianUint16(0);
var signatureLength = receivedRecord.ExtractBigEndianUint16(2);
var signature = receivedRecord.Skip(4).Take(signatureLength).ToArray();
                    
//hashAlgorithm is SHA256 or SHA384, depending on the cipher suite negotiated.
//transmittedHandshakeBytes contains up to the server's Certificate message.
var handshakeHash = hashAlgorithm.ComputeHash(transmittedHandshakeBytes.ToArray());
byte[] contentToSign = [
    .. Enumerable.Repeat<byte>(0x20, 64),
    .. Encoding.ASCII.GetBytes("TLS 1.3, server CertificateVerify"),
    0,
    .. handshakeHash
];

var isEcdsa = algorithm is SignatureAlgorithm.ecdsa_secp256r1_sha256 or SignatureAlgorithm.ecdsa_secp384r1_sha384;
var is256 = algorithm is SignatureAlgorithm.ecdsa_secp256r1_sha256 or SignatureAlgorithm.rsa_pss_rsae_sha256;

var hashAlgName = is256 ? HashAlgorithmName.SHA256 : HashAlgorithmName.SHA384;

//ServerCertificates is an array of X509Certificate2 instances,
//generated from the server's Certificate message.
var certificateOk = isEcdsa ?
    ServerCertificates[0].GetECDsaPublicKey().VerifyData(contentToSign, signature, hashAlgName) :
    ServerCertificates[0].GetRSAPublicKey().VerifyData(contentToSign, signature, hashAlgName, RSASignaturePadding.Pss);

//Result: certificateOk = true only when using RSA

Worth mentioning:

• I’m testing with a few servers, for each one I’m doing two tests: one with ecdsa_secp256r1_sha256 and ecdsa_secp384r1_sha384 as the signature algorithms offered in the ClientHello message (to force the usage of ECDSA) and another one with rsa_pss_rsae_sha256 and rsa_pss_rsae_sha384 (to force RSA).
• I’m hitting properly working public servers, so no self-signed or expired certificates are involved.
• TlsHandler is a similar project that implements a TLS 1.3 server, and apparently i’s doing the ECDSA verification in the same way as I do (although I haven’t tested this code).

And that’s the thing. What am I missing here?