Ideas to implement x509 PEM client certificate validation in real-time.

[elfarolab]

Is your feature request related to a problem? Please describe.
Real-time x509 PEM client certificate validation and authorization using an integrated key-value store of certificate IDs with an in-memory cache DB like Redis.

Problem Description
We're designing an infrastructure with thousands of IoT sensors connected via NB-IoT and CAT-M1 to a server pushing telemetry updates with CoAP. Each device has a unique certificate issued by our dedicated CA and intermediates. After reviewing existing PKI systems, OCSP responders, and reverse proxies, we've found that traditional certificate validation methods (CRL and OCSP) aren't so so fancy anymore.
To address this, we're considering implementing custom client certificate validation logic within the server using libcoap and Redis to check for revoked or disabled IoT sensor devices in real-time.

Proposed Solution
We're looking for a way to check the client certificate ID during the DTLS 1.2 handshake. In the current libcoap version, we've identified the "coap_dtls_cn_callback_t" callback function, which could be used to extract the certificate ID and check it against our Redis database.
However, the API documentation indicates that this callback is invoked after libcoap has performed TLS-level validation checks.

Questions

• Is there another query method or callback available to get the client certificate ID during the early phase of the DTLS v1.2 handshake?
• Can the coap_dtls_cn_callback_t callback be used for this purpose, or are there other alternatives?

Additional
If this feature is of interest to others, we're willing to develop and test a solution and submit a pull request for an example or necessary modifications to libcoap.

Potential Approach
One possible approach is to use the coap_dtls_cn_callback_t callback to extract the certificate ID and check it against the Redis database. If the certificate is revoked or disabled, the callback could return an error, terminating the connection.
To improve performance, we could implement caching mechanisms, such as storing recently invalidated certificates in memory or using Redis' built-in caching features.

Open Questions

• Are there any existing libcoap features or APIs that can be leveraged for this purpose?
• Are there any security considerations or best practices we should be aware of when implementing custom certificate validation logic?

[mrdeep1]

Once the peer's certificate has been received in the TLS handshake, assuming verify_peer_cert is set in the coap_dtls_pki_t PKI definition, then the Certificate is checked based on the other checking options set/unset in coap_dtls_pki_t. Then the coap_dtls_cn_callback_t callback is invoked from the same checking function. Then the TLS handshake continues.

The different TLS libraries vary slightly, but the main principle is the same.

coap_dtls_cn_callback_t callback is usually only called for the first certificate in the certificate chain, but may get called by a problematic CA or an other interim CA issue where depth is not 0.

Any analysis of the data in the coap_dtls_cn_callback_t callback needs to properly bounds checked as 'bad' certificates may be presented.