DTLS 1.3 RSA-PSS Signature Scheme Support

[adrianosela]

DTLS 1.3 RSA-PSS Signature Scheme Support

Address part of #776

Adds support for the RSA_PSS_RSAE_SHA256 digital signature algorithm, which is mandatory-to-implement per RFC 8446 section 9.1.

Also adds support for:

• RSA_PSS_RSAE_SHA384
• RSA_PSS_RSAE_SHA512
• RSA_PSS_PSS_SHA256
• RSA_PSS_PSS_SHA384
• RSA_PSS_PSS_SHA512

This is all dead code, but I opted for a small, self-contained, PR. Next up is a bigger change: implementing the signature_algorithms_cert extension, but perhaps this PR is enough to start the conversation for #776 (comment). cc:// @theodorsm

Basically, breaks SelectSignatureScheme:

• from func([]Algorithm, crypto.PrivateKey) (Algorithm, error)
• __to func([]Algorithm, crypto.PrivateKey, protocol.Version) (Algorithm, error)

But if we don't want to break the API we could do something like this:

type config struct {
    protocolVersion protocol.Version
}

type Option func(*config)

func WithProtocolVersion(protocolVersion protocol.Version) Option {
    return func(c *config) { c.protocolVersion = protocolVersion }
}

and

func SelectSignatureScheme(algs []Algorithm, priv crypto.PrivateKey, opts ...Option)  (Algorithm, error) {
    cfg := &config{
        protocolVersion: protocol.Version1_2, // default to 1.2
    }
    for _, opt := range opts {
        opt(cfg)
    }
    ....
}

and can still be called the way it is called today:

SelectSignatureScheme(algs, priv)

but also

SelectSignatureScheme(algs, priv, WithProtocolVersion(protocol.Version1_3))

I'm new here so I don't know how much we care or don't care about breaking this contract - but either way, there is a way forward (and I really like the functional optional config options).

[adrianosela]

Also, AFAICT this is the last of the mandatory-to-Implement signatures (RSA_PKCS1_SHA256, RSA_PSS_RSAE_SHA256, and ECDSA_SECP256R1_SHA256)? cc:// @theodorsm

[codecov]

Codecov Report

❌ Patch coverage is 86.53846% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.59%. Comparing base (9121462) to head (3974186).

Files with missing lines	Patch %	Lines
crypto.go	50.00%	10 Missing and 2 partials ⚠️
pkg/crypto/signaturehash/signaturehash.go	90.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #778      +/-   ##
==========================================
+ Coverage   81.40%   81.59%   +0.18%     
==========================================
  Files         103      103              
  Lines        5717     5803      +86     
==========================================
+ Hits         4654     4735      +81     
- Misses        684      689       +5     
  Partials      379      379              

Flag	Coverage Δ
go	81.59% <86.53%> (+0.18%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[adrianosela]

@JoTurk

Some early thoughts on this from you would be really awesome :D - if you have time, of course.

This PR implements what's noted above, but wanted to also use it as a discussion point for this comment from @theodorsm :

I am currently unsure how we should split this implementation between DTLS 1.2 and DTLS 1.3.

[theodorsm]

We should be very strict on breaking current public APIs. We should rather offer a separate API like SelectSignatureScheme13 or as a functional optional config .

[adrianosela]

My preference is the functional optional config, but we don't use that pattern anywhere else in pion/dtls or pion/* afaict. What is your preference?

[theodorsm]

We are moving towards the functional optional config: #765, #731 (comment)

[theodorsm]

Some quick comments to add to the discussion:

We are having a disucssion about DTLS 1.2 and DTLS 1.3 architectural separation in PR #738. So far I am thinking to split off the code in conn.go where DTLS 1.3 will have it's own handshaker and flights. Most other things related to DTLS 1.3 should be suffixed by 13.

I support the functional optional config as this is the way forward for Pion (#765).

I will take a closer look at this draft in the coming week.

[JoTurk]

for #765 I'll open a PR soon for the config pattern, I did work on it but I forgot to push it.

[adrianosela]

for #765 I'll open a PR soon for the config pattern, I did work on it but I forgot to push it.

Will hold off working on this till then :)