Skip to content

Stop advertising unsupported RSA signatures in DTLS 1.2#157

Merged
xnorpx merged 3 commits into
mainfrom
fix/rsa-signature-advertisement
Jul 11, 2026
Merged

Stop advertising unsupported RSA signatures in DTLS 1.2#157
xnorpx merged 3 commits into
mainfrom
fix/rsa-signature-advertisement

Conversation

@xnorpx

@xnorpx xnorpx commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Stop advertising RSA/SHA-256 and RSA/SHA-384 as supported DTLS 1.2 signature algorithms. Dimpl recognizes these wire values for parsing, but its signature implementations support only ECDSA.

Why

RSA support was removed from dimpl, but the DTLS 1.2 SignatureAndHashAlgorithm::supported() list still included RSA. That list feeds both the default signature_algorithms extension and server-side CertificateRequest algorithm selection.

As a result, dimpl could advertise RSA to a peer even though it could not complete the corresponding signature verification. A peer selecting an advertised RSA algorithm could therefore cause the handshake to fail. The advertisement was also inconsistent with the ECDSA client certificate type requested by the server.

Changes

  • Separate recognized wire algorithms from algorithms dimpl can actually use:
    • all() retains ECDSA and RSA values so incoming messages remain parseable.
    • supported() now contains only ECDSA/SHA-256 and ECDSA/SHA-384.
  • Size SignatureAndHashAlgorithmVec from the recognized set so parsing mixed ECDSA/RSA input remains bounded without losing recognized values.
  • Build the default DTLS 1.2 signature_algorithms extension from the supported set.
  • Restrict CertificateRequest signature selection to algorithms supported by dimpl's verifiers.
  • Add regression coverage for parsing/filtering, serialization, algorithm classification, and server-side selection.

Testing

  • cargo fmt --all -- --check
  • cargo test --all-features
    • 565 passed
    • 2 ignored
    • 0 failed

@algesten

Copy link
Copy Markdown
Owner

@xnorpx

The parser already consumes and discards signature algorithms we don't support. The ArrayVec therefore only ever contains algorithms from supported(), which is the same pattern we use for named groups and DTLS 1.3 signature schemes.

Sizing it from all() reserves space for values that can never be stored and breaks that invariant. Keep all() as the set of recognised wire values, but size the parsed collection from supported().

And I added a test.

@xnorpx xnorpx merged commit 44989ab into main Jul 11, 2026
46 checks passed
@xnorpx xnorpx deleted the fix/rsa-signature-advertisement branch July 11, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants