Fix RFC 2231 parameter value decoding#139
Open
gaoflow wants to merge 1 commit into
Open
Conversation
Two spec-legal cases were mis-decoded in parse_param_content: - A multibyte character split across continuation segments (name*0*=...; name*1*=...) was charset-decoded per segment before the resulting strings were concatenated, so a character split at a segment boundary decoded as two incomplete byte fragments (U+FFFD). Percent-decode each encoded segment to octets, concatenate the octets of consecutive encoded segments, and charset-decode once (RFC 2231 Section 3). - A quoted parameter value containing ';' was truncated because the value was split on every ';'. Split only on semicolons outside a quoted-string (RFC 2045).
gaoflow
force-pushed
the
fix-rfc2231-param-decoding
branch
from
July 20, 2026 12:46
32f08bf to
32c3d26
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
parse_param_contentmis-decodes two spec-legal parameter forms. Both are reproduced against Python'semail(get_param/collapse_rfc2231_value) and Go'smime.ParseMediaType, which agree on the correct output.Multibyte character split across continuation segments
RFC 2231 §3 lets a long parameter value be split into numbered continuation sections (
name*0*=charset''octets; name*1*=octets; …), and a multi-octet character may legally be split across two sections. The correct algorithm percent-decodes each encoded section to octets, concatenates the octets, then charset-decodes once. mailparse instead charset-decodes each section independently and concatenates the resulting strings, so a UTF-8 character split at a section boundary becomes two incomplete byte fragments, each decoding to U+FFFD:decodes
nameto"\u{FFFD}\u{FFFD}.txt"instead of"☃.txt"(snowman U+2603 = E2 98 83, split E2 98 | 83). Single-byte charsets are unaffected — per-segment and concatenated decoding coincide there — which is why this went unnoticed.The fix defers charset-decoding: encoded segments are percent-decoded to octets, the octets of consecutive encoded segments are concatenated, and each run is charset-decoded once with the section-
*0*charset. An unencoded (literal) continuation segment ends the octet run, preserving the existing mixed encoded/unencoded behaviour.Quoted value containing
;A parameter value may be a
quoted-string(RFC 2045), which can legally contain;. The value was split on every;, sotruncated
nameto"\"foo"instead of"foo;bar.txt". Splitting now ignores;inside a quoted-string (respecting\escapes).All existing parameter tests pass unchanged; new regression tests cover a 2- and 3-segment split, a single-byte-charset control, and the quoted-
;cases.cargo test,cargo fmt --check, andcargo clippyare green.