Skip to content

Fix surrogate decoding and invalid hex handling#466

Open
MichaelChirico wants to merge 1 commit into
jeroen:masterfrom
MichaelChirico:fix-surrogate-decoding-assert
Open

Fix surrogate decoding and invalid hex handling#466
MichaelChirico wants to merge 1 commit into
jeroen:masterfrom
MichaelChirico:fix-surrogate-decoding-assert

Conversation

@MichaelChirico

Copy link
Copy Markdown
Contributor

This was an error produced by a fuzzer we set up for yajl, fixed by Gemini, and ported to this bundled version by Gemini.


YAJL's string decoder could crash (SIGABRT) or read out-of-bounds when encountering invalid unicode escape sequences or surrogate pairs:

  1. hexToDigit used assert to validate hex characters, causing crashes on invalid input (e.g., \u005g).
  2. Surrogate decoding lacked bounds checks, potentially reading past the end of the input buffer if a high surrogate was followed by a truncated \u escape.
  3. The parser did not validate that the second surrogate in a pair was actually a valid low surrogate before combining them.

Solution:
Improve robustness of unicode decoding in yajl_encode.c:

  • Modify hexToDigit to return a success status (int) instead of asserting, allowing callers to handle parsing errors gracefully.
  • Add bounds checks (end + 6 < len) before attempting to parse a low surrogate.
  • Verify that the parsed surrogate is in the valid low surrogate range (0xDC00 to 0xDFFF) before combining.
  • Fall back to the replacement character ? and break parsing of the escape sequence on any invalid input or bounds violation.

Avoid crashes/asserts when decoding invalid surrogate pairs or truncated unicode escapes in yajl_string_decode.
@MichaelChirico MichaelChirico force-pushed the fix-surrogate-decoding-assert branch from 35285b5 to 6c2ca00 Compare July 1, 2026 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant