Skip to content

fuzzing: fix serialization of strings containing null bytes#769

Open
dguido wants to merge 1 commit into
masterfrom
dev/fix-string-serialization
Open

fuzzing: fix serialization of strings containing null bytes#769
dguido wants to merge 1 commit into
masterfrom
dev/fix-string-serialization

Conversation

@dguido

@dguido dguido commented Jan 22, 2026

Copy link
Copy Markdown
Member

Summary

  • Fix serialization of strings containing null bytes (\x00) and other non-printable control characters
  • Strings that cannot be reliably represented in JSON are now hex-encoded with a hex: prefix
  • Add comprehensive unit tests for round-trip serialization of strings with null bytes

Problem

When serializing ABI values to JSON for corpus storage, strings containing null bytes were being incorrectly saved as empty strings. This is because JSON cannot properly represent arbitrary binary data in strings - null bytes and other control characters can be lost during the []byte -> string conversion or JSON marshaling.

Solution

The fix:

  1. Add stringNeedsHexEncoding() helper to detect strings that contain null bytes, invalid UTF-8 sequences, or control characters (except common whitespace like \t, \n, \r)
  2. In encodeJSONArgument(): Encode such strings as hex:<hex-encoded-bytes>
  3. In decodeJSONArgument(): Decode strings with the hex: prefix back to their original bytes

This approach:

  • Maintains backward compatibility with existing corpus files containing normal strings
  • Only applies hex encoding when necessary
  • Follows the same pattern used for BytesTy serialization

Test plan

  • Add TestStringWithNullBytes - verifies round-trip serialization for various null byte scenarios
  • Add TestStringNeedsHexEncoding - verifies the helper function correctly identifies strings needing encoding
  • Run existing tests to ensure no regressions: go test -v ./fuzzing/valuegeneration/...
  • Run linter: golangci-lint run --timeout 5m

Fixes #279


Generated with Claude Code

Strings containing null bytes (\x00) or other non-printable control
characters were being incorrectly serialized to JSON, causing them to
be lost when saved to the corpus. This fix uses hex encoding with a
"hex:" prefix for strings that contain characters that cannot be
reliably represented in JSON.

The fix:
- Add stringNeedsHexEncoding() to detect strings requiring encoding
- Encode such strings as "hex:<hex-encoded-bytes>" in encodeJSONArgument
- Decode hex-prefixed strings back to original bytes in decodeJSONArgument
- Add comprehensive tests for round-trip serialization

Fixes #279

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate 0x00/empty string serialization

1 participant