Skip to content

Add TAP coverage for corrupt WAL summary and backup manifest files#76

Draft
NikolayS wants to merge 1 commit into
masterfrom
claude/testplan-file-mutation-smoke
Draft

Add TAP coverage for corrupt WAL summary and backup manifest files#76
NikolayS wants to merge 1 commit into
masterfrom
claude/testplan-file-mutation-smoke

Conversation

@NikolayS

Copy link
Copy Markdown
Owner

What

Two new deterministic, mutation-based TAP tests that prove the "corrupt file in, clean error out" contract: generate a real input file, apply fixed-offset / fixed-value mutations (no randomness), run the consuming tool on each mutant, and assert it exits nonzero with a clean error message and is never killed by a signal (crash / assertion failure / core dump).

  • src/bin/pg_walsummary/t/003_corrupt.pl
  • src/bin/pg_verifybackup/t/011_manifest_mutation.pl
  • registered both in the respective meson.build

Why

Commit ee654419d5 ("Add additional sanity checks when reading a blkreftable") hardened src/common/blkreftable.c with fork-number and chunk-size validation after external fuzzing found that a corrupted file could index off the end of the fork-number or chunk-entry arrays and crash. No regression test exercised that path, so nothing guards those checks (or the surrounding truncation / overflow handling) against regressing. The mutations here target exactly the fields ee654419d5 validates (fork numbers, chunk sizes) plus the structural boundaries it does not (magic, CRC, EOF, plausible-but-too-large counts).

Note: command_fails_like() only checks nonzero exit + stderr — a signal death still counts as "fails". These tests capture $? directly and assert the low 7 bits (signal) are zero, so a crash is caught as a distinct failure.

Existing vs. added coverage

pg_walsummary — previously only 001_basic.pl (help/version/usage) and 002_blocks.pl (happy-path parsing of a valid summary). No corrupt-input coverage existed. 003_corrupt.pl is entirely new.

Mutation matrix (parses the real file's on-disk layout, then mutates precise offsets):

Category Mutation Expected clean error
Truncation 0 bytes, mid-magic (2B), after-magic (4B), mid-header, mid-chunk, mid-CRC ends unexpectedly
Bit flip corrupt magic wrong magic number
Bit flip fork number too large (16) / negative (-5) invalid fork number
Bit flip corrupt limit_block (valid → CRC catches) wrong checksum
Field oversized chunk-size value (0xFFFF > 4096) chunk N has invalid size
Count oversized nchunks (0xFFFFFFFF) oversized chunk size array
Count large-but-plausible nchunks (reads off EOF) ends unexpectedly

pg_verifybackup005_bad_manifest.pl already feeds hand-written tiny manifests to hit individual JSON parse / checksum / size errors, and 003_corruption.pl appends one byte to a real manifest (→ checksum mismatch) and swaps manifests. What was missing: truncating and bit-flipping a real, full-sized manifest at the byte level. 011_manifest_mutation.pl adds only those missing shapes:

Category Mutation Expected clean error
Truncation 0 bytes, 1 byte, midpoint, drop final newline parse error / checksum mismatch
Structural bit flip top-level {[, corrupt version key, corrupt file-object start could not parse backup manifest
Bogus field flip a file Size digit, flip a file Checksum hex char, flip a Manifest-Checksum hex char manifest checksum mismatch

Test evidence

Built with meson setup build --buildtype=debug -Dcassert=true -Dtap_tests=enabled (asserts on, so an out-of-bounds access would trap as SIGABRT).

pg_walsummary / 003_corrupt                OK   41 subtests passed
pg_verifybackup / 011_manifest_mutation    OK   32 subtests passed

Neighbours confirmed still green: pg_walsummary/001_basic, 002_blocks, pg_verifybackup/005_bad_manifest.

No mutation produced a crash or signal exit — every corrupt input was rejected cleanly, confirming the reader/parser hardening holds. No live bug found.

Closes #70

🤖 Generated with Claude Code

https://claude.ai/code/session_01PtxA6B4d47ietk5dGsseuP


Generated by Claude Code

Commit ee65441 hardened the block reference table reader
(src/common/blkreftable.c) with fork-number and chunk-size sanity
checks after external fuzzing found that a corrupted file could index
off the end of the fork-number or chunk-entry arrays and crash. No
regression test exercised the "corrupt file in, clean error out"
contract, so nothing guards against those checks (or the surrounding
truncation/overflow handling) regressing.

Add two deterministic mutation-based TAP tests. Both generate a real
input file, apply fixed-offset/fixed-value mutations (no randomness),
run the consuming tool on each mutant, and assert it exits nonzero with
a clean error message and is never killed by a signal (crash /
assertion failure / core dump) -- which command_fails_like() alone does
not verify.

pg_walsummary/t/003_corrupt.pl: produces a real WAL summary file (node
with summarize_wal=on, DML, wait for pg_wal/summaries), parses its
on-disk structure, and mutates it: truncation at structural boundaries
(0 bytes, mid-magic, mid-header, mid-chunk, mid-CRC), bit flips in the
magic/fork-number/limit-block fields, an oversized chunk-size value,
and oversized/large nchunks counts. This directly targets the fields
ee65441 validates (fork numbers, chunk sizes) plus the boundaries it
does not (magic, CRC, EOF, plausible-but-too-large counts).

pg_verifybackup/t/011_manifest_mutation.pl: takes a real base backup
and mutates its full-sized backup_manifest at the byte level. The
existing 005_bad_manifest.pl feeds hand-written tiny manifests to hit
individual JSON parse errors, and 003_corruption.pl appends one byte to
the manifest; neither truncates or bit-flips a real manifest. This adds
truncation at fixed offsets, structural JSON bit flips at located anchor
points, and bogus size/checksum fields, asserting a clean, signal-free
failure in every case.

No mutation produced a crash, confirming the reader rejects malformed
input cleanly.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01PtxA6B4d47ietk5dGsseuP
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add file-mutation smoke tests for pg_walsummary and pg_verifybackup

2 participants