Pass fork number to rm_mask so wal_consistency_checking covers visibility map pages

[NikolayS]

Two-patch series. Patch 1 is the backpatch candidate (15–19) and is fully independent of patch 2; patch 2 is master-only (it changes the rm_mask signature, an API break for out-of-core custom resource managers).

[PATCH 1/2] 19c6691 — Don't log VM/FSM pages as standard in CREATE DATABASE WAL_LOG copy (backpatch 15–19)

RelationCopyStorageUsingBuffer() — the CREATE DATABASE STRATEGY=WAL_LOG copy path, introduced in upstream commit 9c08aea6a3090a396be334cc58c511edab05776a (PG 15) — WAL-logged every copied page with log_newpage_buffer(dstBuf, true), claiming the standard page layout for every fork. Visibility map and FSM pages keep their entire payload between pd_lower (24) and pd_upper (BLCKSZ), so the standard-page "hole" optimization elided the whole payload from the FPI, and replay zero-fills the hole. Result: a standby, a PITR restore, or crash recovery on the primary itself reconstructs the copied database's VM and FSM forks as all-zeros while a non-crashed primary keeps the template's bits.

The damage direction is safe-but-bad: lost VM bits / FSM knowledge cannot cause false visibility, but they force extra heap reads for index-only scans, redundant vacuum work (including re-freezing), and poorer free space reuse — which is presumably why it went unnoticed for four releases, and it was invisible to wal_consistency_checking because of the masking blindness fixed in patch 2.

Fix: claim the standard layout for all forks except VISIBILITYMAP_FORKNUM and FSM_FORKNUM (init forks share the standard layout with main, so they keep hole compression):

		if (use_wal)
			log_newpage_buffer(dstBuf,
							   forkNum != VISIBILITYMAP_FORKNUM &&
							   forkNum != FSM_FORKNUM);

The other bulk-copy paths (RelationCopyStorage() via bulk_write.c, log_newpage_range() callers) already pass page_std = false where appropriate.

New always-running TAP test (in patch 1)

src/test/recovery/t/055_create_database_walog.pl (plain test, no PG_TEST_EXTRA needed): builds a template database with populated VM and FSM (insert 5000 rows, delete every 5th, VACUUM (FREEZE)), runs CREATE DATABASE ... STRATEGY WAL_LOG, waits for standby replay, then compares pg_visibility_map_summary() and pg_freespace() for the copied table on primary vs standby, with sanity checks that the primary's VM/FSM are non-trivial so the comparison can't pass vacuously. pg_visibility/pg_freespacemap added to the recovery EXTRA_INSTALL.

Kill-test record (fix reverted via git apply -R of the bufmgr.c hunk, rebuild, refresh tmp_install, rerun):

▶ 1/1 - standby has same visibility map contents as primary after WAL_LOG copy FAIL
▶ 1/1 - standby has same free space map contents as primary after WAL_LOG copy FAIL
#          got: '0|0'          # standby VM: zero pages all-visible/all-frozen
#     expected: '87|87'        # primary VM: 87 pages all-visible, 87 all-frozen
#          got: '87|0'         # standby FSM: 0 bytes avail
#     expected: '87|145504'    # primary FSM: 145504 bytes avail

With the fix restored: recovery/055_create_database_walog OK, 4 subtests passed.

[PATCH 2/2] 77d654c — Pass fork number to rm_mask so wal_consistency_checking covers VM pages (master only)

The blindness: verifyBackupPageConsistency() called rm_mask(page, blkno) with no fork number, so heap_mask() treated VM pages as heap pages; its mask_unused_space() memsets [pd_lower, pd_upper) — the entire VM bitmap — so every VM-page comparison passed vacuously. The stale-VM-bit corruption class addressed by ed62d26 was undetectable.

API change:

void (*rm_mask) (char *pagedata, BlockNumber blkno, ForkNumber forknum);

Updated RmgrData, verifyBackupPageConsistency() (passes the decoded block's fork), the custom-rmgr docs, and all nine in-core mask routines: heap_mask (Heap+Heap2), btree_mask, hash_mask, gin_mask, gist_mask, seq_mask, spg_mask, brin_mask, generic_mask. Heap/Heap2 are the only rmgrs that register VM blocks (verified: all VM buffer registration goes through heapam.c/pruneheap.c). heap_mask() on VISIBILITYMAP_FORKNUM masks only page LSN + checksum, preserving the bitmap; asserts MAIN_FORKNUM otherwise.

Discovery narrative: with patch 2 alone (pre-fix), 027_stream_regress under PG_TEST_EXTRA=wal_consistency_checking FATALed seconds into the suite on the first heap insert into a copied catalog:

FATAL:  inconsistent page found, rel 1663/16384/1247, forknum 2, blkno 0
CONTEXT:  WAL redo at 0/03442088 for Heap/INSERT: off: 27, flags: 0x01;
          blkref #0: rel 1663/16384/1247, blk 15 FPW;
          blkref #1: rel 1663/16384/1247, fork 2, blk 0 FPW

The XLOG/FPI record that had copied that VM page was 73 bytes total (a full image is ~8 kB) — empirical proof the bitmap was elided as a hole. That finding is what patch 1 fixes; the new coverage detects real VM divergence on first contact.

PageClearAllVisible verdict — keep, documented: after ed62d26 every PD_ALL_VISIBLE transition is WAL-logged, so the mask could in principle go. But log_heap_prune_and_freeze() intentionally skips the heap-page FPI and LSN stamp when the only change is setting PD_ALL_VISIBLE with checksums/wal_log_hints off (stamping the LSN without an FPI could make recovery skip an earlier FPI needed to repair a torn page). The flag's on-disk state is therefore not strictly LSN-ordered and can transiently diverge around recovery restarts without indicating corruption. Conservative call: keep the masking; mask_page_hint_bits() now documents exactly why, retaining the XXX for future tightening.

Provenance and validation

• Base: d374280 (master, v19devel). Build: meson setup build --buildtype=debug -Dcassert=true -Dtap_tests=enabled, gcc, Linux x86_64.
• git diff --check clean.
• Final tree (both patches):
  • recovery/055_create_database_walog OK (4 subtests, ~3 s)
  • PG_TEST_EXTRA=wal_consistency_checking recovery/027_stream_regress OK (11 subtests, ~88–118 s) — full regression suite replayed through a standby with wal_consistency_checking = all, zero inconsistent pages
• Kill-tests: 055 fails as shown above with patch 1's bufmgr hunk reverted; 027 under wal_consistency_checking fails with the VM inconsistency above when patch 2's detection is present without patch 1's fix.

Closes #72

🤖 Generated with Claude Code

https://claude.ai/code/session_01PtxA6B4d47ietk5dGsseuP