Test suite has no negative vectors outside the scripts suite

[melvincarvalho]

The suite runs six suites live in the browser. Only one of them can fail a validator that is too permissive.

expect: 'reject' appears exactly once in build.js — in the scripts suite, which runs Core's script_tests.json and inherits its expected-invalid vectors. Every other suite uses expect: null:

Suite	Asserts rejections?
Spending (scripts)	yes — Core's vectors carry expected-invalid cases
Headers	no
Difficulty	no
Blocks	no
Transactions & money	no
Light clients (SPV)	no

The fixtures match. All seven non-script vector files are real, valid mainnet data:

genesis-block.json          header-chain-100k.json
merkleblock-block100000.json  merkleblock-first-segwit.json
pruned-window-100000.json   retarget-32256.json  retarget-modern.json

There are no adversarial fixtures at all.

Why this matters

A positive-only suite can only catch rejects-valid failures — cases where the validator wrongly refuses good data. It is structurally incapable of catching accepts-invalid failures, where the validator wrongly admits bad data. For a consensus validator that is the dangerous direction: it is the one that makes you follow a chain the network has rejected.

This is not hypothetical. Two independent reviews have now found precisely that class of bug:

• Five of the eight findings in #3 are classified accepts-invalid.
• #5 (no powLimit or overflow bound in checkProofOfWork) is accepts-invalid, and five of the six suites could never have detected it, because not one of them ever feeds a bad header and demands a rejection.
• #6 (the witness-nonce crash) likewise: the only segwit fixtures are real blocks with well-formed 32-byte reserved values.

The bugs the reviews keep surfacing are exactly the class the suite cannot see. That correlation is the finding.

Suggested fix

Add a negative fixture set and let the non-script suites carry expect: 'reject'. Starting candidates, all cheap to construct and all corresponding to real Core rejections:

• header with nBits above powLimit, and with an overflowing nBits (0xff123456)
• header with a timestamp beyond the 2-hour future bound; header at or below median-time-past
• coinbase witness reserved value that is not exactly 32 bytes
• block whose merkle tree contains a duplicated subtree (CVE-2012-2459) — see the leaf-count table in Block-level malleability coverage: merkle duplicate, 64-byte tx, witness commitment kernel#4
• block containing a 64-byte transaction; the construction recipe is in Block-level malleability coverage: merkle duplicate, 64-byte tx, witness commitment kernel#4
• coinbase scriptSig shorter than 2 or longer than 100 bytes
• non-minimal BIP34 height push
• merkleblock with unconsumed hashes or flag bits

A related gap upstream: packages/kernel has no test files of its own. npm test runs node --test packages/*/test/*.test.js, which matches only packages/swiftsync/test/. The consensus engine's entire coverage is this browser suite, so a hole here is a hole everywhere.

The README's claim — "across all of Bitcoin's rules, from real test vectors" — is accurate about which rules are exercised and quietly misleading about which direction is tested. Worth fixing the coverage rather than the wording.

[melvincarvalho]

Correction to the last paragraph: the engine does have tests.

I wrote that packages/kernel has no test files and that "the consensus engine's entire coverage is this browser suite." That was wrong, and I should have checked the canonical source before saying it.

packages/kernel is a generated snapshot of bitcoin-desktop/schema, which carries a real suite: 17 test files, 2,241 LOC, 137 passing tests — blocks, headers, spv, interpreter, script, script-vectors, taproot, reorg, roundtrip and more. The snapshot simply doesn't vendor the tests, which is a reasonable thing for a CDN-servable artifact not to ship.

The substance of this issue is unaffected, and the correction sharpens it.

The negative-coverage gap exists upstream too. test/spv.test.js in the canonical repo verifies real mainnet proofs and never feeds a bad target. That is precisely why bitcoin-kernel/kernel#5 — an SPV inclusion proof forgeable for free, with nonce: 1 — sat undetected through 137 green tests and six green browser suites.

So the finding is stronger than first stated: it is not that one suite lacks negative vectors, it is that neither the canonical suite nor the browser suite asserts rejections outside the script vectors, and the bug that slipped through both is exactly the accepts-invalid class this predicts.

bitcoin-desktop/schema#72 adds the first negative-vector file upstream (test/pow.test.js, nine vectors, 137 → 146). The suggested fixture list in this issue still applies here, for the browser suite.