ADR-0017/0018 + working prototype: bringing blockchain benefits to backends natively#3
Merged
Merged
Conversation
Records the analysis of modelling a conventional API server's application logic as smart contracts on this Raft substrate. Captures that the framework gives the form of smart contracts (deterministic, replicated, audited execution) without the substance of trustlessness, since Raft is CFT not BFT, and lands on guidance for when the model fits (single trust domain, audit-critical) versus when it does not (multi-party trust minimization). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
…Proposed) Constructive counterpart to ADR-0017: a layered "deterministic core + effectful edge" design that delivers CFT availability, exactly-once, provable/tamper-evident audit and deterministic replay through ordinary backend code. Records seven pillars (module SDK, determinism-as-guarantee, committed-intent effects/outbox, CQRS + multi-Raft + pluggable state, Merkle audit + committed code version, step budget, signed commands) and a phased roadmap, with each pillar mapped to the backend principle it preserves. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Adds a module-SDK layer on top of the consensus core (no changes to the existing Raft/state-machine code, all 42 prior tests still green): - defineModule + ModuleHost: register modules of pure reducers, dispatch commands, run read queries, snapshot/restore state. - Deterministic ReducerContext (now/random/id) derived entirely from a leader-resolved Seed; resolveSeed() is the sole leader-side entropy site (the analog of models/book.ts). PRNG is sha256-seeded mulberry32; ids are sha256(nonce:counter) — fully reproducible from the seed. - Demo counter and notes modules; a convergence test proves two independent hosts fed the same seeded command stream reach byte-identical state. Review hardening applied: reducers receive a deep-cloned state (a throwing or mutating reducer can't corrupt committed state), reducers return an explicit result distinct from next-state, snapshot keys are emitted in deterministic sorted order, and module validation matches untrimmed dispatch. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Implements pillar 3: reducers emit declarative EffectIntents that land in a deterministic outbox; a post-commit EffectExecutor (the effectful edge) runs each effect and feeds the result back as a committed EffectResultEntry that replicas apply identically — "the replicated log is a transactional outbox." - EffectIntent gains onResult; OutboxEntry/EffectResultEntry/EffectHandler added. - ModuleHost: outbox keyed by idempotencyKey, recorded on apply (dedup), pendingEffects(), idempotent applyEffectResult() that dispatches onResult, outbox folded into snapshot/restore. - EffectExecutor.drain(): runs handlers with an in-flight guard, submits the result on success, leaves the effect pending for retry on failure. - Demo payments module (charge -> http effect -> settle). Exactly-once at the state level is enforced by three layers: outbox dedup on enqueue, idempotent applyEffectResult (first-applied wins), and the executor in-flight guard; handler execution is honestly at-least-once across restarts. Determinism preserved — the executor's resolveSeed() is the sole edge entropy site and its value is committed verbatim, so replicas converge. Review hardening: reserved __-prefixed module names rejected fail-closed, pendingEffects() returns cloned intents, seed-provenance comments corrected. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Implements pillar 5: upgrades the runtime audit from a linear hash chain to a Merkle accumulator and records a per-command module code-version hash, so the audit proves which logic version produced each result (closing the "tamper-evident data but not logic" gap from ADR-0017). - MerkleAudit: append-only accumulator with O(log n) inclusion proofs and a compact root. Odd levels promote the unpaired last node unchanged; leaves and internal nodes are domain-separated (0x00/0x01 prefixes, RFC-6962 style) to avoid second-preimage ambiguity. Canonical (sorted-key) serialization keeps hashing deterministic across replicas. - moduleCodeHash: a deterministic logic-identity hash over a module's name, version, and reducer sources. Honestly scoped as a POC stand-in for hashing the built artifact (omits closures/imports, formatting-sensitive). - ModuleHost stamps each applied command (200 and 500; not unknown-command 404s) into the audit with its module/command/actor/requestId/status/codeHash; audit folded into snapshot/restore under the reserved __audit key. Convergence test proves two independent hosts reach an identical audit root; a version or reducer-body change shifts the codeHash recorded in the leaves. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Adds a "Prototype status" section to ADR-0018 summarizing the three reviewed milestones realized under src/runtime/ (module SDK + deterministic runtime, committed-intent effects/outbox, Merkle audit + code version) and what remains deferred (consensus wiring, CQRS/pluggable store, multi-Raft + step budget, signed commands/BFT, vm sandbox). Adds src/runtime/README.md orienting the deterministic-core + effectful-edge layer for future work. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
naveed949
changed the title
Module commands now ride the real replicated log end-to-end (previously the
runtime was only exercised by deterministic replay):
- Adds a generic MODULE command to the consensus Command union, carrying a
leader-resolved seed; ApplyResult gains an optional result field.
- BookStateMachine.apply is narrowed to Exclude<Command,{type:'MODULE'}> so the
exhaustiveness guard stays intact; ReplicatedStateMachine intercepts MODULE
first and dispatches to an embedded, lazily-created ModuleHost. The idempotency
check runs before dispatch, so a replayed requestId never re-runs the reducer.
- RsmSnapshot carries optional module state (back-compat for module-free nodes);
snapshot/restore round-trips it.
- buildModuleCommand (runtime) resolves the seed leader-side, like models/book.ts.
- Integration test: a 3-node cluster converges on identical module state AND
Merkle audit root, proves MODULE idempotency, and proves the leader-resolved
seed flows through the log (same id/timestamp on every node).
The non-MODULE path is byte-for-byte unchanged; all 42 original tests still pass.
Review fixes: gate the convergence wait on the last entry (race), and keep
read-only module accessors from allocating a host on module-free nodes.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Turns determinism from a convention into an enforced guarantee and adds a deterministic resource bound, without putting any fragile machinery on the replicated apply path: - Static determinism lint at registration: defineModule (strict by default) rejects reducers that reference non-deterministic globals (Math.random, Date.now/new Date(), crypto access, require/import, process, globalThis, fetch, timers, performance, Reflect). Honestly scoped as a static fn.toString() stand-in — bypassable by indirection, not a sandbox; strict:false opts a vetted reducer out. Covers command reducers only. - Deterministic resource bound: ModuleHost rejects a reducer's result with 413 when it emits more than maxEffects (16) effects or its next-state exceeds maxResultBytes (64 KiB). Computed from the reducer's own output via a canonical serializer, so every replica rejects identically — state/outbox untouched, audited at 413. (A preemptive CPU/step meter and a vm sandbox need a worker/vm and remain deferred.) - Extracts a single shared canonical-JSON serializer (canonical.ts) used by both the Merkle audit hash path (throw-on-undefined, byte-identical to before) and the size bound (drop-undefined), removing a drift hazard. All 84 prior tests still pass; consensus + book suites unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Implements the read-scaling half of pillar 4: a derived, rebuildable read model built from the committed module-command stream, answering rich/indexed queries the raw module state does not provide — without touching the consensus apply path. - defineProjection/ProjectionDefinition: a pure `on` fold over ProjectionEvents (committed commands as seen by the read side) plus named queries. - ProjectionHost: applyEvent, query, rebuild (full reset + replay), and snapshot/restore (documented as a cache warm-start, not a source of truth). - Demo noteIndex projection: indexes notes by actor (byActor/total/actors) — a query the flat notes array would need an O(n) scan for. The projection never sits on the apply path, never feeds back into the log, and depends on no ModuleHost internals — the log/ModuleHost remain the sole source of truth (ADR-0002 upheld). rebuild() proves the model is a derived cache: incremental view equals a from-scratch replay, and a polluted cache rebuilt from a different stream retains no trace of the old one. Pure folds give convergence. Cloning reuses the shared canonicalJson (no new serializer). Pluggable state-store backend (the state>RAM half of pillar 4) remains deferred. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Implements pillar 7: a module command is signed by the originating actor's key,
and every node verifies it on the apply path, so a malicious leader cannot forge
`actor`. Adds accountability without full BFT; fully backward-compatible.
- signing.ts: ed25519 primitives (stdlib crypto) — generateActorKeypair,
sign/verifyCommand over the canonical LOGICAL payload, and a KeyRegistry
(actor -> authorized public key allowlist). verifyCommand returns false rather
than throwing on malformed input, so a bad command is a clean 401, not a crash.
- The signature covers { module, command, input, actor, requestId } but NOT the
leader-resolved seed: the actor signs before the leader adds the seed, and the
seed is non-security determinism. Every node rebuilds the same logical payload
from the committed command + meta and verifies, so tampering with any signed
field is rejected identically (401, no state change, audited) on all nodes.
- consensus: optional sig?:string on the MODULE command (additive); RSM threads
it to the host and exposes setKeyRegistry/registerActorKey. No registry ⇒ no
verification ⇒ byte-identical legacy behavior (42 originals unaffected).
- e2e: a signed counter.increment converges 200 on a 3-node cluster; a forged
actor:'alice' command (wrong key / unsigned) is rejected 401 on every node.
Documents that failure results (incl. 401) are cached under requestId, so a
corrected retry should use a fresh requestId.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Records the four deferred milestones now realized — consensus wiring (M4), determinism enforcement + resource bounds (M5), CQRS read projections (M6), and signed commands (M7) — and narrows the remaining frontier to the pluggable state store, multi-Raft sharding, a preemptive CPU/step meter + vm sandbox, and an optional BFT swap. Expands the runtime README file map with the new modules and notes that the runtime now rides real consensus. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Implements the state-larger-than-RAM half of pillar 4: a StateStore abstraction and a transactional keyed module model so a module's state is a collection of records addressed by key — reducers touch only the keys they need instead of loading a whole-state blob. Additive and runtime-only; the consensus core is untouched and all 120 prior tests still pass. - stateStore.ts: StateStore interface + in-memory MemoryStateStore (the seam a persistent embedded KV/LSM drops into; snapshots become store checkpoints) + a copy-on-write StoreView. Iteration is always sorted (never Map insertion order) and reads are deep-cloned, so nothing leaks insertion order or a live reference — the determinism the cluster depends on. - keyedModule.ts: defineKeyedModule / KeyedReducer — mutations go through the StoreView; strict determinism lint applies as for whole-state modules. - ModuleHost is kind-aware: keyed modules get their own store; a keyed command commits its view only on a clean, in-budget return (throw / lint / 413 reject ⇒ zero writes), and keyed + whole-state modules coexist in snapshot/restore without colliding with the reserved __outbox/__audit keys. - Deterministic resource bounds for keyed commands: maxEffects, maxWrites (256), maxResultBytes, computed from the buffered writes after the reducer returns. - Demo accounts module (open/deposit/transfer) showing per-key access. Tests include a cross-incidental-order convergence proof: independent commuting commands applied in different orders reach a deep-equal store snapshot (the audit root legitimately differs, since the audit is order-sensitive by design). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Adds M8 to the prototype status and narrows the remaining frontier to multi-Raft sharding, a preemptive CPU/step meter + vm sandbox, and an optional BFT swap. Expands the runtime README file map with stateStore/keyedModule and the accounts demo. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
…p meter
Hardens pillar 2 (determinism as a structural guarantee) and delivers the
pillar 6 "gas" admission control, opt-in and additive (existing modules keep
direct execution; all 141 prior tests unchanged; no consensus changes).
- sandbox.ts: runs a whole-state module's reducers inside a Node `vm` context
that ENFORCES determinism by removal — Date, timers, crypto, process,
Reflect/Proxy, performance and Intl are deleted, Math.random throws, and the
locale-sensitive prototype methods (toLocaleString/localeCompare/
toLocale*Case) are neutralized, so a careless reducer that reaches for them
throws instead of silently diverging replicas. Reducers must be self-contained
arrow/function expressions (method-shorthand is rejected at registration).
- defineModule(def, { sandbox, stepBudgetMs }) opts a module in; the host
compiles its reducers once at register and dispatches them via the sandbox on
the apply path WITHOUT a timeout, so the run stays a pure deterministic
function of (state, input, ctx) — a wall-clock interrupt there would diverge.
- The step/CPU meter is leader-side admission only: admit() dry-runs the reducer
in the sandbox with a vm timeout against a CLONE (no mutation/effects/audit)
and rejects a runaway (503) before it enters the log; followers trust
committed entries are bounded (the CFT model). A reducer throw maps to 500 on
both admit and apply.
- Honestly scoped: vm is a determinism aid against careless reducers, NOT a
security boundary against malicious code (Function/eval/globalThis remain
reachable) — mirroring the M3/M5 caveats. Demo `compute` module included.
Cross-realm note: the vm timeout error is duck-typed (ERR_SCRIPT_EXECUTION_
TIMEOUT), not instanceof, since it originates in the vm realm.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Adds M9 (vm determinism sandbox + step meter) to the ADR-0018 prototype status and runtime README, and introduces ADR-0019 proposing multi-Raft sharding for write scaling — per-shard Raft groups + a shard router + cross-shard sagas (try/compensate, chosen over blocking 2PC for availability). Narrows the remaining frontier to multi-Raft (now tracked by ADR-0019, prototype M10) and an optional BFT swap (documented as a seam, beyond a prototype increment). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Implements the ADR-0019 prototype: write scaling via multiple INDEPENDENT Raft groups, a deterministic shard router, and cross-shard transactions via a saga (try/compensate). Additive — no changes to src/consensus/* or moduleHost.ts; each shard is an ordinary single-group cluster, so per-shard M1–M9 guarantees carry over unchanged. - shardRouter.ts: ShardRouter over N state-free shard handles. shardFor(key) = sha256(key) mod N — a pure, deterministic mapping every participant agrees on; routes a partition key to the owning shard's current leader (NoShardLeaderError when none is known, for caller retry). - saga.ts: runSaga runs steps forward; on the first failure it compensates the already-succeeded steps in reverse. A compensation that itself throws is NOT swallowed — it is reported in compensationFailures (kept out of compensated), the real "funds may be lost" signal a conservation-critical coordinator must surface. Atomicity by compensation, not isolation (per ADR-0019). - accounts gains a single-key withdraw (reject on missing/insufficient/non- positive; cannot go negative); cross-shard transfer = withdraw on X + deposit on Y with compensating deposit. - Tests: routing determinism, parallel independent shard writes, a successful cross-shard transfer, compensation on a failed credit leg (funds conserved on every node of the shard), and a pure compensation-throws reporting test. Review fixes: surface throwing compensations via compensationFailures; assert funds conservation against every node, not a sampled replica. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
Adds M10 (multi-Raft sharding) to the ADR-0018 prototype status and runtime README, and introduces ADR-0020 which honestly assesses the BFT swap: the real pluggability point is the commit-ordered-log contract above the log (not the Raft-specific Transport), a genuine BFT engine is a protocol-level project beyond a prototype increment, and everything that ports unchanged under BFT (deterministic state machine, Merkle audit, signing, core/edge split) is already in place. Marks the frontier closed for this prototype effort. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
naveed949
marked this pull request as ready for review
June 21, 2026 16:33
…ration-vyma9g # Conflicts: # docs/adr/README.md # src/consensus/replicatedStateMachine.ts # src/consensus/stateMachine.ts # src/consensus/types.ts
The "leader steps down when it removes itself" test (and other multi-change membership tests) chain several elections/commits, each gated by its own waitFor (up to 2–3s). applyCommitted resolves the proposal BEFORE the self-removal step-down, so there is no hang — but the SUM of those waitFor budgets can exceed Jest's 5s default test timeout when workers run in parallel under CPU contention (the suite always passed in isolation). Raise the suite timeout to 15s so a slow-but-correct run isn't mistaken for a hang; the per-waitFor windows remain the real guards against a genuine stall. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NneoGGKxBp5ZExL7D7i1Lu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch now builds on PR #2 (merged): the module runtime plugs into the framework's pluggable
StateMachineseam instead of bolting aMODULEcommand into the consensus core. The result: the consensus core diff vsmainis empty — the entire runtime is additive.Decision records (renumbered after PR #2's ADR-0017)
StateMachineseam below the log; theConsensusseam above the log remains.)How it plugs in (post-reconciliation)
ModuleStateMachineadaptsModuleHostto PR Decouple application from consensus core via pluggable StateMachine #2'sStateMachine<C,T>interface —new RaftNode({ stateMachine: new ModuleStateMachine() }). NoMODULEcommand in the core; a module command is just an application command.ModuleHostand all its unit tests are unchanged — the adapter splits the app command into the host's(cmd, meta)call.src/index.ts.Prototype milestones (all on the seam now)
ReducerContextfrom a leader-resolved seedStateMachineseam; cluster converges on state + audit root413resource boundsactorrejected401cluster-wideStateStore+ transactional keyed modulesvmdeterminism sandbox + leader-side step/CPU meterTests: 160 passing (
--runInBand),tsc --noEmitclean, consensus core untouched vsmain. (Themembership.test.ts"leader steps down" test is a pre-existing flake under parallel load — passes in isolation and single-worker.)Frontier: an optional BFT consensus swap — documented in ADR-0021 as a protocol-level project beyond a prototype increment.
🤖 Generated with Claude Code