perf(blockchain): serve chain height and L1 head from memory - #3921
perf(blockchain): serve chain height and L1 head from memory#3921brbrr wants to merge 4 commits into
Conversation
|
Claude finished @brbrr's task in 6m 19s —— View job Review: serve chain height and L1 head from memory
No blockers — the caching direction is sound and the invalidation ordering is right where it matters. Four inline comments; the first is the one I'd want addressed before merge. SummaryImportant
Nits
What I checked and found clean
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
6a4d4f7 to
03c7cbb
Compare
|
Claude finished @brbrr's task in 8m 28s —— View job Review: serve chain height and L1 head from memory
Note on this run: the pushed tree is byte-identical to the revision I reviewed earlier (same blob SHAs for all three files, still one commit SummaryNo blockers. The direction is sound and the invalidation ordering is right where it matters. Important
Nits
Verified clean
Build / test
I could not run |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3921 +/- ##
==========================================
+ Coverage 74.99% 75.19% +0.19%
==========================================
Files 446 447 +1
Lines 40312 40290 -22
==========================================
+ Hits 30233 30297 +64
+ Misses 7967 7898 -69
+ Partials 2112 2095 -17
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
03c7cbb to
1c2c1a8
Compare
|
Claude finished @brbrr's task in 11m 48s —— View job Review: serve chain height and L1 head from memory
What changed since the last pass. Contrary to the previous run's note, this tree is not identical — four of the six earlier points were taken up: the No blockers. Three new comments below, plus notes on what's still open and two earlier nits I'd now drop. SummaryImportant
Nits
Two earlier nits I'd now drop — I chased both further and they're smaller than they looked:
Verified clean
Build / test
I could not run |
|
Claude finished @brbrr's task in 5m 32s —— View job Review: serve chain height and L1 head from memory
What
I checked the two new tests are non-vacuous: drop the constructor priming (lines 210-214) and the first goes red; drop SummaryNo blockers. Two comments, one carried over. Important
Nits
Verified clean
Build / test
I could not run |
|
Claude finished @brbrr's task in 4m 40s —— View job Review: serve chain height and L1 head from memory
This revision is clean — both remaining items from the previous pass are resolved, and I found nothing new. No inline comments this round. What
|
| Previous finding | Resolution |
|---|---|
update.BlockNumber was a cacheHeads-only nil deref in SetL1Head |
Fixed — blockchain.go:433-436 stores nil (unknown) for a nil update, so both modes agree, plus TestClearingTheL1HeadEmptiesTheCache |
| Field doc named the bypass case but not the concurrency case | Fixed — blockchain.go:109-113 now names both: "written through Blockchain by one writer at a time: a migration on the raw db.KeyValueStore, or a concurrent store and revert, leave it stale" |
On the headsMu mutex I'd raised three times: documenting the constraint instead of enforcing it is a legitimate call here, and the reworded comment now says the thing a future reader actually needs. I re-verified the invariant independently once more and it holds tightly — sync/sync.go:359 is the only Store caller in the tree, both revertTask call sites are conc/stream callbacks on the same verifiers stream (sync/sync.go:191, and :364 from inside storeTask on that goroutine), p2p/sync holds a *blockchain.Blockchain but only ever reads it (Height, GlobalStateRootByBlockNumber, StateAtBlockNumber, SanityCheckNewHeight — grep finds no Store( anywhere under p2p/), and sequencer mode takes the if cfg.Sequencer branch at node/node.go:326 so synchronizer is never constructed. If a second writer ever does appear, the comment is now the thing that flags it.
Verified clean
- Writer inventory (re-grepped).
core.WriteChainHeight/DeleteChainHeightexist only atblockchain/statebackend/block_ops.go:157,199,202, reachable only viaStore/Finalise/RevertHead;core.WriteL1Headonly atblockchain/blockchain.go:425. No migration, pruner, mempool, or p2p path touches either key, so the documented invariant is grep-checkable. - The nil-update path is genuinely correct, not just non-panicking.
WriteL1Head(db, nil)marshals a nil pointer,GetL1Headunmarshals it back to a zeroL1Headwith no error (core/accessors.go:206-220) — so cachingnil("unknown") and letting readers fall through to the database gives exactly the same observable result as the pre-PR code. The test asserts that. - No feed or callback inside the write window.
stateBackend.Store(statebackend.go:79-108) does everything in onedatabase.Writebatch and notifies nobody, so the gap between the commit and the deferredcacheChainHeight()is not observable by any in-process subscriber — a stale-low cache in that window still names a block that exists. - Invalidation ordering.
RevertHeadnils before the commit and refreshes after, so readers fall back for the whole window;Store/Finalisecan only be stale-low. The asymmetry is deliberate and correctly commented. EventFilterdeliberately back on the database (blockchain.go:519-521) with the reason written down —Eventsre-reads latest every call by design, so the constructor must not disagree with it.- Tests are non-vacuous. Spot-checked each: drop the
update == nilguard andTestClearingTheL1HeadEmptiesTheCachepanics; dropdefer b.cacheChainHeight()fromRevertHeadand the newTestRevertsubtest atblockchain_test.go:1294counts a read and fails; drop the constructor priming andTestHeadsAreCachedWhenTheDatabaseIsAlreadyPopulatedfails.headReadCounteronly interceptsGet, which is the right shape for both accessors today. --remote-dbwiring (node/node.go:258-260) is correct and correctly scoped;node.go:427also makes a remote node's synchronizer read-only, so it never reaches the write paths at all.- No value copies of
Blockchainnow that it holdsatomic.Pointerfields — vet's copylocks is clean.
One thing for the PR description (not a code issue)
HeadState() still resolves the head itself (statebackend/statebackend.go:17, deprecated.go:22), which is where every block_id: "latest" state query lands via stateByBlockID — getNonce, getStorageAt, getClass*, estimateFee, simulateTransactions, traceCall. EventFilter.Events (event_filter.go:161) and SetRangeEndBlockToL1Head (event_filter.go:106) also still read the database, the first now deliberately. That's a reasonable scope — Head/HeadsHeader/Height are worth caching on their own, and the constructor priming helps a restarted node immediately — but the title reads as "no more head reads", so a line saying which reads were measured and which remain would save the next person a benchmark.
Build / test
go vet ./blockchain/... — clean (it type-checks the test files, so the new tests compile). On this commit golangci-lint, CodeQL, Code Quality, Dependency Review and Test Documentation Build are green; Juno Test, Deploy and Test on Dev and Container Security were still in progress when I finished.
I could not run go test ./blockchain/ -race — the command needed approval in this environment — so I verified the new tests by reading them rather than executing them. Please confirm Juno Test goes green.
• branch perf/blockchain-head-cache
EgeCaner
left a comment
EgeCaner
left a comment
There was a problem hiding this comment.
Here I worry about cache DB consistency, lets somehow make it more robust. Also I guess we can update the caches directly after writes without requiring to read the ChainHeight or L1Head back from DB in order to populate the cache
| @@ -3,6 +3,7 @@ package blockchain | |||
| import ( | |||
There was a problem hiding this comment.
Here I worry about the tight window where db is updated and cache is not, we might have to synchronize them, or invalidate cache before starting writing to DB, so we don't read stale cache value, at the moment nothing hints cache is stale. We will still have concurrent DB read/write case but it might still worth
There was a problem hiding this comment.
A reader does a lock-free atomic load, so a reader is safe as it is. The pair {database commit, cache refresh} in a writer is not atomic, but the node has only one writer.
Store and RevertHead are stream callbacks on the same stream, and storeTask calls revertTask on that same goroutine. There is only one caller of Store in the tree.
Thus a mutex would protect an interleave that no call path can cause, and it would tell a future reader that this package expects concurrent writers. I prefer to keep the rule in the field comment, where a grep can check it. It would make sense adding a mutex when we'll get a second writer.
You also asked to set the cache directly instead of a read of the database. Store and Finalise can do this, because it writes exactly block.Number. RevertHead cannot: Blockchain does not know the new height unless it reads the database or repeats the rule "delete at genesis, else N-1" in a second place. RevertHead also cannot calculate the new height when the cache is already nil. Thus the choice is one read-back rule for all three methods, or a mixed scheme with the height rule in two places. I decided to keep the read-back.
|
Claude finished @brbrr's task in 3m 47s —— View job Review: serve chain height and L1 head from memory
This revision is clean — I found nothing new. No inline comments this round. What
|
2 participants
No description provided.