telemetry: record linking, the one expensive thing the server never mentioned - #640
Merged
Conversation
…entioned A compile has compile_start/compile_end, a forked test JVM has five events joined on pid, and a link had nothing. That is the operation which forks a whole toolchain — for Scala Native, NIR -> LLVM IR -> clang -> executable — and #603 already charges it a cost against the machine governor. So it was accounted for and then invisible, and "why was this build slow" could not be answered for any build that links. link_start/link_end carry project, workspace, platform, release_mode, is_test and the usual duration/success. `release_mode` is there because it is the field that predicts the number: the same project linked Debug and linked ReleaseFast differ by a large factor, and a total with both mixed in explains nothing. `platform` for the same reason across the four toolchains. Emitted through one `withLinkMetrics` helper rather than the same guaranteeCase block copied into both link handlers. `guaranteeCase`, not `flatMap`, for the reason the compile path uses it: a link that is cancelled or throws must still emit its end event, or the start is left dangling and the duration is unknowable — the bug shape fork_end exists to close. Both readers are updated in the same commit, deliberately. #630 was the fix for these two having drifted apart, and the dashboard silently dropping six of twenty event types via `case _ => ()` is what that drift looked like: - server-metrics gains a Linking card, red when a link failed. - The CI summariser groups by platform and mode, so a Scala Native release link and a Scala.js debug link are not summed into one meaningless total. Scope worth stating: this instruments the SERVER's link path, which is what a real build uses. It does not measure the linking inside bleep-bsp-tests, where the suites drive the bridges directly in a forked test JVM — that cost is already visible per suite via suite_finished, which is how it was measured when those suites were tagged `slow`. Verified against synthetic events: the summariser reports `3 links (1 failed), total 1.8m` split into `Scala Native release 1.5m` / `Scala Native 14.0s` / `Scala.js 3.0s`, and the dashboard renders `Linking — 3 links, 107.0s, 1 release-mode, 1 failed`. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A compile has
compile_start/compile_end. A forked test JVM has five events joined on pid. A link had nothing — and that is the operation which forks a whole toolchain (for Scala Native: NIR → LLVM IR → clang → executable). #603 already charges it a cost against the machine governor, so it was accounted for and then invisible, and "why was this build slow" could not be answered for any build that links.The events
link_start/link_end, carryingproject,workspace,platform,release_mode,is_test, plus duration and success.release_modeis included because it is the field that predicts the number — the same project linkedDebugversusReleaseFastdiffers by a large factor, and a total with both mixed in explains nothing.platformfor the same reason across the four toolchains.Emitted through a single
withLinkMetricshelper rather than the same block copied into both link handlers.guaranteeCaserather thanflatMap, for the reason the compile path uses it: a link that is cancelled or throws must still emit its end event, or the start dangles and the duration is unknowable — the bug shapefork_endexists to close.Both readers, same commit — deliberately
#630 was the fix for these two drifting apart, and the dashboard silently dropping six of twenty event types via
case _ => ()is what that drift looked like. So:server-metricsgains aLinkingcard, red when a link failed.Scope, stated plainly
This instruments the server's link path, which is what a real build uses. It does not measure the linking inside
bleep-bsp-tests, where the suites drive the bridges directly inside a forked test JVM. That cost is already visible per suite viasuite_finished— which is exactly how it was measured when those suites were taggedslowin #639.Verification
Against synthetic events (Scala.js debug 3s, Scala Native release 90s, Scala Native test 14s failing):
and the dashboard renders
Linking — 3 links, 107.0s, 1 release-mode, 1 failedwith the red accent. 107s = 3 + 90 + 14, so the arithmetic is right and the release-mode split lands where it should.Full
bleep compilegreen.🤖 Generated with Claude Code