Skip to content

feat: dispatch warning event on timer saturation#569

Closed
jerome-benoit wants to merge 1 commit into
tinylibs:mainfrom
jerome-benoit:feat/timer-saturation-warning
Closed

feat: dispatch warning event on timer saturation#569
jerome-benoit wants to merge 1 commit into
tinylibs:mainfrom
jerome-benoit:feat/timer-saturation-warning

Conversation

@jerome-benoit

Copy link
Copy Markdown
Collaborator

Summary

Add a 'warning' event to BenchEvents and TaskEvents that is dispatched on both the Bench and the Task instances when the latency samples of a task are dominated by the timer resolution.

const bench = new Bench()
bench.add('hot fn', () => regex.test(line))

bench.addEventListener('warning', evt => {
  // evt.task is typed as Task (warning is in BenchEventsWithTask)
  console.warn(`[${evt.task.name}] timer saturation detected`)
})

await bench.run()

Implementation

Saturation is detected by a new internal detectTimerSaturation(samples, mad) helper in src/utils.ts. It returns true on any of:

  • more than half of the samples are zero
  • the number of distinct sample values is below max(3, min(10, n / 1000))
  • the median absolute deviation is zero with more than 100 samples

The event is dispatched in Task#processRunResult right after this.#result is set with a 'completed' or 'aborted-with-statistics' state, before the 'cycle' event so that cycle listeners can observe the warning state if they need to.

Risk

Runtime-compatible — the event is purely informative and has no listener by default. The only observable change is widening the BenchEvents string union with 'warning', which is a soft compile-time signal for users who exhaustively switch on event types and treat unknown variants as errors. No bump beyond feat: is warranted because no existing API contract is removed or changed in shape.

Add a 'warning' event to BenchEvents and TaskEvents that is dispatched on
both the Bench and the Task instances when the latency samples of a task
are dominated by the timer resolution.

Saturation is detected by a new internal detectTimerSaturation helper in
src/utils.ts. It reports true on any of:
- more than half of the samples are zero
- the number of distinct sample values is below max(3, min(10, n / 1000))
- the median absolute deviation is zero with more than 100 samples

The event is purely informative and has no listener by default, so the
change is runtime-compatible. It only widens the BenchEvents string union,
which is a soft compile-time signal for users who exhaustively switch on
event types.
@jerome-benoit jerome-benoit requested a review from Uzlopak as a code owner May 30, 2026 00:35
@pkg-pr-new

pkg-pr-new Bot commented May 30, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/tinylibs/tinybench@569

commit: 5159de5

@jerome-benoit

Copy link
Copy Markdown
Collaborator Author

Closed in favor of #571 which consolidates this feature with #568 and #570. The audit found that the saturation detection in this PR must operate on the raw latency samples and the raw MAD; running it after the in-place overhead correction from #568 inflates zeroCount with artificially clamped samples and zeroes the MAD on truncated distributions, both producing false positives. The consolidated PR also tightens the heuristic with an n >= 10 minimum-sample guard (avoiding systematic false positives on iterations: 1 benchmarks) and replaces the new Set(samples).size distinct-count with an O(1)-extra-space scan that exploits the sorted-ascending invariant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant