feat(saturation): bisect + drain search, plateau detection, transition flags

[brayniac]

Summary

Rewrites the concurrency saturation search (#17 — the last of the review groups). The old search climbed multiplicatively only, judged each rung on a single window, stopped after N consecutive failures, and gated throughput against a linear extrapolation from a moving baseline — so it undershot the true ceiling by up to (step_multiplier − 1), was noise-sensitive, and mismodeled plateaus.

New design — a pure, unit-tested SearchPlanner state machine separated from a thin async driver:

• Climb → drain → re-probe → bisect → confirm. Climb multiplicatively until an SLO breaks; drop to the last-good concurrency and drain to a clean slate (no measurement) before re-probing the failed rung, distinguishing a transient/metastable failure from a genuine one. On a genuine failure, binary-search the exact ceiling, measuring each rung from a drained state, then confirm the boundary over several windows (M-of-N) so a single noisy window can't decide it.
• Marginal-gain throughput gate (replaces linear extrapolation): a rung trips it when tokens/s falls below min_throughput_ratio of the throughput projected from the previous rung (a fixed pre-plateau baseline during bisection) — detecting the plateau directly.
• Transition flags: the knee (saturation onset) and any transient recoveries, in the console summary and JSON. Each step is labeled with its phase (climb/bisect/confirm) and whether it was drained.
• Driver: grows via add_permits, shrinks via forget_permits, and on a drain waits for the in-flight gauge to fall to target (bounded by the sample window) plus a short settle before measuring.

Config

• min_throughput_ratio keeps its name with the new marginal semantics (documented).
• stop_after_failures is now unused (accepted for backward compatibility).

Test plan

• cargo test — 94 lib + 16 integration tests pass, 0 failures. New planner_tests drive the planner against modeled servers: exact-knee bisection (converges to 50 where a multiplicative climb would report 40), no-compliant-at-start, transient recovery after drain, throughput-plateau detection (111), and confirm-window step-down (49).
• cargo clippy --all-targets — clean
• cargo fmt --check — clean
• Independent review of the planner termination/convergence + drain interaction. It caught a real issue — the back-off drop was being measured as a full window (spurious results row + wasted ~sample_window per transient); fixed by making the drop a drain-only Action::Drain (no measurement). Declined its suggestion to drop the throughput gate during re-probe, since that would break plateau detection (a pure-latency transient has scaling throughput and passes; only a real plateau re-fails).
• The async driver drain/forget_permits shrink is build + reasoning-verified; exercising the full climb→bisect→confirm against a live server is the recommended smoke test (low max_concurrency, watch the printed table).

Generated with Claude Code