perf: two-phase linesearch (cheap scout + full verify)#165
Open
hmzh-khn wants to merge 3 commits into
Open
Conversation
Tests for :twophase linesearch method: - Solver convergence, same solution as geometric - Step sizes bounded by 1.0 - Works with recompute_policy_in_linesearch=false - Works with different initial conditions Co-Authored-By: Claude Opus 4.6 <[email protected]>
Scout phase uses cheap residual evaluation (stale K, no recomputation) to find a promising step size via geometric reduction. Verify phase then confirms with full K recomputation at the candidate. If full eval rejects the scout's choice, tries 1-2 smaller steps before giving up. This separates the cheap scouting (~0.3ms/eval) from expensive verification (~3.3ms/eval), reducing total K recomputations. Changes: - src/nonlinear_kkt.jl: Add :twophase branch with scout/verify logic - src/types.jl: Add :twophase to VALID_LINESEARCH_METHODS - test/test_nonlinear_solver_options.jl: Add validation test Co-Authored-By: Claude Opus 4.6 <[email protected]>
Compares all 5 linesearch methods on 3-player and 4-player problems. Results on 4-player chain (T=5, s=4, c=2): - :geometric: 2.51ms, 6652KB - :armijo: 2.48ms, 6652KB - :armijo_quadratic: 2.55ms, 6652KB - :armijo_interp: 2.53ms, 6652KB - :twophase: 2.03ms, 6066KB (19% faster, 9% less allocation) Two-phase shows improvement by avoiding full K recomputation when the cheap eval accepts the scout's candidate. Co-Authored-By: Claude Opus 4.6 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
:twophaselinesearch method that separates cheap scouting from expensive verification. Scout phase uses stale K (no recomputation) with geometric reduction to find a promising step size. Verify phase recomputes K only at the candidate and falls back to 1-2 smaller steps if verification fails.Bead:
ry8PR chain: PR1 (warm-start α) → PR2 (quadratic) → PR3 (cubic) → PR4 (this)
Changes
src/nonlinear_kkt.jl: Add:twophasebranch with cheap/full residual closures, scout phase (geometric with stale K), verify phase (1-3 full K evals)src/types.jl: Add:twophasetoVALID_LINESEARCH_METHODStest/test_twophase_linesearch.jl: 5 testsets, 17 tests covering: solver convergence, same solution as geometric, step bounds, recompute_policy=false, different initial conditionstest/test_tiers.jl,test/test_test_tiers.jl: Add to FAST tiertest/test_nonlinear_solver_options.jl: Add:twophasevalidationscripts/benchmark_twophase_linesearch.jl: Benchmark of all 5 methodsDesign
The cheap eval uses the K matrices from the current z_est. For small α (most common), K changes little — good approximation. For large α, cheap eval may accept a step that full eval rejects. The verify phase handles this with 1-2 fallback attempts at smaller step sizes.
Testing
julia --project=. test/test_twophase_linesearch.jlBenchmark Results (4-player chain)
Two-phase is 19% faster and 9% less allocation on this benchmark.
Changelog
🤖 Generated with Claude Code