Perf #8: In-place ldiv! for K = M \ N#159
Open
hmzh-khn wants to merge 4 commits into
Open
Conversation
Tests verify: - _solve_K! accepts K_buffer kwarg and produces identical results - K_buffer works with dense, sparse, regularization paths - Allocation reduction when K_buffer is provided - Error paths (singular, non-finite) work with K_buffer - compute_K_evals accepts K_buffers dict - Backward compatibility (K_buffer=nothing) Co-Authored-By: Claude Opus 4.6 <[email protected]>
Changes: - _solve_K!: Add K_buffer kwarg. When provided, uses lu!/lu + ldiv! to write result directly into pre-allocated buffer instead of allocating. Dense path without regularization uses lu!(M) (overwrites M, safe because M_fns! refreshes it each iteration). With regularization, uses lu(M) to preserve M for finally-block cleanup. Sparse path uses copyto! instead of ldiv! because lu(sparse(M)) + ldiv! allocates more overhead. - compute_K_evals: Add K_buffers dict parameter, passed through to _solve_K!. - run_nonlinear_solver: Pre-allocate K_buffers alongside M/N buffers. - MixedHierarchyGames.jl: Import lu, lu!, ldiv! from LinearAlgebra. Benchmark results (dense path, no regularization): 10×10: 47.9% allocation reduction, 1808 vs 3472 bytes/call 30×30: 49.5% allocation reduction, 13264 vs 26248 bytes/call 100×100: 49.9% allocation reduction, 164928 vs 328928 bytes/call All changes are backward-compatible (K_buffer defaults to nothing). Co-Authored-By: Claude Opus 4.6 <[email protected]>
Measures allocation and timing for _solve_K! with and without K_buffer across dense, sparse, and regularization paths at multiple matrix sizes. Co-Authored-By: Claude Opus 4.6 <[email protected]>
Co-Authored-By: Claude Opus 4.6 <[email protected]>
hmzh-khn
changed the base branch from
main
to
perf/sparse-threshold-heuristic
February 17, 2026 23:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pre-allocate K result buffers and use
lu!+ldiv!in_solve_K!to eliminate per-solve allocation of the K result matrix. For Lane Change 4P with 65 iterations × 4 players, this eliminates 260 matrix allocations per solve.Changes
src/MixedHierarchyGames.jl: Importlu,lu!,ldiv!from LinearAlgebrasrc/nonlinear_kkt.jl:_solve_K!: AddK_bufferkwarg. Dense path useslu!(M)+ldiv!(K_buffer, F, N)when no regularization (overwrites M, safe becauseM_fns!refreshes each iteration). With regularization, useslu(M)(preserves M for finally-block cleanup). Sparse path usescopyto!instead ofldiv!(benchmarked:lu(sparse(M))+ldiv!allocates more).compute_K_evals: AddK_buffersdict parameter, passed to_solve_K!run_nonlinear_solver: Pre-allocateK_buffersalongside M/N bufferstest/test_ldiv_inplace.jl: 22 tests covering correctness, allocation, error paths, integrationtest/test_tiers.jl: Register new test filescripts/benchmark_ldiv_inplace.jl: Allocation benchmark scriptBenchmark Results (Dense path, no regularization)
Dense + regularization: ~20% savings. Sparse path: neutral (copyto! approach).
Key Design Decisions
lu!(M) only when regularization == 0:
lu!(M)overwrites M with LU factors. Safe becauseM_fns!refreshes M each iteration. When regularization > 0, thefinallyblock must subtract λ from the original M diagonal, so we uselu(M)which copies.Sparse path uses copyto! not ldiv!: Benchmarking showed
lu(sparse(M))+ldiv!actually allocates more and runs slower thansparse(M) \ N. The sparse K_buffer only saves the result allocation viacopyto!.Testing
test/test_ldiv_inplace.jlK_bufferdefaults tonothing)Changelog