Skip to content

perf(rust): reduce sort32 bucket overhead#398

Open
dexhunter wants to merge 1 commit into
sparkjsdev:mainfrom
dexhunter:perf/sort32-review-step1-20260715
Open

perf(rust): reduce sort32 bucket overhead#398
dexhunter wants to merge 1 commit into
sparkjsdev:mainfrom
dexhunter:perf/sort32-review-step1-20260715

Conversation

@dexhunter

Copy link
Copy Markdown

Summary

  • count valid keys in fixed-size chunks to reduce loop overhead
  • reuse one mutable bucket reference per scatter instead of indexing each bucket twice
  • return before the scatter passes when there are no active splats
  • add regression tests for invalid keys, finite-value boundaries, and stable ordering

Performance

The benchmark builds the production Wasm sorter from a pinned upstream archive, overlays only rust/spark-rs/src/sort.rs, and compares baseline and candidate calls on the same process and fixtures. It validates exact active counts, descending order, stable equal-key ordering, unchanged input/padding, and Wasm size before printing the metric.

sort32_paired_latency_score is the candidate/baseline latency ratio, so lower is better and 1.0 is the baseline. Three final-patch confirmations measured 0.750563, 0.768874, and 0.760215. The median is 0.760215, a 24.0% latency reduction. The suite includes small inputs and one-million-key random, periodic, and adversarial layouts.

Autoresearch trajectory: https://dashboard.weco.ai/share/4avZ2RktRLkZmJ1sWBU1Tiqqd20Zndg7

Validation

  • cargo test --manifest-path rust/spark-rs/Cargo.toml (2 passed)
  • strict paired Wasm evaluator across 10 input layouts
  • git diff --check

@dexhunter

Copy link
Copy Markdown
Author

Rechecked against current main (fec6d05): this still merges cleanly, and rust/spark-rs/src/sort.rs has not changed upstream since #327, so the patch is still confined to that one file (+53/-16).

The amber check state is not a failure — Spark CI Linux and Spark CI Windows are both sitting in action_required awaiting fork-workflow approval, so they have never actually run. I ran the same steps locally against main plus this patch: npm run lint clean across 84 files, npm run test 1/1, and cargo test --manifest-path rust/spark-rs/Cargo.toml 2/2 (the two regression tests this PR adds).

Separately, and this applies to any Rust-only change rather than to this PR specifically: build-dist.yml triggers on paths: ['src/**'], so a commit confined to rust/ does not regenerate the committed dist/ bundles that inline the wasm. #327 landed on 2026-06-08 and the next Bump dist build was 2026-07-15 (8a033bc1), immediately after the unrelated src/ change in #378 — so that sort work reached dist/ about five weeks after it merged. Happy to add rust/** to that trigger, here or as a separate PR, if that would be useful — or to leave it alone if the dist refresh is meant to be deliberate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant