Skip to content

add mobench support to ProveKit main#450

Open
dcbuild3r wants to merge 7 commits into
mainfrom
dcbuilder/pr429-main-mobench-fixtures
Open

add mobench support to ProveKit main#450
dcbuild3r wants to merge 7 commits into
mainfrom
dcbuilder/pr429-main-mobench-fixtures

Conversation

@dcbuild3r

@dcbuild3r dcbuild3r commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator

Reopened from #429 using the new dcbuilder/* branch naming.

Old PR: #429
Replacement branch: dcbuilder/pr429-main-mobench-fixtures
Old head branch: dcbuild3r/main-mobench-fixtures
Head SHA: e14fa60beadc25bf1e7d238c9bdbf8f6237fa9f2

Previous PR CI / benchmark runs from #429:

Summary

  • add main-based mobench fixtures for passport, OPRF, and P-256 bigcurve
  • run BrowserStack triad profile across Vivo Y21, Pixel 7, S24 and iOS fallback triad
  • fix Android prove timing so fixture clone/setup is outside the measured prove span
  • use jemalloc as the Android fallback allocator under the native C FFI allocator
  • harden Android incomplete-fixture reporting with timeout/build/kill-evidence fields

Android mobench fix note

The previous failing run was https://github.com/worldfnd/provekit/actions/runs/26007468825. The missing Vivo Y21 monolithic cell did not recover a BrowserStack session payload or summary.json; available artifacts only show the BrowserStack fetch timeout after 7200s for build c943753d95fd0b34f5775aa0a3bc6ff58cbcc3ca.

Before/failure memory from that run:

  • Vivo Y21 OPRF: 328 MB process peak, 10.061s
  • Vivo Y21 fragmented age check: 1187 MB process peak, 65.342s
  • Vivo Y21 monolithic age check: no row; BrowserStack timeout/no summary
  • S24 OPRF: 426 MB process peak, 2.809s

What changed:

  • Android and iOS now use the same semantic boundary for prove benches: per-iteration fixture setup happens before the measured closure, and profile_phase("prove") wraps only the prover entry point. Previously prepared.clone() ran inside profile_phase("prove"), so clone cost and peak memory were charged to proving.
  • Android native C FFI fallback allocation now routes through jemalloc instead of Bionic malloc when no host callback/mmap allocator is active.
  • If BrowserStack returns no summary, failure.json now records attempts, fetch timeout seconds, build id, and any LMK/OOM/SIGKILL lines recovered from attempt/device logs.

Latest benchmark numbers were reposted as a PR comment from #429.

Validation

  • cargo fmt --all
  • cargo test -p bench-mobile --lib
  • cargo test -p bench-mobile --test examples_smoke
  • cargo test -p bench-mobile --test passport_smoke
  • cargo check -p provekit-ffi --target aarch64-linux-android with NDK 26.1 aarch64-linux-android34-clang
  • ruby -e 'require "yaml"; YAML.load_file(".github/workflows/mobile-bench-reusable.yml")'
  • git diff --check

@dcbuild3r ChatGPT Codex Connector

dcbuild3r commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator Author

Latest mobile benchmark numbers

Reposted from #429 comment #429 (comment).

Source: Mobile Bench PR Auto run 26368378239 on dcbuild3r/main-mobench-fixtures at e14fa60b.
Mobench version: 0.1.42. Artifact summaries report 2 measured runs per function/device plus 1 warmup run.

iOS

Benchmark Device Runs Warmup Mean/run Median p95 Min Max Wall total CPU median/run CPU total CPU/wall Peak growth Peak memory Process peak
OPRF iPhone 16 Pro Max 2 1 766.622ms 766.622ms 775.181ms 758.064ms 775.181ms 1.533s 3.364s 6.728s 438.8% 10.95 MB 10.95 MB 362.28 MB
OPRF iPhone 14 2 1 1.096s 1.096s 1.118s 1.075s 1.118s 2.192s 4.497s 8.994s 410.2% 32.55 MB 32.55 MB 206.95 MB
OPRF iPhone SE 2022 2 1 1.085s 1.085s 1.086s 1.085s 1.086s 2.171s 4.486s 8.972s 413.3% 33.48 MB 33.48 MB 209.44 MB
Passport fragmented iPhone 16 Pro Max 2 1 4.751s 4.751s 4.805s 4.696s 4.805s 9.501s 21.524s 43.049s 453.1% 150.95 MB 150.95 MB 1042.44 MB
Passport fragmented iPhone 14 2 1 6.598s 6.598s 6.625s 6.570s 6.625s 13.195s 28.438s 56.877s 431.0% 264.30 MB 264.30 MB 845.34 MB
Passport fragmented iPhone SE 2022 2 1 7.053s 7.053s 7.233s 6.874s 7.233s 14.106s 30.661s 61.323s 434.7% 265.72 MB 265.72 MB 851.73 MB
Passport complete iPhone 16 Pro Max 2 1 4.499s 4.499s 4.514s 4.484s 4.514s 8.997s 19.933s 39.866s 443.1% 376.08 MB 376.08 MB 1594.42 MB
Passport complete iPhone 14 2 1 7.772s 7.772s 8.889s 6.654s 8.889s 15.543s 28.034s 56.069s 360.7% 487.56 MB 487.56 MB 1220.12 MB
Passport complete iPhone SE 2022 2 1 7.443s 7.443s 7.505s 7.381s 7.505s 14.886s 30.253s 60.507s 406.5% 494.86 MB 494.86 MB 1104.30 MB

Android

Benchmark Device Runs Warmup Mean/run Median p95 Min Max Wall total CPU median/run CPU total CPU/wall Peak growth Peak memory Process peak
OPRF Samsung Galaxy S24 2 1 2.725s 2.725s 2.762s 2.688s 2.762s 5.449s 9.657s 19.315s 354.4% 89.95 MB 89.95 MB 488.93 MB
OPRF Google Pixel 7 2 1 3.900s 3.900s 3.943s 3.858s 3.943s 7.800s 11.518s 23.036s 295.3% 18.23 MB 18.23 MB 237.57 MB
OPRF Samsung Galaxy M32 2 1 6.681s 6.681s 6.926s 6.436s 6.926s 13.362s 22.641s 45.282s 338.9% n/a n/a 352.76 MB
Passport fragmented Samsung Galaxy S24 2 1 14.516s 14.516s 14.697s 14.336s 14.697s 29.033s 54.556s 109.112s 375.8% 202.75 MB 202.75 MB 929.28 MB
Passport fragmented Google Pixel 7 2 1 26.237s 26.237s 28.870s 23.605s 28.870s 52.475s 74.610s 149.220s 284.4% 217.21 MB 217.21 MB 789.64 MB
Passport fragmented Samsung Galaxy M32 2 1 40.629s 40.629s 41.183s 40.075s 41.183s 81.257s 137.225s 274.451s 337.8% 284.89 MB 284.89 MB 859.60 MB
Passport complete Samsung Galaxy S24 2 1 11.250s 11.250s 11.671s 10.829s 11.671s 22.500s 46.655s 93.310s 414.7% 376.77 MB 376.77 MB 1157.10 MB
Passport complete Google Pixel 7 2 1 19.856s 19.856s 22.382s 17.330s 22.382s 39.712s 63.704s 127.408s 320.8% 304.69 MB 304.69 MB 980.12 MB
Passport complete Samsung Galaxy M32 2 1 28.921s 28.921s 30.002s 27.840s 30.002s 57.842s 102.707s 205.414s 355.1% 365.05 MB 365.05 MB 1220.69 MB

Memory growth is the benchmark-reported peak growth above baseline; process peak is the reported process RSS peak.

@dcbuild3r dcbuild3r mentioned this pull request Jun 3, 2026
Closed
@github-actions

github-actions Bot commented Jun 3, 2026 •
edited
Loading

Copy link
Copy Markdown

CSP benchmarks

Metric Value
Workflow status [PASS] success
Commit 353e21725c07
Run #29412973002
Circuits benchmarked 21
Iterations averaged per circuit 3

Prover time, peak RSS, peak heap, and verifier time are arithmetic means across the iterations. Peak heap comes from the largest peak memory entry in provekit-cli prove's tracing output; peak RSS is reported by /usr/bin/time -v (max-resident-set-size).

Each metric cell shows the current value followed by the percentage delta against the latest successful main run #29247978240. (new) marks circuits absent from the baseline.

Results
Circuit Constraints Witnesses Prover time Peak RSS Peak heap Verifier time Proof size PKP size
ecdsa_p256 143,321 (±0.0%) 258,215 (±0.0%) 1.74 s (-0.8%) 226 MB (±0.0%) 188 MB (±0.0%) 90 ms (±0.0%) 307 KB (+2.8%) 811 KB (±0.0%)
keccak_1024 788,814 (±0.0%) 1,490,812 (±0.0%) 5.14 s (+0.5%) 897 MB (±0.0%) 874 MB (±0.0%) 450 ms (±0.0%) 370 KB (+0.5%) 5.88 MB (±0.0%)
keccak_128 157,522 (±0.0%) 304,529 (±0.0%) 1.23 s (-0.3%) 236 MB (±0.0%) 202 MB (±0.0%) 110 ms (±0.0%) 309 KB (-0.2%) 1.26 MB (±0.0%)
keccak_2048 1,507,374 (±0.0%) 2,840,532 (±0.0%) 10.13 s (-0.1%) 1.70 GB (±0.0%) 1.68 GB (±0.0%) 847 ms (±0.0%) 392 KB (+1.9%) 11.97 MB (±0.0%)
keccak_256 247,782 (±0.0%) 474,010 (±0.0%) 1.42 s (+0.2%) 290 MB (±0.0%) 249 MB (±0.0%) 150 ms (±0.0%) 329 KB (-0.3%) 1.88 MB (±0.0%)
keccak_512 428,126 (±0.0%) 812,944 (±0.0%) 2.64 s (-0.5%) 542 MB (±0.0%) 455 MB (±0.0%) 250 ms (±0.0%) 352 KB (±0.0%) 3.32 MB (±0.0%)
poseidon2_12 479 (±0.0%) 563 (±0.0%) 40 ms (±0.0%) 12.82 MB (-0.3%) 3.26 MB (±0.0%) 10 ms (+203.0%) 72.54 KB (-1.6%) 436 KB (±0.0%)
poseidon2_16 556 (±0.0%) 719 (±0.0%) 40 ms (±0.0%) 13.35 MB (±0.0%) 3.46 MB (±0.0%) 10 ms (±0.0%) 75.38 KB (+0.3%) 530 KB (±0.0%)
poseidon2_2 231 (±0.0%) 278 (±0.0%) 30 ms (-9.9%) 11.95 MB (±0.0%) 2.72 MB (±0.0%) — 67.58 KB (-1.4%) 108 KB (±0.0%)
poseidon2_4 529 (±0.0%) 535 (±0.0%) 30 ms (±0.0%) 12.16 MB (+0.3%) 2.86 MB (±0.0%) — 73.04 KB (-0.1%) 31.66 KB (±0.0%)
poseidon2_8 363 (±0.0%) 423 (±0.0%) 40 ms (±0.0%) 12.76 MB (-0.3%) 3.15 MB (±0.0%) — 70.77 KB (-0.3%) 365 KB (±0.0%)
poseidon_12 504 (±0.0%) 524 (±0.0%) 40 ms (±0.0%) 13.25 MB (-1.3%) 3.53 MB (±0.0%) 10 ms (±0.0%) 72.05 KB (-2.1%) 569 KB (±0.0%)
poseidon_16 609 (±0.0%) 633 (±0.0%) 40 ms (±0.0%) 13.79 MB (+0.3%) 3.98 MB (±0.0%) 10 ms (±0.0%) 73.25 KB (+0.8%) 717 KB (±0.0%)
poseidon_2 240 (±0.0%) 249 (±0.0%) 30 ms (±0.0%) 11.94 MB (-0.3%) 2.68 MB (±0.0%) — 65.40 KB (-1.3%) 59.18 KB (±0.0%)
poseidon_4 297 (±0.0%) 309 (±0.0%) 33 ms (-9.3%) 12.55 MB (±0.0%) 3.10 MB (±0.0%) — 69.61 KB (-0.1%) 332 KB (±0.0%)
poseidon_8 402 (±0.0%) 418 (±0.0%) 33 ms (-16.8%) 12.81 MB (±0.0%) 3.29 MB (±0.0%) 10 ms (+203.0%) 70.99 KB (-0.9%) 448 KB (±0.0%)
sha256_1024 196,940 (±0.0%) 339,764 (±0.0%) 1.33 s (-0.5%) 271 MB (-1.1%) 236 MB (±0.0%) 163 ms (±0.0%) 306 KB (-0.5%) 1.91 MB (+2.3%)
sha256_128 46,398 (±0.0%) 80,974 (±0.0%) 330 ms (±0.0%) 76.22 MB (+0.9%) 55.50 MB (±0.0%) 40 ms (±0.0%) 262 KB (-12.5%) 503 KB (-1.0%)
sha256_2048 345,399 (±0.0%) 612,724 (±0.0%) 2.60 s (-1.0%) 516 MB (±0.0%) 436 MB (±0.0%) 300 ms (±0.0%) 344 KB (+0.2%) 3.00 MB (-4.5%)
sha256_256 67,904 (±0.0%) 117,944 (±0.0%) 570 ms (-0.6%) 124 MB (+1.1%) 98.23 MB (±0.0%) 60 ms (±0.0%) 276 KB (-0.7%) 719 KB (-1.2%)
sha256_512 110,916 (±0.0%) 191,884 (±0.0%) 693 ms (-1.0%) 152 MB (-0.8%) 126 MB (±0.0%) 90 ms (±0.0%) 296 KB (+0.2%) 1.12 MB (+2.6%)

@dcbuild3r dcbuild3r added the bench Run mobile benchmarks on PRs label Jun 11, 2026
@github-actions

github-actions Bot commented Jun 11, 2026 •
edited
Loading

Copy link
Copy Markdown

Mobench Benchmark Results

IOS — iPhone 14 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 1087.2 1062.7 1128.2 1081.3 1128.2 23.053s 33.64 MB 212.19 MB

5 iterations · 1 warmup · avg is primary metric

IOS — iPhone 16 Pro Max (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 795.6 780.0 822.3 789.8 822.3 17.143s 8.08 MB 364.77 MB

5 iterations · 1 warmup · avg is primary metric

Note: memory growth excludes warmup/baseline retained before the measured iteration.

IOS — iPhone SE 2022 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 1114.4 1093.2 1129.4 1125.1 1129.4 23.275s 34.23 MB 209.06 MB

5 iterations · 1 warmup · avg is primary metric

IOS — iPhone 14 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-complete-age-check-prove 6992.6 6687.8 7536.5 6915.6 7536.5 143.580s 494.42 MB 1304.48 MB

5 iterations · 1 warmup · avg is primary metric

IOS — iPhone 16 Pro Max (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-complete-age-check-prove 4441.8 4292.8 4553.1 4496.7 4553.1 97.787s 350.23 MB 1575.81 MB

5 iterations · 1 warmup · avg is primary metric

Note: memory growth excludes warmup/baseline retained before the measured iteration.

IOS — iPhone SE 2022 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-complete-age-check-prove 7775.8 7496.8 8841.8 7510.9 8841.8 154.651s 494.72 MB 1232.14 MB

5 iterations · 1 warmup · avg is primary metric

IOS — iPhone 14 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 7299.0 6645.4 7924.5 7356.0 7924.5 155.265s 265.25 MB 946.48 MB

5 iterations · 1 warmup · avg is primary metric

IOS — iPhone 16 Pro Max (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 4882.4 4701.4 5129.4 4824.8 5129.4 109.552s 174.45 MB 1086.69 MB

5 iterations · 1 warmup · avg is primary metric

Note: memory growth excludes warmup/baseline retained before the measured iteration.

IOS — iPhone SE 2022 (iOS unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 8069.9 6823.2 9465.6 7638.7 9465.6 170.763s 265.22 MB 932.34 MB

5 iterations · 1 warmup · avg is primary metric

Ios Sina Plot

ios Sina plot

ANDROID — Google Pixel 7 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 4068.0 3878.5 4227.9 4109.6 4227.9 56.925s 2.95 MB 236.42 MB

5 iterations · 1 warmup · avg is primary metric

ANDROID — Samsung Galaxy M32 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 6467.5 6351.8 6608.8 6457.1 6608.8 112.532s — 349.68 MB

5 iterations · 1 warmup · avg is primary metric

ANDROID — Samsung Galaxy S24 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
oprf-prove 2720.3 2557.9 2941.9 2701.2 2941.9 49.069s 118.68 MB 509.63 MB

5 iterations · 1 warmup · avg is primary metric

Note: memory growth excludes warmup/baseline retained before the measured iteration.

ANDROID — Google Pixel 7 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-complete-age-check-prove 19824.1 17421.5 21567.1 20842.8 21567.1 300.506s 327.86 MB 978.55 MB

1 iterations · 0 warmup · avg is primary metric

ANDROID — Samsung Galaxy M32 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-complete-age-check-prove 28159.9 27704.9 28553.9 28289.3 28553.9 516.652s 394.57 MB 1212.76 MB

1 iterations · 0 warmup · avg is primary metric

ANDROID — Google Pixel 7 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 25703.6 24471.1 30313.1 24529.9 30313.1 367.291s 224.49 MB 796.35 MB

5 iterations · 1 warmup · avg is primary metric

ANDROID — Samsung Galaxy M32 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 39759.5 38717.5 40839.1 39943.6 40839.1 684.708s 275.04 MB 848.70 MB

1 iterations · 0 warmup · avg is primary metric

ANDROID — Samsung Galaxy S24 (Android unknown)

Benchmark Avg ms Best Worst Median P95 CPU total Peak growth Process peak
passport-fragmented-age-check-prove 14648.3 14391.5 15066.4 14636.5 15066.4 276.273s 245.88 MB 972.73 MB

5 iterations · 1 warmup · avg is primary metric

Android fixture incomplete

  • Function: bench_mobile::bench_passport_complete_age_check_prove
  • Device: Samsung Galaxy S24-14.0
  • Iterations/Warmup: 5 / 1
  • Fetch timeout: 1800s
  • BrowserStack build: 20fc4a6d403b16f947724e9ddf0e8ffe4f5fe644
  • Reason: Android benchmark process produced no summary.json before the 1800s BrowserStack fetch timeout; likely app/process death or watchdog timeout. Check BrowserStack build logs for LMK/OOM/SIGKILL evidence.

Android Sina Plot

android Sina plot

Posted by mobench at 2026-06-21 10:46 UTC

@vercel

vercel Bot commented Jun 20, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs-provekit-org Ready Ready Preview Jul 15, 2026 11:49am

Request Review

@dcbuild3r

dcbuild3r commented Jun 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

@Bisht13 ready for review once the fresh rerun settles.

I pushed 7dbdafb9 to add the five-sample mobile trigger and kicked off full CI with [mobench:both:5]: https://github.com/worldfnd/provekit/actions/runs/27893152208

@dcbuild3r dcbuild3r force-pushed the dcbuilder/pr429-main-mobench-fixtures branch from 7dbdafb to 6b3d103 Compare July 15, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

bench Run mobile benchmarks on PRs

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@dcbuild3r