Add mitata benchmark for recommended config linting#2632
Merged
NullVoxPopuli merged 8 commits intoember-cli:masterfrom Mar 24, 2026
Merged
Add mitata benchmark for recommended config linting#2632NullVoxPopuli merged 8 commits intoember-cli:masterfrom
NullVoxPopuli merged 8 commits intoember-cli:masterfrom
Conversation
Contributor
|
looking good so far: |
Introduces a benchmark suite that measures how fast the recommended config lints files of various sizes (.js, .gjs, .gts), with automatic PR comparison comments so performance regressions are visible on every PR. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Fixes bench:compare failure caused by pnpm lockfile config mismatch when .npmrc (which sets auto-install-peers=false) was missing. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The frozen lockfile install can fail when the control branch's lockfile was generated with different pnpm settings. Fall back gracefully since exact reproducibility of the control's node_modules is less critical than being able to run the benchmark at all. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Medium fixtures: ~400-470 lines (was ~85-110) Large fixtures: ~690-980 lines (was ~180-335) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The bench fixtures are plain Ember source files, not test files. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The bench fixtures are Ember app source files (ES modules) that aren't meant to be linted by the project's own eslint config (which uses sourceType: 'script'). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
49b1ebf to
345784e
Compare
- Move bench scripts to alphabetical position in package.json (required by lint:package-json) - Set cwd on the type-aware linting test's ESLint instance so its **/*.gts glob doesn't accidentally pick up bench fixture files Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
NullVoxPopuli
approved these changes
Mar 24, 2026
This was referenced Apr 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.js,.gjs, and.gtsfixture filesmasterand posting a performance comparison comment with summary table and boxplotspnpm bench,pnpm bench:compare,pnpm bench:summary) for running benchmarks during developmentModeled after the benchmark infrastructure in ember-eslint-parser.
New files
tests/lint.bench.mjsLinterAPI + mitatatests/bench/*.{js,gjs,gts}scripts/bench-compare.mjsgit archivescripts/bench-utils.mjsscripts/format-bench-comment.mjsscripts/format-bench-cli.mjsscripts/run-bench.shscripts/local-bench-summary.sh.github/workflows/bench-compare.ymlTest plan
pnpm benchruns successfully and produces outputpnpm bench:compareworks againstmaster🤖 Generated with Claude Code