Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
85 lines (65 loc) · 4.61 KB

CONTRIBUTING.md

File metadata and controls

85 lines (65 loc) · 4.61 KB
Raw

Contributing to semantic-diff

Thanks for your interest. This project has a small, load-bearing discipline that keeps it honest. Please read this before opening a PR — a change that violates the discipline will be sent back even if the code is otherwise fine.

The one rule: every correctness claim ships a must-fail negative control

A green test proves nothing on its own — models and humans alike write a test that merely mirrors the implementation. So every correctness test must include a deliberately-wrong variant that the test rejects. Concretely:

  • If you add a case where a change should be detected, also add (or point at) the case where an equivalent cosmetic edit is correctly ignored — and vice versa.
  • The differential suite (tests/differential.sh) is the model: control #2 flips <→<= and asserts exactly one function changes; the cosmetic control asserts CHANGED: 0. One without the other is not evidence.
  • The scale gate (tests/scale_regex_syntax.sh) computes the ground-truth failing set live (by running the real suite without the driver) and asserts the selector chose a superset. That "compare against reality" pattern is the gold standard — prefer it when you can.

A test that has never failed for the right reason is not a test. Write it red first, watch it fail, then make it green.

The dangerous direction is a false UNCHANGED

The whole design biases toward over-selection. A skipped test that should have run (a false UNCHANGED) is the one truly bad outcome; running a few extra tests is merely inefficient. When you are unsure whether a change is representable, fail loud (INCOMPLETE → run everything) or widen — never report a silent "nothing changed." If your change could introduce a path where an affected test is silently skipped, it needs a control that would catch exactly that.

Honesty over hype

This project's credibility comes from stating its limits plainly.

  • Keep every concrete claim true to the code — line counts, suite counts, the measured selectable-fraction and wall-clock numbers. If you change behavior, update the numbers in README.md / docs/prior-art.md in the same PR.
  • The mechanism is borrowed (from rustc internals, rustowl, and cuda-oxide) and we say so. Don't reframe borrowed work as novel; the new work is cross-build stability + the sound RTS argument.
  • If something doesn't work, document it in docs/prior-art.md "Measured limitations." A known gap is an asset; a hidden one is a landmine.

The dev loop (container-only)

The fingerprinting driver links rustc_private and can only build inside the pinned-nightly container (see README.md "Requirements"). The pure engine + CLI build on host stable.

# Fast host loop — the pure index/diff/select engine + CLI:
cargo test --workspace

# Build the container once, then run the gates (this is where driver changes are exercised):
docker build -t semantic-diff-dev .
for g in m0_double_build differential select workspace scale_regex_syntax; do
  docker run --rm -v "$PWD":/work -w /work semantic-diff-dev bash tests/$g.sh
done

The pinned toolchain is load-bearing. StableHasher output drifts across nightlies, so a fingerprint is only stable within a frozen toolchain. Treat the driver as a versioned plugin: any toolchain bump (the Dockerfile / rust-toolchain.toml nightly date) invalidates every committed fingerprint and requires a full re-baseline. Bump deliberately, never incidentally.

PR expectations

  • The full host suite (cargo test --workspace) is green.
  • All container gates pass, including the M0 gate (tests/m0_double_build.sh) and the negative controls in tests/differential.sh. If your change touches the driver, canonicalization, or the config gate, re-run all of them and paste the output.
  • New behavior comes with its must-fail negative control (see above).
  • Docs that make claims about behavior are updated to match.

Scope

In scope: the fingerprint pipeline (canonicalization, the config gate, cross-crate handling), the diff/select engine, reachability, and honest measurement of the tool on real crates.

Out of scope (documented punts, not accidental gaps): full cross-crate body fingerprinting of a dependency's non-generic MIR (it isn't in crate metadata; we fail loud instead); folding crate version / --features / --cfg into the config gate (re-baseline across those axes for now); and anything that would trade a soundness guarantee for precision. If you want to work on one of these, open an issue first so we can agree on the boundary.