Skip to content

feat: add public-safe story regression lane (#1701)#1712

Open
prompt-driven-github[bot] wants to merge 1 commit into
mainfrom
change/issue-1701
Open

feat: add public-safe story regression lane (#1701)#1712
prompt-driven-github[bot] wants to merge 1 commit into
mainfrom
change/issue-1701

Conversation

@prompt-driven-github

Copy link
Copy Markdown
Contributor

Summary

Adds a first-class, public-safe story regression lane that runs pytest -m story with a deterministic/offline harness — locally via make regression-stories and in CI as a non-blocking story-regression PR job that needs no LLM/cloud credentials. Story-marked tests are auto-discovered (no manual registration) and a per-story pass/fail summary is reported.

Closes #1701

Changes Made

This issue is build/CI/test-harness infrastructure. Per the Step 8 analysis, no .prompt files were modified: the only prompt-backed candidate (prompts/regression_bash.prompt) generates a credential-requiring live-LLM exerciser — the opposite of this offline lane — and the live Makefile is hand-maintained (its stale prompts/Makefile_makefile.prompt does not back the real targets). All edits are additive and direct.

Build / Harness

  • Makefile — new .PHONY: regression-stories target running the offline tests/story_regression.sh; added to help.
  • tests/story_regression.sh (new) — public-safe prelude (scratch HOME, stripped API/cloud credentials, PDD_RUN_LLM_TESTS=0); runs pytest -m story single-process; treats "no tests collected" (exit 5) as a green no-op so the lane can't silently false-green during the story backfill window.
  • pytest.ini — register the story marker.
  • tests/conftest.py — per-story summary plugin (pytest_runtest_makereport tally + pytest_terminal_summary) reporting story count, total story tests, and per-story PASS/FAIL. Renders only when story tests actually ran, so the normal unit run is unaffected.

CI

  • .github/workflows/unit-tests.yml — non-blocking story-regression PR job modeled on public-cli-regression (draft-PR guard, timeout-minutes: 20, no secrets) running make regression-stories.

Documentation Updated

  • docs/ci.md — document story-regression as the third default PR job; broaden the fork-safety statement to cover make regression-stories.
  • CONTRIBUTING.md — add a "Story Regression Lane" subsection.

Review Checklist

  • PDD conventions followed (no prompt regeneration; hand-maintained Makefile untouched at the prompt layer)
  • make regression-stories runs pytest -m story offline with no API keys
  • CI story-regression job is non-blocking and credential-free
  • Documentation is up to date

Next Steps After Merge

  1. No modules to sync (no .prompt changes) — ./sync_order.sh is a no-op for this PR.
  2. Run make regression-stories to verify the lane is green.
  3. Story-backed tests land via the separate sub-issue (Backfill the story regression suite: top flows + previously-fixed bug regressions #1703 backfill) and are auto-picked-up by the story marker; the merge-blocking coverage gate is tracked separately (CI gate: every user story must have a passing, non-stale regression test #1702) and is out of scope here.

Created by pdd change workflow

Make the story-backed regression tests a first-class CI lane:

- Makefile: new `regression-stories` target running `pytest -m story`
  via the offline tests/story_regression.sh harness (+ .PHONY/help).
- tests/story_regression.sh: deterministic/offline prelude (scratch HOME,
  stripped credentials, PDD_RUN_LLM_TESTS=0); auto-discovers story-marked
  tests; treats "no tests collected" (exit 5) as a green no-op lane.
- pytest.ini: register the `story` marker.
- tests/conftest.py: per-story pass/fail summary plugin
  (pytest_runtest_makereport tally + pytest_terminal_summary).
- .github/workflows/unit-tests.yml: non-blocking `story-regression` PR
  job (draft guard, timeout, no secrets).
- docs/ci.md, CONTRIBUTING.md: document the lane.

Closes #1701

Co-Authored-By: Claude Opus 4 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Story regression lane: make regression-stories + public-safe CI integration

1 participant