Skip to content

Commit 8edb01f

Browse files
committed
Require causal diagnosis before repair
1 parent 244b797 commit 8edb01f

1 file changed

Lines changed: 13 additions & 1 deletion

File tree

  • plugins/testforge/skills/software-verification

plugins/testforge/skills/software-verification/SKILL.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,19 @@ Keep diagnostic and reproduction commands capability-matched, read-only where po
7171
- `scripts/normalize_test_results.py` for JUnit XML, Jest JSON, or generic command records.
7272
- `scripts/assemble_report.py` only after the manifest and referenced evidence validate.
7373

74-
Classify failures before patching: `PRODUCT_DEFECT`, `TEST_DEFECT`, `ENVIRONMENT_FAILURE`, `FLAKY_OR_NONDETERMINISTIC`, `EXPECTED_CONTRACT_CHANGE`, `TOOLING_FAILURE`, or `INSUFFICIENT_EVIDENCE`. Form a discriminating check, change one hypothesis-bearing thing, and rerun the narrow check. Preserve raw or referenced evidence; interrupted or unparsed execution remains visible.
74+
## Diagnose before repair
75+
76+
When any test, command, validator, build, tool, or verification step returns an unexpected result, stop the repair loop. Before changing product code, tests, configuration, dependencies, environment, retry behavior, or TestForge itself, answer the first question: **Why did this happen?**
77+
78+
Preserve the exact failure. Find the earliest observed divergence from expected behavior. Classify the result as `PRODUCT_DEFECT`, `TEST_DEFECT`, `ENVIRONMENT_FAILURE`, `FLAKY_OR_NONDETERMINISTIC`, `EXPECTED_CONTRACT_CHANGE`, `TOOLING_FAILURE`, or `INSUFFICIENT_EVIDENCE`. Keep plausible causes live until evidence separates them. Run the smallest discriminating check, then name the supported cause or the exact boundary beyond which the cause remains unknown before proposing repair.
79+
80+
This causal gate is proportional, not optional. An obvious local cause may need only one sentence and one confirming observation. A complex failure needs a differential. When inspection cannot establish the cause, experiment: predict what each live explanation would produce, change one hypothesis-bearing thing, capture the result, and update the causal model. A workaround that makes the symptom disappear is not a diagnosis.
81+
82+
If a repair reveals a different failure, reopen the causal model before patching again. Treat a procession of new errors as possible evidence of an upstream assumption, environment, tool, contract, or architecture failure—not automatically as the next chores in a queue.
83+
84+
Sometimes reasonable investigation leaves an irreducible transient or nondeterministic event. In that bounded case, “shit happens” may be the honest conclusion. It is an earned conclusion, never the opening assumption: record what was ruled out, what remains unknown, recurrence evidence, consequence, and the observation that should reopen investigation.
85+
86+
After the cause is established or honestly bounded, make the smallest cause-directed change and rerun the narrow check. Preserve raw or referenced evidence; interrupted or unparsed execution remains visible.
7587

7688
When execution is unavailable, deliver unexecuted tests, copy-ready commands, and the exact lost guarantee. Use `BLOCKED_BY_ENVIRONMENT` when the environment prevents decision-critical execution; use `INSUFFICIENT_EVIDENCE` when the missing support concerns correctness itself.
7789

0 commit comments

Comments
 (0)