Problem
packages/explicabl-core/src/reporting/saveReport.ts unconditionally writes all five conformance categories (pkce, jwt_verify, scope_enforcement, allowlist, expiry) as "pass" with a fresh timestamp — no test is consulted. The committed docs/conformance.json reflects this, and the README's "MCP/Auth Conformance" badge reads that file's version.
During a repo-visibility push this is the single highest-risk artifact: a skeptical evaluator finds saveReport.ts in one file-read, sees the hardcoded passes, and that discredits every other claim in the repo. There is currently zero mapping from any category to any test, and the root tests/smoke.test.ts is a 7-line "runner runs" assertion.
Fix (pick one, do before the star push)
- Preferred: replace
saveReport.ts with a script that derives each category's pass/fail from actual test results (one describe-block per category, real assertions), so the badge means something.
- Or: remove the badge from the README and delete
docs/conformance.json + saveReport.ts until real conformance tests exist. An absent badge is honest; a hardcoded one is not.
Acceptance
- The conformance JSON is produced only from executed tests, OR the badge/file are removed.
- No category can report
pass without a corresponding assertion having run.
Problem
packages/explicabl-core/src/reporting/saveReport.tsunconditionally writes all five conformance categories (pkce,jwt_verify,scope_enforcement,allowlist,expiry) as"pass"with a fresh timestamp — no test is consulted. The committeddocs/conformance.jsonreflects this, and the README's "MCP/Auth Conformance" badge reads that file'sversion.During a repo-visibility push this is the single highest-risk artifact: a skeptical evaluator finds
saveReport.tsin one file-read, sees the hardcoded passes, and that discredits every other claim in the repo. There is currently zero mapping from any category to any test, and the roottests/smoke.test.tsis a 7-line "runner runs" assertion.Fix (pick one, do before the star push)
saveReport.tswith a script that derives each category's pass/fail from actual test results (one describe-block per category, real assertions), so the badge means something.docs/conformance.json+saveReport.tsuntil real conformance tests exist. An absent badge is honest; a hardcoded one is not.Acceptance
passwithout a corresponding assertion having run.