Skip to content

audit: SDD dimension penalizes non-AWOS repos — documented SDD-01 gatekeeper skip not implemented #160

Description

@AlexanderMakarov

Effort: ~1–2 days for the gatekeeper fix (option 1 below); ~4–5 days if the dimension is genericized (option 2).

Profit: medium — every non-AWOS repo (i.e., nearly every client repo in an org scan) currently shows a penalized spec-driven-development dimension, so the difference is visible on almost every real audit; correctness is easy to check (the gating rule is written in the dimension doc but not implemented). The deeper value is credibility: an external tester already read the dimension as "designed to be awos (and spec-driven development) promo" (Slack thread) — in a report marketed as anchored to industry standards, a dimension whose points require installing the vendor's own framework undermines the rest of the numbers.

Problem

Two layers:

  1. Docs promise gating the engine doesn't implement. dimensions/spec-driven-development.md declares SDD-01 (AWOS installed) a gatekeeper: every other SDD check carries "Skip-When: SDD-01 is FAIL". But all seven SDD records in references/standards.toml are applies_when = "always", and the engine's applies_when interpreter only understands topology flags — check-level dependency skipping does not exist. Result: a non-AWOS repo hard-FAILs the whole dimension (FAILs count in the coverage denominator; the promised SKIPs would have been neutral).
  2. The dimension scores vendor-framework adoption. SDD-01/02/03/05/06/07 award points specifically for AWOS artifacts (.awos/, context/product/*, the AWOS spec triad, Status: fields, **[Agent: …]** annotations), source = "AWOS convention". Only SDD-04 is framework-agnostic (AWOS/Kiro/Agent-OS/specs//docs/specs/) — and it is documented as gated behind SDD-01 anyway. A repo with a disciplined DECISIONS.md / ADR practice earns zero SDD credit.

Fix — decide the dimension's posture, then implement it

  • Option 1 (minimum): implement the documented gatekeeper. When SDD-01 FAILs, SDD-02..07 become SKIP (neutral, out of the denominator), so not using AWOS is not penalized — the dimension simply reports "spec-driven workflow not adopted" via SDD-01 alone. Requires a check-dependency mechanism in the engine (or SDD-01-derived topology flag) plus standards.toml updates.
  • Option 2 (fuller): genericize the dimension — credit any spec/design-doc discipline (ADRs, DECISIONS.md, design-docs/, Kiro, Agent-OS) in the SDD-02/05/06-style checks, keeping AWOS as one recognized convention among several. SDD-04 already models this.

Either way, the report should stop reading as self-promotion on repos that never opted into AWOS.

Definition of done

  • Explicit maintainer decision (option 1 or 2) recorded in this issue.
  • Engine behavior matches the dimension doc: on a non-AWOS fixture, SDD-02..07 are SKIP (option 1) or score generic spec practice (option 2) — no hard FAILs solely for AWOS absence; engine test pins this.
  • dimensions/spec-driven-development.md, references/standards.toml, and detector logic are mutually consistent (no documented Skip-When that the engine ignores).
  • Coverage ratio on a non-AWOS repo no longer degraded by AWOS-artifact checks; verified on one real non-AWOS repo before/after.
  • dist/ rebuilt and committed; plugin version bumped.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions