diff --git a/docs/problems/README-history.md b/docs/problems/README-history.md index 2e2a02e7..01754e2b 100644 --- a/docs/problems/README-history.md +++ b/docs/problems/README-history.md @@ -1084,3 +1084,5 @@ Last reviewed: 2026-04-28 **AFK iter 7 — P139 transitioned Open → Verificati > Last reviewed: 2026-06-28 **P394 captured** — release publishes a new version of every package, not just the changed ones (lightweight aside via /wr-itil:capture-problem). > Last reviewed: 2026-06-28 **batch transition** — P373 verifying, P383 verifying (post-release K→V after the cohort graduation shipped @windyroad/itil@0.54.6 + jtbd; manual transition per P389 — the graduated changesets carried no Release-vehicle seed so the enumerator skipped them). + +> Last reviewed: 2026-06-28 **P395 captured** — external-comms agent silently dormant on the credibility axis; no nudge to author the missing RISK-POLICY section (lightweight aside via /wr-itil:capture-problem). diff --git a/docs/problems/README.md b/docs/problems/README.md index 267249d2..e1ed1a75 100644 --- a/docs/problems/README.md +++ b/docs/problems/README.md @@ -1,6 +1,6 @@ # Problem Backlog -> Last reviewed: 2026-06-28 **P395 captured** — external-comms agent silently dormant on the credibility axis; no nudge to author the missing RISK-POLICY section (lightweight aside via /wr-itil:capture-problem). +> Last reviewed: 2026-06-28 **P396 + P397 captured** — verification-queue drain has no self-firing cadence so the verifying queue bloats unbounded (188) until a human asks (P396) + the silent evidence-based close auto-closes external reporters' issues against the confirmation/14-day promise we post (P397) (lightweight aside via /wr-itil:capture-problem). > Run `/wr-itil:review-problems` to refresh WSJF rankings. @@ -11,6 +11,8 @@ Dev-work queue only. Verification Pending (`.verifying.md`, WSJF multiplier 0) a | WSJF | ID | Title | Severity | Status | Effort | Reported | Origin | |------|-----|-------|----------|--------|--------|----------|--------| | 1.5 | P395 | external-comms agent silently goes dormant on the Outbound Credibility / Self-Own axis when RISK-POLICY.md lacks the `## Outbound Credibility / Self-Own` section — no nudge to author it, so the shipped capability stays invisible (P375 rot-class). Blocks P384's eval-floor. Fix: extend risk-scorer-scaffold-nudge.sh to detect the missing-section predicate. | 3 Medium | Open | M | 2026-06-28 | internal | +| 1.5 | P396 | Verification-queue drain (verifying → closed) has no self-firing cadence — runs only on manual `/wr-itil:review-problems` / `run-retro` / `manage-problem review`, or the narrow work-problems Step 0c AND-gate. The verifying queue bloats unbounded (**188** on 2026-06-28) until a human asks, then drains in one large batch. P375 cadence-rollup class; sibling P295. Fix: a SessionStart nudge / tiered auto-drain that surfaces the queue. | 3 Medium | Open | M | 2026-06-28 | internal | +| 1.5 | P397 | Silent evidence-based verifying-close auto-closes external reporters' GitHub issues (`gh issue close` via update-upstream / the P363 inbound leg) against the "we'll close after **your** confirmation OR a 14-day quiet period" promise we post — it closes on **our** in-session evidence, reporter-independent, often within days. ADR-044 evidence=verified is fine for internal tickets, wrong across the external trust boundary. Fix: don't `gh issue close` inbound/upstream issues until reporter-confirmed or the quiet period elapses. | 3 Medium | Open | M | 2026-06-28 | internal | | 1.5 | P394 | Release publishes a new version of EVERY @windyroad/* package, not just the changed ones — likely `updateInternalDependencies: patch` cascading bumps across the 13-package internal dependency graph; floods the adopter version stream + dilutes the change audit-trail. | 3 Medium | Open | M | 2026-06-28 | internal | | 9.0 | P384 | wr-risk-scorer:external-comms agent has no credibility / self-own review axis — a leak-clean message can still ask for already-held info, restate what the recipient told us as new, or carry a careless error (wrong name/company, stale account claim). Agent-prose axis added (RFC-032): `asks-for-already-held-info` / `restates-prior-as-new` / `plainly-careless-error` composing with the existing confidential-leak axis; home-repo RISK-POLICY section + paired promptfoo eval deferred. **Tier 1: Inbound-reported** (#283). | 9 Medium | Known Error | M | 2026-06-26 | inbound-reported (#283) | | 6.0 | P305 | Post-Edit silent revert of working-tree files before commit — root cause confirmed 2026-06-08 as parallel `claude -p` iter dispatch race on shared working tree (P228-fix + P213-fix iters overlapped 13:58 → 14:41 on 2026-06-08; 14:28 mystery revert sits inside the parallel window; Edit tool's read-modify-write is not atomic across processes — last-writer-wins clobbers sibling iter's edits). Workaround: re-apply edit + immediate `git add`. Fix strategy: three candidate options (A serialize iter dispatch / B per-iter `git worktree` / C post-Edit verification helper) queued substance-confirm to outstanding_questions per ADR-074 — DO NOT build before user ratifies | 6 Med | Known Error | M | 2026-05-26 | internal | diff --git a/docs/problems/open/396-verification-queue-drain-has-no-self-firing-cadence-bloats-to-188.md b/docs/problems/open/396-verification-queue-drain-has-no-self-firing-cadence-bloats-to-188.md new file mode 100644 index 00000000..f42b8595 --- /dev/null +++ b/docs/problems/open/396-verification-queue-drain-has-no-self-firing-cadence-bloats-to-188.md @@ -0,0 +1,57 @@ +# Problem 396: Verification-queue drain has no self-firing cadence — the verifying queue bloats unbounded (188) until a human asks + +**Status**: Open +**Reported**: 2026-06-28 +**Priority**: 3 (Medium) — Impact: 3 x Likelihood: 1 (deferred — re-rate at next /wr-itil:review-problems) +**Origin**: internal +**Effort**: M (deferred — re-rate at next /wr-itil:review-problems) +**JTBD**: JTBD-001 +**Persona**: plugin-developer + +## Description + +The verification-queue drain (Verification Pending → Closed) has no self-firing cadence. It runs ONLY when a human invokes `/wr-retrospective:run-retro`, `/wr-itil:review-problems`, or `/wr-itil:manage-problem review`, OR when the AFK `/wr-itil:work-problems` orchestrator Step 0c pre-flight promotes it — but only under a narrow AND gate (deferred-placeholder count ≥ 3 AND `docs/problems/README.md` cadence age > 7 days). There is no session-start nudge and no scheduled trigger that surfaces the verifying queue on its own. + +Consequence: in a normal session nothing drains the queue, so it accumulates without bound until a human remembers to ask, then closes in one large batch. This is the same class the `feedback_automatic_cadence_or_it_doesnt_happen` memory names — a governance action with no automatic cadence never happens; the human ends up holding the memory — and the same shape as P375 (a deferral that names a re-entry point is NOT a self-firing cadence). + +## Symptoms + +- The `docs/problems/verifying/` queue is currently **188 tickets** in this repo — fixes released, sitting un-closed because no surface drains them automatically. +- Witnessed batch drains close 30+ tickets at once (a separate session reported 47 → 17, closing 30), because the queue had been allowed to grow between manual invocations. +- A user has to explicitly ask to "examine the queue of problems in the verifying state" — the system never surfaces it proactively. + +## Workaround + +Manually invoke `/wr-itil:review-problems` or `/wr-retrospective:run-retro` periodically. Relies entirely on the human remembering — which is the defect. + +## Impact Assessment + +- **Who is affected**: plugin-developer / maintainer running the backlog; secondarily reporters whose fixed tickets stay open-in-tracker because the close (and the linked `gh issue close`) never fires until a drain runs. +- **Frequency**: continuous — every session that doesn't manually drain lets the queue grow. +- **Severity**: backlog truthfulness erodes (188 stale "verifying" entries); batch-drains become large and lossy-feeling; couples to P396's sibling defect where the eventual drain closes external reporters' issues automatically (see Related). +- **Analytics**: `ls docs/problems/verifying/ | wc -l` trend over time; gap between fix-release commits and the corresponding Verifying→Closed transition. + +## Root Cause Analysis + +### Investigation Tasks + +- [ ] Re-rate Priority and Effort at next /wr-itil:review-problems +- [ ] Investigate root cause — confirm there is no SessionStart hook / scheduled trigger that surfaces the verifying queue; map the three on-demand entry points + the work-problems Step 0c AND-gate +- [ ] Decide the cadence surface — e.g. a SessionStart nudge that reports verifying-queue size + age (mirroring the architect/jtbd oversight nudges), or a tiered auto-drain trigger, per the P375 cadence-rollup design +- [ ] Create reproduction test (queue size grows across N sessions with no drain unless manually invoked) + +## Dependencies + +- **Blocks**: backlog truthfulness (verifying tickets reflect real un-verified state, not drain-cadence lag) +- **Blocked by**: (none) +- **Composes with**: P375 (named-re-entry vs self-firing cadence — rollup parent), P295 (ADR-043 deep-context-analysis needs automatic cadence not on-demand-only — closest cadence sibling) + +## Related + +Captured via /wr-itil:capture-problem. Hang-off-check skipped — candidate-cap short-circuit (sub-step 2b): the mechanical pre-filter on shared skill-refs (`/wr-retrospective:run-retro`, `/wr-itil:review-problems`, "verification queue") matched 158 open/verifying candidates (> 5 cap), so subagent dispatch was skipped per the SKILL contract; re-evaluate absorption at next /wr-itil:review-problems. + +- **P375** (`docs/problems/open/375-repo-conflates-named-re-entry-point-with-self-firing-cadence.md`) — the cadence-rollup parent. This is a textbook instance of its "capability exists but nothing self-fires it" class; tracked as a rollup child, not absorbed (consistent with how P375's other instances are handled). +- **P295** (`docs/problems/verifying/295-adr-043-deep-context-analysis-needs-automatic-cadence-not-on-demand-only.md`) — closest sibling: the same on-demand-only-rot defect on the context-analysis surface. The verifying-queue drain is the same defect on the verification-close surface. +- **P253** — no house-cleaning cadence for cruft/deprecation removal — same cadence-gap family. +- **Sibling defect (capture pending this session)**: the silent evidence-based close auto-closes external reporters' GitHub issues against the "we'll close after your confirmation or 14-day quiet period" promise — the *close* side of the same broken cadence (the *trigger* side is this ticket). +- **Witness**: `docs/problems/verifying/` = 188 tickets on 2026-06-28; surfaced when the user asked why issues were being auto-closed and why they had to ask to examine the verifying queue. diff --git a/docs/problems/open/397-evidence-close-auto-closes-reporter-issues-against-stated-promise.md b/docs/problems/open/397-evidence-close-auto-closes-reporter-issues-against-stated-promise.md new file mode 100644 index 00000000..dbde332a --- /dev/null +++ b/docs/problems/open/397-evidence-close-auto-closes-reporter-issues-against-stated-promise.md @@ -0,0 +1,60 @@ +# Problem 397: Silent evidence-based close auto-closes external reporters' GitHub issues against the confirmation/14-day promise we post + +**Status**: Open +**Reported**: 2026-06-28 +**Priority**: 3 (Medium) — Impact: 3 x Likelihood: 1 (deferred — re-rate at next /wr-itil:review-problems) +**Origin**: internal +**Effort**: M (deferred — re-rate at next /wr-itil:review-problems) +**JTBD**: JTBD-301 +**Persona**: plugin-user + +## Description + +When a Verification Pending ("verifying") ticket is closed, two things happen that compose into a trust-boundary defect: + +1. **The close is silent and evidence-based, not reporter-confirmed.** `/wr-retrospective:run-retro` Step 4a closes verifying tickets on in-session evidence (test invocations, dependent commits, hook firings) as a "silent agent action per P135 / ADR-044" — no per-ticket `AskUserQuestion`. The framework has resolved "in-session evidence = verified." +2. **The close propagates to the external GitHub issue.** `/wr-itil:update-upstream` runs `gh issue close` on the Verifying → Closed transition for both reported-upstream tickets (Step 5b) and inbound-reported `**Origin**: inbound-reported (#NN)` tickets (the P363 inbound leg). + +The contradiction: the fix-released comment we post to reporters (the Known Error → Verification Pending template) explicitly promises *"We'll close this issue after **your** confirmation OR after a 14-day quiet period."* But the drain closes on **our** in-session evidence — reporter-independent, often within days. So a reporter watches their GitHub issue get auto-closed before they confirmed and before 14 days elapsed, directly contradicting the commitment in our own comment. + +ADR-044's "evidence = verified" framework-resolution is defensible for *internal* tickets (the maintainer is both author and verifier). It crosses a different trust boundary when the close auto-propagates to an *external* party's issue and overrides a promise we made them. The reporter is not the one who verified, and the quiet-period we cited never ran. + +## Symptoms + +- A reporter files an issue; we post "we'll close after your confirmation or 14 days"; days later the issue is auto-closed by an evidence-based drain with a "closed after verification" comment the reporter never triggered. +- The user's observed alarm: "Why are issues being automatically closed?" — reporters' issues closing without their confirmation. + +## Workaround + +Exclude inbound-reported / reported-upstream tickets from the silent evidence-close pass by hand (close them only after the reporter confirms or the 14-day quiet period genuinely elapses). No mechanism enforces this today. + +## Impact Assessment + +- **Who is affected**: plugin-user (external reporters) whose issues are closed before they confirm; secondarily the maintainer's credibility on a trust-ramp product. +- **Frequency**: every inbound-reported / reported-upstream verifying ticket caught by an evidence-based drain before reporter confirmation or the 14-day window. +- **Severity**: breaks an explicit promise made to an external party (JTBD-301 reporter feedback loop); reputational, distinct from a functional break. Compounded by P396 (the drain runs in large delayed batches, so many issues close at once). +- **Analytics**: diff `gh issue close` events on inbound/upstream issues against (a) whether a reporter-confirmation comment exists and (b) whether 14 days elapsed since the fix-released comment. + +## Root Cause Analysis + +### Investigation Tasks + +- [ ] Re-rate Priority and Effort at next /wr-itil:review-problems +- [ ] Investigate root cause — confirm run-retro Step 4a close-on-evidence reaches the `gh issue close` leg for inbound/upstream tickets without a reporter-confirmation or elapsed-quiet-period gate +- [ ] Decide the fix — e.g. evidence-close may transition the LOCAL ticket but MUST NOT `gh issue close` an external reporter's issue until reporter-confirmed OR the 14-day quiet period actually elapses; OR change the promise wording to match the evidence-close behaviour (less good — the promise is the right one) +- [ ] Create reproduction test (inbound verifying ticket + in-session evidence → assert the external issue is NOT auto-closed before confirmation/quiet-period) + +## Dependencies + +- **Blocks**: trustworthy reporter feedback loop (JTBD-301 — reporters' issues reflect their own verification, not our internal evidence) +- **Blocked by**: (none) +- **Composes with**: P363 (inbound fix-released verdict + `gh issue close` leg — the close mechanism), P396 (the drain's missing cadence — the trigger side of the same broken loop) + +## Related + +Captured via /wr-itil:capture-problem. Hang-off-check skipped — candidate-cap short-circuit (sub-step 2b): mechanical pre-filter on shared signals (P363 / `/wr-itil:update-upstream` / verifying-close / ADR-044) matched > 5 open/verifying candidates, so subagent dispatch was skipped per the SKILL contract; re-evaluate absorption at next /wr-itil:review-problems. Nearest parent is P363 (it owns the inbound `gh issue close` leg) but this defect is the close-policy trust boundary, not P363's leg-correctness — surfaced for review-time absorption decision. + +- **P363** (`docs/problems/known-error/363-inbound-reported-tickets-never-receive-fix-released-verdict-on-originating-issue.md`) — owns the inbound fix-released verdict + `gh issue close` machinery. This ticket is the *should-we-auto-close-at-all* policy question sitting on top of P363's *how-to-close* mechanism. +- **P396** (`docs/problems/open/396-verification-queue-drain-has-no-self-firing-cadence-bloats-to-188.md`) — the trigger side: the drain has no cadence, so it fires late and in big batches. This ticket is the close side: when it does fire, it closes external issues against the stated promise. +- **P048** (`docs/problems/verifying/048-manage-problem-does-not-detect-verification-candidates.md`) — owns the 14-day quiet-period default that this defect's promise cites. +- **Witness**: surfaced 2026-06-28 when the user asked "Why are issues being automatically closed?" after a reporter saw issue auto-closure. The fix-released comments posted this session on #274 / #284 / #282 each carry the "your confirmation OR 14-day quiet period" promise the evidence-close would override.