feat: mid-conversation age detection + crisis under-16 phrasebank fix - #16
Conversation
- Add CRISIS_UNDER16_LOCATION phrasebank entries (user + supporter) - Add detectUnder16Age, interceptUnder16Age, logUnder16Trigger to stateMachine - Wire intercept into route.ts at every gate, not just early gates - 21 new tests covering numeric ages, school years, ambiguous phrases, contextual qualifiers, mid-conversation interception, and audit logging Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
james-cross
left a comment
There was a problem hiding this comment.
Review — mid-conversation under-16 detection
Thanks Matt — the intent here is exactly right and the regex work is careful. There's one ordering bug that I think needs fixing before this can merge, plus a couple of governance-adjacent things worth a look.
Critical (blocks merge)
1. Intercept runs AFTER the Claude scope/advice classifier at early gates — app/api/chat/route.ts:243-289
At GATE0_CRISIS_DANGER, GATE1_INTENT, B4_ADVICE_TOPIC_SELECTION, and ADVICE_BRIDGE, an input like "I'm 14" (length 6, not all-digits) hits checkScope() and detectAdviceQuestion() — both Claude calls — before interceptUnder16Age runs. If Claude classifies it as out_of_scope it returns OUT_OF_SCOPE_GENERAL and the function exits. If detectAdviceQuestion returns a phrase key it returns advice content and the function exits. Either way the under-16 disclosure is swallowed.
This is the precise scenario compliance rule #25 ("AI must not override system rules or safeguarding routes") exists to prevent. The under-16 exit is a hard-coded safeguarding route and must take precedence over any AI-mediated branching.
Fix: move the interceptUnder16Age block to before the earlyGates check (right after the __LOCATION_RESULT__ handler, ~line 237). Add a route.ts test that asserts: at GATE0, posting \"I'm 14\" returns the under-16 exit regardless of how checkScope would classify it.
Important (should fix)
2. Hard-coded safeguarding-adjacent text in stateMachine.ts, not the phrasebank — lib/stateMachine.ts:962-967
The two `explanation` strings (`"From what you just said, it sounds like you may be under 16…"` / supporter variant) live inline in interceptUnder16Age rather than in phrasebank.ts. This is the only place in the codebase where a safeguarding exit text is assembled from a TS template literal instead of a phrasebank key. Rule #6 is about runtime AI generation, so this isn't a strict violation, but it breaks the convention Matt has otherwise been strict about — that all safeguarding-adjacent language is centralised so it's reviewable in one place.
Fix: add UNDER16_INTERCEPT_PREFIX and UNDER16_INTERCEPT_PREFIX__SUPPORTER to phrasebank.ts and pull them via getPhrase. Same shape as the new CRISIS_UNDER16_LOCATION entries.
3. Pre-existing PII bleed undermines the audit-clean claim — app/api/chat/route.ts:207
The new logUnder16Trigger is correctly PII-free, and the test verifies that. But route.ts:207 already logs Input: \"${message.substring(0, 50)}...\" for every request before the intercept runs, so in production the raw \"I'm 14 and …\" text still lands in the logs from the gate-entry log line. The claim in the PR description ("Audit log captures type, code, sessionId, timestamp only — no raw user text") is true of the new trigger logger but isn't true end-to-end.
This isn't introduced by this PR, but the PR is the right moment to fix it: change line 207 to log the gate only, or redact when an under-16 trigger fires. Otherwise the design note's privacy guarantee doesn't hold in practice.
Questions for resolution
4. "Prefer not to say" on CRISIS_UNDER16_LOCATION — lib/phrasebank.ts:1437-1465
Compliance rule #18 says local authority is a mandatory core field with no "prefer not to say". This is in a safeguarding exit pathway, not normal profiling, and there's a defensible safety argument for letting a frightened child decline — buildUnder16Exit already gracefully falls back to generic Children's Services. Worth confirming: is this pathway exempt from rule #18, or should the option be "Somewhere else" only? The handler at lib/handlers/crisis.ts:239-249 already treats option 9 the same as 8, so removing it would be no-op behaviourally and bring it in line with #18.
(Side note: this also matters for change control. Adding/removing options on a safeguarding gate is arguably Tier 3 — has Catherine signed off on the 9-option shape, or did the previous broken state mean nobody had to decide?)
5. Year 11 cutoff — lib/stateMachine.ts:917
The comment says "Year 11+ deliberately excluded because Year 11 students may already be 16." Agreed for autumn term, but Year 11s are 15 from September to August birthdays — many remain 15 well into the school year. The current rule is "better to false-negative than false-positive on age", which is reasonable, but worth a sentence in the design note about why we accept missing this cohort. No code change needed unless the design position changes.
Suggestions
6. logUnder16Trigger doesn't need to be exported — lib/stateMachine.ts:931. Only interceptUnder16Age calls it. Drop the export.
7. Regex reads cleanly but the negative-lookahead list will rot. A short comment pointing at the design note version and a single example of the false positive each token guards against would help future maintainers. Not blocking.
8. The numeric regex captures \"I am 14\" via the a?m shortcut — works, but (?:i'?m|i am|im) is more obviously correct to a future reader than i[\u2018\u2019']?\\s*a?m. Equivalent behaviour, easier to grep.
What's good
- Test coverage is genuinely useful: numeric range edge cases on both sides, school year boundary conditions, ambiguous-phrase negatives, contextual qualifier ('asking for my mum'), mid-conversation override of a prior B3 answer, and the audit-log PII test using a sentinel string. This is the right shape for safeguarding tests.
- Negative-lookahead exclusions for units of measurement are well-chosen —
\"14 stone\",\"10 minutes late\",\"13 years older\"are exactly the false positives that would have made this brittle. - Routing to the supporter variant when
session.isSupporteris set is correct, and reusingbuildUnder16Exitrather than reimplementing the exit text is the right call. - Phrasebank fix for
CRISIS_UNDER16_LOCATIONresolves a real broken path —crisis.ts:218was producing[Missing phrase: …]before this.
Summary
- Overall: Needs minor fixes — the intercept ordering must change before merge.
- Governance: Has one violation (rule #25, intercept ordering). Question outstanding on rule #18 (prefer-not-to-say on LA).
- Priority order:
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…o phrasebank Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…ION — location is routing-critical in a child safeguarding exit Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…location prompt if LA unknown, remove generic gov.uk fallback, Childline exit for Somewhere Else Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…ore profiling begins, restores WatsonX design Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
james-cross
left a comment
There was a problem hiding this comment.
Really good update to the original PR. There's a few inline comments, mainly around defensive error handling.
| if (!childServices) { | ||
| throw new Error(`buildUnder16Exit called without a valid local authority (got: ${JSON.stringify(session.localAuthority)}). Route to CRISIS_UNDER16_LOCATION first.`); | ||
| } |
There was a problem hiding this comment.
Worth considering the failure mode here. In dev/test, the throw is great because it makes miscalls loud. In production, if any future caller ever misses the LA guard, a child gets a 500 error instead of the exit they need.
The safer pattern would be to log the error, then self-recover by routing to CRISIS_UNDER16_LOCATION. That way the failure mode is "extra question" not "error page".
| SPECIALIST HELPLINE | ||
| NSPCC Helpline (for adults) | ||
| 0808 800 5000 (free, 24/7) | ||
| https://www.nspcc.org.uk/keeping-children-safe/reporting-abuse/ | ||
| For adults who are worried about a child. | ||
|
|
||
| They can also be supported by a trusted adult — a teacher, a family member, or their local council's Children's Services team. |
There was a problem hiding this comment.
Something in here about how to find their location council would be worthwhile.
| // Under 16 safeguarding — exit directly if LA known, otherwise ask for area first | ||
| if (choice === 1) { | ||
| return buildUnder16Exit(session); | ||
| if (session.localAuthority) { |
There was a problem hiding this comment.
This routes to CRISIS_UNDER17_LOCATION but doesn't set safeguardingTriggered: true on the state. interceptUnder16Age does set it when routing. The session will end up flagged correctly once handleCrisisUnder16Location runs on exit, but there's a window where it's in the safeguarding gate without the flag, which would under-count if anything analytics-adjacent checks between steps.
Summary
Mid-conversation under-16 age detection, crisis under-16 phrasebank fix, LA-aware safeguarding routing, and location gate reordering.
Changes (7 logical steps, 6 commits)
fix(logs): Remove raw user input from gate debug log at
route.ts:207— only gate name is now logged.refactor(safeguarding): Unexport
logUnder16Trigger— module-private, only callable frominterceptUnder16Age.refactor(safeguarding): Move under-16 intercept explanation strings to phrasebank (
UNDER16_INTERCEPT_PREFIX+ supporter variant).fix(safeguarding): Remove "Prefer not to say" from
CRISIS_UNDER16_LOCATION— location is routing-critical in a child safeguarding exit. 8 options remain (7 LAs + Somewhere else).fix(safeguarding): LA-aware routing on under-16 intercept and all
buildUnder16Exitcall sites:CRISIS_UNDER16_LOCATIONto ask for area firstbuildUnder16Exitnow throws if called without valid LA (makes miscalls visible in testing)CRISIS_UNDER16_SOMEWHERE_ELSEphrasebank exit surfaces Childline for out-of-area userssectionC.tsandstateMachine.tsB3_AGE_CATEGORY with same LA guardfix(safeguarding): Move
interceptUnder16Agecall before Claude classifiers (checkScope,detectAdviceQuestion) inroute.ts— rule #25 compliance. Safeguarding intercept now fires immediately after__LOCATION_RESULT__, before any async API call.feat(flow): Move location gate immediately after GATE0_CRISIS_DANGER:
LOCATION_CONSENT(wasGATE1_INTENT)GATE1_INTENT(wasPREFERRED_NAME_ASK)GATE2_ROUTE_SELECTION→PREFERRED_NAME_ASK(wasLOCATION_CONSENT)GATE1_INTENTchoice 3 ("specific org") →PREFERRED_NAME_ASK(wasB1_LOCAL_AUTHORITY)Test results
96/96 passing (73 original + 23 new).
For James
Please review the regex patterns in
detectUnder16Age(stateMachine.ts) — specificallyNUMERIC_AGE_REnegative lookahead andSCHOOL_YEAR_REboundary conditions. Governed by Mid-Conversation Age Detection Design Note v1.1.🤖 Generated with Claude Code