feat(insights): Phase 1 — post-game LLM summaries (M0a + M1)#262
feat(insights): Phase 1 — post-game LLM summaries (M0a + M1)#262mwickett wants to merge 8 commits into
Conversation
… playerKey pseudonymization)
Single non-streaming Messages call on claude-opus-4-8 (adaptive thinking, low effort, no sampling params). Throws on refusal/empty output; sums all billed token fields. Document ANTHROPIC_API_KEY + CRON_SECRET in .env.example.
Durable per-game recap row (status machine + retryCount + sourceStatsHash) with unique (game_id, prompt_version). Hand-authored migration verified identical to Prisma's canonical DDL via migrate diff --from-empty.
buildGameRecap builds deterministic facts in playerKey space (winner at rank 1 even on tiebreaks); recapHash stable-hashes them; pseudonymizeRecap projects to an LLM-facing payload with no real names or internal ids; grounding flags ungrounded numbers. 14 tests.
Split enqueue (sync, primary, writes durable pending row) from run (async LLM call), scheduled via after() and flag-gated on llm-features. Triggered from updateGameAsFinished AND syncGameCompletionAfterScoreWrite so edits to finished games regenerate (hash-gated). Sweep for retries. 9 tests.
getGameSummary reads the primary (avoids replica lag on a just-finished game). GameSummaryCard renders ready/pending/insufficient states; the page shows it flag-gated when finished. /api/insights/sweep retries stuck rows, guarded by CRON_SECRET. 4 component tests.
- Race-safe terminal writes: runGameSummary persists ready/failed/insufficient via a hash-conditional updateMany, so a stale in-flight run can't overwrite newer work (Codex BLOCKER). - Deterministic recap hash: build calcGameStats input in sorted playerKey order so player/score DB ordering can't change biggestRound/worstRound. - Reset retryCount when the source hash changes, so post-edit work is sweepable. - Pseudonymizer never falls back to a raw playerKey. - Render the recap card only when a row exists (no fake 'pending' for games finished while the flag was off). 2 new tests (172 total).
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d57d678661
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // (audienceUserId is null in MVP — neutral recap). The row is also the durable | ||
| // job unit: status + retryCount drive enqueue/run/sweep. sourceStatsHash is | ||
| // stored (not in the unique key) so score edits after finish are detected. | ||
| model GameSummary { |
There was a problem hiding this comment.
This repo checks in the custom Prisma client under src/generated/prisma, but this commit adds GameSummary without updating those generated files (rg gameSummary src/generated/prisma returns nothing). As a result, a checkout of this commit has no prisma.gameSummary delegate and npm run typecheck -- --pretty false fails with TS2339 at every new summary query, so the app cannot typecheck until the client is regenerated/committed or generation is added before typecheck/build.
Useful? React with 👍 / 👎.
| // Flag-gated entry point used by the game-finish paths. Writes the durable | ||
| // pending row synchronously, then runs the LLM after the response is sent. | ||
| export async function scheduleGameSummary(gameId: string): Promise<void> { | ||
| if (!(await isLlmFeaturesEnabled())) return; |
There was a problem hiding this comment.
Because this flag check is evaluated for the user who happened to finish or edit the game, a game-level summary is never enqueued when that editor lacks llm-features. The game page separately checks the viewing user's flag and renders only the stored getGameSummary row, so during a per-user rollout a flagged teammate can open the finished game and see no recap, or continue seeing a stale recap after an unflagged member edits finished scores; enqueueing needs to be based on a game/org/global condition or refresh existing summaries regardless of the editor's flag.
Useful? React with 👍 / 👎.
…scheduling - failed run-writes are now conditional on status != ready, so a concurrent or late failure can't knock a succeeded summary back to failed. - enqueue no longer re-schedules when the row already matches the hash, preventing duplicate runs/cost on repeated same-source triggers (the sweep still retries non-ready rows). - scheduleGameSummary is best-effort on the finish/edit paths (try/catch) so a summary failure can never fail the game finish or score edit. 3 new tests (174).
Codex review trail (Phase 1)Five Codex passes, all findings addressed:
Status: The two final fixes ( |
1 participant
LLM Insights — Phase 1: post-game summaries (M0a + M1)
First slice of the LLM Insights feature (design spec:
docs/superpowers/specs/2026-06-13-llm-insights-design.md). After a Dutch Blitz game finishes, Claude writes a short, warm, neutral recap grounded in the real scores. Ships dark behind the existingllm-featuresflag.This is the integration base; later phases (chat, metric-plan DSL, dashboard tiles, RLS substrate) land as their own PRs into
feature/llm-insights.What's in it
M0a — Claude foundation
@anthropic-ai/sdk+ a singleton client;generateSummaryText(Opus 4.8, adaptive thinking, low effort, no sampling params; throws on refusal/empty; sums all billed tokens).GameSummarymodel + migration (hand-authored, verified identical to Prisma's canonical DDL). The row doubles as the durable job unit (status machine +retryCount+sourceStatsHash).M1 — Post-game summary
buildGameRecap→recapHash→pseudonymize→summaryPrompt→grounding. Built entirely in playerKey space: real names and internal IDs never reach the model; names are re-substituted server-side after generation.after(); a/api/insights/sweepcron route (CRON_SECRET-guarded) retries anything stuck. Terminal writes are hash-conditional so a stale in-flight run can't clobber newer work.syncGameCompletionAfterScoreWrite) — hash-gated so unchanged scores are a no-op.Review
Two Codex review rounds (plan + implementation), all findings addressed:
after()), the finished-game-edit trigger, playerKey-keyed pseudonymization,migrate dev→ hand-authored migration,satisfiesoveras, refusal/empty handling,sourceUpdatedAtfrom source data.Verification
npm test→ 28 suites / 172 tests pass ·npm run typecheckclean ·npm run lintclean.🔴 Needs a human before this does anything in prod
ANTHROPIC_API_KEYin the deployment env (+ local.envfor manual E2E). Documented in.env.example.prisma migrate deploy(build script). For local testing, apply it against a dev DB.llm-featuresPostHog flag for the intended users — both generation and rendering are gated on it.CRON_SECRETand add a Vercel cron hittingGET /api/insights/sweep(e.g. every 10 min) for retry durability.Known limitations (by design, for MVP)
breakTie(core scoring) keeps the first DB-order candidate. Impact is only a rare redundant regeneration — within any render the recap agrees with the scoreboard. The underlying core-scoring non-determinism is tracked as a separate follow-up (out of scope here per repo convention: don't change scoring logic in an insights PR).