docs: ADR-030 agenda storage model, mutability, and summaries (MON-209) - #298
Conversation
Settles the four decisions MON-210 was blocked on: - one denormalized agenda_items table, not an extension of votes, with the exact column set named so the migration needs no further calls - soft-state refresh (upsert + last_seen_at, never DELETE), which keeps the upsert-only rule intact rather than carving an exception for a mutable source - seance publique only; commissions stay out despite better dossier coverage, because committee hearings are a different product - summaries only where there is something to summarize: stub objets get no LLM call, the UI renders the point type instead Grounded in the MON-208 measurements (24-day median lead time, 27% cancellation rate, 31% dossier coverage, 16% stub objets).
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🧪 dbt data-health check — ✅ passedValidates prod data health. A failure means current prod data is stale/broken, not necessarily that this PR is wrong. |
Walid-peach
left a comment
There was a problem hiding this comment.
Attention Score
100/100 - SAFE TO SKIM
- Base 100
- no Must Fix, no Should Fix findings
- no high-risk paths (
docs/only, 82 added lines, 1 file) tests/penalty does not apply: noapi/,rag/, orscripts/change- Testing / Validation Gaps: none applicable to a decision record
Reason you can skim: additive documentation only, every quantitative claim re-verified against the MON-208 data, and the two load-bearing choices (séance-only scope, soft-state refresh) were made by the maintainer rather than assumed here.
Summary
Adds ADR-030, settling the four questions that blocked MON-210: table shape, refresh semantics for a mutable source, ingestion scope, and summary policy. Also adds rule 14 to the standing rules list. Purely additive - 82 lines, no existing text modified, no code or schema.
The substantive contribution is that it resolves an apparent contradiction rather than papering over it. The agenda is the first mutable source in the pipeline, which looked like it required an exception to the upsert-only / no-DELETE rule in CLAUDE.md. The ADR shows the rule holds given a last_seen_at freshness column, and says so explicitly, so a future reader does not find two conflicting rules.
Must Fix
None.
Should Fix
None.
Nice to Have
- The ADR names
procedure_labelrather thanprocedurefor the column mapping the feed'sprocedurefield. The rename avoids a Postgres non-reserved keyword; that reasoning is implicit in the column comment and could be stated outright so MON-210 does not "fix" it back.
Testing / Validation Gaps
None applicable - a decision record has nothing executable. What was verified rather than asserted:
point_uidpresent and unique on 3 644 / 3 644 séance ODJ points, and réunion uids unique on 7 469 / 7 469, so the proposed primary key is implementable.- Every column in the
CREATE TABLEmaps to a field confirmed present in the export byscripts/explore_agenda.py(#294). - All quantitative claims recomputed from the source data: 69% without a dossier reference, 16% stub
objet, 27% of séancesSupprimé, 72% commission dossier coverage against 31% for séances,procedureon 5.1% of points.
Documentation / Reviewer Notes
- The claim worth a second opinion is decision 3, séance-only. It is the one place the ADR knowingly leaves better data on the table: commission réunions have more than double the dossier coverage and are the only forward-looking content that exists year-round, which matters because the page will be empty until the session returns in late September. It is a product judgment, not a technical one, and the ADR records a cheap reversal path (
--typeargument plus a backfill). - No CLAUDE.md change in this PR by design: its table inventory gains
agenda_itemswhen MON-210 creates it, not when an ADR describes it. - Prose avoids em dashes per project style; the
ADR-030 —heading keeps one deliberately so it matches the format of all 29 preceding ADR headings.
Verdict
Ready to merge. MON-210 is unblocked on merge and has no schema decisions left to make.
Settles the four architectural questions blocking MON-210, grounded in the MON-208 measurements rather than guesswork. Docs only: one new ADR, no schema, no code.
The decisions
1. One denormalized
agenda_itemstable, not an extension ofvotes. The ADR names the exact column set, so MON-210 writes the migration with no remaining calls to make.point_uidis the primary key - verified present and unique across all 3 644 séance ODJ points, so no surrogate (MON-77 removed exactly such a surrogate elsewhere).votesis keyed on the scrutin uid and only exists once a vote happened; an agenda item exists before, and often without, a vote - 69% carry no dossier reference at all. Merging them would mean nullable-everything rows that break the assumption that avotesrow means a vote occurred, and would corruptmart_vote_summaryand the RAG vote chunks downstream.2. Soft-state refresh: upsert on
point_uid, stamplast_seen_atevery run, never DELETE. An item is visible only if it was seen in the latest completed run and is not cancelled.This is the part that matters most. The agenda is the first mutable source in the pipeline (27% of séances end
Supprimé), and the obvious reading was that CLAUDE.md decision 8 - "upsert-only, no DELETE" - needed an exception. It doesn't.etatcovers cancellations the Assemblée announces;last_seen_atcovers items that quietly vanish from the export, which is the churn case MON-208 could not measure. Together they make the unmeasured risk structurally harmless instead of leaving an open question the page would eventually expose. The rule is unchanged and now applies here too.3. Séance publique only. Commissions stay out despite better dossier coverage (72% vs 31%) and being the only forward content year-round. "What your committee is hearing this week" is a different product from "what gets voted this week". The parser is type-agnostic, so adding them later is a
--typeargument and a backfill, not a redesign.4. Dossier linking binds to the earliest scrutin on or after the sitting date. Latest would link a 2024 agenda item to a 2026 vote on the same bill; earliest overall would link forward to a vote that already happened.
5. Summaries only where there is something to summarize. Stub
objetvalues (16%, entire contentDiscussionorQuestions au Gouvernement) get no LLM call; the UI renderspoint_type. Sending one word to an LLM would produce invented specifics on a civic-transparency site.Acceptance criteria
CREATE TABLEwith per-column commentsdocs/decisions.mdonlyDownstream effects
009_agenda.sql.etat.point_typewhereversummary_plainis NULL, never a placeholder.Test plan
Docs only - nothing executable changed. Verified the
CREATE TABLEis implementable:point_uidpresent and unique on 3 644/3 644 points, réunion uids unique on 7 469/7 469, and every named column maps to a field confirmed present in the export byscripts/explore_agenda.py(merged in #294).Notes
Prose avoids em dashes per project style; the
ADR-030 —heading keeps one deliberately, matching the format of all 29 preceding ADR headings.No CLAUDE.md change yet: its table list gains
agenda_itemswhen MON-210 actually creates it, not when the ADR describes it.