Skip to content

docs: ADR-030 agenda storage model, mutability, and summaries (MON-209) - #298

Merged
Walid-peach merged 1 commit into
masterfrom
feature/mon-209-adr-agenda-storage-model-mutability-and-summary-generation
Aug 2, 2026
Merged

docs: ADR-030 agenda storage model, mutability, and summaries (MON-209)#298
Walid-peach merged 1 commit into
masterfrom
feature/mon-209-adr-agenda-storage-model-mutability-and-summary-generation

Conversation

@Walid-peach

Copy link
Copy Markdown
Owner

Settles the four architectural questions blocking MON-210, grounded in the MON-208 measurements rather than guesswork. Docs only: one new ADR, no schema, no code.

The decisions

1. One denormalized agenda_items table, not an extension of votes. The ADR names the exact column set, so MON-210 writes the migration with no remaining calls to make. point_uid is the primary key - verified present and unique across all 3 644 séance ODJ points, so no surrogate (MON-77 removed exactly such a surrogate elsewhere).

votes is keyed on the scrutin uid and only exists once a vote happened; an agenda item exists before, and often without, a vote - 69% carry no dossier reference at all. Merging them would mean nullable-everything rows that break the assumption that a votes row means a vote occurred, and would corrupt mart_vote_summary and the RAG vote chunks downstream.

2. Soft-state refresh: upsert on point_uid, stamp last_seen_at every run, never DELETE. An item is visible only if it was seen in the latest completed run and is not cancelled.

This is the part that matters most. The agenda is the first mutable source in the pipeline (27% of séances end Supprimé), and the obvious reading was that CLAUDE.md decision 8 - "upsert-only, no DELETE" - needed an exception. It doesn't. etat covers cancellations the Assemblée announces; last_seen_at covers items that quietly vanish from the export, which is the churn case MON-208 could not measure. Together they make the unmeasured risk structurally harmless instead of leaving an open question the page would eventually expose. The rule is unchanged and now applies here too.

3. Séance publique only. Commissions stay out despite better dossier coverage (72% vs 31%) and being the only forward content year-round. "What your committee is hearing this week" is a different product from "what gets voted this week". The parser is type-agnostic, so adding them later is a --type argument and a backfill, not a redesign.

4. Dossier linking binds to the earliest scrutin on or after the sitting date. Latest would link a 2024 agenda item to a 2026 vote on the same bill; earliest overall would link forward to a vote that already happened.

5. Summaries only where there is something to summarize. Stub objet values (16%, entire content Discussion or Questions au Gouvernement) get no LLM call; the UI renders point_type. Sending one word to an LLM would produce invented specifics on a civic-transparency site.

Acceptance criteria

Criterion Where
ADR covers table choice, mutability, séance-vs-commission, linking, summaries ADR-030, all five numbered
States its relationship to the upsert-only decision "Impact" - unchanged, no exception created
Names the concrete table and column set for MON-210 Full CREATE TABLE with per-column comments
No schema or code change lands here Diff is docs/decisions.md only

Downstream effects

  • MON-210 implements the named table verbatim in 009_agenda.sql.
  • MON-212 must filter on freshness and state: an item not seen in the last run is invisible regardless of etat.
  • MON-213 renders point_type wherever summary_plain is NULL, never a placeholder.
  • Rule 14 added to "Rules for future development sessions".

Test plan

Docs only - nothing executable changed. Verified the CREATE TABLE is implementable: point_uid present and unique on 3 644/3 644 points, réunion uids unique on 7 469/7 469, and every named column maps to a field confirmed present in the export by scripts/explore_agenda.py (merged in #294).

Notes

Prose avoids em dashes per project style; the ADR-030 — heading keeps one deliberately, matching the format of all 29 preceding ADR headings.

No CLAUDE.md change yet: its table list gains agenda_items when MON-210 actually creates it, not when the ADR describes it.

Settles the four decisions MON-210 was blocked on:

- one denormalized agenda_items table, not an extension of votes, with
  the exact column set named so the migration needs no further calls
- soft-state refresh (upsert + last_seen_at, never DELETE), which keeps
  the upsert-only rule intact rather than carving an exception for a
  mutable source
- seance publique only; commissions stay out despite better dossier
  coverage, because committee hearings are a different product
- summaries only where there is something to summarize: stub objets get
  no LLM call, the UI renders the point type instead

Grounded in the MON-208 measurements (24-day median lead time, 27%
cancellation rate, 31% dossier coverage, 16% stub objets).
@vercel

vercel Bot commented Aug 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
mon-elu Ready Ready Preview Aug 2, 2026 12:20am

@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown

🧪 dbt data-health check — ✅ passed

00:18:58  52 of 62 PASS source_not_null_raw_votes_vote_id ................................ [PASS in 0.87s]
00:18:58  56 of 62 START test source_unique_raw_deputies_deputy_id ....................... [RUN]
00:18:59  53 of 62 PASS source_not_null_raw_votes_voted_at ............................... [PASS in 0.90s]
00:18:59  57 of 62 START test source_unique_raw_votes_vote_id ............................ [RUN]
00:18:59  54 of 62 PASS source_relationships_raw_vote_positions_deputy_id__deputy_id__source_raw_deputies_  [PASS in 1.03s]
00:18:59  58 of 62 START test unique_mart_deputy_scorecard_deputy_id ..................... [RUN]
00:18:59  55 of 62 PASS source_relationships_raw_vote_positions_vote_id__vote_id__source_raw_votes_  [PASS in 1.08s]
00:18:59  59 of 62 START test unique_mart_party_alignment_deputy_id ...................... [RUN]
00:18:59  56 of 62 PASS source_unique_raw_deputies_deputy_id ............................. [PASS in 1.00s]
00:18:59  60 of 62 START test unique_mart_vote_summary_vote_id ........................... [RUN]
00:19:00  57 of 62 PASS source_unique_raw_votes_vote_id .................................. [PASS in 0.89s]
00:19:00  61 of 62 START test unique_stg_deputies_deputy_id .............................. [RUN]
00:19:00  58 of 62 PASS unique_mart_deputy_scorecard_deputy_id ........................... [PASS in 0.92s]
00:19:00  62 of 62 START test unique_stg_votes_vote_id ................................... [RUN]
00:19:00  59 of 62 PASS unique_mart_party_alignment_deputy_id ............................ [PASS in 0.86s]
00:19:00  60 of 62 PASS unique_mart_vote_summary_vote_id ................................. [PASS in 0.87s]
00:19:00  61 of 62 PASS unique_stg_deputies_deputy_id .................................... [PASS in 0.87s]
00:19:01  62 of 62 PASS unique_stg_votes_vote_id ......................................... [PASS in 0.89s]
00:19:02  
00:19:02  Finished running 62 data tests in 0 hours 0 minutes and 21.19 seconds (21.19s).
00:19:02  
00:19:02  Completed successfully
00:19:02  
00:19:02  Done. PASS=62 WARN=0 ERROR=0 SKIP=0 TOTAL=62

Validates prod data health. A failure means current prod data is stale/broken, not necessarily that this PR is wrong.

@Walid-peach Walid-peach left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attention Score

100/100 - SAFE TO SKIM

  • Base 100
  • no Must Fix, no Should Fix findings
  • no high-risk paths (docs/ only, 82 added lines, 1 file)
  • tests/ penalty does not apply: no api/, rag/, or scripts/ change
  • Testing / Validation Gaps: none applicable to a decision record
    Reason you can skim: additive documentation only, every quantitative claim re-verified against the MON-208 data, and the two load-bearing choices (séance-only scope, soft-state refresh) were made by the maintainer rather than assumed here.

Summary

Adds ADR-030, settling the four questions that blocked MON-210: table shape, refresh semantics for a mutable source, ingestion scope, and summary policy. Also adds rule 14 to the standing rules list. Purely additive - 82 lines, no existing text modified, no code or schema.

The substantive contribution is that it resolves an apparent contradiction rather than papering over it. The agenda is the first mutable source in the pipeline, which looked like it required an exception to the upsert-only / no-DELETE rule in CLAUDE.md. The ADR shows the rule holds given a last_seen_at freshness column, and says so explicitly, so a future reader does not find two conflicting rules.

Must Fix

None.

Should Fix

None.

Nice to Have

  • The ADR names procedure_label rather than procedure for the column mapping the feed's procedure field. The rename avoids a Postgres non-reserved keyword; that reasoning is implicit in the column comment and could be stated outright so MON-210 does not "fix" it back.

Testing / Validation Gaps

None applicable - a decision record has nothing executable. What was verified rather than asserted:

  • point_uid present and unique on 3 644 / 3 644 séance ODJ points, and réunion uids unique on 7 469 / 7 469, so the proposed primary key is implementable.
  • Every column in the CREATE TABLE maps to a field confirmed present in the export by scripts/explore_agenda.py (#294).
  • All quantitative claims recomputed from the source data: 69% without a dossier reference, 16% stub objet, 27% of séances Supprimé, 72% commission dossier coverage against 31% for séances, procedure on 5.1% of points.

Documentation / Reviewer Notes

  • The claim worth a second opinion is decision 3, séance-only. It is the one place the ADR knowingly leaves better data on the table: commission réunions have more than double the dossier coverage and are the only forward-looking content that exists year-round, which matters because the page will be empty until the session returns in late September. It is a product judgment, not a technical one, and the ADR records a cheap reversal path (--type argument plus a backfill).
  • No CLAUDE.md change in this PR by design: its table inventory gains agenda_items when MON-210 creates it, not when an ADR describes it.
  • Prose avoids em dashes per project style; the ADR-030 — heading keeps one deliberately so it matches the format of all 29 preceding ADR headings.

Verdict

Ready to merge. MON-210 is unblocked on merge and has no schema decisions left to make.

@Walid-peach Walid-peach added the safe-to-skim PR review score above threshold - agent review likely sufficient label Aug 2, 2026
@Walid-peach
Walid-peach merged commit 7460f3f into master Aug 2, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe-to-skim PR review score above threshold - agent review likely sufficient

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant