feat(annotator): Antecedent Annotator — full-stack workbench for the labeled Id./supra corpus#829
Open
medelman17 wants to merge 33 commits into
Open
feat(annotator): Antecedent Annotator — full-stack workbench for the labeled Id./supra corpus#829medelman17 wants to merge 33 commits into
medelman17 wants to merge 33 commits into
Conversation
…./supra corpus tool) Approved brainstorming output — the brief handed to the external frontend design pass. This repo's scope: the backend (API + Postgres + eyecite-ts resolver-prefill + CourtListener-replica corpus pipeline + label export) and the UI data contract. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…ad contract) Bite-sized TDD plan for the foundation slice: apps/annotator workspace package, pure prefill mapping (eyecite-ts -> UI contract), Postgres schema, replica ingestion, and the read API serving the contract. API-write / adjudication / export are follow-on plans. Core eyecite-ts package untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Add the annotator pnpm workspace package with package.json, tsconfig, vitest config, and .env.example. Adds a minimal src/index.ts placeholder to satisfy tsc TS18003 (no-inputs error) until Task 2 replaces it with real contract types. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…e db Adds 7-table schema (documents, citations, backrefs, annotators, batches, batch_items, labels), a postgres.js client factory (makeSql), an idempotent migration runner with _migrations tracking, a docker-compose.yml db-only service on port 5433, and updates package.json scripts to load .env via --env-file-if-exists so pnpm migrate works after cp .env.example .env. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
5433 is used by another local docker project; the in-container port (and the api service's db:5432 in Task 9) are unchanged.
Without .catch(), a failed migration became an unhandled rejection (noisy, unreliable exit code). Now logs the error, closes the client, and exits 1.
Add upsertDocumentPayload/getDocumentPayload (transactional upsert + select), vitest setup file that loads .env for integration tests, and a round-trip integration test against the live annotator-db container. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…rsist round-trip test Code review (Task 6): on-conflict only updated source/text, leaving court/year stale on re-ingest. Add them to the SET clause (proven by a new failing-first re-ingest test). Also lock in the jsonb candidates array, span tuple, parties, and null court/year round-trips.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…rything-real build
…nts, label-preserving upsert Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…chema Adds migration 003 (batch mode column + batch_reviewers join table), four new endpoints (GET/POST /annotators, POST /batches, GET /batches/:id) in a src/routes/batches.ts module, and a 10-test integration suite covering happy paths, 400 validation, FK-violation → 400, and auto-generated UUIDs. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…nts API Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…t preservation, bad-reviewer 400 Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Add pure `canonicalCategory`/`cohenKappa` functions in `src/kappa.ts` (11 unit tests, no DB) and a `GET /batches/:id/agreement` endpoint (3 integration tests) that computes inter-annotator κ for exactly-2-reviewer batches; returns null-kappa stub for single- or >2-reviewer batches. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…port (TDD) Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…FK, share decisionFromColumns Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
… demo batch/labels Co-Authored-By: Claude Opus 4.8 <[email protected]>
…agreements) for dashboard Co-Authored-By: Claude Opus 4.8 <[email protected]>
…act types, API client Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…/label/resume) Co-Authored-By: Claude Opus 4.8 <[email protected]>
…ve API Port adjudicator.jsx → adjudicator.tsx and dashboard.jsx → dashboard.tsx from the design prototype to the real Vite+React+TS app backed by the live Hono API. Extends DocViewer with an optional tintMap prop (mk-cite--tint-a/b/gold) for reviewer colour highlights. Updates api.postGold to send the flat body the backend expects. App.tsx replaces the two placeholder tabs with the real surfaces. Co-Authored-By: Claude Opus 4.8 <[email protected]>
…g semantics, live region, contrast, reduced-motion Co-Authored-By: Claude Opus 4.8 <[email protected]>
Moves every hook that was called after a conditional early-return guard above the guard so the number of hooks called per render is constant: - dashboard.tsx: useMemo (totals/KPIs) guarded on loadState.phase==="ready" - reviewer.tsx: useCallback (onKey) + bind useEffect hoisted; handler body guards on ready and derives doc/backref/items from loadState inline; the two post-return render-only helpers (undo, nextUnlabeled) that had no JSX callers are removed (their logic lives in the Local variants inside onKey) - adjudicator.tsx: keyboard useEffect + reset-draft useEffect hoisted; each body guards on ready and reads queue/docs/backref from loadState No behavior, visual, or a11y changes; noUnusedLocals passes; build clean. Co-Authored-By: Claude Opus 4.8 <[email protected]>
…tart README Adds Dockerfile for the API image (builds eyecite-ts at the workspace root before starting the Hono server) and a multi-stage Dockerfile for the web (Vite build → nginx). Extends docker-compose.yml with api and web services; nginx proxies /api/ to the api container. Includes a .dockerignore to keep build context lean and a full quickstart README. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…endent batch-042 case, unused var)
… a11y residual, resolver gap)
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #829 +/- ##
=======================================
Coverage 98.51% 98.51%
=======================================
Files 136 136
Lines 6243 6243
Branches 2066 2066
=======================================
Hits 6150 6150
Misses 93 93 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This was referenced Jul 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
A local, dockerized workbench where legal experts record what each "Id."/"supra" back-reference actually points to (its antecedent). The eyecite-ts engine pre-computes each back-reference's best guess + ranked candidates into Postgres; the human confirms the guess with one keystroke or corrects it. The output is a clean, exportable gold corpus of
Id./supra → antecedentlabels.Why it matters: that labeled corpus is the missing ingredient gating the resolver research — measuring how often the engine is wrong, training a learned ranker, and calibrating abstention. None existed; this is the tool that produces it.
Before / Now
docker compose up.What's included
apps/annotator, Hono + postgres.js, 4 migrations): read API (document contract), label-write (POST /labels, next-unlabeled, resume, batches, annotators), adjudication queue + gold decisions, Cohen's κ inter-annotator agreement, and gold NDJSON export. A purebuildDocumentPayloadmapsextractCitations(text, { resolve: true })→ the UI contract (engine guess, ranked candidates, confidence, warnings,isBuriedAsidevia publicfullSpancontainment).pnpm ingest, over-sampling parenthetical-dense hard cases) + a seed of four canonical hard-case opinions (Hogue/Corsello buried-paren, same-name Smith, supra, prose-introduced Yellen chain, string-cite abstain, nested Bryant/Davis) run through the real engine.apps/annotator/web, Vite + React + TS): the three surfaces from the approved Claude-design handoff, ported faithfully (dark theme, Spectral/IBM Plex, keyboard-first), wired to the live API and a11y-hardened (focus-visible, keyboard-operable controls, dialog semantics, live region, contrast, reduced-motion, 24px targets).postgres + api + webvia compose; nginx proxies/api.The eyecite-ts core (
src/) is untouched — the annotator is aprivateworkspace app importing the published surface, so there's no changeset (nothing to release).Verification (what was actually run)
tscclean +vite buildclean.pnpm lint(CI gate) exits 0;git diff main..HEAD -- src/is empty (core unchanged).docker compose up --build→/healthzok, web SPA serves,/api/healthzthrough the nginx proxy ok, seededbatch-042(κ≈0.52) flows web→nginx→api→db.Notable finding
Seeding through the real engine surfaced the real-vs-assumed divergences the corpus exists to quantify: the engine resolves an extra
shortFormCaseback-reference in one opinion and does not resolve a statute-section§ …, suprain another — a candidateresolveSupraenhancement. Captured (with the deferred frontend decomposition + residual a11y items) indocs/superpowers/plans/2026-06-03-antecedent-annotator-followups.md.Test plan
docker compose -f apps/annotator/docker-compose.yml up --build, then open http://localhost:8080docker compose -f apps/annotator/docker-compose.yml exec api pnpm seed; verify Review / Adjudicate / Corpus load real datacd apps/annotator && pnpm exec vitest run🤖 Generated with Claude Code