Skip to content

feat(annotator): Antecedent Annotator — full-stack workbench for the labeled Id./supra corpus#829

Open
medelman17 wants to merge 33 commits into
mainfrom
worktree-feat+antecedent-annotator
Open

feat(annotator): Antecedent Annotator — full-stack workbench for the labeled Id./supra corpus#829
medelman17 wants to merge 33 commits into
mainfrom
worktree-feat+antecedent-annotator

Conversation

@medelman17

Copy link
Copy Markdown
Owner

What this is

A local, dockerized workbench where legal experts record what each "Id."/"supra" back-reference actually points to (its antecedent). The eyecite-ts engine pre-computes each back-reference's best guess + ranked candidates into Postgres; the human confirms the guess with one keystroke or corrects it. The output is a clean, exportable gold corpus of Id./supra → antecedent labels.

Why it matters: that labeled corpus is the missing ingredient gating the resolver research — measuring how often the engine is wrong, training a learned ranker, and calibrating abstention. None existed; this is the tool that produces it.

Before / Now

  • Before: eyecite-ts extracts and resolves citations, but there was no way to capture human ground-truth on short-form antecedents.
  • Now: sample/seed real opinions → engine prefill → Postgres → a keyboard-first React workbench (Review · Adjudicate · Corpus) → exportable gold JSONL. The whole stack comes up with docker compose up.

What's included

  • Backend (apps/annotator, Hono + postgres.js, 4 migrations): read API (document contract), label-write (POST /labels, next-unlabeled, resume, batches, annotators), adjudication queue + gold decisions, Cohen's κ inter-annotator agreement, and gold NDJSON export. A pure buildDocumentPayload maps extractCitations(text, { resolve: true }) → the UI contract (engine guess, ranked candidates, confidence, warnings, isBuriedAside via public fullSpan containment).
  • Corpus: replica ingestion (pnpm ingest, over-sampling parenthetical-dense hard cases) + a seed of four canonical hard-case opinions (Hogue/Corsello buried-paren, same-name Smith, supra, prose-introduced Yellen chain, string-cite abstain, nested Bryant/Davis) run through the real engine.
  • Frontend (apps/annotator/web, Vite + React + TS): the three surfaces from the approved Claude-design handoff, ported faithfully (dark theme, Spectral/IBM Plex, keyboard-first), wired to the live API and a11y-hardened (focus-visible, keyboard-operable controls, dialog semantics, live region, contrast, reduced-motion, 24px targets).
  • Docker: postgres + api + web via compose; nginx proxies /api.

The eyecite-ts core (src/) is untouched — the annotator is a private workspace app importing the published surface, so there's no changeset (nothing to release).

Verification (what was actually run)

  • Backend: 83 vitest integration tests pass against Postgres (TDD throughout; files run serially for determinism).
  • Frontend: tsc clean + vite build clean.
  • pnpm lint (CI gate) exits 0; git diff main..HEAD -- src/ is empty (core unchanged).
  • Full stack: docker compose up --build/healthz ok, web SPA serves, /api/healthz through the nginx proxy ok, seeded batch-042 (κ≈0.52) flows web→nginx→api→db.

Notable finding

Seeding through the real engine surfaced the real-vs-assumed divergences the corpus exists to quantify: the engine resolves an extra shortFormCase back-reference in one opinion and does not resolve a statute-section § …, supra in another — a candidate resolveSupra enhancement. Captured (with the deferred frontend decomposition + residual a11y items) in docs/superpowers/plans/2026-06-03-antecedent-annotator-followups.md.

Test plan

  • docker compose -f apps/annotator/docker-compose.yml up --build, then open http://localhost:8080
  • docker compose -f apps/annotator/docker-compose.yml exec api pnpm seed; verify Review / Adjudicate / Corpus load real data
  • (optional, with the db up) cd apps/annotator && pnpm exec vitest run

🤖 Generated with Claude Code

medelman17 and others added 30 commits June 3, 2026 16:42
…./supra corpus tool)

Approved brainstorming output — the brief handed to the external frontend design
pass. This repo's scope: the backend (API + Postgres + eyecite-ts resolver-prefill
+ CourtListener-replica corpus pipeline + label export) and the UI data contract.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…ad contract)

Bite-sized TDD plan for the foundation slice: apps/annotator workspace package,
pure prefill mapping (eyecite-ts -> UI contract), Postgres schema, replica
ingestion, and the read API serving the contract. API-write / adjudication /
export are follow-on plans. Core eyecite-ts package untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Add the annotator pnpm workspace package with package.json, tsconfig,
vitest config, and .env.example. Adds a minimal src/index.ts placeholder
to satisfy tsc TS18003 (no-inputs error) until Task 2 replaces it with
real contract types.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…e db

Adds 7-table schema (documents, citations, backrefs, annotators, batches,
batch_items, labels), a postgres.js client factory (makeSql), an idempotent
migration runner with _migrations tracking, a docker-compose.yml db-only
service on port 5433, and updates package.json scripts to load .env via
--env-file-if-exists so pnpm migrate works after cp .env.example .env.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
5433 is used by another local docker project; the in-container port (and the api service's db:5432 in Task 9) are unchanged.
Without .catch(), a failed migration became an unhandled rejection (noisy, unreliable exit code). Now logs the error, closes the client, and exits 1.
Add upsertDocumentPayload/getDocumentPayload (transactional upsert + select),
vitest setup file that loads .env for integration tests, and a round-trip
integration test against the live annotator-db container.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…rsist round-trip test

Code review (Task 6): on-conflict only updated source/text, leaving court/year stale on re-ingest. Add them to the SET clause (proven by a new failing-first re-ingest test). Also lock in the jsonb candidates array, span tuple, parties, and null court/year round-trips.
…nts, label-preserving upsert

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…chema

Adds migration 003 (batch mode column + batch_reviewers join table), four new
endpoints (GET/POST /annotators, POST /batches, GET /batches/:id) in a
src/routes/batches.ts module, and a 10-test integration suite covering happy
paths, 400 validation, FK-violation → 400, and auto-generated UUIDs.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…t preservation, bad-reviewer 400

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Add pure `canonicalCategory`/`cohenKappa` functions in `src/kappa.ts`
(11 unit tests, no DB) and a `GET /batches/:id/agreement` endpoint
(3 integration tests) that computes inter-annotator κ for exactly-2-reviewer
batches; returns null-kappa stub for single- or >2-reviewer batches.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…port (TDD)

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…FK, share decisionFromColumns

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…agreements) for dashboard

Co-Authored-By: Claude Opus 4.8 <[email protected]>
…act types, API client

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…ve API

Port adjudicator.jsx → adjudicator.tsx and dashboard.jsx → dashboard.tsx
from the design prototype to the real Vite+React+TS app backed by the live
Hono API. Extends DocViewer with an optional tintMap prop (mk-cite--tint-a/b/gold)
for reviewer colour highlights. Updates api.postGold to send the flat body the
backend expects. App.tsx replaces the two placeholder tabs with the real surfaces.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
…g semantics, live region, contrast, reduced-motion

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Moves every hook that was called after a conditional early-return guard
above the guard so the number of hooks called per render is constant:
- dashboard.tsx: useMemo (totals/KPIs) guarded on loadState.phase==="ready"
- reviewer.tsx: useCallback (onKey) + bind useEffect hoisted; handler body
  guards on ready and derives doc/backref/items from loadState inline; the
  two post-return render-only helpers (undo, nextUnlabeled) that had no JSX
  callers are removed (their logic lives in the Local variants inside onKey)
- adjudicator.tsx: keyboard useEffect + reset-draft useEffect hoisted; each
  body guards on ready and reads queue/docs/backref from loadState

No behavior, visual, or a11y changes; noUnusedLocals passes; build clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
…tart README

Adds Dockerfile for the API image (builds eyecite-ts at the workspace root
before starting the Hono server) and a multi-stage Dockerfile for the web
(Vite build → nginx). Extends docker-compose.yml with api and web services;
nginx proxies /api/ to the api container. Includes a .dockerignore to keep
build context lean and a full quickstart README.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
@codecov

codecov Bot commented Jun 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.51%. Comparing base (34133d0) to head (e14a95e).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #829   +/-   ##
=======================================
  Coverage   98.51%   98.51%           
=======================================
  Files         136      136           
  Lines        6243     6243           
  Branches     2066     2066           
=======================================
  Hits         6150     6150           
  Misses         93       93           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant