Skip to content

feat(weather-recon): Phase 2d — almanac, 188/188 stations on Wasabi (#184)#196

Merged
robbiebyrd merged 6 commits into
mainfrom
feat/weather-recon-almanac
Jul 13, 2026
Merged

feat(weather-recon): Phase 2d — almanac, 188/188 stations on Wasabi (#184)#196
robbiebyrd merged 6 commits into
mainfrom
feat/weather-recon-almanac

Conversation

@robbiebyrd

Copy link
Copy Markdown
Collaborator

Phase 2d of the Weather app (#184): the almanac — and with it, all four datasets of the data pipeline are now live. 188/188 stations have almanac JSON on Wasabi under weather/almanac/: record high/low (with years), 1971–2000 normal high/low, and record precip for Sept 9–12, computed from GHCN-Daily using only data through 2001-09-08 (a record set during the replay window can never appear — belt-and-suspenders: the fetch is cutoff-bounded AND the math re-filters).

What's in here

  • weather_recon/ghcn.py — GHCN id derivation (USW000+WBAN), ghcnd-stations.txt parsing, haversine k-nearest matching, almanac computation (tenths→1 dp °C/mm, ties→latest year).
  • weather_recon/flow_almanac.py — candidate ladder (overrides → derived → k-nearest), cached fetches, per-station JSON + index.json (same key_prefix/key_pattern shape as the radar index).
  • k-nearest fallback (found live): the single nearest GHCN gauge near airports is often precip-only — first run gapped 12 of 17 Canadian stations. Trying the 5 nearest in distance order rescued all 13 gaps with zero manual overrides (CYYZ→CA006158733, CYUL→CA007025250, MMVA→MXN00027054).

Verification

  • 90/90 pytest, ruff clean; final review independently verified the k-nearest sort, hand-checked the fixture geometry, and reordered the test fixture so the ordering assertion can actually fail an unsorted implementation.
  • Live on Wasabi: 188 almanacs + index (0 gaps); sampled KORD/KJFK/CYYZ/MMMX/KLAX/KDCA — all record years ≤ 2001, normals plausible (Toronto Sept-9 normal high 23.3 °C, KORD record high 35.0 °C/1983).

⚠️ Pre-Phase-3 blocker (infra repo)

The whole weather/ prefix currently 404s through files.911realtime.org — the file-proxy allow-list (Traefik Ingress path rules in Keeping-History/infra) needs a weather/ entry before the frontend can fetch radar frames, almanacs, or any of it. Objects are confirmed present via the S3 API.

What's next

Phase 3: the weather streamer channel (observations + forecasts, flights-style). Phase 4: the Classicy Weather app itself.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W179bqEPFP92a3JrsPkD5c

Robbie Byrd and others added 6 commits July 12, 2026 23:53
…ions

Phase 2d-2 live run of load_weather_almanac published 175/188 with 13 gaps
(CYEG CYHZ CYOW CYQR CYQT CYSJ CYUL CYWG CYXE CYXY CYYZ CYZF MMVA — 12 of 17
Canadian stations + Villahermosa). Root cause: nearest_ghcn returned only the
single closest ghcnd-stations.txt entry; near several airports that entry is
a precip-only or sparse gauge, the flow's <300-valid-TMAX demotion rejected
it, and there was no next-nearest fallback.

Add nearest_ghcn_candidates(lat, lon, stations, max_km=20.0, limit=5),
returning up to 5 nearest station ids within max_km ordered by distance;
reimplement nearest_ghcn on top of it so existing callers/tests are
unaffected. flow_almanac now extends its candidate list with all k-nearest
ids instead of just the single nearest, so a sparse first match demotes to
the next-nearest rather than a recorded gap.

Before: 175/188 published, 13 gaps.
After:  188/188 published, 0 gaps. No GHCN_OVERRIDES entries were needed —
the k-nearest fallback alone resolved every gap, including the Canadian
majors (CYYZ -> CA006158733, CYUL -> CA007025250) and MMVA -> MXN00027054.

Co-Authored-By: Claude Fable 5 <[email protected]>
… docstring gate wording

Final-review fixes: NEARBY_STATIONS deliberately out of distance order so
the ordering assertions can fail a naive unsorted implementation; the
almanac candidate-demotion docstring now states the real >=300-TMAX gate.

Co-Authored-By: Claude Fable 5 <[email protected]>
@github-actions

Copy link
Copy Markdown

Playwright E2E — 1 passed · 0 failed · 0 flaky · 0 skipped

Full report

@robbiebyrd robbiebyrd merged commit 49a1980 into main Jul 13, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant