feat(weather-recon): Phase 2a — observations pipeline, 19k METARs loaded (#184)#193
Merged
Conversation
…elds, fail-fast env, doc fixes) Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W179bqEPFP92a3JrsPkD5c
Implements parse_tenths, parse_wnd, parse_gust, parse_vis_km, sky_from_gf1, weather_from_mw1, and raw_metar_from_rem parsers for NCEI ISD CSV fields. All 14 tests pass with fixtures from real KORD 2001-09 data. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W179bqEPFP92a3JrsPkD5c
WMO ww code 47 genuinely means fog (fog, sky invisible), so the WW_CODES entry was correct and the test fixture was wrong. Restore "47": "fog" and exercise the unknown-code path with 39 (blowing snow band) instead. Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W179bqEPFP92a3JrsPkD5c
…+ dedupe Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W179bqEPFP92a3JrsPkD5c
Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W179bqEPFP92a3JrsPkD5c
Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W179bqEPFP92a3JrsPkD5c
…ad identifiers pick_station_rows() picks the isd-history row that exactly matches a curated ICAO and nominally covers 2001-09-09..12, but for 10 stations that row was a dead end against the live NCEI global-hourly API (placeholder USAF, or a file with zero FM-15/METAR rows for the window). Add an ISD_OVERRIDES map, verified against the live API, that repoints 9 of them at a working USAF-WBAN for the same airport; KFLG has no working alternate and stays empty. Reloaded weather_stations (188) and weather_observations (18315 -> 19176). Co-Authored-By: Claude Fable 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01W179bqEPFP92a3JrsPkD5c
|
✅ Playwright E2E — 1 passed · 0 failed · 0 flaky · 0 skipped |
This was referenced Jul 13, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 2a of the Weather app (#184): the observations pipeline. Real hourly METAR/SPECI conditions for 2001-09-09..12 are now loaded in production Directus — 19,176 observations across 187/188 stations.
What's in here
weather_recon/obs.py— ISD global-hourly field parsers (WND/TMP/DEW/SLP/VIS/GF1/MW1/OC1/REM → typed columns) + row shaping: METAR/SPECI filter, per-timestamp dedupe (FM-15 wins), raw METAR extraction. Test fixtures are real captured KORD Sept-2001 values.weather_recon/fetch_ncei.py— per-station subset fetch from NCEI's data service (endDate-inclusive, dynamic columns) with per-(station, window) disk cache — retries and re-runs replay from disk.weather_recon/flow_observations.py—load-weather-observationsflow: fetch 188 stations → shape → idempotentdelete_all+ insert.ISD_OVERRIDESinscripts/build_stations.py— 9 stations (CYUL, KBGR, KCXY, KDAL, KMHT, KMRY, KOAK, KSMF, MMUN) had isd-history ids with no 2001-09 data (placeholder-USAF rows); each override was validated against live NCEI before acceptance. KFLG genuinely has no Sept-2001 ISD data and stays as a dataless station.insert_manyfields param, fail-fast env checks in_client, doc fixes.Verification
FU PLUME DSNT NW DRFTG SE— the WTC smoke plume in the real record. Idempotent reload confirmed.Follow-ups (Phase 2b)
README refresh (stale layout section), decide whether observations gets a work-pool deployment registration, header sanity-check on cached fetches.
🤖 Generated with Claude Code
https://claude.ai/code/session_01W179bqEPFP92a3JrsPkD5c