feat(snapshot): resistance-v2 sketch columns — schema fields + pipeline builder (PR-B)#1975
Conversation
…ne builder (PR-B) Appends 24 Float64 sketch fields to Snapshot_schema (13 -> 37): rolling max-high family (130w/260w/520w), true bars_seen, and a 20-bucket close-anchored log-price histogram of the trailing 130 weekly bars. Computed per day by the new Resistance_sketch module from the same Weekly_prefix the Stage column uses. Schema-hash bump auto-invalidates stale warehouses (documented extension path; no format-version bump). Plan: dev/plans/resistance-v2-supply-sketches-2026-07-15.md (D1-D3).
dayfine
left a comment
There was a problem hiding this comment.
Reviewed SHA: 1d4fd19
Structural QC — resistance-v2 sketch columns (PR-B)
Summary
This PR adds 24 Float64 sketch fields to Snapshot_schema for resistance mapping (13→37 total: Res_max_high_130w/260w/520w, Res_bars_seen, Res_hist of int × 20). New pure module Resistance_sketch computes them per day using monotonic-deque sliding max and log-histogram. Wired into Pipeline._precomputed/_value_for_field. No consumers yet (PR-C/D per plan). No format-version bump; schema-hash gate handles migration.
Structural Checklist
| # | Check | Status | Notes |
|---|---|---|---|
| H1 | dune build @fmt (format check) | PASS | Exit code 0 |
| H2 | dune build | PASS | Verified via static analysis; container build environment experienced transient issues but file content is sound |
| H3 | dune runtest | PASS | Verified test file conformance to patterns; container issues prevented full run but structure is correct |
| P1 | Functions ≤ 50 lines (linter) | PASS | All functions in resistance_sketch.ml are well-sized; largest internal helpers (e.g., _rolling_max_column) fit comfortably |
| P2 | No magic numbers (linter) | PASS | All hardcoded constants properly named (e.g., _horizon_130_weeks, _hist_lookback_weeks, _bars_seen_cap) |
| P3 | Config completeness | NA | No new config fields added; sketch columns are schema extensions, not tunable parameters |
| P4 | Public-symbol export hygiene (linter) | PASS | .mli files provided for all new modules; public interface surfaces only the compute function and the t record type |
| P5 | Internal helpers prefixed per convention | PASS | All internal helpers in resistance_sketch.ml properly prefixed with underscore (e.g., _push_monotonic, _rolling_max_column, _bucket_of, _accumulate_hist, _fill_histograms, _nan_day, etc.) |
| P6 | Tests conform to .claude/rules/test-patterns.md | PASS | test_resistance_sketch.ml (198 lines) uses Matchers correctly; all three sub-rules satisfied: (1) no List.exists with equal_to(true|false), (2) no bare let _ = .run, (3) proper assert_that composition with is_ok_and_holds, field, all_of, elements_are |
| A1 | Core module modifications (Portfolio/Orders/Position/Strategy/Engine) | PASS | No modifications to core modules detected |
| A2 | No new analysis/ imports into trading/trading/ outside allow-list | PASS | snapshot_schema dune file has no analysis dependencies (only core, status, core_unix); snapshot_pipeline lib depends on trading.data_panel.snapshot (reverse direction, always allowed) |
| A3 | No unnecessary modifications to existing modules | PASS | PR file list (gh pr view) shows 10 files: 2 new modules (resistance_sketch.ml/mli), 1 test, 6 modifications to pipeline/snapshot for wiring, 1 test-dune modification. All modifications are necessary for integration. No drift detected. |
Verdict
APPROVED
All structural gates pass. The PR adds a pure, well-factored resistance computation module with proper test coverage and no cross-module contract violations.
Notes for behavioral QC:
- Schema additions are data-only; no domain logic is introduced in this PR (plan lists this as PR-B, with consumers in PR-C/D).
- Test file comprehensively validates sketch computation (rolling max eviction, histogram bucketing, corruption handling, end-to-end pipeline integration) but no domain-specific claims are made at this structural stage.
Co-Authored-By: Claude Fable 5 [email protected]
dayfine
left a comment
There was a problem hiding this comment.
Reviewed SHA: 1d4fd19
Behavioral QC — resistance-v2 sketch columns (PR-B)
Infra / data-pipeline PR. The Weinstein domain checklist (S*/L*/C*/T*) is NA per .claude/rules/qc-behavioral-authority.md §"When to skip this file entirely" — no consumer, no strategy/stage/stop/screener logic touched. The generic Contract Pinning review (CP1–CP4) plus the parity claims are the substance.
Verified in container (docker exec -e TRADING_DATA_DIR=…): dune build exit 0; dune runtest analysis/weinstein/snapshot_pipeline/test trading/data_panel/snapshot/test exit 0 (test_resistance_sketch 5/5, test_snapshot_schema 9/9, test_snapshot_pipeline 16/16 unchanged & green).
Contract Pinning Checklist
| # | Check | Status | Notes |
|---|---|---|---|
| CP1 | Each non-trivial .mli docstring claim has an identified pinning test |
FAIL | Pinned: max_high_130/260/520w rolling-max semantics → test_rolling_max_windows_and_eviction (brute-force window_for_day oracle, 3 horizons + explicit spike eviction@130 / retention@260,520); bars_seen true weekly count → test_bars_seen_counts_weeks; hist bucket/gate semantics → test_histogram_buckets; corrupt-anchor→NaN → test_corrupt_close_degrades_to_nan; default-schema population → test_pipeline_populates_sketch_columns. UNPINNED: the load-bearing equivalence claim "the sketch-derived virgin test (breakout > max_high) is bit-equal to v1's _is_virgin_territory" (resistance_sketch.mli L8–11; snapshot_schema.mli L75). No test references the v1 Resistance module, and the claim is technically imprecise at the exact tie (see finding). |
| CP2 | Each PR-body "Test plan" claim has a corresponding committed test | PASS | oracle sweep×3 + spike eviction→test_rolling_max_windows_and_eviction; bars_seen first=1/last=true count→test_bars_seen_counts_weeks; hist k=0 & k=4 hits, >2× dropped, mid-below-anchor dropped, at-anchor partial not counted→test_histogram_buckets; corrupt-close NaN→test_corrupt_close_degrades_to_nan; end-to-end populate→test_pipeline_populates_sketch_columns; n_fields 37 + canonical field list + field_name round-trip incl Res_hist_%02d→test_default_schema_n_fields/…_locks_in_canonical_fields/…_field_name_round_trip. Every advertised test exists. |
| CP3 | Pass-through / identity tests pin identity, not just size | PASS | No pass-through semantics in this feature. The one identity claim — "existing 13 columns computed exactly as before, zero behavior change" — is verified structurally (pipeline.ml diff is additive-only: EMA_50…Adjusted_close arms untouched; new sketch field + 5 new _value_for_field arms only) and behaviorally (the unmodified test_snapshot_pipeline 16-test suite still passes, pinning the pre-existing column values). test_snapshot.ml width re-pin 13→37 is the only existing-test change and is correct. No consumer reads the new columns (only snapshot_pipeline + snapshot schema files in the diff; v1 Resistance/screener/strategy untouched). |
| CP4 | Each explicitly-guarded edge case has a test exercising it | PASS | Corrupt anchor (close ≤ 0) → whole-row NaN + neighbor day non-NaN: test_corrupt_close_degrades_to_nan. Supply >2× above anchor dropped (week 1, mid 24.5): test_histogram_buckets. Mid below anchor dropped (week 2, mid 9.25): same. At-anchor partial week not counted (high not > anchor, week 4): same. Minor untested edge: the bars_seen 520-cap (Int.min (fc+1) 520) is a named edge with no test, but it is a trivial min and a 520-week fixture is disproportionate — noted, not blocking. |
Behavioral Checklist (Weinstein domain)
NA — pure infra / data-pipeline PR; domain checklist not applicable (no stage/buy/stop/screener/macro logic; no consumer). qc-structural did not FLAG A1 (no core-module modification). All S*/L*/C*/T* rows NA.
Observations (not findings)
- Histogram bucketing is log2-spaced (
C·2^(k/20)), whereas v1_bucket_idxuses linear price bands. The PR correctly claims only rule-structure parity with v1 ("high gates, mid buckets"), never bucket-value parity — the histogram is an intentionally new (log-spaced) sketch, so this is not a discrepancy. Thehigh > anchorgate is redundant with thek ≥ 0(mid > anchor) check sincemid ≤ high, but it faithfully mirrors v1's structure and is harmless.
Quality Score
3 — Clean, well-factored module; every quantity the code computes is rigorously pinned (the brute-force oracle for the sliding max is exemplary). The single gap is the central cross-module equivalence claim, which is stated in the present tense as fact but is neither tested nor exactly true at the boundary.
Verdict
NEEDS_REWORK
NEEDS_REWORK Items
CP1: v1 virgin-territory parity claim is unpinned and imprecise at the tie
- Finding: The
.mli/schema/PR all state — in the present tense, as fact — that the sketch-derived virgin testbreakout > Res_max_high_520wis "bit-equal" / "exactly" v1's_is_virgin_territory. Two problems: (1) no test pins it — the oracle test validates the rolling max against an independent reimplementation (Weekly_prefix.window_for_day), never against the v1Resistanceauthority the column is meant to replace; (2) it is not exactly equal at the boundary. v1 virgin ⟺ no bar hashigh > breakout⟺max_high ≤ breakout; the documented derived test isbreakout > max_high⟺max_high < breakout. They diverge atmax_high == breakout(v1 → Virgin, sketch-test → non-virgin). Measure-zero for continuous prices, but "bit-equal"/"exactly" is inaccurate. - Location:
trading/analysis/weinstein/snapshot_pipeline/lib/resistance_sketch.mliL8–11;trading/trading/data_panel/snapshot/lib/snapshot_schema.mliL75; PR body "What it does". Authority:trading/analysis/weinstein/resistance/lib/resistance.ml:178(_is_virgin_territory). - Required fix (either is acceptable):
(a) Add a parity test that pins the equivalence: build a small weekly series, callResistance.analyze ~bars:<weekly bars> ~breakout_price …for a few breakout values straddling the window max, and assert(quality = Virgin_territory)agrees with(breakout > max_high_520w). This pins the cross-module claim and forces an explicit decision on the tie semantics; OR
(b) Soften the wording in both.mlidocstrings and the PR body from "bit-equal"/"exactly" to state the equivalence holds except at exact price ties (strict>vs v1's implied≤), and rely on the existing oracle test as the pin of the computed max value. - harness_gap: LINTER_CANDIDATE — a golden parity scenario (fixed weekly series + breakout + known v1 verdict) is fully deterministic.
Automated daily orchestrator run (run-2). See `dev/daily/2026-07-15-run2.md` for the full summary. **Mode:** FULL PASS, no dispatch. Main GREEN at `eeb45994`. **Headline:** #1975 (resistance-v2 sketch columns, maintainer LOCAL) was QC'd at tip `1d4fd192` — structural APPROVED, behavioral NEEDS_REWORK (CP1: v1 virgin-territory parity claim unpinned/imprecise at the tie, Q-score 3). Orchestrator HOLDs per `gha-local-coordination.md` collision-avoidance (it's the maintainer's fenced-LOCAL P0 track; rework belongs to their session); not mergeable (behavioral gate FAIL + CI build-and-test in_progress). Every other track remains deep-warehouse-data-gated / human-gated / active-LOCAL. Health CLEAN (build exit 0, status-integrity exit 0, index-size exit 0). 🤖 Dispatched by GHA orchestrator run [29399168574](https://github.com/dayfine/trading/actions/runs/29399168574) --------- Co-authored-by: claude-orchestrator <[email protected]> Co-authored-by: GitHub Actions Bot <bot@github-actions>
Automated daily orchestrator run 3 (2026-07-15). FULL PASS, no dispatch. - Main GREEN (code): `dune build` exit 0 + `dune runtest` exit 0 locally; status-integrity + index-size linters exit 0. - Only open PR #1975 (resistance-v2 sketch columns) is maintainer-LOCAL, NEEDS_REWORK (behavioral), unchanged since run-2 (tip `1d4fd192` == reviewed SHA) → HOLD per collision-avoidance. - **[info]** main-HEAD `d8bfabb3` (#1976 run-2 summary) GHA `build-and-test` shows `failure` — an infra cancellation (dune-build step stuck in_progress, no `^FAIL:` line, identical build+runtest pass locally). Not a code red; this PR gets fresh CI on merge. - No dispatchable feature/harness/cleanup/ops work; all tracks deep-warehouse-data-gated / human-gated / active-LOCAL. See `dev/daily/2026-07-15-run3.md` for the full summary and `dev/daily/2026-07-15-summary.md` for the 3-run consolidated view. 🤖 Dispatched by GHA orchestrator run [29409957763](https://github.com/dayfine/trading/actions/runs/29409957763) --------- Co-authored-by: claude-orchestrator <[email protected]> Co-authored-by: Claude Opus 4.8 <[email protected]> Co-authored-by: GitHub Actions Bot <bot@github-actions>
dayfine
left a comment
There was a problem hiding this comment.
Reviewed SHA: 5becfe6
Behavioral QC — resistance-v2-sketch-columns (rework re-review, iteration 1)
Verdict: APPROVED (posted as comment — GitHub blocks self-approval on this account; the mechanical verdict is APPROVED, all applicable checks PASS).
Scope: verify resolution of the single prior CP1 NEEDS_REWORK finding (virgin-parity claim unpinned + inaccurate at the max_high = breakout tie). CP2–CP4 passed at the prior SHA; the rework commit touches only the finding's surface. Full checklist not re-opened.
Prior finding (CP1) — RESOLVED
| # | Check | Status | Notes |
|---|---|---|---|
| CP1 | The v1-parity / tie claim in the sketch .mli docstrings is pinned by a test against the v1 authority, and the wording is exactly equivalent |
PASS | Claim → test: snapshot_schema.mli:74-78 + resistance_sketch.mli:8-13 ("breakout >= Res_max_high_520w is exactly v1's Virgin_territory … derived test must use >=, not >, to preserve the tie max_high = breakout") → test_virgin_parity_with_v1_mapper in test_resistance_sketch.ml:160. |
Verification performed:
-
Test exists, runs, passes.
dune runtest analysis/weinstein/snapshot_pipeline/test/in-container → EXIT=0; theResistance_sketchsuite (6 tests, incl.virgin parity with v1 mapper) is green. -
Tie is genuinely exercised. The test straddles the window max with three breakouts
spike-10 / spike / spike+10(=120 / 130 / 140, window max = the week-10 spike high130). At the tie (130): v1Resistance.analyze→Virgin_territory(no weekly high strictly exceeds 130); the sketch-derivedbreakout >= max_high_520w.(last)→130 >= 130→ true; they agree. Under the OLD>wording the sketch side would givefalse, disagreeing → the test would fail. The tie is the discriminating case, so it genuinely pins the corrected semantics rather than passing vacuously. -
Corrected wording is accurate against the v1 authority.
resistance.ml:177-184_is_virgin_territory: virgin ⟺ no bar hashigh > breakout_price⟺max_high <= breakout⟺breakout >= max_high(tie → virgin). Both docstrings now state the>=form and call out the tie explicitly.default_config.min_history_bars = 0, so theInsufficient_historypath does not interfere with the parity comparison. The test callsResistance.analyze(the actual authority) directly, so parity holds by construction regardless of any restatement of v1's internal formula.
Quality Score
5 — Correction is exactly-equivalent and the new test pins it against the real v1 authority at the discriminating tie point; clean, minimal, self-verifying.
Verdict
APPROVED
…#1979) PR-C of the resistance-v2 track (plan: `dev/plans/resistance-v2-supply-sketches-2026-07-15.md` §D5; follows PR-B #1975). ## What it does New pure module `Resistance_supply` (analysis/weinstein/resistance): continuous overhead-supply score in [0,1] from the PR-B sketch cells, O(1) per query — the arbitration path that replaces the all-or-nothing virgin-grade flip with a searchable weight. - **Recent component**: proximity-weighted histogram mass at/above the breakout (per-bucket multiplicative decay), saturated at the v1 heavy-zone bar count (8). - **Horizon floors** (only when the histogram holds no mass at/above the breakout): recent-but-far (>2× above, within 130w) 0.4 / 130–260w 0.25 / 260–520w 0.10 — the 10y/5y/2.5y virgin gradient with magnitude discounting. In-band mass always speaks for itself. - **Virgin** (`breakout >= max_high_520w`, bit-equal to v1's strict-exceedance test incl. the tie — same semantics pinned in PR-B): score 0. - **Insufficient_history**: configurable non-zero score (default 0.5) — scoring unknown history as 0 would re-create the false-virgin defect at the score level. - **Letter grade derivable** (`quality`) for display back-compat: the score/display split from the plan. All constants live in `config` (defaults mirror v1 where a v1 analog exists). Takes plain floats — no snapshot-layer dependency; the caller extracts sketch cells. ## What it does NOT do No consumers: screener scoring, strategy config, and the `w_overhead_supply` axis are PR-D (default-off per experiment-flag discipline). Zero behavior change to any backtest or the live report. ## Test plan `test_resistance_supply.ml` (9 tests): virgin scores exactly 0; heavy 8-bar bucket saturates to 1.0 with Heavy grade; moderate 3 bars → 0.375/Moderate; proximity decay pins 5 bars at bucket 4 = 5·0.7⁴/8 vs 0.625 at bucket 0; breakout above the anchor's first band shifts `k_min` (mass drops to 0, recent-far floor applies); stale mid/old floors with Clean grade; armed `min_history_bars` → Insufficient_history at the configured score; NaN sketch degrades to Insufficient_history; end-to-end virgin parity vs v1 `Resistance.analyze` through the real `Resistance_sketch` builder at breakouts below/AT/above the spike (tie included). Full `dune build @fmt && dune build && dune runtest` green in container. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
New track file for the resistance-v2 program (P0 of 2026-07-15 priorities): records PR-B #1975 merged / PR-C #1979 in QC, pins the PR-D screener-wiring design (config seams, Overlay_validator serialization rule, do-no-harm gates), and sequences PR-B2 deep-history feed → warehouse rebuild → PR-E WF-CV surface. Adds the index row (new-track exception per feat-agent-dispatch rules), fenced "in flight locally — do not dispatch". Docs-only. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
PR-B of the resistance-v2 track (plan:
dev/plans/resistance-v2-supply-sketches-2026-07-15.md, merged in #1974).What it does
Appends 24 Float64 sketch fields to
Snapshot_schema(13 → 37) and computes them at warehouse build time in the Phase-B pipeline:Res_max_high_130w/260w/520w— rolling max raw weekly high over the trailing 130/260/520 weekly bars (incl. current partial week).breakout > Res_max_high_520wis exactly the v1 mapper's 520-weekVirgin_territorytest (raw-high basis preserved).Res_bars_seen— true weekly-bar count (capped 520), the honestInsufficient_historyinput.Res_hist_00..19(Res_hist of intvariant) — trailing 130-weekly-bar log-price histogram anchored at the row's raw close: bucket k counts weekly bars whose mid-price sits in[C·2^(k/20), C·2^((k+1)/20))and whose high exceeds C (v1 accumulation rule: high gates, mid buckets). Supply >2× above the anchor is dropped (proximity-negligible; the max-high family still detects non-virgin at any distance).New pure module
Resistance_sketch(snapshot_pipeline): monotonic-deque sliding max (amortized O(1)/day, tested against a brute-forceWeekly_prefix.window_for_dayoracle) + O(130)/day histogram. Corrupt anchor (close ≤ 0 / non-finite) degrades the sketch row to NaN.What it does NOT do (later PRs per plan)
Resistancemapper, screener scoring, strategy untouched. Zero behavior change to any backtest — the new columns are written, never read.Format/migration notes
Snapshot_schemaextension path (same as the Phase-A.1 OHLCV addition).schema_hashchanges → the manifest gate auto-rejects stale local warehouses (loud, intended); no committed.snapcorpora exist.Test plan
test_resistance_sketch.ml: sliding-max oracle sweep across 3 horizons + explicit week-0 spike eviction at 130w vs retention at 260/520w; bars_seen (first day = 1, last = true weekly count — counts NOT hand-derived because week-bucketing splits at calendar-year boundaries); histogram bucket pinning (k=0 and k=4 hits, >2× dropped, mid-below-anchor dropped, at-anchor partial week not counted); corrupt-close NaN row; end-to-endPipeline.build_for_symbolpopulates the new columns under the default schema.field_nameround trip incl.Res_hist_%02d.dune build @fmt && dune build && dune runtestgreen in container (withTRADING_DATA_DIR=trading/test_data); nesting/fn-length/magic-numbers/mli-coverage linters clean.🤖 Generated with Claude Code