Skip to content

Commit 3952941

Browse files
author
Marcel
committed
Update paper/results to completed heldout80 and publish rolling latest release
1 parent fb840a2 commit 3952941

14 files changed

Lines changed: 67 additions & 56 deletions

.github/workflows/ci.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ jobs:
120120
Rolling paper build from `main`.
121121
122122
Source commit: `${{ github.sha }}`
123-
prerelease: true
124-
make_latest: false
123+
prerelease: false
124+
make_latest: true
125125
files: release/main.pdf
126126
overwrite_files: true

README.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,10 @@ Completed evidence so far:
1919
- `layer16,last1` was a false lead: useful in early pilots, not robust on held-out data.
2020
- Broader training washed out the site-specific signal.
2121
- The oracle rows are reference patches, not ceilings.
22+
- Completed heldout80: `targeted_mlp(layer34,last1)` reached `h8=0.1156`, `h16=0.0641` vs no-patch `h8=0.0172`, `h16=0.0133`.
2223

23-
Active validation:
24-
- `helmas3n/artifacts/reports/targeted_site_study_v5_holdout80/` is the current held-out80 rerun.
24+
Latest validation artifact:
25+
- `helmas3n/artifacts/reports/targeted_site_study_v5_holdout80/` is the completed heldout80 report.
2526

2627
## Where to start reading
2728

helmas3n/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,14 @@ Each sample stores:
102102
3. KV uplift baseline + direct comparison support.
103103
4. Flow refinement placeholder module (not default).
104104

105+
## Latest Results
106+
107+
- Completed heldout80 targeted-site rerun: `artifacts/reports/targeted_site_study_v5_holdout80/`.
108+
- `targeted_mlp(layer34,last1)` is the strongest learned method:
109+
- `h1=0.7750`, `h4=0.2031`, `h8=0.1156`, `h16=0.0641`
110+
- no-patch baseline: `h1=0.0375`, `h4=0.0250`, `h8=0.0172`, `h16=0.0133`
111+
- `targeted_mlp(layer16,last1)` does not generalize on heldout80.
112+
105113
## Notes
106114

107115
- `configs/extract.yaml` now defines non-identical low/full regime controls via `text_overrides`.

paper/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,4 +34,4 @@ The repository tracks paper sources and generated analysis assets. LaTeX tempora
3434

3535
## Scope of the draft
3636

37-
The draft is written as a living research document. It uses the completed pilot, layer-sweep, suffix-span, and targeted-site evidence as the current result base, while calling out the larger held-out80 validation as the active next gate.
37+
The draft is written as a living research document. It now includes the completed heldout80 targeted-site validation as part of the core evidence base, alongside the completed pilot, layer-sweep, and suffix-span studies.
0 Bytes
Binary file not shown.
0 Bytes
Binary file not shown.
0 Bytes
Binary file not shown.
-614 Bytes
Binary file not shown.
-7.91 KB
Loading
Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
1-
\begin{tabular}{lcccccccc}
1+
\begin{tabular}{lcccccc}
22
\toprule
3-
Method & Pilot h1 & Pilot h4 & Pilot h8 & Pilot h16 & Held-out h1 & Held-out h4 & Held-out h8 & Held-out h16 \\
3+
Method & h1 & h4 & h8 & h16 & $\Delta$h8 & $\Delta$h16 \\
44
\midrule
5-
No patch & 0.000000 & 0.006250 & 0.009375 & 0.012500 & 0.000000 & 0.000000 & 0.000000 & 0.006250 \\
6-
Identity & 0.000000 & 0.006250 & 0.009375 & 0.012500 & 0.000000 & 0.000000 & 0.000000 & 0.006250 \\
7-
Broad MLP & 0.000000 & 0.012500 & 0.021875 & 0.017188 & 0.000000 & 0.000000 & 0.012500 & 0.012500 \\
8-
Targeted MLP @ layer16 & 0.000000 & 0.006250 & 0.009375 & 0.010937 & 0.000000 & 0.000000 & 0.000000 & 0.000000 \\
9-
Targeted MLP @ layer34 & 0.825000 & 0.212500 & 0.115625 & 0.067187 & 0.700000 & 0.175000 & 0.100000 & 0.056250 \\
10-
Oracle layer16 & 0.000000 & 0.012500 & 0.021875 & 0.017188 & 0.000000 & 0.000000 & 0.000000 & 0.000000 \\
11-
Oracle layer34 & 0.250000 & 0.068750 & 0.037500 & 0.028125 & 0.400000 & 0.100000 & 0.050000 & 0.031250 \\
12-
Oracle stride & 0.200000 & 0.081250 & 0.046875 & 0.031250 & 0.300000 & 0.075000 & 0.050000 & 0.025000 \\
5+
No patch & 0.037500 & 0.025000 & 0.017188 & 0.013281 & 0.000000 & 0.000000 \\
6+
Identity & 0.037500 & 0.025000 & 0.017188 & 0.013281 & 0.000000 & 0.000000 \\
7+
Broad MLP & 0.075000 & 0.028125 & 0.018750 & 0.010937 & 0.001562 & -0.002344 \\
8+
Targeted MLP @ layer16 & 0.012500 & 0.006250 & 0.010937 & 0.007812 & -0.006250 & -0.005469 \\
9+
Targeted MLP @ layer34 & 0.775000 & 0.203125 & 0.115625 & 0.064062 & 0.098438 & 0.050781 \\
10+
Reference layer16 & 0.012500 & 0.006250 & 0.014063 & 0.013281 & -0.003125 & 0.000000 \\
11+
Reference layer34 & 0.387500 & 0.103125 & 0.060937 & 0.036719 & 0.043750 & 0.023438 \\
12+
Reference stride & 0.237500 & 0.068750 & 0.046875 & 0.028906 & 0.029687 & 0.015625 \\
1313
\bottomrule
1414
\end{tabular}

0 commit comments

Comments
 (0)