Evgenii Vyaltsev (ORCID 0009-0004-3712-6798), Daniil Vyaltsev — June 2026.
A prior gate showed implicit multi-hop reasoning at 3B is retrieval-and-elimination, with real traversal only as written chain-of-thought (self-delivery). One door stayed open: maybe a latent scratchpad exists that observation just doesn't surface. So we search for it — by optimization, with frozen weights — and ask whether the state the optimizer finds composes or merely injects the answer.
With weights frozen, a compact latent state H* (k virtual reasoning vectors injected between the cached context and the question) is gradient-optimized to make failed implicit chains emit the absent target D. Because H* has k×d free params (enough to encode the answer), five pre-registered controls separate a composition mode from memorized RAG. The optimizer makes the chains emit D (k=1 already suffices) — but:
| control | signal | Llama-3.2-3B | Qwen2.5-3B | composition needs |
|---|---|---|---|---|
| C1 | code-fact REMOVED, still emits D | 0.94 | 0.81 | ≈ 0 |
| C4 | P(B) raw → under-H* | 0.0012 → 0.0003 | 0.0000 → 0.0001 | rise from 0 |
| C4 | P(C) raw → under-H* | 0.0001 → 0.0001 | 0.0000 → 0.0000 | rise from 0 |
| C3 | false-hop follow rate | 0.00 | 0.00 | ≈ 0.80 (as CoT) |
| C2 | transfer to chain Y / breaks | 0.12 / 0.88 | 0.12 / 0.88 | high / low |
| C5 | eff-dim({H*_i}) | 13/16 | 13/16 | ≪ n |
Verdict: LEARNED-RAG, instance-specific — no latent composition mode at 3B. H* carries the answer with the chain's code fact removed (C1), leaves the intermediates lens-invisible (C4), is inert to hop corruption (C3), does not transfer (C2/C5). A CoT-informed initialization does not rescue it (ruling out an optimization-barrier / reachability artifact). Replicates cross-model.
This inverts the burden of proof: not "we didn't observe a latent composition state" but "we optimized directly for one, with maximal freedom, and the best solution reachable is answer-injection." A negative existence result, not a failure to observe.
| gate | lever | verdict |
|---|---|---|
| γ (DOI 10.5281/zenodo.20514229) | attention | TRANSPORT |
| delivery ladder | internal skill shortcut | NO — fix is delivery |
| synthesis | implicit reasoning | retrieval+elimination; CoT = self-delivery |
| latent scratchpad (this paper) | optimized latent state | answer-injection, not composition |
This model class delivers, and composes only through an external buffer (the written scratchpad); it does not internally compute — and a latent internal scratchpad cannot be optimized into existence at 3B.
3B, synthetic attr→person→zone→code chains, failed-implicit population, N≤4096, greedy, k≤8, Layer-0/full-unroll injection. A NULL = "no compact latent composition state at 3B on this design", NOT "never". Deep mid-stack injection and k=32 deferred (k=1 already saturates accuracy → capacity is not the bottleneck). The null does not transfer up.
paper6_latent_scratchpad.tex / .pdf preprint (6 pp)
make_figures.py recomputes figures from data/
figures/ fig1 C1 capacity, fig2 C4 lens, fig3 C2/C3
latent_scratchpad.py the optimizer (frozen weights, backprop to H* only)
lens_chain.py the logit-lens harness (C4)
grep_check.md confirms no kernel/bundle/sparse edit (vanilla bases)
PREREGISTRATION/PREREGISTRATION.md five controls + verdict map, fixed before running
data/
scratchpad_results.jsonl per-model aggregates (acc_k, C1–C5, cot-init)
per_case_*.csv raw per-case logs (both models)
python3 -m pip install matplotlib numpy
python3 make_figures.py
pdflatex paper6_latent_scratchpad.tex && pdflatex paper6_latent_scratchpad.texText/figures/data: CC-BY-4.0. Code: MIT.
See CITATION.cff; full BibTeX added once the Zenodo DOI is minted.