Skip to content

Seqev/latent-scratchpad-search

Repository files navigation

No Latent Scratchpad — the last open door, closed

DOI

Evgenii Vyaltsev (ORCID 0009-0004-3712-6798), Daniil Vyaltsev — June 2026.

A prior gate showed implicit multi-hop reasoning at 3B is retrieval-and-elimination, with real traversal only as written chain-of-thought (self-delivery). One door stayed open: maybe a latent scratchpad exists that observation just doesn't surface. So we search for it — by optimization, with frozen weights — and ask whether the state the optimizer finds composes or merely injects the answer.

TL;DR — we optimized for it and it's still delivery (both models)

With weights frozen, a compact latent state H* (k virtual reasoning vectors injected between the cached context and the question) is gradient-optimized to make failed implicit chains emit the absent target D. Because H* has k×d free params (enough to encode the answer), five pre-registered controls separate a composition mode from memorized RAG. The optimizer makes the chains emit D (k=1 already suffices) — but:

control signal Llama-3.2-3B Qwen2.5-3B composition needs
C1 code-fact REMOVED, still emits D 0.94 0.81 ≈ 0
C4 P(B) raw → under-H* 0.0012 → 0.0003 0.0000 → 0.0001 rise from 0
C4 P(C) raw → under-H* 0.0001 → 0.0001 0.0000 → 0.0000 rise from 0
C3 false-hop follow rate 0.00 0.00 ≈ 0.80 (as CoT)
C2 transfer to chain Y / breaks 0.12 / 0.88 0.12 / 0.88 high / low
C5 eff-dim({H*_i}) 13/16 13/16 ≪ n

Verdict: LEARNED-RAG, instance-specific — no latent composition mode at 3B. H* carries the answer with the chain's code fact removed (C1), leaves the intermediates lens-invisible (C4), is inert to hop corruption (C3), does not transfer (C2/C5). A CoT-informed initialization does not rescue it (ruling out an optimization-barrier / reachability artifact). Replicates cross-model.

This inverts the burden of proof: not "we didn't observe a latent composition state" but "we optimized directly for one, with maximal freedom, and the best solution reachable is answer-injection." A negative existence result, not a failure to observe.

The chain arc is closed

gate lever verdict
γ (DOI 10.5281/zenodo.20514229) attention TRANSPORT
delivery ladder internal skill shortcut NO — fix is delivery
synthesis implicit reasoning retrieval+elimination; CoT = self-delivery
latent scratchpad (this paper) optimized latent state answer-injection, not composition

This model class delivers, and composes only through an external buffer (the written scratchpad); it does not internally compute — and a latent internal scratchpad cannot be optimized into existence at 3B.

Scope (strict)

3B, synthetic attr→person→zone→code chains, failed-implicit population, N≤4096, greedy, k≤8, Layer-0/full-unroll injection. A NULL = "no compact latent composition state at 3B on this design", NOT "never". Deep mid-stack injection and k=32 deferred (k=1 already saturates accuracy → capacity is not the bottleneck). The null does not transfer up.

Repository layout

paper6_latent_scratchpad.tex / .pdf   preprint (6 pp)
make_figures.py                       recomputes figures from data/
figures/                              fig1 C1 capacity, fig2 C4 lens, fig3 C2/C3
latent_scratchpad.py                  the optimizer (frozen weights, backprop to H* only)
lens_chain.py                         the logit-lens harness (C4)
grep_check.md                         confirms no kernel/bundle/sparse edit (vanilla bases)
PREREGISTRATION/PREREGISTRATION.md    five controls + verdict map, fixed before running
data/
  scratchpad_results.jsonl            per-model aggregates (acc_k, C1–C5, cot-init)
  per_case_*.csv                      raw per-case logs (both models)

Reproduce

python3 -m pip install matplotlib numpy
python3 make_figures.py
pdflatex paper6_latent_scratchpad.tex && pdflatex paper6_latent_scratchpad.tex

License

Text/figures/data: CC-BY-4.0. Code: MIT.

Citation

See CITATION.cff; full BibTeX added once the Zenodo DOI is minted.

About

We optimize a compact latent state (frozen weights) to force failed multi-hop chains to output the missing answer D. 5 pre-registered controls show it simply injects D: carries it without the code-fact, leaves intermediates invisible, inert to hop corruption, and doesn’t transfer. No latent composition at 3B (Llama-3.2-3B, Qwen2.5-3B).

Topics

Resources

License

Unknown, Unknown licenses found

Licenses found

Unknown
LICENSE-CODE.txt
Unknown
LICENSE-TEXT-DATA.txt

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors