STR Coherence Signal

This repository reproduces the results in:

Trajectory-Level Coherence in Language Models via Soft Topological Return

Haelio Tang · April 2026

Overview

We introduce the Soft Topological Return (STR), a kernel-based recurrence measure that captures trajectory-level coherence in language model hidden states. STR distinguishes coherent from conflicting generation dynamics with statistical significance ($p = 0.0013$, Cohen's $d_z = 0.48$, $N = 50$).

Setup

pip install -r requirements.txt

Requirements: Python 3.9+, ~500MB disk for GPT-2 weights (downloaded automatically on first run).

Run

# Full experiment: generates results + figures
python run.py

# Plot only (from cached results)
python run.py --plot-only

Output

File	Description
figures/fig1.png	3-panel figure: scatter, histogram, bar chart
figures/fig2.png	Per-sample waterfall of STR differences
data/results/bridge_50.json	Raw numerical results

Key Results

Metric	Value
Correct > Conflict	36/50 (72%)
Mean STR (correct)	0.2221 ± 0.0171
Mean STR (conflict)	0.2109 ± 0.0212
Paired t-test	t = 3.40, p = 0.0013
Cohen's d_z	0.48 (medium)

Repository Structure

str-coherence/
├── README.md
├── requirements.txt
├── run.py                  ← one-command reproduction
├── core/
│   ├── __init__.py
│   ├── str.py              ← STR computation (adaptive σ)
│   └── trajectory.py       ← hidden state extraction
├── data/
│   └── results/            ← output JSON
└── figures/                ← output figures

Notes

• Uses GPT-2 (124M parameters, local). No API key required.
• Fully deterministic: greedy decoding, no sampling.
• Model weights are downloaded automatically via HuggingFace transformers.
• Runs in ~10 minutes on CPU; ~2 minutes with GPU.

License

MIT

Citation

@article{tang2026str,
  title={Trajectory-Level Coherence in Language Models via Soft Topological Return},
  author={Tang, Haelio},
  year={2026},
  note={arXiv preprint}
}