Skip to content

unixsysdev/llm_tests

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 

Repository files navigation

LLM Tests

Experiments for understanding and manipulating reasoning in language models, specifically Qwen3-0.6B.

Projects

This repository contains two related projects exploring different aspects of how LLMs process and generate reasoning.

1. Model View (model_view/)

Layer-by-layer token prediction visualization.

Generates responses while tracking what each layer of the model would predict at every step. This reveals how information flows through the network and when the model "knows" the answer.

Key features:

  • Shows token predictions from all 28 layers simultaneously
  • Supports multiple sampling presets (deterministic, balanced, creative, chaotic)
  • Compares reasoning vs non-reasoning mode outputs

Example insight: Early layers predict random multilingual tokens, middle layers start showing relevant content, and only the final layers produce coherent answers.

cd model_view
pip install -r requirements.txt
python layer_full_generation.py

See model_view/README.md for details.

2. Model Tests (model_tests/)

Reasoning direction intervention experiments.

Replication of the R1D1 paper "Is Reasoning in Language Models Mediated by a Single Direction?" The goal is to find and manipulate a "reasoning direction" in the model's activation space.

Key experiments:

  • Extract activations during thinking vs non-thinking generation
  • Compute a reasoning direction (difference vector)
  • Test interventions: Can we suppress reasoning? Can we induce it?
cd model_tests
pip install -r requirements.txt
python run_experiment.py --quick --device mps

See model_tests/README.md for details.

Model

Both projects use Qwen3-0.6B with its native thinking mode toggle:

  • enable_thinking=True: Model uses <think>...</think> tags for reasoning
  • enable_thinking=False: Model gives direct answers

This provides a clean comparison since the same model architecture is used in both modes.

Requirements

  • Python 3.10+
  • PyTorch with MPS/CUDA support
  • Transformers library
  • ~2GB disk space for model weights

Hardware

Tested on Apple Silicon (MPS). Should work on CUDA with --device cuda.

About

Mechanistic-interpretability experiments for visualizing and manipulating Qwen reasoning

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages