Building a small LLM in PyTorch from scratch - a step-by-step journey inspired by Stanford CS336.

The NoteBook and Assignments implemention via Learning CS336 Spring 2026😛

Comprehensive illustrated notes for Stanford CS336 (Spring 2025) — Language Modeling from Scratch. Live: qihongruan.github.io/cs336/

实现CS336的作业1，并从头开始构建一个transformer模型。Implement CS336's job 1 and build a transformer from scratch

CS336 Final Project - Vietnamese Legal Text Retrieval

Learning CS336...

Rutgers CS:336 | Principles of Info and Data Management Spring 2022

CS336 作业 5：基于 Qwen2.5 模型的 LLM 对齐与推理强化学习。完整实现了监督微调（SFT）与组相对策略优化（GRPO）算法，并在 GSM8K 数据集上完成零样本、在策与离策的训练与评估对比。

Implementation of a Decoder-only Transformer language model from scratch for CS336, featuring a byte-level BPE tokenizer, RoPE, Multi-Head Self-Attention and SwiGLU FFN. Trained on TinyStories with 1.39 Val Loss.

Project for Information Retrieval Courses

Implementation-first Stanford CS336 study repo covering tokenization, transformer/language-modeling fundamentals, training loops, and evaluation notes.

Self-study Stanford CS336: BPE, transformers, systems, scaling laws, data pipelines & GRPO alignment — all assignments verified.

learning cs336

Decoder-only transformer LLM built from scratch in PyTorch — hand-written BPE tokenizer, model, optimizer, and training loop. My implementation of Stanford CS336 Assignment 1

Living comparison table of LLM architectural choices (norm, attention, MoE, positional encoding, and more) from the Original Transformer (2017) to frontier models (2026). Based on Harm de Vries's figure, Sebastian Raschka's Big LLM Architecture Comparison, and Tatsunori Hashimoto's Stanford CS 336 lecture.

160+ pages of handwritten notes from Stanford CS336: Language Modeling from Scratch, covering language modeling, LLM systems, and full-stack AI engineering from data to deployment and evaluation and RL alignment

Curated learning materials for Stanford CS336. Comprehensive resources covering the course's key concepts and practice content.