A decoder-only Transformer built from scratch using CuPy only — no PyTorch, no autograd, no magic. Every forward pass, backward pass, and gradient derived and implemented by hand. Includes full training loop, Adam optimizer, LayerNorm, and causal self-attention on GPU.
deep-learning neural-network transformer gpt language-model backpropagation cupy llms from-scratch-implementation no-pytorch
-
Updated
Mar 30, 2026 - Python