RetroAGI

General purpose machine learning agent for retro games.

RetroAGI is organized as a progressive-resolution curriculum for training and promoting an architecture toward full Super Mario Bros:

1. Synthetic 1D fully validates the architecture on procedural sequence data: tensor contracts, hierarchy behavior, losses, gradients, baselines, primitive labels, k-step world-model outcomes, checkpointing, and deterministic metrics.
2. Block SMB trains all trainable game-facing models on a simplified synthetic version of SMB: Block ViT perception plus the hierarchical actor/world-model/critic policy in fast scenario-driven tasks.
3. Full SMB asset-mock perception bootstraps the Full SMB ViT with full game assets arranged into synthetic scenarios before any policy relies on full emulator observations.
4. Full SMB verifies and validates inference in the emulator, then continues training the transferred models at full fidelity.

The stage code is separated, but all stages share the same core contract:

observation -> hierarchical actor -> action
observation + action -> world model prediction
prediction -> critic -> actor refinement
carried LSTM state -> next initial actor decision

Shared actor/world-model/critic components live in retroagi/core. Stage adapters live in retroagi/stages/* and convert stage-native observations into the common A/B/C timescale tensors.

Project Layout

retroagi/
  core/
    interfaces.py      # StageSpec, StageBatch, shared adapter protocol
    models.py          # reusable actor, world model, critic, controller
  stages/
    synthetic_1d/      # procedural one-dimensional validation
    block_smb/         # pygame-ce SMB-like scenarios and adapter
    full_smb/          # stable-retro adapter and emulator runner
scripts/               # compatibility wrappers and older experiments

Architecture

See the architecture diagram for the hierarchical actor/world-model/critic flow.

Current Status

• Synthetic 1D is the architecture validation stage and is reproducible from a clean checkout with deterministic datasets, baselines, checkpoint save/restore, primitive-control supervision, k-step primitive-outcome supervision, and held-out metrics.
• Block SMB is the simplified synthetic game-training stage. It has a pygame-ce environment, fixed success thresholds, Block ViT perception, policy training, evaluation, resume, recording, ablations, structured logs, and optional TensorBoard or W&B tracking. Generated scenarios now use the versioned block_smb_mc_v1 Monte Carlo distribution so policy training can cover parameterized ground-truth tasks rather than only a small fixed set. Fresh Block SMB train/distill CLI runs use the initial real-volume MC target of 512 train, 128 validation, and 256 test samples, with failure-focused train oversampling biased toward full_smb_opening_proxy, unless a smoke/sweep override is passed.
• Full SMB has the asset-mock ViT bootstrapping rung, a stable-retro stage adapter, headless smoke evaluation, Block SMB policy transfer, continued Full SMB training, transfer-vs-scratch comparison tooling, and a documented local benchmark run at artifacts/full_smb/documented_benchmark_seed0/.
• Operations are covered by CI, native install instructions, stage operations guidance, and a clean-checkout reproducibility procedure.

Supported Platforms

RetroAGI supports Linux x86-64 and macOS Apple Silicon with Python 3.12 through 3.14. It pins PyTorch 2.9.1 with torchvision 0.24.1. CPU-only execution is the baseline; CUDA and Apple Metal/MPS acceleration are selected automatically when available. CUDA 12.8 is the primary Linux GPU target, CUDA 13.0 is the secondary Linux GPU target, and MPS is the native macOS GPU target.

Reference training runs were performed on an Intel NUC with a 12th-generation Intel processor and 64 GB of RAM, connected over Thunderbolt to a Razer Core X V2 external GPU enclosure housing an NVIDIA Tesla V100 with 32 GB of memory.

See the compatibility matrix and installation commands before creating an environment.

The stage semantics define observations, actions, rewards, episode endings, and resets across the curriculum.

The Block SMB success thresholds define the deterministic fixed-scenario bar for considering a policy known-good. The Block SMB Monte Carlo curriculum plan defines the generalization gate: parameterized scenario families, sampled train/validation/test/stress splits, coverage metrics, held-out evaluation, and distribution-level promotion criteria before Full SMB transfer. The universal retro oracle roadmap lays out the path from scripted Block SMB oracle labels to a learned cross-game AI teacher that improves through maximum-likelihood training and EM relabeling. The current known-good baseline is a scripted policy artifact at artifacts/block_smb/known_good_scripted_seed20260622/. Full SMB cannot commit ROM-derived emulator artifacts, so its checked-in counterpart is the documented local benchmark manifest at artifacts/full_smb/documented_benchmark_seed0/benchmark_manifest.json.

The operations reference lists hardware expectations, runtime targets, metrics, and artifact locations for each stage.

The reproducibility procedure starts from a clean checkout and walks through installation, tests, smoke training, and artifact verification.

The tensor contracts define hierarchy and vision shapes, dtypes, normalization ranges, and timescales.

The AI teaching curriculum provides a 10-week path for using the project to teach practical AI engineering.

Usage

1. Create and activate a supported Python environment using the commands in docs/compatibility.md. For a fully traceable run from a fresh clone, follow docs/reproducibility.md.
2. Run a curriculum stage:

   python -m retroagi.stages.synthetic_1d.train
   retroagi train --game smb --stage block --epochs 5 \
     --vision-checkpoint data/block_vit/block_vit.pth \
     --checkpoint data/block_smb/policy.pth \
     --output artifacts/block_smb/latest/run_summary.json
   retroagi resume --game smb --stage block \
     --checkpoint data/block_smb/policy.pth --epochs 10
   retroagi evaluate --game smb --stage block \
     --checkpoint data/block_smb/policy.pth
   retroagi record --game smb --stage block \
     --checkpoint data/block_smb/policy.pth \
     --record-dir artifacts/block_smb/recordings
   retroagi transfer --game smb --stage full \
     --block-policy-checkpoint data/block_smb/policy.pth \
     --full-smb-vision-checkpoint data/vit/full_smb_vit.pth \
     --output-checkpoint data/full_smb/transferred_policy.pth
   retroagi train --game smb --stage full \
     --init-checkpoint data/full_smb/transferred_policy.pth \
     --full-smb-vision-checkpoint data/vit/full_smb_vit.pth \
     --perception-mode freeze \
     --updates-per-epoch 1 \
     --rollout-steps 64 \
     --checkpoint data/full_smb/policy.pth
   retroagi evaluate --game smb --stage full \
     --checkpoint data/full_smb/policy.pth \
     --evaluation-episodes 3
   retroagi record --game smb --stage full \
     --checkpoint data/full_smb/policy.pth \
     --record-dir artifacts/full_smb/recordings \
     --recording-path artifacts/full_smb/recording_manifest.npz
   retroagi play --game smb --stage full \
     --checkpoint data/full_smb/policy.pth \
     --task-set fixed_benchmark \
     --level 1-1 \
     --frame-skip 4 \
     --action-repeat 2 \
     --render-mode human \
     --deterministic-policy \
     --inspection-overlay \
     --render \
     --fps 30 \
     --record-output artifacts/full_smb/play_manifest.npz
   retroagi play --game smb --stage full \
     --human \
     --task-set smoke \
     --level 1-1 \
     --render-mode human \
     --render \
     --fps 30
   retroagi compare --game smb --stage full \
     --transfer-checkpoint data/full_smb/transferred_policy.pth \
     --scratch-trained-checkpoint data/full_smb/scratch_policy.pth \
     --fine-tuned-checkpoint data/full_smb/fine_tuned_policy.pth \
     --known-good-checkpoint data/full_smb/known_good_policy.pth \
     --task-set fixed_benchmark \
     --seed 0 \
     --seed 1 \
     --output artifacts/full_smb/policy_suite_comparison.json
   retroagi evaluate --game smb --stage full --steps 500 --seed 0

   The retroagi command is the preferred entry point for selecting a game and fidelity rung. Use --game smb --stage synthetic|block|full; legacy stage aliases such as block-smb and full-smb remain accepted. Stage-specific options are forwarded to the selected implementation; the legacy retroagi-block-smb command remains available for Block SMB-only workflows. Full SMB commands accept --task-set, --task, --level, --state, --scenario, and --frame-skip to select repeatable task starts. Full SMB play mode also accepts --render-mode human|none, --action-repeat, --deterministic-policy or --sampling-policy, and --record-output for the playback manifest. Add --inspection-overlay to stream a compact policy-inspection HUD to stderr with action probabilities, reward terms, score/progress signals, termination reason, and fixed-task threshold gates. Preserved Full SMB runs should use the canonical layout from full_smb_artifact_layout("<run>") under artifacts/full_smb/<run>/ for summaries, logs, recordings, videos, evaluation reports, comparisons, tracking output, and checkpoint copies. Full SMB policy training records its perception choice in every checkpoint: --perception-mode freeze reuses a frozen Full SMB ViT checkpoint, fine_tune includes trainable ViT parameters in the optimizer, and replace starts from a fresh trainable Full SMB ViT. Full SMB trainer checkpoints also capture the resolved rollout/update shape, vector environment request, reward config, loss weights, recording paths, tracking config, deterministic mode, training source provenance, and the validated Full SMB A/B/C stage-batch contract. Scratch training is selected by omitting --resume and --init-checkpoint, which builds the policy via the shared architecture factory before consuming Full SMB stage batches. --init-checkpoint starts transferred-policy fine-tuning from either a raw Block SMB policy checkpoint or an existing Full SMB transfer checkpoint. Rollouts carry recurrent world-model state only while the episode continues and drop it on manual reset, death, timeout, completion, game over, termination, or truncation boundaries. Training summaries and checkpoints now include compact rollout replay records with actions, rewards, done/truncated flags, episode masks, boundary reasons, scenario/task/emulator-state IDs when available, selected Full SMB signals, and reward terms. Full SMB training also records numerical safety settings and metrics: finite loss/logit/gradient checks, gradient clipping, reward-scale bounds, reward/value prediction bounds, action entropy, and early-stop reasons for NaN or exploding values. Real Full SMB emulator runs require the optional stable-retro backend: pip install ".[full-smb]". Unit tests, Block SMB training, and CI smoke training do not install or build that native emulator dependency. Each versioned checkpoint also writes a JSON sidecar beside the .pth file containing the resolved config, metrics, code revision, runtime environment, specs, metadata, and state keys. Block SMB ablations can be run with paired switches such as --disable-vision, --disable-world-model, --disable-critic-feedback, --disable-hierarchy, --disable-recurrent-state, and --disable-checkpoint-transfer. The resolved ablation config is written beside each run summary and checkpoint. Training can write structured JSONL events with --log-path; deterministic evaluation cadence is controlled with --evaluation-interval-epochs. Optional TensorBoard and Weights & Biases tracking can be installed with pip install ".[tracking]" and enabled with --tracking-backend tensorboard|wandb. Low-level controller gain schedules are selectable with --controller-schedule constant|linear. The Full SMB random-agent runner is headless by default; pass --render only for local visual inspection. Full SMB policy transfer reuses Block SMB actor/world-model/critic weights. Before that transferred policy is used for Full SMB inference or continued training, the Full SMB ViT must be bootstrapped on synthetic scenarios made from full-game assets and loaded as the versioned Full SMB ViT checkpoint. Transfer comparisons evaluate the transferred policy and a scratch Full SMB baseline on identical seeded observation batches. Learned-dynamics imagination is selectable with --imagined-rollout-horizon and --imagined-rollout-weight. Target-network stabilization is selectable with --target-network-mode, --target-network-tau, and --target-network-instability-threshold.
3. Run the test suite:

   python -m unittest discover -s scripts/tests -v

Legacy wrappers still work:

python scripts/simple_transformer.py
python scripts/run.py

Training

Train the Block SMB vision transformer directly from procedural pygame-ce rollouts. Semantic masks and Mario positions are generated exactly from the renderer palette:

python scripts/vit/train_block_vit.py \
  --epochs 20 \
  --samples-per-epoch 2048 \
  --val-samples 512

The best checkpoint and its JSON metrics are written to data/block_vit/. Training can be continued with:

python scripts/vit/train_block_vit.py --epochs 40 --resume data/block_vit/block_vit.pth

Synthetic 1D validation

Stage 1 trains the shared hierarchical actor/world-model/critic stack on synthetic data. It now mirrors the Block SMB controller contract: the B-level primitive decoder is supervised for button combo, hold duration, release, cancel, replan, post-release action, and hazard-window timing, and the LSTM world model predicts k-step primitive outcomes for progress, support loss, collision/death risk, terminal outcome, continue, cancel, and replan. The deterministic validation path is covered by:

timeout 120s python3 -m unittest -v scripts.tests.test_synthetic_1d

Expected CPU runtime for this Stage 1 suite is under 20 seconds on the current development machine; the held-out baseline demonstration itself runs in about 7 seconds. It trains a small AgentWorldModelCritic on deterministic Synthetic 1D train data and evaluates on the fixed test split. The expected result is that the trained policy's held-out controller_mse is below both declared baselines: seeded random and train-marginal simple, with an asserted margin of at least 25% below the simple baseline. The same suite also verifies finite CPU gradients, decreasing total loss, deterministic data/permutation seeding, shared-schema checkpoint save/restore, and these evaluation metrics: controller_mse, controller_mae, controller_rmse, error_B, and accuracy_A. Training history additionally records loss_primitive_labels and loss_primitive_outcome so architecture ideas fail early if the primitive contract does not learn on perfectly known 1-D ground truth.

Block SMB policy training uses AgentWorldModelCritic through the shared stage contract and can be run, resumed, evaluated, and recorded from the CLI. Full SMB keeps emulator interaction isolated behind retroagi/stages/full_smb and supports deterministic smoke evaluation plus transferred-policy comparison.