feat: persistent baseline cache (CACHEON_BASELINE_CACHE_DIR)

[ai-hpc]

Summary

• Adds CACHEON_BASELINE_CACHE_DIR env var to validator/config.py
• validator/gpu_eval.py: loads BaselineCache JSON before calling run_baseline(); saves after a fresh run
• tests/baseline_cache/README.md: format docs, usage, and key derivation explanation

Motivation

The vLLM v0.22.0 baseline container takes ~12 minutes per eval round (10 min model load + 2 min inference). With this cache, repeated runs against the same (block_hash, baseline_digest, PROMPT_ENGINE_VERSION) skip the container entirely — critical for fast iteration during miner development.

Usage

python3 scripts/run_validator_eval.py \
  --block-hash 0x<hash> \
  --miner-image <image> \
  --model-volume /models \
  --baseline-cache-dir ./tests/baseline_cache

First run saves tests/baseline_cache/<key>.json. Subsequent runs load it, printing:

INFO Loaded baseline from cache: key=5ea34e493462daee file=... (10 prompts)

Test plan

• Run eval once with --baseline-cache-dir → verify JSON written + log line
• Run again → verify baseline container NOT started + same results

[xavierlyu]

That script doesn't exist in the repo. gpu_eval is an env-var-driven container entrypoint (python -m validator.gpu_eval)

[xavierlyu]

This never hits in production because each block is 12s. the comment is a bit misleading.

This is fine if the intent is local dev/CI only (the README does say so), but the inline comment in gpu_eval.py reads like a prod optimization: # Baseline cache: skip the 12-min vLLM container if a cached result # exists for this (block_hash, baseline_digest, prompt_version) key

[xavierlyu]

The baseline is the teacher-forcing reference that every challenger is scored against. Loading it from an unsigned JSON file on disk (gated only by a single env var) means anyone who can write that file controls scoring. If CACHEON_BASELINE_CACHE_DIR is accidentally set on a real validator (shared .env, misconfigured deploy), it silently loads whatever is in that directory with no validation that the prompts match the current block. For a subnet where this feeds directly into weight-setting, that's a meaningful risk

[xavierlyu]

i think tests/baseline_cache/ should be gitignored