Skip to content

Add ABIDES arena#104

Open
Muhtasham wants to merge 1 commit intoCodeClash-ai:mainfrom
Muhtasham:feat/abides-arena
Open

Add ABIDES arena#104
Muhtasham wants to merge 1 commit intoCodeClash-ai:mainfrom
Muhtasham:feat/abides-arena

Conversation

@Muhtasham
Copy link
Copy Markdown
Contributor

@Muhtasham Muhtasham commented May 5, 2026

Summary

  • add an ABIDES financial-market simulation arena backed by abides-sim/abides
  • pin upstream ABIDES to c4bf157678928934417aba6073eb0651aeaf6d15, constrain Python dependencies, and pin pip in the arena image
  • add the runtime adapter, starter abides_agent.py, example config, arena docs, and unit coverage

Runtime behavior

  • evaluates each CodeClash player in its own identical seeded ABIDES market world per sim, with an exchange, market maker, and background zero-intelligence traders
  • compares players by average mark-to-market profit across the configured market seeds
  • computes scores from the player object holdings and final exchange price rather than mutable ABIDES agent.type / kernel.meanResultByAgentType accounting
  • guards player lifecycle callbacks so one crashing submission receives CRASH_SCORE without crashing other players
  • reloads player modules per sim, discards simulator stdout to /dev/null, and isolates ABIDES summary logs in temporary directories so runtime artifacts do not leak into the repo workspace

Hardening

  • fixed validation-order bypass by importing the real ABIDES TradingAgent before executing the submitted module
  • penalized missing player scores with CRASH_SCORE instead of treating absent ABIDES results as neutral 0.0
  • changed from shared-kernel submitted-agent competition to isolated per-player worlds to avoid direct opponent-object access through self.kernel.agents
  • added regression coverage for validation heredoc ordering and missing score handling

Verification

  • uv run pytest tests/arenas/test_abides.py -> 7 passed
  • uv run pytest tests/arenas -> 190 passed
  • uv run pytest -> 192 passed
  • uv run mkdocs build
  • uv run pre-commit run --all-files
  • uv run python main.py configs/examples/ABIDES__dummy__r1__s2.yaml -o /tmp/codeclash-abides-smoke-review-1777981176
    • rebuilt codeclash/abides from scratch
    • both players passed validation
    • produced round scores, e.g. -1736.0 vs -2297.0 depending on player ordering
    • produced metadata.json, game.log, tournament.log, and rounds/round_0.tar.gz / rounds/round_1.tar.gz
  • manual runtime crash-isolation check with one agent raising in wakeup gave the bad agent -1000000.0 while the good agent still scored normally
  • test ! -e log after local runtime/smoke checks confirmed no ABIDES log artifact leaked into the repo root

@Muhtasham Muhtasham force-pushed the feat/abides-arena branch from 54128a6 to bbc23f7 Compare May 5, 2026 11:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant