Skip to content

Add ABIDES arena#104

Open
Muhtasham wants to merge 6 commits into
CodeClash-ai:mainfrom
Muhtasham:feat/abides-arena
Open

Add ABIDES arena#104
Muhtasham wants to merge 6 commits into
CodeClash-ai:mainfrom
Muhtasham:feat/abides-arena

Conversation

@Muhtasham

@Muhtasham Muhtasham commented May 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add an ABIDES financial-market simulation arena backed by abides-sim/abides
  • pin upstream ABIDES to c4bf157678928934417aba6073eb0651aeaf6d15, constrain Python dependencies, and pin pip in the arena image
  • expose a restricted CodeClash policy protocol: submitted abides_agent.py defines decide(observation) and returns declarative buy/sell limit-order intents
  • add the trusted runtime adapter, starter policy, example config, arena docs, and unit coverage

Runtime Behavior

  • evaluates each CodeClash player in identical seeded ABIDES market worlds with an exchange, market maker, and background zero-intelligence traders
  • compares players by average mark-to-market profit across configured market seeds
  • keeps ABIDES kernel, exchange, ledgers, order books, and order construction inside trusted runtime code
  • calls submitted decide(observation) code out-of-process with a per-decision timeout and passes only plain observation dictionaries
  • validates and clamps returned order intents before submitting trusted ABIDES LimitOrders
  • computes scores from trusted exchange ORDER_EXECUTED messages plus final exchange price, not from mutable submitted-code state
  • runs each player simulation in a child process with a per-player timeout so import/runtime hangs receive CRASH_SCORE without stalling the whole round
  • prepends the submitted file's parent directory during policy import so normal multi-file submissions like from helper import X work

Hardening

  • replaced the native TradingAgent subclass submission contract with a restricted protocol boundary
  • added validation_timeout, decision_timeout, and player_timeout config knobs
  • changed missing abides_results.json handling from neutral 0.0 ties to CRASH_SCORE with error details
  • uses isolated per-player worlds to avoid direct opponent-object access while keeping matched seeds/configs for fair score comparison
  • retains trusted execution-ledger checks so only simulator-processed player orders affect scored cash/shares

Verification

  • uv run ruff check codeclash/arenas/abides/abides.py codeclash/arenas/abides/runtime/run_abides.py tests/arenas/test_abides.py -> passed
  • uv run pytest -q tests/arenas/test_abides.py -> 9 passed
  • uv run pytest -q tests/arenas -> 194 passed
  • uv run pytest -q -> 196 passed
  • uv run pre-commit run --files codeclash/arenas/abides/abides.py codeclash/arenas/abides/runtime/run_abides.py codeclash/arenas/abides/runtime/abides_agent.py codeclash/arenas/abides/runtime/README.md tests/arenas/test_abides.py configs/examples/ABIDES__dummy__r1__s2.yaml docs/reference/arenas/abides.md -> passed
  • docker build -t codeclash/abides -f codeclash/arenas/abides/ABIDES.Dockerfile . -> passed
  • direct Docker runtime smoke with two starter policies:
    • both players produced per-sim status: "ok"
    • orders_submitted: 18, policy_errors: 0
    • average scores were -900.0 for both players
  • full CodeClash dummy smoke: uv run python main.py configs/examples/ABIDES__dummy__r1__s2.yaml -o "$SMOKE_DIR"
    • both players passed validation
    • both rounds completed as ties
    • per-sim details included trusted ledger fields cash, shares, orders_submitted, policy_errors, and status
  • adversarial direct Docker checks:
    • impossible huge quantity/price intents were clamped/bounded and produced normal finite scores/positions, not inflated profit
    • infinite-loop decide() calls hit the decision timeout and did not stall the simulation

Note: uv run mkdocs build --strict was previously blocked by pre-existing docs warnings about README.md/nav entries; normal docs build was already verified earlier in this PR.

@Muhtasham Muhtasham force-pushed the feat/abides-arena branch from 54128a6 to bbc23f7 Compare May 5, 2026 11:42
@Muhtasham Muhtasham force-pushed the feat/abides-arena branch from 2879449 to f9ccb8f Compare June 23, 2026 01:01
@Muhtasham Muhtasham force-pushed the feat/abides-arena branch from f9ccb8f to 6adff22 Compare June 24, 2026 22:49
@Muhtasham Muhtasham requested a review from john-b-yang June 25, 2026 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant