Skip to content

Release NetOpsBench 0.2 scalable benchmark runtime - #6

Merged
yyyyyt123 merged 38 commits into
mainfrom
codex/benchmark-simulator-cleanup
Jul 28, 2026
Merged

Release NetOpsBench 0.2 scalable benchmark runtime#6
yyyyyt123 merged 38 commits into
mainfrom
codex/benchmark-simulator-cleanup

Conversation

@yyyyyt123

Copy link
Copy Markdown
Collaborator

Summary

This PR delivers the NetOpsBench 0.2 benchmark/runtime refactor and validated large-topology support:

  • shared IncidentEngine and public simulator SDK contracts;
  • native Rust Pingmesh and background traffic client agent;
  • stable xlarge and fat-tree K8/K12 topology profiles and tier-aware fault placement;
  • scale-aware Pingmesh, BGP, traffic, recovery, cleanup, and quarantine semantics;
  • high-fanout device-down lifecycle and corrected large profile;
  • removal of superseded internal paths and obsolete tests.

This is an intentional 0.2 alpha API/runtime boundary cleanup. NetOpsBench-RL remains outside this repository and consumes the public SDK.

Validation

  • 744 passed, 7 deselected non-real Python tests
  • Python coverage: 83.34%
  • Ruff, Black, configured mypy boundary: passed
  • Rust fmt, Clippy, and 29 tests: passed
  • wheel import and CLI smoke outside checkout: passed
  • native client image smoke: passed
  • git diff --check: passed

Real infrastructure validation completed for xlarge, fat-tree K8/K12, and the corrected large profile. The final large gate verified 64 clients, 256 background flows, complete 4,032-path and 16,128 pair-port Pingmesh coverage, clean healthy observations, leaf/spine device-down behavior, recovery, and residue-free teardown.

The completed xlarge DeepSeek campaign produced 70/70 case-valid, cleanup-successful, coverage-complete cases. Agent accuracy remains a model baseline result rather than a release gate.

Known follow-up

A deliberately broader, non-CI mypy scan reports nine annotation-only issues in topology builders. They do not affect runtime behavior or the configured release gate and can be handled in a separate type-only cleanup.

yyyyyt123 added 30 commits July 19, 2026 23:19
@yyyyyt123
yyyyyt123 merged commit 7fe1e80 into main Jul 28, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant