feat(controller): implement control plane in Rust (phases 1–4) + e2e test

[zzylol]

Summary

Implements the control plane described in docs/control-plane-design.md across all four phases, in Rust, verified end-to-end against the custom OTel collector and SDK load generator.

---

Design (from docs/control-plane-design.md)

The controller is the "brain" that observes the query workload and drives configuration across the entire pipeline — from agent OTel collectors to SimpleStore precomputation — to minimize bandwidth and latency while meeting accuracy SLAs.

Four coupled decisions per metric × workload:

1. Sketch vs. raw — based on query type, bandwidth budget, and latency SLA
2. Sketch type — Quantile → DDSketch/KLL, Cardinality → HLL, Frequency → CountSketch/CountMinSketch
3. Dimension aggregation — minimal aggregate_by set to satisfy group-by + filter requirements
4. Time window strategy — window mode when LatencySLA ≥ TimeWindow, batch otherwise

Planning algorithm:

1. Classify aggregation types → candidate sketches
2. Determine minimum label dimensions (GroupByLabels ∪ LabelFilter keys)
3. Score candidates by bandwidth/CPU/accuracy using benchmark table → pick lowest bandwidth meeting AccuracySLA
4. Select window strategy (window vs. batch)
5. Build CollectionPlan → push via HTTP config provider → register precompute jobs
6. Monitor: if actual_accuracy < AccuracySLA → re-plan

Config delivery: The opamp extension is omitted from generated configs because opamp-go expects binary protobuf while the controller speaks JSON, causing handshake errors. Configs are delivered via the OTel HTTP config provider (--config=http://controller:8080/api/v1/config/<metric>) instead.

---

Implementation

Phase 1 — Static planner

• types.rs: domain types (QueryWorkload, CollectionPlan, AgentCollectorConfig, SketchType, etc.)
• analyzer.rs: QuerySpec → QueryWorkload with duration parsing and label deduplication
• planner/rules.rs: deterministic sketch selection (Quantile→DDSketch, Cardinality→HLL, Frequency→CountSketch)
• config/agent.rs: full OTel collector YAML generation (receivers, processors, exporters, service pipeline)
• config/backend.rs: backend merge-processor YAML generation
• store/mod.rs: plan store with rollback and TTL expiry
• opamp/mod.rs: OpAMP WebSocket server (agent registry, push, push_all)

Phase 2 — Cost model

• planner/cost_model.rs: benchmark-driven Pareto selection (lowest bandwidth meeting AccuracySLA); ValidUntil TTL
• QuerySpec.sketch_type override: pin a specific sketch type, bypassing the cost model (useful when a collector supports only a subset)

Phase 3 — Feedback loop

• monitor/mod.rs: async Prometheus scraper; delta-based CPU, bandwidth, and error-rate violation detection; OnViolationFn callback

Phase 4 — Precompute integration

• config/precompute.rs: eligibility check (repeat_every < latency_sla), job builder, PrecomputeClient (register/deregister via reqwest)

HTTP config provider endpoint

• GET /api/v1/config/:metric — serves the full collector YAML for any planned metric, enabling --config=http://controller:8080/api/v1/config/<metric> without requiring OpAMP protobuf

---

Test plan

• 71 unit tests across all modules (cargo test)
• End-to-end test script: tests/otel_controller_e2e_test.sh
  • Starts controller → submits plan → fetches YAML config → starts ddsketchcol via HTTP config provider → runs e2esdkbench (500 series × 25k data-points/s × 30 s) → verifies collector processed data
• Human-readable walkthrough: docs/e2e-test-guide.md

bash tests/otel_controller_e2e_test.sh           # full run (builds both binaries)
bash tests/otel_controller_e2e_test.sh --skip-build  # fast re-run

🤖 Generated with Claude Code

[zzylol]

@milindsrivastava1997 FYI