Skip to content

refactor!: audit and prune redundant planning enums - #447

Merged
zzylol merged 5 commits into
mainfrom
fix/428-prune-planning-enums
Sep 21, 2026
Merged

zzylol merged 5 commits into
mainfrom
fix/428-prune-planning-enums

Conversation

@zzylol

@zzylol zzylol commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Why

Closes #428. Remove redundant planning representations while retaining enums with implemented consumers. The enum audit and consumer migration are included below.

What

  • Remove ExecutableOperator, ExecutableDagNode.operator, MaterializationMedium, and SummaryWindowFramework::Extension(String).
  • Re-export the shared CostUnit from mapping; recurring formulas reject total-cost units.
  • Retain PhysicalHandoffKind, CacheProfile, EvaluationSchedule, QueryTimeScope, Predictability, and ExponentialHistogram because they drive estimation, lifecycle selection, or guarantee validation.

How

Use ExecutableOperatorPayload as the sole operator identity. Keep network/materialization byte dimensions separate, without an unused storage-medium label. Limit window frameworks to those with defined planning semantics.

Before this PR

Compiling a sum into an executable DAG emitted both operator: "SummaryAgg" and payload.kind: "summary_agg"; these could disagree. Declaring a materialization required a medium even though all media used the same byte counter and coefficient. Mapping and shared cost annotations used distinct Rust CostUnit types.

After this PR

The same sum emits only payload.kind: "summary_agg" in a version-2 executable DAG. A materialization is {"kind":"materialization"} and preserves its byte estimate. Mapping and annotations share one cost-unit type; recurring formulas reject CostUnits instead of interpreting a total as a rate.

Evidence

The executable-DAG round-trip test checks payload-only node identity, rejection of the removed field, and rejection of document version 1. Physical handoff integration coverage preserves distinct once-only materialization and per-consumer transfer multiplicities. The rate-unit regression failed before the guard was added.

Verification

  • Unit and end-to-end tests: cargo test --workspace --locked — 1,165 passed.
  • Formatting: cargo fmt --all -- --check — passed.
  • Lint: cargo clippy --workspace --all-targets --all-features --locked -- -D warnings — passed.
  • Viewer: 24 passed, 6 skipped.
  • MetricsQL external-consumer check and vendored-parser baseline verification — passed.

New tests verify that opaque window extensions cannot enter planning and that shared total-cost units cannot enter recurring formulas. Existing DAG and physical handoff tests verify the simplified wire contracts.

Architectural decisions

Keep enums with actual semantic consumers: for example, predictable advance notice enables prepared state, moving real-time lookbacks require deletion support, and exponential histograms require framework-specific accuracy evidence. Remove the opaque window extension because name validation alone supplies no framework semantics.

Planning enum audit and migration

This records the current code audit for issue #428 for library and wire-contract
consumers. Retention decisions depend on implemented consumers, not future uses.

Type or variant Decision Current consumer or redundancy
PhysicalHandoffKind Keep physical_handoff_cost::estimate_physical_handoffs distinguishes network endpoint validation and network bytes from materialization bytes.
CacheProfile Keep analytical_cost resolves cache evidence into hit/miss work; physical_plan_cost_model rejects cache evidence for boundary/storage models that cannot account for it.
SummaryWindowFramework::ExponentialHistogram Keep summary_maintenance_cost/window.rs checks framework-specific accuracy evidence; candidate ranking composes its error with the root guarantee and checks the required accuracy.
SummaryWindowFramework::Extension(String) Remove Only the name was validated; no registered implementation or window-specific semantic validation consumed it.
EvaluationSchedule Keep Lifecycle planning emits one-shot, per-update, or on-read evaluation. The estimator rejects guarantees whose schedule disagrees with lifecycle and data arrival. Prepared state can be one-shot or per-update, depending on arrival.
QueryTimeScope Keep Lifecycle planning requires deletion support for moving real-time/mixed lookbacks, but not historical as-of reads.
Predictability Keep A predictable one-time query with advance notice can use prepared state; its known-at timestamp bounds activation.
ExecutableOperator and ExecutableDagNode.operator Remove Every tag duplicated the tagged payload and allowed inconsistent node states. Match the payload in Rust and read payload.kind in JSON.
Planner-local CostUnit Consolidate Mapping re-exports asap_types::cost::CostUnit. Recurring formulas require CostUnitsPerSecond; totals are rejected.
MaterializationMedium Remove No in-repository estimator or export consumer branches on memory, disk, or object store. All use the same materialization byte counter and coefficient.

Consumer migration

The executable DAG document version is 2. Nodes no longer contain an
operator field: for example, a summary aggregate has only
"payload": {"kind": "summary_agg", ...} for its operator identity. Update
consumers to match ExecutableOperatorPayload or inspect payload.kind.
Version 1 documents are not supported; regenerate them with the new compiler.
The strict node decoder rejects the removed field.

Boundary materialization JSON changes from
{"kind":"materialization","medium":"disk"} to
{"kind":"materialization"}. Rust callers use BoundaryKind::Materialization.
Remove the medium from supplied boundary profiles. Network evidence and byte
counters are unchanged. A boundary is still explicitly declared; ordinary
in-memory dataflow does not become a materialization.

Window frameworks accept tumbling, sliding, and exponential_histogram.
Opaque {"extension":"..."} values are rejected. New frameworks need defined
planning and accuracy semantics before entering this contract.

Existing mapping imports of CostUnit remain valid as re-exports of the shared
type. Its existing shared serde representation is unchanged; as_str() retains
cost_units_per_second and also names totals as cost_units.

Known downstream migration

The local ASAPQuery backend checkout still imports ExecutableOperator in
crates/asap_types/src/executable_plan.rs and constructs the duplicated field
in runtime tests. Those consumers must migrate to the payload before upgrading
the planner dependency. Its window compiler uses Extension only in a test
that rejects an unsupported hierarchical rollup; move that check to invalid
wire input or an unsupported layout. No materialization-medium consumer was
found there. This PR changes ASAPPlanner only.

Limitations and follow-up

Breaking API/wire cleanup: regenerate version-1 executable DAG documents; consumers must match the payload, remove materialization media from profiles, and stop supplying window extensions. The local ASAPQuery backend still imports the old executable operator API and must migrate before upgrading its planner dependency; its only window-extension use is a test rejecting an unsupported layout. No materialization-medium consumer was found there. Backend changes are outside this PR.

Human review — do not complete with an agent

  • The MVP boundary is correct.
  • New conceptual layers or public interfaces are necessary.
  • The before/after description matches the intended product behavior.
  • Human reviewer:
  • Decision and rationale:

zzylol and others added 5 commits September 18, 2026 20:43
Downstream repos depend on Sketchlib by bare git URL; Cargo treats ?rev=
as a distinct source, so consumers got two copies of asap_sketchlib.
Cargo.lock keeps builds reproducible.

Closes #450

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@zzylol
zzylol merged commit 8514931 into main Sep 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Refactor] Audit and prune redundant planning enums

2 participants