Skip to content

feat(planner): compose exact functions with summaries - #359

Merged
zzylol merged 19 commits into
mainfrom
feat/general-exact-summary-composition
Sep 9, 2026
Merged

zzylol merged 19 commits into
mainfrom
feat/general-exact-summary-composition

Conversation

@zzylol

@zzylol zzylol commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Closes #171

Why

The planner could build nested accumulator summaries, but it could not preserve a useful inner summary when an exact function had to run across the summary maintenance/read boundary. Those cases collapsed to an opaque raw fallback.

What

  • add one general ValueOperation IR node whose semantic operation is independent from ExecutionTiming (MaintenanceTime or ReadTime)
  • retain concrete function identity; rate, irate, and increase are distinct intents and irate is no longer lowered as rate
  • register accuracy behavior by function definition: exact sum, average, and extrema have separate proven rules; counter rate functions explicitly have no distribution-free approximate-input bound
  • preserve unknown accuracy when no applicable rule exists instead of inventing Lipschitz(1); later accuracy-target checks therefore fail closed
  • allow runtimes to gate support by concrete function and execution placement
  • add typed execution-data-state validation, cost-based global composition selection, CSE, DAG export, and lifecycle traversal

How

Before: an exact fold over a summary readout, or a summary over an exact function without accumulator state, remained an opaque KeepPreAsap region. irate also lost its identity by lowering to Rate, and unclassified exact functions could inherit a placeholder accuracy rule.

After: the planner selects and materializes ValueOperation(operation, timing) around the independently selected summary plan. Function definitions supply semantic identity and accuracy/capability rules; execution timing remains an independent physical property. No function- or issue-specific physical node is introduced.

For exact samples, rate, irate, and increase remain exact. For approximate samples, reset detection and boundary extrapolation do not receive a guessed bound: the guarantee remains unknown unless a deployment registers a proof-backed rule with sufficient evidence.

Review fixes

  • Validate concrete operation/child pairs against root targets, instead of rejecting all compositions.
  • Retain the caller-model-validated plan through selection and materialization; never recompute its guarantee with the default model.
  • Share built-in accuracy and accumulator facts in one function registry. Runtime capability remains deployment-specific.
  • Regression tests cover custom-rule acceptance/provenance and rejection of unproven exactness.

Validation

  • cargo fmt --all -- --check
  • cargo check --workspace
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test -p asap-types --lib (182 passed)
  • cargo test -p asap-aware-mapping --lib (392 passed)
  • cargo test -p asap-frontend-promql
  • cargo test -p asap-integration-tests --test exact_composition (13 passed)
  • cargo test --workspace previously reached the environment linker limit for several large DataFusion test binaries (lld bus error); focused suites above pass.

zzylol and others added 16 commits August 29, 2026 16:38
…ase boundaries (#171)

Add phase-explicit post-ASAP nodes SummaryExpr::{ExactTransform, ExactPostProcess}
carrying a non-exhaustive ExactOperator::Aggregate payload (never an intact
QueryExpr subtree), plus an ExecutionAvailability {UpdateValue, SummaryState,
ReadoutValue} derivation/validation (post_asap::phase) returning typed
PhaseErrors at construction. construct_summary_agg now validates its edge, so
a maintained summary over a query-time readout falls back conservatively
instead of producing an unexecutable plan.

Add ExactCompositionStrategy (registered in default_strategies) proposing
Replacement::ExactComposition candidates that reference the child target
rather than selecting a child; PlanSpace::global_selection commits the
compatible parent/child pair using the issue's cost-units-per-second
formulas (postprocess/pretransform vs raw-recompute baseline), counts shared
child state once, and GlobalSelection::materialize links the committed
decisions into one validated DAG with shared Rc identity.

Cost hooks: CostModel::mixed_execution_capabilities and
exact_composition_cost_inputs (unknowns stay None, never zero; missing
statistics keep KeepPreAsap). DAG export gains explicit per-node stage,
decision provenance, cost unit and child-decision links (additive).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@zzylol zzylol changed the title Compose exact functions with summary plans feat(planner): compose exact functions with summaries Sep 9, 2026
@zzylol
zzylol merged commit 373826b into main Sep 9, 2026
3 checks passed
@zzylol
zzylol deleted the feat/general-exact-summary-composition branch September 21, 2026 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compose exact operators with summary plans across explicit update/readout boundaries

1 participant