Skip to content

Separate execution placement from logical semantics and cost ingestion-time alternatives #530

Description

@zzylol

Execution at ingestion time or query time should be a physical planning choice based on cost and runtime capabilities, not an intrinsic restriction on a logical computation. This is a concrete follow-up to #528 and complements #520; the intended representation follows #511.

Problem

The merge/scalar migration in #528 (commit 477decd) preserves current execution correctness by restricting maintenance arithmetic. In crates/types/src/ir/timing.rs, forces_query_time and ingestion-time binary validation use per_series_rows to require matching series populations. In crates/asap-aware-mapping/src/summary_maintenance_lifecycle.rs, lifecycle enumeration disables retained alternatives when that validation fails.

For example:

sum(sum_over_time(m[1m]) + sum_over_time(n[1m]))

The current AlignedBinary implementation requires identical key sets, so it cannot implement general matching between different selectors. This is a limitation of that physical implementation, not proof that the computation must run at query time. An ingestion-time implementation could maintain both operands, match labels, process updates/removals, and recompute affected results; its state and update cost may be high.

Requested change

  • Preserve logical semantics independently of execution placement. Treat every computation as eligible in principle for either placement, with any required evaluation context and dependencies represented explicitly.
  • Enumerate ingestion-time and query-time physical alternatives. Include incremental maintenance or recomputation where supported, and account for state, matching, update propagation, recomputation, and query-read costs.
  • Keep the identical-key requirement on AlignedBinary itself. Use a general matching implementation for different populations, or report that physical alternative as unsupported by the current runtime. Do not turn missing implementation support into a logical query-time-only restriction.
  • Replace the logical timing/lifecycle exclusions introduced in refactor: unify logical operators and scalar expressions #528 with physical capability checks and cost-based selection. Retain correctness guards until an appropriate implementation exists; simply deleting validation is not sufficient.
  • Coordinate with Move execution timing out of the logical DAG, then rename PostAsapDag — after #511 #520 so moving timing metadata also removes timing-dependent changes in operator meaning. This issue focuses on placement eligibility and physical alternatives, rather than DAG renaming.

Acceptance criteria

  • The example above retains the same logical meaning at either placement. With both physical implementations available, changing workload/update/read costs can change the selected placement.
  • Both implementations agree for disjoint and overlapping label sets, empty inputs, updates, removals/staleness, and window movement. Preserve the existing regression tests maintained_arithmetic_over_different_selectors_matches_prometheus and maintained_arithmetic_over_one_selector_executes.
  • Unsupported ingestion-time execution is reported as a physical capability gap, without silently using AlignedBinary or declaring the logical computation query-time-only.
  • Placement changes preserve matching, scalar/vector, and empty-result semantics; they do not change which computation the graph represents.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions