Conversation
The backend decided precompute-versus-Prometheus placement by enumerating materialization-key masks and compiling every mask as its own candidate. Placement is now a summary-maintenance lifecycle chosen per unique state: Planner enumerates each state's alternatives and the backend prices them with its own unit costs, charging shared state once with all consumers' demand. ContinuouslyMaintained state is precomputed. Ephemeral state is rebuilt at query time and is offered only when raw series are readable from Prometheus; a query that keeps no state then runs natively over range-selector Scans, compiled once by Planner's PromQL Fallback lowering. Retention adds estimated state bytes x retained panes x the new lifecycle_costs.store_per_byte_second (default 0, preserving placements). Window-layout tests that exercise retained state pin it by disallowing the query-time raw source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A cheap store serves the query from precomputed state without raw reads; an expensive store makes the backend read the range selector from Prometheus at query time and compute the quantile itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One CostModel prices every summary a Planner enumeration lists, so a root with several states can be enumerated and bound in one call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unpriced retained state was moved to query time whenever rebuilding was priced; it now stays retained, the placement used without lifecycle evidence. Retention scales by input cardinality for per-series and grouped state, and raw selectors without a metric name are not offered as query-time sources, since their Scan cannot be priced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Sep 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #793.
Why
Placement (precompute a summary vs. compute it at query time) was decided by enumerating materialization-key masks over counter/max leaves and fully compiling each mask as its own quote candidate. The owner-approved layering makes placement a summary-maintenance lifecycle choice priced by the backend's own costs.
What
9a13ae4(integration/planner-for-backend; feat(precompute): separate idle and deadline closure #486/PromQL range warm path ignores step_ms and differential E2E does not detect it #487 PromQL Fallback physical coverage). Lockfile changes only the rev.compiler/placement.rs: for every unique summary state (after sharing), Planner'senumerate_summary_maintenance_lifecycleslists alternatives; one backendCostModelprices them — build,maintenance_per_update× ingestion rate,read× reads, retention, retirement — with demand from all consumers, so shared state is charged once.ContinuouslyMaintainedwins by default.lifecycle_costs.store_per_byte_second(default 0): retention adds estimated state bytes × retained panes (window ÷ evaluation interval) × partitions (input cardinality for per-series/grouped state) × price. An unpriced alternative never displaces retention.Ephemeralis offered only when raw series are readable from Prometheus at query time (mixed PromQL, backend-local target, notrequire_backend_local_execution). A query rebuilds all of its states or none. With no state left it runs natively: Planner's PromQL Fallback lowering compiles the query once over raw-series inputs, bound to range-selectorScans (the feat: bind query-time raw series inputs to the Prometheus endpoint #792 raw source). Otherwise counter/max leaves are externalized as before.materialization_candidates.rs,enabled_materialization_keys,MaterializationSearchCoverage) is removed; the inventory is the selected candidate plus the native exact alternative (plus existing forests, removed in the next PR).Scanas anexact_backendsource; decisions are recorded asdeployment.lifecycle_placementtrace events.Before / After
quantile_over_time(0.99, m[1m]), default snapshot costs:store_per_byte_secondcontinuously_maintained){__name__="m"}[60000ms]from Prometheus and computes the quantileStartup of the 44-query issue workload (
issue_workloads_execute_warm_at_successive_evaluationssnapshot, process start →/api/v1/health, debug build): 69.3 s → 38.6 s (164 → 84 candidates). Still above the 30 s budget; the next PR fixes it.Behaviour differences
materialization_search_coverageand candidate identities no longer hash a mask.require_backend_local_execution.Ephemeralis priced as rebuilding the selected state per read with the backend unit costs; it is realized as an exact query-time program over raw series (or exact subtrees), which is at least as accurate.Validation
cargo fmt --all -- --check,cargo clippy --workspace --all-targets --locked -- -D warningscargo test --workspace --locked --lib,cargo test -p control_plane --locked --tests(newtests/lifecycle_placement.rs: cheap/expensive store, no raw source, shared state priced once)cargo test -p data_plane --locked --test asapquery_compatibility_process_e2e -- --test-threads=1: 25/26; newlifecycle_placement_processe2e (store price flips placement) passes;issue_workloads_execute_warm_at_successive_evaluationsfails its 30 s readiness budget, as it does on chore: repin Planner integration and take over storage formats and panes #793 (69 s there).🤖 Generated with Claude Code