Conversation
This was referenced Oct 3, 2026
zzylol
force-pushed
the
stack/528-05b-local-alternatives
branch
from
October 3, 2026 16:07
102ad03 to
bca78d8
Compare
zzylol
force-pushed
the
stack/528-05-promql
branch
2 times, most recently
from
October 3, 2026 16:08
fb30215 to
d133a1d
Compare
zzylol
force-pushed
the
stack/528-05b-local-alternatives
branch
from
October 3, 2026 16:08
bca78d8 to
ce188ad
Compare
zzylol
force-pushed
the
stack/528-05-promql
branch
from
October 3, 2026 16:33
d133a1d to
d61bf7b
Compare
zzylol
force-pushed
the
stack/528-05b-local-alternatives
branch
from
October 3, 2026 16:33
ce188ad to
ba07130
Compare
zzylol
force-pushed
the
stack/528-05-promql
branch
from
October 3, 2026 17:02
d61bf7b to
130fc54
Compare
zzylol
force-pushed
the
stack/528-05b-local-alternatives
branch
2 times, most recently
from
October 3, 2026 17:11
681ba3a to
6f174e1
Compare
zzylol
force-pushed
the
stack/528-05-promql
branch
2 times, most recently
from
October 3, 2026 17:23
4cd558c to
7f65a80
Compare
zzylol
force-pushed
the
stack/528-05b-local-alternatives
branch
2 times, most recently
from
October 3, 2026 17:29
aeb762b to
bc4c4d5
Compare
zzylol
force-pushed
the
stack/528-05-promql
branch
from
October 3, 2026 17:29
7f65a80 to
569b845
Compare
zzylol
force-pushed
the
stack/528-05b-local-alternatives
branch
from
October 3, 2026 17:40
bc4c4d5 to
043e1d5
Compare
zzylol
force-pushed
the
stack/528-05-promql
branch
from
October 3, 2026 17:40
569b845 to
00c2776
Compare
zzylol
marked this pull request as draft
October 3, 2026 19:29
Contributor
Author
|
Parked as draft until Phase C (see #528). Stage 1 Pass 1 local alternatives belong to the #509 end-to-end work, which comes after finishing #511 (Phase A) and the #572 reorganization (Phase B). It will move into the 🤖 Generated with Claude Code |
zzylol
force-pushed
the
stack/528-05-promql
branch
from
October 3, 2026 19:32
00c2776 to
2918b78
Compare
zzylol
force-pushed
the
stack/528-05b-local-alternatives
branch
2 times, most recently
from
October 3, 2026 19:45
b044f4f to
7b4972b
Compare
zzylol
force-pushed
the
stack/528-05-promql
branch
from
October 3, 2026 19:46
2918b78 to
21b014f
Compare
zzylol
force-pushed
the
stack/528-05b-local-alternatives
branch
from
October 3, 2026 19:47
7b4972b to
561c6ed
Compare
zzylol
force-pushed
the
stack/528-05-promql
branch
from
October 3, 2026 19:47
21b014f to
008b0d0
Compare
zzylol
force-pushed
the
stack/528-05b-local-alternatives
branch
from
October 3, 2026 20:09
561c6ed to
8219c17
Compare
zzylol
force-pushed
the
stack/528-05-promql
branch
2 times, most recently
from
October 3, 2026 20:48
e3febef to
ca8b839
Compare
zzylol
force-pushed
the
stack/528-05b-local-alternatives
branch
from
October 3, 2026 20:48
8219c17 to
941c696
Compare
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The PromQL frontend's unified module was promoted to the crate root later in the stack; asap_frontend_promql::unified no longer exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
changed the base branch from
stack/528-05-promql
to
stack/528-08-cleanup
October 3, 2026 21:42
zzylol
force-pushed
the
stack/528-05b-local-alternatives
branch
from
October 3, 2026 21:42
941c696 to
517cd55
Compare
This was referenced Oct 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem: Pass 1 has no logical-only, unranked candidate list over the unified IR
#509 §1 "Logical ASAP-aware optimization", Pass 1: Local candidate generation, says:
#509 §"Stages and their decisions" also says that only stage 3 (plan selection) may discard a valid candidate, and that it does so with the deployment's empirical cost and accuracy models. Pass 1 gets only the logical DAGs, the accuracy requirements,
time_selectionand the repetition interval. It gets no cost model.Its candidate table includes the exact option for every computation:
Sum(x) by (g)TopK(k, x) by (g)Distinct(x)Quantile(x, window)Before this PR, the only enumeration is the legacy
replacement::realizations_for_intent(intent, cost_model). It works on the legacyQueryExprgraph, not onOperatorNode/QueryRootfrom #511. It also breaks the Pass 1 contract in three ways:It needs a cost model and ranks with it. The list is "exhaustive and ranked (most-preferred first) via
cost_model". Ranking is a stage 3 decision.It drops exact execution for approximate requests. For an approximate request it returns only the sketches.
Countalso gets its exact accumulator, but no intent getsPassThrough(the original sub-DAG run exactly):So "Exact quantile" from the table above is missing. Stage 3 can never choose it, even when it would be cheaper.
It gives an exact request an approximate choice.
Count{Exact}returns[ExactAggregate{Count}, Sketch(UnivMon)].This PR covers the per-aggregate part of Pass 1 on the unified IR: find every single-measure aggregate reachable from the workload roots, and list its exact and summary choices without ranking. It does not build replacement sub-DAGs, apply rewrite rules (for example recognizing
Entropy/L2forms), produce Hydra (a grouping-level choice), split multi-measure aggregates, or do any Pass 2 sharing.Proposed method
The new module
asap_aware_mapping::logical_candidatesruns in Pass 1. It takes the named roots from a frontend (for example #540'slower_promql_query_workload) and returns an inventory. It does not change the roots.enumerate_local_logical_candidates(roots):QueryRoot::validate_structure()on every root. A structural error is returned asLogicalCandidateError::Structure.QueryRoot::Operator(node)that isnode. ForQueryRoot::Scalar(expr)it isexpr.operator_refs()(for example the plan insidescalar(...)or a SQL scalar subquery).OperatorNode::reachable. That walk also follows operator nodes referenced from scalar expressions inside operators, becauseOperatorNode::children()includes them.HashSetofRcpointers is shared across all roots, so a producer read by two roots becomes one target.timing.is_some()(AssignedTiming). Execution timing is assigned in physical planning (docs: propose workload-wide planning, summary sharing, and materialization #509 stage 2), so it must not be present yet.NonASAPOp::Aggregatewith exactly one measure, callslocal_realizations_for_intentand stores aLocalLogicalTarget { target, alternatives }. An aggregate with several measures is skipped and stays as it is in the roots.local_realizations_for_intent(intent)builds the list in a fixed order:Realization::PassThroughfirst, always. This is exact execution of the original sub-DAG.Realization::ExactAggregate { kind, params }when the intent has an exact mergeable accumulator:Count,Sum,Min,Max,Rate,IRate,Increase.Countgets it for both exact and approximate targets.accuracy_target(intent)isSome) and that target is notExact:(epsilon, delta)with the existingaccuracy_budget.Epsilon(e)usesDEFAULT_DELTA = 0.01.epsilonis finite and> 0anddeltais finite and in(0, 1)(InvalidAccuracy).Realization::Sketchper algorithm insummary_candidates(intent), in catalog order. Each is sized withdefault_size_params(algorithm, intent, epsilon, delta).The function takes no cost model, accuracy model, runtime capabilities or storage policy. The sketch sizes are nominal dimensions from the built-in sizing contracts. They do not certify that a deployment meets the accuracy target; stage 3 checks that. The list order has no preference meaning.
The legacy ranked path stays for the existing pipeline until the planner cutover. This PR adds the new entry point beside it.
Key code interfaces
crates/asap-aware-mapping/src/logical_candidates.rs(new;pub mod logical_candidatesinlib.rs):The alternatives reuse the existing
Realizationenum fromreplacement.rsunchanged. This PR produces only these variants:Usage, from the frontend to the inventory:
Fields
LocalLogicalTargettargetRc<OperatorNode>NonASAPOp::Aggregatenode, the sameRcas in the roots (pointer-equal, not a copy). It keeps the source, grouping (reduction), filters, input expressions and evaluation context.timingisNone.alternativesVec<Realization>PassThrough.LocalLogicalCandidates<Id>IdrootsVec<(Id, QueryRoot)>targetsVec<LocalLogicalTarget>LogicalCandidateErrorStructure(SchemaDerivationError)QueryRoot::validate_structure()failed for a root.AssignedTimingtiming: Some(_). Pass 1 input must have no execution timing assigned; timing is a Stage 2 materialization decision.InvalidAccuracyepsilonis not finite or<= 0, ordeltais not finite or not in(0, 1).local_realizations_for_intentintent: &AggIntentQuantile,Cardinality,FrequencyL2,FrequencyEntropy,Count,TopK) decides whether sketches are added.Ok(Vec<Realization>)PassThrough, then the exact accumulator if any, then sketches insummary_candidatesorder.enumerate_local_logical_candidatesroots: Vec<(Id, QueryRoot)>roots.Ok(LocalLogicalCandidates<Id>)Realizationvariants produced herePassThroughExactAggregate { kind, params }kind: ExactKindis one ofCount,Sum,Min,Max,Rate,IRate,Increase.params: ExactParamsis the matching variant with the same name (exact accumulators have no tuning parameters).Sketch(SketchKind)SketchKind::new(algorithm, default_size_params(...)).SketchKind::algorithm()returns theSketchAlgorithm.Examples
End to end: #509 Example 1's rate query
Input (from
tests/logical_candidates.rs,promql_lowering_reaches_local_candidates_without_execution_timing): a PromQL workload with one entry,sum by (job) (rate(http_requests_total[1m])), accuracyEpsilonDelta { epsilon: 0.05, delta: 0.01 }, ingestion interval 1 s.feat(promql): lower queries to unified operator and scalar IR #540's
lower_promql_query_workloadproduces one operator root:enumerate_local_logical_candidates(vec![(0, root)])validates the root, walks the four nodes once each, and finds no assigned timing.Both aggregates have one measure, so both become targets. Neither
SumnorRatecarries an accuracy target, so no sketch is added even though the query is approximate:alternativesAggregate{by (job), [Sum]}PassThrough,ExactAggregate{Sum}Aggregate{PerEntity, [Rate]}PassThrough,ExactAggregate{Rate}The test asserts that some target offers
ExactAggregate { kind: Rate }and that every target still hastiming: None. No cost model is passed anywhere.Shared producer read by two roots
scalar_root_producers_are_discovered_once: oneCardinalityaggregate overflowsis used both asQueryRoot::Scalar(ScalarExpr::ScalarSubquery(producer))and asQueryRoot::Operator(producer). The result has 2rootsand 1target, andRc::ptr_eq(&targets[0].target, &producer)holds. Both roots still export withcompile_logical_asap_query(..).validate(), and the producer has notimingand noguarantee.Per-intent results
From
tests/logical_candidates.rsand the unit test inlogical_candidates.rs. "approx" meansEpsilonDelta { epsilon: 0.05, delta: 0.01 }.CountPassThrough,ExactAggregate{Count},Cms,CountSketch,UnivMon(unit test checks PassThrough, exact Count and UnivMon)CountExactPassThrough,ExactAggregate{Count}; no sketchCardinality{cols: [0]}PassThrough,Hll,Theta,Kmv,UnivMon(#509 Example 2)Cardinality{cols: [0, 1]}PassThrough,Hll,Theta,Kmv; noUnivMonfor a tupleFrequencyL2/FrequencyEntropyPassThrough,UnivMonQuantile{q: 0.99}PassThrough,Kll,DDSketchQuantile{q: 0.99}Exact[PassThrough]TopK{k: 10}PassThrough,CmsWithHeap,CountSketchWithHeapSum,Min,Max,Rate,IRate,IncreasePassThrough, matchingExactAggregateAvg,StdDev,HistogramQuantile,Extension, …[PassThrough]CountEpsilon(NaN)Err(InvalidAccuracy)CountEpsilonDelta{epsilon: 0.1, delta: 0.0}Err(InvalidAccuracy)timing = Some(QueryTime)Err(AssignedTiming)Aggregatewith two measuresrootsCompared with the legacy path
realizations_for_intentCount, approxExactAggregate{Count}; noPassThroughPassThrough,ExactAggregate{Count}, sketches in catalog orderQuantile, approxKll/DDSketchonlyPassThrough,Kll,DDSketchCount,ExactExactAggregate{Count},UnivMonPassThrough,ExactAggregate{Count}Extensioncost_model.realize_extension(..)PassThroughonlyOut of scope
SummaryAgg/SummaryEstimatenodes). The descriptors are not executable plans, and taking the first one is not a selection policy.Extensionintents.API notes and boundaries:
docs/develop_docs/local-logical-candidates.md.Stack and validation
Revised logical foundation 5/5 · Previous: #540 · Subsequent physical scopes pending reorganization · Tracker: #528
Order: #567 → #560 → #537 → #539 → #540 → #561.
🤖 Generated with Claude Code