Skip to content

feat(planner): Stage 1 whole-workload candidates and the stage_pipeline devtool - #575

Draft
zzylol wants to merge 6 commits into
stack/528-05b-local-alternativesfrom
stack/509-c2-stage1-workload
Draft

zzylol wants to merge 6 commits into
stack/528-05b-local-alternativesfrom
stack/509-c2-stage1-workload

Conversation

@zzylol

@zzylol zzylol commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #561 (Pass 1 local alternatives), which sits on #543 (Phase C). Next: Stage 2/3 (#576), then the Example 1 acceptance tests (#577).

Why

#509 defines a Stage 1 candidate as one logical ASAP DAG for the whole workload. #561 only lists the alternatives for each target aggregate. It builds no DAG, so nothing after Pass 1 has a candidate to work on, and nobody can look at what Stage 0 and Stage 1 produce.

What

  • logical_candidates::compose_logical_candidate(inventory, choice) builds one whole-workload candidate from one choice per Pass 1 target.
  • stage_pipeline devtool: lowers a workload, runs Pass 1, enumerates the Cartesian product of local choices (capped with --max-candidates and recorded as capped), and writes an asap-stage-pipeline/v1 document with stage0_logical and stage1_logical_asap.
  • Committed fixture tools/dag-viewer/examples/planner-layering-example1.json, which a test regenerates and checks.
  • Two fixes that the composed DAGs exposed:
    • FinalizeExactAccumulator named its output state, so a consumer that reads the sample value (for example sum by (job) over a finalized per-series rate) found no value column. It now takes the name of the aggregate it replaces.
    • Top-K summary items are keyed by the PromQL series identity column. Pass 1 rejected rows that carry the full series identity, and the runtime needs explicit item columns for keyed summaries.

How

compose_logical_candidate replaces each chosen target:

  • sketch → SummaryAgg → SummaryEstimate;
  • exact accumulator → SummaryAgg → FinalizeExactAccumulator;
  • pass-through → unchanged.

Untouched sub-DAGs keep their Rc identity. Each SummaryAgg declares trusted whole-source coverage (#570). No timing is assigned; with #543 everything defaults to query time later.

Example 1 (#509 planner-layering)

Q1 sum by (job) (rate(http_requests_total[1m])) (exact), Q2 topk by (job) (10, sum_over_time(http_requests_total[1m])) (ε = 0.01, δ = 0.001).

Before this PR: Pass 1 returns four target inventories (rate, sum, sum_over_time, topk). There is no Stage 1 candidate and no way to view one.

After this PR: cargo run -p asap-devtools --bin stage_pipeline -- --example planner-layering-1 --out out.json writes Stage 0 (1 workload DAG, two roots) and Stage 1 (24 candidates):

Q1 choices (4) × Q2 choices (6)
rate: exact / Rate accumulator × sum: exact / Sum accumulator sum_over_time: exact / Sum accumulator × topk: exact / CMS+heap / CountSketch+heap

So the counts are 1 → 24, for example L20 = Q1 exact (Sum acc, Rate acc) · Q2 exact (Sum acc).

Known gaps (tracked in #577's acceptance tests)

  • The doc expects 6 Stage 1 candidates and this gives 24, because Pass 1 also offers exact-accumulator options (Rate and Sum) and CountSketch+heap.
  • There is no Hydra option and no Pass 2 shared-input variant.

🤖 Generated with Claude Code

zzylol and others added 6 commits October 3, 2026 21:40
#509 defines a Stage 1 candidate as one logical ASAP DAG for the whole
workload. #561 lists local alternatives per target but builds no DAG.

compose_logical_candidate(inventory, choice) replaces each chosen target
aggregate with SummaryAgg -> SummaryEstimate (sketch) or SummaryAgg ->
FinalizeExactAccumulator (exact accumulator). Pass-through targets and
untouched sub-DAGs keep their identity. Each SummaryAgg declares trusted
whole-source coverage (#570).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`stage_pipeline --example planner-layering-1 --out <file>` (or
`--promql <q>...`) lowers the workload, runs Pass 1, enumerates the
Cartesian product of local alternatives (capped by --max-candidates and
recorded as `capped`), and writes an asap-stage-pipeline/v1 document with
stage0_logical and stage1_logical_asap only. Generated fixture for #509
Example 1 is committed; a test regenerates it and validates every DAG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…valuation naming

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…alizes

FinalizeExactAccumulator kept the state column's name (`state`), so a
consumer reading the sample value (e.g. sum by (job) over a finalized
per-series rate) found no `value` column at runtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…column

Rows that carry the full series identity are closed; Pass 1 rejected them
and the runtime needs explicit item columns for keyed summaries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r naming fix

The finalized exact accumulator now names its column after the aggregate
it realizes (value/sum instead of state). Point the stage_pipeline test at
the renamed fixture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant