fix(ir): top-k sketch readouts return selected rows; Example 1 expectations follow planner output - #579
Draft
zzylol wants to merge 2 commits into
Draft
fix(ir): top-k sketch readouts return selected rows; Example 1 expectations follow planner output#579zzylol wants to merge 2 commits into
zzylol wants to merge 2 commits into
Conversation
zzylol
added a commit
that referenced
this pull request
Oct 4, 2026
Follows #579: the top-k sketch readout derives one row per selected item instead of a packed Utf8 column, matching the exact Sort → Limit shape. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 4, 2026
SummaryEstimate{TopK} derived `partition keys + topk Utf8`, an encoding
no runtime operator produces. The runtime's keyed evaluation, the exact
Sort -> Limit path and EvaluatePopulation{TopK} all return the selected
rows, so every CountSketch+heap Example 1 candidate failed to compile.
Derive partition keys + item identity columns + Float64 `value`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 6-candidate spec did not account for Pass 1's exact-accumulator options (user decision). Un-ignore the count-only tests, check that the runtime compiles exactly the candidates Stage 3 finds valid and rejects Count-Min + heap for the same reason, and list the 24 candidates in the acceptance spec for manual review. Hydra and shared-input tests stay ignored, naming the missing feature. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
stack/509-c5-runtime-consistency
branch
from
October 5, 2026 06:21
5796e23 to
8d68e5a
Compare
zzylol
force-pushed
the
stack/572-b1-types-modules
branch
from
October 5, 2026 06:21
dfaa958 to
7eda06e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebased on main d4869a7 (DF 54).
Why
#509 Example 1 runs end to end (1 → 24 → 24 → 1), but only 8 of the 24 Stage 2 candidates compiled in the runtime (
physical_planner::compile). The acceptance tests still expected the spec's 6 candidates.What
ASAPOp::output_schemaderivedSummaryEstimate{TopK}aspartition keys + topk Utf8. No runtime operator produces or reads that encoding. The runtime's keyed evaluation, the exact Sort → Limit path andEvaluatePopulation{TopK}all return the selected rows, sokeyed_evaluationrejected the IR's schema ("invalid keyed evaluation shape"). The IR now derives the selected rows: partition keys + item identity columns + Float64value. With this, all three top-k implementations of Q2 return one row per selected series.sum_over_timeof raw samples. Nothing in the workload or catalog declareshttp_requests_totalnon-negative, and neither existing proof (UnitCount,ResetAwareCounterDerivative) applies. The legacy rule also leaves this caseUnknownOrSigned. Proving it would need a metric-type input (a counter or non-negative declaration) and a matching proof variant. The runtime and Stage 3 rules are unchanged and agree.How
crates/types/src/ir/operator/asap.rs: addsranked_rows_schemaforSketchStatistic::TopK. New unit test:structure_contract::topk_readout_derives_selected_rows.stage2_count_sketch_heap_topk_compiles_in_the_physical_planner.stage2_runtime_compiles_exactly_the_candidates_stage3_finds_valid(replaces the ignored "every candidate compiles" test) checks that the runtime compiles the 16 valid candidates and rejects the 8 CMS+heap candidates, for the same reason Stage 3 gives.stage1_has_24_candidates_covering_every_combination(wasstage1_has_six…),stage2_keeps_every_logical_candidate,stage2_preserves_logical_choices,stage2_exact_topk_is_sort_then_limit(8 exact),stage2_summary_topk_is_build_then_estimate, andstage3_charges_each_node_once, which now covers the priced (valid) candidates.stage1_q2_summary_families_are_heap_sketches_and_hydra(no Hydra),stage1_keeps_independent_and_shared_variants(Pass 2 sharing, Hydra),stage3_shared_input_is_not_costlier(Pass 2 sharing, Hydra).stage_pipeline. Only the CountSketch/CMS estimate node schemas changed.Before this PR: 8 of 24 Example 1 candidates compile in the runtime. The 8 CountSketch+heap candidates fail on the readout shape, and the 8 CMS+heap candidates fail on the non-negative weight rule. 10 acceptance tests are ignored.
After this PR: 16 of 24 compile, which is every candidate Stage 3 finds valid. The 8 CMS+heap candidates are rejected by both the runtime and Stage 3 for the non-negative weight reason. Stage 3 still selects P20 (all exact, 79.401 cpu ms), and every rejection reason is unchanged. 3 tests are ignored, each for a missing feature.
Follow-up: §5.2 of the
docs/asap-primitive-schemabranch still saystopkUtf8 and should say "selected rows". This PR does not touch that branch.🤖 Generated with Claude Code