Conversation
zzylol
force-pushed
the
stack/509-x9-pass1-coverage
branch
from
October 4, 2026 20:18
31ecc84 to
7c0019e
Compare
zzylol
force-pushed
the
stack/509-x9b-hydra-pass1
branch
from
October 4, 2026 20:18
79f0dff to
b89974e
Compare
This was referenced Oct 4, 2026
A count with `by` groups and an approximate target gets one more alternative (#580 W7): one shared Count-Min grid for every group, as the SummaryAgg contract of #600 states. The update is a unit weight per row, hashed by a non-null Utf8/Int64/Bool column (a grouping column first, else the PromQL series identity). Grouping is a new per-alternative axis on LocalLogicalTarget, carried through tumbling forms. Stage 3's analytical guarantee is now grouping-aware: for HydraCms it is pass1::grouping::hydra_guarantee over the inner Count-Min, with collision term e/shared_columns and failure probability e^-shared_rows (relative to the whole input's weight). Pass 1 sizes the inner sketch and the grid for ε/2 and δ/2 each so the sum meets the target. Other shared groupings have no model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#604 gave RootDemand a latency_ms and a logical candidate several physical candidates. The Hydra count test sets no latency bound and checks the all-query-time physical candidate, which comes first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
stack/509-x9b-hydra-pass1
branch
from
October 5, 2026 06:21
b89974e to
70a252c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack: Wave 2 chain: #599 → #601 → #600 → #604 → #606 → #603 → #605
Rebased on main d4869a7 (DF 54). At this head
cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warningsandcargo test --workspacepass (1642 passed, 0 failed, 23 ignored).Why
#580 item F / W7: Hydra for count and point queries first. Part of #509. #600 added the executor's HydraCms kernel and stated the IR the planner must emit, but Stage 1 never offered Hydra. Stage 3's analytical check also ignored grouping, so it would have certified a shared grid with a per-group sketch's guarantee.
What
SELECT src_ip, COUNT(*) FROM flows GROUP BY src_ipat ε = 0.1, throughplan_stages:Countacc, CMS, CountSketch, UnivMonSummaryAgg{Sketch(Cms), SharedMultiSubpopulation{HydraCms}}→SummaryEstimate{PointCount{SampleValue, None}}. It passes Stage 3's accuracy check, is priced at 0.0157 (the exact accumulator costs 0.0073), compiles, and executes toa=3, b=2, c=1.Selection does not change in these examples:
Countaccumulator, is still selected.How
logical_candidates::add_hydra_alternatives): for a singleCountwith an approximate target andbygroups (notwithout), it emits executor: HydraCms kernel for count and point queries #600's contract:Constant(1.0), with theUnitCountproof.width/depthequal across the family and the grid.LocalLogicalTarget.groupingsis parallel toalternatives, likewindowsandabsorbs. Tumbling forms copy it, since HydraCms merges per executor: HydraCms kernel for count and point queries #600. The capability rule resets it; count targets are never keyed.hydra_guaranteeadds the grid's collision term to the inner sketch's error. So the inner sketch and the grid are each sized for ε/2 and δ/2 (default_size_params+default_hydra_params). At ε = δ = 0.01 that gives a 544 × 6 inner sketch in a 6 × 544 grid.estimators::local_guaranteeis now grouping-aware. For HydraCms it returns the existingpass1::grouping::hydra_guaranteeover the inner Count-Min, with collision bounde/shared_columnsand failure probabilitye^-shared_rows. HydraCountSketch and HydraKll getNone, which means uncertified.stage_pipelinelabels these candidates "HydraCms".pass1/grouping.rsHydraGroupingStrategy, whose guarantee composition is reused.Review points
e·N·(1/shared_columns + 1/width). A small group's count can therefore be dominated by the error, whereas a per-group sketch's bare count is exact. The target is read the same way for both.shared_rows × shared_columnscells, each the size of one per-group sketch. At ε = 0.01 that exceeds the executor's default memory limit, so the execution test uses ε = 0.1.Tests
All four failed or did not exist on the parent:
pass1_sql_coverage::grouped_count_offers_a_priced_executable_hydra_plan(written first; it failed with "a Hydra candidate is generated") checks the contract, Stage 3 pricing, compilation and execution.logical_candidates::grouped_approximate_count_offers_hydrachecks that Hydra is offered only forbyplus an approximate target, and checks the sizes.estimators::cms::hydra_guarantee_adds_the_shared_grid_term: a grid sized like one per-group sketch misses ε, and one sized for the ε/2 split meets it.stage_pipeline_selection::sql_hydra_count_dp_equals_exhaustive: the DP picks the exhaustive minimum over the 18 combinations.Not in this PR
PointCount{Named(item), Some(v)}): noAggIntentasks for one. A filtered count,COUNT(*) FILTER (WHERE service = 'checkout') GROUP BY job, could use it, so it belongs with Pass 2 sharing and Stage 1 coverage parity with the retired MajorPass #580 J (filtered aggregates).Gates
-D warnings): pass.cargo test --workspace: 1,642 passed, 23 ignored. fix(planner): SQL COUNT(*) candidates count rows, not a sample value #603 had 1,638, and this PR adds 4 tests.Stacked on #603. Integration with #604:
fix: integrate with #604setslatency_ms: Nonein the Hydra test's demand and checks the all-query-time physical candidate, since a logical candidate now has several. The numbers above are unchanged on the rebased chain.🤖 Generated with Claude Code