Skip to content

feat: support grouped TopK heaps on the Planner graph - #802

Open
zzylol wants to merge 2 commits into
feat/per-series-arithmetic-warmfrom
fix/topk-by-heap-groups
Open

zzylol wants to merge 2 commits into
feat/per-series-arithmetic-warmfrom
fix/topk-by-heap-groups

Conversation

@zzylol

@zzylol zzylol commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Stacked on #801.

Why

Grouped temporal TopK over a raw heap, e.g. topk by (job) (2, count_over_time(m[5s])), failed at install with raw heap items that exclude identity labels are not readable. The heap is partitioned by job, so Planner's heap item is the series identity without job. The group labels are not lost: storage keeps them as the heap state's group key. Stored-state readout just never put them back when it rendered the item.

What

  • The backend's stored-state readout (SummaryState::topk_items, topk_ranked, readout_bound) now gets the stored group labels and adds each missing non-empty one back into a decoded series identity before rendering the series key. Planner still does the heap estimate and the ranking. No Planner change is needed.
  • Removes the install-time rejection in raw_dag.rs.
  • Repins ASAPPlanner to 344369e (integration/planner-for-backend, adds e2e.sh list misses ignored Rust tests outside data_plane and crates #490). Only the rev strings change in Cargo.lock.

Before this PR

topk by (job) (2, count_over_time(top_endpoint_qps[5s])) → the data plane refuses the physical plan at startup. On main and at #795/#797, the same query was also refused at install (raw DAG item expression is unsupported / raw heap item expression is unsupported).

After this PR

The query installs and returns the top 2 per job. Each row carries job plus item = the whole series key, the same shape the legacy heap kernel produced for identity items:

{job="a", item="top_endpoint_qps{endpoint=\"gamma\",job=\"a\"}"} 200
{job="a", item="top_endpoint_qps{endpoint=\"alpha\",job=\"a\"}"} 100
{job="b", item="top_endpoint_qps{endpoint=\"zeta\",job=\"b\"}"}  150
{job="b", item="top_endpoint_qps{endpoint=\"delta\",job=\"b\"}"}  75

Validation

  • New e2e registered_temporal_topk_by_{cms,count_sketch}_heap (instant and range queries). It fails before the fix with the install error and passes after. The ungrouped variants still pass.
  • New unit weighted_frequency_items_restore_group_labels.
  • cargo fmt --check, cargo clippy --workspace --all-targets --locked -- -D warnings, cargo test --workspace --locked --lib, cargo test -p control_plane --locked --tests, cargo test -p data_plane --locked --test asapquery_compatibility_process_e2e -- --test-threads=1 (28 passed).

🤖 Generated with Claude Code

zzylol and others added 2 commits September 30, 2026 10:06
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A topk by heap partitions by its group labels, so Planner's weighted
frequency item identity excludes them. Stored-state readout now merges
the stored group labels back into each series identity before rendering
it, and install no longer rejects identity items that exclude labels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant