docs: define unified operators and SQL/PromQL scalar boundaries - #511
Merged
Merged
Conversation
Add a design proposal for merging the pre- and post-ASAP operator sets
into one Operator { Basic(OriginalOp), Ext(ASAPOp) } tree, and list it
in the proposals index.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
reviewed
Oct 1, 2026
zzylol
previously approved these changes
Oct 1, 2026
This was referenced Oct 2, 2026
Open
Contributor
|
@Selvomega while you are drafting the implementation plan for this PR, some points I think may be useful:
|
Collaborator
Author
I don't quite understand. the main branch where I am branching out has no notion of LogicalASAPDAG, and I thought we have agreed this morning that I do this PR without the new DAG layering, and the DAG layering PR will instead depend on this operator flattening PR. Btw I am currently having Claude writing the code but it hits the usage limit so I have to stop for a while |
zzylol
added a commit
that referenced
this pull request
Oct 2, 2026
Merge main into refactor/operator-flattening while retaining the unified OperatorNode graph proposed in #511. Port native physical compilation, aggregate filters, shared summaries and lifecycle placement to that graph. Remove ScalarBridge, preserve scalar workload roots as owned expressions, and lower mixed PromQL arithmetic and comparisons to Project/Filter with complete series identity and explicit vector/scalar conversions. Preserve native execution coverage and correct cross-selector maintenance eligibility and empty PromQL aggregation behavior. Validation: 1,569 workspace tests pass; workspace all-target/all-feature Clippy with warnings denied, rustfmt, and MetricsQL external consumer pass. Vendored MetricsQL baseline retains its main-branch sources and checker: lib baseline passes, but one existing doctest diagnostic hash differs under rustc 1.98.1 (same expected failing test set).
This was referenced Oct 2, 2026
Resolve the conflict in docs/design_docs/proposals/operator-sharing.md by keeping this PR's version. Since this branch was cut, main changed two lines in sections that this PR rewrites: - the MaintainPopulation/ReadPopulation timing row (lifecycle-set population timing, from the lifecycle stack), and - PlanSpace -> CandidateLogicalASAPDAGs (#514). Neither line exists in this PR's rewritten §2, which already assigns timing to physical planning under #509. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Define a common operator graph before and after ASAP optimization, and separate query operators from scalar expressions using explicit SQL/PromQL semantic mappings.
Before this PR
Relational subplans are hidden behind wrappers, and ordinary operations have separate representations above and below summaries. The scalar/operator proposal largely copies the current mixed representation without establishing language coverage.
After this PR
Operator::NonASAPandOperator::ASAP, with a commonOperatorNodecarrying result kind, schema, guarantee and timing. Keep one canonical set of operation payloads across both documents; distinguish structure and timing validation from planner accuracy assessment.NonASAPOpandScalarExpragainst DataFusion 55.1.0 and Prometheus 3.15.0, with tables for direct mappings, equivalent lowerings and remaining semantic gaps.PromqlScalarBridge: for example,up * 2uses a projection with a scalar literal. Retain real scalar/vector conversion and SQL subquery semantics, with visible operator dependencies.Documentation only. The repository still depends on DataFusion 43.0.0 and a PromQL parser declaring 3.8 compatibility; this PR does not upgrade runtime dependencies.
Validation
Reviewed the pinned upstream definitions and language references. Checked both documents' operator lists, scalar placement, links, references, code fences and whitespace. Compiled the combined Rust structure sketches with placeholders for existing payloads; checked that type definitions are unique and operation lists remain complete. No runtime tests run.