Skip to content

feat(runtime): compile unified operator and scalar graphs - #541

Draft
zzylol wants to merge 4 commits into
stack/528-05-promqlfrom
stack/528-06-native
Draft

zzylol wants to merge 4 commits into
stack/528-05-promqlfrom
stack/528-06-native

Conversation

@zzylol

@zzylol zzylol commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Rebuilt on the revised stack (2026-10-03)

This PR was based on the old stack/528-legacy-physical-base. It now sits on the revised #540 (Phase A in #528), as four commits:

  1. Restore execution timing assignment (ir/timing.rs). The revised feat(ir): add logical sub-DAG sharing and export without execution timing #537 exports only a logical DAG with no execution timing. The runtime still needs every node timed (ingestion vs query time), so the lifecycle → timing expansion comes back unchanged, except that it now carries coverage.

  2. Export timed plans as PhysicalASAPDAG (ir/physical_export.rs, the docs: propose workload-wide planning, summary sharing, and materialization #509 name for the old PostAsapDAG). It reuses the logical export's payloads and node ids, and adds each node's and edge's data state, window compatibility and coverage. Tests: the timed export round-trips through JSON; a query-time producer feeding an ingestion-time consumer is rejected; an untimed plan is rejected.

  3. The runtime itself, unchanged apart from names. The runtime's own split into precompute and query is renamed from PhysicalASAPDAG to CompiledPhysicalPlan, so it no longer collides with the docs: propose workload-wide planning, summary sharing, and materialization #509 plan type.

  4. One root per batch query. Like the logical export (feat(ir): add logical sub-DAG sharing and export without execution timing #537), a PhysicalASAPDAG holds roots, one per query of the batch, and shared sub-DAGs are exported once. compile_physical_asap_workload(&[Rc<OperatorNode>]) exports a timed batch, and the runtime compiles all roots. Scalar query roots are not physical nodes yet.

Validation: the workspace passes clippy with -D warnings, and cargo test --workspace passes 2,027 tests with 0 failures.


Revised Phase A stack (see #528): #567 → #560 → #537 → #539 → #540 → #541 → #542 → #543. The pre-rebuild description is in the PR history.

🤖 Generated with Claude Code

@zzylol
zzylol force-pushed the stack/528-05-promql branch from 5686cf7 to d9223e3 Compare October 2, 2026 18:32
@zzylol
zzylol force-pushed the stack/528-06-native branch 2 times, most recently from b22fa53 to d82bcde Compare October 2, 2026 19:40
@zzylol
zzylol force-pushed the stack/528-05-promql branch 2 times, most recently from 26da80b to 51a5484 Compare October 2, 2026 21:14
@zzylol
zzylol force-pushed the stack/528-06-native branch 2 times, most recently from 337a0ff to a94bd59 Compare October 2, 2026 21:22
@zzylol
zzylol force-pushed the stack/528-05-promql branch from 51a5484 to b58f2b4 Compare October 2, 2026 21:22
@zzylol
zzylol force-pushed the stack/528-06-native branch from a94bd59 to d84830e Compare October 2, 2026 21:25
@zzylol
zzylol force-pushed the stack/528-05-promql branch 2 times, most recently from 8fb402b to 2535016 Compare October 2, 2026 21:56
@zzylol
zzylol force-pushed the stack/528-06-native branch from d84830e to b631137 Compare October 2, 2026 21:56
@zzylol
zzylol force-pushed the stack/528-05-promql branch from 2535016 to e55e580 Compare October 3, 2026 02:31
@zzylol
zzylol force-pushed the stack/528-06-native branch 2 times, most recently from 075d8e2 to d9dc49c Compare October 3, 2026 02:39
@zzylol
zzylol force-pushed the stack/528-05-promql branch from e55e580 to 397e56a Compare October 3, 2026 02:39
@zzylol
zzylol changed the base branch from stack/528-05-promql to stack/528-legacy-physical-base October 3, 2026 14:57
@zzylol
zzylol changed the base branch from stack/528-legacy-physical-base to stack/528-05-promql October 3, 2026 19:55
@zzylol
zzylol force-pushed the stack/528-06-native branch from d9dc49c to 04f9db2 Compare October 3, 2026 19:55
@zzylol
zzylol marked this pull request as draft October 3, 2026 20:02
@zzylol
zzylol force-pushed the stack/528-05-promql branch from 008b0d0 to e3febef Compare October 3, 2026 20:09
@zzylol
zzylol force-pushed the stack/528-06-native branch from 04f9db2 to 0ebce24 Compare October 3, 2026 20:17
zzylol added a commit that referenced this pull request Oct 3, 2026
Cut today's planner over to the unified OperatorNode IR. Frontends return
OperatorNode / QueryRoot, ParsedWorkload keeps scalar roots, Pass 1
(ASAPStrategies), the existing identical-sub-DAG sharing, selection, DAG
assembly and lifecycle costing all run on OperatorNode, and PlanOutput exposes
the whole workload DAG. The native compiler from #541 becomes the canonical
physical_planner and consumes the PhysicalASAPDAG export.

Ported from the earlier #542 (95eef55) without new #509 stage logic. Legacy
QueryExpr/SummaryNode modules stay compiled for their own tests but are no
longer re-exported from post_asap; the cleanup PR removes them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol and others added 2 commits October 3, 2026 20:47
The logical export is phase-free. Physical planning still needs the lifecycle
timing expansion; bring it back unchanged except for carrying coverage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The logical export carries no execution timing. Physical planning output
needs it: reuse the logical payloads and node ids, and add each node's and
edge's data state plus window compatibility, as the earlier post-ASAP export
did. Coverage is carried through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol and others added 2 commits October 3, 2026 20:47
Match the logical export: a batch is one DAG whose roots are its queries,
with shared sub-DAGs exported once. The runtime compiles all roots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 3, 2026
Cut today's planner over to the unified OperatorNode IR. Frontends return
OperatorNode / QueryRoot, ParsedWorkload keeps scalar roots, Pass 1
(ASAPStrategies), the existing identical-sub-DAG sharing, selection, DAG
assembly and lifecycle costing all run on OperatorNode, and PlanOutput exposes
the whole workload DAG. The native compiler from #541 becomes the canonical
physical_planner and consumes the PhysicalASAPDAG export.

Ported from the earlier #542 (95eef55) without new #509 stage logic. Legacy
QueryExpr/SummaryNode modules stay compiled for their own tests but are no
longer re-exported from post_asap; the cleanup PR removes them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the stack/528-06-native branch from 0ebce24 to aaa8467 Compare October 3, 2026 20:48
@zzylol
zzylol force-pushed the stack/528-05-promql branch from e3febef to ca8b839 Compare October 3, 2026 20:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant