Skip to content

docs: take node timing from lifecycle assignment in the flattening proposal - #481

Closed
zzylol wants to merge 2 commits into
mainfrom
docs/flattening-lifecycle-timing
Closed

zzylol wants to merge 2 commits into
mainfrom
docs/flattening-lifecycle-timing

Conversation

@zzylol

@zzylol zzylol commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Why

The owner-approved planner layering (#480, "Output layers") makes the summary maintenance lifecycle layer the only source of execution timing: logical Post-ASAP / PlanSpace decides what to compute, the lifecycle layer decides placement, physical compilation splits by timing, and the deployment prices lifecycle assignments. The flattening proposal merged in #469 instead derived timing from operator kinds with derive_timings.

What

Docs only, docs/design_docs/proposals/operator-sharing.md:

  • derive_timings / derive_timings_at → apply_lifecycle_timings(root, &LifecycleAssignment, memo). The timing slot stays; Unset means no assignment applied, and export rejects it.

  • Validity checks (e.g. ingestion work cannot depend on a query-time result) become validation of the applied assignment (§2.3, §4, §5).

  • Notes that some logical candidates hard-code timing today (exact-composition placements, maintained populations, feat: expose physical-ready logical candidates in Planner selection #472's Rate→Sum pair); they become lifecycle choices.

  • Stage table: stage 4 applies current timing as the initial assignment; stage 5 switches to lifecycle-applied timing; stage 6 removes the fallbacks. Tests and open questions updated.

  • derive_guarantees is unchanged. decoupling_op_and_expr.md does not mention timing, so it is not changed.

  • §6 export: replace "largest connected non-ASAP subtree as one Relational fragment (with DagInput leaves)" with one post-ASAP node per non-ASAP operator; children become edges. Physical compilation then corresponds node by node (a node may expand to helper operators numbered from it). Tests and consumer table updated accordingly.

  • Wording: the timing source is "physical design (summary materialization)", matching docs: propose workload-wide planning, summary sharing, and materialization #509; the LifecycleAssignment open question now points to SummaryMaintenanceLifecyclePlan / execution_timed_dag() from feat: derive execution timing from a summary lifecycle plan #482.

Before this PR

derive_timings runs on each assembled root, sharing one memo, top-down: a root is query time, a set node keeps its value, a node of fixed kind takes that kind's time, and every other node takes its parent's.

Stage 4: … copying each node's guarantee and today's derived timing into the slots …

After this PR

The summary maintenance lifecycle layer chooses a lifecycle per unique summary state; a chosen assignment determines every node's timing … It is the only source of timing. apply_lifecycle_timings writes the assignment into the slots … Unset means no assignment has been applied; physical compilation and export reject it.

Stage 4: … copying each node's guarantee into its slot and applying each node's current timing as the initial lifecycle assignment …

Related

#469 (original proposal), #480 (output layers), #476

🤖 Generated with Claude Code

…oposal

The per-node timing slot is filled by applying a summary maintenance
lifecycle assignment instead of a derive_timings pass over the IR.
Validity checks become validation of the applied assignment; the stage
plan copies current timing as the initial assignment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol

zzylol commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

On the open question about LifecycleAssignment: #482 answers it with the existing type. A chosen assignment is a SummaryMaintenanceLifecyclePlan (from #476's SummaryMaintenanceLifecycleCandidates::select), and SummaryMaintenanceLifecyclePlan::execution_timed_dag() turns the per-state choice into per-node timing. This proposal could name those instead of a new LifecycleAssignment type.

Replace the largest-non-ASAP-subtree fragment export with one Relational node
per operator, so physical compilation corresponds node by node. Name the
timing source as physical design (summary materialization) and point the open
LifecycleAssignment question at SummaryMaintenanceLifecyclePlan (#482).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Sep 30, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@Selvomega Selvomega left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't merge this PR. I will update my design doc based on this PR as well as recent updates in main

@zzylol

zzylol commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

@Selvomega given the new planner architecture design and #511, this PR can be closed.

@zzylol zzylol closed this Oct 1, 2026
zzylol added a commit that referenced this pull request Oct 2, 2026
…ation (#509)

* docs: define the Planner and deployment layering contract

Add the layering proposal, its proposals-index entry, and an Output layers
section in the input/output/workflow design. Every Planner layer outputs all
legal candidates; the deployment keeps every summary-family candidate and
selects with its own costs. Design DAG names are marked as the target API with
the current main type alongside.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: treat source binding as a PhysicalDAG execution step

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: place summary materialization under physical planning

Split physical planning into physical design (summary materialization,
workload-level, like materialized-view selection) and physical
implementation (per-node lowering and cutting). The annotated graph stays a
Post-ASAP DAG; materialization is decided in design and realized by the cut.
State node-by-node correspondence and the Fallback exception that the
operator-flattening proposal removes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: point per-operator export at #481

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: state family retention and timing-sensitive lowering plainly

Describe current Planner behaviour (no family pruning) and the backend gap,
and replace the Binary 'exception' with the general rule that a timing-
sensitive node needs one compilation per distinct timing, with an example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: drop Binary lowering detail from the layering proposal

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: drop mixed-placement and compatibility sections from the layering proposal

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: drop the implementation-status section from the layering proposal

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: leave input-output-workflow unchanged in the layering PR

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: move selection into ASAPPlanner; deployment supplies prices

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: address review of layering proposal

Planner takes the query and data workloads plus the deployment's cost
model, accuracy requirements and capabilities, and returns one optimal
PhysicalDAG; candidate sets stay internal. Name the annotated stage
MaterializedPostASAPDAG, tabulate what each DAG encodes, fold timing and
selection into the layer descriptions, rename physical design to summary
materialization, and make the example trace one query through every DAG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: name layer 2 summary lifecycle planning; drop deployment ranking row

A lifecycle fixes materialization, timing, maintenance, retention and
window framework together, so the stage and its DAG are named after the
lifecycle (LifecyclePostASAPDAG) rather than one of those aspects. The
deployment row no longer mentions ranking or selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: rename layering proposal to planner-layering.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: name the DAG stages LogicalPostASAPDAG and PhysicalPostASAPDAG

Every stage after the frontend is a Post-ASAP DAG; the prefix says how
far planning has gone: Logical, Lifecycle, Physical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Update planner-layering.md

* Update README.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* Update planner-layering.md

* docs: clarify planning contracts and simplify examples

* docs: restore detailed planning stages diagram

* docs: align stage diagram with planning contracts

* docs: restore planner layering proposal to 8c29c2e

* Update planner-layering.md

* Update planner-layering.md

* docs: address remaining planner-layering review comments

Replace "executable", "lower", "price" and "costed" wording, note that
materialized output may live on disk or in memory, keep Hydra's per-job
heap explicit, mark SQL entropy/L2 recognition as frontend TODO, and fill
in the sliding-window merge citation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants