docs: propose operator flattening (#468)

[Selvomega]

What

Adds a design proposal for #468: docs/design_docs/proposals/operator-flattening.md. Docs only.

The proposal merges the pre- and post-ASAP operator sets into one tree: Operator { Basic(OriginalOp), Ext(ASAPOp) }, where every operator's children are Rc<Operator>. This removes KeepPreAsap, the duplicated ValueOperation operators, SummaryNode and SummarySchema, and with them the three problems in #468.

Worth reviewing

• QueryExpr is split into operators (OriginalOp) and expressions (ScalarExpr).
• Pre vs. post is checked once at the optimizer entry, not by a type parameter.
• Only SummaryAgg / SummaryEstimate store a guarantee; the rest is derived into a GuaranteeIndex.
• Column.dtype becomes ColumnType { DataType(..), ASAPType(..) }.
• Six implementation stages, each leaving main green.

🤖 Generated with Claude Code

[zzylol]

From #468, the essential change is to use one operator representation before and after ASAP optimization, so relational and summary operators can compose directly within the same DAG.

Concretely, this means two things:

1. Remove duplicate definitions and implementations of relational operators. Keep a single set of Project, Filter, Aggregate, Join, SetOp, etc., shared by pre-ASAP and post-ASAP plans.

2. Remove the opaque KeepPreAsap boundary. Give all operators a common child interface, allowing relational operators to consume summary readouts and summary operators to reference relational subtrees and share inputs. Input/output types and semantic validation still determine which connections are legal.

Optimization can then replace selected nodes while preserving the surrounding structure and sharing relationships:

SetOp                         Aggregate(avg)    SummaryAgg(Kll)
├── SummaryEstimate                    \        /
│   └── SummaryAgg                        Scan
└── SummaryEstimate
    └── SummaryAgg

This goal does not require removing or redesigning per-node schema, guarantee, or timing. These can remain; how they are stored, derived, and propagated is a separate design decision.

[Selvomega]

From #468, the essential change is to use one operator representation before and after ASAP optimization, so relational and summary operators can compose directly within the same DAG.

Concretely, this means two things:

1. Remove duplicate definitions and implementations of relational operators. Keep a single set of Project, Filter, Aggregate, Join, SetOp, etc., shared by pre-ASAP and post-ASAP plans.
2. Remove the opaque KeepPreAsap boundary. Give all operators a common child interface, allowing relational operators to consume summary readouts and summary operators to reference relational subtrees and share inputs. Input/output types and semantic validation still determine which connections are legal.

Optimization can then replace selected nodes while preserving the surrounding structure and sharing relationships:

SetOp                         Aggregate(avg)    SummaryAgg(Kll)
├── SummaryEstimate                    \        /
│   └── SummaryAgg                        Scan
└── SummaryEstimate
    └── SummaryAgg

This goal does not require removing or redesigning per-node schema, guarantee, or timing. These can remain; how they are stored, derived, and propagated is a separate design decision.

This is exactly what I want to do conceptually.

I don't think the proposal is trying to "remove" or "redesign" the schema, timing and guarantee. I think it is just proposing how to implement that under the new operator definition.

[zzylol]

@Selvomega I think the proposed split comes from a useful distinction, but we should separate two design decisions:

1. Classify operators and expressions by their roles.

Operators describe data flow between plan nodes; scalar expressions describe value computation inside an operator. For example, Project is an operator, while price * 2 is one of its expressions. Expressions can be evaluated per row or over a batch.

With that distinction, OriginalOp is a confusing name: it describes history rather than semantics. A clearer classification would be:

• RelationalOp: Scan, Filter, Project, Aggregate, Join, SetOp, etc.
• TimeSeriesOp: time-series-specific operations, if a separate category is useful.
• SummaryOp: SummaryAgg, SummaryEstimate, SummaryMerge, etc.
• ScalarExpr: columns, literals, arithmetic, comparisons, scalar functions, etc., used within operators.

These can be conceptual categories without requiring additional enum wrappers. All operators should use a common plan-node representation and child interface, preserving per-node schema, guarantee, and timing.

2. Treat summary optimization as selective subgraph replacement.

These operator categories are not successive planning stages. When a rule applies and its semantic and accuracy requirements are satisfied, a relational or time-series operator/subgraph can be replaced with a subgraph containing summary operators. For example:

Project                         Project
  Aggregate(percentile)    →      SummaryEstimate(percentile)
    Filter                          SummaryAgg(Kll)
      Scan                            Filter
                                        Scan

The surrounding operators remain unchanged. The replacement must provide the output expected by its parent—for example, a percentile value from SummaryEstimate, rather than raw sketch state from SummaryAgg. The resulting DAG can freely mix these operator categories through valid connections and preserve shared inputs.

This keeps #468 focused on sharing operator definitions and removing the opaque KeepPreAsap boundary. Separating ScalarExpr is a reasonable structural cleanup, but the proposal should explain whether it is necessary for that goal or an independent change.