feat(promql): lower queries to unified operator and scalar IR

[zzylol]

Problem: PromQL and MetricsQL still lower to the mixed legacy graph, not to #511's operators and scalar expressions

#511 §1 states the problem: the current representation "mixes query-plan operators and scalar expressions", so "an expression can be placed where a table input is expected and fail only when the plan is checked". #511 §2.3 "Composition without a bridge" and §3.3 "PromQL semantic mapping" give the required PromQL shapes:

2                    → ScalarExpr: Literal(2.0)
time()               → ScalarExpr: EvalTimestamp
up * 2               → Project(sample * 2.0, child = instant selection of up)
vector(time())       → PromqlVectorFromScalar(EvalTimestamp)
scalar(sum(up)) + 1  → ScalarExpr: Arithmetic(PromqlScalarFromVector(Aggregate(...)), Literal(1.0))

§3.3 adds: up{job="api"} is TimeRange(Instant) over a scan and up[5m] is TimeRange(Range). up > 0 is a Filter. up > bool 0 is a Project with Case(Compare(...), 1.0, 0.0). Vector/vector comparisons keep vector_match and set return_bool. Per-sample math is a Project with a scalar FunctionCall whose "scalar parameters can be expressions; do not require constant-only AggIntent::Math payloads". §3.4 lists the gaps that "the frontend must reject … until it is supplied", including native-histogram samples, dynamic aggregate/sampling parameters and fill modifiers. §4 requires the §2.3 examples to "need no PromqlScalarBridge or equivalent constant-wrapper node".

#509 §0 "Language-specific frontends" sets the same rule from the planner side: a construct "that cannot be represented faithfully is rejected", and the frontend keeps series identity, evaluation timing and missing-data semantics.

Before this PR, the PromQL frontend (crates/frontend-promql/src/promql.rs) and the MetricsQL frontend (crates/frontend-metricsql/src/lib.rs) build only the legacy QueryExpr. On the same queries:

PromQL	Legacy QueryExpr (before)	Problem
2	PromqlScalarBridge leaf	constant-wrapper operator
time()	EvalTimestamp operator leaf	a scalar placed as an operator
up * 2	BinaryOp{Mul, up, PromqlScalarBridge(2)}	scalar wrapped as a BinaryOp input
-up	BinaryOp{Mul, up, PromqlScalarBridge(-1)}	same
up > bool 0	BinaryOp{CompareBool, up, PromqlScalarBridge(0)}	same
abs(up)	Aggregate{[Math(Abs)]}	per-sample map modeled as an aggregate
round(up, scalar(sum(other)))	rejected: Math::Round { to_nearest } needs a constant	constant-only payload
up vs up[1s]	both TimeRange{1s}	instant and range selection look the same
histogram_quantile(0.9, native_latency) (no catalog entry, no le)	generic sketchable Quantile	native/undeclared samples silently treated as raw
histogram_count(v)	Aggregate{[HistogramCount]}	native-histogram sample type not represented (#511 §3.4)

This PR covers the PromQL rows of #511 §2.3 and §3.3, and the representable subset of MetricsQL. It also rejects the §3.4 PromQL gaps listed above. It does not change the planner callers: they keep using the legacy entry points until the cutover. It does not upgrade the parser or add runtime execution.

Proposed method

This is the frontend stage (#509 §0). Both frontends get a parallel unified module. It builds the name-based UnresolvedOp / UnresolvedScalar tree from asap-frontend-common (added in #539) and resolves it with resolve_root / resolve_scalar_root into OperatorNode / ScalarExpr. No legacy QueryExpr is built and no conversion between graphs is added.

PromQL (crates/frontend-promql/src/unified/):

1. Workload checks. lower_promql_query_workload requires QueryLanguage::PromQL, runs workload.validate(), and reads data_ingestion_interval at now_ms. Expired or future evidence fails. Each entry is lowered with its own requirements.accuracy.target().
2. Native histograms first. The lowerer collects every metric name in the query. If the histogram catalog declares any of them Native, the query is rejected (UnsupportedFeature).
3. Root kind. If the parser's value_type() is Scalar, the query becomes QueryRoot::Scalar through lower_scalar. Otherwise it becomes QueryRoot::Operator through walk. A scalar-typed expression at an operator position is an error, so a scalar never becomes an operator node.
4. Scalars (lower_scalar). Constant sub-expressions fold to Literal(Float64). Otherwise -x → Negative, scalar arithmetic → Arithmetic, time() → EvalTimestamp, pi() → Literal(π), scalar(v) → PromqlScalarFromVector(<operator plan of v>), and a < bool b → Case(Compare → 1.0, else 0.0). A scalar comparison without bool is rejected. All scalar nodes carry ExprSemantics::Promql.
5. Vector/scalar operations. If exactly one operand is scalar-typed, the result is PromqlScalarOp { child, scalar, op, scalar_left, return_bool }. The resolver turns it into a Filter (comparison without bool, sample values kept) or a Project (arithmetic, or comparison with bool). An operator read inside the scalar (for example scalar(sum(up))) stays a visible child of that node.
6. Vector/vector operations become BinaryOp { operator: BinaryOperator { kind, vector_match, .. }, return_bool, lhs, rhs }. CompareBool is no longer used to encode bool.
7. Per-sample functions. Math functions (abs, round, clamp*, …), unary minus and calendar functions become PromqlMap. The resolver projects the sample through FunctionCall { name: "promql_<fn>", args } (or Negative) and keeps the full series identity. Extra arguments go through lower_scalar, so they can be dynamic (time() - 1, scalar(...)). round(v) gets a default 1.0. Unary minus keeps the metric name; math and calendar functions drop it.
8. Selectors. m{…} → TimeRange { range: data_ingestion_interval, kind: Instant } over the scan. m{…}[w] → TimeRange { range: w, kind: Range }. offset/@ wrap the scan in TimeShift.
9. Generic topk/bottomk → Limit { n: Some(k), partition_by: none } over Sort { value, partition_by: keys }. topk over count_over_time/sum_over_time stays the heavy-hitter AggIntent::TopK.
10. histogram_quantile. A declared ClassicBucket, or classic-bucket evidence (by (le), a _bucket metric, an le matcher), gives exact HistogramQuantile. A declared RawSamples gives the sketchable Quantile with the entry's accuracy. Mixed declarations, Native, and undeclared metrics without bucket evidence are rejected. histogram_count/sum/avg/stddev/stdvar/fraction are rejected.
11. Remaining §3.4 gaps stay rejected: dynamic limitk/limit_ratio parameters, non-constant min_of/max_of, and fill modifiers.

MetricsQL (crates/frontend-metricsql/src/unified/mod.rs) has the same structure as the legacy MetricsQL lowerer, with these changes: a bare number literal becomes QueryRoot::Scalar(Literal). A number literal operand becomes PromqlScalarOp. Unary minus becomes PromqlScalarOp(* -1). Matchers carry ExprSemantics::Promql. Range windows become TimeRange(Range). Everything it did not support before is still rejected.

The old entry points stay so each stack layer builds on its own. The unified modules are documented as "promoted to the root API at planner cutover".

Key code interfaces

PromQL, crates/frontend-promql/src/unified/mod.rs:

pub use error::PromqlError;
pub use histogram::{HistogramCatalog, HistogramKind};

/// Vector roots only; a scalar root is an UnsupportedFeature error.
pub fn lower_promql_workload(
    workload: &PlanningWorkload,
    now_ms: u64,
) -> Result<Vec<Rc<OperatorNode>>, PromqlError>;

pub fn lower_promql_workload_with_histograms(
    workload: &PlanningWorkload,
    histograms: HistogramCatalog,
    now_ms: u64,
) -> Result<Vec<Rc<OperatorNode>>, PromqlError>;

/// Scalar and vector query roots.
pub fn lower_promql_query_workload(
    workload: &PlanningWorkload,
    now_ms: u64,
) -> Result<Vec<asap_types::ir::QueryRoot>, PromqlError>;

pub fn lower_promql_query_workload_with_histograms(
    workload: &PlanningWorkload,
    histograms: HistogramCatalog,
    now_ms: u64,
) -> Result<Vec<asap_types::ir::QueryRoot>, PromqlError>;

unified/error.rs:

pub enum PromqlError {
    InvalidWorkload(WorkloadError),
    Parse(String),
    UnsupportedFunction(String),
    UnsupportedAggregateOp(String),
    UnsupportedFeature(String),
    MissingArgument(String),
    InvalidParameter(String),
    WrongLanguage(String),
    Convert(ResolveDAGError),
}

unified/histogram.rs:

pub enum HistogramKind { ClassicBucket, Native, RawSamples }
impl HistogramKind {
    pub fn is_sketchable(self) -> bool; // true only for RawSamples
}

#[derive(Debug, Clone, Default)]
pub struct HistogramCatalog(HashMap<String, HistogramKind>);
impl HistogramCatalog {
    pub fn new() -> Self;
    pub fn with(self, metric: impl Into<String>, kind: HistogramKind) -> Self;
    pub fn kind_of(&self, metric: &str) -> Option<HistogramKind>;
    pub fn is_empty(&self) -> bool;
}

MetricsQL, crates/frontend-metricsql/src/unified/mod.rs:

pub enum MetricsqlError {
    Parse(String),
    UnsupportedFeature(String),
    Resolve(String),
}
pub fn parse_metricsql(query: &str) -> Result<MetricsqlExpr, MetricsqlError>;
pub fn canonical_metricsql(query: &str) -> Result<String, MetricsqlError>;
/// Vector roots only.
pub fn lower_metricsql(query: &str, accuracy: AccuracyTarget)
    -> Result<Rc<OperatorNode>, MetricsqlError>;
/// Scalar constants become QueryRoot::Scalar without an operator.
pub fn lower_metricsql_query(query: &str, accuracy: AccuracyTarget)
    -> Result<asap_types::ir::QueryRoot, MetricsqlError>;

Usage:

use asap_frontend_promql::unified::{lower_promql_query_workload, HistogramCatalog, HistogramKind};
let roots = lower_promql_query_workload(&workload, now_ms)?;   // Vec<QueryRoot>
match &roots[0] {
    QueryRoot::Operator(node) => node.validate_structure()?,
    QueryRoot::Scalar(expr) => { expr.scalar_type(&Default::default())?; }
}

The output types (QueryRoot, OperatorNode, ScalarExpr, NonASAPOp) and the unresolved tree (UnresolvedOp::PromqlMap, UnresolvedOp::PromqlScalarOp, resolve_root, resolve_scalar_root) come from earlier PRs in the stack. This PR only uses them. The crate-private entry point is PromqlLowerer::lower_query_with_ingestion_interval(query, accuracy, interval) -> Result<QueryRoot>. Test helpers live in tests/unified_support.rs.

Fields

PromQL lowering functions

Parameter / result	Type	Meaning
workload	&PlanningWorkload	Must have language = PromQL (else WrongLanguage) and pass validate() (else InvalidWorkload). Each entry from query_workload.entries() is lowered with its own requirements.accuracy.target(). data_workload.data_ingestion_interval is required.
now_ms	u64	Planning time in Unix ms. The ingestion-interval evidence must be valid at this time (value_at(now_ms)), else InvalidWorkload(UnavailableDataIngestionInterval). Expiry is inclusive.
histograms	HistogramCatalog	Sample-type declarations for this call only. Installed in a thread-local guard and restored on return, so it does not leak into later calls.
result of *_query_workload*	Vec<QueryRoot>	One root per entry, in entry order. Scalar for scalar-typed queries, Operator otherwise.
result of lower_promql_workload*	Vec<Rc<OperatorNode>>	Same, but a scalar root is UnsupportedFeature("scalar root: use lower_promql_query_workload").

PromqlError

Variant	Meaning
InvalidWorkload(WorkloadError)	Workload validation failed, or the ingestion interval is missing or not usable at now_ms.
Parse(String)	promql-parser rejected the string.
UnsupportedFunction(String)	A function with no lowering (name included).
UnsupportedAggregateOp(String)	An aggregation operator with no lowering.
UnsupportedFeature(String)	A construct that is rejected on purpose: native histograms, mixed histogram declarations, undeclared non-bucket histogram_quantile, a scalar at an operator position, a scalar root on the vector-only API, fill modifiers, and similar.
MissingArgument(String)	A required argument is missing.
InvalidParameter(String)	An argument has the wrong shape, for example a scalar comparison without bool or a fractional topk k.
WrongLanguage(String)	The workload language is not PromQL.
Convert(ResolveDAGError)	Name resolution in asap-frontend-common failed.

HistogramKind (what a client declares for a metric)

Variant	is_sketchable	histogram_quantile result
ClassicBucket	false	exact HistogramQuantile (cumulative le buckets)
Native	false	rejected. The IR has no native-histogram sample type yet. Any query that names the metric is rejected.
RawSamples	true	generic Quantile with the entry's accuracy target. This is an explicit extension, not standard PromQL histogram semantics.

HistogramCatalog: a map from metric name to HistogramKind. new() is empty. with(metric, kind) adds or replaces one entry (builder style). kind_of(metric) looks one up. is_empty() reports an empty catalog. If no metric in the argument is declared, only classic-bucket evidence is accepted.

MetricsQL functions and MetricsqlError

Item	Meaning
query: &str	MetricsQL source text.
accuracy: AccuracyTarget	Copied into Count (count_over_time), Cardinality (count) and Quantile intents.
MetricsqlExpr	Re-export of metricsql_parser::ast::Expr. parse_metricsql returns it; canonical_metricsql returns its to_string().
lower_metricsql_query result	QueryRoot::Scalar(Literal) for a bare number, QueryRoot::Operator otherwise. lower_metricsql rejects the scalar case.
Parse(String)	Parser error.
UnsupportedFeature(String)	Construct not supported (offset/@, subquery step, keep_metric_names, vector-matching modifiers, if/ifnot/default, …).
Resolve(String)	Name resolution failed.

Examples

End to end: up * scalar(sum(up))

Input: lower_promql_query_workload on a PromQL workload with this one query, Exact accuracy and a 1 s ingestion interval (from tests/unified_scalar_design.rs, scalar_plan_dependencies_remain_visible).

1. The parser types the root as an instant vector, so walk is used.

2. walk_binary sees that the right operand is scalar-typed. The left side up lowers to TimeRange{1s, Instant} over Scan{up}. The right side goes through lower_scalar: scalar(...) → PromqlScalarFromVector(<plan of sum(up)>).

3. The result is PromqlScalarOp { op: Mul, scalar_left: false, return_bool: false }. The resolver adds series identity to the open PromQL schema and builds a Project whose value column is Arithmetic(Mul, sample, PromqlScalarFromVector(..)).

4. Output:

   QueryRoot::Operator(
     Project{ value = sample * PromqlScalarFromVector(·), labels/time kept }
     ├── TimeRange{1s, Instant} ── Scan{up}                         (child 1)
     └── Aggregate{[Sum]} ── TimeRange{1s, Instant} ── Scan{up}      (child 2, read by the scalar)
   )
   

   The test asserts node.children().len() == 2: the plan read by the scalar is a visible dependency, not hidden inside the expression.

Before/after on the #511 examples

From tests/unified_scalar_design.rs, tests/unified_promql_lowering.rs and tests/unified_promql_conformance.rs:

PromQL	Result of this PR	Test
2, time(), scalar(sum(up)) + 1, 1 < bool 2, -time()	QueryRoot::Scalar, type-checks with an empty schema	standalone_scalars_are_expressions
up * 2	Project, value = Arithmetic(Mul, sample, Literal(2.0)), plus promql_drop_metric_name; series identity and time_index kept	arithmetic_projects_the_sample_and_preserves_full_identity_and_time
up > 0, 0 < up	Filter; output schema equals the input schema	non_bool_comparisons_keep_vector_samples_even_with_scalar_on_left
up > bool 0	Project with Case	bool_comparison_projects_zero_or_one
-up	Project with Negative; schema unchanged (metric name kept)	unary_minus_preserves_identity
abs(up), round(up, scalar(sum(other))), clamp(up, time() - 1, time()), year(up), hour()	Project with FunctionCall; validate_structure passes	pointwise_functions_are_typed_scalar_projections
sum by (job) (rate(http_requests_total[5m]))	Aggregate{by [2], [Sum]} over Aggregate{[Rate]} over TimeRange{5m}	sum_by_over_rate_groups_the_outer_sum
a / on(host) b	BinaryOp with vector_match { kind: On, labels: ["host"] }	binary_op_with_on_grouping
bare selector vs [w] of the same length	TimeRange{Instant} vs TimeRange{Range}	instant_and_range_selectors_of_equal_length_stay_distinct
histogram_quantile(0.9, sum by (le) (rate(..._bucket[5m])))	HistogramQuantile	heuristic_baseline_is_unchanged_without_a_catalog
histogram_quantile(0.9, foo_bucket) with foo_bucket: RawSamples	Quantile	declared_raw_extension_and_native_gap_override_the_heuristic
histogram_quantile(0.9, latency_seconds) with ClassicBucket	HistogramQuantile	declared_classic_bucket_fixes_the_false_negative

Rejected on purpose:

PromQL	Error	Test
histogram_quantile(0.9, native_latency), no catalog	UnsupportedFeature (needs classic-bucket evidence)	heuristic_baseline_is_unchanged_without_a_catalog
foo_bucket declared Native (even a bare selector)	UnsupportedFeature	declared_raw_extension_and_native_gap_override_the_heuristic
histogram_count(v) and other native accessors, histogram_fraction	UnsupportedFeature	native_histogram_accessors_are_explicit_gaps, histogram_fraction_is_an_explicit_gap
limitk(scalar(foo), m), limit_ratio(time() % 17 / 17, m), limitk(NaN, m)	rejected	dynamic_and_non_finite_sample_params_are_rejected
1 < 2, time() > 0 (scalar comparison without bool)	error (the parser or InvalidParameter)	scalar_comparison_without_bool_is_rejected
a + fill(0) b and other fill modifiers	UnsupportedFeature	fill_modifiers_are_rejected_not_ignored
non-constant min_of/max_of	rejected	non_constant_min_of_max_of_is_rejected__GAP

Out of scope

• Planner callers. They keep the legacy lower_promql_workload / lower_metricsql until the cutover. The temporary parallel entry points are removed at the end of the stack.
• §3.4 gaps (native-histogram samples, dynamic aggregate/sampling parameters such as quantile(scalar(q), up), fill modifiers, request-context durations such as step()). They are rejected, not supported.
• Parser upgrades and runtime execution of the new IR.
• Open point: the instant selector lookback is the workload's data_ingestion_interval, as in the legacy lowerer. docs: define unified operators and SQL/PromQL scalar boundaries #511 §2.1 says the Instant lookback "is not the ingestion interval". This PR adds the Instant kind but does not change the horizon.
• MetricsQL: no new unified MetricsQL test file in this diff. Selectors without a range lower to a bare Scan (no TimeRange(Instant)), as in the legacy MetricsQL lowerer.

Stack and validation

Revised logical foundation 4/5 · Previous: #539 · Next: #561 · Tracker: #528

Order: #567 → #560 → #537 → #539 → #540 → #561. Splits the PromQL and MetricsQL lowering portion of #528. Restacked onto phase-free logical export: test helpers export LogicalASAPDAG without lifecycle assignment.

Validation: workspace all-target check; PromQL and MetricsQL tests, including the new scalar-design and conformance suites; workspace all-target/all-feature Clippy with warnings denied. Complete first-five validation: 1,961 workspace tests/doctests, formatting and workspace/all-target/all-feature Clippy pass.

🤖 Generated with Claude Code