Conversation
Replace the no-common-snapshot reject with a bounded-lag admission: query-time raw inputs may share a physical program with stored native batches, whose lag the executor checks. Add the stored window grid helpers used to find the newest complete window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Raw inputs are read at t_q; each stored input uses its newest complete window t_s and is admitted only when t_q - t_s <= max_lag (default one slide of that output, override --max-stored-input-lag-ms). Beyond the bound the query misses to the exact fallback. The observed lag is traced and reported in execution stats. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #802.
Why
#792 added query-time raw-series inputs but rejected any physical plan that also read stored state ("no common snapshot"). The owner decided that such mixes are allowed under a bounded-staleness rule, which lets one plan combine fresh raw series with maintained state.
What
asap_types: the reject rule is replaced by an admission rule. Raw inputs may share a program with stored native batches, the only stored inputs whose lag can be checked. NewMaterializationBinding::slide_ms/latest_window_end_at_or_beforegive the stored window grid.data_plane:execute_storedbinds raw inputs at t_q. For a mixed plan, each stored input takes the newest complete window at or before t_q (t_s). If a window is incomplete it steps back one slide, but only whilet_q - t_s <= max_lag. Beyond the bound it returns a capability miss, which is the existing exact fallback, before any raw read.--max-stored-input-lag-ms(ASAPQueryEngine::with_max_stored_input_lag_ms). Unset means one slide of each stored output. I used deployment config rather than a plan field because it needs no plan-format or control-plane change. It sits next to--remote-write-revision-freshness-ms. A per-query field can follow when the Planner emits mixed plans.x-asap-stored-input-lag-ms, and the internal warning is stripped.Before this PR
sum_over_time(m[5m]) + <stored per-series Sum>with the raw leaf bound to Prometheus fails validation:query-time raw inputs cannot be mixed with installed state.After this PR
At t_q = 1000s, with a 300s window sliding every 60s:
sum_over_timeover (700s, 1000s] plus that stored Sum. The response carriesx-asap-stored-input-lag-ms: 40000.Semantics
t_q - t_s <= max_lagfor every stored input; otherwise the query fails closed to exact. A mix is never silently stale.Not in scope
placement.rsand candidate selection are untouched).--remote-write-revision-dir) pin[t_q - lookback, t_q]. There, a mixed plan is served only when t_q is on the window grid; otherwise it falls back to exact, which is safe but less available.Validation
cargo fmt --all --checkcargo clippy --workspace --all-targets --locked -- -D warningscargo test --workspace --locked --libcargo test -p data_plane --locked --test asapquery_compatibility_process_e2e -- --test-threads=1(28 passed)raw_source.rsmixed_inputs: within bound equals the exact reference; configured bound admits or rejects an older window; beyond bound falls back with no raw read; a terminal store error is not stepped past; stored-only reads the t_q window; raw inputs are rejected beside exact readouts. The three mixed-execution tests fail with the old reject rule.🤖 Generated with Claude Code