Conversation
This was referenced Oct 3, 2026
Contributor
Author
|
Closing: this passes the deployment's accuracy model into Pass 1, which conflicts with the agreed #572 decision (Q22): Stage 1 uses only the planner's built-in analytical rules; the deployment's accuracy model is used in Stage 3 (plan-selection) only, and Pass 1 keeps candidates without an analytical rule with 🤖 Generated with Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem: Pass 1 ignores the deployment's accuracy model, so it drops candidates that model would certify
#509 makes accuracy a deployment input. §Goal says the deployment supplies "its empirical cost model, empirical accuracy model and capabilities", and Assumption 5 says ASAPPlanner "uses the deployment's cost and accuracy models, or built-in defaults when the deployment supplies none". §Pass 1: Local candidate generation keeps "every candidate that is not provably unable to meet its accuracy requirement". §Scenarios, "Adding a better cost or accuracy estimation", allows the model to change the candidate set "where a model also changes sizing or admits a summary that has no built-in guarantee". #509 Example 2 is that case: one UnivMon state answers
Distinct,EntropyandL2.Before this PR,
MajorPassused two different accuracy models:PlanningModels::with_accuracytherefore reached only the final root check. A candidate that the built-in model cannot certify is already gone by then.The built-in model gives UnivMon a guarantee for one statistic only (
accuracy/estimators/univmon.rs):distinct_over_time(m[5m])CardinalityNoneentropy_over_time(m[5m])FrequencyEntropyNonel2_over_time(m[5m])FrequencyL2NoneA deployment model that does certify these statistics could not change this. The shared-UnivMon test from #519 had to rebuild
MajorPassby hand (lower,search_workload_with_targets,global_selection_with_summary_maintenance_lifecycles, assemble,share_common_sub_dags) so it could pass its own model toASAPStrategies.Scope. This PR covers the #509 requirement that Pass 1 uses the deployment accuracy model. It does not add a calibrated UnivMon error model, does not change sizing rules, and does not change plan selection. Native execution of the UnivMon readouts is #552.
Proposed method
default_strategies_with_models(cost, accuracy, evidence). It builds the same five default strategies as before.ASAPStrategiesandHydraGroupingStrategynow receive the caller'sAccuracyModelinstead of&DEFAULT_ACCURACY_MODEL.default_strategies_with_evidence(cost, evidence)with the same signature. It now callsdefault_strategies_with_modelswith&DEFAULT_ACCURACY_MODEL, so callers that do not supply a model keep the built-in behavior.MajorPass::optimize(Pass 1 entry,pass/major.rs) builds its strategies withmodels.accuracy. Pass 1 candidate construction and the final root check now use the same model.No new types. The allocator stays
DEFAULT_ALLOCATOR(EqualSplitAllocator).Pipeline position: Pass 1 (logical candidate generation). Pass 2 sharing, lifecycle selection and assembly are unchanged.
Key code interfaces
crates/asap-aware-mapping/src/replacement.rs:crates/asap-aware-mapping/src/pass/major.rs, inMajorPass::optimize:The models come from the existing
PlanningModels(pass/mod.rs, unchanged):Usage, as in the new test:
Fields
default_strategies_with_models:cost_model&'a dyn CostModelASAPStrategies,HydraGroupingStrategyandExactCompositionStrategy.accuracy_model&'a dyn AccuracyModellocal_guarantee), composes them (propagate) and checks targets (satisfies) while Pass 1 builds summary candidates. Passed toASAPStrategiesandHydraGroupingStrategy.MajorPasspassesmodels.accuracy, the same model it passes to the root check.evidence&'a dyn AccuracyEvidenceProviderASAPStrategiesandHydraGroupingStrategy.Vec<Box<dyn ReplacementStrategy + 'a>>ASAPStrategies,HydraGroupingStrategy,SharedSubDAGStrategy,AvgToSumOverCountStrategy,ExactCompositionStrategy. Same list asdefault_strategies_with_evidencereturned before.default_strategies_with_evidence:cost_modelandevidenceas above. The accuracy model is fixed toDefaultAccuracyModel. Behavior is unchanged for its callers.PlanningModels(unchanged; listed because this PR changes whereaccuracyis used):cost&dyn CostModelPlanningModels::new,with_cost;builtin()usesDefaultCostModel.accuracy&dyn AccuracyModelnew,with_accuracy;builtin()usesDefaultAccuracyModel.evidence&dyn AccuracyEvidenceProvidernew,with_evidence;builtin()usesNoAccuracyEvidence.Examples
#509 Example 2 through
e2e_planTest:
e2e_certified_frequency_evaluations_share_one_univmon_stateincrates/planner/tests/summary_sharing.rs. It replaces the hand-builtcertified_frequency_evaluations_share_one_univmon_state.Input. Three PromQL queries, each with
AccuracyTarget::Epsilon(0.02):Models:
PlanningModels::builtin().with_cost(&CHEAP_SUMMARY).with_accuracy(&UnivMonEvidence).UnivMonEvidenceis a synthetic test model. For any UnivMon family it returns a guarantee with metricRelativeValue, bound0.01and failure probability0.01. For every other family, and forpropagateandsatisfies, it defers toDefaultAccuracyModel. It exercises sharing, not runtime accuracy.What this PR changes.
MajorPasspassesUnivMonEvidenceintoASAPStrategies. The UnivMon candidate for each query now gets a guarantee in Pass 1 and stays in the candidate set. Selection and Pass 2 sharing then work as before.Output. The test checks:
output.plans.len() == 3.unique_deployments(&output) == 1: one deployed state across the workload, by pointer.Rc.ASAPOp::SummaryEstimatewhosesummary_inputisASAPOp::SummaryAggwith familySketch(UnivMon, _).guaranteeisSomeand satisfiesEpsilon(0.02)underDefaultAccuracyModel.satisfies.Which model each step uses
MajorPassbefore this PRDefaultAccuracyModelmodels.accuracyMajorPassafter this PRmodels.accuracymodels.accuracydefault_strategies_with_evidence(cost, evidence)DefaultAccuracyModel(unchanged)default_strategies_with_models(cost, acc, evidence)accOut of scope
PointCounttotal; the test model is synthetic.Stack and validation
Stack 9 · Base: #543 · Next: #552 · Closes #523. Continues the #528 review stack after #543.
Validation:
cargo test --locked -p asap-planner --test summary_sharing e2e_certified_frequency_evaluations_share_one_univmon_state.🤖 Generated with Claude Code