Skip to content

fix(asap-tools): regenerate recommended-config results with p99 rank error - #784

Merged
milindsrivastava1997 merged 1 commit into
mainfrom
fix/752-results-summary
Oct 5, 2026
Merged

milindsrivastava1997 merged 1 commit into
mainfrom
fix/752-results-summary

Conversation

@zzylol

@zzylol zzylol commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Before this PR

results_summary.csv from #752 still had the p99 runs' relative value errors: 0.1055 for the recommended KLL k=200, 0.4582 for the default K=20. It marked the recommended configuration as failing its 0.01 target. The README reports rank error (0.0019 / 0.0153) and says the recommendation meets the target, so a consumer of the CSV would conclude the opposite of the README.

After this PR

The CSV is regenerated by summarize on main, which computes p99 rank error against the replayed trace values:

Run measured_error meets_target
p99 recommended, KLL k=200 0.0019 True
p99 default, KLL K=20 0.0153 False

These match the README. Two other differences come from the current summarize:

  • the default rows no longer repeat the recommended row's predicted_error;
  • the p99 rows leave measured_error_all_keys empty, because the all-keys ARE does not apply to a quantile.

All other values are unchanged.

Validation

Ran on clnode109, which holds #752's experiment outputs:

python3 recommended_sketch_configs/recommended_sketch_configs.py summarize \
  --recommendations /mydata/asap/recommendations.csv \
  --experiments-dir /mydata/asap/experiment_outputs \
  --google-trace /data/cluster_traces/google/part-00000-of-00500.csv.gz

The code was ASAPQuery main at cbabfd6. The generator's output-name collision, the other CR finding on #752, is being fixed separately.

🤖 Generated with Claude Code

…error

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@milindsrivastava1997
milindsrivastava1997 merged commit 4885a2e into main Oct 5, 2026
11 checks passed
@milindsrivastava1997
milindsrivastava1997 deleted the fix/752-results-summary branch October 5, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants