test (aggregation-fuzzer): exercise spilling under a bounded memory pool - #23979
test (aggregation-fuzzer): exercise spilling under a bounded memory pool#23979naman-modi wants to merge 3 commits into
Conversation
|
Hey @2010YOUY01 this tries to close #23658, and while implementing the solution, I hit some design questions, which I wanted to discuss with you: I added a bounded-pool knob to the fuzzer, but through the SQL/SessionContext path I can't produce the plan shape the ordered streams need: sorted input always plans as a single-partition Single-mode aggregate, and multi-partition repartitions away the ordering. So under a memory limit everything falls back to the legacy So, currently, this PR exercises the legacy hash-aggregate spill path broadly (accumulators, pressure, sorted/unsorted), but not the ordered streams. what i can think of:
Which do you think about this? |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #23979 +/- ##
==========================================
- Coverage 81.29% 81.28% -0.01%
==========================================
Files 1110 1110
Lines 385205 385205
Branches 385205 385205
==========================================
- Hits 313145 313132 -13
- Misses 53580 53589 +9
- Partials 18480 18484 +4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Hey @2010YOUY01 / @alamb, any thoughts on this approach for fuzzy memory testing? |
c643f6d to
0e4d1ad
Compare
|
Thank you! The high-level idea LGTM, it's on my list to review further. |
2fc1e48 to
bf7909f
Compare
|
Hey @2010YOUY01, when you get a chance, can you please review this? |
e7dccfe to
d4619dd
Compare
Which issue does this PR close?
Rationale for this change
The aggregation fuzzer always runs under the default unbounded memory pool, so it never exercises the aggregate spill paths (including the ordered-aggregation spilling from #23657).
What changes are included in this PR?
SessionContextGenerator. A context is either unbounded (today's behavior) or bounded by aFairSpillPoolthat forces a spill.peak_mem_usedmetric on the baseline plan, and bounded contexts use a range of fractions (2x, 0.5x, 0.4x, 0.334x) of it.ResourcesExhaustedand is skipped and logged, not failed.Are these changes tested?
Yes, no regression, and added a new
test_generated_context_spillsasserts a spill actually happens.Are there any user-facing changes?
No, it's Test-only.