fix: partition not used yet panic - #3769
Merged
BubbleCal merged 3 commits intoMay 2, 2025
Merged
Conversation
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
BubbleCal
marked this pull request as ready for review
May 2, 2025 10:00
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3769 +/- ##
==========================================
- Coverage 78.59% 78.58% -0.01%
==========================================
Files 272 272
Lines 102094 102085 -9
Branches 102094 102085 -9
==========================================
- Hits 80238 80222 -16
- Misses 18683 18694 +11
+ Partials 3173 3169 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
BubbleCal
added a commit
that referenced
this pull request
Aug 14, 2025
We've seen some bugs that could be reproduced with some operations in some order, so adding this for finding out such bugs: - #4108 - #4156 - #3769 These weekly tests would enumerate all permutations of write operations, and run all of them, to make sure we can cover all cases. And we would run these tests on latest 2 stable version and main, to find out any regression and compatibility issues This would take a long time, so we run these weekly, rather than on each commit. and we've seen some bugs which happened on only large dataset, we can run some tests over a large dataset weekly, but not adding these for now --------- Signed-off-by: BubbleCal <bubble-cal@outlook.com>
BubbleCal
added a commit
that referenced
this pull request
Aug 12, 2026
## What changed? Extend the cross-version index maintenance-sequence search with a bounded `IVF_PQ` + `BTREE` prefilter scenario. - cover every single maintenance operation and ordered operation pair across valid writer/reader splits - add five curated deeper lifecycles, including the two-vector-delta plus scalar-unindexed-row state from #3769 - compare scalar-index results with a full scan - compare indexed ANN prefilter results with exact index-free KNN and require recall@10 >= 0.5 - cap vector search at four shards and avoid exponential sequence growth - include `IVF_PQ` in manual compat-pair `all` ## Why is this needed? The existing sequence search covers scalar and FTS indexes but not vector/scalar prefilter interactions. That is the main remaining unique signal in the legacy recurring test. This adds deterministic, isolated cross-version coverage with correctness oracles before that recurring matrix is removed. ## Validation - deterministic bounded-generator test - current runtime: all 103 max-length-5 cases passed across four shards in 10.51s - Pylance 9.0.1 writer to 10.0.0 reader: - all 103 cases passed in one shard - the default four-shard run passed in 9.07s after environment setup - existing `BTREE` sequence smoke passed - `uv run make lint` - workflow YAML parse - `git diff --check`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
the bug is that we may call
executemultiple times, and it causes a panic in datafusionRepartitionExec, the conditions to repro:This also improves the performance of vector search with prefilter when there are more than 1 deltas, because before it performed scalar index query & scanning for each delta, but now it does only once