fix(index): make vector append segment-set-native - #8047
Merged
Conversation
Xuanwo
marked this pull request as ready for review
July 28, 2026 09:37
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
BubbleCal
reviewed
Jul 28, 2026
BubbleCal
left a comment
Contributor
There was a problem hiding this comment.
I left three inline comments on compatibility, logical-index validation, and the ANN regression test.
BubbleCal
approved these changes
Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #6398.
A logical vector index can contain immutable physical segments trained with different IVF and quantizer models. The append path currently validates the entire segment set before selecting work and then tries to encode unindexed fragments as though every segment shared one model. This makes append fail on valid heterogeneous segment sets.
This makes append segment-set-native. Append builds one new physical segment over only unindexed fragments using the complete model of the deterministic manifest suffix segment, while preserving all existing segment metadata and fragment coverage. Explicit merge selects its requested suffix before compatibility validation, and steady-state rebalance continues to rewrite only one segment. Explicit retrain remains the operation that source-rebuilds and unifies models.
The compatibility guard remains fail-closed and now applies to legacy and V3 storage, including IVF, quantizer/codebook/rotation, metric, dimension, and index-type metadata. This intentionally leaves distributed merge planning in #7730 out of scope.