test: lock exact file-version wire contracts - #8019
Merged
Merged
Conversation
…mat-identity # Conflicts: # rust/lance-file/src/reader.rs # rust/lance/src/dataset/transaction.rs
…rsion-stack-02-fixtures
20 tasks
Xuanwo
marked this pull request as ready for review
July 29, 2026 05:33
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
BubbleCal
approved these changes
Jul 29, 2026
Xuanwo
added a commit
that referenced
this pull request
Jul 29, 2026
Part 3/12 of #7877. Builds on merged #8019. This is an independently reviewable step toward the final layout demonstrated in #7979. This PR makes `lance-file::versions::v1` the canonical owner of the frozen V1 reader, writer, metadata, page table, and encoding grammar. The ambiguous compiled `previous` namespace and `lance_io::encodings` compatibility layer are removed. V1 keeps its historical byte-buffer materialization private; legacy IVF payload I/O remains explicitly owned by its index format, while protobuf tensor conversion validates its own byte contract. No general Arrow conversion API is introduced. Manifest dictionary materialization now dispatches once on the exact manifest version and delegates V1 bytes to the V1 implementation. Current-format readers and writers are not reorganized here, and no later-version dispatch API is introduced. V1 wire behavior remains fixed by the checked-in fixtures from #8019. The historical `lance.file.previous.*` cache-key identifiers are intentionally retained because they are persisted cache identity, not source-level ownership.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 2/12 of #7877. Builds on merged #7879.
This is an independently reviewable step toward the final layout demonstrated in #7979.
Before moving reader and writer ownership, this PR locks stable file bytes and reader behavior with checked-in baseline fixtures for V1, V2.0, V2.1, and V2.2. Each stable standard writer must reproduce its fixture byte-for-byte, and the current readers validate every logical column and row across primitive, nullable UTF-8, nullable list, nullable dictionary, blob, multi-batch, and multi-page data.
V2.0 also has independent self-described and mini-Lance fixtures: the standard writer preserves footer
(0, 3), while both embedded forms preserve(2, 0). The V1 assertion explicitly preserves its historical null materialization semantics.The fixtures are reproducible from locked baseline commit
3a72f8a61e14613f517dded6816d4bfc77817c93; the checked-in generator requires two separate runs to agree before accepting the bytes. V2.3 remains unstable, so it has current-revision determinism and round-trip coverage instead of a historical fixture. No production dispatch changes are included.