Skip to content

fix(indexing): write FTS metadata even without partitions - #8482

Merged
westonpace merged 3 commits into
lance-format:mainfrom
cmccabe:colin_write_metadata_without_parts
Aug 11, 2026
Merged

fix(indexing): write FTS metadata even without partitions#8482
westonpace merged 3 commits into
lance-format:mainfrom
cmccabe:colin_write_metadata_without_parts

Conversation

@cmccabe

@cmccabe cmccabe commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

During a distributed FTS index build, write canonical metadata even when no partitions are produced. This distinguishes a valid empty index segment from a failed or incomplete build.

@github-actions github-actions Bot added A-index Vector index, linalg, tokenizer bug Something isn't working labels Aug 11, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added K-changes Latest Gatekeeper recommendation requests changes. and removed K-changes Latest Gatekeeper recommendation requests changes. labels Aug 11, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 11, 2026
During a distributed FTS index build, write canonical metadata even when no
partitions are produced. This distinguishes a valid empty index segment from a
failed or incomplete build.
@cmccabe
cmccabe force-pushed the colin_write_metadata_without_parts branch from 802dd82 to 90a7983 Compare August 11, 2026 18:11
@lance-gatekeeper lance-gatekeeper Bot removed the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 11, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 11, 2026
@LuQQiu

LuQQiu commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

@cmccabe emmm wonder do we support search those empty partition FTS yet....
@BubbleCal do we support an FTS index segment without any partitions?

@LuQQiu
LuQQiu requested a review from BubbleCal August 11, 2026 19:25
@lance-gatekeeper lance-gatekeeper Bot removed the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 11, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

The reviewed FTS fix remains unchanged, and the additional transaction-test edits use the canonical file-version resolver without altering behavior. A finalized FTS segment with no partitions is supported: its end-to-end search returns an empty result rather than an error.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 11, 2026
@cmccabe

cmccabe commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

@LuQQiu : we do actually test loading an empty index in test_all_empty_string_documents_build_empty_index. If there is a bug related to empty index support, I think that would just merit a follow-on change?

@lance-gatekeeper lance-gatekeeper Bot removed the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 11, 2026
@westonpace
westonpace merged commit 67273e0 into lance-format:main Aug 11, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-index Vector index, linalg, tokenizer bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants