fix(fts): propagate nested multimatch limits - #8073
Merged
BubbleCal merged 1 commit intoJul 29, 2026
Merged
Conversation
BubbleCal
marked this pull request as ready for review
July 29, 2026 12:17
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Xuanwo
approved these changes
Jul 29, 2026
BubbleCal
deleted the
yang/oss-1599-fix-nested-multimatch-limit-propagation-in-compound-fts
branch
July 29, 2026 13:59
sbrunk
added a commit
to sbrunk/lance
that referenced
this pull request
Jul 30, 2026
lance-format#8073 made `FtsSearchParams::limit` the recursive planning contract for FTS nodes: `Some(k)` permits bounded top-k execution, `None` requires every candidate so a compound parent can finish composing scores before its own fetch. `combined_fields` already honors it, but no test nested it under a compound parent, so a regression that let the ambient scanner limit reach the child would have gone unnoticed. Add `test_nested_combined_fields_limit_propagation` over MUST, SHOULD, `BoostQuery`, and the standalone query, mirroring `test_nested_multimatch_limit_propagation`, and lift the three-fragment two-column fixture both tests need into `compound_fts_dataset`.
sbrunk
added a commit
to sbrunk/lance
that referenced
this pull request
Jul 30, 2026
lance-format#8073 gave the `MultiMatch` arm the same `(score DESC, row_id ASC)` sort expressions `combined_fields_sort_exprs` builds, so the block existed twice verbatim. Rename the helper `fts_score_sort_exprs`, drop the combined_fields-only framing from its doc comment, and call it from both arms. The fts.rs test helper likewise hand-rolled the comparator lance-format#8073 added as `compare_scored_rows`; use that instead. Neither path changes behavior.
sbrunk
added a commit
to sbrunk/lance
that referenced
this pull request
Aug 2, 2026
lance-format#8073 gave the `MultiMatch` arm the same `(score DESC, row_id ASC)` sort expressions `combined_fields_sort_exprs` builds, so the block existed twice verbatim. Rename the helper `fts_score_sort_exprs`, drop the combined_fields-only framing from its doc comment, and call it from both arms. The fts.rs test helper likewise hand-rolled the comparator lance-format#8073 added as `compare_scored_rows`; use that instead. Neither path changes behavior.
sbrunk
added a commit
to sbrunk/lance
that referenced
this pull request
Aug 5, 2026
Upstream lance-format#8073 gave the `MultiMatch` arm the same `(score DESC, row_id ASC)` sort expressions `combined_fields_sort_exprs` builds, so the block existed twice verbatim. Rename the helper `fts_score_sort_exprs`, drop the combined_fields-only framing from its doc comment, and call it from both arms. No behavior change.
sbrunk
added a commit
to sbrunk/lance
that referenced
this pull request
Aug 6, 2026
Upstream lance-format#8073 gave the `MultiMatch` arm the same `(score DESC, row_id ASC)` sort expressions `combined_fields_sort_exprs` builds, so the block existed twice verbatim. Rename the helper `fts_score_sort_exprs`, drop the combined_fields-only framing from its doc comment, and call it from both arms. No behavior change.
sbrunk
added a commit
to sbrunk/lance
that referenced
this pull request
Aug 10, 2026
Upstream lance-format#8073 gave the `MultiMatch` arm the same `(score DESC, row_id ASC)` sort expressions `combined_fields_sort_exprs` builds, so the block existed twice verbatim. Rename the helper `fts_score_sort_exprs`, drop the combined_fields-only framing from its doc comment, and call it from both arms. No behavior change.
sbrunk
added a commit
to sbrunk/lance
that referenced
this pull request
Aug 10, 2026
Upstream lance-format#8073 gave the `MultiMatch` arm the same `(score DESC, row_id ASC)` sort expressions `combined_fields_sort_exprs` builds, so the block existed twice verbatim. Rename the helper `fts_score_sort_exprs`, drop the combined_fields-only framing from its doc comment, and call it from both arms. No behavior change.
sbrunk
added a commit
to sbrunk/lance
that referenced
this pull request
Aug 12, 2026
Upstream lance-format#8073 gave the `MultiMatch` arm the same `(score DESC, row_id ASC)` sort expressions `combined_fields_sort_exprs` builds, so the block existed twice verbatim. Rename the helper `fts_score_sort_exprs`, drop the combined_fields-only framing from its doc comment, and call it from both arms. No behavior change.
sbrunk
added a commit
to sbrunk/lance
that referenced
this pull request
Aug 14, 2026
Upstream lance-format#8073 gave the `MultiMatch` arm the same `(score DESC, row_id ASC)` sort expressions `combined_fields_sort_exprs` builds, so the block existed twice verbatim. Rename the helper `fts_score_sort_exprs`, drop the combined_fields-only framing from its doc comment, and call it from both arms. No behavior change.
sbrunk
added a commit
to sbrunk/lance
that referenced
this pull request
Aug 14, 2026
Upstream lance-format#8073 gave the `MultiMatch` arm the same `(score DESC, row_id ASC)` sort expressions `combined_fields_sort_exprs` builds, so the block existed twice verbatim. Rename the helper `fts_score_sort_exprs`, drop the combined_fields-only framing from its doc comment, and call it from both arms. No behavior change.
sbrunk
added a commit
to sbrunk/lance
that referenced
this pull request
Aug 16, 2026
Upstream lance-format#8073 gave the `MultiMatch` arm the same `(score DESC, row_id ASC)` sort expressions `combined_fields_sort_exprs` builds, so the block existed twice verbatim. Rename the helper `fts_score_sort_exprs`, drop the combined_fields-only framing from its doc comment, and call it from both arms. No behavior change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the bug?
BooleanQueryandBoostQueryrecursively plan their children without a limit so outer score composition remains exact. NestedMultiMatchQuery, however, applied its final fetch from the ambient scanner limit instead of the recursively supplied FTS search parameters.Linear: OSS-1599
What issues or incorrect behavior does the bug cause?
A nested MultiMatch could discard candidates before an outer MUST clause, SHOULD score accumulation, or BoostQuery demotion finished combining scores. This could omit the true top-k or return incomplete scores. Equal-score rows could also appear in a different order between bounded and exhaustive execution.
How does this PR fix the problem?
FtsSearchParams::limitas the recursive planning contract for MultiMatch fetches.Noneas complete execution for compound parents and document that future competitive-score pruning needs a separate contract._score DESC, _rowid ASC.This is a correctness and planning-semantics fix. It does not replace the current MultiMatch Union/Aggregate/Sort execution.
Validation
cargo test -p lance --lib test_nested_multimatch_limit_propagation -- --nocapturecargo test -p lance --lib io::exec::fts::tests -- --nocapture(15 passed)cargo test -p lance --lib dataset::tests::dataset_index::test_fts_ -- --nocapture(22 passed)cargo clippy -p lance -p lance-index --lib --tests -- -D warningscargo fmt --all -- --checkgit diff --check