Skip to content

Add excludeRowIds to search method#58

Draft
steFaiz wants to merge 4 commits into
apache:mainfrom
steFaiz:add_exclude_row_ids
Draft

Add excludeRowIds to search method#58
steFaiz wants to merge 4 commits into
apache:mainfrom
steFaiz:add_exclude_row_ids

Conversation

@steFaiz

@steFaiz steFaiz commented Jul 13, 2026

Copy link
Copy Markdown

Purpose

Parts of apache/paimon#8322

This PR add an excludeRowIds param to vector search, so that those deleted vectors will not affect choosing real-top-K similar vectors.

Core changes:

  1. Add a new method with optional including row ids and excluding row ids
  2. Move RowIdFilter struct to a new common file. Re export this class in ivfpq file to ensure backward compatibility
  3. NO public API changes

Tests

Add tests for core, c, c++, java, python module

@steFaiz steFaiz closed this Jul 13, 2026
@steFaiz steFaiz reopened this Jul 13, 2026
@steFaiz

steFaiz commented Jul 14, 2026

Copy link
Copy Markdown
Author

@JingsongLi PTAL if you have some time, thanks!

@steFaiz steFaiz marked this pull request as draft July 15, 2026 07:33
@steFaiz

steFaiz commented Jul 15, 2026

Copy link
Copy Markdown
Author

I think this may be not necessary. For RBM, storing includeRowIds as fullRange NOT exclusiveRowIds and directly storing exclusiveRowIds may have a similar storage cost.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant