Skip to content

[core] Add physical search and full-text archive primitives#8649

Merged
JingsongLi merged 1 commit into
apache:masterfrom
JingsongLi:codex/pk-fulltext-primitives
Jul 15, 2026
Merged

[core] Add physical search and full-text archive primitives#8649
JingsongLi merged 1 commit into
apache:masterfrom
JingsongLi:codex/pk-fulltext-primitives

Conversation

@JingsongLi

Copy link
Copy Markdown
Contributor

Summary

Introduce the reusable physical-position search model and immutable full-text archive primitives needed by primary-key table indexes. This isolates the storage and ranking foundation before exposing primary-key full-text configuration or query APIs.

Changes

  • Add shared physical search positions, scored results, deterministic ranking, and position-reader abstractions, then adapt the existing primary-key vector path to use them.
  • Extend source-backed primary-key index metadata to retain ordered source files and an optional definition fingerprint.
  • Add full-text archive state, projected data-file reading, archive building, and index-file handling with continuous row IDs across ordered sources.
  • Cover metadata compatibility, null-row ordinals, multi-source archives, reader ownership, ranking, and vector compatibility with focused tests.

Testing

  • JDK 8: compile paimon-core and its required modules
  • Run 9 focused core test classes for source metadata, full-text archive primitives, indexed split reading, ranking, and vector compatibility
  • Verify this is exactly one commit based on the target branch merge base

Notes

@JingsongLi JingsongLi marked this pull request as ready for review July 15, 2026 05:24
@JingsongLi JingsongLi force-pushed the codex/pk-fulltext-primitives branch 2 times, most recently from 35ad209 to 67620a6 Compare July 15, 2026 05:44
@JingsongLi JingsongLi force-pushed the codex/pk-fulltext-primitives branch from 67620a6 to 258f005 Compare July 15, 2026 06:02

@leaves12138 leaves12138 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Reviewed the physical search and full-text archive primitives, including the vector-index adaptations. The related paimon-core tests pass locally.

@JingsongLi JingsongLi merged commit 31a3e0a into apache:master Jul 15, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants