feat(search): AND semantics, field boosting, recency + graph distance ranking

[tstapler]

Summary

• AND semantics: Multi-term queries now require all terms (FTS5 AND), fixing the root bug where FtsQueryBuilder was overriding FTS5's own AND default with an explicit OR join. OR fallback fires automatically when AND returns zero results so no query goes empty-handed.
• Field boosting: Page-title FTS hits receive a 5× score multiplier (PAGE_BOOST) over body-text hits, implemented in Kotlin to avoid a schema migration. bm25_score is now surfaced on SearchedPage/SearchedBlock; RankedSearchHit sealed class provides a unified ranked list in SearchResult.ranked.
• Recency signal: Exponential decay (RECENCY_HALFLIFE_DAYS = 14) gives results edited today up to 2× score boost, tapering to 1× for content older than ~6 weeks.
• Graph distance signal: Results on pages 1 hop away from the current page (via block_references) receive a 3× multiplier (GRAPH_BOOST). A single selectNeighbourPageUuids UNION query pre-fetches the full neighbour set — no N+1.

Test plan

• FtsQueryBuilderTest — 27 cases covering AND/OR semantics, phrase segments, injection stripping, buildOr() fallback
• SearchRepositoryIntegrationTests — 10 integration cases including:
  • AND semantics (only blocks with all terms match)
  • OR fallback fires when AND returns empty
  • Field boosting (page title ranks above block body)
  • Graph distance boost (linked-page block ranks above unlinked block)
  • Recency boost (recently-edited block ranks above 90-day-old block)
• Pre-existing BlockStateManagerTest.unobservePage_clears_state failure confirmed on main — not introduced by this PR

🤖 Generated with Claude Code

[github-actions]

Benchmark Results

Comparing 5c8d254 (this PR) vs 105928d (baseline)
Graph config: small — ? pages

Metric	This PR	Baseline	Delta
Phase 1 TTI	9ms	11ms	(-2ms)
Phase 2	11ms	3ms	(+8ms)
Phase 3 index	10ms	15ms	(-5ms)
Total	29ms	28ms	(+1ms)
Write p95 (baseline)	28ms	33ms	-
Write p95 (under load)	-1ms	-1ms	-
Jank factor	-0.04x	-0.03x	-

Flamegraph not available

Top allocation hotspots (this PR)

62.6% byte[][k]
9.6% java.lang.Object[][k]
4.5% java.lang.StringBuilder_[k]
2.5% java.lang.String_[k]
2.5% jdk.internal.org.objectweb.asm.SymbolTable$Entry_[k]

[Copilot]

Pull request overview

This PR updates the SQLDelight-backed search pipeline to use FTS5 default AND semantics (with an OR fallback), and introduces cross-type (page + block) ranking with field boosting, recency, and 1-hop graph-distance signals.

Changes:

• Switch FtsQueryBuilder multi-term joining to AND and add buildOr() for fallback queries.
• Surface bm25_score from FTS queries and add a unified RankedSearchHit list on SearchResult.
• Add 1-hop neighbor-page lookup SQL and apply page-title, recency, and graph-distance multipliers during ranking.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
kmp/src/commonMain/kotlin/dev/stapler/stelekit/search/FtsQueryBuilder.kt	Implements AND-joined queries and adds OR-building fallback helper.
kmp/src/jvmTest/kotlin/dev/stapler/stelekit/search/FtsQueryBuilderTest.kt	Updates/extends unit tests to validate AND semantics and buildOr() behavior.
kmp/src/commonMain/sqldelight/dev/stapler/stelekit/db/SteleDatabase.sq	Adds neighbor-page UUID query and exposes BM25 scores for blocks/pages FTS results.
kmp/src/commonMain/kotlin/dev/stapler/stelekit/repository/SqlDelightSearchRepository.kt	Adds OR fallback logic, computes ranked cross-type results with boosts, and populates BM25 fields.
kmp/src/commonMain/kotlin/dev/stapler/stelekit/repository/GraphRepository.kt	Extends search result models to carry BM25 and introduces ranked hit types.
kmp/src/jvmTest/kotlin/dev/stapler/stelekit/repository/SearchRepositoryIntegrationTests.kt	Adds integration tests for AND/OR fallback, field boost, graph boost, and recency boost.

Comments suppressed due to low confidence (2)

kmp/src/commonMain/sqldelight/dev/stapler/stelekit/db/SteleDatabase.sq:477

• In this query bm25(blocks_fts) is computed as bm25_score and then recomputed again in ORDER BY bm25(blocks_fts). Consider ordering by the selected alias to avoid evaluating BM25 twice per row.

    bm25(blocks_fts) AS bm25_score
FROM blocks_fts bm
JOIN blocks b ON b.id = bm.rowid
WHERE blocks_fts MATCH :query
ORDER BY bm25(blocks_fts)

kmp/src/commonMain/sqldelight/dev/stapler/stelekit/db/SteleDatabase.sq:527

• Same pattern in the pages FTS query: bm25(pages_fts) is selected as bm25_score and recomputed in ORDER BY bm25(pages_fts). Ordering by the alias can avoid the extra BM25 evaluation.

    bm25(pages_fts) AS bm25_score
FROM pages_fts pf
JOIN pages p ON p.rowid = pf.rowid
WHERE pages_fts MATCH :query
ORDER BY bm25(pages_fts)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The last assertion in this test repeats generateUuid(12) again, so the block that contains only "2025" (generateUuid(13)) is never asserted. This makes the test redundant and the message misleading; update the final assertion to check uuid 13 (or remove the duplicate assertion).

Suggested change

	assertFalse(generateUuid(12) in uuids, "Block with only 'meeting' should not match '2025 tax'")
	assertFalse(generateUuid(13) in uuids, "Block with only '2025' should not match '2025 tax'")

[Copilot]

Duration.Companion.milliseconds is imported but never used in this file. Please remove it to avoid unused-import warnings and keep imports tidy.

Suggested change

import kotlin.time.Duration.Companion.milliseconds

[Copilot]

abs(sp.bm25Score) is not a monotonic transform of BM25 across all possible return values. Since the SQL queries rank by ORDER BY bm25(...) (smaller = more relevant), converting to a cross-type “higher is better” score should use a monotonic mapping (e.g., -sp.bm25Score). With the current abs, if bm25() ever returns positive values, worse matches can incorrectly get higher scores.

[Copilot]

For SearchScope.CURRENT_PAGE, this code path executes only the AND-built ftsQuery via searchBlocksByContentFtsInPage(...) and does not apply the OR fallback used in the other search paths when the AND query returns zero rows. This can still produce empty results for multi-term queries in current-page scope.

[Copilot]

Same scoring issue for blocks: abs(sb.bm25Score) can invert relevance ordering if BM25 is positive. Use a monotonic transform consistent with ORDER BY bm25(...) (e.g., -sb.bm25Score) so higher score always means more relevant.

[Copilot]

The KDoc and constant name say “half-life”, but exp(-daysSince / RECENCY_HALFLIFE_DAYS) does not halve the bonus at RECENCY_HALFLIFE_DAYS (it yields ~0.367 of the bonus). If you want a true half-life, the exponent needs a ln(2) factor (or rename the constant/documentation to match the implemented decay).

[Copilot]

runBlocking is imported but not used anywhere in this test file. Please remove the unused import.

Suggested change

import kotlinx.coroutines.runBlocking

[Copilot]

selectNeighbourPageUuids uses SELECT DISTINCT ... in each branch, but the branches are combined with UNION (which is distinct by default). The inner DISTINCT is redundant work; consider removing it (or using UNION ALL with a single outer SELECT DISTINCT).

Suggested change

	SELECT DISTINCT to_b.page_uuid AS page_uuid
	FROM block_references br
	JOIN blocks from_b ON from_b.uuid = br.from_block_uuid
	JOIN blocks to_b ON to_b.uuid = br.to_block_uuid
	WHERE from_b.page_uuid = :pageUuid AND to_b.page_uuid != :pageUuid
	UNION
	SELECT DISTINCT from_b.page_uuid AS page_uuid
	SELECT to_b.page_uuid AS page_uuid
	FROM block_references br
	JOIN blocks from_b ON from_b.uuid = br.from_block_uuid
	JOIN blocks to_b ON to_b.uuid = br.to_block_uuid
	WHERE from_b.page_uuid = :pageUuid AND to_b.page_uuid != :pageUuid
	UNION
	SELECT from_b.page_uuid AS page_uuid