Unified search: emit rendered-html/css on /_search render requests by habdelra · Pull Request #5171 · cardstack/boxel

habdelra · 2026-06-09T17:09:57Z

Phase 1 / Stream A step 3 (server turn-on) of Unify prerendered-card-search and card-search: /_federated-search starts honoring render and emitting rendered-html / css.

Stacked on #5167. Review the top commit here; base retargets to main as the stack lands.

Behavior

A request with an explicit render member is prefer-HTML: each result resolves to prerendered HTML where the index has it and falls back to the full live card otherwise — both in one data array.

HTML-backed row → an identity-only card (id + meta incl. renderType + meta.identityOnly + links.self + a rendered-html relationship, no attributes) plus a rendered-html resource (and its deduped css, id = md5(href)) in included.
No-HTML row → the full live card, exactly as /_search returns today (loadLinks runs over only these rows).
dataOnly: true → full live cards only.

A request without render (and without dataOnly) returns the live-card document unchanged. Prefer-HTML emission is gated on an explicitly-provided render member so consumers that don't yet read the unified shape are unaffected regardless of merge order; flipping to the project default (omitting render → prefer-HTML) is tracked on the cleanup ticket, once the host consumers migrate. The prerendered endpoints keep their existing path as the compat shim — byte-identical, covered by the existing prerendered goldens.

Implementation

runtime-common/unified-search.ts: pure builders (css / rendered-html / identity-only card) + the per-row used-render-type / scoped-css-dep helpers.
RealmIndexQueryEngine.searchUnified: one render projection, the per-row resolution policy, css deduped in included, render-type echo. The projection now also selects types so an identity-only card carries its actual adoptsFrom (distinct from the ancestor it was rendered as).
Realm.search routes to searchUnified when render is present; searchRealms / combineSearchResults merge the unified document by (type, id).
handle-search resolves the render type (filter.on default, "native" escape valve) and folds the resolved spec into the job-scoped cache key.

Tests

Mixed-index wire test: index, null the HTML for one row, then assert rendered-html + identity-only card for HTML rows and a full live card for the nulled row, all in one data array.
dataOnly is byte-identical to a plain live query; cache-key segregation by render / dataOnly / cardUrls; the resource builders; (type, id) merge dedup.
The 14 prerendered compat goldens stay byte-identical.

Locally: runtime-common + realm-server type-check clean; unified-search contracts 26/26; the search-test + search-prerendered shards 54/54.

🤖 Generated with Claude Code

/_federated-search honors an explicit render member: it resolves each result to prerendered HTML where indexed and falls back to the full live card otherwise. A request without render (and dataOnly) returns the live-card document unchanged, so consumers not yet reading the unified shape are unaffected; the prerendered endpoints keep their existing path as the compat shim. - runtime-common/unified-search.ts: pure builders for the css (md5 of the encoded href), rendered-html, and identity-only card resources, plus the used-render-type / scoped-css-dep helpers the result mapper runs per row. - RealmIndexQueryEngine.searchUnified: one render projection. HTML rows ship an identity-only card + rendered-html (deduped css in included); no-HTML rows ship the full live card assembled via loadLinks over only those rows; the render type is echoed; a file-meta query resolves to the live document. The render projection now also selects types so an identity-only card carries its actual adoptsFrom (distinct from the ancestor it was rendered as). - Realm.search routes to searchUnified when render is present; SearchOpts carries the resolved render spec; searchRealms / combineSearchResults already merge the unified document by (type, id). - handle-search resolves the render type (filter.on default, "native" escape valve), gates emission on an explicitly-provided render member, and folds the resolved spec into the job-scoped cache key. Tests: the mixed-index wire test (null the HTML for one row, assert rendered-html + identity-only card for HTML rows and a full live card for the nulled row in one data array); dataOnly is byte-identical to a plain live query; cache-key segregation by render / dataOnly / cardUrls; the resource builders; and the prerendered compat goldens stay byte-identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6407f94afa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

… unrenderable formats, fix host mock type - searchUnified now applies the query's sparse fieldset and threads linkFields to loadLinks for the live fallback rows, so a fallback card matches exactly what the same /_search query returns (it previously over-serialized). - handle-search rejects an explicit render.format that is a valid Format but not backed by prerendered HTML (e.g. isolated / edit) with a 400 invalid- render, instead of silently coercing it to fitted. - Widen the host realm-server-mock's SearchableRealm.search return type to the unified document so a Realm (whose search now returns it) stays assignable — fixes the host lint:types failure. Tests: a render request with an empty sparse fieldset yields a fallback card with no attributes; an unsupported render.format returns 400. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

github-actions · 2026-06-09T17:42:59Z

Preview deployments

Host staging preview

Host production preview

Host Test Results

1 files 1 suites 1h 51m 19s ⏱️
3 002 tests 2 987 ✅ 15 💤 0 ❌
3 021 runs 3 006 ✅ 15 💤 0 ❌

Results for commit 41c8d00.

Realm Server Test Results

1 files ± 0 1 suites ±0 10m 6s ⏱️ -41s
1 606 tests + 14 1 606 ✅ + 14 0 💤 ±0 0 ❌ ±0
3 020 runs +1 337 3 020 ✅ +1 337 0 💤 ±0 0 ❌ ±0

Results for commit a4b2513. ± Comparison against earlier commit 41c8d00.

…ch-_search-renderdataonly-members-emit-rendered # Conflicts: # packages/runtime-common/index.ts # packages/runtime-common/realm-index-query-engine.ts

Copilot

Pull request overview

This PR turns on “prefer-HTML” emission for /_federated-search only when the caller explicitly includes a render member, returning unified results where each row either (a) ships prerendered HTML (rendered-html + deduped css) alongside an identity-only card, or (b) falls back to a full live card when HTML is missing.

Changes:

Add pure unified-search builders (css / rendered-html / identity-only card) and helpers for parsing/deriving render metadata.
Implement RealmIndexQueryEngine.searchUnified() and route Realm.search() to it when opts.render is present, while keeping the live-card response unchanged when render is omitted.
Update federated handler behavior and extend endpoint/contract tests to cover mixed HTML-backed + live fallback results, cache-key segregation, and format validation.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
packages/runtime-common/unified-search.ts	Adds pure helper/builder functions for unified-search resources and render metadata parsing.
packages/runtime-common/search-utils.ts	Extends `SearchOpts` and updates federated search typing to return unified documents.
packages/runtime-common/resource-types.ts	Adds `meta.renderType` to identity-only `card` metadata.
packages/runtime-common/realm.ts	Routes `Realm.search()` to `searchUnified()` when `opts.render` is provided.
packages/runtime-common/realm-index-query-engine.ts	Implements `searchUnified()` mapping logic, CSS dedupe, and fallback `loadLinks` behavior.
packages/runtime-common/index.ts	Exports unified-search helpers from the runtime-common barrel.
packages/runtime-common/index-query-engine.ts	Extends the render projection to also select `types` for correct adoptsFrom on identity-only cards.
packages/realm-server/tests/unified-search-contracts-test.ts	Adds unit coverage for unified-search helper/builder functions.
packages/realm-server/tests/server-endpoints/search-test.ts	Adds/updates integration tests for prefer-HTML behavior, cache key behavior, and invalid formats.
packages/realm-server/handlers/handle-search.ts	Gates prefer-HTML emission on explicit `render`, validates prerenderable formats, and folds resolved render spec into cache key.
packages/host/tests/helpers/realm-server-mock/routes.ts	Updates mock realm typing to reflect unified search return shape.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Strip temporal phrasing from the comments added in this work (today's, not yet, unchanged) so each states the contract as timeless fact: the live-card document for a no-render request, the live /_search exclusion of error rows, and the full live card's standard wireformat. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ttachRealmInfo cast - handle-search.ts: fold `dataOnly` into the run-time `searchRealms` opts alongside `render`, so the run-time opts name the requested mode the same members the job-scoped cache key does. SearchOpts already declares `dataOnly`; absence of `render` still selects the live path, so this is contract-completeness with no behavior change. - realm-index-query-engine.ts: widen `attachRealmInfo` to accept `UnifiedSearchCollectionDocument` and narrow each resource by its `type` discriminant (card / file-meta) instead of casting the whole document to `LinkableCollectionDocument`. rendered-html / css resources are skipped by the discriminant exactly as the prior runtime guard skipped them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

habdelra · 2026-06-10T14:20:08Z

+    // make every key unique and defeat the cache) and ride here instead, where
+    // `searchRealms` stamps the SQL + loadLinks stages onto the collector this
+    // handler emits.
    let runSearchOpts =


[Claude Code 🤖]

Forwarding a Codex finding from the stacked host PR #5180.

cardUrls is folded into the cache key (cacheKeyOpts.cardUrls above) but not into runSearchOpts, which is what's passed to searchRealms → Realm.search. The query engine applies the IN (...) filter from opts.cardUrls (index-query-engine.ts ~620), so on this unified federated path a cardUrls-scoped request keys the cache correctly but isn't actually filtered at runtime — the filter is silently dropped.

Threading parsed.cardUrls into runSearchOpts when non-empty (alongside render / dataOnly) should restore it.

[Claude Code 🤖] Fixed. cardUrls now rides the run-time search opts, not just the cache key: the handler threads parsed.cardUrls into runSearchOpts, Realm.search forwards it into the engine opts, and it's added to SearchOpts. The SQL i.url IN (...) subset filter now actually narrows the result set. Added an endpoint test: a cardUrls-scoped federated query returns only the requested card.

habdelra · 2026-06-10T14:20:38Z

[Claude Code 🤖]

Forwarding a Codex finding from the stacked host PR #5180 (the host consumer relies on this).

combineSearchResults initializes meta as { page: { total: 0 } } and never carries meta.renderType from the per-realm docs, so a federated result loses the collection render type that the per-realm responses echo. The host consumer renders fallback/live card rows under doc.meta.renderType, so federated card rows would silently lose the resolved ancestor type (single-realm responses are unaffected). File rows carry no render type by design, so they're unaffected.

The resolved render type is consistent within a search, so carrying the first non-null per-realm meta.renderType onto the combined meta should suffice.

…s the federated merge Two review-finding fixes on the unified federated-search server path. cardUrls subset filter: a `cardUrls`-scoped request folded the subset into the cache key but not the run-time search opts, so the SQL `i.url IN (...)` filter was never applied — the full result set came back, cached under a subset-specific key. Thread `cardUrls` from the handler through `Realm.search` into the engine opts (and add it to `SearchOpts`) so the subset actually narrows the results. combineSearchResults renderType: the federated merge initialized fresh metadata and summed only `page.total`, dropping the collection-level `meta.renderType` the per-realm docs echo. A host consumer renders live/fallback card rows under it, so they hydrated with `undefined`. Carry the first non-null per-realm `meta.renderType` onto the combined document. Tests: a cardUrls-scoped federated query returns only the requested subset; combineSearchResults preserves the resolved renderType and fabricates none for native/per-row searches. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

habdelra · 2026-06-10T14:58:20Z

[Claude Code 🤖] Fixed the forwarded combineSearchResults finding (renderType dropped in the federated merge). The merge now carries the first non-null per-realm meta.renderType onto the combined document, so a host consumer renders live/fallback card rows under the resolved ancestor type; native/per-row searches (no collection-level type) stay unset. Both this and the cardUrls fix landed in a4b2513 on this branch, with contract + endpoint tests.

chatgpt-codex-connector Bot reviewed Jun 9, 2026

View reviewed changes

Comment thread packages/runtime-common/realm-index-query-engine.ts Outdated

Comment thread packages/realm-server/handlers/handle-search.ts Outdated

habdelra changed the base branch from cs-11432-unified-search-collapse-federated-runners-mergers-handlers to main June 9, 2026 23:31

Merge remote-tracking branch 'origin/main' into cs-11433-unified-sear…

10d3862

…ch-_search-renderdataonly-members-emit-rendered # Conflicts: # packages/runtime-common/index.ts # packages/runtime-common/realm-index-query-engine.ts

habdelra requested a review from Copilot June 9, 2026 23:35

Copilot started reviewing on behalf of habdelra June 9, 2026 23:35 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread packages/realm-server/handlers/handle-search.ts

Comment thread packages/runtime-common/realm-index-query-engine.ts Outdated

habdelra and others added 2 commits June 9, 2026 19:43

habdelra requested a review from a team June 9, 2026 23:54

habdelra commented Jun 10, 2026

View reviewed changes

FadhlanR approved these changes Jun 10, 2026

View reviewed changes

habdelra merged commit dd53478 into main Jun 10, 2026
97 of 102 checks passed

habdelra mentioned this pull request Jun 10, 2026

Unified search: FileDef parity in the host consumer (identity-only file-meta rows) #5180

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Unified search: emit rendered-html/css on /_search render requests#5171

Unified search: emit rendered-html/css on /_search render requests#5171
habdelra merged 6 commits into
mainfrom
cs-11433-unified-search-_search-renderdataonly-members-emit-rendered

habdelra commented Jun 9, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jun 9, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

habdelra Jun 10, 2026

Uh oh!

habdelra Jun 10, 2026

Uh oh!

habdelra commented Jun 10, 2026

Uh oh!

habdelra commented Jun 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

habdelra commented Jun 9, 2026

Behavior

Implementation

Tests

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview deployments

Host Test Results

Realm Server Test Results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

habdelra Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

habdelra Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

habdelra commented Jun 10, 2026

Uh oh!

habdelra commented Jun 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Jun 9, 2026 •

edited

Loading