Skip to content

Hand over the corpus health panel and the privilege trap behind it - #2539

Merged
BigSimmo merged 4 commits into
mainfrom
claude/corpus-health-panel-na3hbs
Sep 2, 2026
Merged

Hand over the corpus health panel and the privilege trap behind it#2539
BigSimmo merged 4 commits into
mainfrom
claude/corpus-health-panel-na3hbs

Conversation

@BigSimmo

@BigSimmo BigSimmo commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Summary

What the handover records

The finding both fixes exist for. The cookie-bound user client cannot read documents or document_index_quality at all: schema.sql:5299 revokes all public table privileges from anon/authenticated, 20260725000000_audit_security_remediation.sql:81 re-applies that revoke after every earlier grant, and no later migration restores it. A policy cannot hand back a missing SQL SELECT, so the owner-read policies sit behind a privilege the role does not hold. Both modules degraded the resulting permission-denied to null by design, which is why the hub's document count rendered "document count unavailable" from the day it shipped and nobody noticed.

The variant check, recorded as closed rather than left as a hunt. Every createSupabaseServerClient caller in src/ was audited on main at 45a3dcacb: access.ts and src/app/auth/callback/route.ts are auth-only with no table read; src/lib/sources/document-source-loader.ts already used the correct shape (user client for the viewer id, admin client plus withOwnerReadScope for the query); the two developer-area modules are fixed. No third instance exists.

Six rules a later change must not undo, each named with the test that holds it — chiefly that the explicit owner_id filter is now the whole owner-scoping guarantee rather than a second layer over row-level security, and that restoring the user client would read nothing while looking healthy.

What is not verified, stated plainly: neither surface has been seen against the real library or on screen, because a cloud container has neither. The document carries the exact confirmation steps, including reading which of the five resolveQualitySpread cases the live data reports. The unverified uniform-placeholder-score report is why that derivation exists — but a uniform reading is a prompt to investigate, not a verdict. assessDocumentIndexQuality starts at 1 and only subtracts penalties before rounding to three decimals (src/lib/index-quality.ts:138-167), so a cleanly extracted library legitimately scores 1.000 on every document. Uniform 0.00 is the strong reading, because zero is also the column default; uniform 1.00 needs corroboration from the stored issues and metrics before anything is recorded against scoring. (Raised in review of this PR and corrected on the panel, the handover and the queued request in c12632a03.)

Verification

  • npm run verify:pr-local — green, all 19 steps completed, none failed:
 Test Files  947 passed (947)
      Tests  12055 passed | 1 skipped (12056)
✓ Compiled successfully in 117s
- completed: check:runtime, check:installed-lock-parity, format:changed, sitemap:check,
  check:repo-awareness-snapshot, docs:check-index, docs:check-inventory, docs:check-scripts,
  docs:check-links, check:branch-review-ledger, check:outstanding-issues,
  check:ledger-write-discipline, lint, typecheck, test, build, check:rag:fixtures,
  check:medication-interactions, check:medication-lexicon-report
- failed: (none)
- not reached: (none)

An earlier run of this same change failed check:repo-awareness-snapshot, because a new file under docs/ makes the committed snapshot stale. Regenerated with the command the gate names and re-run clean. Recorded rather than dropped.

  • npm run verify:ui — not applicable; no component, route, style or browser behaviour changes.
  • npm run verify:release before release or handoff confidence claims
  • npm run eval:retrieval:quality — not applicable; no retrieval, ranking, selection, chunking or scoring behaviour is touched.
  • npm run eval:rag -- --limit 15 + npm run eval:quality -- --rag-only — not applicable; answer generation is untouched.
  • npm run check:production-readiness — provider-backed, and not run. This PR adds no executable code: a Markdown document, an append-only ledger request, and two regenerated snapshots.
  • npm run check:deployment-readiness when deployment startup, hosting, or rollout behavior changed

Risk and rollout

  • Risk: None to runtime. No executable code changes. The realistic failure is the document being wrong, so every factual claim in it is either a file-and-line citation or a recorded gate result, and the variant-check table was produced by auditing each caller rather than from memory.
  • Rollback: Revert the commit. The document and the queued request disappear together; the two snapshots regenerate from whatever remains.
  • Provider or production effects: None. No migration, no schema change, no environment variable, no provider call.
  • RAG impact: none.

Clinical Governance Preflight

  • Source-backed claims still require linked source verification before clinical use
  • No patient-identifiable document workflow was introduced or expanded without explicit governance approval
  • Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
  • Service-role keys and private document access remain server-only
  • Demo/synthetic content remains clearly separated from real clinical sources
  • Source metadata, review status, and outdated/unknown-source behavior remain conservative
  • Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Evidence for each, in order:

  1. No clinical claim is made or rendered. The document describes index bookkeeping and access control; it adds no content a clinician could read in place of a linked source.
  2. No document workflow was added or widened. Nothing executable changed, and the handover adds no upload, export, sharing or retrieval path.
  3. Untouched. No migration and no Supabase configuration change; the project ref appears in this PR only inside the preflight line above.
  4. Reinforced rather than altered. The document's whole purpose is to record why these reads must run server-side through the service-role client with an explicit owner filter, and to stop a later change reverting them to a client that reads nothing.
  5. Unchanged. The handover notes that demo mode has no Supabase configuration, so no read is attempted and the panel renders its explicit "Nothing was read" notice rather than presenting synthetic figures as the real library.
  6. More conservative, not less: the recorded rules are the null-never-zero rule, the per-read guards, and the requirement that the page state what an empty or healthy reading does not prove.
  7. Checked, and the classification is unchanged: this adds no decision-support behaviour and no executable code. It documents an internal developer diagnostic behind the developer-area gate.

Notes

  • The queued request is docs/outstanding-issues-inbox/f5078359-d4ea-4a91-b376-d5bbecb452a6.json. Run npm run issues:reconcile from a dedicated fresh-base branch after this lands; it is append-only and merge-safe until then. (It replaced 42fd3baa-589b-4f2e-9434-dc259810b9ef.json, which carried the same uniform-score overclaim; that request never reached main, so no immutable record was rewritten. The Cursor Bugbot note below still names the withdrawn id because it reviewed the earlier commit.)
  • Both snapshots are generated files. If either conflicts with another PR, regenerate rather than hand-resolving.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XG7wQurapeZwWRsNhHA1PY


Generated by Claude Code


Note

Low Risk
Documentation and generated ledger snapshots only; no runtime, migration, or credential behavior changes in this diff.

Overview
Adds docs/corpus-health-panel-handover.md so a later session knows what merged in #2504/#2512, why the cookie-bound Supabase client could not read documents / document_index_quality, and six rules (service-role reads, explicit owner_id, null not 0, etc.) that must not be reverted. It also records that nothing has been verified against a live library and lists concrete confirmation steps.

Queues ledger inbox 42fd3baa-589b-4f2e-9434-dc259810b9ef (action: add) so startup/reconcile surfaces a P2 task to exercise the developer hub document count and /mockups/development/corpus-health with real Supabase + an admin session.

Regenerates data/repo-awareness-snapshot.json (new doc entry, refreshed revision/counts) and data/outstanding-issues-snapshot.json (ledger SHA, pending: 0 → 9, including this add plus other pending reconcile actions bundled in the snapshot). No application or schema code changes.

Reviewed by Cursor Bugbot for commit ab8ee08. Configure here.


Note

Cursor Bugbot is generating a summary for commit 8338b04. Configure here.

…ege trap behind it

Both changes are merged and nothing is in flight, so this records what a fresh
session needs and what it must not undo.

The substance is the finding both fixes exist for: the cookie-bound user client
cannot read `documents` or `document_index_quality` at all. `schema.sql:5299`
revokes all `public` table privileges from `anon`/`authenticated`, migration
`20260725000000` re-applies that revoke after every earlier grant, and no later
migration restores it -- so the owner-read policies sit behind a privilege the
role does not hold, and a policy cannot hand back a missing SQL SELECT. Both
modules degraded the resulting permission-denied to `null` by design, which is
why the hub's document count rendered "document count unavailable" from the day
it shipped and nobody saw it.

The variant check is recorded as closed rather than left as a hunt: every
`createSupabaseServerClient` caller in `src/` was audited on main at 45a3dca.
`access.ts` and the auth callback are auth-only; `document-source-loader.ts`
already used the correct shape (user client for the viewer id, admin client plus
`withOwnerReadScope` for the query). No third instance exists.

Also records the six rules a later change must not undo -- chiefly that the
explicit `owner_id` filter is now the whole owner-scoping guarantee rather than a
second layer over row-level security, and that restoring the user client would
read nothing while looking healthy -- and names the tests that hold each one.

What is NOT verified is stated as plainly: neither surface has been seen against
the real library or on screen, because a cloud container has neither. The
handover carries the exact confirmation steps, including reading which of the
five `resolveQualitySpread` cases the live data reports -- the unverified
uniform-placeholder-score report is the reason that derivation exists, and a
`uniform` reading is a finding against the scoring pipeline rather than a panel
defect.

A ledger inbox request queues the same confirmation, so a fresh session is told
at startup rather than having to find this file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XG7wQurapeZwWRsNhHA1PY
@coderabbitai

coderabbitai Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 5fe5166d-a88a-4a18-b7e3-79a8710e25ef


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@supabase

supabase Bot commented Sep 2, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-02T05:28:52.773358Z ab8ee08 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cursor

cursor Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_e76832a5-09bb-4902-84c8-79e5749e676f)

…panel-na3hbs

# Conflicts:
#	data/repo-awareness-snapshot.json
@BigSimmo
BigSimmo enabled auto-merge September 2, 2026 05:27

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ab8ee0825f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/corpus-health-panel-handover.md Outdated
…coring is broken

Raised as a P2 by Codex on #2539 and verified against the source before acting.
`assessDocumentIndexQuality` starts `qualityScore` at 1 and only subtracts
penalties, then rounds to three decimals (`src/lib/index-quality.ts:138-167`).
A corpus that extracted cleanly -- no issues, no duplicate chunks, adequate
heading and section-path coverage, nothing visual to score -- therefore lands on
`1.000` for every document, legitimately. Equality proves an absence of observed
spread and nothing more, so the false positive is not a corner case: it is what a
healthy library looks like.

Three surfaces carried the overclaim and all three are corrected:

- The panel told the reader the figures were unusable "until scoring writes real
  values", which presumes a fault. It now says the tie cannot rank or separate
  the documents, that this is not by itself evidence the scorer is broken and
  why, and to read the issues and metrics on the rows before concluding
  anything. The `0.00` note stays as it was: zero is the column default, so that
  reading really is what a corpus nobody scored looks like.
- The handover recorded a uniform reading as confirming the placeholder-score
  rumour. It now separates the strong case (uniform `0.00`, where
  `extraction_quality` also defaults to `unknown`) from the weak one (uniform
  `1.00`), and requires corroboration from the stored issues, metrics and the
  worker write path before anything is recorded against scoring.
- The queued ledger request said the same thing and is replaced with a corrected
  one. It had not reached `main`, so nothing immutable was rewritten.

This is the overclaiming the panel exists to avoid, in the panel's own prose --
the reason its caveats are page content pinned by tests rather than comments.
The corrected wording is pinned the same way, proven by deleting the caveat and
watching "says outright when every scored document carries the identical score"
go red, then restoring the file byte-identical by SHA-256.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XG7wQurapeZwWRsNhHA1PY

BigSimmo commented Sep 2, 2026

Copy link
Copy Markdown
Owner Author

Advisory UI is red on this head, and it is not this PR's failure. Recording it rather than leaving it silent, and deliberately not fixing it here.

What failed. One test, tests/ui-tools-search-mode-mockup.spec.ts:126 — "matches the exact displayed tool title after normalising punctuation". 127 passed, 2 skipped, 1 failed. The lane is continue-on-error: true, so it does not block the merge.

Locator: locator('[data-testid="tools-search-mode-mockup"]:visible').getByText('1 tool', { exact: true })
Expected: visible ... element(s) not found

Why it is not this PR's. The diff is one Markdown file, one ledger request and two regenerated snapshots — no executable code, so the mockup renders from exactly the same sources as on main. Confirmed by running the pure ranking function this page counts, rather than inferring it from the blast radius:

query="Compare"        count=2  -> differentials, clinical-dictionary
query="Risk & Safety"  count=3  -> risk-safety, safety-plan, differentials

So the page correctly renders 3 tools and the assertion still pins 1 tool. That is a stale assertion on main, not a flake — deterministic, and it will fail identically on any PR that touches a mockup surface (the lane only runs when advisory_ui_changed is true, which is why it has not been loud). The sibling test asserting 2 tools for Compare still passes, which is why only this one went red.

The fix, for whoever picks it up. Not ported here, because no fix exists to port yet and adding one would make a documentation-only PR carry an unrelated executable change — including through its governance preflight, which states no executable change. The count is incidental to what this test is actually about (that & normalises and the exact title matches), and this same file already avoids pinning absolutes elsewhere ("reads the rendered count rather than pinning an absolute"). So drop the count assertion rather than renumber it to 3 tools, which would only go stale again the next time a tool is added:

-    await expect(mockup.getByText("1 tool", { exact: true })).toBeVisible();
     await expect(mockup.getByRole("heading", { level: 2, name: "Risk & Safety" }).first()).toBeVisible();

Not verified in a browser from here — this container has no display and the check above was the ranking function, not the rendered page. Worth confirming the heading is the first result before landing it.


Generated by Claude Code

…panel-na3hbs

# Conflicts:
#	data/outstanding-issues-snapshot.json
@cursor

cursor Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_fc1c810d-2095-4d53-a643-4036d50d414f)

BigSimmo commented Sep 2, 2026

Copy link
Copy Markdown
Owner Author

Updating my note above on one point: a fix now exists. #2554 carries it — it drops the absolute count and asserts the ranking instead, which is what that test is named for.

Deliberately not ported into this PR. The usual reason to port a fix rather than wait is that the failure blocks the PR; this one cannot — Advisory UI is continue-on-error: true, and every required check here is green on 8338b0411. Porting a test change into a documentation-only PR would mean rewriting its own summary and governance preflight, both of which state that nothing executable changed, to buy a lane that was never blocking. So the same failure will show once more on this PR and disappear once #2554 lands.

Everything else above stands: deterministic, not a flake, and not caused by this diff.


Generated by Claude Code

@BigSimmo
BigSimmo merged commit 686ce37 into main Sep 2, 2026
30 of 33 checks passed
@BigSimmo
BigSimmo deleted the claude/corpus-health-panel-na3hbs branch September 2, 2026 06:42
BigSimmo pushed a commit that referenced this pull request Sep 2, 2026
Second sync: main advanced again (PR #2539) touching the same generated
data/repo-awareness-snapshot.json. Regenerated with
npm run snapshot:repo-awareness against the merged tree rather than
hand-resolving the conflict.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FDiC2BK8XcPbstaJf7So2x
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants