From dedeb64ef1210f67ce782614bbcaf76bff27faab Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 10 Jul 2026 20:27:32 +1000 Subject: [PATCH 01/20] docs(plan): EQL v3 integration test suite, then type robustness MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan for a comprehensive real-crypto integration suite across the EQL v3 adapters, and the three refactors it unblocks. Reviewing #535 surfaced four structural gaps: - The Supabase v3 adapter has zero real-crypto coverage. Its only "live" suite stubs ZeroKMS with `hm-${plaintext}`, so equality assertions are true by construction. - The canonical per-domain types from `@cipherstash/eql` are imported nowhere; both adapters erase payloads to `unknown` and drop the Result wrapper. - v3 JSON is unimplemented, not merely untested — the bundle ships eql_v3_json but the SDK admits no 'json' cast kind. - The Supabase and Drizzle adapters ship from inside stack, unlike prisma-next. Sequenced tests first, so each later refactor lands against a suite that proves query correctness: PR1 harness + matrices, PR2 type robustness, PR3 adapter package split, PR4 v3 JSON. `docs/` does not ship in the stash tarball, so no changeset. --- .../2026-07-10-eql-v3-integration-tests.md | 320 ++++++++++++++++++ 1 file changed, 320 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md diff --git a/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md b/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md new file mode 100644 index 000000000..c259a17ad --- /dev/null +++ b/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md @@ -0,0 +1,320 @@ +# EQL v3 Integration Test Suite Plan (tests → type robustness → packaging → JSON) + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Give every EQL v3 adapter a **comprehensive, real-crypto integration suite** that proves query correctness against a real database — for each domain type, generate a sample of plaintext, single-encrypt some rows and bulk-encrypt the rest, insert them through the adapter, then exercise every valid query operation over a range of values and assert against a plaintext oracle. Then, standing on that suite, restore the erased EQL v3 types, split the adapters into their own packages, and add v3 JSON support. + +**Architecture:** One capability catalog, one plaintext oracle, one driver. Everything is **derived from `V3_MATRIX`**, never hand-listed: which operations must work, which must be rejected, which rows to seed. The catalog is `satisfies Record` keyed off the real `AnyEncryptedV3Column` union, so adding a domain to the SDK fails compilation until a catalog row exists. A single `IntegrationAdapter` interface abstracts Drizzle from Supabase, so each per-family test file is three lines and the two adapters cannot drift in what they claim to cover. Integration tests are **separate from unit tests** — own vitest config, own `test:integration` script, own CI workflow — and they **fail loudly** rather than skipping when credentials or a database are absent. EQL v3 is installed by the real `stash eql install --eql-version 3` CLI, so an installer regression fails CI instead of hiding behind a test-only code path. + +**Tech Stack:** TypeScript, `vitest` (`vitest run --config vitest.integration.config.ts`), `@cipherstash/stack` (`/eql/v3` catalog, `/v3` typed client, `/supabase`, `/eql/v3/drizzle`), `@cipherstash/eql@3.0.0` (canonical wire types; `/sql` release manifest), `@cipherstash/protect-ffi@0.29`, `postgres` (direct DDL), `@supabase/postgrest-js`, `drizzle-orm`, `stash` CLI (`packages/cli`), Docker (`ghcr.io/cipherstash/postgres-eql:17-2.3.1`, `supabase/postgres`, `postgrest/postgrest:v12.2.12`), `turbo`, `biome`. + +## Global Constraints + +1. **No `describe.skipIf` in integration suites.** A false gate is a silent whole-suite skip on a green job. Missing credentials, database, or PostgREST must throw with an actionable message. `AutoStrategy` lets a local `~/.cipherstash` profile stand in for `CS_*` env vars, so local dev stays frictionless. +2. **`pnpm test` must never run integration tests.** They live behind `test:integration` and their own workflows. +3. **Coverage is compile-enforced.** `V3_MATRIX` must keep its `satisfies Record`, and the test kit must be typechecked in CI, or a new domain silently goes untested. +4. **The oracle is the plaintext**, never a restatement of the query. Expected row sets are computed by filtering the seeded plaintext with `comparePlain`, not by asserting a literal. +5. **Never assert `localeCompare` ordering.** Strings compare by codepoint, dates by instant, bigints numerically. +6. **Connect direct to port 5432, never a pooler.** The `SET ROLE` / advisory-lock flakiness seen in `supabase-v3-grants-pg` is a PgBouncer transaction-mode artifact. + +## Ground-truth notes (verified against `feat/eql-v3-text-search-schema`, 2026-07-10) + +- **The Supabase v3 adapter has zero real-crypto coverage.** `packages/stack/__tests__/supabase-v3-pgrest-live.test.ts` runs against a real PostgREST but stubs ZeroKMS. Its equality term is `` `hm-${value}` `` (`__tests__/helpers/v3-envelope.ts:22`), so `.in('nickname', ['ada','nobody']) → ['ada']` is true by construction. The file header says as much: "What is faked is ZeroKMS — and only ZeroKMS." +- **`drizzle-v3/operators-live-pg.test.ts` is a genuine end-to-end suite** across all covered domains — real `EncryptionV3`, real Postgres, known rows, plaintext oracle (`plainValue`, `comparePlain`, `expectedKeysFor`, `sortedKeysFor`). It seeds **only** via `bulkEncryptModels`; the single-`encryptModel` insert path is never exercised. +- **The Supabase `insert()` branches on shape**: array → `bulkEncryptModels` (`query-builder.ts:452`), single object → `encryptModel` (`:474`). The live suite only ever inserts a single object, so the array path is untested. +- **The canonical EQL v3 types are imported nowhere.** `@cipherstash/eql@3.0.0` exports ~130 per-domain types (`IntegerOrd = { v, i, c, op: OpeCllw }` and its term-only twin `IntegerOrdQuery = { v, i, op }`), but no `.ts` source file imports them; only `@cipherstash/eql/sql` is consumed. Both adapters erase to `unknown`: `eql/v3/drizzle/operators.ts:51,55` type the client's `encrypt`/`bulkEncrypt` as returning `unknown`; `encryptOperands` returns `Promise`; `supabase/query-builder-v3.ts:393,433,463` do the same, and the `Result` wrapper collapses to `{ data?: unknown }`. +- **v3 JSON is unimplemented, not merely untested.** The bundle ships `public.eql_v3_json` and `public.eql_v3_jsonb_entry` (51 `CREATE DOMAIN`s, 56 `eql_v3.{ste_vec,jsonb_path,selector,contained_by}` functions), but the SDK models 41 domains and `eql/v3/columns.ts` admits only `bigint | boolean | date | number | string | timestamp` as `cast_as`. There is no `'json'` kind, so a v3 JSON column cannot be declared. **The gap is in the core v3 schema; both adapters inherit it.** `eql/v3/drizzle/codec.ts:38` already carries defensive SteVec decode logic (`sv[0].c`) for documents nothing can currently create. +- **`@cipherstash/stack/drizzle` is not `@cipherstash/drizzle`.** They are a fork: `@cipherstash/drizzle@3.0.3` peer-depends on the predecessor SDK `@cipherstash/protect@12` and exports `createProtectOperators` (2,038 lines); stack's in-tree copy exports `createEncryptionOperators` (1,945 lines); ~645 lines have diverged. There is no dependency between them — hence no cycle today, only duplication. Docs reference the stack subpath 13× and the package 2×. +- **Ordering is safe on OPE.** EQL 3.0.0 pins `_ord` domains to `op` (CLLW-OPE), which orders via a native `bytea` btree, so `ORDER BY eql_v3.ord_term(col)` works on every provider without superuser. `_ord_ore` domains remain block-ORE, are superuser-only, and are absent on managed Postgres — see `docs/eql-v3-ord-term-ordering-defect.md`. +- **`postgres-eql:17-2.3.1` ships EQL v2, not v3.** Every DB variant must install v3 at setup. +- **`./supabase` and `./drizzle` are published stack subpaths** (`npm view @cipherstash/stack exports`); `./eql/v3/drizzle` is not. +- **Moving an adapter out of stack requires promoting internals.** `src/supabase` imports six non-public modules (`@/encryption/helpers`, `@/encryption/operations/base-operation`, `@/eql/v3/columns`, `@/eql/v3/domain-registry`, `@/types`, `@/utils/logger`); `src/eql/v3/drizzle` imports three (`@/types`, `@/schema/match-defaults`, `@/encryption/operations/base-operation`). Note `@/types` is the internal `src/types.ts`, distinct from the published `./types` → `types-public`. + +## Sequencing + +Tests first, so every later refactor lands against a suite that actually proves query correctness. + +| PR | Contents | +|----|----------| +| **PR1** (this plan, Tasks 1–5) | Integration harness + Supabase v3 and Drizzle v3 real-crypto matrices. **No source moves.** | +| **PR2** | Type robustness: import `@cipherstash/eql` per-domain types; stop erasing `Result` and the query-type encoding. | +| **PR3** | Adapter package split: `@cipherstash/stack-drizzle`, `@cipherstash/stack-supabase`. Tests move with the code. | +| **PR4** | v3 JSON support: `types.Json()` + ste_vec, plus its family file in the existing matrix. v3 only. | +| *(independent, any time after PR1)* | Prisma-next live real-crypto suite; best-effort v2 coverage on the same harness. | + +**Why the split is PR3 and not PR1.** It means promoting nine internal stack modules to public API and removing two published subpaths. That is a larger, riskier change than the test work, and it is exactly the refactor that wants an integration suite standing behind it. Doing it first would also churn every test import twice. Tests therefore land in `packages/stack/integration/` and relocate in PR3; because the shared kit is a workspace package from day one, that move is a path change, not a rewrite. + +## File Structure + +``` +packages/test-kit/ # NEW — private, unpublished, no build step + package.json # "private": true; exports "." -> ./src/index.ts + src/ + catalog.ts # MOVED from stack/__tests__/v3-matrix/catalog.ts + families.ts # FamilyName, domainsForFamily() + oracle.ts # plainValue, comparePlain, expectedKeysFor, sortedKeysFor + rows.ts # planTable(), planRows() — data gen + single/bulk split + ops.ts # QueryOpKind, positiveOps(), negativeOps() + adapter.ts # IntegrationAdapter interface + run-family-suite.ts # the driver + env.ts # requireIntegrationEnv(), requireEncryptionClientV3() + install.ts # shells out to `stash eql install --eql-version 3` + index.ts + +vitest.shared.ts # NEW — resolve.alias block, spread into every vitest config + +local/ + docker-compose.postgres.yml # NEW — postgres-eql, no PostgREST + docker-compose.supabase.yml # NEW — supabase/postgres (by digest) + PostgREST + supabase-init.sql # NEW — authenticator LOGIN password + docker-compose.yml # existing; keep or alias + +packages/stack/ + integration/ # NEW — excluded from `pnpm test` + vitest.integration.config.ts + global-setup.ts # requireIntegrationEnv() — fails before docker work + supabase/ + adapter.ts # SupabaseAdapter implements IntegrationAdapter + {integer,smallint,bigint,real-double,numeric,date,timestamp,text,boolean}.integration.test.ts + wire.integration.test.ts # PORTED from __tests__/supabase-v3-pgrest-live.test.ts + drizzle-v3/ + adapter.ts # DrizzleAdapter implements IntegrationAdapter + {integer,smallint,bigint,real-double,numeric,date,timestamp,text,boolean}.integration.test.ts + relational.integration.test.ts # joins, exists/notExists, pagination, and/or/not, bigint + matrix-sql.integration.test.ts # PORTED from __tests__/v3-matrix/matrix-live-pg.test.ts + __tests__/v3-matrix/catalog.ts # becomes a one-line re-export shim + +.github/ + actions/integration-setup/action.yml # NEW — checkout + pnpm + node + install + workflows/integration-drizzle.yml # NEW — db: [postgres] + workflows/integration-supabase.yml # NEW — db: [supabase] +``` + +--- + +### Task 1: Commit this plan + +**Files:** +- Create: `docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md` + +`docs/` does not ship in the `stash` tarball (only `skills/` does, per `CLAUDE.md:21`), so no changeset is required. + +- [ ] **Step 1: Write the plan doc and commit** + +--- + +### Task 2: `packages/test-kit` — the shared harness + +Move the catalog into a private workspace package so both adapter suites (and, after PR3, both adapter packages) import one source of truth. Consumed as TypeScript source via a vitest alias so every package sees **one module instance** — stack's unit tests do `expect(builder(…)).toBeInstanceOf(spec.ColumnClass)`, which fails across module instances. + +**Files:** +- Create: `packages/test-kit/{package.json,tsconfig.json,src/*.ts}` +- Create: `vitest.shared.ts` +- Modify: `packages/stack/__tests__/v3-matrix/catalog.ts` → one-line re-export shim +- Modify: `pnpm-workspace.yaml`, every `vitest.config.ts` + +**Interfaces:** + +```ts +export type Plain = string | number | bigint | boolean | Date + +export type QueryOpKind = + | 'eq' | 'ne' | 'in' | 'notIn' + | 'gt' | 'gte' | 'lt' | 'lte' | 'between' | 'notBetween' + | 'contains' | 'order' | 'isNull' | 'isNotNull' + +export type QueryOp = + | { kind: 'eq'|'ne'|'gt'|'gte'|'lt'|'lte'; column: string; value: Plain } + | { kind: 'in'|'notIn'; column: string; values: Plain[]; asRawFilter?: boolean } + | { kind: 'between'|'notBetween'; column: string; lo: Plain; hi: Plain } + | { kind: 'contains'; column: string; needle: string } + | { kind: 'order'; column: string; direction: 'asc'|'desc' } + | { kind: 'isNull'|'isNotNull'; column: string } + +export interface IntegrationAdapter { + readonly name: 'drizzle' | 'supabase' + readonly supportedOps: ReadonlySet // supabase omits 'order' + readonly alwaysRejectedOps: ReadonlySet // supabase: {'order'} + setup(): Promise + teardown(): Promise + createTable(t: TableSpec): Promise + insertSingle(t: TableSpec, row: PlainRow): Promise // MUST hit encryptModel + insertBulk(t: TableSpec, rows: PlainRow[]): Promise // MUST hit bulkEncryptModels + run(t: TableSpec, op: QueryOp): Promise // → matching rowKeys + expectRejected(t: TableSpec, op: QueryOp): Promise +} + +export function runFamilySuite(family: FamilyName, make: () => IntegrationAdapter): void +``` + +Add one field to `DomainSpec`: + +```ts +scope: { covered: true } | { covered: false; reason: string } +``` + +The nine ORE domains (eight numeric/date `_ord_ore` plus `text_ord_ore`) are `{ covered: false, reason: 'ORE is superuser-only; absent on managed Postgres — follow-up' }`. The driver skips them and `console.info`s the reason. Compile-enforcement survives, the exclusion is visible, and the follow-up flips flags rather than adding rows. + +- [ ] **Step 1: Scaffold `packages/test-kit`** (private, no build, `exports: { ".": "./src/index.ts" }`), add to `pnpm-workspace.yaml` +- [ ] **Step 2: Move `catalog.ts`**, repoint its imports from `@/eql/v3`/`@/schema` to `@cipherstash/stack/eql/v3`/`@cipherstash/stack/schema`, add the `scope` field +- [ ] **Step 3: Leave the re-export shim** at `packages/stack/__tests__/v3-matrix/catalog.ts` so the ~10 unit tests importing it need no churn +- [ ] **Step 4: Add `vitest.shared.ts`** aliasing stack subpaths and `@cipherstash/test-kit` to source; spread into each package's vitest config +- [ ] **Step 5: Lift the oracle** (`plainValue`, `comparePlain`, `expectedKeysFor`, `sortedKeysFor`) out of `drizzle-v3/operators-live-pg.test.ts` into `oracle.ts` +- [ ] **Step 6: Write `ops.ts`** — the capability→operation table and `positiveOps`/`negativeOps` +- [ ] **Step 7: Write `rows.ts`** — `planTable()` and `planRows()` (see Task 4 notes on the single/bulk split) +- [ ] **Step 8: Ensure the kit is typechecked in CI** (add to `test:types`), then run `pnpm test` and commit + +--- + +### Task 3: The harness — docker, CLI install, loud env gate + +**Files:** +- Create: `local/docker-compose.postgres.yml`, `local/docker-compose.supabase.yml`, `local/supabase-init.sql` +- Create: `packages/stack/integration/{vitest.integration.config.ts,global-setup.ts}` +- Create: `.github/actions/integration-setup/action.yml` +- Modify: `turbo.json`, `packages/stack/{package.json,vitest.config.ts}` + +**Install via the real CLI.** The harness runs `stash eql install --eql-version 3 --database-url $DATABASE_URL` after `turbo run build --filter stash`. This replaces the hand-rolled `installEqlV3IfNeeded` + `SUPABASE_PERMISSIONS_SQL_V3` apply, and means every integration run exercises the installer customers actually use. `isInstalled` is generation-aware, so repeat local runs are cheap. + +**`supabase-init.sql`.** The `supabase/postgres` image already ships `anon`, `authenticated`, `service_role`, `authenticator`, and a non-superuser `postgres`. This file only guarantees the login PostgREST needs: + +```sql +ALTER ROLE authenticator WITH LOGIN PASSWORD 'authpass' NOINHERIT; +GRANT anon, authenticated, service_role TO authenticator; +``` + +**Do not** reuse `local/postgrest-roles.sql` here: its `IF NOT EXISTS` create-branch skips the password when the role already exists, and PostgREST auth then fails. + +**Loud failure.** `requireIntegrationEnv(requires)` throws, naming exactly what is missing and how to supply it. For CipherStash it accepts **either** the four `CS_*` vars **or** an existing `~/.cipherstash` profile — mirroring `examples/prisma/test/e2e/global-setup.ts:82-90` — then lets protect-ffi's AutoStrategy resolve. It runs in vitest `globalSetup` so it fails before any container work is wasted. + +- [ ] **Step 1: Write the two compose files** (`supabase/postgres` pinned by digest) and `supabase-init.sql`; comment the never-a-pooler constraint +- [ ] **Step 2: Bring both stacks up locally** and confirm `eql_v3.version()` matches the pinned release after `stash eql install --eql-version 3` +- [ ] **Step 3: Confirm the CLI's grants suffice for `anon` without a superuser connection** — the step most likely to surprise us. If they do not, wire a `SUPABASE_ADMIN_URL` for the install only +- [ ] **Step 4: Write `env.ts`** (`requireIntegrationEnv`, `requireEncryptionClientV3`) and `install.ts` +- [ ] **Step 5: Add the integration vitest config + `global-setup.ts`**; add `exclude: ['integration/**']` to the base config; add the `test:integration` script and the `turbo.json` task (`dependsOn: ['^build']`, `cache: false`) +- [ ] **Step 6: Verify** `pnpm test` runs zero integration tests, and that unsetting `PGRST_URL` produces a red failure with an actionable message — **not** a green skip +- [ ] **Step 7: Commit** + +--- + +### Task 4: Supabase v3 real-crypto integration suite + +The headline gap. Nine per-family files, each three lines; all the work lives in `adapter.ts` and the shared driver. + +**Files:** +- Create: `packages/stack/integration/supabase/adapter.ts` + nine `*.integration.test.ts` +- Create: `packages/stack/integration/supabase/wire.integration.test.ts` (ported) +- Create: `.github/workflows/integration-supabase.yml` +- Modify: `packages/stack/__tests__/live-coverage-guard.test.ts` (trim moved assertions) +- Delete: `packages/stack/__tests__/supabase-v3-pgrest-live.test.ts` (after porting) + +**Adapter notes.** Build through the public `encryptedSupabaseV3(client, { schemas, databaseUrl })` so the shipped construction path — introspection, declared-schema merge, real `Encryption({ eqlVersion: 3 })` — is what runs. `supportedOps` = all except `order`; `alwaysRejectedOps = {'order'}` (PostgREST cannot emit `ORDER BY eql_v3.ord_term(col)`, and the adapter refuses by design). Synthesize what the builder lacks: `between` → `.gte(lo).lte(hi)`; `notBetween` → `.or('c.lt.lo,c.gt.hi')`; `notIn` → `.or('c.not.in.(…)')`. + +**Single vs bulk.** `planRows` splits the value rows into disjoint halves: single = `{A, C}` via `.insert(obj)` (→ `encryptModel`), bulk = `{B, D}` via `.insert(array)` (→ `bulkEncryptModels`). The per-domain `eq`-over-every-row loop necessarily matches rows from both halves; on top of that, one explicit crossover assertion per family (a match-all predicate must return a superset of `{aSingle, aBulk}`) makes "the bulk path mangled a row" a red test rather than a coincidence. + +**Rejection matrix, derived.** + +```ts +const OPS_BY_CAPABILITY = { + equality: ['eq','ne','in','notIn'], + orderAndRange: ['gt','gte','lt','lte','between','notBetween','order'], + freeTextSearch: ['contains'], +} +// isNull/isNotNull are structural, never capability-gated. +``` + +Flipping a capability flag must flip a positive test into a negative one. Verify this by perturbing `text_match` to `equality: true` and watching its `eq` test change from a passing rejection to a failing query. + +**Why the wire suite survives.** `supabase-v3-pgrest-live.test.ts` is *not* superseded. It uniquely proves the `23514` rejection of a narrowed `encryptQuery`-shaped term (a shape the adapter cannot itself produce, so it must be hand-built), dense PostgREST parse edges, plaintext-passthrough containment on array/jsonb columns, and the grants as `anon`. The stub is the point: **no CipherStash credentials**, so it runs wherever the DB-only jobs run. Document the deliberate overlap at the top of the file, or someone will "clean up" one of the two. + +**The two PR #535 fixes map here.** The raw `.filter(col,'in',[…])` element-wise encryption fix is covered by running every eq-capable domain through *both* `.in()` and the raw-filter path (`asRawFilter: true`) against the same oracle with real ciphertext. The PostgREST select-alias `Date` reconstruction fix is covered in the `date` and `timestamp` family files by asserting `select('row_key, ts:')` yields a real `Date`; this needs a small `{ alias }` option on `SupabaseAdapter.run`, kept off the shared interface (Drizzle has no such concept). + +**`live-coverage-guard.test.ts` must be trimmed in this same commit.** It asserts `LIVE_SUPABASE_PGREST_ENABLED` is true in CI; once these suites leave `tests.yml`, that job stops setting `PGRST_URL` and the guard goes red. Delete only the assertions whose suites moved; ~15 non-adapter live suites still use `LIVE_*`. + +- [ ] **Step 1: Write `SupabaseAdapter`** against `encryptedSupabaseV3` +- [ ] **Step 2: Write one family file** (`integer`) and get it green against the supabase compose stack +- [ ] **Step 3: Verify the suite catches the #535 bugs** — revert only `packages/stack/src/supabase/`, confirm the raw-filter `in` and alias-`Date` tests go red, restore +- [ ] **Step 4: Verify the rejection matrix is derived** — perturb one capability flag, confirm the corresponding test flips, revert +- [ ] **Step 5: Fill in the remaining eight family files** +- [ ] **Step 6: Port `supabase-v3-pgrest-live.test.ts`** → `wire.integration.test.ts`, de-gated, with the overlap documented +- [ ] **Step 7: Trim `live-coverage-guard.test.ts`**; add `integration-supabase.yml` +- [ ] **Step 8: Run the full suite against `supabase/postgres`, then commit** + +--- + +### Task 5: Port the Drizzle v3 suite onto the harness + +**Files:** +- Create: `packages/stack/integration/drizzle-v3/adapter.ts` + nine `*.integration.test.ts` +- Create: `packages/stack/integration/drizzle-v3/relational.integration.test.ts` +- Create: `packages/stack/integration/matrix-sql.integration.test.ts` (ported) +- Create: `.github/workflows/integration-drizzle.yml` +- Delete: `packages/stack/__tests__/drizzle-v3/operators-live-pg.test.ts`, `__tests__/v3-matrix/matrix-live-pg.test.ts` (after porting) + +The per-domain logic becomes `DrizzleAdapter` + `runFamilySuite`. The bespoke, non-family-shaped tests move to `relational.integration.test.ts`: plain-table joins, `exists`/`notExists` correlated subqueries, `limit`/`offset` pagination, `and`/`or`/`not` disjoint-predicate proofs, the statically-typed bigint round-trip, and the >4-value bulk in-list. Text-specific edge guards (short-needle rejection, astral `👍`, NFD normalization) are richer than the capability matrix and stay as bespoke assertions in the text family file. + +`supportedOps` = all kinds; `alwaysRejectedOps` = ∅. Ordering uses `ORDER BY eql_v3.ord_term(col)`, safe because `_ord` is OPE. + +`matrix-live-pg.test.ts` is **kept, not deleted for redundancy** — it tests the `eql_v3.*` SQL operators directly, independent of any ORM (`contained_by`, empty-string domain CHECK accept/reject, storage round-trip). It moves to `matrix-sql.integration.test.ts` and is de-gated. + +This suite currently seeds only via `bulkEncryptModels`; the harness's single/bulk split closes that gap for free. + +- [ ] **Step 1: Write `DrizzleAdapter`** +- [ ] **Step 2: Port one family file**, confirm parity with the assertions it replaces +- [ ] **Step 3: Fill in the remaining eight** +- [ ] **Step 4: Move the bespoke tests** into `relational.integration.test.ts`; confirm nothing is lost by diffing the `it(` titles against the original +- [ ] **Step 5: Port `matrix-live-pg.test.ts`** → `matrix-sql.integration.test.ts`, de-gated +- [ ] **Step 6: Delete the originals**; add `integration-drizzle.yml` +- [ ] **Step 7: Run both workflows' suites locally, then commit** + +--- + +## CI + +One workflow per adapter, not an adapter×db matrix: the cross-product is meaningless (each adapter is fixed to one DB variant), and `paths:` filters are per-workflow — a shared workflow would pull the ~2 GB `supabase/postgres` image on Drizzle-only PRs. Keep `strategy.matrix.db` as a one-element list so the shape is ready if that changes. + +Both workflows: fork-PR gated via `if: github.event.pull_request.head.repo.full_name == github.repository` (a clean skip, as `prisma-next-e2e.yml` does); keep `require-cs-secrets` as a **fast pre-flight** before the docker pull; pass secrets via job-level `env:` rather than writing `.env` files (`dotenv/config` does not override an already-set `process.env`, so job env wins and ~5 `.env`-writing steps disappear). + +## Verification + +Observed, not assumed. + +1. **Harness boots.** `docker compose -f local/docker-compose.supabase.yml up -d --wait`, `stash eql install --eql-version 3`, then `eql_v3.version()` matches the pinned release and `curl -s localhost:3000/ | jq '.paths|keys'` lists the test table. +2. **Loud failure works.** Run with `PGRST_URL` unset → fails with the actionable message; does **not** skip green. +3. **Profile fallback works.** Unset all four `CS_*` with `~/.cipherstash` present → suite runs. Move the profile aside → suite fails loudly. +4. **The suite catches the bugs it exists for.** Revert only `packages/stack/src/supabase/` and re-run: the raw-filter `in` tests and the alias-`Date` tests must go red. (This is how the current PR #535 fixes were validated — 4 of 6 live tests failed against unfixed source.) +5. **The rejection matrix is derived, not decorative.** Give `text_match` `equality: true` → its `eq` test must turn from a passing rejection into a failing query. Revert. +6. **Ordering works on the managed-Postgres variant.** `asc`/`desc` on an `_ord` (OPE) column returns true plaintext order against `supabase/postgres`. +7. **CI shape.** `pnpm test` at the repo root runs zero integration tests; a fork PR shows the integration jobs as *skipped*, not failed. + +## PR2 — type robustness (outline) + +Import the canonical per-domain types from `@cipherstash/eql` and thread them through. Stop typing `OperandEncryptionClient.encrypt`/`bulkEncrypt` as returning `unknown` (`eql/v3/drizzle/operators.ts:51,55`); stop collapsing `Result<…>` to `{ data?: unknown }` (`:94-101`); stop returning `Promise` from `encryptOperands` and from `encryptCollectedTerms`/`bulkEncryptGroup`/`encryptGroupPerTerm` (`supabase/query-builder-v3.ts:393,433,463`). Where a `JSON.stringify` is genuinely required at the SQL boundary, it should serialize a *typed* envelope, not an `unknown`. The PR1 suite is what makes this safe to attempt. + +## PR3 — adapter package split (outline) + +``` +@cipherstash/stack (core) +@cipherstash/stack-drizzle deps: stack ← stack/src/drizzle (v2) + src/eql/v3/drizzle +@cipherstash/stack-supabase deps: stack ← stack/src/supabase +@cipherstash/prisma-next deps: stack (existing precedent) + +@cipherstash/drizzle@3.x peer: @cipherstash/protect@12 — untouched, maintenance +``` + +stack does **not** re-export the new packages, so no build cycle arises — the shape `@cipherstash/prisma-next` already proves in production. `@cipherstash/protect` is succeeded by stack, so `@cipherstash/drizzle@3.x` and `packages/protect` enter maintenance and the two forked Drizzle implementations stop diverging by attrition. Once protect sunsets, `stack-drizzle` can reclaim the `@cipherstash/drizzle` name in a major — do not attempt that now. + +Work: promote nine internal stack modules to public API; drop `./drizzle`, `./supabase`, `./eql/v3/drizzle` from `exports`/`typesVersions`/`tsup.config.ts` (stack is 0.19.0, so a minor bump carries the break under 0.x semver); update 15 doc/skill references; move the integration suites into the new packages; follow the `fta-v3.yml` complexity gate to the moved code or it silently loses its gate. + +## PR4 — v3 JSON (outline) + +Add a `'json'` `cast_as` kind and a `types.Json()` column to the core v3 schema, backed by `public.eql_v3_json` / `public.eql_v3_jsonb_entry` and the bundle's 56 ste_vec functions. Surface the query operations (`jsonb_path_exists`, selectors, containment) on both adapters. `eql/v3/drizzle/codec.ts:38` already decodes SteVec documents, so the read path is partly there. Add a `json` family file to the existing matrix. v3 only — v2 `searchableJson` is not a priority. + +## Risks + +- **Split blast radius (PR3).** Nine internal modules become public API, two published subpaths disappear, 15 doc references change. If any module is too internal to expose, relocate the shared piece to `packages/schema` or the test kit — a design call to make before coding, not during. +- **Two Drizzle implementations stay forked** until protect sunsets. Anyone fixing a bug in one should check the other; worth a comment atop both operator files. +- **`supabase/postgres` is ~2 GB.** Pin by digest, lean on the Blacksmith layer cache, rely on path-filtered triggering. If CI minutes bite, `postgres-eql` + `postgrest-roles.sql` reproduces the role surface at ~10% of the size — revisit then, not now. +- **CLI-based install adds a build step** (`turbo run build --filter stash`) to every integration job. Worth it: it is the only thing that tests the installer. +- **Source-aliasing stack's public surface in vitest** is the price of one shared catalog plus a working `instanceof`. Centralize it in `vitest.shared.ts` rather than copy-pasting. +- **Deliberate overlap** between the stub wire suite and the real-crypto matrix must be documented, or it will be "cleaned up". From e8d4587129b4dbf250506239b5d24d2bc00daf9a Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 10 Jul 2026 20:27:50 +1000 Subject: [PATCH 02/20] fix(stack): encrypt raw filter in-lists element-wise; restore alias Date MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two defects found reviewing #535, both verified against a real PostgREST. **Raw `.filter(col, 'in', […])` encrypted the whole list as one ciphertext.** The raw-filter path reached `in` with none of the element-splitting the `in()`, `not(…, 'in', …)` and `.or()` paths perform. The two wire formats then failed differently, which is why it went unnoticed: v2's `("json")` composite literal is already parenthesized, so PostgREST parsed it as a one-element list and answered `200 []` — a filter that silently matched nothing. v3's bare `{…}` envelope is not, so PostgREST rejected the request with PGRST100. Each element is now encrypted separately and the operand rendered as a quoted list literal, reusing `collectInListTerms` and `formatInListOperand`. As on the `not` path, a PostgREST list literal now throws — pass an array. Plaintext columns are untouched, including postgrest-js's own quirk of rendering `.filter(col,'in',[array])` as an unparenthesized `in.a,b`. **Aliased date columns lost their `Date` reconstruction.** `selectKeyToDb` was dropped from `buildSelectString`, so `.select('ts:createdAt')` returned an ISO string where the typed surface promises a `Date`. Restored by extracting `resolveSelectToken`, shared by `addJsonbCastsV3` and the new `selectKeyToDbV3` — the keys reported are now, by construction, the keys the cast helper causes PostgREST to emit, so the two cannot drift again. Six new tests, four of which fail against the unfixed source. --- .changeset/supabase-in-list-operands.md | 18 +++ .../__tests__/supabase-v3-builder.test.ts | 88 +++++++++++++++ .../__tests__/supabase-v3-pgrest-live.test.ts | 88 +++++++++++++++ packages/stack/src/supabase/helpers.ts | 103 +++++++++++++----- .../stack/src/supabase/query-builder-v3.ts | 39 +++++-- packages/stack/src/supabase/query-builder.ts | 49 ++++++++- 6 files changed, 342 insertions(+), 43 deletions(-) diff --git a/.changeset/supabase-in-list-operands.md b/.changeset/supabase-in-list-operands.md index dfd1fb2e5..360e23370 100644 --- a/.changeset/supabase-in-list-operands.md +++ b/.changeset/supabase-in-list-operands.md @@ -27,6 +27,24 @@ rendered as `not.in.(…)`. Passing a PostgREST list literal (`'(a,b)'`) for an encrypted column now throws instead of silently matching nothing — pass an array. +**`filter(col, 'in', […])` encrypted the whole list as a single ciphertext.** +The raw `.filter()` path reached `in` with none of the element-splitting the +`in()`, `not(…, 'in', …)` and `.or()` paths perform, so the entire list operand +was encrypted as one equality term. The two wire formats then failed +differently, which is why this went unnoticed: **v2**'s `("json")` composite +literal is already parenthesized, so PostgREST parsed it as a one-element list +and answered `200 []` — a filter that silently matched nothing. **v3**'s bare +`{…}` envelope is not, so PostgREST rejected the request outright with +`PGRST100 (failed to parse filter)`. + +Each element is now encrypted separately and the operand rendered as a quoted +PostgREST list literal. As on the `not` path, passing a list literal +(`'(a,b)'`) for an encrypted column now throws instead — pass an array. + +Plaintext columns are unaffected, including the pre-existing quirk that +postgrest-js renders `.filter(col, 'in', [array])` as an unparenthesized +`in.a,b` that PostgREST rejects; pass a list literal there, or use `.in()`. + **`is(col, null)` is now allowed on every column**, including storage-only encrypted ones (`types.Boolean`, `types.Integer`, …). `is` is never encrypted and a NULL plaintext is stored as a SQL NULL, so `IS NULL` is not merely legal diff --git a/packages/stack/__tests__/supabase-v3-builder.test.ts b/packages/stack/__tests__/supabase-v3-builder.test.ts index e3fea4e5a..c72d55d92 100644 --- a/packages/stack/__tests__/supabase-v3-builder.test.ts +++ b/packages/stack/__tests__/supabase-v3-builder.test.ts @@ -977,6 +977,40 @@ describe('encryptedSupabaseV3 wire encoding', () => { '2026-01-02T03:04:05.000Z', ) }) + + // A caller-chosen PostgREST alias keys the row by neither the property nor + // the DB name, so the reconstruction has to follow the select string. This + // regressed once when the alias→DB map was dropped from `buildSelectString`. + it('reconstructs a Date on a row keyed by a caller-chosen alias', async () => { + const rows = [ + { id: 1, ts: fakeEnvelope(new Date('2026-01-02T03:04:05.000Z'), 'ts') }, + ] + const { es, supabase } = v3Instance(rows) + + const { data } = await es.from('users', users).select('id, ts:createdAt') + + // The alias survives into the emitted select, so PostgREST keys rows `ts`. + expect(supabase.callsFor('select')[0].args[0]).toBe( + 'id, ts:created_at::jsonb', + ) + const row = data![0] as unknown as Record + expect(row.ts).toBeInstanceOf(Date) + expect((row.ts as Date).toISOString()).toBe('2026-01-02T03:04:05.000Z') + }) + + // An alias onto the raw DB name takes the other `resolveSelectToken` branch. + it('reconstructs a Date on an alias of the raw DB column name', async () => { + const rows = [ + { at: fakeEnvelope(new Date('2026-01-02T03:04:05.000Z'), 'at') }, + ] + const { es } = v3Instance(rows) + + const { data } = await es.from('users', users).select('at:created_at') + + const row = data![0] as unknown as Record + expect(row.at).toBeInstanceOf(Date) + expect((row.at as Date).toISOString()).toBe('2026-01-02T03:04:05.000Z') + }) }) // `notFilterOperator` was asserted only on the operator, never on the @@ -1582,6 +1616,60 @@ describe('v3 raw filter() resolves the query type from the operator', () => { expect(status).toBe(200) }) + // The raw path reached `in` with no element-split, so the whole list was + // encrypted as one equality term and the filter matched nothing. It now + // mirrors the `not(…, 'in', …)` contract exactly. + it('encrypts a raw in-list element-wise, not as one whole-list term', async () => { + const { es, supabase } = v3Instance() + + const { error, status } = await es + .from('users', users) + .select('id') + .filter('nickname', 'in', ['ada', 'grace']) + + expect(error).toBeNull() + expect(status).toBe(200) + + const [call] = supabase.callsFor('filter') + expect(call.args[0]).toBe('nickname') + expect(call.args[1]).toBe('in') + + // One envelope per element, quoted into a PostgREST list literal — never a + // single ciphertext of the literal string `("ada","grace")`. + const operand = call.args[2] as string + expect(operand.startsWith('(')).toBe(true) + expect(operand.endsWith(')')).toBe(true) + const plaintexts = [...operand.matchAll(/\\"pt\\":\\"([^\\]+)\\"/g)].map( + (m) => m[1], + ) + expect(plaintexts).toEqual(['ada', 'grace']) + }) + + it('rejects a raw in-list passed as a PostgREST list literal', async () => { + const { es } = v3Instance() + + const { error, status } = await es + .from('users', users) + .select('id') + .filter('nickname', 'in', '("ada","grace")') + + expect(status).toBe(500) + expect(error?.message).toMatch(/requires an array of values/) + }) + + it('leaves a raw in-list on a plaintext column untouched', async () => { + const { es, supabase } = v3Instance() + + const { error } = await es + .from('users', users) + .select('id') + .filter('id', 'in', [1, 2]) + + expect(error).toBeNull() + const [call] = supabase.callsFor('filter') + expect(call.args[2]).toEqual([1, 2]) + }) + it('rejects cs on an ord column, which has no freeTextSearch capability', async () => { const { es } = v3Instance() diff --git a/packages/stack/__tests__/supabase-v3-pgrest-live.test.ts b/packages/stack/__tests__/supabase-v3-pgrest-live.test.ts index cbaf6470e..66c57a436 100644 --- a/packages/stack/__tests__/supabase-v3-pgrest-live.test.ts +++ b/packages/stack/__tests__/supabase-v3-pgrest-live.test.ts @@ -509,6 +509,94 @@ describeLiveSupabasePgrest('supabase v3 adapter over real PostgREST', () => { expect(data).toEqual([]) }) + // The `in()` method path, executed for the first time against a real server. + // Each element is its own quote-dense envelope, so the operand is the densest + // list PostgREST has to parse outside an or-tree. + it('matches an encrypted in-list through the in() method', async () => { + const { data, error } = await from() + .select('row_key') + .in('nickname', ['ada', 'nobody']) + + expect(error).toBeNull() + expect(data.map((r: { row_key: string }) => r.row_key)).toEqual(['ada']) + }) + + // The RAW filter path reached `in` with no element-split: the whole list was + // encrypted as one equality term, so the request parsed and returned zero + // rows. A mock records the emitted operand and cannot tell that apart from a + // correct one — only a real server proves the predicate selects `ada`. + it('matches an encrypted in-list through the raw filter() path', async () => { + const { data, error } = await from() + .select('row_key') + .filter('nickname', 'in', ['ada', 'nobody']) + + expect(error).toBeNull() + expect(data.map((r: { row_key: string }) => r.row_key)).toEqual(['ada']) + }) + + // The same call must still EXCLUDE a row whose value is absent from the list. + // Without this, a filter that matched everything would pass the test above. + it('excludes a row whose value is absent from a raw in-list', async () => { + const { data, error } = await from() + .select('row_key') + .filter('nickname', 'in', ['nobody', 'someone']) + + expect(error).toBeNull() + expect(data).toEqual([]) + }) + + // A PostgREST list literal cannot be encrypted element-wise; the adapter + // refuses it rather than emit a filter that silently matches nothing. + it('rejects a raw in-list passed as a PostgREST list literal', async () => { + const { error } = await from() + .select('row_key') + .filter('nickname', 'in', '("ada","nobody")') + + expect(error?.message).toMatch(/requires an array of values/) + }) + + // A plaintext column keeps postgrest-js's own encoding, untouched. Passing a + // list LITERAL is the only form its raw `.filter()` renders correctly. + it('leaves a raw in-list literal on a plaintext column to postgrest-js', async () => { + const { data, error } = await from() + .select('row_key') + .filter('row_key', 'in', '("ada","nobody")') + + expect(error).toBeNull() + expect(data.map((r: { row_key: string }) => r.row_key)).toEqual(['ada']) + }) + + // Pins what we did NOT change. postgrest-js renders `.filter(col,'in',[…])` + // as an unparenthesized `in.ada,nobody`, which PostgREST rejects with + // PGRST100 — true of every plaintext column, before this change and after. + // The encrypted path cannot inherit that: it must take an array (nothing else + // can be encrypted element-wise) and so builds the literal itself. Asserting + // the asymmetry keeps a future "helpfully format plaintext arrays too" from + // landing unnoticed as a behaviour change. + it('does not rescue a raw in-list ARRAY on a plaintext column', async () => { + const { error } = await from() + .select('row_key') + .filter('row_key', 'in', ['ada', 'nobody']) + + expect(error?.code).toBe('PGRST100') + expect(error?.details).toContain('expecting "("') + }) + + // A caller-chosen alias keys the row by neither the property nor the DB name. + // PostgREST has to parse `ts:created_at::jsonb` and key the row `ts`, and the + // adapter has to follow that alias back to the column's `cast_as` to rebuild + // the `Date`. Both halves are only observable against a real server. + it('reconstructs a Date under a caller-chosen select alias', async () => { + const { data, error } = await from() + .select('row_key, ts:createdAt') + .eq('row_key', 'ada') + + expect(error).toBeNull() + expect(data).toHaveLength(1) + expect(data[0].ts).toBeInstanceOf(Date) + expect((data[0].ts as Date).toISOString()).toBe(ADA_CREATED.toISOString()) + }) + it('updates and deletes through encrypted WHERE operands', async () => { const updated = await from() .update({ note: 'changed' }) diff --git a/packages/stack/src/supabase/helpers.ts b/packages/stack/src/supabase/helpers.ts index 91ca28453..b3856bd60 100644 --- a/packages/stack/src/supabase/helpers.ts +++ b/packages/stack/src/supabase/helpers.ts @@ -117,42 +117,85 @@ export function addJsonbCastsV3( return columns .split(',') .map((col) => { - const trimmed = col.trim() - - if (!trimmed) return col - if (trimmed.includes('::')) return col - if (trimmed.includes('(') || trimmed.includes('.')) return col + const resolved = resolveSelectToken(col.trim(), propToDb, dbNames) + if (resolved === null) return col const leadingWhitespace = col.match(/^(\s*)/)?.[1] ?? '' + return `${leadingWhitespace}${resolved.emit}` + }) + .join(',') as DbSelect +} - // Already-aliased token: `alias:column` - const aliasMatch = trimmed.match( - /^([A-Za-z_][A-Za-z0-9_]*):([A-Za-z_][A-Za-z0-9_]*)$/, - ) - if (aliasMatch) { - const [, alias, name] = aliasMatch - const db = - lookupDbName(propToDb, name) ?? (dbNames.has(name) ? name : undefined) - if (db !== undefined) { - return `${leadingWhitespace}${alias}:${db}::jsonb` - } - return col - } +/** + * The result-row key each encrypted column comes back under for a given select + * string, mapped to its DB column name. + * + * The two differ whenever PostgREST renames: `createdAt` on a `created_at` + * column keys rows by `createdAt`, and a caller-chosen alias (`ts:createdAt`) + * keys them by `ts`. Consumers that resolve per-column config by DB name — + * `postprocessDecryptedRow` reading `cast_as` to rebuild `Date` values — need + * this bridge, because the row key alone does not identify the column. + * + * Shares {@link resolveSelectToken} with {@link addJsonbCastsV3} so the keys + * this reports are exactly the keys that helper causes PostgREST to emit. Any + * token the cast helper leaves untouched is absent here. + */ +export function selectKeyToDbV3( + columns: string, + propToDb: Record, +): Record { + const dbNames = new Set(Object.values(propToDb)) + const keyToDb: Record = Object.create(null) - const db = lookupDbName(propToDb, trimmed) - if (db !== undefined) { - return db === trimmed - ? `${leadingWhitespace}${trimmed}::jsonb` - : `${leadingWhitespace}${trimmed}:${db}::jsonb` - } + for (const col of columns.split(',')) { + const resolved = resolveSelectToken(col.trim(), propToDb, dbNames) + if (resolved !== null) keyToDb[resolved.key] = resolved.db + } - if (dbNames.has(trimmed)) { - return `${leadingWhitespace}${trimmed}::jsonb` - } + return keyToDb +} - return col - }) - .join(',') as DbSelect +/** + * Resolve one select-string token to the row key it produces, the DB column it + * names, and the `::jsonb`-cast text to emit for it. `null` for a token this + * helper does not rewrite: empty, already cast, a function call or foreign-table + * path (parens/dots), or a name belonging to no encrypted column. + */ +function resolveSelectToken( + trimmed: string, + propToDb: Record, + dbNames: ReadonlySet, +): { key: string; db: string; emit: string } | null { + if (!trimmed) return null + if (trimmed.includes('::')) return null + if (trimmed.includes('(') || trimmed.includes('.')) return null + + // Already-aliased token: `alias:column` + const aliasMatch = trimmed.match( + /^([A-Za-z_][A-Za-z0-9_]*):([A-Za-z_][A-Za-z0-9_]*)$/, + ) + if (aliasMatch) { + const [, alias, name] = aliasMatch + const db = + lookupDbName(propToDb, name) ?? (dbNames.has(name) ? name : undefined) + if (db === undefined) return null + return { key: alias, db, emit: `${alias}:${db}::jsonb` } + } + + const db = lookupDbName(propToDb, trimmed) + if (db !== undefined) { + return { + key: trimmed, + db, + emit: db === trimmed ? `${trimmed}::jsonb` : `${trimmed}:${db}::jsonb`, + } + } + + if (dbNames.has(trimmed)) { + return { key: trimmed, db: trimmed, emit: `${trimmed}::jsonb` } + } + + return null } /** diff --git a/packages/stack/src/supabase/query-builder-v3.ts b/packages/stack/src/supabase/query-builder-v3.ts index b76c8adee..ccdf7364f 100644 --- a/packages/stack/src/supabase/query-builder-v3.ts +++ b/packages/stack/src/supabase/query-builder-v3.ts @@ -12,7 +12,7 @@ import type { ScalarQueryTerm, } from '@/types' import { logger } from '@/utils/logger' -import { addJsonbCastsV3 } from './helpers' +import { addJsonbCastsV3, selectKeyToDbV3 } from './helpers' import { EncryptedQueryBuilderImpl, EncryptionFailedError, @@ -146,6 +146,15 @@ export class EncryptedQueryBuilderV3Impl< private columnSchemas: Record /** Column builders keyed by BOTH property name and DB name. */ private v3Columns: Record + /** + * Result-row key → DB column name for the columns the current select + * produces, including caller-chosen PostgREST aliases (`ts:createdAt` keys + * rows by `ts`). Populated by {@link buildSelectString}; consumed by + * {@link postprocessDecryptedRow} so aliased date columns still get `Date` + * reconstruction. Empty on paths with no recorded select (an insert or update + * that returns rows), which fall back to the static property/DB names. + */ + private selectKeyToDb: Record = Object.create(null) constructor( tableName: string, @@ -283,6 +292,7 @@ export class EncryptedQueryBuilderV3Impl< protected override buildSelectString(): DbSelect | null { if (this.selectColumns === null) return null + this.selectKeyToDb = selectKeyToDbV3(this.selectColumns, this.propToDb) return addJsonbCastsV3(this.selectColumns, this.propToDb) } @@ -534,18 +544,27 @@ export class EncryptedQueryBuilderV3Impl< protected override postprocessDecryptedRow( row: Record, ): Record { - const out: Record = { ...row } + // Every key an encrypted column can appear under: the keys this select + // actually produces (including caller-chosen aliases like `ts:createdAt`), + // plus the static property and DB names as a fallback for paths that record + // no select. Aliases win — `selectKeyToDb` describes the row in hand. + const keyToDb: Record = Object.assign( + Object.create(null), + this.selectKeyToDb, + ) for (const [property, dbName] of Object.entries(this.propToDb)) { + keyToDb[property] ??= dbName + keyToDb[dbName] ??= dbName + } + + const out: Record = { ...row } + for (const [key, dbName] of Object.entries(keyToDb)) { const castAs = this.columnSchemas[dbName]?.cast_as if (!DATE_LIKE_CAST_SET.has(castAs as string)) continue - // Rows are keyed by property name when selected via the aliasing cast - // helper, but a caller selecting by raw DB name gets DB-name keys. - for (const key of property === dbName ? [property] : [property, dbName]) { - const value = out[key] - if (value == null || value instanceof Date) continue - if (typeof value === 'string' || typeof value === 'number') { - out[key] = new Date(value) - } + const value = out[key] + if (value == null || value instanceof Date) continue + if (typeof value === 'string' || typeof value === 'number') { + out[key] = new Date(value) } } return out diff --git a/packages/stack/src/supabase/query-builder.ts b/packages/stack/src/supabase/query-builder.ts index 7f9d84af5..ca604e3d8 100644 --- a/packages/stack/src/supabase/query-builder.ts +++ b/packages/stack/src/supabase/query-builder.ts @@ -690,10 +690,32 @@ export class EncryptedQueryBuilderImpl< for (let i = 0; i < dbSpace.rawFilters.length; i++) { const rf = dbSpace.rawFilters[i] if (!isEncryptedColumn(rf.column, this.encryptedColumnNames)) continue - if (!isEncryptableTerm(rf.operator, rf.value)) continue const column = tableColumns[rf.column] if (!column) continue + if (rf.operator === 'in') { + // Same contract as the `not(…, 'in', …)` path: a PostgREST list literal + // (`'("a","b")'`) cannot be encrypted element-wise, and encrypting it + // whole matches nothing. Refuse it rather than emit a filter that + // silently returns no rows. + if (!Array.isArray(rf.value)) { + throw new Error( + `filter("${rf.column}", "in", …) on an encrypted column requires an array of values, ` + + `not a PostgREST list literal — each element must be encrypted separately`, + ) + } + collectInListTerms( + 'in', + rf.value, + column, + this.queryTypeForRawOp(rf.operator), + (inIndex) => ({ source: 'raw', rawIndex: i, inIndex }), + ) + continue + } + + if (!isEncryptableTerm(rf.operator, rf.value)) continue + pushTerm( rf.value as JsPlaintext, column, @@ -945,6 +967,7 @@ export class EncryptedQueryBuilderImpl< const notValueMap = new Map() const notInMap = new Map() // "notIndex:inIndex" -> value const rawValueMap = new Map() + const rawInMap = new Map() // "rawIndex:inIndex" -> value const orStringConditionMap = new Map() // "orIndex:condIndex" -> value const orStructuredConditionMap = new Map() @@ -974,7 +997,11 @@ export class EncryptedQueryBuilderImpl< } break case 'raw': - rawValueMap.set(mapping.rawIndex, encValue) + if (mapping.inIndex !== undefined) { + rawInMap.set(`${mapping.rawIndex}:${mapping.inIndex}`, encValue) + } else { + rawValueMap.set(mapping.rawIndex, encValue) + } break // `inIndex` widens the key to address one element of an `in` list, so a // whole-condition value and a per-element value never collide. @@ -1145,6 +1172,22 @@ export class EncryptedQueryBuilderImpl< // Apply raw filters for (let i = 0; i < dbSpace.rawFilters.length; i++) { const rf = dbSpace.rawFilters[i] + + // An encrypted `in` list was encrypted element-wise; reassemble it into + // the quoted PostgREST list literal, exactly as the `not` path does. A + // plaintext column keeps its operand untouched. + if ( + rf.operator === 'in' && + Array.isArray(rf.value) && + isEncryptedColumn(rf.column, this.encryptedColumnNames) + ) { + const values = rf.value.map((v, j) => + rawInMap.has(`${i}:${j}`) ? rawInMap.get(`${i}:${j}`) : v, + ) + q = q.filter(rf.column, rf.operator, formatInListOperand(values)) + continue + } + const value = rawValueMap.has(i) ? rawValueMap.get(i) : rf.value q = q.filter(rf.column, rf.operator, value) } @@ -1465,7 +1508,7 @@ type TermMapping = | { source: 'filter'; filterIndex: number; inIndex?: number } | { source: 'match'; matchIndex: number; column: string } | { source: 'not'; notIndex: number; inIndex?: number } - | { source: 'raw'; rawIndex: number } + | { source: 'raw'; rawIndex: number; inIndex?: number } | { source: 'or-string' orIndex: number From ce390d84ea427288fae06d3de683a0a28ddaab95 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 10 Jul 2026 20:58:11 +1000 Subject: [PATCH 03/20] test(test-kit): extract the v3 domain catalog into a shared harness package MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The integration suites need one catalog, one oracle, and one operation table shared across packages: after the adapter split, Supabase and Drizzle tests live in different packages, and neither may reach into stack's __tests__ tree. Duplicating the catalog would let the two copies disagree about which domains exist — exactly the drift the `satisfies Record` was written to prevent. `packages/test-kit` is private, unpublished, and has no build step; it is consumed as TypeScript source. That keeps `pnpm test` independent of `pnpm build`, which matters because the dts build is currently fragile. Resolution is aliased in three places that must stay in step: `vitest.shared.ts` (runtime), `packages/test-kit/tsconfig.json` and `packages/stack/tsconfig.json` (compile time). `tsc` does not read Vitest's aliases — without the tsconfig half, the catalog's builders lose their precise return types and the `.test-d.ts` matrix silently collapses to `never`. New `DomainSpec.deferred` records why a domain is skipped rather than dropping its row, so removing coverage stays a visible diff. The nine block-ORE domains are deferred: their opclass is superuser-only, so they are absent on managed Postgres. `test-kit-families.test.ts` asserts the families partition the catalog. The `satisfies Record<…>` only forces a row to EXIST; a domain whose slug matched no family prefix would belong to no test file and go untested while the suites stayed green. That test immediately caught such a bug: `eqlTypeSlug` strips only the `public.` schema, so slugs are `eql_v3_integer_ord`, and the family prefixes matched nothing. Verified: catalog coverage still fails the build when a domain row is removed (named explicitly); 1554 unit tests pass; stack and test-kit both typecheck. --- .github/workflows/tests.yml | 7 + .../stack/__tests__/test-kit-families.test.ts | 92 +++ packages/stack/__tests__/v3-matrix/catalog.ts | 308 +-------- packages/stack/tsconfig.json | 16 +- packages/stack/vitest.config.ts | 5 + packages/test-kit/package.json | 21 + packages/test-kit/src/adapter.ts | 60 ++ packages/test-kit/src/catalog.ts | 651 ++++++++++++++++++ packages/test-kit/src/families.ts | 114 +++ packages/test-kit/src/index.ts | 53 ++ packages/test-kit/src/ops.ts | 101 +++ packages/test-kit/src/oracle.ts | 102 +++ packages/test-kit/src/rows.ts | 106 +++ packages/test-kit/tsconfig.json | 42 ++ pnpm-lock.yaml | 13 + vitest.shared.ts | 44 ++ 16 files changed, 1434 insertions(+), 301 deletions(-) create mode 100644 packages/stack/__tests__/test-kit-families.test.ts create mode 100644 packages/test-kit/package.json create mode 100644 packages/test-kit/src/adapter.ts create mode 100644 packages/test-kit/src/catalog.ts create mode 100644 packages/test-kit/src/families.ts create mode 100644 packages/test-kit/src/index.ts create mode 100644 packages/test-kit/src/ops.ts create mode 100644 packages/test-kit/src/oracle.ts create mode 100644 packages/test-kit/src/rows.ts create mode 100644 packages/test-kit/tsconfig.json create mode 100644 vitest.shared.ts diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index cb7864e92..b1ec881d3 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -72,6 +72,13 @@ jobs: - name: Type tests (stack) run: pnpm --filter @cipherstash/stack run test:types + # The v3 domain catalog lives in the test kit, and its + # `satisfies Record` is what forces a new SDK + # domain to be covered. That check only fires under `tsc`, so without this + # step a domain added to stack would slip through untested. + - name: Type tests (test-kit — enforces v3 domain coverage) + run: pnpm --filter @cipherstash/test-kit run test:types + - name: Lint — no hardcoded package-manager runners run: pnpm run lint:runners diff --git a/packages/stack/__tests__/test-kit-families.test.ts b/packages/stack/__tests__/test-kit-families.test.ts new file mode 100644 index 000000000..6a9d951c1 --- /dev/null +++ b/packages/stack/__tests__/test-kit-families.test.ts @@ -0,0 +1,92 @@ +import { + deferredForFamily, + domainsForFamily, + eqlTypeSlug, + FAMILY_NAMES, + isCovered, + typedEntries, + V3_MATRIX, +} from '@cipherstash/test-kit' +import { describe, expect, it } from 'vitest' + +/** + * The integration suites are split one file per plaintext family, and each file + * asks `domainsForFamily(name)` which domains it owns. Nothing in that lookup + * forces the families to COVER the catalog: add a domain whose slug matches no + * family prefix (`interval_ord`, say) and it simply belongs to no file — the + * suites stay green while the domain goes untested. + * + * The catalog's `satisfies Record<…>` cannot catch that; it only forces a ROW to + * exist. These tests close the gap by asserting the families partition the + * catalog: every domain lands in exactly one family, and covered + deferred + * accounts for all of them. + */ +describe('test-kit families partition the v3 catalog', () => { + const bare = (eqlType: string) => eqlTypeSlug(eqlType).replace(/^eql_v3_/, '') + const allBare = typedEntries(V3_MATRIX).map(([eqlType]) => bare(eqlType)) + + it('assigns every covered domain to exactly one family', () => { + const owners = new Map() + for (const family of FAMILY_NAMES) { + for (const domain of domainsForFamily(family)) { + owners.set(domain.bare, [...(owners.get(domain.bare) ?? []), family]) + } + } + + const multiplyOwned = [...owners].filter(([, fams]) => fams.length > 1) + expect(multiplyOwned).toEqual([]) + + const coveredBare = typedEntries(V3_MATRIX) + .filter(([, spec]) => isCovered(spec)) + .map(([eqlType]) => bare(eqlType)) + .sort() + + expect([...owners.keys()].sort()).toEqual(coveredBare) + }) + + it('accounts for every catalog domain as covered or deferred', () => { + const covered = FAMILY_NAMES.flatMap((f) => + domainsForFamily(f).map((d) => d.bare), + ) + const deferred = FAMILY_NAMES.flatMap((f) => + deferredForFamily(f).map((d) => d.bare), + ) + + expect([...covered, ...deferred].sort()).toEqual([...allBare].sort()) + }) + + it('defers exactly the block-ORE domains, each with a reason', () => { + const deferred = FAMILY_NAMES.flatMap((f) => deferredForFamily(f)) + + expect(deferred.map((d) => d.bare).sort()).toEqual([ + 'bigint_ord_ore', + 'date_ord_ore', + 'double_ord_ore', + 'integer_ord_ore', + 'numeric_ord_ore', + 'real_ord_ore', + 'smallint_ord_ore', + 'text_ord_ore', + 'timestamp_ord_ore', + ]) + for (const { reason } of deferred) expect(reason).toMatch(/superuser-only/) + }) + + it('never hands a family a domain from a neighbouring prefix', () => { + // `date` must not claim `timestamp_ord`, and `integer` must not claim + // `smallint`. The prefix match is anchored on `_` or end-of-string; a naive + // `startsWith` would fail this. + expect(domainsForFamily('date').map((d) => d.bare)).toEqual([ + 'date', + 'date_eq', + 'date_ord', + ]) + expect(domainsForFamily('text').map((d) => d.bare)).toEqual([ + 'text', + 'text_eq', + 'text_match', + 'text_ord', + 'text_search', + ]) + }) +}) diff --git a/packages/stack/__tests__/v3-matrix/catalog.ts b/packages/stack/__tests__/v3-matrix/catalog.ts index 896375595..6077d63cc 100644 --- a/packages/stack/__tests__/v3-matrix/catalog.ts +++ b/packages/stack/__tests__/v3-matrix/catalog.ts @@ -1,303 +1,11 @@ /** - * Type-driven v3 test matrix — single source of truth. + * The v3 domain catalog moved to `@cipherstash/test-kit` so the integration + * suites — which after the adapter split live in other packages — can import + * the same single source of truth. A package must not reach into another + * package's `__tests__` tree, and duplicating the catalog would let the two + * copies disagree about which domains exist. * - * The TypeScript analog of the Rust `eql_v3` `scalar_matrix!` harness - * (`encrypt-query-language/tests/sqlx`): one declarative catalog drives both a - * runtime `it.each` matrix (`matrix.test.ts`) and type-level assertions - * (`matrix.test-d.ts`), instead of hand-rolling per-domain test bodies. - * - * COVERAGE IS MANDATORY. The catalog is `satisfies Record`, and `EqlV3TypeName` is derived from the real column union - * (`AnyEncryptedV3Column`). Add a domain to the SDK and this file fails to - * compile until it has a row — the compile-time analog of, and stronger than, - * the Rust `test:matrix:inventory` cross-check (it names each missing domain). - * - * Every field here is consumed by a test: `builder`/`ColumnClass` by the - * instanceof check, `castAs` + `indexes` by the `build()` `toStrictEqual`, and - * `capabilities` by `getQueryCapabilities()`/`isQueryable()`. - */ - -import type { - AnyEncryptedV3Column, - EqlTypeForColumn, - QueryCapabilities, -} from '@/eql/v3' -import { - EncryptedBigintColumn, - EncryptedBigintEqColumn, - EncryptedBigintOrdColumn, - EncryptedBigintOrdOreColumn, - EncryptedBooleanColumn, - EncryptedDateColumn, - EncryptedDateEqColumn, - EncryptedDateOrdColumn, - EncryptedDateOrdOreColumn, - EncryptedDoubleColumn, - EncryptedDoubleEqColumn, - EncryptedDoubleOrdColumn, - EncryptedDoubleOrdOreColumn, - EncryptedIntegerColumn, - EncryptedIntegerEqColumn, - EncryptedIntegerOrdColumn, - EncryptedIntegerOrdOreColumn, - EncryptedNumericColumn, - EncryptedNumericEqColumn, - EncryptedNumericOrdColumn, - EncryptedNumericOrdOreColumn, - EncryptedRealColumn, - EncryptedRealEqColumn, - EncryptedRealOrdColumn, - EncryptedRealOrdOreColumn, - EncryptedSmallintColumn, - EncryptedSmallintEqColumn, - EncryptedSmallintOrdColumn, - EncryptedSmallintOrdOreColumn, - EncryptedTextColumn, - EncryptedTextEqColumn, - EncryptedTextMatchColumn, - EncryptedTextOrdColumn, - EncryptedTextOrdOreColumn, - EncryptedTextSearchColumn, - EncryptedTimestampColumn, - EncryptedTimestampEqColumn, - EncryptedTimestampOrdColumn, - EncryptedTimestampOrdOreColumn, - types, -} from '@/eql/v3' -import type { ColumnSchema } from '@/schema' - -/** - * The canonical union of every v3 domain name — derived STRAIGHT from the real - * column union (`AnyEncryptedV3Column`) via the exported `EqlTypeForColumn` - * helper, not hand-copied. This is the key set the `Record` below must cover. - */ -export type EqlV3TypeName = EqlTypeForColumn - -/** One row of the type-driven matrix: everything a test needs about a domain. */ -export type DomainSpec = Readonly<{ - /** Column builder under test. */ - builder: (name: string) => AnyEncryptedV3Column - /** Concrete class the builder must instantiate (`toBeInstanceOf`). */ - ColumnClass: new ( - ...args: never[] - ) => AnyEncryptedV3Column - /** Plaintext axis emitted by `build().cast_as`. */ - castAs: ColumnSchema['cast_as'] - /** Semantic capability flags (`getQueryCapabilities()`). */ - capabilities: QueryCapabilities - /** - * The full `build().indexes` output — stored as DATA per row (like the Rust - * harness) rather than derived from `capabilities`, because `text_search` - * overrides `build()` to emit `unique + ore + match` where the capability → - * index rule would omit `unique` for an order-capable column. - */ - indexes: ColumnSchema['indexes'] - /** - * Representative + edge plaintext values that MUST round-trip through live - * encrypt/decrypt (consumed by `matrix-live.test.ts`). Typed as the loose - * plaintext union rather than per-row: the precise `castAs → plaintext` axis - * is already proven at the type level in `matrix.test-d.ts` (`InferPlaintext`), - * and a per-row generic would break the single `satisfies Record<…>` that is - * this file's coverage mechanism. Numeric samples are split integer-vs- - * fractional: `build()` emits `cast_as:'number'` uniformly so the FFI can't - * tell `integer` from `double`, and a fractional value on an int-named domain is - * untested territory (it would truncate against a real narrow PG column). - */ - samples: ReadonlyArray - /** - * Values that MUST fail encryption. Number domains reject `NaN`/`±Infinity` - * and `bigint` domains reject values outside the signed 64-bit (`int8`) range - * — both via the same client-side guard (`assertValidNumericValue`). Domains - * whose plaintext type cannot express an invalid value omit this. - */ - errorSamples?: ReadonlyArray -}> - -/** - * `Object.entries` that preserves the literal key union instead of widening to - * `string` — so `eqlType` in the runtime matrix stays `EqlV3TypeName`. - */ -export function typedEntries( - obj: Record, -): Array<[K, V]> { - return Object.entries(obj) as Array<[K, V]> -} - -/** The schema the concrete EQL v3 domains live in — see `EqlV3TypeName`. */ -export const EQL_V3_DOMAIN_SCHEMA = 'public' - -/** - * A `public.eql_v3_integer_ord` domain reduced to a bare `integer_ord`, suitable as a - * column identifier in the test tables. Shared by every suite that builds a - * table from the matrix, so a future domain-schema move is a one-line change - * rather than five. + * This shim keeps the ~12 unit suites that import `../v3-matrix/catalog` + * unchanged. Import from `@cipherstash/test-kit` in anything new. */ -export function eqlTypeSlug(eqlType: EqlV3TypeName | string): string { - const prefix = `${EQL_V3_DOMAIN_SCHEMA}.` - return eqlType.startsWith(prefix) ? eqlType.slice(prefix.length) : eqlType -} - -// Capability shorthands (mirror the SDK's internal presets). -const STORAGE = { - equality: false, - orderAndRange: false, - freeTextSearch: false, -} as const -const EQ = { - equality: true, - orderAndRange: false, - freeTextSearch: false, -} as const -const ORD = { - equality: true, - orderAndRange: true, - freeTextSearch: false, -} as const -const MATCH_ONLY = { - equality: false, - orderAndRange: false, - freeTextSearch: true, -} as const -const FULL = { - equality: true, - orderAndRange: true, - freeTextSearch: true, -} as const - -// Index shorthands (mirror `build().indexes`). Type-annotated rather than -// `as const`: annotation contextually types the literals so enum fields like -// `kind: 'ngram'` stay checked against the schema while arrays remain MUTABLE -// — `ColumnSchema['indexes']` rejects the `readonly` arrays `as const` produces. -type Indexes = ColumnSchema['indexes'] -const NONE: Indexes = {} -const UNIQUE_IDX: Indexes = { unique: { token_filters: [] } } -// Ordering flavour is pinned by the domain name (eql-3.0.0): `_ord_ore` -// domains are block-ORE (`ore` index, `ob` term); plain `_ord` domains and -// `text_search` are CLLW-OPE (`ope` index, `op` term). -const ORE_IDX: Indexes = { ore: {} } -const OPE_IDX: Indexes = { ope: {} } -// Text order domains (`text_ord`, `text_ord_ore`) carry BOTH `hm` (unique) and -// their ordering term: their eql_v3 SQL domains require `hm` because text -// equality is HMAC-based, unlike numeric/date order domains which answer -// equality via the ordering term. -const TEXT_ORD_IDX: Indexes = { unique: { token_filters: [] }, ope: {} } -const TEXT_ORD_ORE_IDX: Indexes = { unique: { token_filters: [] }, ore: {} } -const MATCH_BLOCK: NonNullable['match'] = { - tokenizer: { kind: 'ngram', token_length: 3 }, - token_filters: [{ kind: 'downcase' }], - k: 6, - m: 2048, - include_original: true, -} -const MATCH_IDX: Indexes = { match: MATCH_BLOCK } -const TEXT_SEARCH_IDX: Indexes = { - unique: { token_filters: [] }, - ope: {}, - match: MATCH_BLOCK, -} - -// Sample plaintexts per plaintext axis, consumed by `matrix-live.test.ts`. -// Numeric sets are split by domain width: integers (incl. type bounds) for -// smallint/integer, fractionals for real/double/numeric. See `DomainSpec.samples`. -const SMALLINT_S = [0, -1, 32767, -32768] as const -const INTEGER_S = [0, -42, 2147483647, -2147483648] as const -// Full i64 bounds: i64::MAX, i64::MIN, zero, a negative, and a mid value beyond -// Number.MAX_SAFE_INTEGER to prove protect-ffi 0.28 round-trips a JS bigint -// losslessly. Values OUTSIDE this range are rejected client-side (BIGINT_ERR) -// before protect-ffi, mirroring how NUM_ERR guards the number domains. -const BIGINT_S = [ - 4611686018427387904n, - -42n, - 9223372036854775807n, - -9223372036854775808n, - 0n, -] as const -const REAL_S = [0, 77.5, -117.25, 0.5] as const -const DOUBLE_S = [0, -117.123456, 1e15, -1e15] as const -const NUMERIC_S = [0, 12345.678, -42, -0.5] as const -const TEXT_S = ['', 'ada@example.com', 'Ada Lovelace'] as const -// Text domains with an ordering term require non-empty positive samples to -// satisfy the real Postgres domain checks. Empty-string rejection is covered in -// matrix-live-pg. -const TEXT_ORE_S = [ - 'ada@example.com', - 'grace@example.com', - 'zora@example.org', -] as const -const BOOLEAN_S = [true, false] as const -const DATE_S = [ - new Date('2026-07-01T00:00:00.000Z'), - new Date('1970-01-01T00:00:00.000Z'), -] as const -// Timestamp domains (`cast_as: 'timestamp'`) preserve the full instant, unlike -// `date` which truncates to midnight. These samples deliberately carry a -// NON-ZERO time-of-day so the live round-trip actually detects a regression -// that truncates to the day boundary — reusing the midnight-only `DATE_S` here -// would let such a regression pass silently (see matrix.test.ts). -const TS_S = [ - new Date('2026-07-01T12:34:56.000Z'), - new Date('1970-01-01T23:59:59.000Z'), -] as const -// Every number domain rejects these via the global encrypt guard. -const NUM_ERR = [ - Number.NaN, - Number.POSITIVE_INFINITY, - Number.NEGATIVE_INFINITY, -] as const -// Every bigint domain rejects values outside the signed 64-bit (`int8`) range -// via the same guard — the bigint analog of NUM_ERR's NaN/±Infinity: i64::MAX+1 -// and i64::MIN-1. -const BIGINT_ERR = [9223372036854775808n, -9223372036854775809n] as const - -// biome-ignore format: one row per domain reads as a table; keep it dense. -export const V3_MATRIX = { - // integer - 'public.eql_v3_integer': { builder: types.Integer, ColumnClass: EncryptedIntegerColumn, castAs: 'number', capabilities: STORAGE, indexes: NONE, samples: INTEGER_S, errorSamples: NUM_ERR }, - 'public.eql_v3_integer_eq': { builder: types.IntegerEq, ColumnClass: EncryptedIntegerEqColumn, castAs: 'number', capabilities: EQ, indexes: UNIQUE_IDX, samples: INTEGER_S, errorSamples: NUM_ERR }, - 'public.eql_v3_integer_ord_ore': { builder: types.IntegerOrdOre, ColumnClass: EncryptedIntegerOrdOreColumn, castAs: 'number', capabilities: ORD, indexes: ORE_IDX, samples: INTEGER_S, errorSamples: NUM_ERR }, - 'public.eql_v3_integer_ord': { builder: types.IntegerOrd, ColumnClass: EncryptedIntegerOrdColumn, castAs: 'number', capabilities: ORD, indexes: OPE_IDX, samples: INTEGER_S, errorSamples: NUM_ERR }, - // smallint - 'public.eql_v3_smallint': { builder: types.Smallint, ColumnClass: EncryptedSmallintColumn, castAs: 'number', capabilities: STORAGE, indexes: NONE, samples: SMALLINT_S, errorSamples: NUM_ERR }, - 'public.eql_v3_smallint_eq': { builder: types.SmallintEq, ColumnClass: EncryptedSmallintEqColumn, castAs: 'number', capabilities: EQ, indexes: UNIQUE_IDX, samples: SMALLINT_S, errorSamples: NUM_ERR }, - 'public.eql_v3_smallint_ord_ore': { builder: types.SmallintOrdOre, ColumnClass: EncryptedSmallintOrdOreColumn, castAs: 'number', capabilities: ORD, indexes: ORE_IDX, samples: SMALLINT_S, errorSamples: NUM_ERR }, - 'public.eql_v3_smallint_ord': { builder: types.SmallintOrd, ColumnClass: EncryptedSmallintOrdColumn, castAs: 'number', capabilities: ORD, indexes: OPE_IDX, samples: SMALLINT_S, errorSamples: NUM_ERR }, - // bigint (int8) — native JS bigint round-trip on protect-ffi 0.28; ungated - 'public.eql_v3_bigint': { builder: types.Bigint, ColumnClass: EncryptedBigintColumn, castAs: 'bigint', capabilities: STORAGE, indexes: NONE, samples: BIGINT_S, errorSamples: BIGINT_ERR }, - 'public.eql_v3_bigint_eq': { builder: types.BigintEq, ColumnClass: EncryptedBigintEqColumn, castAs: 'bigint', capabilities: EQ, indexes: UNIQUE_IDX, samples: BIGINT_S, errorSamples: BIGINT_ERR }, - 'public.eql_v3_bigint_ord_ore': { builder: types.BigintOrdOre, ColumnClass: EncryptedBigintOrdOreColumn, castAs: 'bigint', capabilities: ORD, indexes: ORE_IDX, samples: BIGINT_S, errorSamples: BIGINT_ERR }, - 'public.eql_v3_bigint_ord': { builder: types.BigintOrd, ColumnClass: EncryptedBigintOrdColumn, castAs: 'bigint', capabilities: ORD, indexes: OPE_IDX, samples: BIGINT_S, errorSamples: BIGINT_ERR }, - // date - 'public.eql_v3_date': { builder: types.Date, ColumnClass: EncryptedDateColumn, castAs: 'date', capabilities: STORAGE, indexes: NONE, samples: DATE_S }, - 'public.eql_v3_date_eq': { builder: types.DateEq, ColumnClass: EncryptedDateEqColumn, castAs: 'date', capabilities: EQ, indexes: UNIQUE_IDX, samples: DATE_S }, - 'public.eql_v3_date_ord_ore': { builder: types.DateOrdOre, ColumnClass: EncryptedDateOrdOreColumn, castAs: 'date', capabilities: ORD, indexes: ORE_IDX, samples: DATE_S }, - 'public.eql_v3_date_ord': { builder: types.DateOrd, ColumnClass: EncryptedDateOrdColumn, castAs: 'date', capabilities: ORD, indexes: OPE_IDX, samples: DATE_S }, - // timestamp - 'public.eql_v3_timestamp': { builder: types.Timestamp, ColumnClass: EncryptedTimestampColumn, castAs: 'timestamp', capabilities: STORAGE, indexes: NONE, samples: TS_S }, - 'public.eql_v3_timestamp_eq': { builder: types.TimestampEq, ColumnClass: EncryptedTimestampEqColumn, castAs: 'timestamp', capabilities: EQ, indexes: UNIQUE_IDX, samples: TS_S }, - 'public.eql_v3_timestamp_ord_ore': { builder: types.TimestampOrdOre, ColumnClass: EncryptedTimestampOrdOreColumn, castAs: 'timestamp', capabilities: ORD, indexes: ORE_IDX, samples: TS_S }, - 'public.eql_v3_timestamp_ord': { builder: types.TimestampOrd, ColumnClass: EncryptedTimestampOrdColumn, castAs: 'timestamp', capabilities: ORD, indexes: OPE_IDX, samples: TS_S }, - // numeric - 'public.eql_v3_numeric': { builder: types.Numeric, ColumnClass: EncryptedNumericColumn, castAs: 'number', capabilities: STORAGE, indexes: NONE, samples: NUMERIC_S, errorSamples: NUM_ERR }, - 'public.eql_v3_numeric_eq': { builder: types.NumericEq, ColumnClass: EncryptedNumericEqColumn, castAs: 'number', capabilities: EQ, indexes: UNIQUE_IDX, samples: NUMERIC_S, errorSamples: NUM_ERR }, - 'public.eql_v3_numeric_ord_ore': { builder: types.NumericOrdOre, ColumnClass: EncryptedNumericOrdOreColumn, castAs: 'number', capabilities: ORD, indexes: ORE_IDX, samples: NUMERIC_S, errorSamples: NUM_ERR }, - 'public.eql_v3_numeric_ord': { builder: types.NumericOrd, ColumnClass: EncryptedNumericOrdColumn, castAs: 'number', capabilities: ORD, indexes: OPE_IDX, samples: NUMERIC_S, errorSamples: NUM_ERR }, - // text - 'public.eql_v3_text': { builder: types.Text, ColumnClass: EncryptedTextColumn, castAs: 'string', capabilities: STORAGE, indexes: NONE, samples: TEXT_S }, - 'public.eql_v3_text_eq': { builder: types.TextEq, ColumnClass: EncryptedTextEqColumn, castAs: 'string', capabilities: EQ, indexes: UNIQUE_IDX, samples: TEXT_S }, - 'public.eql_v3_text_match': { builder: types.TextMatch, ColumnClass: EncryptedTextMatchColumn, castAs: 'string', capabilities: MATCH_ONLY, indexes: MATCH_IDX, samples: TEXT_S }, - 'public.eql_v3_text_ord_ore': { builder: types.TextOrdOre, ColumnClass: EncryptedTextOrdOreColumn, castAs: 'string', capabilities: ORD, indexes: TEXT_ORD_ORE_IDX, samples: TEXT_ORE_S }, - 'public.eql_v3_text_ord': { builder: types.TextOrd, ColumnClass: EncryptedTextOrdColumn, castAs: 'string', capabilities: ORD, indexes: TEXT_ORD_IDX, samples: TEXT_ORE_S }, - 'public.eql_v3_text_search': { builder: types.TextSearch, ColumnClass: EncryptedTextSearchColumn, castAs: 'string', capabilities: FULL, indexes: TEXT_SEARCH_IDX, samples: TEXT_ORE_S }, - // boolean - 'public.eql_v3_boolean': { builder: types.Boolean, ColumnClass: EncryptedBooleanColumn, castAs: 'boolean', capabilities: STORAGE, indexes: NONE, samples: BOOLEAN_S }, - // real - 'public.eql_v3_real': { builder: types.Real, ColumnClass: EncryptedRealColumn, castAs: 'number', capabilities: STORAGE, indexes: NONE, samples: REAL_S, errorSamples: NUM_ERR }, - 'public.eql_v3_real_eq': { builder: types.RealEq, ColumnClass: EncryptedRealEqColumn, castAs: 'number', capabilities: EQ, indexes: UNIQUE_IDX, samples: REAL_S, errorSamples: NUM_ERR }, - 'public.eql_v3_real_ord_ore': { builder: types.RealOrdOre, ColumnClass: EncryptedRealOrdOreColumn, castAs: 'number', capabilities: ORD, indexes: ORE_IDX, samples: REAL_S, errorSamples: NUM_ERR }, - 'public.eql_v3_real_ord': { builder: types.RealOrd, ColumnClass: EncryptedRealOrdColumn, castAs: 'number', capabilities: ORD, indexes: OPE_IDX, samples: REAL_S, errorSamples: NUM_ERR }, - // double - 'public.eql_v3_double': { builder: types.Double, ColumnClass: EncryptedDoubleColumn, castAs: 'number', capabilities: STORAGE, indexes: NONE, samples: DOUBLE_S, errorSamples: NUM_ERR }, - 'public.eql_v3_double_eq': { builder: types.DoubleEq, ColumnClass: EncryptedDoubleEqColumn, castAs: 'number', capabilities: EQ, indexes: UNIQUE_IDX, samples: DOUBLE_S, errorSamples: NUM_ERR }, - 'public.eql_v3_double_ord_ore': { builder: types.DoubleOrdOre, ColumnClass: EncryptedDoubleOrdOreColumn, castAs: 'number', capabilities: ORD, indexes: ORE_IDX, samples: DOUBLE_S, errorSamples: NUM_ERR }, - 'public.eql_v3_double_ord': { builder: types.DoubleOrd, ColumnClass: EncryptedDoubleOrdColumn, castAs: 'number', capabilities: ORD, indexes: OPE_IDX, samples: DOUBLE_S, errorSamples: NUM_ERR }, -} as const satisfies Record +export * from '@cipherstash/test-kit/catalog' diff --git a/packages/stack/tsconfig.json b/packages/stack/tsconfig.json index 515c043f4..b2627eb5a 100644 --- a/packages/stack/tsconfig.json +++ b/packages/stack/tsconfig.json @@ -35,7 +35,21 @@ // Paths "paths": { - "@/*": ["./src/*"] + "@/*": ["./src/*"], + + // The v3 domain catalog lives in `@cipherstash/test-kit`, which imports + // stack through its PUBLIC subpaths (packages outside stack cannot use + // `@/`). `tsc` does not read Vitest's `resolve.alias`, so without these + // the catalog's builders lose their precise return types and the + // `.test-d.ts` matrix collapses to `never`. Mirrors `vitest.shared.ts` + // and `packages/test-kit/tsconfig.json`; keep the three in step. + "@cipherstash/test-kit": ["../test-kit/src/index.ts"], + "@cipherstash/test-kit/catalog": ["../test-kit/src/catalog.ts"], + "@cipherstash/stack": ["./src/index.ts"], + "@cipherstash/stack/eql/v3": ["./src/eql/v3/index.ts"], + "@cipherstash/stack/schema": ["./src/schema/index.ts"], + "@cipherstash/stack/v3": ["./src/encryption/v3.ts"], + "@cipherstash/stack/supabase": ["./src/supabase/index.ts"] } } } diff --git a/packages/stack/vitest.config.ts b/packages/stack/vitest.config.ts index 2f8acabec..257d2f7d2 100644 --- a/packages/stack/vitest.config.ts +++ b/packages/stack/vitest.config.ts @@ -1,9 +1,14 @@ import { resolve } from 'node:path' import { defineConfig } from 'vitest/config' +import { sharedAlias } from '../../vitest.shared' export default defineConfig({ resolve: { alias: { + // `@cipherstash/test-kit` + the stack subpaths its catalog imports, all + // resolved to source. Must precede `@/` only in spirit — the keys do not + // overlap — but keep it first so the shared block is obvious. + ...sharedAlias, '@/': resolve(__dirname, './src') + '/', // The installed `@cipherstash/{protect-ffi,auth}` only export `.`; their // `/wasm-inline` subpaths (imported by `src/wasm-inline.ts`) are not diff --git a/packages/test-kit/package.json b/packages/test-kit/package.json new file mode 100644 index 000000000..e8e465feb --- /dev/null +++ b/packages/test-kit/package.json @@ -0,0 +1,21 @@ +{ + "name": "@cipherstash/test-kit", + "version": "0.0.0", + "private": true, + "description": "Shared EQL v3 test harness: the domain catalog, the plaintext oracle, and the integration-suite driver. Consumed as TypeScript source — no build step.", + "type": "module", + "exports": { + ".": "./src/index.ts", + "./catalog": "./src/catalog.ts" + }, + "scripts": { + "test:types": "tsc --noEmit -p tsconfig.json" + }, + "dependencies": { + "@cipherstash/stack": "workspace:*" + }, + "devDependencies": { + "typescript": "catalog:repo", + "vitest": "catalog:repo" + } +} diff --git a/packages/test-kit/src/adapter.ts b/packages/test-kit/src/adapter.ts new file mode 100644 index 000000000..aabc574d8 --- /dev/null +++ b/packages/test-kit/src/adapter.ts @@ -0,0 +1,60 @@ +import type { QueryOp, QueryOpKind } from './ops.ts' +import type { PlainRow, TableSpec } from './rows.ts' + +/** + * The seam between the shared driver and one adapter under test. + * + * The driver decides WHAT to assert — derived from the domain's capabilities — + * and the adapter decides HOW to express it. Keeping the adapters behind one + * interface is what stops Drizzle and Supabase from quietly covering different + * operations while both suites read as "comprehensive". + */ +export interface IntegrationAdapter { + readonly name: 'drizzle' | 'supabase' + + /** + * Operations this adapter can express at all, independent of any domain. + * Supabase omits `order`: PostgREST cannot emit `ORDER BY eql_v3.ord_term(col)`, + * and a bare `ORDER BY` would sort the raw ciphertext envelope. + */ + readonly supportedOps: ReadonlySet + + /** + * Operations this adapter refuses on ANY encrypted column, whatever its + * capabilities. Supabase: `{'order'}`. This is the half of the contract the + * capability matrix cannot derive — an ORE-capable column still cannot be + * ordered through PostgREST — so it is stated once, here, and pinned by a test. + */ + readonly alwaysRejectedOps: ReadonlySet + + setup(): Promise + teardown(): Promise + + /** Create the table: one column per family domain, every encrypted column NULLable. */ + createTable(table: TableSpec): Promise + + /** Insert one row through the adapter's single-encrypt path (`encryptModel`). */ + insertSingle(table: TableSpec, row: PlainRow): Promise + + /** Insert many rows through the adapter's bulk path (`bulkEncryptModels`). */ + insertBulk(table: TableSpec, rows: readonly PlainRow[]): Promise + + /** + * Run `op` and return the matching row keys. + * + * For `order`, return them in the order the database produced, and exclude + * NULLs — the seed set contains one all-NULL row, and NULL ordering is a + * property of the SQL dialect, not of the encrypted ordering term. Every + * other kind returns keys sorted ascending, so the driver can compare sets + * without caring about row order. + */ + run(table: TableSpec, op: QueryOp): Promise + + /** + * Assert `op` is rejected. Each adapter owns its error shape — + * `EncryptionOperatorError` for Drizzle, a `[supabase v3]` Error for + * Supabase — so the driver stays adapter-agnostic. Always async: wrap a + * synchronous throw. + */ + expectRejected(table: TableSpec, op: QueryOp): Promise +} diff --git a/packages/test-kit/src/catalog.ts b/packages/test-kit/src/catalog.ts new file mode 100644 index 000000000..76b3dd526 --- /dev/null +++ b/packages/test-kit/src/catalog.ts @@ -0,0 +1,651 @@ +/** + * Type-driven v3 test matrix — single source of truth. + * + * The TypeScript analog of the Rust `eql_v3` `scalar_matrix!` harness + * (`encrypt-query-language/tests/sqlx`): one declarative catalog drives both a + * runtime `it.each` matrix (`matrix.test.ts`) and type-level assertions + * (`matrix.test-d.ts`), instead of hand-rolling per-domain test bodies. + * + * COVERAGE IS MANDATORY. The catalog is `satisfies Record`, and `EqlV3TypeName` is derived from the real column union + * (`AnyEncryptedV3Column`). Add a domain to the SDK and this file fails to + * compile until it has a row — the compile-time analog of, and stronger than, + * the Rust `test:matrix:inventory` cross-check (it names each missing domain). + * + * Every field here is consumed by a test: `builder`/`ColumnClass` by the + * instanceof check, `castAs` + `indexes` by the `build()` `toStrictEqual`, and + * `capabilities` by `getQueryCapabilities()`/`isQueryable()`. + */ + +import type { + AnyEncryptedV3Column, + EqlTypeForColumn, + QueryCapabilities, +} from '@cipherstash/stack/eql/v3' +import { + EncryptedBigintColumn, + EncryptedBigintEqColumn, + EncryptedBigintOrdColumn, + EncryptedBigintOrdOreColumn, + EncryptedBooleanColumn, + EncryptedDateColumn, + EncryptedDateEqColumn, + EncryptedDateOrdColumn, + EncryptedDateOrdOreColumn, + EncryptedDoubleColumn, + EncryptedDoubleEqColumn, + EncryptedDoubleOrdColumn, + EncryptedDoubleOrdOreColumn, + EncryptedIntegerColumn, + EncryptedIntegerEqColumn, + EncryptedIntegerOrdColumn, + EncryptedIntegerOrdOreColumn, + EncryptedNumericColumn, + EncryptedNumericEqColumn, + EncryptedNumericOrdColumn, + EncryptedNumericOrdOreColumn, + EncryptedRealColumn, + EncryptedRealEqColumn, + EncryptedRealOrdColumn, + EncryptedRealOrdOreColumn, + EncryptedSmallintColumn, + EncryptedSmallintEqColumn, + EncryptedSmallintOrdColumn, + EncryptedSmallintOrdOreColumn, + EncryptedTextColumn, + EncryptedTextEqColumn, + EncryptedTextMatchColumn, + EncryptedTextOrdColumn, + EncryptedTextOrdOreColumn, + EncryptedTextSearchColumn, + EncryptedTimestampColumn, + EncryptedTimestampEqColumn, + EncryptedTimestampOrdColumn, + EncryptedTimestampOrdOreColumn, + types, +} from '@cipherstash/stack/eql/v3' +import type { ColumnSchema } from '@cipherstash/stack/schema' + +/** + * The canonical union of every v3 domain name — derived STRAIGHT from the real + * column union (`AnyEncryptedV3Column`) via the exported `EqlTypeForColumn` + * helper, not hand-copied. This is the key set the `Record` below must cover. + */ +export type EqlV3TypeName = EqlTypeForColumn + +/** One row of the type-driven matrix: everything a test needs about a domain. */ +export type DomainSpec = Readonly<{ + /** Column builder under test. */ + builder: (name: string) => AnyEncryptedV3Column + /** Concrete class the builder must instantiate (`toBeInstanceOf`). */ + ColumnClass: new ( + ...args: never[] + ) => AnyEncryptedV3Column + /** Plaintext axis emitted by `build().cast_as`. */ + castAs: ColumnSchema['cast_as'] + /** Semantic capability flags (`getQueryCapabilities()`). */ + capabilities: QueryCapabilities + /** + * The full `build().indexes` output — stored as DATA per row (like the Rust + * harness) rather than derived from `capabilities`, because `text_search` + * overrides `build()` to emit `unique + ore + match` where the capability → + * index rule would omit `unique` for an order-capable column. + */ + indexes: ColumnSchema['indexes'] + /** + * Representative + edge plaintext values that MUST round-trip through live + * encrypt/decrypt (consumed by `matrix-live.test.ts`). Typed as the loose + * plaintext union rather than per-row: the precise `castAs → plaintext` axis + * is already proven at the type level in `matrix.test-d.ts` (`InferPlaintext`), + * and a per-row generic would break the single `satisfies Record<…>` that is + * this file's coverage mechanism. Numeric samples are split integer-vs- + * fractional: `build()` emits `cast_as:'number'` uniformly so the FFI can't + * tell `integer` from `double`, and a fractional value on an int-named domain is + * untested territory (it would truncate against a real narrow PG column). + */ + samples: ReadonlyArray + /** + * Values that MUST fail encryption. Number domains reject `NaN`/`±Infinity` + * and `bigint` domains reject values outside the signed 64-bit (`int8`) range + * — both via the same client-side guard (`assertValidNumericValue`). Domains + * whose plaintext type cannot express an invalid value omit this. + */ + errorSamples?: ReadonlyArray + /** + * Why the integration matrix does NOT exercise this domain. Absent means it + * is covered — the common case, so the exclusion is what has to be spelled + * out, not the inclusion. + * + * A deferred domain still needs a row here: the `satisfies Record<…>` below + * is the coverage mechanism, and dropping the row would hide the domain + * rather than document why it is skipped. The integration driver reads this + * field, skips the domain, and prints the reason — so removing coverage is + * always a visible, reviewable diff, never a silent omission. The unit + * matrix (`matrix.test.ts`) ignores it and exercises every domain. + */ + deferred?: string +}> + +/** + * Whether the integration matrix exercises this domain. See {@link DomainSpec.deferred}. + * + * Read through these accessors, not by touching `.deferred`: `V3_MATRIX` is + * `as const`, so a row that omits the field has no such property on its literal + * type and a direct access does not compile. Widening to `DomainSpec` here makes + * the optional field readable from every row. + */ +export function isCovered(spec: DomainSpec): boolean { + return spec.deferred === undefined +} + +/** Why the integration matrix skips this domain, or `undefined` if it does not. */ +export function deferredReason(spec: DomainSpec): string | undefined { + return spec.deferred +} + +/** + * `Object.entries` that preserves the literal key union instead of widening to + * `string` — so `eqlType` in the runtime matrix stays `EqlV3TypeName`. + */ +export function typedEntries( + obj: Record, +): Array<[K, V]> { + return Object.entries(obj) as Array<[K, V]> +} + +/** The schema the concrete EQL v3 domains live in — see `EqlV3TypeName`. */ +export const EQL_V3_DOMAIN_SCHEMA = 'public' + +/** + * A `public.eql_v3_integer_ord` domain reduced to a bare `integer_ord`, suitable as a + * column identifier in the test tables. Shared by every suite that builds a + * table from the matrix, so a future domain-schema move is a one-line change + * rather than five. + */ +export function eqlTypeSlug(eqlType: EqlV3TypeName | string): string { + const prefix = `${EQL_V3_DOMAIN_SCHEMA}.` + return eqlType.startsWith(prefix) ? eqlType.slice(prefix.length) : eqlType +} + +// Capability shorthands (mirror the SDK's internal presets). +const STORAGE = { + equality: false, + orderAndRange: false, + freeTextSearch: false, +} as const +const EQ = { + equality: true, + orderAndRange: false, + freeTextSearch: false, +} as const +const ORD = { + equality: true, + orderAndRange: true, + freeTextSearch: false, +} as const +const MATCH_ONLY = { + equality: false, + orderAndRange: false, + freeTextSearch: true, +} as const +const FULL = { + equality: true, + orderAndRange: true, + freeTextSearch: true, +} as const + +// Index shorthands (mirror `build().indexes`). Type-annotated rather than +// `as const`: annotation contextually types the literals so enum fields like +// `kind: 'ngram'` stay checked against the schema while arrays remain MUTABLE +// — `ColumnSchema['indexes']` rejects the `readonly` arrays `as const` produces. +type Indexes = ColumnSchema['indexes'] +const NONE: Indexes = {} +const UNIQUE_IDX: Indexes = { unique: { token_filters: [] } } +// Ordering flavour is pinned by the domain name (eql-3.0.0): `_ord_ore` +// domains are block-ORE (`ore` index, `ob` term); plain `_ord` domains and +// `text_search` are CLLW-OPE (`ope` index, `op` term). +const ORE_IDX: Indexes = { ore: {} } +const OPE_IDX: Indexes = { ope: {} } +// Text order domains (`text_ord`, `text_ord_ore`) carry BOTH `hm` (unique) and +// their ordering term: their eql_v3 SQL domains require `hm` because text +// equality is HMAC-based, unlike numeric/date order domains which answer +// equality via the ordering term. +const TEXT_ORD_IDX: Indexes = { unique: { token_filters: [] }, ope: {} } +const TEXT_ORD_ORE_IDX: Indexes = { unique: { token_filters: [] }, ore: {} } +const MATCH_BLOCK: NonNullable['match'] = { + tokenizer: { kind: 'ngram', token_length: 3 }, + token_filters: [{ kind: 'downcase' }], + k: 6, + m: 2048, + include_original: true, +} +const MATCH_IDX: Indexes = { match: MATCH_BLOCK } +const TEXT_SEARCH_IDX: Indexes = { + unique: { token_filters: [] }, + ope: {}, + match: MATCH_BLOCK, +} + +// Sample plaintexts per plaintext axis, consumed by `matrix-live.test.ts`. +// Numeric sets are split by domain width: integers (incl. type bounds) for +// smallint/integer, fractionals for real/double/numeric. See `DomainSpec.samples`. +const SMALLINT_S = [0, -1, 32767, -32768] as const +const INTEGER_S = [0, -42, 2147483647, -2147483648] as const +// Full i64 bounds: i64::MAX, i64::MIN, zero, a negative, and a mid value beyond +// Number.MAX_SAFE_INTEGER to prove protect-ffi 0.28 round-trips a JS bigint +// losslessly. Values OUTSIDE this range are rejected client-side (BIGINT_ERR) +// before protect-ffi, mirroring how NUM_ERR guards the number domains. +const BIGINT_S = [ + 4611686018427387904n, + -42n, + 9223372036854775807n, + -9223372036854775808n, + 0n, +] as const +const REAL_S = [0, 77.5, -117.25, 0.5] as const +const DOUBLE_S = [0, -117.123456, 1e15, -1e15] as const +const NUMERIC_S = [0, 12345.678, -42, -0.5] as const +const TEXT_S = ['', 'ada@example.com', 'Ada Lovelace'] as const +// Text domains with an ordering term require non-empty positive samples to +// satisfy the real Postgres domain checks. Empty-string rejection is covered in +// matrix-live-pg. +const TEXT_ORE_S = [ + 'ada@example.com', + 'grace@example.com', + 'zora@example.org', +] as const +const BOOLEAN_S = [true, false] as const +const DATE_S = [ + new Date('2026-07-01T00:00:00.000Z'), + new Date('1970-01-01T00:00:00.000Z'), +] as const +// Timestamp domains (`cast_as: 'timestamp'`) preserve the full instant, unlike +// `date` which truncates to midnight. These samples deliberately carry a +// NON-ZERO time-of-day so the live round-trip actually detects a regression +// that truncates to the day boundary — reusing the midnight-only `DATE_S` here +// would let such a regression pass silently (see matrix.test.ts). +const TS_S = [ + new Date('2026-07-01T12:34:56.000Z'), + new Date('1970-01-01T23:59:59.000Z'), +] as const +// Every number domain rejects these via the global encrypt guard. +const NUM_ERR = [ + Number.NaN, + Number.POSITIVE_INFINITY, + Number.NEGATIVE_INFINITY, +] as const +// Every bigint domain rejects values outside the signed 64-bit (`int8`) range +// via the same guard — the bigint analog of NUM_ERR's NaN/±Infinity: i64::MAX+1 +// and i64::MIN-1. +const BIGINT_ERR = [9223372036854775808n, -9223372036854775809n] as const + +// biome-ignore format: one row per domain reads as a table; keep it dense. +/** + * The `_ord_ore` domains are block-ORE, whose operator class is superuser-only: + * the eql-3.0.0 bundle self-skips those statements on `insufficient_privilege`, + * so on managed Postgres (Supabase) the ORE domains are not installed at all. + * The integration matrix runs against both a plain and a managed Postgres, so it + * covers the OPE-backed `_ord` domains — which order via a native `bytea` btree + * and behave identically on every provider — and defers ORE to a suite that can + * assert its superuser-only install. See `docs/eql-v3-ord-term-ordering-defect.md`. + */ +const ORE_DEFERRED = + 'block-ORE opclass is superuser-only, so these domains are absent on managed Postgres; covered by a follow-up suite' + +export const V3_MATRIX = { + // integer + 'public.eql_v3_integer': { + builder: types.Integer, + ColumnClass: EncryptedIntegerColumn, + castAs: 'number', + capabilities: STORAGE, + indexes: NONE, + samples: INTEGER_S, + errorSamples: NUM_ERR, + }, + 'public.eql_v3_integer_eq': { + builder: types.IntegerEq, + ColumnClass: EncryptedIntegerEqColumn, + castAs: 'number', + capabilities: EQ, + indexes: UNIQUE_IDX, + samples: INTEGER_S, + errorSamples: NUM_ERR, + }, + 'public.eql_v3_integer_ord_ore': { + builder: types.IntegerOrdOre, + ColumnClass: EncryptedIntegerOrdOreColumn, + castAs: 'number', + capabilities: ORD, + indexes: ORE_IDX, + samples: INTEGER_S, + errorSamples: NUM_ERR, + deferred: ORE_DEFERRED, + }, + 'public.eql_v3_integer_ord': { + builder: types.IntegerOrd, + ColumnClass: EncryptedIntegerOrdColumn, + castAs: 'number', + capabilities: ORD, + indexes: OPE_IDX, + samples: INTEGER_S, + errorSamples: NUM_ERR, + }, + // smallint + 'public.eql_v3_smallint': { + builder: types.Smallint, + ColumnClass: EncryptedSmallintColumn, + castAs: 'number', + capabilities: STORAGE, + indexes: NONE, + samples: SMALLINT_S, + errorSamples: NUM_ERR, + }, + 'public.eql_v3_smallint_eq': { + builder: types.SmallintEq, + ColumnClass: EncryptedSmallintEqColumn, + castAs: 'number', + capabilities: EQ, + indexes: UNIQUE_IDX, + samples: SMALLINT_S, + errorSamples: NUM_ERR, + }, + 'public.eql_v3_smallint_ord_ore': { + builder: types.SmallintOrdOre, + ColumnClass: EncryptedSmallintOrdOreColumn, + castAs: 'number', + capabilities: ORD, + indexes: ORE_IDX, + samples: SMALLINT_S, + errorSamples: NUM_ERR, + deferred: ORE_DEFERRED, + }, + 'public.eql_v3_smallint_ord': { + builder: types.SmallintOrd, + ColumnClass: EncryptedSmallintOrdColumn, + castAs: 'number', + capabilities: ORD, + indexes: OPE_IDX, + samples: SMALLINT_S, + errorSamples: NUM_ERR, + }, + // bigint (int8) — native JS bigint round-trip on protect-ffi 0.28; ungated + 'public.eql_v3_bigint': { + builder: types.Bigint, + ColumnClass: EncryptedBigintColumn, + castAs: 'bigint', + capabilities: STORAGE, + indexes: NONE, + samples: BIGINT_S, + errorSamples: BIGINT_ERR, + }, + 'public.eql_v3_bigint_eq': { + builder: types.BigintEq, + ColumnClass: EncryptedBigintEqColumn, + castAs: 'bigint', + capabilities: EQ, + indexes: UNIQUE_IDX, + samples: BIGINT_S, + errorSamples: BIGINT_ERR, + }, + 'public.eql_v3_bigint_ord_ore': { + builder: types.BigintOrdOre, + ColumnClass: EncryptedBigintOrdOreColumn, + castAs: 'bigint', + capabilities: ORD, + indexes: ORE_IDX, + samples: BIGINT_S, + errorSamples: BIGINT_ERR, + deferred: ORE_DEFERRED, + }, + 'public.eql_v3_bigint_ord': { + builder: types.BigintOrd, + ColumnClass: EncryptedBigintOrdColumn, + castAs: 'bigint', + capabilities: ORD, + indexes: OPE_IDX, + samples: BIGINT_S, + errorSamples: BIGINT_ERR, + }, + // date + 'public.eql_v3_date': { + builder: types.Date, + ColumnClass: EncryptedDateColumn, + castAs: 'date', + capabilities: STORAGE, + indexes: NONE, + samples: DATE_S, + }, + 'public.eql_v3_date_eq': { + builder: types.DateEq, + ColumnClass: EncryptedDateEqColumn, + castAs: 'date', + capabilities: EQ, + indexes: UNIQUE_IDX, + samples: DATE_S, + }, + 'public.eql_v3_date_ord_ore': { + builder: types.DateOrdOre, + ColumnClass: EncryptedDateOrdOreColumn, + castAs: 'date', + capabilities: ORD, + indexes: ORE_IDX, + samples: DATE_S, + deferred: ORE_DEFERRED, + }, + 'public.eql_v3_date_ord': { + builder: types.DateOrd, + ColumnClass: EncryptedDateOrdColumn, + castAs: 'date', + capabilities: ORD, + indexes: OPE_IDX, + samples: DATE_S, + }, + // timestamp + 'public.eql_v3_timestamp': { + builder: types.Timestamp, + ColumnClass: EncryptedTimestampColumn, + castAs: 'timestamp', + capabilities: STORAGE, + indexes: NONE, + samples: TS_S, + }, + 'public.eql_v3_timestamp_eq': { + builder: types.TimestampEq, + ColumnClass: EncryptedTimestampEqColumn, + castAs: 'timestamp', + capabilities: EQ, + indexes: UNIQUE_IDX, + samples: TS_S, + }, + 'public.eql_v3_timestamp_ord_ore': { + builder: types.TimestampOrdOre, + ColumnClass: EncryptedTimestampOrdOreColumn, + castAs: 'timestamp', + capabilities: ORD, + indexes: ORE_IDX, + samples: TS_S, + deferred: ORE_DEFERRED, + }, + 'public.eql_v3_timestamp_ord': { + builder: types.TimestampOrd, + ColumnClass: EncryptedTimestampOrdColumn, + castAs: 'timestamp', + capabilities: ORD, + indexes: OPE_IDX, + samples: TS_S, + }, + // numeric + 'public.eql_v3_numeric': { + builder: types.Numeric, + ColumnClass: EncryptedNumericColumn, + castAs: 'number', + capabilities: STORAGE, + indexes: NONE, + samples: NUMERIC_S, + errorSamples: NUM_ERR, + }, + 'public.eql_v3_numeric_eq': { + builder: types.NumericEq, + ColumnClass: EncryptedNumericEqColumn, + castAs: 'number', + capabilities: EQ, + indexes: UNIQUE_IDX, + samples: NUMERIC_S, + errorSamples: NUM_ERR, + }, + 'public.eql_v3_numeric_ord_ore': { + builder: types.NumericOrdOre, + ColumnClass: EncryptedNumericOrdOreColumn, + castAs: 'number', + capabilities: ORD, + indexes: ORE_IDX, + samples: NUMERIC_S, + errorSamples: NUM_ERR, + deferred: ORE_DEFERRED, + }, + 'public.eql_v3_numeric_ord': { + builder: types.NumericOrd, + ColumnClass: EncryptedNumericOrdColumn, + castAs: 'number', + capabilities: ORD, + indexes: OPE_IDX, + samples: NUMERIC_S, + errorSamples: NUM_ERR, + }, + // text + 'public.eql_v3_text': { + builder: types.Text, + ColumnClass: EncryptedTextColumn, + castAs: 'string', + capabilities: STORAGE, + indexes: NONE, + samples: TEXT_S, + }, + 'public.eql_v3_text_eq': { + builder: types.TextEq, + ColumnClass: EncryptedTextEqColumn, + castAs: 'string', + capabilities: EQ, + indexes: UNIQUE_IDX, + samples: TEXT_S, + }, + 'public.eql_v3_text_match': { + builder: types.TextMatch, + ColumnClass: EncryptedTextMatchColumn, + castAs: 'string', + capabilities: MATCH_ONLY, + indexes: MATCH_IDX, + samples: TEXT_S, + }, + 'public.eql_v3_text_ord_ore': { + builder: types.TextOrdOre, + ColumnClass: EncryptedTextOrdOreColumn, + castAs: 'string', + capabilities: ORD, + indexes: TEXT_ORD_ORE_IDX, + samples: TEXT_ORE_S, + deferred: ORE_DEFERRED, + }, + 'public.eql_v3_text_ord': { + builder: types.TextOrd, + ColumnClass: EncryptedTextOrdColumn, + castAs: 'string', + capabilities: ORD, + indexes: TEXT_ORD_IDX, + samples: TEXT_ORE_S, + }, + 'public.eql_v3_text_search': { + builder: types.TextSearch, + ColumnClass: EncryptedTextSearchColumn, + castAs: 'string', + capabilities: FULL, + indexes: TEXT_SEARCH_IDX, + samples: TEXT_ORE_S, + }, + // boolean + 'public.eql_v3_boolean': { + builder: types.Boolean, + ColumnClass: EncryptedBooleanColumn, + castAs: 'boolean', + capabilities: STORAGE, + indexes: NONE, + samples: BOOLEAN_S, + }, + // real + 'public.eql_v3_real': { + builder: types.Real, + ColumnClass: EncryptedRealColumn, + castAs: 'number', + capabilities: STORAGE, + indexes: NONE, + samples: REAL_S, + errorSamples: NUM_ERR, + }, + 'public.eql_v3_real_eq': { + builder: types.RealEq, + ColumnClass: EncryptedRealEqColumn, + castAs: 'number', + capabilities: EQ, + indexes: UNIQUE_IDX, + samples: REAL_S, + errorSamples: NUM_ERR, + }, + 'public.eql_v3_real_ord_ore': { + builder: types.RealOrdOre, + ColumnClass: EncryptedRealOrdOreColumn, + castAs: 'number', + capabilities: ORD, + indexes: ORE_IDX, + samples: REAL_S, + errorSamples: NUM_ERR, + deferred: ORE_DEFERRED, + }, + 'public.eql_v3_real_ord': { + builder: types.RealOrd, + ColumnClass: EncryptedRealOrdColumn, + castAs: 'number', + capabilities: ORD, + indexes: OPE_IDX, + samples: REAL_S, + errorSamples: NUM_ERR, + }, + // double + 'public.eql_v3_double': { + builder: types.Double, + ColumnClass: EncryptedDoubleColumn, + castAs: 'number', + capabilities: STORAGE, + indexes: NONE, + samples: DOUBLE_S, + errorSamples: NUM_ERR, + }, + 'public.eql_v3_double_eq': { + builder: types.DoubleEq, + ColumnClass: EncryptedDoubleEqColumn, + castAs: 'number', + capabilities: EQ, + indexes: UNIQUE_IDX, + samples: DOUBLE_S, + errorSamples: NUM_ERR, + }, + 'public.eql_v3_double_ord_ore': { + builder: types.DoubleOrdOre, + ColumnClass: EncryptedDoubleOrdOreColumn, + castAs: 'number', + capabilities: ORD, + indexes: ORE_IDX, + samples: DOUBLE_S, + errorSamples: NUM_ERR, + deferred: ORE_DEFERRED, + }, + 'public.eql_v3_double_ord': { + builder: types.DoubleOrd, + ColumnClass: EncryptedDoubleOrdColumn, + castAs: 'number', + capabilities: ORD, + indexes: OPE_IDX, + samples: DOUBLE_S, + errorSamples: NUM_ERR, + }, +} as const satisfies Record diff --git a/packages/test-kit/src/families.ts b/packages/test-kit/src/families.ts new file mode 100644 index 000000000..e895c9521 --- /dev/null +++ b/packages/test-kit/src/families.ts @@ -0,0 +1,114 @@ +import { + type DomainSpec, + deferredReason, + type EqlV3TypeName, + eqlTypeSlug, + isCovered, + typedEntries, + V3_MATRIX, +} from './catalog.ts' + +/** + * Integration suites are split one file per plaintext family, not one per + * domain (39 files per adapter) nor one per capability tier (which buries the + * plaintext axis that actually drives the samples and the oracle). A family is + * the set of domains sharing a plaintext type and sample set — `integer`, + * `integer_eq`, `integer_ord`, `integer_ord_ore` — so one file seeds one table + * and exercises every capability tier that plaintext supports. + */ +export type FamilyName = + | 'integer' + | 'smallint' + | 'bigint' + | 'real-double' + | 'numeric' + | 'date' + | 'timestamp' + | 'text' + | 'boolean' + +/** + * The bare-domain prefixes each family owns. `real-double` pairs two prefixes + * because both are IEEE floats over the same sample set; every other family is + * one prefix. + * + * Matched against the BARE domain (`integer_ord`), not the slug: `eqlTypeSlug` + * strips only the `public.` schema, so a slug is `eql_v3_integer_ord`. The slug + * stays the physical column name, matching the existing matrix suites. + */ +const FAMILY_PREFIXES: Readonly> = { + integer: ['integer'], + smallint: ['smallint'], + bigint: ['bigint'], + 'real-double': ['real', 'double'], + numeric: ['numeric'], + date: ['date'], + timestamp: ['timestamp'], + text: ['text'], + boolean: ['boolean'], +} + +export const FAMILY_NAMES = Object.keys(FAMILY_PREFIXES) as FamilyName[] + +/** One family column: its physical column name, its domain, and its catalog row. */ +export type FamilyDomain = Readonly<{ + /** Physical column name in the test table, e.g. `eql_v3_integer_ord`. */ + slug: string + /** The domain without its `eql_v3_` prefix, e.g. `integer_ord`. Drives family matching. */ + bare: string + eqlType: EqlV3TypeName + spec: DomainSpec +}> + +/** `public.eql_v3_integer_ord` → `integer_ord`. */ +function bareDomain(eqlType: EqlV3TypeName | string): string { + return eqlTypeSlug(eqlType).replace(/^eql_v3_/, '') +} + +/** + * `integer` must not swallow `smallint`, and `date` must not swallow + * `timestamp` — but `text` legitimately owns `text_search`. Match on the prefix + * followed by end-of-string or `_`, so `date` claims `date` and `date_ord` + * while `timestamp*` stays with `timestamp`. A naive `startsWith` would not. + */ +function belongsTo(bare: string, prefix: string): boolean { + return bare === prefix || bare.startsWith(`${prefix}_`) +} + +function rowsMatching( + family: FamilyName, + covered: boolean, +): Array<[EqlV3TypeName, DomainSpec]> { + const prefixes = FAMILY_PREFIXES[family] + return typedEntries(V3_MATRIX) + .filter(([, spec]) => isCovered(spec) === covered) + .filter(([eqlType]) => + prefixes.some((p) => belongsTo(bareDomain(eqlType), p)), + ) +} + +/** + * Every domain in a family that the integration matrix covers, in catalog order. + * Deferred domains (see {@link DomainSpec.deferred}) are excluded here rather + * than skipped per-test, so a family file cannot accidentally assert against a + * domain the database does not have. + */ +export function domainsForFamily(family: FamilyName): FamilyDomain[] { + return rowsMatching(family, true).map(([eqlType, spec]) => ({ + slug: eqlTypeSlug(eqlType), + bare: bareDomain(eqlType), + eqlType, + spec, + })) +} + +/** Domains a family declares but the matrix skips, with the reason. Reported by the driver. */ +export function deferredForFamily( + family: FamilyName, +): Array<{ slug: string; bare: string; reason: string }> { + return rowsMatching(family, false).map(([eqlType, spec]) => ({ + slug: eqlTypeSlug(eqlType), + bare: bareDomain(eqlType), + reason: deferredReason(spec) ?? '', + })) +} diff --git a/packages/test-kit/src/index.ts b/packages/test-kit/src/index.ts new file mode 100644 index 000000000..af54e53de --- /dev/null +++ b/packages/test-kit/src/index.ts @@ -0,0 +1,53 @@ +/** + * Shared EQL v3 test harness. + * + * The domain catalog is the single source of truth: it is + * `satisfies Record` over the real column union, so + * adding a domain to the SDK fails this package's typecheck until a row exists. + * Everything else here derives from it — which operations a domain must answer, + * which it must reject, and what the expected rows are. + * + * Consumed as TypeScript source (no build). See `vitest.shared.ts` at the repo + * root for the runtime aliases and `tsconfig.json` here for the compile-time ones. + */ + +export type { IntegrationAdapter } from './adapter.ts' +export { + type DomainSpec, + deferredReason, + EQL_V3_DOMAIN_SCHEMA, + type EqlV3TypeName, + eqlTypeSlug, + isCovered, + typedEntries, + V3_MATRIX, +} from './catalog.ts' +export { + deferredForFamily, + domainsForFamily, + FAMILY_NAMES, + type FamilyDomain, + type FamilyName, +} from './families.ts' +export { + negativeOps, + type Plain, + positiveOps, + type QueryOp, + type QueryOpKind, +} from './ops.ts' +export { + comparePlain, + containsPlain, + expectedKeysFor, + plainValue, + sortedKeysFor, +} from './oracle.ts' +export { + NULL_ROW_KEY, + type PlainRow, + planRows, + planTable, + type RowPlan, + type TableSpec, +} from './rows.ts' diff --git a/packages/test-kit/src/ops.ts b/packages/test-kit/src/ops.ts new file mode 100644 index 000000000..9bf6724d1 --- /dev/null +++ b/packages/test-kit/src/ops.ts @@ -0,0 +1,101 @@ +import type { QueryCapabilities } from '@cipherstash/stack/eql/v3' + +/** Every query operation an adapter can be asked to perform on an encrypted column. */ +export type QueryOpKind = + | 'eq' + | 'ne' + | 'in' + | 'notIn' + | 'gt' + | 'gte' + | 'lt' + | 'lte' + | 'between' + | 'notBetween' + | 'contains' + | 'order' + | 'isNull' + | 'isNotNull' + +export type Plain = string | number | bigint | boolean | Date + +export type QueryOp = + | { + kind: 'eq' | 'ne' | 'gt' | 'gte' | 'lt' | 'lte' + column: string + value: Plain + } + | { + kind: 'in' | 'notIn' + column: string + values: readonly Plain[] + /** + * Supabase only: drive the raw `.filter(col, 'in', […])` path instead of + * `.in()`. They are different code paths — the raw path reached `in` with + * no element-split and encrypted the whole list as one term — so both + * must be proven against the same oracle. + */ + asRawFilter?: boolean + } + | { kind: 'between' | 'notBetween'; column: string; lo: Plain; hi: Plain } + | { kind: 'contains'; column: string; needle: string } + | { kind: 'order'; column: string; direction: 'asc' | 'desc' } + | { kind: 'isNull' | 'isNotNull'; column: string } + +/** + * Which operations each capability unlocks. This table is the ONLY place the + * mapping lives: the positive tests and the rejection tests are both derived + * from it, so flipping a capability flag in the catalog turns a passing + * rejection into a failing query (and vice versa) rather than silently + * disagreeing with the SDK. + * + * `orderAndRange` implies equality in the SDK's presets, but that is a property + * of the CATALOG rows (`ORD` sets `equality: true`), not of this table — do not + * encode the implication twice. + */ +const OPS_BY_CAPABILITY = { + equality: ['eq', 'ne', 'in', 'notIn'], + orderAndRange: ['gt', 'gte', 'lt', 'lte', 'between', 'notBetween', 'order'], + freeTextSearch: ['contains'], +} as const satisfies Record + +/** + * Null presence is structural, not cryptographic: a NULL plaintext is stored as + * a SQL NULL on every domain, including storage-only ones. So `isNull` / + * `isNotNull` are never capability-gated, and never appear in the rejection set. + */ +const STRUCTURAL_OPS: readonly QueryOpKind[] = ['isNull', 'isNotNull'] + +function enabledBy(kind: QueryOpKind, caps: QueryCapabilities): boolean { + return ( + Object.keys(OPS_BY_CAPABILITY) as Array + ).some( + (cap) => + caps[cap] && + (OPS_BY_CAPABILITY[cap] as readonly QueryOpKind[]).includes(kind), + ) +} + +/** Operations this adapter supports AND this domain's capabilities allow. Must succeed. */ +export function positiveOps( + caps: QueryCapabilities, + supported: ReadonlySet, +): Set { + return new Set([...supported].filter((kind) => enabledBy(kind, caps))) +} + +/** + * Operations this adapter supports but this domain's capabilities forbid. Must + * be rejected — by a compile error in typed usage, and by a thrown error here. + * Structural ops are excluded: they are legal on every domain. + */ +export function negativeOps( + caps: QueryCapabilities, + supported: ReadonlySet, +): Set { + return new Set( + [...supported].filter( + (kind) => !enabledBy(kind, caps) && !STRUCTURAL_OPS.includes(kind), + ), + ) +} diff --git a/packages/test-kit/src/oracle.ts b/packages/test-kit/src/oracle.ts new file mode 100644 index 000000000..26fefaf66 --- /dev/null +++ b/packages/test-kit/src/oracle.ts @@ -0,0 +1,102 @@ +import type { DomainSpec } from './catalog.ts' +import type { Plain } from './ops.ts' + +/** + * The expected result of every query is computed from the PLAINTEXT, never + * restated as a literal. Lifted from `drizzle-v3/operators-live-pg.test.ts`, + * which is where these rules were originally worked out; the comments are the + * load-bearing part. + */ + +/** + * Row `i` of a domain takes `samples[min(i, len - 1)]`. Domains with fewer + * distinct samples than there are rows therefore REUSE their last value, which + * is deliberate: it produces ties, so ordering tie-breaks and range boundaries + * are exercised rather than vacuously satisfied. + */ +export function plainValue( + spec: DomainSpec, + rowKeys: readonly string[], + rowKey: string, +): Plain { + const index = rowKeys.indexOf(rowKey) + if (index < 0) throw new Error(`Unknown row key: ${rowKey}`) + const sample = spec.samples[Math.min(index, spec.samples.length - 1)] + if (sample === undefined) { + throw new Error(`Domain has no samples for row ${rowKey}`) + } + return sample +} + +export function comparePlain(left: Plain, right: Plain): number { + if (left instanceof Date && right instanceof Date) { + return left.getTime() - right.getTime() + } + if (typeof left === 'number' && typeof right === 'number') { + return left - right + } + // bigint order domains (`bigint_ord`/`bigint_ord_ore`) carry i64 samples + // beyond Number.MAX_SAFE_INTEGER, so they must be compared as bigints — the + // subtraction is narrowed to -1/0/1 because callers expect a `number`. + if (typeof left === 'bigint' && typeof right === 'bigint') { + return left < right ? -1 : left > right ? 1 : 0 + } + if (typeof left === 'string' && typeof right === 'string') { + // eql_v3 text ordering is BYTEWISE, not locale-collated: the oracle must + // model codepoint order, not `localeCompare` (which folds case, reorders + // punctuation vs letters, and is locale-dependent). Text samples must stay + // ASCII/unambiguous so UTF-16 code-unit order == the byte order the DB + // actually sorts by. + return left < right ? -1 : left > right ? 1 : 0 + } + if (typeof left === 'boolean' && typeof right === 'boolean') { + return Number(left) - Number(right) + } + throw new Error( + `Unsupported ordered values: ${String(left)}, ${String(right)}`, + ) +} + +/** Row keys whose plaintext satisfies `predicate`. The NULL row is never included. */ +export function expectedKeysFor( + spec: DomainSpec, + rowKeys: readonly string[], + predicate: (value: Plain) => boolean, +): string[] { + return rowKeys.filter((rowKey) => + predicate(plainValue(spec, rowKeys, rowKey)), + ) +} + +/** + * Oracle for the encrypted ORDER BY. Domains with fewer samples than rows give + * two rows equal values, so the comparison alone does not determine the row + * order — ties break on `rowKey` ascending, which the query mirrors with a + * secondary `ORDER BY row_key`. Without that both sides would be arbitrary and + * the test would flake rather than prove ordering. + */ +export function sortedKeysFor( + spec: DomainSpec, + rowKeys: readonly string[], + direction: 'asc' | 'desc', +): string[] { + return [...rowKeys].sort((left, right) => { + const cmp = comparePlain( + plainValue(spec, rowKeys, left), + plainValue(spec, rowKeys, right), + ) + if (cmp !== 0) return direction === 'asc' ? cmp : -cmp + return left < right ? -1 : left > right ? 1 : 0 + }) +} + +/** + * The `contains` oracle. The match index downcases and tokenizes into 3-grams + * with `include_original: true`, so a needle matches iff it appears in the + * downcased value — with the caveat that a needle longer than the tokenizer + * window contributes a whole-value token no stored trigram supplies. Family + * files choose needles accordingly; this models the plaintext side only. + */ +export function containsPlain(value: Plain, needle: string): boolean { + return String(value).toLowerCase().includes(needle.toLowerCase()) +} diff --git a/packages/test-kit/src/rows.ts b/packages/test-kit/src/rows.ts new file mode 100644 index 000000000..f42795b78 --- /dev/null +++ b/packages/test-kit/src/rows.ts @@ -0,0 +1,106 @@ +import type { FamilyDomain, FamilyName } from './families.ts' +import type { Plain } from './ops.ts' +import { plainValue } from './oracle.ts' + +/** The table one family file creates: one column per covered domain in the family. */ +export type TableSpec = Readonly<{ + /** Unique per run so concurrent suites and reruns never collide. */ + name: string + runId: string + columns: readonly FamilyDomain[] +}> + +export type PlainRow = Readonly<{ + rowKey: string + runId: string + /** Plaintext per column slug. Empty on the NULL row. */ + values: Readonly> +}> + +export type RowPlan = Readonly<{ + /** Row keys carrying values, in ascending order — the oracle's domain. */ + rowKeys: readonly string[] + byKey: Readonly> + /** Inserted one at a time, through the adapter's `encryptModel` path. */ + singleKeys: readonly string[] + /** Inserted as one array, through the adapter's `bulkEncryptModels` path. */ + bulkKeys: readonly string[] + /** Every encrypted column NULL. Exercises `isNull`/`isNotNull` on every domain. */ + nullRow: PlainRow +}> + +/** + * Three value rows minimum, so a predicate that over-matches by one row is + * detectable; four when any domain in the family has that many distinct samples, + * so the extra sample (usually a type bound) reaches the database. More rows buy + * nothing: the oracle already covers ties via the `samples[min(i, len-1)]` reuse. + */ +const MIN_VALUE_ROWS = 3 +const MAX_VALUE_ROWS = 4 + +const ROW_KEYS = ['a', 'b', 'c', 'd'] as const +const NULL_ROW_KEY = 'null' + +function valueRowCount(domains: readonly FamilyDomain[]): number { + const widest = Math.max(...domains.map((d) => d.spec.samples.length)) + return Math.min(Math.max(widest, MIN_VALUE_ROWS), MAX_VALUE_ROWS) +} + +/** + * A table name unique to this family and run. Postgres identifiers cap at 63 + * bytes and the run id is a caller-supplied short hex string, so this stays well + * inside the limit for every family name. + */ +export function planTable( + family: FamilyName, + domains: readonly FamilyDomain[], + runId: string, +): TableSpec { + if (domains.length === 0) { + throw new Error( + `Family "${family}" has no covered domains — check FAMILY_PREFIXES and the catalog's deferred rows`, + ) + } + return { + name: `v3_it_${family.replace(/-/g, '_')}_${runId}`, + runId, + columns: domains, + } +} + +/** + * Seed rows plus the single/bulk split. + * + * The halves are DISJOINT and interleaved (`a`,`c` single; `b`,`d` bulk) so that + * any per-domain predicate matching more than one row necessarily spans both + * encryption paths. That is what turns "the bulk path mangled a row" into a red + * test rather than a coincidence — before this, the Supabase adapter's array + * insert (`bulkEncryptModels`) and the Drizzle single insert (`encryptModel`) + * were each untested on their respective sides. + */ +export function planRows( + domains: readonly FamilyDomain[], + runId: string, +): RowPlan { + const count = valueRowCount(domains) + const rowKeys = ROW_KEYS.slice(0, count) + + const byKey: Record = {} + for (const rowKey of rowKeys) { + const values: Record = {} + for (const domain of domains) { + values[domain.slug] = plainValue(domain.spec, rowKeys, rowKey) + } + byKey[rowKey] = { rowKey, runId, values } + } + + return { + rowKeys, + byKey, + singleKeys: rowKeys.filter((_, i) => i % 2 === 0), + bulkKeys: rowKeys.filter((_, i) => i % 2 === 1), + nullRow: { rowKey: NULL_ROW_KEY, runId, values: {} }, + } +} + +export { NULL_ROW_KEY } diff --git a/packages/test-kit/tsconfig.json b/packages/test-kit/tsconfig.json new file mode 100644 index 000000000..0524347d6 --- /dev/null +++ b/packages/test-kit/tsconfig.json @@ -0,0 +1,42 @@ +{ + "compilerOptions": { + "lib": ["ES2022"], + "target": "ES2022", + "module": "ESNext", + "moduleDetection": "force", + "esModuleInterop": true, + + "moduleResolution": "bundler", + "allowImportingTsExtensions": true, + "verbatimModuleSyntax": true, + "noEmit": true, + + // Mirrors `packages/stack/tsconfig.json`: protect-ffi 0.24+ uses + // conditional exports, and bundler resolution omits `node` by default. + "customConditions": ["node"], + + "strict": true, + "skipLibCheck": true, + "noFallthroughCasesInSwitch": true, + "noUncheckedIndexedAccess": true, + + // Resolve stack to SOURCE, not `dist`. Without this the kit — and every + // unit suite that imports the catalog through it — would only typecheck + // after `pnpm build`, coupling `pnpm test` to a build step. The matching + // runtime aliases live in `vitest.shared.ts`. + "baseUrl": ".", + "paths": { + "@cipherstash/stack": ["../stack/src/index.ts"], + "@cipherstash/stack/eql/v3": ["../stack/src/eql/v3/index.ts"], + "@cipherstash/stack/schema": ["../stack/src/schema/index.ts"], + "@cipherstash/stack/v3": ["../stack/src/encryption/v3.ts"], + "@cipherstash/stack/supabase": ["../stack/src/supabase/index.ts"], + + // Resolving stack to source means we also inherit stack's OWN internal + // alias: `src/eql/v3/columns.ts` imports `@/schema`. Without this the + // kit's typecheck fails inside stack's sources rather than in the kit. + "@/*": ["../stack/src/*"] + } + }, + "include": ["src/**/*.ts"] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 2613ace6b..5c8575260 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -670,6 +670,19 @@ importers: specifier: catalog:repo version: 0.41.0 + packages/test-kit: + dependencies: + '@cipherstash/stack': + specifier: workspace:* + version: link:../stack + devDependencies: + typescript: + specifier: catalog:repo + version: 5.9.3 + vitest: + specifier: catalog:repo + version: 3.2.6(@types/node@25.8.0)(jiti@2.7.0)(lightningcss@1.30.2)(terser@5.44.1)(tsx@4.22.1)(yaml@2.9.0) + packages/wizard: dependencies: '@anthropic-ai/claude-agent-sdk': diff --git a/vitest.shared.ts b/vitest.shared.ts new file mode 100644 index 000000000..15147279a --- /dev/null +++ b/vitest.shared.ts @@ -0,0 +1,44 @@ +import { resolve } from 'node:path' + +/** + * Resolve `@cipherstash/test-kit` and the `@cipherstash/stack` public subpaths + * it imports to SOURCE rather than `dist`. + * + * The kit has no build step, and its catalog imports stack through the public + * subpaths (not stack's internal `@/` alias) because packages outside stack — + * `@cipherstash/drizzle`, and the adapter packages after the split — have to be + * able to import it too. Left unaliased, those specifiers resolve to + * `packages/stack/dist`, which would couple `pnpm test` to a prior `pnpm build`. + * The matching compile-time mapping is `packages/test-kit/tsconfig.json` `paths`. + * + * Spread into each package's vitest config rather than copied, so the two + * cannot drift: + * + * import { sharedAlias } from '../../vitest.shared' + * resolve: { alias: { ...sharedAlias, '@/': … } } + */ +const repoRoot = __dirname + +export const sharedAlias: Record = { + '@cipherstash/test-kit/catalog': resolve( + repoRoot, + 'packages/test-kit/src/catalog.ts', + ), + '@cipherstash/test-kit': resolve(repoRoot, 'packages/test-kit/src/index.ts'), + '@cipherstash/stack/eql/v3': resolve( + repoRoot, + 'packages/stack/src/eql/v3/index.ts', + ), + '@cipherstash/stack/schema': resolve( + repoRoot, + 'packages/stack/src/schema/index.ts', + ), + '@cipherstash/stack/v3': resolve( + repoRoot, + 'packages/stack/src/encryption/v3.ts', + ), + '@cipherstash/stack/supabase': resolve( + repoRoot, + 'packages/stack/src/supabase/index.ts', + ), +} From e9d72ac2752aa391bd5844aab0da70399dc5a481 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 10 Jul 2026 21:27:30 +1000 Subject: [PATCH 04/20] =?UTF-8?q?test(integration):=20harness=20=E2=80=94?= =?UTF-8?q?=20docker=20variants,=20CLI=20install,=20loud=20env=20gate?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the two database variants the integration suites run against, installs EQL v3 through the real `stash eql install`, and replaces the credential `describe.skipIf` gates with a failure that names what is missing. Installing via the CLI (not a hand-rolled SQL bundle apply) means an installer regression fails the integration jobs instead of hiding behind a test-only code path that reimplements half of it. `--supabase --direct`: the first applies the eql_v3 + eql_v3_internal grants, the second suppresses an interactive prompt that would hang CI. Measured, not assumed, against `supabase/postgres:17.4.1.048`: - The CLI installs fine as the NON-superuser `postgres` role. No superuser connection, no SUPABASE_ADMIN_URL — the plan had budgeted for both. - `_ord_ore` domains are NOT "absent" on managed Postgres, as the plan claimed. All 51 domains and all 33 ORE comparison functions install; only the btree opclass is skipped (`pg_opclass` count 0 vs 1 on plain Postgres). So ORE columns are creatable and `eql_v3.gt`/`lt` still compare correctly, but `ORDER BY ord_term_ore` silently falls back to bytea order. Corrected in the catalog's deferral reason and in the plan doc; pinned by a harness test that asserts the opclass count against `rolsuper`. Three fixture bugs found by running it rather than reasoning about it: - `authenticator` is a RESERVED role; even `postgres` cannot set its password ("only superusers can modify it"). It must come from an initdb script. - That script must sort AFTER `migrate.sh`, which is where the image creates the roles. A `99-` prefix sorts BEFORE it (`'9' < 'm'`) and aborts the boot. - `pg_isready` is not a readiness check for either image: both boot a temporary server for init that listens on the unix socket only. `up --wait` returned while the real server was still restarting and the EQL install died with "Connection terminated unexpectedly". The postgres variant now probes TCP; the supabase variant probes the last thing init does. Ports are 55432/55433/55430, never the defaults: a developer running this very likely also has `local/docker-compose.yml`, a `supabase start` stack, or their own Postgres on 5432, and shadowing the database they think they are talking to is worse than failing to bind. Verified end to end: unconfigured runs fail loudly with an actionable message; a `~/.cipherstash` profile stands in for the four CS_* vars; both variants boot clean and install with no sleeps; `pnpm test` still runs 1554 unit tests and zero integration tests. --- .github/actions/integration-setup/action.yml | 45 +++++++++ .../2026-07-10-eql-v3-integration-tests.md | 4 +- local/docker-compose.postgres.yml | 44 +++++++++ local/docker-compose.supabase.yml | 83 +++++++++++++++++ local/supabase-init.sql | 32 +++++++ packages/stack/integration/global-setup.ts | 27 ++++++ .../integration/harness.integration.test.ts | 91 +++++++++++++++++++ packages/stack/integration/vitest.config.ts | 48 ++++++++++ packages/stack/package.json | 5 +- packages/stack/vitest.config.ts | 7 +- packages/test-kit/src/catalog.ts | 23 +++-- packages/test-kit/src/env.ts | 85 +++++++++++++++++ packages/test-kit/src/index.ts | 7 ++ packages/test-kit/src/install.ts | 79 ++++++++++++++++ turbo.json | 82 ++++++++++++----- 15 files changed, 626 insertions(+), 36 deletions(-) create mode 100644 .github/actions/integration-setup/action.yml create mode 100644 local/docker-compose.postgres.yml create mode 100644 local/docker-compose.supabase.yml create mode 100644 local/supabase-init.sql create mode 100644 packages/stack/integration/global-setup.ts create mode 100644 packages/stack/integration/harness.integration.test.ts create mode 100644 packages/stack/integration/vitest.config.ts create mode 100644 packages/test-kit/src/env.ts create mode 100644 packages/test-kit/src/install.ts diff --git a/.github/actions/integration-setup/action.yml b/.github/actions/integration-setup/action.yml new file mode 100644 index 000000000..d8ec179f7 --- /dev/null +++ b/.github/actions/integration-setup/action.yml @@ -0,0 +1,45 @@ +name: Integration test setup +description: >- + Checkout, pnpm, Node, dependencies, and a built `stash` CLI — the common + preamble for every integration workflow. The CLI build is required, not + incidental: the integration harness installs EQL v3 by shelling out to + `stash eql install --eql-version 3`, so an installer regression fails the job + rather than hiding behind a test-only SQL apply. + +inputs: + node-version: + description: Node major to run the suites on. + required: false + default: '22' + +runs: + using: composite + steps: + - name: Checkout Repo + uses: actions/checkout@v6 + + - uses: pnpm/action-setup@v6.0.8 + name: Install pnpm + with: + run_install: false + + - name: Install Node.js + uses: actions/setup-node@v6 + with: + node-version: ${{ inputs.node-version }} + cache: 'pnpm' + + # node-pty's install hook falls back to `node-gyp rebuild` when no + # linux-x64 prebuild matches. pnpm/action-setup v6 no longer ships node-gyp + # on PATH, so install it explicitly. + - name: Install node-gyp + shell: bash + run: npm install -g node-gyp + + - name: Install dependencies + shell: bash + run: pnpm install --frozen-lockfile + + - name: Build the stash CLI (used to install EQL v3) + shell: bash + run: pnpm exec turbo run build --filter stash diff --git a/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md b/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md index c259a17ad..4f4876d9d 100644 --- a/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md +++ b/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md @@ -25,7 +25,9 @@ - **The canonical EQL v3 types are imported nowhere.** `@cipherstash/eql@3.0.0` exports ~130 per-domain types (`IntegerOrd = { v, i, c, op: OpeCllw }` and its term-only twin `IntegerOrdQuery = { v, i, op }`), but no `.ts` source file imports them; only `@cipherstash/eql/sql` is consumed. Both adapters erase to `unknown`: `eql/v3/drizzle/operators.ts:51,55` type the client's `encrypt`/`bulkEncrypt` as returning `unknown`; `encryptOperands` returns `Promise`; `supabase/query-builder-v3.ts:393,433,463` do the same, and the `Result` wrapper collapses to `{ data?: unknown }`. - **v3 JSON is unimplemented, not merely untested.** The bundle ships `public.eql_v3_json` and `public.eql_v3_jsonb_entry` (51 `CREATE DOMAIN`s, 56 `eql_v3.{ste_vec,jsonb_path,selector,contained_by}` functions), but the SDK models 41 domains and `eql/v3/columns.ts` admits only `bigint | boolean | date | number | string | timestamp` as `cast_as`. There is no `'json'` kind, so a v3 JSON column cannot be declared. **The gap is in the core v3 schema; both adapters inherit it.** `eql/v3/drizzle/codec.ts:38` already carries defensive SteVec decode logic (`sv[0].c`) for documents nothing can currently create. - **`@cipherstash/stack/drizzle` is not `@cipherstash/drizzle`.** They are a fork: `@cipherstash/drizzle@3.0.3` peer-depends on the predecessor SDK `@cipherstash/protect@12` and exports `createProtectOperators` (2,038 lines); stack's in-tree copy exports `createEncryptionOperators` (1,945 lines); ~645 lines have diverged. There is no dependency between them — hence no cycle today, only duplication. Docs reference the stack subpath 13× and the package 2×. -- **Ordering is safe on OPE.** EQL 3.0.0 pins `_ord` domains to `op` (CLLW-OPE), which orders via a native `bytea` btree, so `ORDER BY eql_v3.ord_term(col)` works on every provider without superuser. `_ord_ore` domains remain block-ORE, are superuser-only, and are absent on managed Postgres — see `docs/eql-v3-ord-term-ordering-defect.md`. +- **Ordering is safe on OPE.** EQL 3.0.0 pins `_ord` domains to `op` (CLLW-OPE), which orders via a native `bytea` btree, so `ORDER BY eql_v3.ord_term(col)` works on every provider without superuser. +- **`_ord_ore` is not "absent" on managed Postgres — it is silently wrong.** Measured against `supabase/postgres:17.4.1.048` after `stash eql install --eql-version 3 --supabase --direct` as the non-superuser `postgres` role: all 51 `public.eql_v3_*` domains install, and so do the 33 ORE comparison functions; only the btree **opclass** is skipped (`pg_opclass` count 0). An `_ord_ore` column is therefore creatable and `eql_v3.gt`/`lt` still compare in true ORE order, but `ORDER BY eql_v3.ord_term_ore(col)` falls back to raw `bytea` order and mis-sorts with no error. That is why ORE gets its own suite rather than a skip. See `docs/eql-v3-ord-term-ordering-defect.md`. +- **The CLI installs fine as a non-superuser.** `stash eql install --eql-version 3 --supabase --direct --database-url …` runs non-interactively as `postgres` on `supabase/postgres` and grants `anon` USAGE on **both** `eql_v3` and `eql_v3_internal`. No `SUPABASE_ADMIN_URL` is needed. (`--supabase` without `--direct`/`--migration` prompts, so pass `--direct`.) - **`postgres-eql:17-2.3.1` ships EQL v2, not v3.** Every DB variant must install v3 at setup. - **`./supabase` and `./drizzle` are published stack subpaths** (`npm view @cipherstash/stack exports`); `./eql/v3/drizzle` is not. - **Moving an adapter out of stack requires promoting internals.** `src/supabase` imports six non-public modules (`@/encryption/helpers`, `@/encryption/operations/base-operation`, `@/eql/v3/columns`, `@/eql/v3/domain-registry`, `@/types`, `@/utils/logger`); `src/eql/v3/drizzle` imports three (`@/types`, `@/schema/match-defaults`, `@/encryption/operations/base-operation`). Note `@/types` is the internal `src/types.ts`, distinct from the published `./types` → `types-public`. diff --git a/local/docker-compose.postgres.yml b/local/docker-compose.postgres.yml new file mode 100644 index 000000000..81ce55bef --- /dev/null +++ b/local/docker-compose.postgres.yml @@ -0,0 +1,44 @@ +# Plain Postgres for the Drizzle v3 integration suite. +# +# Drizzle talks straight to Postgres, so there is no PostgREST here and no +# anon/authenticated role surface — that is the Supabase variant's job. +# +# Ports are deliberately NOT the defaults. A developer running this is very +# likely also running `local/docker-compose.yml`, a `supabase start` stack, or +# their own Postgres on 5432. Binding 5432 here would either fail to start or, +# worse, shadow the database they think they are talking to. +# +# Connect DIRECT to the database, never through a pooler: `SET ROLE` and session +# advisory locks (both used by the EQL installer) are unstable over PgBouncer in +# transaction mode. +name: cs-integration-postgres + +services: + postgres: + # Ships EQL v2 preinstalled; EQL v3 is installed by the harness via + # `stash eql install --eql-version 3`, which is the point — the integration + # suites exercise the installer customers actually run. + image: ghcr.io/cipherstash/postgres-eql:17-2.3.1 + environment: + PGPORT: 5432 + POSTGRES_DB: cipherstash + POSTGRES_USER: cipherstash + PGUSER: cipherstash + POSTGRES_PASSWORD: password + ports: + - "55432:5432" + # No volume: every run starts from a clean database, so a stale EQL + # generation can never make a suite pass for the wrong reason. + # + # `-h 127.0.0.1` is load-bearing. During init the entrypoint runs a TEMPORARY + # server that listens on the unix socket ONLY; a socket-based `pg_isready` + # succeeds against it, so `up --wait` returns while the real server is still + # restarting and the next connection dies with "Connection terminated + # unexpectedly" (observed: the EQL install failed exactly there). TCP is not + # published until the final server is up, so probing it is the honest check. + healthcheck: + test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U cipherstash -d cipherstash"] + interval: 1s + timeout: 5s + start_period: 5s + retries: 30 diff --git a/local/docker-compose.supabase.yml b/local/docker-compose.supabase.yml new file mode 100644 index 000000000..4931db194 --- /dev/null +++ b/local/docker-compose.supabase.yml @@ -0,0 +1,83 @@ +# Supabase-flavoured Postgres + PostgREST for the Supabase v3 integration suite. +# +# `supabase/postgres` rather than the plain `postgres-eql` image, because the +# thing under test is what a managed Supabase project actually does: the +# `postgres` role is NOT a superuser there (verified: `pg_roles.rolsuper = f`), +# and the EQL v3 install, the `ALTER DEFAULT PRIVILEGES FOR ROLE postgres` in the +# grants block, and the ORE opclass self-skip all behave differently under that +# constraint. A plain Postgres would make every privilege check pass vacuously. +# +# NOT the full `supabase start` stack: no Kong, GoTrue, Studio or Storage. The +# adapter speaks PostgREST, and PostgREST is what we run. +# +# Images are pinned by DIGEST, not tag — Supabase re-pushes tags. +# +# Ports are deliberately NOT the defaults. A developer running this very likely +# also has `local/docker-compose.yml`, a `supabase start` stack, or their own +# Postgres bound to 5432/3000. Binding those here would either fail to start or, +# worse, silently shadow the database they think they are talking to. +# +# Connect DIRECT to the database, never through a pooler: `SET ROLE` and session +# advisory locks (both used by the EQL installer) are unstable over PgBouncer in +# transaction mode. +name: cs-integration-supabase + +services: + db: + image: supabase/postgres:17.4.1.048@sha256:68f90b9a227f56b583d2c1aef2fc683c4dc693a3c5ebf5191903e966318d87ee + environment: + POSTGRES_PASSWORD: password + ports: + - "55433:5432" + volumes: + # Supplies the one thing the image omits: a password for `authenticator`. + # + # The `zz-` prefix is load-bearing. The Postgres entrypoint runs + # `/docker-entrypoint-initdb.d/*` in ASCII order, and this image creates + # its roles from `migrate.sh` in that same directory. A `99-` prefix sorts + # BEFORE `migrate.sh` (`'9' < 'm'`), so the script would run against a + # database where `authenticator` does not exist yet and abort the boot. + - ./supabase-init.sql:/docker-entrypoint-initdb.d/zz-cipherstash-authenticator.sql:ro + # No volume for PGDATA: every run starts clean, so a stale EQL generation can + # never make a suite pass for the wrong reason. + # + # `pg_isready` is NOT a sufficient readiness check here. This image boots a + # temporary server to run `migrate.sh`, then restarts it; `pg_isready` + # succeeds against the temporary one, so `up --wait` returns while the real + # server is still coming up and the next connection dies with "Connection + # terminated unexpectedly" (observed: the EQL install failed exactly there). + # + # Probe the LAST thing init does instead — the `authenticator` password set + # by our `zz-` script. It is true only once every init step has run. + healthcheck: + test: + - CMD-SHELL + - psql -U postgres -d postgres -tAc "SELECT 1 FROM pg_authid WHERE rolname='authenticator' AND rolpassword IS NOT NULL" | grep -q 1 + interval: 2s + timeout: 5s + start_period: 10s + retries: 60 + + postgrest: + image: postgrest/postgrest:v12.2.12@sha256:be3b6ab31a94e3e89f08edad40be0234ae0951e3297f2043d797c219165f5afa + depends_on: + db: + condition: service_healthy + environment: + # Connect as `authenticator`, then SET ROLE to `anon` for unauthenticated + # requests. Pointing PostgREST at the owner instead would make every grant + # check pass vacuously — proving the Supabase grants work is half the point. + PGRST_DB_URI: "postgres://authenticator:authpass@db:5432/postgres" + PGRST_DB_SCHEMAS: public + PGRST_DB_ANON_ROLE: anon + # Lets `NOTIFY pgrst, 'reload schema'` reach the server, so a table created + # after boot becomes visible without a restart. + PGRST_DB_CHANNEL_ENABLED: "true" + ports: + - "55430:3000" + # No healthcheck: the postgrest image is distroless — it has no shell, no + # wget, no curl, so every in-container probe fails with "executable file not + # found" and `docker compose up --wait` reports the service unhealthy while + # it is in fact serving. Readiness is asserted from the harness instead, by + # polling the root endpoint until the table under test appears in the OpenAPI + # output — which is what actually has to be true before a suite can run. diff --git a/local/supabase-init.sql b/local/supabase-init.sql new file mode 100644 index 000000000..2bcd11cd5 --- /dev/null +++ b/local/supabase-init.sql @@ -0,0 +1,32 @@ +-- Make `authenticator` usable by PostgREST. +-- +-- `supabase/postgres` already ships `anon`, `authenticated`, `service_role` and +-- `authenticator`, and `authenticator` is already `LOGIN NOINHERIT` and already a +-- member of the other three. What it does NOT ship is a password (`pg_authid` +-- reports `rolpassword IS NULL`), so PostgREST cannot connect. That is the only +-- thing this file supplies. +-- +-- Do NOT reuse `local/postgrest-roles.sql` here. It is written for plain +-- Postgres, where the roles do not exist: its `CREATE ROLE ... IF NOT EXISTS` +-- branch sets the password only on first create, so against an image that +-- already has `authenticator` the password is silently never set and PostgREST +-- fails to authenticate. +-- +-- Runs as `supabase_admin` via /docker-entrypoint-initdb.d, which is the image's +-- superuser — and it must, because `authenticator` is a RESERVED role that even +-- the `postgres` role cannot modify (`ERROR: "authenticator" is a reserved role, +-- only superusers can modify it`). It therefore cannot be done from the harness. +-- +-- It must also be mounted with a filename that sorts AFTER `migrate.sh`, which is +-- where this image creates the roles. See the compose file. +-- +-- Nothing else in the integration harness needs a superuser: the EQL v3 install +-- runs as the non-superuser `postgres` role, exactly as a customer's Supabase +-- project does. + +ALTER ROLE authenticator WITH LOGIN PASSWORD 'authpass' NOINHERIT; + +-- Idempotent no-ops on this image, kept so the file also works against a plain +-- Postgres that has had the roles created some other way. +GRANT anon, authenticated, service_role TO authenticator; +GRANT USAGE ON SCHEMA public TO anon, authenticated, service_role; diff --git a/packages/stack/integration/global-setup.ts b/packages/stack/integration/global-setup.ts new file mode 100644 index 000000000..de61d79c2 --- /dev/null +++ b/packages/stack/integration/global-setup.ts @@ -0,0 +1,27 @@ +import { + databaseUrl, + dbVariant, + installEqlV3, + type Requirement, + requireIntegrationEnv, +} from '@cipherstash/test-kit' + +/** + * Fail fast, before a single container second is spent, and install EQL v3 once + * per run through the real `stash eql install`. + * + * Runs in Vitest's `globalSetup`, so a misconfigured run reports one clear error + * rather than the same error once per test file. + */ +export async function setup(): Promise { + const variant = dbVariant() + + // Every integration suite encrypts, so CipherStash credentials are always + // required. The Supabase variant additionally needs PostgREST — the adapter + // does not speak Postgres. + const requirements: Requirement[] = ['cipherstash', 'database'] + if (variant === 'supabase') requirements.push('pgrest') + requireIntegrationEnv(requirements) + + await installEqlV3(databaseUrl(), variant) +} diff --git a/packages/stack/integration/harness.integration.test.ts b/packages/stack/integration/harness.integration.test.ts new file mode 100644 index 000000000..7dc9d1a49 --- /dev/null +++ b/packages/stack/integration/harness.integration.test.ts @@ -0,0 +1,91 @@ +import { releaseManifest } from '@cipherstash/eql/sql' +import { databaseUrl, dbVariant, pgrestUrl } from '@cipherstash/test-kit' +import postgres from 'postgres' +import { afterAll, expect, it } from 'vitest' + +/** + * Proves the harness itself, so a failure in the adapter suites is never + * ambiguous between "the adapter is broken" and "the database was never set up". + * + * `globalSetup` has already run `stash eql install --eql-version 3` against the + * configured database by the time this file executes. Nothing here is skippable: + * an unconfigured run throws in `globalSetup`, before any test is collected. + */ +const sql = postgres(databaseUrl(), { prepare: false }) + +afterAll(async () => { + await sql.end() +}) + +it('installed the pinned EQL v3 release through the real CLI', async () => { + const [row] = await sql< + { version: string }[] + >`SELECT eql_v3.version() AS version` + + expect(row?.version).toBe(releaseManifest.eqlVersion) +}) + +it('installed the concrete public.eql_v3_* domains', async () => { + const [row] = await sql<{ count: string }[]>` + SELECT count(*)::text AS count + FROM pg_type t + JOIN pg_namespace n ON n.oid = t.typnamespace + WHERE n.nspname = 'public' AND t.typname LIKE 'eql_v3\\_%' + ` + + // The SDK models 41 of them; the bundle ships more (json, jsonb_entry, the + // `*_ord_ope` twins). Assert the floor, not the exact count, so a bundle that + // adds a domain does not fail a test that is really about "the install ran". + expect(Number(row?.count)).toBeGreaterThanOrEqual(41) +}) + +/** + * The ORE opclass is superuser-only. This is the observable difference between + * the two database variants, and the reason the nine `_ord_ore` domains are + * `deferred` in the catalog: on managed Postgres their columns exist and compare + * correctly, but `ORDER BY eql_v3.ord_term_ore(col)` silently falls back to raw + * bytea order. Pinning it here means a bundle change that alters the privilege + * story shows up as a harness failure rather than as a mysterious ordering bug. + */ +it('installs the ORE opclass only when the connecting role is a superuser', async () => { + const [role] = await sql<{ is_super: boolean }[]>` + SELECT rolsuper AS is_super FROM pg_roles WHERE rolname = current_user + ` + const [opclass] = await sql<{ count: string }[]>` + SELECT count(*)::text AS count + FROM pg_opclass o + JOIN pg_namespace n ON n.oid = o.opcnamespace + WHERE n.nspname LIKE 'eql_v3%' + ` + + const opclasses = Number(opclass?.count) + if (role?.is_super) { + expect(opclasses).toBeGreaterThan(0) + } else { + expect(opclasses).toBe(0) + } +}) + +it.runIf(dbVariant() === 'supabase')( + 'grants anon USAGE on both eql_v3 and eql_v3_internal', + async () => { + // `eql_v3_internal` is load-bearing: the SECURITY INVOKER extractors resolve + // it with the CALLER's privileges, so without this grant every encrypted + // filter fails for `anon` with "permission denied for schema". + const [row] = await sql<{ eql_v3: boolean; internal: boolean }[]>` + SELECT has_schema_privilege('anon', 'eql_v3', 'USAGE') AS eql_v3, + has_schema_privilege('anon', 'eql_v3_internal', 'USAGE') AS internal + ` + + expect(row).toEqual({ eql_v3: true, internal: true }) + }, +) + +it.runIf(dbVariant() === 'supabase')( + 'serves PostgREST as anon through the authenticator role', + async () => { + const response = await fetch(`${pgrestUrl()}/`) + + expect(response.status).toBe(200) + }, +) diff --git a/packages/stack/integration/vitest.config.ts b/packages/stack/integration/vitest.config.ts new file mode 100644 index 000000000..ab87d7e14 --- /dev/null +++ b/packages/stack/integration/vitest.config.ts @@ -0,0 +1,48 @@ +import { resolve } from 'node:path' +import { defineConfig } from 'vitest/config' +import { sharedAlias } from '../../../vitest.shared' + +/** + * Integration suites: real ZeroKMS, real Postgres, real PostgREST. + * + * Deliberately a SEPARATE config from `packages/stack/vitest.config.ts`, which + * excludes `integration/**`. `pnpm test` must stay runnable with no credentials + * and no database; these run only under `test:integration:*`, from their own CI + * jobs. That separation is what lets the integration suites throw on missing + * configuration instead of skipping. + * + * `SUITE_GLOB` selects the adapter, so one config serves both without a second + * near-identical file. CI passes it per job; locally it defaults to everything. + */ +const SUITE_GLOB = + process.env['CS_IT_SUITE'] ?? 'integration/**/*.integration.test.ts' + +export default defineConfig({ + resolve: { + alias: { + ...sharedAlias, + '@/': resolve(__dirname, '../src') + '/', + '@cipherstash/protect-ffi/wasm-inline': resolve( + __dirname, + '../__tests__/helpers/stub-protect-ffi-wasm-inline.ts', + ), + '@cipherstash/auth/wasm-inline': resolve( + __dirname, + '../__tests__/helpers/stub-auth-wasm-inline.ts', + ), + }, + }, + test: { + root: resolve(__dirname, '..'), + include: [SUITE_GLOB], + globalSetup: [resolve(__dirname, 'global-setup.ts')], + // Real crypto round-trips over the network. The unit config uses 30s for the + // same reason; seeding a family table encrypts every sample, so hooks need + // more headroom than tests do. + testTimeout: 60_000, + hookTimeout: 180_000, + // One database, shared tables. File-level parallelism would have two family + // suites installing EQL and reloading the PostgREST schema cache at once. + fileParallelism: false, + }, +}) diff --git a/packages/stack/package.json b/packages/stack/package.json index c476443cd..6cbac0229 100644 --- a/packages/stack/package.json +++ b/packages/stack/package.json @@ -224,7 +224,8 @@ "db:eql-v3:install": "tsx scripts/install-eql-v3.ts", "test": "vitest run", "test:types": "vitest --run --typecheck.only", - "release": "tsup" + "release": "tsup", + "test:integration": "vitest run --config integration/vitest.config.ts" }, "devDependencies": { "@cipherstash/eql": "3.0.0", @@ -257,7 +258,7 @@ "uuid": "14.0.0", "zod": "3.25.76" }, - "//optionalDependencies": "@cipherstash/auth ships per-platform native bindings as optional peerDependencies. pnpm does not auto-install platform-matched optional peer deps, so we declare them here as optionalDependencies — pnpm then picks the binary matching the host's os/cpu (from each sub-package's own package.json) and ignores the rest. Required because @cipherstash/stack re-exports the Node auth strategies (OidcFederationStrategy, AccessKeyStrategy, …). All seven names share a single catalog entry to keep them in lockstep.", + "//optionalDependencies": "@cipherstash/auth ships per-platform native bindings as optional peerDependencies. pnpm does not auto-install platform-matched optional peer deps, so we declare them here as optionalDependencies \u2014 pnpm then picks the binary matching the host's os/cpu (from each sub-package's own package.json) and ignores the rest. Required because @cipherstash/stack re-exports the Node auth strategies (OidcFederationStrategy, AccessKeyStrategy, \u2026). All seven names share a single catalog entry to keep them in lockstep.", "optionalDependencies": { "@cipherstash/auth-darwin-arm64": "catalog:repo", "@cipherstash/auth-darwin-x64": "catalog:repo", diff --git a/packages/stack/vitest.config.ts b/packages/stack/vitest.config.ts index 257d2f7d2..b2aeac8c1 100644 --- a/packages/stack/vitest.config.ts +++ b/packages/stack/vitest.config.ts @@ -1,5 +1,5 @@ import { resolve } from 'node:path' -import { defineConfig } from 'vitest/config' +import { configDefaults, defineConfig } from 'vitest/config' import { sharedAlias } from '../../vitest.shared' export default defineConfig({ @@ -26,6 +26,11 @@ export default defineConfig({ }, }, test: { + // Integration suites live in `integration/` and require credentials, a + // database and PostgREST. They THROW rather than skip when unconfigured, so + // they must never be picked up by `pnpm test` — that is the whole reason + // they are a separate config and a separate CI job. + exclude: [...configDefaults.exclude, 'integration/**'], // Live suites make real ZeroKMS / CTS network round-trips. The vast // majority of these tests already pass an explicit `, 30000)` per-test // timeout (300+ call sites); a handful were written without one and so diff --git a/packages/test-kit/src/catalog.ts b/packages/test-kit/src/catalog.ts index 76b3dd526..62fbfe356 100644 --- a/packages/test-kit/src/catalog.ts +++ b/packages/test-kit/src/catalog.ts @@ -282,15 +282,24 @@ const BIGINT_ERR = [9223372036854775808n, -9223372036854775809n] as const // biome-ignore format: one row per domain reads as a table; keep it dense. /** * The `_ord_ore` domains are block-ORE, whose operator class is superuser-only: - * the eql-3.0.0 bundle self-skips those statements on `insufficient_privilege`, - * so on managed Postgres (Supabase) the ORE domains are not installed at all. - * The integration matrix runs against both a plain and a managed Postgres, so it - * covers the OPE-backed `_ord` domains — which order via a native `bytea` btree - * and behave identically on every provider — and defers ORE to a suite that can - * assert its superuser-only install. See `docs/eql-v3-ord-term-ordering-defect.md`. + * the eql-3.0.0 bundle self-skips those statements on `insufficient_privilege`. + * + * Measured against `supabase/postgres:17.4.1.048` after `stash eql install + * --eql-version 3 --supabase --direct`, connected as the non-superuser + * `postgres` role: all 51 `public.eql_v3_*` domains install, and so do the 33 + * ORE comparison functions. Only the btree OPCLASS is skipped. So an `_ord_ore` + * column is creatable, and `eql_v3.gt`/`lt`/`gte`/`lte` still compare in true + * ORE order — but `ORDER BY eql_v3.ord_term_ore(col)` falls back to raw `bytea` + * ordering and sorts DETERMINISTICALLY WRONG, with no error. A matrix whose + * ordering assertions must hold on every provider cannot straddle that, so ORE + * gets its own suite. + * + * The OPE-backed `_ord` domains have no such split: they order via a native + * `bytea` btree and behave identically everywhere. See + * `docs/eql-v3-ord-term-ordering-defect.md`. */ const ORE_DEFERRED = - 'block-ORE opclass is superuser-only, so these domains are absent on managed Postgres; covered by a follow-up suite' + 'block-ORE opclass is superuser-only, so ORDER BY silently mis-sorts on managed Postgres; covered by a follow-up suite' export const V3_MATRIX = { // integer diff --git a/packages/test-kit/src/env.ts b/packages/test-kit/src/env.ts new file mode 100644 index 000000000..c03d1ae62 --- /dev/null +++ b/packages/test-kit/src/env.ts @@ -0,0 +1,85 @@ +import { existsSync } from 'node:fs' +import { homedir } from 'node:os' +import { join } from 'node:path' + +/** + * Integration suites FAIL when they are not configured. They never skip. + * + * The suites they replace were gated by `describe.skipIf(...)`, which turns a + * missing credential into a silent whole-suite skip on a green job — the exact + * failure `live-coverage-guard.test.ts` exists to paper over. Throwing removes + * the need for that guard: an unconfigured run is red, and the message says how + * to fix it. + */ + +export type Requirement = 'cipherstash' | 'database' | 'pgrest' + +/** + * CipherStash credentials resolve from EITHER the four `CS_*` variables OR a + * local `~/.cipherstash` profile written by `stash auth login`. protect-ffi + * builds an `AutoStrategy` that consults both, so a developer who has logged in + * once needs no environment at all; CI, which has no profile, supplies the vars. + * + * Mirrors `examples/prisma/test/e2e/global-setup.ts`, which already resolves + * credentials this way. + */ +function hasCipherStashCredentials(): boolean { + if (process.env['CS_WORKSPACE_CRN']) return true + return existsSync(join(homedir(), '.cipherstash')) +} + +const HINTS: Record string | null> = { + cipherstash: () => + hasCipherStashCredentials() + ? null + : 'CipherStash credentials. Either set CS_WORKSPACE_CRN, CS_CLIENT_ID, ' + + 'CS_CLIENT_KEY and CS_CLIENT_ACCESS_KEY, or run `stash auth login` once ' + + '(writes ~/.cipherstash, which AutoStrategy picks up).', + + database: () => + process.env['DATABASE_URL'] + ? null + : 'DATABASE_URL. Start a database and point at it:\n' + + ' docker compose -f local/docker-compose.postgres.yml up -d --wait\n' + + ' export DATABASE_URL=postgres://cipherstash:password@localhost:55432/cipherstash\n' + + ' ...or, for the Supabase variant:\n' + + ' docker compose -f local/docker-compose.supabase.yml up -d --wait\n' + + ' export DATABASE_URL=postgres://postgres:password@localhost:55433/postgres', + + pgrest: () => + process.env['PGRST_URL'] + ? null + : 'PGRST_URL. The Supabase adapter speaks PostgREST, not Postgres:\n' + + ' docker compose -f local/docker-compose.supabase.yml up -d --wait\n' + + ' export PGRST_URL=http://localhost:55430', +} + +/** + * Throw unless every requirement is satisfied, naming all of them at once — a + * developer fixing one variable at a time, one failed run at a time, is the + * thing this is trying to avoid. + */ +export function requireIntegrationEnv(requires: readonly Requirement[]): void { + const missing = requires + .map((requirement) => HINTS[requirement]()) + .filter((hint): hint is string => hint !== null) + + if (missing.length === 0) return + + throw new Error( + `Integration suite cannot run — missing configuration:\n\n - ${missing.join('\n\n - ')}\n\n` + + 'This suite FAILS rather than skips: a green skip would hide a real regression.', + ) +} + +/** The database URL, having asserted it exists. */ +export function databaseUrl(): string { + requireIntegrationEnv(['database']) + return process.env['DATABASE_URL'] as string +} + +/** The PostgREST base URL, having asserted it exists. */ +export function pgrestUrl(): string { + requireIntegrationEnv(['pgrest']) + return process.env['PGRST_URL'] as string +} diff --git a/packages/test-kit/src/index.ts b/packages/test-kit/src/index.ts index af54e53de..e9583aa26 100644 --- a/packages/test-kit/src/index.ts +++ b/packages/test-kit/src/index.ts @@ -22,6 +22,12 @@ export { typedEntries, V3_MATRIX, } from './catalog.ts' +export { + databaseUrl, + pgrestUrl, + type Requirement, + requireIntegrationEnv, +} from './env.ts' export { deferredForFamily, domainsForFamily, @@ -29,6 +35,7 @@ export { type FamilyDomain, type FamilyName, } from './families.ts' +export { type DbVariant, dbVariant, installEqlV3 } from './install.ts' export { negativeOps, type Plain, diff --git a/packages/test-kit/src/install.ts b/packages/test-kit/src/install.ts new file mode 100644 index 000000000..227372efd --- /dev/null +++ b/packages/test-kit/src/install.ts @@ -0,0 +1,79 @@ +import { execFile } from 'node:child_process' +import { existsSync } from 'node:fs' +import { dirname, resolve } from 'node:path' +import { fileURLToPath } from 'node:url' +import { promisify } from 'node:util' + +const execFileAsync = promisify(execFile) + +/** `packages/test-kit/src` → repo root. */ +const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '../../..') +const STASH_BIN = resolve(REPO_ROOT, 'packages/cli/dist/bin/stash.js') + +export type DbVariant = 'postgres' | 'supabase' + +/** + * Which database the integration suite is pointed at. + * + * `PGRST_URL` is only ever set for the Supabase variant — the Drizzle suite + * talks straight to Postgres — so its presence identifies the variant without a + * second environment variable to keep in sync. `CS_IT_DB_VARIANT` overrides, + * for the case where that stops being true. + */ +export function dbVariant(): DbVariant { + const explicit = process.env['CS_IT_DB_VARIANT'] + if (explicit === 'postgres' || explicit === 'supabase') return explicit + return process.env['PGRST_URL'] ? 'supabase' : 'postgres' +} + +/** + * Install EQL v3 by running the real `stash eql install`, not by applying the + * SQL bundle directly. + * + * The installer is a shipped product surface — it vendors the bundle, verifies + * it against the release manifest, decides what to skip on a non-superuser, and + * applies the Supabase role grants. Driving it here means an installer + * regression fails the integration jobs instead of hiding behind a test-only + * code path that reimplements half of it. + * + * Notes on the flags: + * - `--supabase` applies the `eql_v3` AND `eql_v3_internal` grants to + * anon/authenticated/service_role. The SECURITY INVOKER extractors need both. + * - `--direct` is required alongside it: `--supabase` alone prompts for an + * install mode (migration vs direct) and would hang a CI job. + * - Verified against `supabase/postgres:17.4.1.048` as the NON-superuser + * `postgres` role: no superuser connection is needed. + */ +export async function installEqlV3( + databaseUrl: string, + variant: DbVariant = dbVariant(), +): Promise { + if (!existsSync(STASH_BIN)) { + throw new Error( + `The stash CLI is not built (${STASH_BIN} is missing).\n` + + 'The integration suites install EQL v3 through the real CLI. Build it first:\n' + + ' pnpm exec turbo run build --filter stash', + ) + } + + const args = ['eql', 'install', '--eql-version', '3'] + if (variant === 'supabase') args.push('--supabase', '--direct') + args.push('--database-url', databaseUrl) + + try { + await execFileAsync(process.execPath, [STASH_BIN, ...args], { + cwd: REPO_ROOT, + // The CLI is a clack TUI. Without a TTY it takes its non-interactive + // paths, which is what we want; `CI` makes that explicit. + env: { ...process.env, CI: '1' }, + maxBuffer: 32 * 1024 * 1024, + }) + } catch (cause) { + const detail = cause instanceof Error ? cause.message : String(cause) + throw new Error( + `stash eql install --eql-version 3${variant === 'supabase' ? ' --supabase --direct' : ''} failed.\n` + + 'This is the real installer, so a failure here is a real bug — do not work around it.\n' + + detail, + ) + } +} diff --git a/turbo.json b/turbo.json index 6bfd9160b..7a9936878 100644 --- a/turbo.json +++ b/turbo.json @@ -1,27 +1,59 @@ { - "$schema": "https://turbo.build/schema.json", - "tasks": { - "build": { - "dependsOn": ["^build"], - "inputs": ["$TURBO_DEFAULT$", ".env*"] - }, - "release": { - "dependsOn": ["^build"], - "inputs": ["$TURBO_DEFAULT$", ".env*"] - }, - "dev": { - "cache": false, - "persistent": true - }, - "test": { - "dependsOn": ["^build"], - "inputs": ["$TURBO_DEFAULT$", ".env*"], - "cache": false - }, - "test:e2e": { - "dependsOn": ["^build", "build"], - "inputs": ["$TURBO_DEFAULT$", ".env*"], - "cache": false - } - } + "$schema": "https://turbo.build/schema.json", + "tasks": { + "build": { + "dependsOn": [ + "^build" + ], + "inputs": [ + "$TURBO_DEFAULT$", + ".env*" + ] + }, + "release": { + "dependsOn": [ + "^build" + ], + "inputs": [ + "$TURBO_DEFAULT$", + ".env*" + ] + }, + "dev": { + "cache": false, + "persistent": true + }, + "test": { + "dependsOn": [ + "^build" + ], + "inputs": [ + "$TURBO_DEFAULT$", + ".env*" + ], + "cache": false + }, + "test:e2e": { + "dependsOn": [ + "^build", + "build" + ], + "inputs": [ + "$TURBO_DEFAULT$", + ".env*" + ], + "cache": false + }, + "test:integration": { + "dependsOn": [ + "^build", + "build" + ], + "inputs": [ + "$TURBO_DEFAULT$", + ".env*" + ], + "cache": false + } + } } From c67d0173539283d68b8dfe5fc18579477609f512 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 10 Jul 2026 21:45:07 +1000 Subject: [PATCH 05/20] docs(test-kit): pin down why ORE ordering is silently wrong, not absent MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up to the harness commit, which said `ORDER BY ord_term_ore` "falls back to raw bytea ordering" without explaining the mechanism or bounding the damage. Measured on the two variants: - `eql_v3_internal.ore_block_256` is a COMPOSITE type, not a domain over bytea. Ordering it resolves the btree opclass `ore_block_256_operator_class`, which self-skips on `insufficient_privilege`. With no opclass, Postgres does not error — it falls back to built-in RECORD comparison, which walks down to the raw `bytes` field and compares bytewise. - Bytewise order over ORE ciphertext is deterministic and stable but uncorrelated with plaintext order: over 200 random well-formed terms it disagreed with the ORE comparator on 87. A stable, plausible-looking, silently wrong ordering. - RANGE FILTERS ARE UNAFFECTED. The `<`/`>`/`<=`/`>=` operators are backed by `ore_block_256_lt` → the ORE comparator, and operators need no opclass. Only `ORDER BY`, which resolves an opclass rather than an operator, takes the fallback. A btree index on the column cannot be created at all. Proof the two variants take different code paths: `ORDER BY` over a malformed 1-byte term returns rows on supabase/postgres, and raises `Malformed ORE term` from `ore_block_256_lt` on plain Postgres. Comment-only; no behaviour change. --- .../2026-07-10-eql-v3-integration-tests.md | 6 ++++- packages/test-kit/src/catalog.ts | 26 ++++++++++++++----- 2 files changed, 25 insertions(+), 7 deletions(-) diff --git a/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md b/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md index 4f4876d9d..4a30e788d 100644 --- a/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md +++ b/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md @@ -26,7 +26,11 @@ - **v3 JSON is unimplemented, not merely untested.** The bundle ships `public.eql_v3_json` and `public.eql_v3_jsonb_entry` (51 `CREATE DOMAIN`s, 56 `eql_v3.{ste_vec,jsonb_path,selector,contained_by}` functions), but the SDK models 41 domains and `eql/v3/columns.ts` admits only `bigint | boolean | date | number | string | timestamp` as `cast_as`. There is no `'json'` kind, so a v3 JSON column cannot be declared. **The gap is in the core v3 schema; both adapters inherit it.** `eql/v3/drizzle/codec.ts:38` already carries defensive SteVec decode logic (`sv[0].c`) for documents nothing can currently create. - **`@cipherstash/stack/drizzle` is not `@cipherstash/drizzle`.** They are a fork: `@cipherstash/drizzle@3.0.3` peer-depends on the predecessor SDK `@cipherstash/protect@12` and exports `createProtectOperators` (2,038 lines); stack's in-tree copy exports `createEncryptionOperators` (1,945 lines); ~645 lines have diverged. There is no dependency between them — hence no cycle today, only duplication. Docs reference the stack subpath 13× and the package 2×. - **Ordering is safe on OPE.** EQL 3.0.0 pins `_ord` domains to `op` (CLLW-OPE), which orders via a native `bytea` btree, so `ORDER BY eql_v3.ord_term(col)` works on every provider without superuser. -- **`_ord_ore` is not "absent" on managed Postgres — it is silently wrong.** Measured against `supabase/postgres:17.4.1.048` after `stash eql install --eql-version 3 --supabase --direct` as the non-superuser `postgres` role: all 51 `public.eql_v3_*` domains install, and so do the 33 ORE comparison functions; only the btree **opclass** is skipped (`pg_opclass` count 0). An `_ord_ore` column is therefore creatable and `eql_v3.gt`/`lt` still compare in true ORE order, but `ORDER BY eql_v3.ord_term_ore(col)` falls back to raw `bytea` order and mis-sorts with no error. That is why ORE gets its own suite rather than a skip. See `docs/eql-v3-ord-term-ordering-defect.md`. +- **`_ord_ore` is not "absent" on managed Postgres — it is silently wrong, and only for `ORDER BY`.** Measured against `supabase/postgres:17.4.1.048` after `stash eql install --eql-version 3 --supabase --direct` as the non-superuser `postgres` role: all 51 `public.eql_v3_*` domains install, and so do the 33 ORE comparison functions; only the btree opclass `eql_v3_internal.ore_block_256_operator_class` is skipped (`pg_opclass` count 0 vs 1 on plain Postgres). + - **Range filters still work everywhere.** `<`/`>`/`<=`/`>=` on `ore_block_256` are backed by `ore_block_256_lt` → the ORE comparator, and operators need no opclass. + - **`ORDER BY eql_v3.ord_term_ore(col)` does not error — it sorts, wrongly.** `ore_block_256` is a *composite* type, so with no opclass to resolve, Postgres falls back to built-in record comparison, which walks down to the raw `bytes` field and compares bytewise. That order is deterministic and stable but uncorrelated with plaintext: over 200 random well-formed terms it disagreed with the ORE comparator on **87**. Proof that the two variants take different paths: `ORDER BY` on a malformed 1-byte term returns rows on Supabase and raises `Malformed ORE term` on plain Postgres. + - A btree **index** on such a column cannot be created at all. + - That is why ORE gets its own suite rather than a skip. See `docs/eql-v3-ord-term-ordering-defect.md`. - **The CLI installs fine as a non-superuser.** `stash eql install --eql-version 3 --supabase --direct --database-url …` runs non-interactively as `postgres` on `supabase/postgres` and grants `anon` USAGE on **both** `eql_v3` and `eql_v3_internal`. No `SUPABASE_ADMIN_URL` is needed. (`--supabase` without `--direct`/`--migration` prompts, so pass `--direct`.) - **`postgres-eql:17-2.3.1` ships EQL v2, not v3.** Every DB variant must install v3 at setup. - **`./supabase` and `./drizzle` are published stack subpaths** (`npm view @cipherstash/stack exports`); `./eql/v3/drizzle` is not. diff --git a/packages/test-kit/src/catalog.ts b/packages/test-kit/src/catalog.ts index 62fbfe356..cdbfdfb97 100644 --- a/packages/test-kit/src/catalog.ts +++ b/packages/test-kit/src/catalog.ts @@ -287,12 +287,26 @@ const BIGINT_ERR = [9223372036854775808n, -9223372036854775809n] as const * Measured against `supabase/postgres:17.4.1.048` after `stash eql install * --eql-version 3 --supabase --direct`, connected as the non-superuser * `postgres` role: all 51 `public.eql_v3_*` domains install, and so do the 33 - * ORE comparison functions. Only the btree OPCLASS is skipped. So an `_ord_ore` - * column is creatable, and `eql_v3.gt`/`lt`/`gte`/`lte` still compare in true - * ORE order — but `ORDER BY eql_v3.ord_term_ore(col)` falls back to raw `bytea` - * ordering and sorts DETERMINISTICALLY WRONG, with no error. A matrix whose - * ordering assertions must hold on every provider cannot straddle that, so ORE - * gets its own suite. + * ORE comparison functions. Only the btree OPCLASS + * (`eql_v3_internal.ore_block_256_operator_class`) is skipped. + * + * What that does and does not break: + * + * - RANGE FILTERS STILL WORK. The `<`/`>`/`<=`/`>=` operators on + * `ore_block_256` are backed by `ore_block_256_lt` → the ORE comparator, and + * operators need no opclass. `eql_v3.gt`/`lt`/`gte`/`lte` compare in true ORE + * order on every provider. + * - `ORDER BY eql_v3.ord_term_ore(col)` DOES NOT ERROR — it sorts, wrongly. + * `ore_block_256` is a COMPOSITE type, so with no opclass to resolve, Postgres + * falls back to its built-in record comparison, which walks down to the raw + * `bytes` field and compares bytewise. Bytewise order over ORE ciphertext is + * deterministic and stable but uncorrelated with plaintext order: measured + * over 200 random well-formed terms, it disagreed with the ORE comparator on + * 87 of them. A stable, plausible-looking, silently wrong ordering. + * - A btree INDEX on such a column cannot be created at all. + * + * A matrix whose ordering assertions must hold on every provider cannot straddle + * that, so ORE gets its own suite. * * The OPE-backed `_ord` domains have no such split: they order via a native * `bytea` btree and behave identically everywhere. See From bd63c8ca14dfce1f06ca85d3768183e7fb35daad Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 10 Jul 2026 22:46:42 +1000 Subject: [PATCH 06/20] test(integration): supabase v3 driver + adapter, integer family on real crypto MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First real-crypto coverage of the Supabase v3 adapter. 62 tests against real ZeroKMS ciphertext, a real PostgREST, and supabase/postgres. The driver (`@cipherstash/test-kit/suite`) derives everything from the domain's catalog row: which operations must work, which must be rejected, and what rows each should return, computed from a plaintext oracle. A family file is three lines. The adapter is built through the public `encryptedSupabaseV3` factory so the shipped construction path runs — introspection over `pg`, declared-schema verification, a real `Encryption({eqlVersion: 3})` client. Both the `in()` method and the raw `filter(col, 'in', […])` path are driven against the same oracle, because they are different code paths. Rows are split into disjoint interleaved halves — single-encrypt {a,c}, bulk-encrypt {b,d} — so any predicate matching more than one row necessarily spans both, and an explicit crossover assertion pins it. Verified the suite is worth having, rather than assuming it: - Reverting `src/supabase/` to the base commit fails exactly four tests, all of them `filter(in)` — the raw-filter bug this PR fixes, caught end to end. - Removing `equality` from `integer_eq`'s catalog row flips its four positive tests into rejection tests, which then fail because the column really does support equality. The rejection matrix is derived, not decorative. Two integration hazards found by running it: - `@cipherstash/test-kit`'s barrel must not import `vitest`, even transitively. `globalSetup` runs in a different context from the test workers and imports that barrel, so a `vitest` reached from there dies with "Vitest failed to access its internal state". The driver therefore lives behind its own subpath. - The kit resolves to source in another package, outside this config's root, so Vitest externalizes it and loads it through Node rather than the transform pipeline. It must be listed in `server.deps.inline`. --- .../stack/integration/supabase/adapter.ts | 215 ++++++++++++++ .../supabase/integer.integration.test.ts | 4 + packages/stack/integration/vitest.config.ts | 10 + packages/stack/tsconfig.json | 1 + packages/test-kit/package.json | 39 +-- packages/test-kit/src/index.ts | 6 + packages/test-kit/src/run-family-suite.ts | 281 ++++++++++++++++++ vitest.shared.ts | 5 + 8 files changed, 542 insertions(+), 19 deletions(-) create mode 100644 packages/stack/integration/supabase/adapter.ts create mode 100644 packages/stack/integration/supabase/integer.integration.test.ts create mode 100644 packages/test-kit/src/run-family-suite.ts diff --git a/packages/stack/integration/supabase/adapter.ts b/packages/stack/integration/supabase/adapter.ts new file mode 100644 index 000000000..5ef160089 --- /dev/null +++ b/packages/stack/integration/supabase/adapter.ts @@ -0,0 +1,215 @@ +import { + databaseUrl, + type IntegrationAdapter, + type PlainRow, + type QueryOp, + type QueryOpKind, + type TableSpec, +} from '@cipherstash/test-kit' +import postgres from 'postgres' +import { encryptedTable } from '@/eql/v3' +import { encryptedSupabaseV3 } from '@/supabase' +import { + makePostgrestClient, + reloadSchemaCache, +} from '../../__tests__/helpers/pgrest' + +/** + * The Supabase v3 adapter under real ZeroKMS ciphertext. + * + * Built through the PUBLIC `encryptedSupabaseV3` factory, so the shipped + * construction path runs: introspection over `pg`, declared-schema verification + * against the live domains, and a real `Encryption({ eqlVersion: 3 })` client. + * A hand-assembled builder would skip all three. + * + * The table therefore has to exist before the instance is constructed — + * introspection reads it — which is why `createTable` also builds the client. + */ + +/** PostgREST cannot emit `ORDER BY eql_v3.ord_term(col)`, and a bare `ORDER BY` */ +/** sorts the raw ciphertext envelope. The adapter refuses ordering outright. */ +const SUPPORTED_OPS: ReadonlySet = new Set([ + 'eq', + 'ne', + 'in', + 'notIn', + 'gt', + 'gte', + 'lt', + 'lte', + 'between', + 'notBetween', + 'contains', + 'isNull', + 'isNotNull', + 'order', +]) + +/** + * `order` is refused on EVERY encrypted column, whatever its capabilities. That + * is the half of the contract the capability matrix cannot derive — an + * ORE-capable column still cannot be ordered through PostgREST — so it is + * declared here and pinned by a test on every domain. + */ +const ALWAYS_REJECTED: ReadonlySet = new Set(['order']) + +type Instance = Awaited> + +// biome-ignore lint/suspicious/noExplicitAny: the builder is generic over a row type the driver does not know +type Query = any + +export function makeSupabaseAdapter(): IntegrationAdapter { + let sql: postgres.Sql + let instance: Instance + let tableName: string + + const from = (): Query => instance.from(tableName) + + /** Apply `op` to a `select('row_key')` query. May throw synchronously — the + * capability and ordering guards fire at call time, not on execute. */ + const applyOp = (op: QueryOp): Query => { + const q = from().select('row_key') + switch (op.kind) { + case 'eq': + return q.eq(op.column, op.value) + case 'ne': + return q.neq(op.column, op.value) + case 'gt': + return q.gt(op.column, op.value) + case 'gte': + return q.gte(op.column, op.value) + case 'lt': + return q.lt(op.column, op.value) + case 'lte': + return q.lte(op.column, op.value) + case 'in': + // Two distinct code paths: `.in()` and the raw `.filter(col, 'in', […])`. + // The raw one encrypted the whole list as a single term until recently, + // so both must be proven against the same oracle. + return op.asRawFilter + ? q.filter(op.column, 'in', [...op.values]) + : q.in(op.column, [...op.values]) + case 'notIn': + return q.not(op.column, 'in', [...op.values]) + case 'between': + // The builder exposes no `between`; it is `gte AND lte` by definition. + return q.gte(op.column, op.lo).lte(op.column, op.hi) + case 'notBetween': + // Structured `.or()`, not an or-STRING: a string operand would have to + // stringify Dates and bigints, and the parser would then have to guess + // them back. The structured form carries the plaintext through intact. + return q.or([ + { column: op.column, op: 'lt', value: op.lo }, + { column: op.column, op: 'gt', value: op.hi }, + ]) + case 'contains': + return q.contains(op.column, op.needle) + case 'isNull': + return q.is(op.column, null) + case 'isNotNull': + return q.not(op.column, 'is', null) + case 'order': + return q.order(op.column) + } + } + + return { + name: 'supabase', + supportedOps: SUPPORTED_OPS, + alwaysRejectedOps: ALWAYS_REJECTED, + + async setup() { + sql = postgres(databaseUrl(), { prepare: false }) + }, + + async teardown() { + if (tableName) await sql.unsafe(`DROP TABLE IF EXISTS ${tableName}`) + await sql.end() + }, + + async createTable(table: TableSpec) { + tableName = table.name + + const columns = table.columns + .map((column) => `"${column.slug}" ${column.eqlType}`) + .join(',\n ') + + await sql.unsafe(`DROP TABLE IF EXISTS ${table.name}`) + await sql.unsafe(` + CREATE TABLE ${table.name} ( + row_key TEXT PRIMARY KEY, + ${columns} + ) + `) + // The EQL grants cover `eql_v3` objects, not application tables. PostgREST + // runs as `anon`; without this every query is a 401. + await sql.unsafe( + `GRANT SELECT, INSERT, UPDATE, DELETE ON ${table.name} TO anon, authenticated`, + ) + await reloadSchemaCache(sql, table.name) + + const schema = encryptedTable( + table.name, + Object.fromEntries( + table.columns.map((column) => [ + column.slug, + column.spec.builder(column.slug), + ]), + ) as never, + ) + + // `schemas` is a Record keyed by table name, not an array — the factory + // rejects a key that disagrees with the table's own name. + instance = await encryptedSupabaseV3(makePostgrestClient(), { + schemas: { [table.name]: schema } as never, + databaseUrl: databaseUrl(), + }) + }, + + async insertSingle(_table: TableSpec, row: PlainRow) { + // A single object routes through `encryptModel`; an array routes through + // `bulkEncryptModels`. Both must be exercised. + const { error } = await from().insert({ + row_key: row.rowKey, + ...row.values, + }) + if (error) + throw new Error(`insertSingle(${row.rowKey}): ${error.message}`) + }, + + async insertBulk(_table: TableSpec, rows: readonly PlainRow[]) { + const models = rows.map((row) => ({ row_key: row.rowKey, ...row.values })) + const { error } = await from().insert(models) + if (error) throw new Error(`insertBulk: ${error.message}`) + }, + + async run(_table: TableSpec, op: QueryOp): Promise { + const { data, error } = await applyOp(op) + if (error) throw new Error(`${op.kind}(${op.column}): ${error.message}`) + return (data as { row_key: string }[]).map((row) => row.row_key) + }, + + async expectRejected(_table: TableSpec, op: QueryOp) { + // The guards fire in two places: capability checks throw synchronously at + // call time, while a term that reaches `execute` surfaces as a Result + // error. Accept either; the point is that the query never runs. + try { + const { error } = await applyOp(op) + if (!error) { + throw new Error( + `Expected ${op.kind}("${op.column}") to be rejected, but it succeeded`, + ) + } + } catch (cause) { + if ( + cause instanceof Error && + cause.message.startsWith('Expected ') && + cause.message.includes('to be rejected') + ) { + throw cause + } + // A thrown guard is a rejection. + } + }, + } +} diff --git a/packages/stack/integration/supabase/integer.integration.test.ts b/packages/stack/integration/supabase/integer.integration.test.ts new file mode 100644 index 000000000..a1463710a --- /dev/null +++ b/packages/stack/integration/supabase/integer.integration.test.ts @@ -0,0 +1,4 @@ +import { runFamilySuite } from '@cipherstash/test-kit/suite' +import { makeSupabaseAdapter } from './adapter' + +runFamilySuite('integer', makeSupabaseAdapter) diff --git a/packages/stack/integration/vitest.config.ts b/packages/stack/integration/vitest.config.ts index ab87d7e14..6f31ce677 100644 --- a/packages/stack/integration/vitest.config.ts +++ b/packages/stack/integration/vitest.config.ts @@ -36,6 +36,16 @@ export default defineConfig({ root: resolve(__dirname, '..'), include: [SUITE_GLOB], globalSetup: [resolve(__dirname, 'global-setup.ts')], + server: { + deps: { + // `@cipherstash/test-kit` resolves to source in ANOTHER package, i.e. + // outside this config's root, so Vitest externalizes it and loads it + // through Node rather than the transform pipeline. Its driver imports + // `vitest`, and a `vitest` imported outside a worker cannot reach the + // runner's state: "Vitest failed to access its internal state". + inline: [/packages\/test-kit/], + }, + }, // Real crypto round-trips over the network. The unit config uses 30s for the // same reason; seeding a family table encrypts every sample, so hooks need // more headroom than tests do. diff --git a/packages/stack/tsconfig.json b/packages/stack/tsconfig.json index b2627eb5a..122bb3e96 100644 --- a/packages/stack/tsconfig.json +++ b/packages/stack/tsconfig.json @@ -44,6 +44,7 @@ // `.test-d.ts` matrix collapses to `never`. Mirrors `vitest.shared.ts` // and `packages/test-kit/tsconfig.json`; keep the three in step. "@cipherstash/test-kit": ["../test-kit/src/index.ts"], + "@cipherstash/test-kit/suite": ["../test-kit/src/run-family-suite.ts"], "@cipherstash/test-kit/catalog": ["../test-kit/src/catalog.ts"], "@cipherstash/stack": ["./src/index.ts"], "@cipherstash/stack/eql/v3": ["./src/eql/v3/index.ts"], diff --git a/packages/test-kit/package.json b/packages/test-kit/package.json index e8e465feb..9709ecc7d 100644 --- a/packages/test-kit/package.json +++ b/packages/test-kit/package.json @@ -1,21 +1,22 @@ { - "name": "@cipherstash/test-kit", - "version": "0.0.0", - "private": true, - "description": "Shared EQL v3 test harness: the domain catalog, the plaintext oracle, and the integration-suite driver. Consumed as TypeScript source — no build step.", - "type": "module", - "exports": { - ".": "./src/index.ts", - "./catalog": "./src/catalog.ts" - }, - "scripts": { - "test:types": "tsc --noEmit -p tsconfig.json" - }, - "dependencies": { - "@cipherstash/stack": "workspace:*" - }, - "devDependencies": { - "typescript": "catalog:repo", - "vitest": "catalog:repo" - } + "name": "@cipherstash/test-kit", + "version": "0.0.0", + "private": true, + "description": "Shared EQL v3 test harness: the domain catalog, the plaintext oracle, and the integration-suite driver. Consumed as TypeScript source \u2014 no build step.", + "type": "module", + "exports": { + ".": "./src/index.ts", + "./catalog": "./src/catalog.ts", + "./suite": "./src/run-family-suite.ts" + }, + "scripts": { + "test:types": "tsc --noEmit -p tsconfig.json" + }, + "dependencies": { + "@cipherstash/stack": "workspace:*" + }, + "devDependencies": { + "typescript": "catalog:repo", + "vitest": "catalog:repo" + } } diff --git a/packages/test-kit/src/index.ts b/packages/test-kit/src/index.ts index e9583aa26..96fb520f5 100644 --- a/packages/test-kit/src/index.ts +++ b/packages/test-kit/src/index.ts @@ -9,6 +9,12 @@ * * Consumed as TypeScript source (no build). See `vitest.shared.ts` at the repo * root for the runtime aliases and `tsconfig.json` here for the compile-time ones. + * + * This barrel MUST NOT import `vitest`, directly or transitively. Vitest's + * `globalSetup` runs in a different context from the test workers, and a + * `vitest` import reached from there fails with "Vitest failed to access its + * internal state". `global-setup.ts` imports this barrel, so the test driver + * lives behind its own subpath: `@cipherstash/test-kit/suite`. */ export type { IntegrationAdapter } from './adapter.ts' diff --git a/packages/test-kit/src/run-family-suite.ts b/packages/test-kit/src/run-family-suite.ts new file mode 100644 index 000000000..e1010769a --- /dev/null +++ b/packages/test-kit/src/run-family-suite.ts @@ -0,0 +1,281 @@ +import { afterAll, beforeAll, describe, expect, it } from 'vitest' +import type { IntegrationAdapter } from './adapter.ts' +import type { DomainSpec } from './catalog.ts' +import { + deferredForFamily, + domainsForFamily, + type FamilyDomain, + type FamilyName, +} from './families.ts' +import { negativeOps, type Plain, positiveOps, type QueryOp } from './ops.ts' +import { + comparePlain, + containsPlain, + expectedKeysFor, + plainValue, +} from './oracle.ts' +import { planRows, planTable } from './rows.ts' + +/** + * The shared driver. One family file per adapter is three lines; everything a + * test asserts is derived from the domain's catalog row, so Drizzle and Supabase + * cannot quietly cover different operations while both read as "comprehensive". + * + * What is asserted, per domain: + * - every operation the domain's capabilities allow, over a RANGE of values, + * against a plaintext oracle; + * - every operation they forbid, rejected; + * - that both the single-encrypt and bulk-encrypt insert paths produced + * queryable ciphertext. + */ + +/** Distinct sample values for a domain, in ascending order, deduplicated by `comparePlain`. */ +function distinctValues(spec: DomainSpec, rowKeys: readonly string[]): Plain[] { + const values = rowKeys.map((key) => plainValue(spec, rowKeys, key)) + const sorted = [...values].sort(comparePlain) + return sorted.filter( + (value, i) => i === 0 || comparePlain(value, sorted[i - 1] as Plain) !== 0, + ) +} + +/** + * A needle guaranteed to be answerable: the first three characters of the + * value, downcased. `token_length` is 3, so a shorter needle blooms to nothing + * and the adapters reject it — see `requireAnswerableNeedle`. + */ +function needleFor(value: Plain): string | null { + const text = String(value) + return text.length >= 3 ? text.slice(0, 3).toLowerCase() : null +} + +/** A representative operand for a rejected operation — the value never reaches the DB. */ +function sampleOpFor( + kind: QueryOp['kind'], + column: string, + value: Plain, +): QueryOp { + switch (kind) { + case 'in': + case 'notIn': + return { kind, column, values: [value] } + case 'between': + case 'notBetween': + return { kind, column, lo: value, hi: value } + case 'contains': + return { kind, column, needle: needleFor(value) ?? 'abc' } + case 'order': + return { kind, column, direction: 'asc' } + case 'isNull': + case 'isNotNull': + return { kind, column } + default: + return { kind, column, value } + } +} + +export function runFamilySuite( + family: FamilyName, + makeAdapter: () => IntegrationAdapter, +): void { + const adapter = makeAdapter() + const domains = domainsForFamily(family) + const deferred = deferredForFamily(family) + + // Unique per run so a crashed run never leaves a table that shadows the next. + const runId = Math.random().toString(36).slice(2, 8) + const table = planTable(family, domains, runId) + const plan = planRows(domains, runId) + const { rowKeys, byKey } = plan + + describe(`v3 ${adapter.name} — ${family}`, () => { + beforeAll(async () => { + await adapter.setup() + await adapter.createTable(table) + + // Disjoint, interleaved halves: any predicate matching more than one row + // necessarily spans both encryption paths. + for (const key of plan.singleKeys) { + await adapter.insertSingle(table, byKey[key] as never) + } + await adapter.insertBulk( + table, + plan.bulkKeys.map((key) => byKey[key] as never), + ) + // Every encrypted column NULL. Goes through the single path; the bulk path + // is already proven by the value rows. + await adapter.insertSingle(table, plan.nullRow) + }, 300_000) + + afterAll(async () => { + await adapter.teardown() + }) + + if (deferred.length > 0) { + it.skip(`defers ${deferred.map((d) => d.bare).join(', ')}: ${deferred[0]?.reason}`, () => {}) + } + + for (const domain of domains) { + describe(domain.bare, () => { + const { slug, spec } = domain + const values = distinctValues(spec, rowKeys) + const min = values[0] as Plain + const max = values[values.length - 1] as Plain + + const positive = positiveOps(spec.capabilities, adapter.supportedOps) + const negative = negativeOps(spec.capabilities, adapter.supportedOps) + + const expectRows = async (op: QueryOp, expected: string[]) => { + const rows = await adapter.run(table, op) + expect([...rows].sort()).toEqual([...expected].sort()) + } + + const keysWhere = (predicate: (value: Plain) => boolean) => + expectedKeysFor(spec, rowKeys, predicate) + + if (positive.has('eq')) { + it.each(values)('eq(%s) selects exactly its rows', async (value) => { + await expectRows( + { kind: 'eq', column: slug, value }, + keysWhere((v) => comparePlain(v, value) === 0), + ) + }) + } + + if (positive.has('ne')) { + it('ne excludes exactly its rows', async () => { + await expectRows( + { kind: 'ne', column: slug, value: min }, + keysWhere((v) => comparePlain(v, min) !== 0), + ) + }) + } + + if (positive.has('in')) { + // Both the `in()` method and the raw `filter(col, 'in', [...])` path: + // they are different code paths, and the raw one encrypted the whole + // list as a single term until recently. + for (const asRawFilter of [false, true]) { + const label = asRawFilter ? 'filter(in)' : 'in()' + + it(`${label} selects the union of the listed values`, async () => { + const listed = values.slice(0, 2) + await expectRows( + { kind: 'in', column: slug, values: listed, asRawFilter }, + keysWhere((v) => listed.some((l) => comparePlain(v, l) === 0)), + ) + }) + + it(`${label} excludes rows whose value is absent from the list`, async () => { + // Guards against a predicate that matches everything: without this, + // a filter that ignored its operand would pass the test above. + await expectRows( + { kind: 'in', column: slug, values: [min], asRawFilter }, + keysWhere((v) => comparePlain(v, min) === 0), + ) + }) + } + } + + if (positive.has('notIn')) { + it('notIn excludes the listed values', async () => { + await expectRows( + { kind: 'notIn', column: slug, values: [min] }, + keysWhere((v) => comparePlain(v, min) !== 0), + ) + }) + } + + for (const kind of ['gt', 'gte', 'lt', 'lte'] as const) { + if (!positive.has(kind)) continue + it.each([ + min, + max, + ])(`${kind}(%s) matches the oracle`, async (bound) => { + const cmp = { + gt: (c: number) => c > 0, + gte: (c: number) => c >= 0, + lt: (c: number) => c < 0, + lte: (c: number) => c <= 0, + }[kind] + await expectRows( + { kind, column: slug, value: bound }, + keysWhere((v) => cmp(comparePlain(v, bound))), + ) + }) + } + + if (positive.has('between')) { + it('between(min, max) spans every row', async () => { + await expectRows( + { kind: 'between', column: slug, lo: min, hi: max }, + [...rowKeys], + ) + }) + + it('between(v, v) selects only that value', async () => { + await expectRows( + { kind: 'between', column: slug, lo: min, hi: min }, + keysWhere((v) => comparePlain(v, min) === 0), + ) + }) + } + + if (positive.has('notBetween')) { + it('notBetween(min, max) selects nothing', async () => { + await expectRows( + { kind: 'notBetween', column: slug, lo: min, hi: max }, + [], + ) + }) + } + + if (positive.has('contains')) { + const needle = needleFor(min) + it.runIf(needle !== null)( + 'contains matches the rows whose plaintext contains the needle', + async () => { + await expectRows( + { kind: 'contains', column: slug, needle: needle as string }, + keysWhere((v) => containsPlain(v, needle as string)), + ) + }, + ) + } + + // Structural, never capability-gated: a NULL plaintext is a SQL NULL on + // every domain, storage-only ones included. + it('isNull selects only the all-null row', async () => { + await expectRows({ kind: 'isNull', column: slug }, ['null']) + }) + + it('isNotNull selects every value row', async () => { + await expectRows({ kind: 'isNotNull', column: slug }, [...rowKeys]) + }) + + // Proves BOTH encryption paths produced queryable ciphertext. The eq + // loop above already spans them, but only implicitly; assert it. + it('both encrypt paths produced queryable ciphertext', async () => { + const rows = await adapter.run(table, { + kind: 'isNotNull', + column: slug, + }) + expect(rows).toEqual(expect.arrayContaining([...plan.singleKeys])) + expect(rows).toEqual(expect.arrayContaining([...plan.bulkKeys])) + }) + + for (const kind of negative) { + it(`rejects ${kind}: not supported by this domain's capabilities`, async () => { + await adapter.expectRejected(table, sampleOpFor(kind, slug, min)) + }) + } + + for (const kind of adapter.alwaysRejectedOps) { + if (negative.has(kind)) continue // already covered above + it(`rejects ${kind}: refused on every encrypted column by this adapter`, async () => { + await adapter.expectRejected(table, sampleOpFor(kind, slug, min)) + }) + } + }) + } + }) +} diff --git a/vitest.shared.ts b/vitest.shared.ts index 15147279a..525bc12f4 100644 --- a/vitest.shared.ts +++ b/vitest.shared.ts @@ -20,6 +20,11 @@ import { resolve } from 'node:path' const repoRoot = __dirname export const sharedAlias: Record = { + // Longest specifiers first: Vite alias keys are prefix-matched in order. + '@cipherstash/test-kit/suite': resolve( + repoRoot, + 'packages/test-kit/src/run-family-suite.ts', + ), '@cipherstash/test-kit/catalog': resolve( repoRoot, 'packages/test-kit/src/catalog.ts', From 476c1e368a194cb3ed18cc012ab9bef46de49f09 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 10 Jul 2026 22:54:55 +1000 Subject: [PATCH 07/20] test(integration): all nine supabase families + aliased Date reconstruction MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 609 tests across the nine plaintext families, on real ZeroKMS ciphertext against supabase/postgres + PostgREST. Every domain the SDK models except the nine deferred block-ORE ones, every operation its capabilities allow, every operation they forbid, both encrypt paths, against a plaintext oracle. `select-alias.integration.test.ts` covers what the family driver cannot express: `Date` reconstruction across all three ways PostgREST can key a row — the JS property, the raw DB column name, and a caller-chosen alias — plus `select('*')`. It is a projection concern, not a query operation. Proof the suite earns its keep. Against the base commit's `src/supabase/`: 41 failed | 568 passed - 38 × `filter(in)`, on every eq-capable domain in every family: the raw filter path encrypted the whole in-list as one ciphertext. - 3 × aliased `Date`: `selectKeyToDb` was dropped from `buildSelectString`. With the fixes restored: 609 passed, 0 failed. A silent skip found and removed. `contains` derived its needle from the domain's MINIMUM sample, which for text is the empty string — shorter than `token_length`, so `it.runIf` skipped it. `text_match` is the only match-only domain, and free-text search is its only capability: the suite was silently not testing the one thing that domain exists to do. The needle is now taken from the first sample long enough to build one, and a domain that cannot produce one throws instead of skipping. That is the same failure this whole PR exists to eliminate — a test that appears to cover something and does not — reproduced inside the harness meant to prevent it. Worth stating plainly rather than quietly fixing. --- .../supabase/bigint.integration.test.ts | 4 + .../supabase/boolean.integration.test.ts | 4 + .../supabase/date.integration.test.ts | 4 + .../supabase/numeric.integration.test.ts | 4 + .../supabase/real-double.integration.test.ts | 4 + .../supabase/select-alias.integration.test.ts | 128 ++++++++++++++++++ .../supabase/smallint.integration.test.ts | 4 + .../supabase/text.integration.test.ts | 4 + .../supabase/timestamp.integration.test.ts | 4 + packages/test-kit/src/run-family-suite.ts | 51 ++++--- 10 files changed, 194 insertions(+), 17 deletions(-) create mode 100644 packages/stack/integration/supabase/bigint.integration.test.ts create mode 100644 packages/stack/integration/supabase/boolean.integration.test.ts create mode 100644 packages/stack/integration/supabase/date.integration.test.ts create mode 100644 packages/stack/integration/supabase/numeric.integration.test.ts create mode 100644 packages/stack/integration/supabase/real-double.integration.test.ts create mode 100644 packages/stack/integration/supabase/select-alias.integration.test.ts create mode 100644 packages/stack/integration/supabase/smallint.integration.test.ts create mode 100644 packages/stack/integration/supabase/text.integration.test.ts create mode 100644 packages/stack/integration/supabase/timestamp.integration.test.ts diff --git a/packages/stack/integration/supabase/bigint.integration.test.ts b/packages/stack/integration/supabase/bigint.integration.test.ts new file mode 100644 index 000000000..b0bd74ae9 --- /dev/null +++ b/packages/stack/integration/supabase/bigint.integration.test.ts @@ -0,0 +1,4 @@ +import { runFamilySuite } from '@cipherstash/test-kit/suite' +import { makeSupabaseAdapter } from './adapter' + +runFamilySuite('bigint', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/boolean.integration.test.ts b/packages/stack/integration/supabase/boolean.integration.test.ts new file mode 100644 index 000000000..f12b0cbe2 --- /dev/null +++ b/packages/stack/integration/supabase/boolean.integration.test.ts @@ -0,0 +1,4 @@ +import { runFamilySuite } from '@cipherstash/test-kit/suite' +import { makeSupabaseAdapter } from './adapter' + +runFamilySuite('boolean', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/date.integration.test.ts b/packages/stack/integration/supabase/date.integration.test.ts new file mode 100644 index 000000000..971c4a141 --- /dev/null +++ b/packages/stack/integration/supabase/date.integration.test.ts @@ -0,0 +1,4 @@ +import { runFamilySuite } from '@cipherstash/test-kit/suite' +import { makeSupabaseAdapter } from './adapter' + +runFamilySuite('date', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/numeric.integration.test.ts b/packages/stack/integration/supabase/numeric.integration.test.ts new file mode 100644 index 000000000..914f03d3d --- /dev/null +++ b/packages/stack/integration/supabase/numeric.integration.test.ts @@ -0,0 +1,4 @@ +import { runFamilySuite } from '@cipherstash/test-kit/suite' +import { makeSupabaseAdapter } from './adapter' + +runFamilySuite('numeric', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/real-double.integration.test.ts b/packages/stack/integration/supabase/real-double.integration.test.ts new file mode 100644 index 000000000..1797e67f2 --- /dev/null +++ b/packages/stack/integration/supabase/real-double.integration.test.ts @@ -0,0 +1,4 @@ +import { runFamilySuite } from '@cipherstash/test-kit/suite' +import { makeSupabaseAdapter } from './adapter' + +runFamilySuite('real-double', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/select-alias.integration.test.ts b/packages/stack/integration/supabase/select-alias.integration.test.ts new file mode 100644 index 000000000..3fcedb516 --- /dev/null +++ b/packages/stack/integration/supabase/select-alias.integration.test.ts @@ -0,0 +1,128 @@ +import { databaseUrl } from '@cipherstash/test-kit' +import postgres from 'postgres' +import { afterAll, beforeAll, expect, it } from 'vitest' +import { encryptedTable, types } from '@/eql/v3' +import { encryptedSupabaseV3 } from '@/supabase' +import { + makePostgrestClient, + reloadSchemaCache, +} from '../../__tests__/helpers/pgrest' + +/** + * `Date` reconstruction across every way PostgREST can key a result row. + * + * Not expressible through the family driver: it is about the SELECT string, not + * about a query operation, and only the Supabase adapter has the concept. The + * driver proves the operations; this proves the projection. + * + * A decrypted date-like column arrives as an ISO string, and the adapter rebuilds + * the `Date` by looking up the column's `cast_as`. That lookup is keyed by DB + * column name, so it needs a map from the ROW KEY back to the DB column — and the + * row key is whatever PostgREST chose, which is one of three things: + * + * .select('createdAt') -> keyed `createdAt` (the JS property; renamed) + * .select('created_at') -> keyed `created_at` (the raw DB name) + * .select('ts:createdAt') -> keyed `ts` (a caller-chosen alias) + * + * The third case regressed: `selectKeyToDb` was dropped from `buildSelectString`, + * so an aliased date column came back as a string where the typed surface + * promises a `Date`. + */ +const TABLE = `v3_it_alias_${Math.random().toString(36).slice(2, 8)}` +const CREATED = new Date('2026-01-02T03:04:05.000Z') + +// `createdAt` -> `created_at` is the rename the aliasing `prop:db_name::jsonb` +// select exists for; a table whose property equals its DB name cannot express it. +const rows = encryptedTable(TABLE, { + createdAt: types.Timestamp('created_at'), + bornOn: types.Date('born_on'), +}) + +const sql = postgres(databaseUrl(), { prepare: false }) +// biome-ignore lint/suspicious/noExplicitAny: the row type varies per select +let instance: any + +beforeAll(async () => { + await sql.unsafe(`DROP TABLE IF EXISTS ${TABLE}`) + await sql.unsafe(` + CREATE TABLE ${TABLE} ( + row_key TEXT PRIMARY KEY, + created_at public.eql_v3_timestamp, + born_on public.eql_v3_date + ) + `) + await sql.unsafe( + `GRANT SELECT, INSERT, UPDATE, DELETE ON ${TABLE} TO anon, authenticated`, + ) + await reloadSchemaCache(sql, TABLE) + + instance = await encryptedSupabaseV3(makePostgrestClient(), { + schemas: { [TABLE]: rows } as never, + databaseUrl: databaseUrl(), + }) + + const { error } = await instance + .from(TABLE) + .insert({ row_key: 'a', createdAt: CREATED, bornOn: CREATED }) + if (error) throw new Error(`insert: ${error.message}`) +}, 300_000) + +afterAll(async () => { + await sql.unsafe(`DROP TABLE IF EXISTS ${TABLE}`) + await sql.end() +}) + +it('reconstructs a Date keyed by the JS property name', async () => { + const { data, error } = await instance + .from(TABLE) + .select('row_key, createdAt') + + expect(error).toBeNull() + expect(data[0].createdAt).toBeInstanceOf(Date) + expect((data[0].createdAt as Date).toISOString()).toBe(CREATED.toISOString()) +}) + +it('reconstructs a Date keyed by the raw DB column name', async () => { + const { data, error } = await instance + .from(TABLE) + .select('row_key, created_at') + + expect(error).toBeNull() + expect(data[0].created_at).toBeInstanceOf(Date) +}) + +it('reconstructs a Date keyed by a caller-chosen alias', async () => { + const { data, error } = await instance + .from(TABLE) + .select('row_key, ts:createdAt') + + expect(error).toBeNull() + expect(data[0].ts).toBeInstanceOf(Date) + expect((data[0].ts as Date).toISOString()).toBe(CREATED.toISOString()) +}) + +it('reconstructs a Date under an alias of the raw DB column name', async () => { + const { data, error } = await instance + .from(TABLE) + .select('row_key, at:created_at') + + expect(error).toBeNull() + expect(data[0].at).toBeInstanceOf(Date) +}) + +// `date` and `timestamp` are separate `cast_as` values; both are date-like, and +// both must be reconstructed under an alias. +it('reconstructs an aliased date column, not just timestamp', async () => { + const { data, error } = await instance.from(TABLE).select('row_key, d:bornOn') + + expect(error).toBeNull() + expect(data[0].d).toBeInstanceOf(Date) +}) + +it('expands select(*) with every date column reconstructed', async () => { + const { data, error } = await instance.from(TABLE).select('*') + + expect(error).toBeNull() + expect(data[0].createdAt).toBeInstanceOf(Date) + expect(data[0].bornOn).toBeInstanceOf(Date) +}) diff --git a/packages/stack/integration/supabase/smallint.integration.test.ts b/packages/stack/integration/supabase/smallint.integration.test.ts new file mode 100644 index 000000000..38eaa23bc --- /dev/null +++ b/packages/stack/integration/supabase/smallint.integration.test.ts @@ -0,0 +1,4 @@ +import { runFamilySuite } from '@cipherstash/test-kit/suite' +import { makeSupabaseAdapter } from './adapter' + +runFamilySuite('smallint', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/text.integration.test.ts b/packages/stack/integration/supabase/text.integration.test.ts new file mode 100644 index 000000000..d25741d39 --- /dev/null +++ b/packages/stack/integration/supabase/text.integration.test.ts @@ -0,0 +1,4 @@ +import { runFamilySuite } from '@cipherstash/test-kit/suite' +import { makeSupabaseAdapter } from './adapter' + +runFamilySuite('text', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/timestamp.integration.test.ts b/packages/stack/integration/supabase/timestamp.integration.test.ts new file mode 100644 index 000000000..33dbf0ce2 --- /dev/null +++ b/packages/stack/integration/supabase/timestamp.integration.test.ts @@ -0,0 +1,4 @@ +import { runFamilySuite } from '@cipherstash/test-kit/suite' +import { makeSupabaseAdapter } from './adapter' + +runFamilySuite('timestamp', makeSupabaseAdapter) diff --git a/packages/test-kit/src/run-family-suite.ts b/packages/test-kit/src/run-family-suite.ts index e1010769a..91c6eae8b 100644 --- a/packages/test-kit/src/run-family-suite.ts +++ b/packages/test-kit/src/run-family-suite.ts @@ -39,13 +39,26 @@ function distinctValues(spec: DomainSpec, rowKeys: readonly string[]): Plain[] { } /** - * A needle guaranteed to be answerable: the first three characters of the - * value, downcased. `token_length` is 3, so a shorter needle blooms to nothing - * and the adapters reject it — see `requireAnswerableNeedle`. + * A needle guaranteed to be answerable: the first three characters of the first + * sample long enough to produce one, downcased. `token_length` is 3, so a + * shorter needle blooms to nothing and the adapters reject it — see + * `requireAnswerableNeedle`. + * + * THROWS rather than skipping when no sample qualifies. It previously derived + * the needle from the domain's MINIMUM value, which for the text samples is the + * empty string — so `text_match`, the only match-only domain, silently skipped + * the only test that exercises its one capability. A `freeTextSearch` domain + * whose catalog row cannot produce a needle is a catalog bug, and must be loud. */ -function needleFor(value: Plain): string | null { - const text = String(value) - return text.length >= 3 ? text.slice(0, 3).toLowerCase() : null +function needleFrom(values: readonly Plain[]): string { + for (const value of values) { + const text = String(value) + if (text.length >= 3) return text.slice(0, 3).toLowerCase() + } + throw new Error( + 'No sample is long enough to build an answerable needle (>= token_length 3). ' + + 'A freeTextSearch domain must carry at least one such sample in the catalog.', + ) } /** A representative operand for a rejected operation — the value never reaches the DB. */ @@ -62,7 +75,14 @@ function sampleOpFor( case 'notBetween': return { kind, column, lo: value, hi: value } case 'contains': - return { kind, column, needle: needleFor(value) ?? 'abc' } + return { + kind, + column, + needle: + String(value).length >= 3 + ? String(value).slice(0, 3).toLowerCase() + : 'abc', + } case 'order': return { kind, column, direction: 'asc' } case 'isNull': @@ -230,16 +250,13 @@ export function runFamilySuite( } if (positive.has('contains')) { - const needle = needleFor(min) - it.runIf(needle !== null)( - 'contains matches the rows whose plaintext contains the needle', - async () => { - await expectRows( - { kind: 'contains', column: slug, needle: needle as string }, - keysWhere((v) => containsPlain(v, needle as string)), - ) - }, - ) + const needle = needleFrom(values) + it('contains matches the rows whose plaintext contains the needle', async () => { + await expectRows( + { kind: 'contains', column: slug, needle }, + keysWhere((v) => containsPlain(v, needle)), + ) + }) } // Structural, never capability-gated: a NULL plaintext is a SQL NULL on From 48618a4d4ab88b9a3a712716f3deed562fc96a79 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 10 Jul 2026 23:07:41 +1000 Subject: [PATCH 08/20] test(integration): cover the include_original pathological case (#615) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A single 3-character needle was insufficient coverage for free-text search, and the reason is subtle: its whole-value token IS a trigram, so it matches whether or not `include_original` is honoured. That is the degenerate case, and it was the only case the suite exercised. The match index blooms downcased 3-grams; `include_original: true` additionally blooms the whole value as one token. Storage wants that. A QUERY operand must not have it, or the needle's bloom carries a whole-needle token the haystack's bloom cannot contain, and every strict-substring search returns zero rows. Both v3 adapters build match operands with `encrypt` — the full storage envelope — rather than `encryptQuery`, because PostgREST cannot cast a filter value to `eql_v3.query_*`. Both are therefore structurally exposed. It is masked today only because protect-ffi ignores `include_original` outright, so neither side carries the token: two bugs cancel. When protect-ffi is fixed, substring search breaks silently. Two layers of coverage: - The text family now searches four needles per match-capable domain: one trigram (degenerate), a strict interior substring longer than a trigram (the discriminating case), the whole stored value, and a needle absent from every value. - `match-bloom.integration.test.ts` asserts the precondition directly on the wire — bloom(needle) must be a subset of bloom(haystack) for every substring — so a failure names its cause instead of surfacing as "a query returned no rows". It also proves its own sensitivity: the honest operand is contained, and unioning in the bits of one value the haystack never saw is enough to break the subset. An `include_original` token is exactly such a foreign token. These go red the day protect-ffi honours the flag. That is the intent: the fix is to stop the operand carrying the original token, not to relax the tests. --- .../match-bloom.integration.test.ts | 117 ++++++++++++++++++ packages/test-kit/src/run-family-suite.ts | 87 ++++++++++--- 2 files changed, 185 insertions(+), 19 deletions(-) create mode 100644 packages/stack/integration/match-bloom.integration.test.ts diff --git a/packages/stack/integration/match-bloom.integration.test.ts b/packages/stack/integration/match-bloom.integration.test.ts new file mode 100644 index 000000000..563b3cec6 --- /dev/null +++ b/packages/stack/integration/match-bloom.integration.test.ts @@ -0,0 +1,117 @@ +import { expect, it } from 'vitest' +import { EncryptionV3 } from '@/encryption/v3' +import { encryptedTable, types } from '@/eql/v3' + +/** + * The invariant `contains()` rests on, asserted directly on the wire rather than + * inferred from a query result. + * + * `eql_v3.contains(a, b)` is `match_term(a) @> match_term(b)`: bloom-filter + * (`smallint[]`) containment. So a substring search can only work if + * + * bloom(needle) ⊆ bloom(haystack) + * + * The match index tokenizes into downcased 3-grams. With `include_original: true` + * the WHOLE value is bloomed as an extra token. That is right for STORAGE and + * wrong for a QUERY operand: a whole-needle token is, by definition, not a + * trigram of the haystack, so the subset relation breaks and every strict + * substring search returns zero rows — silently. + * + * Both v3 adapters build match operands with `encrypt` (the full storage + * envelope), not `encryptQuery`, because PostgREST cannot cast a filter value to + * `eql_v3.query_*`. They are therefore structurally exposed. Today it is masked: + * protect-ffi ignores `include_original` altogether, so neither side carries the + * whole-value token and two bugs cancel out. See cipherstash/stack#615. + * + * WHEN protect-ffi STARTS HONOURING `include_original`, this file goes red — and + * that is the intent. The subset assertion below is the precondition; the query + * tests in the text family are the symptom. Fixing it means the operand must + * stop carrying the original token (via `encryptQuery`, or by disabling the flag + * on the query path) — not relaxing this test. + */ +const docs = encryptedTable('match_bloom_probe', { + bio: types.TextMatch('bio'), +}) + +const HAYSTACK = 'ada@example.com' + +async function bloomOf( + client: Awaited>, + value: string, +) { + const result = await client.encrypt(value, { column: docs.bio, table: docs }) + if (result.failure) throw new Error(result.failure.message) + const envelope = result.data as { bf?: number[] } + if (!Array.isArray(envelope.bf)) { + throw new Error( + `Expected a bloom filter (bf) on the envelope for "${value}"`, + ) + } + return new Set(envelope.bf) +} + +const isSubset = (needle: Set, haystack: Set) => + [...needle].every((bit) => haystack.has(bit)) + +it.each([ + // Degenerate: the whole needle IS a trigram, so it holds under either + // behaviour. Passing this proves nothing about `include_original`. + { label: '3-character needle (one trigram)', needle: 'ada' }, + // The discriminating cases: strict substrings longer than one trigram. These + // are what break the moment a whole-needle token enters the operand. + { label: 'interior substring, 7 chars', needle: 'a@examp' }, + { label: 'trailing substring, 4 chars', needle: '.com' }, + { label: 'the whole value', needle: HAYSTACK }, +])('bloom(needle) is a subset of bloom(haystack) for a $label', async ({ + needle, +}) => { + const client = await EncryptionV3({ schemas: [docs] }) + const haystack = await bloomOf(client, HAYSTACK) + const probe = await bloomOf(client, needle) + + expect(HAYSTACK.includes(needle)).toBe(true) + expect( + isSubset(probe, haystack), + `bloom("${needle}") is not contained in bloom("${HAYSTACK}"). ` + + 'The operand carries a token the haystack lacks — most likely the ' + + '`include_original` whole-value token. See the file header.', + ).toBe(true) +}, 120_000) + +it('a needle absent from the haystack is NOT a bloom subset', async () => { + // Without this, an implementation that blooms every needle to the empty set + // would satisfy every assertion above. + const client = await EncryptionV3({ schemas: [docs] }) + const haystack = await bloomOf(client, HAYSTACK) + const probe = await bloomOf(client, 'qqqzzz') + + expect(isSubset(probe, haystack)).toBe(false) +}, 120_000) + +it('a needle blooms to a non-empty filter', async () => { + const client = await EncryptionV3({ schemas: [docs] }) + expect((await bloomOf(client, 'ada')).size).toBeGreaterThan(0) +}, 120_000) + +/** + * The subset assertions above are only worth having if they can FAIL. A whole- + * value `include_original` token is, from the bloom's point of view, just a + * token the haystack does not have — so simulate one by unioning in the bits of + * a value the haystack never saw, and confirm the subset relation collapses. + * + * This is a test of the test. It is the reason the substring cases can be + * trusted to catch `include_original` leaking into the query operand, rather + * than passing for some unrelated reason. + */ +it('the subset assertion detects a single foreign token in the operand', async () => { + const client = await EncryptionV3({ schemas: [docs] }) + const haystack = await bloomOf(client, HAYSTACK) + const substring = await bloomOf(client, 'a@examp') + const foreign = await bloomOf(client, 'qqqzzz') + + // The honest operand is contained... + expect(isSubset(substring, haystack)).toBe(true) + // ...and one extra token is enough to break it. + const polluted = new Set([...substring, ...foreign]) + expect(isSubset(polluted, haystack)).toBe(false) +}, 120_000) diff --git a/packages/test-kit/src/run-family-suite.ts b/packages/test-kit/src/run-family-suite.ts index 91c6eae8b..2a971c5a2 100644 --- a/packages/test-kit/src/run-family-suite.ts +++ b/packages/test-kit/src/run-family-suite.ts @@ -38,27 +38,75 @@ function distinctValues(spec: DomainSpec, rowKeys: readonly string[]): Plain[] { ) } +/** A needle to search for, and what it is meant to discriminate. */ +type Needle = Readonly<{ label: string; needle: string }> + /** - * A needle guaranteed to be answerable: the first three characters of the first - * sample long enough to produce one, downcased. `token_length` is 3, so a - * shorter needle blooms to nothing and the adapters reject it — see - * `requireAnswerableNeedle`. + * The needle set for a `freeTextSearch` domain. + * + * A single 3-character needle is NOT sufficient coverage, and the reason is + * subtle enough to be worth stating. + * + * The match index tokenizes into downcased 3-grams. `include_original: true` + * additionally blooms the WHOLE value as one token. Storage wants that; a QUERY + * operand must not have it, or the needle's bloom carries a whole-needle token + * that the haystack's bloom cannot contain, and every strict-substring search + * returns nothing. * - * THROWS rather than skipping when no sample qualifies. It previously derived - * the needle from the domain's MINIMUM value, which for the text samples is the - * empty string — so `text_match`, the only match-only domain, silently skipped - * the only test that exercises its one capability. A `freeTextSearch` domain - * whose catalog row cannot produce a needle is a catalog bug, and must be loud. + * Both v3 adapters build match operands with `encrypt` — the full storage + * envelope — not `encryptQuery`, because PostgREST cannot cast a filter value to + * `eql_v3.query_*`. So both are structurally exposed to exactly that. Today it + * is masked: protect-ffi ignores `include_original` outright, so neither the + * stored value nor the operand carries the whole-value token. Two bugs cancel. + * The day protect-ffi honours the flag, substring search breaks — silently, by + * returning no rows. See cipherstash/stack#615. + * + * A 3-character needle cannot see any of this: its whole-value token IS a + * trigram, so it matches either way. That is the degenerate case. The + * discriminating needle is a strict substring LONGER than `token_length`, taken + * from the interior so no prefix-specific shortcut can satisfy it. + * + * THROWS rather than skipping when no sample qualifies: a `freeTextSearch` + * domain whose catalog row cannot produce a needle is a catalog bug, and must be + * loud. (This previously derived the needle from the domain's MINIMUM value — + * the empty string for text — so `text_match`, the only match-only domain, + * silently skipped the only test of its only capability.) */ -function needleFrom(values: readonly Plain[]): string { - for (const value of values) { - const text = String(value) - if (text.length >= 3) return text.slice(0, 3).toLowerCase() +function needlesFrom(values: readonly Plain[]): Needle[] { + const longest = [...values] + .map(String) + .sort((left, right) => right.length - left.length)[0] + + if (longest === undefined || longest.length < 3) { + throw new Error( + 'No sample is long enough to build an answerable needle (>= token_length 3). ' + + 'A freeTextSearch domain must carry at least one such sample in the catalog.', + ) } - throw new Error( - 'No sample is long enough to build an answerable needle (>= token_length 3). ' + - 'A freeTextSearch domain must carry at least one such sample in the catalog.', - ) + + const value = longest.toLowerCase() + const needles: Needle[] = [ + // Degenerate: exactly one trigram. Matches whether or not `include_original` + // is honoured, because the whole needle IS a trigram. + { label: 'a 3-character needle (one trigram)', needle: value.slice(0, 3) }, + // The whole stored value. Matches under either behaviour. + { label: 'the whole stored value', needle: value }, + // Absent from every sample, so it must never match. Guards against a + // predicate that ignores its operand — and against a bloom false positive + // being mistaken for coverage. + { label: 'a needle absent from every value', needle: 'qqqzzz' }, + ] + + // The discriminating case: a strict substring longer than one trigram, drawn + // from the interior. Fails the moment a whole-needle token enters the operand. + if (value.length >= 6) { + needles.splice(1, 0, { + label: 'a strict interior substring longer than one trigram', + needle: value.slice(2, Math.min(value.length - 1, 8)), + }) + } + + return needles } /** A representative operand for a rejected operation — the value never reaches the DB. */ @@ -250,8 +298,9 @@ export function runFamilySuite( } if (positive.has('contains')) { - const needle = needleFrom(values) - it('contains matches the rows whose plaintext contains the needle', async () => { + it.each(needlesFrom(values))('contains: $label', async ({ + needle, + }) => { await expectRows( { kind: 'contains', column: slug, needle }, keysWhere((v) => containsPlain(v, needle)), From 8f9df8b54a5bc1d4d396ebc5124d9e9044213e0b Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 10 Jul 2026 23:14:50 +1000 Subject: [PATCH 09/20] test(integration): port the wire suite, drop its gate, wire up CI MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes the Supabase half of PR1. 655 tests now run in one job against real ZeroKMS ciphertext, a real PostgREST, and supabase/postgres. - `supabase-v3-pgrest-live.test.ts` moves to `integration/supabase/wire. integration.test.ts`, de-gated. It is NOT superseded by the real-crypto matrix: it uniquely proves the 23514 rejection of a narrowed encryptQuery term (a shape the adapter cannot produce, so it must be hand-built), dense PostgREST parse edges, plaintext-passthrough containment, and the grants as `anon`. Its stubbed ZeroKMS is the point — it needs no credentials. Its `installEqlV3IfNeeded` + `SUPABASE_PERMISSIONS_SQL_V3` calls are gone: `globalSetup` installs both by running the real `stash eql install --eql-version 3 --supabase --direct`. Re-applying them here would only have tested a hand-rolled approximation of the installer. - `LIVE_SUPABASE_PGREST_ENABLED` / `describeLiveSupabasePgrest` are deleted along with the `live-coverage-guard` assertion that policed them. The guard existed because a false gate meant a silent whole-suite skip on a green job; a suite that throws needs no such guard. The remaining `LIVE_*` gates and their assertions stay — their suites have not moved. - `tests.yml` loses the PostgREST step and `PGRST_URL`. The moved suite was its only consumer, so the unit job no longer starts a container it does not need. - `integration-supabase.yml` runs the suites on `supabase/postgres` + PostgREST, fork-PR gated, secrets passed as job env rather than written to a `.env`. `CS_IT_SUITE` scopes the run to the suites this database serves, so the Drizzle suites (plain Postgres, no PostgREST) can have their own job without either provisioning the other's dependencies. Verified: 655 pass / 8 skipped under the exact CI invocation; `pnpm test` still runs 1554 unit tests and zero integration tests; both typechecks clean. --- .github/workflows/integration-supabase.yml | 99 +++++++++++++++++++ .github/workflows/tests.yml | 29 ------ packages/stack/__tests__/helpers/live-gate.ts | 26 +---- .../__tests__/live-coverage-guard.test.ts | 21 ---- .../helpers/pgrest.ts | 4 +- .../stack/integration/supabase/adapter.ts | 5 +- .../supabase/select-alias.integration.test.ts | 5 +- .../supabase/wire.integration.test.ts} | 31 +++--- packages/stack/integration/vitest.config.ts | 17 +++- 9 files changed, 133 insertions(+), 104 deletions(-) create mode 100644 .github/workflows/integration-supabase.yml rename packages/stack/{__tests__ => integration}/helpers/pgrest.ts (93%) rename packages/stack/{__tests__/supabase-v3-pgrest-live.test.ts => integration/supabase/wire.integration.test.ts} (95%) diff --git a/.github/workflows/integration-supabase.yml b/.github/workflows/integration-supabase.yml new file mode 100644 index 000000000..4c8bbeabf --- /dev/null +++ b/.github/workflows/integration-supabase.yml @@ -0,0 +1,99 @@ +name: Integration — Supabase (EQL v3) + +# Real ZeroKMS ciphertext, a real PostgREST, and `supabase/postgres` — the only +# job that proves the Supabase v3 adapter's queries return the right rows. The +# unit suites drive a mock that records strings; they cannot. +# +# Separate from `tests.yml` on purpose: these suites need CipherStash credentials +# and a database, and they THROW rather than skip when unconfigured. Keeping them +# out of the unit job is what lets `pnpm test` stay runnable with neither. + +on: + push: + branches: [main] + paths: + - 'packages/stack/src/supabase/**' + - 'packages/stack/src/eql/v3/**' + - 'packages/stack/integration/**' + - 'packages/test-kit/**' + - 'packages/cli/src/installer/**' + - 'local/docker-compose.supabase.yml' + - 'local/supabase-init.sql' + - '.github/workflows/integration-supabase.yml' + - '.github/actions/integration-setup/**' + pull_request: + branches: ['**'] + paths: + - 'packages/stack/src/supabase/**' + - 'packages/stack/src/eql/v3/**' + - 'packages/stack/integration/**' + - 'packages/test-kit/**' + - 'packages/cli/src/installer/**' + - 'local/docker-compose.supabase.yml' + - 'local/supabase-init.sql' + - '.github/workflows/integration-supabase.yml' + - '.github/actions/integration-setup/**' + +jobs: + integration: + name: Supabase v3 integration (db=${{ matrix.db }}) + runs-on: blacksmith-4vcpu-ubuntu-2404 + # Fork PRs have no secrets. Skip cleanly rather than fail loudly on something + # the contributor cannot fix — `tests.yml` still gives them a green signal. + if: ${{ github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository }} + + strategy: + # A one-element matrix, not a cross-product: the Supabase adapter only ever + # runs against the Supabase variant. Kept as a matrix so adding a second + # database is a one-line change. + matrix: + db: [supabase] + + env: + CS_WORKSPACE_CRN: ${{ secrets.CS_WORKSPACE_CRN }} + CS_CLIENT_ID: ${{ secrets.CS_CLIENT_ID }} + CS_CLIENT_KEY: ${{ secrets.CS_CLIENT_KEY }} + CS_CLIENT_ACCESS_KEY: ${{ secrets.CS_CLIENT_ACCESS_KEY }} + # Job-level env, not a `.env` file: `dotenv/config` does not override an + # already-set `process.env`, so these win and no secret is written to disk. + # + # `postgres` here is deliberately NOT a superuser on this image — that is + # what makes the EQL install, the grants, and the ORE opclass skip behave + # as they do on a real Supabase project. + DATABASE_URL: postgres://postgres:password@localhost:55433/postgres + PGRST_URL: http://localhost:55430 + # Scoped to the suites this database serves. The Drizzle suites talk to + # plain Postgres and get their own job; the harness and bloom suites are + # adapter-agnostic and run here because a database is already up. + CS_IT_SUITE: >- + integration/supabase/**/*.integration.test.ts, + integration/harness.integration.test.ts, + integration/match-bloom.integration.test.ts + + steps: + - uses: actions/checkout@v6 + - uses: ./.github/actions/integration-setup + + # Fast pre-flight: fail in seconds if a secret was rotated or cleared, + # before paying for the ~2 GB supabase/postgres pull. The in-test + # `requireIntegrationEnv` is the correctness guarantee; this is the cheap one. + - name: Require CipherStash secrets + uses: ./.github/actions/require-cs-secrets + with: + workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }} + client-id: ${{ secrets.CS_CLIENT_ID }} + client-key: ${{ secrets.CS_CLIENT_KEY }} + client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }} + + - name: Start ${{ matrix.db }} + run: docker compose -f local/docker-compose.${{ matrix.db }}.yml up -d --wait + + # `globalSetup` installs EQL v3 by shelling out to the real + # `stash eql install --eql-version 3 --supabase --direct`, so an installer + # regression fails here rather than hiding behind a test-only SQL apply. + - name: Supabase v3 integration suites + run: pnpm --filter @cipherstash/stack run test:integration + + - name: Stop ${{ matrix.db }} + if: always() + run: docker compose -f local/docker-compose.${{ matrix.db }}.yml down -v diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index b1ec881d3..df99f4c85 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -103,34 +103,6 @@ jobs: echo "CS_CLIENT_ACCESS_KEY=${{ secrets.CS_CLIENT_ACCESS_KEY }}" >> ./packages/protect/.env echo "DATABASE_URL=postgres://cipherstash:password@localhost:5432/cipherstash" >> ./packages/protect/.env - # PostgREST for `supabase-v3-pgrest-live.test.ts` — the only suite that - # executes the supabase v3 adapter against a real server rather than a - # mock that records strings. - # - # Started as a STEP, not a `services:` entry. The role it connects as - # (`authenticator`, a non-superuser member of `anon`) does not exist in - # the postgres image, and service containers all start before the repo is - # checked out — so there is no point at which a `services:` PostgREST - # could find it. A step runs after the roles exist, deterministically. - # - # `PGRST_DB_ANON_ROLE: anon` is the whole point: pointing it at the DB - # owner would make every grant check pass vacuously. - - name: Start PostgREST - run: | - PGPASSWORD=password psql -h localhost -U cipherstash -d cipherstash \ - -v ON_ERROR_STOP=1 -f ./local/postgrest-roles.sql - docker run -d --name postgrest --network host \ - -e PGRST_DB_URI="postgres://authenticator:authpass@localhost:5432/cipherstash" \ - -e PGRST_DB_SCHEMAS=public \ - -e PGRST_DB_ANON_ROLE=anon \ - -e PGRST_DB_CHANNEL_ENABLED=true \ - postgrest/postgrest:v12.2.12 - for i in $(seq 1 30); do - curl -sf -o /dev/null http://localhost:3000/ && exit 0 - sleep 1 - done - echo "PostgREST did not become ready"; docker logs postgrest; exit 1 - - name: Create .env file in ./packages/stack/ run: | touch ./packages/stack/.env @@ -139,7 +111,6 @@ jobs: echo "CS_CLIENT_KEY=${{ secrets.CS_CLIENT_KEY }}" >> ./packages/stack/.env echo "CS_CLIENT_ACCESS_KEY=${{ secrets.CS_CLIENT_ACCESS_KEY }}" >> ./packages/stack/.env echo "DATABASE_URL=postgres://cipherstash:password@localhost:5432/cipherstash" >> ./packages/stack/.env - echo "PGRST_URL=http://localhost:3000" >> ./packages/stack/.env - name: Create .env file in ./packages/protect-dynamodb/ run: | diff --git a/packages/stack/__tests__/helpers/live-gate.ts b/packages/stack/__tests__/helpers/live-gate.ts index 854bfb4dc..3d972868a 100644 --- a/packages/stack/__tests__/helpers/live-gate.ts +++ b/packages/stack/__tests__/helpers/live-gate.ts @@ -51,24 +51,8 @@ export const LIVE_PG_ENABLED = Boolean(process.env.DATABASE_URL) export const describeLivePgOnly = LIVE_PG_ENABLED ? describe : describe.skip -/** - * True when a real PostgREST is reachable AND a `DATABASE_URL` is configured. - * - * The supabase v3 adapter talks to PostgREST, not to Postgres — the aliasing - * `prop:db_name::jsonb` casts, the `cs` containment mapping, the quoted - * envelopes inside `or=(…)`, and the full storage envelope every `public.*` - * domain CHECK must accept are all things only a real server can execute. - * Everything else in the repo asserts them as strings against a mock. - * - * NO CipherStash creds: the suite builds structurally-valid envelopes itself - * (the domain CHECKs are structural — `v`/`i`/`c` plus the domain's index - * terms), so it can run wherever the DB-only suites run. It proves the WIRE and - * the GRANTS. Cryptographic round-tripping is `drizzle-v3/operators-live-pg`'s - * job and needs the credentials. - */ -export const LIVE_SUPABASE_PGREST_ENABLED = - Boolean(process.env.PGRST_URL) && LIVE_PG_ENABLED - -export const describeLiveSupabasePgrest = LIVE_SUPABASE_PGREST_ENABLED - ? describe - : describe.skip +// The PostgREST gate (`LIVE_SUPABASE_PGREST_ENABLED` / `describeLiveSupabasePgrest`) +// is gone. Its only consumer, `supabase-v3-pgrest-live.test.ts`, moved to +// `integration/supabase/wire.integration.test.ts`, which THROWS on missing +// configuration instead of skipping — so no gate, and nothing for the +// coverage guard to assert. diff --git a/packages/stack/__tests__/live-coverage-guard.test.ts b/packages/stack/__tests__/live-coverage-guard.test.ts index 1d44370dd..882310eb9 100644 --- a/packages/stack/__tests__/live-coverage-guard.test.ts +++ b/packages/stack/__tests__/live-coverage-guard.test.ts @@ -57,7 +57,6 @@ import { LIVE_EQL_V3_PG_ENABLED, LIVE_LOCK_CONTEXT_ENABLED, LIVE_PG_ENABLED, - LIVE_SUPABASE_PGREST_ENABLED, } from './helpers/live-gate' // GitHub Actions always sets CI=true; treat any truthy CI as "must run live". @@ -125,26 +124,6 @@ describe('live-coverage guard', () => { }, ) - it.runIf(IN_CI)( - 'CI must have PGRST_URL so the live supabase PostgREST suite does not silently skip', - () => { - expect( - LIVE_SUPABASE_PGREST_ENABLED, - 'CI must run the live supabase PostgREST suite — ' + - '`LIVE_SUPABASE_PGREST_ENABLED` is false. This needs a ' + - '`DATABASE_URL` AND a `PGRST_URL` pointing at the pinned ' + - '`postgrest/postgrest` service in .github/workflows/tests.yml (no ' + - 'CS_* creds — the domain CHECKs are structural). It is ' + - 'the ONLY suite that executes the adapter against a real PostgREST — ' + - 'the `prop:db_name::jsonb` aliasing selects, the `cs` containment ' + - 'mapping, and the full-envelope filter operands that every `public.*` ' + - 'domain CHECK must accept. Everything else asserts those as strings ' + - 'against a mock, so a false value here means the wire encoding is ' + - 'unproven while CI stays green.', - ).toBe(true) - }, - ) - // Local dev with no creds: nothing to assert. Keep at least one always-run // assertion so the file is never reported as fully empty/pending. it('is always collected (guard file runs outside every live gate)', () => { diff --git a/packages/stack/__tests__/helpers/pgrest.ts b/packages/stack/integration/helpers/pgrest.ts similarity index 93% rename from packages/stack/__tests__/helpers/pgrest.ts rename to packages/stack/integration/helpers/pgrest.ts index bedcee766..56fcc8681 100644 --- a/packages/stack/__tests__/helpers/pgrest.ts +++ b/packages/stack/integration/helpers/pgrest.ts @@ -18,7 +18,9 @@ export function makePostgrestClient(): SupabaseClientLike { const url = process.env.PGRST_URL if (!url) { throw new Error( - 'PGRST_URL is not set — makePostgrestClient() must only be called behind `describeLiveSupabasePgrest`', + 'PGRST_URL is not set. The integration `globalSetup` asserts it before any ' + + 'suite is collected, so reaching this means makePostgrestClient() was ' + + 'called outside the integration harness.', ) } return new PostgrestClient(url) as unknown as SupabaseClientLike diff --git a/packages/stack/integration/supabase/adapter.ts b/packages/stack/integration/supabase/adapter.ts index 5ef160089..2d3a08cae 100644 --- a/packages/stack/integration/supabase/adapter.ts +++ b/packages/stack/integration/supabase/adapter.ts @@ -9,10 +9,7 @@ import { import postgres from 'postgres' import { encryptedTable } from '@/eql/v3' import { encryptedSupabaseV3 } from '@/supabase' -import { - makePostgrestClient, - reloadSchemaCache, -} from '../../__tests__/helpers/pgrest' +import { makePostgrestClient, reloadSchemaCache } from '../helpers/pgrest' /** * The Supabase v3 adapter under real ZeroKMS ciphertext. diff --git a/packages/stack/integration/supabase/select-alias.integration.test.ts b/packages/stack/integration/supabase/select-alias.integration.test.ts index 3fcedb516..dcb2d05a2 100644 --- a/packages/stack/integration/supabase/select-alias.integration.test.ts +++ b/packages/stack/integration/supabase/select-alias.integration.test.ts @@ -3,10 +3,7 @@ import postgres from 'postgres' import { afterAll, beforeAll, expect, it } from 'vitest' import { encryptedTable, types } from '@/eql/v3' import { encryptedSupabaseV3 } from '@/supabase' -import { - makePostgrestClient, - reloadSchemaCache, -} from '../../__tests__/helpers/pgrest' +import { makePostgrestClient, reloadSchemaCache } from '../helpers/pgrest' /** * `Date` reconstruction across every way PostgREST can key a result row. diff --git a/packages/stack/__tests__/supabase-v3-pgrest-live.test.ts b/packages/stack/integration/supabase/wire.integration.test.ts similarity index 95% rename from packages/stack/__tests__/supabase-v3-pgrest-live.test.ts rename to packages/stack/integration/supabase/wire.integration.test.ts index 66c57a436..eb14958a4 100644 --- a/packages/stack/__tests__/supabase-v3-pgrest-live.test.ts +++ b/packages/stack/integration/supabase/wire.integration.test.ts @@ -28,26 +28,21 @@ * `permission denied for schema eql_v3_internal` (see `supabase-v3-grants-pg`). */ -import 'dotenv/config' +import { databaseUrl } from '@cipherstash/test-kit' import postgres from 'postgres' -import { afterAll, beforeAll, expect, it } from 'vitest' +import { afterAll, beforeAll, describe, expect, it } from 'vitest' import type { EncryptionClient } from '@/encryption' import { encryptedTable, types } from '@/eql/v3' import { EncryptedQueryBuilderV3Impl } from '@/supabase/query-builder-v3' -import { SUPABASE_PERMISSIONS_SQL_V3 } from '../../cli/src/installer/grants' -import { installEqlV3IfNeeded } from './helpers/eql-v3' import { - describeLiveSupabasePgrest, - LIVE_SUPABASE_PGREST_ENABLED, -} from './helpers/live-gate' -import { makePostgrestClient, reloadSchemaCache } from './helpers/pgrest' -import { narrowedQueryTerm, storageEnvelope } from './helpers/v3-envelope' + narrowedQueryTerm, + storageEnvelope, +} from '../../__tests__/helpers/v3-envelope' +import { makePostgrestClient, reloadSchemaCache } from '../helpers/pgrest' const TABLE = 'protect_ci_v3_pgrest' -const sql = LIVE_SUPABASE_PGREST_ENABLED - ? postgres(process.env.DATABASE_URL as string, { prepare: false }) - : (undefined as unknown as postgres.Sql) +const sql = postgres(databaseUrl(), { prepare: false }) // A DECLARED table: `createdAt → created_at` is the rename the aliasing // `prop:db_name::jsonb` select exists for, and a synthesized table (property == @@ -211,12 +206,9 @@ function from(): any { const ADA_CREATED = new Date('2026-01-02T03:04:05.000Z') beforeAll(async () => { - if (!LIVE_SUPABASE_PGREST_ENABLED) return - await installEqlV3IfNeeded(sql) - - // The shipped Supabase grants — the thing under test, not a hand-rolled - // approximation. Re-applied after install because the bundle DROPs eql_v3. - await sql.unsafe(SUPABASE_PERMISSIONS_SQL_V3) + // EQL v3 and the Supabase grants are installed once per run by `globalSetup`, + // which shells out to the real `stash eql install --eql-version 3 --supabase`. + // Re-applying them here would only test a hand-rolled approximation. await sql.unsafe(`DROP TABLE IF EXISTS ${TABLE}`) await sql.unsafe(` CREATE TABLE ${TABLE} ( @@ -245,12 +237,11 @@ beforeAll(async () => { }, 180_000) afterAll(async () => { - if (!LIVE_SUPABASE_PGREST_ENABLED) return await sql.unsafe(`DROP TABLE IF EXISTS ${TABLE}`) await sql.end() }) -describeLiveSupabasePgrest('supabase v3 adapter over real PostgREST', () => { +describe('supabase v3 adapter over real PostgREST (wire + grants)', () => { it('inserts raw envelopes that clear every domain CHECK, as anon', async () => { const { error, status } = await from().insert({ row_key: 'ada', diff --git a/packages/stack/integration/vitest.config.ts b/packages/stack/integration/vitest.config.ts index 6f31ce677..a80207052 100644 --- a/packages/stack/integration/vitest.config.ts +++ b/packages/stack/integration/vitest.config.ts @@ -11,11 +11,20 @@ import { sharedAlias } from '../../../vitest.shared' * jobs. That separation is what lets the integration suites throw on missing * configuration instead of skipping. * - * `SUITE_GLOB` selects the adapter, so one config serves both without a second - * near-identical file. CI passes it per job; locally it defaults to everything. + * `CS_IT_SUITE` selects which suites run, so one config serves every adapter + * without a second near-identical file. Comma-separated globs; each CI job scopes + * itself to the adapter it provisioned a database for. Locally it defaults to + * everything. + * + * Scoping matters: the Drizzle suites talk straight to Postgres and the Supabase + * suites need PostgREST, so a job running both would have to provision both. */ -const SUITE_GLOB = +const SUITE_GLOBS = ( process.env['CS_IT_SUITE'] ?? 'integration/**/*.integration.test.ts' +) + .split(',') + .map((glob) => glob.trim()) + .filter(Boolean) export default defineConfig({ resolve: { @@ -34,7 +43,7 @@ export default defineConfig({ }, test: { root: resolve(__dirname, '..'), - include: [SUITE_GLOB], + include: SUITE_GLOBS, globalSetup: [resolve(__dirname, 'global-setup.ts')], server: { deps: { From 9b65ae83e6bea366c31fcdbf74da6ee4ee945127 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 10 Jul 2026 23:44:31 +1000 Subject: [PATCH 10/20] feat(stack): order EQL v3 encrypted columns by their OPE term on Supabase MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The v3 Supabase adapter rejected `order()` on every encrypted column, on the grounds that PostgREST cannot emit `ORDER BY eql_v3.ord_term(col)`. True, but it can emit a jsonb path — and the path selects exactly that term. Measured against a live PostgREST over 10 rows: order=amount->>op.asc -> plaintext order (integer_ord) order=amount->>op.desc -> plaintext order order=name->>op.asc -> plaintext order (text_search) order=amount.asc -> r00,r04,r08,r01,… (sorts the envelope) So the bare-ORDER-BY objection was right and the conclusion was wrong. The builder now rewrites an encrypted ordering column to `col->>op` through a new `orderColumnName` seam — deliberately separate from `filterColumnName`, since filters compare whole envelopes through the `eql_v3.*` operators and must not see the path. Rejection is now keyed on the ordering flavour rather than on encryption: `ope` is supported, `ore` is refused (its `ob` term needs the superuser-only opclass, and such a column cannot hold data on managed Postgres at all), and columns with no ordering term are refused as before. `V3OrderableKeys` widens accordingly. `is(col, true)` had been borrowing the orderable key set — the comment there said the two axes were "threaded separately so they can diverge", and now they have — so it gets its own `V3PlaintextKeys` and stays plaintext-only. The integration matrix asserts ordering against the plaintext oracle for all ten OPE-backed domains, in both directions. Reverting `orderColumnName` to a bare column name turns them red, so they discriminate rather than merely pass. Ties are broken on `row_key` in both the oracle and the query — date and timestamp have two samples across three rows, so without that the test would flake instead of proving order. Also collapses the nine three-line family files into one that iterates `FAMILY_NAMES`, so a family added to the kit is covered without touching it. --- .changeset/supabase-v3-order-by-ope-term.md | 35 ++++++++ .../__tests__/drizzle-v3/operators.test.ts | 4 +- packages/stack/__tests__/schema-v3.test.ts | 6 +- .../__tests__/supabase-v3-builder.test.ts | 20 ++--- .../__tests__/supabase-v3-matrix.test.ts | 44 ++++++---- .../stack/__tests__/supabase-v3.test-d.ts | 23 ++++-- .../stack/integration/supabase/adapter.ts | 32 ++++++-- .../supabase/bigint.integration.test.ts | 4 - .../supabase/boolean.integration.test.ts | 4 - .../supabase/date.integration.test.ts | 4 - .../supabase/families.integration.test.ts | 21 +++++ .../supabase/integer.integration.test.ts | 4 - .../supabase/numeric.integration.test.ts | 4 - .../supabase/real-double.integration.test.ts | 4 - .../supabase/smallint.integration.test.ts | 4 - .../supabase/text.integration.test.ts | 4 - .../supabase/timestamp.integration.test.ts | 4 - .../stack/src/supabase/query-builder-v3.ts | 80 ++++++++++++++----- packages/stack/src/supabase/query-builder.ts | 11 ++- packages/stack/src/supabase/types.ts | 55 ++++++++++--- packages/test-kit/src/run-family-suite.ts | 17 ++++ 21 files changed, 279 insertions(+), 105 deletions(-) create mode 100644 .changeset/supabase-v3-order-by-ope-term.md delete mode 100644 packages/stack/integration/supabase/bigint.integration.test.ts delete mode 100644 packages/stack/integration/supabase/boolean.integration.test.ts delete mode 100644 packages/stack/integration/supabase/date.integration.test.ts create mode 100644 packages/stack/integration/supabase/families.integration.test.ts delete mode 100644 packages/stack/integration/supabase/integer.integration.test.ts delete mode 100644 packages/stack/integration/supabase/numeric.integration.test.ts delete mode 100644 packages/stack/integration/supabase/real-double.integration.test.ts delete mode 100644 packages/stack/integration/supabase/smallint.integration.test.ts delete mode 100644 packages/stack/integration/supabase/text.integration.test.ts delete mode 100644 packages/stack/integration/supabase/timestamp.integration.test.ts diff --git a/.changeset/supabase-v3-order-by-ope-term.md b/.changeset/supabase-v3-order-by-ope-term.md new file mode 100644 index 000000000..320216c6f --- /dev/null +++ b/.changeset/supabase-v3-order-by-ope-term.md @@ -0,0 +1,35 @@ +--- +'@cipherstash/stack': minor +--- + +**`order()` now works on EQL v3 encrypted ordering columns in the Supabase +adapter.** It was rejected outright on every encrypted column. + +A bare `ORDER BY col` on an EQL v3 domain really is wrong — the bundle declares +no btree operator class on any domain, so the sort falls through to jsonb's +default `jsonb_cmp` and compares the envelope's keys in storage order, starting +at the random ciphertext `c`. Measured over ten rows it returns +`r00,r04,r08,r01,…` where the plaintext order is `r00..r09`. No error, a stable +and plausible-looking meaningless order. + +But the correct sort key is reachable without a function call. `eql_v3.ord_term` +returns the domain's `op` term, and OPE is order-preserving, so ordering by the +term reproduces the plaintext order. PostgREST cannot emit +`ORDER BY eql_v3.ord_term(col)`, but it can emit a jsonb path. The builder now +emits `order=col->>op` for an encrypted ordering column, verified against a live +PostgREST for `integer_ord` and `text_search` in both directions. + +The guard is now on the ordering FLAVOUR, not on encryption: + +- **`ope` present → supported.** Every plain `*_ord` domain, plus `text_ord` and + `text_search`. +- **`ore` present → rejected.** The `ob` term is an array of ORE blocks whose + comparison needs the superuser-only operator class, which no jsonb path can + reach. (Such a column cannot hold data on managed Postgres anyway: its domain + CHECK raises `ore_domain_unavailable`.) +- **neither → rejected.** Storage-only, equality-only and match-only columns + carry no ordering term. + +`V3OrderableKeys` widens to match, so `order()` on an ordering column +typechecks. `is(col, true)` is unaffected — it stays plaintext-only, and now has +its own `V3PlaintextKeys` rather than borrowing the orderable set. diff --git a/packages/stack/__tests__/drizzle-v3/operators.test.ts b/packages/stack/__tests__/drizzle-v3/operators.test.ts index a8da7abca..0fe344fa1 100644 --- a/packages/stack/__tests__/drizzle-v3/operators.test.ts +++ b/packages/stack/__tests__/drizzle-v3/operators.test.ts @@ -321,7 +321,9 @@ describe('createEncryptionOperatorsV3 - comparison & range', () => { ) }) - it.each(orderDomains)('%s asc / desc extract the ord term', (eqlType, spec) => { + it.each( + orderDomains, + )('%s asc / desc extract the ord term', (eqlType, spec) => { const { ops, render } = setup() // eql-3.0.0 splits the extractor by ordering flavour: `ord_term` for the // OPE-backed `_ord` domains, `ord_term_ore` for the block-ORE `_ord_ore`. diff --git a/packages/stack/__tests__/schema-v3.test.ts b/packages/stack/__tests__/schema-v3.test.ts index f044eeb72..72f442d7b 100644 --- a/packages/stack/__tests__/schema-v3.test.ts +++ b/packages/stack/__tests__/schema-v3.test.ts @@ -422,7 +422,11 @@ describe('eql_v3 text order domains carry the hm (unique) index (regression)', ( // columns must emit `unique` (hm) IN ADDITION to `ore` (ob), or a real INSERT // fails with `value for domain public.eql_v3_text_ord_ore violates check constraint`. it.each([ - ['text_ord_ore', types.TextOrdOre, { unique: { token_filters: [] }, ore: {} }], + [ + 'text_ord_ore', + types.TextOrdOre, + { unique: { token_filters: [] }, ore: {} }, + ], ['text_ord', types.TextOrd, { unique: { token_filters: [] }, ope: {} }], ] as const)('%s emits both unique (hm) and its ordering index', (_name, builder, expected) => { expect(builder('c').build().indexes).toStrictEqual(expected) diff --git a/packages/stack/__tests__/supabase-v3-builder.test.ts b/packages/stack/__tests__/supabase-v3-builder.test.ts index c72d55d92..da561d618 100644 --- a/packages/stack/__tests__/supabase-v3-builder.test.ts +++ b/packages/stack/__tests__/supabase-v3-builder.test.ts @@ -351,31 +351,33 @@ describe('encryptedSupabaseV3 wire encoding', () => { // The old guard only rejected columns LACKING orderAndRange, i.e. exactly the // columns where the wrongness was obvious, and admitted the ORE-capable ones // where it is silent. - it('rejects order() on an ORE-capable encrypted column', async () => { - const { es } = v3Instance() + it('orders an OPE-backed encrypted column by its op term', async () => { + const { es, supabase } = v3Instance() - const { error, status } = await es + const { error } = await es .from('users', users) .select('id, createdAt') .order('createdAt') - expect(status).toBe(500) - expect(error?.message).toContain('cannot order by encrypted column') - expect(error?.message).toContain('created_at') - expect(error?.message).toContain('timestamp_ord') - expect(error?.message).toContain('ord_term') + expect(error).toBeNull() + // The jsonb path, not a bare column: `ORDER BY created_at` would sort the + // ciphertext envelope through jsonb's default opclass. + expect(supabase.callsFor('order')[0].args[0]).toBe('created_at->>op') }) it('rejects order() on an equality-only encrypted column', async () => { const { es } = v3Instance() + // `nickname` is public.eql_v3_text_eq: an `hm` term and nothing to sort by. + // (`amount` is IntegerOrd — OPE-backed, and therefore orderable.) const { error, status } = await es .from('users', users) .select('id') - .order('amount') + .order('nickname') expect(status).toBe(500) expect(error?.message).toContain('cannot order by encrypted column') + expect(error?.message).toContain('no ordering term') }) it('leaves plaintext columns untouched in order()', async () => { diff --git a/packages/stack/__tests__/supabase-v3-matrix.test.ts b/packages/stack/__tests__/supabase-v3-matrix.test.ts index 25170a309..7ecf052c5 100644 --- a/packages/stack/__tests__/supabase-v3-matrix.test.ts +++ b/packages/stack/__tests__/supabase-v3-matrix.test.ts @@ -157,21 +157,35 @@ describe('supabase v3 wire encoding, every domain', () => { expect(JSON.parse(gte.args[1] as string).c).toBeDefined() }) - // `gte` works — the `>=` operators ARE declared on the ord domains — but - // `order()` does not: no btree OPERATOR CLASS exists on any domain, so - // `ORDER BY col` falls through to jsonb's default opclass and sorts the - // ciphertext envelope. Filtering and sorting resolve through different - // machinery, and only sorting is broken. - it('rejects order() even though gte() is supported', async () => { - const { q, supabase, name } = instanceFor(eqlType, spec) - - const { error, status } = await q.select(`id, ${name}`).order(name) - - expect(status).toBe(500) - expect(error?.message).toContain('cannot order by encrypted column') - expect(error?.message).toContain('ord_term') - expect(supabase.callsFor('order')).toHaveLength(0) - }) + // A bare `ORDER BY col` WOULD be wrong — no btree operator class exists on + // any EQL v3 domain, so it falls through to jsonb's default opclass and + // sorts the ciphertext envelope. The builder never emits one. For an + // OPE-backed column it emits the jsonb path `col->>op`, which selects the + // order-preserving term. For an ORE-backed column there is no such path — + // `ob` is an array of blocks needing the superuser-only comparator — so it + // is refused. + if (spec.indexes.ope) { + it('orders by the OPE term, not by the envelope', async () => { + const { q, supabase, name } = instanceFor(eqlType, spec) + + const { error } = await q.select(`id, ${name}`).order(name) + + expect(error).toBeNull() + const [order] = supabase.callsFor('order') + expect(order.args[0]).toBe(`${name}->>op`) + }) + } else { + it('rejects order() even though gte() is supported', async () => { + const { q, supabase, name } = instanceFor(eqlType, spec) + + const { error, status } = await q.select(`id, ${name}`).order(name) + + expect(status).toBe(500) + expect(error?.message).toContain('cannot order by encrypted column') + expect(error?.message).toContain('ORE ordering term') + expect(supabase.callsFor('order')).toHaveLength(0) + }) + } }) describe.each(matchDomains)('%s (freeTextSearch)', (eqlType, spec) => { diff --git a/packages/stack/__tests__/supabase-v3.test-d.ts b/packages/stack/__tests__/supabase-v3.test-d.ts index 8b042f237..6b6337e30 100644 --- a/packages/stack/__tests__/supabase-v3.test-d.ts +++ b/packages/stack/__tests__/supabase-v3.test-d.ts @@ -105,20 +105,31 @@ describe('encryptedSupabaseV3 typed surface (with schemas)', () => { builder.is('email', null) }) - it('rejects order() on every encrypted column at the type level', async () => { + it('allows order() on encrypted columns that carry an ordering term', async () => { const supabase = await encryptedSupabaseV3(supabaseClient, { schemas: { users }, }) const builder = supabase.from('users') - // No btree opclass exists on any EQL v3 domain, so `ORDER BY col` sorts the - // ciphertext envelope. This holds for the ORE-capable domains too — which is - // the whole point: those are the ones where the wrongness is silent. - // @ts-expect-error — timestamp_ord: ORDER BY sorts ciphertext + // The builder does not emit a bare `ORDER BY col` — that would sort the + // ciphertext envelope through jsonb's default opclass. It emits the jsonb + // path `col->>op`, which selects the OPE term, and OPE is order-preserving. builder.order('createdAt') - // @ts-expect-error — integer_ord: ORDER BY sorts ciphertext builder.order('amount', { ascending: false }) + // `text_search` carries `ope` alongside its match and equality terms. + builder.order('email') + }) + + it('rejects order() on encrypted columns with no ordering term', async () => { + const supabase = await encryptedSupabaseV3(supabaseClient, { + schemas: { users }, + }) + const builder = supabase.from('users') // @ts-expect-error — active is public.eql_v3_boolean: storage only builder.order('active') + // @ts-expect-error — nickname is public.eql_v3_text_eq: equality only + builder.order('nickname') + // @ts-expect-error — bio is public.eql_v3_text_match: match only + builder.order('bio') }) it('still allows order() on a plaintext row key', async () => { diff --git a/packages/stack/integration/supabase/adapter.ts b/packages/stack/integration/supabase/adapter.ts index 2d3a08cae..63037cdee 100644 --- a/packages/stack/integration/supabase/adapter.ts +++ b/packages/stack/integration/supabase/adapter.ts @@ -23,8 +23,12 @@ import { makePostgrestClient, reloadSchemaCache } from '../helpers/pgrest' * introspection reads it — which is why `createTable` also builds the client. */ -/** PostgREST cannot emit `ORDER BY eql_v3.ord_term(col)`, and a bare `ORDER BY` */ -/** sorts the raw ciphertext envelope. The adapter refuses ordering outright. */ +/** + * `order` is supported. The builder rewrites an encrypted ordering column to the + * jsonb path `col->>op`, which selects the OPE term; OPE is order-preserving, so + * PostgREST reproduces the plaintext order. Columns with no ordering term are + * rejected by the capability matrix, exactly like `gt`/`lt`. + */ const SUPPORTED_OPS: ReadonlySet = new Set([ 'eq', 'ne', @@ -43,12 +47,11 @@ const SUPPORTED_OPS: ReadonlySet = new Set([ ]) /** - * `order` is refused on EVERY encrypted column, whatever its capabilities. That - * is the half of the contract the capability matrix cannot derive — an - * ORE-capable column still cannot be ordered through PostgREST — so it is - * declared here and pinned by a test on every domain. + * Nothing is refused adapter-wide. ORE-backed columns cannot be ordered through + * PostgREST, but they are `deferred` in the catalog — they cannot hold data on + * managed Postgres at all — so no test reaches them here. */ -const ALWAYS_REJECTED: ReadonlySet = new Set(['order']) +const ALWAYS_REJECTED: ReadonlySet = new Set() type Instance = Awaited> @@ -106,7 +109,20 @@ export function makeSupabaseAdapter(): IntegrationAdapter { case 'isNotNull': return q.not(op.column, 'is', null) case 'order': - return q.order(op.column) + // Exclude the all-NULL row: NULL ordering is a property of the SQL + // dialect, not of the encrypted ordering term, and the oracle sorts + // value rows only. + // + // The secondary sort on `row_key` is load-bearing. A domain with fewer + // distinct samples than rows gives two rows the SAME plaintext, so the + // encrypted term alone does not determine their order — the oracle + // breaks such ties on `row_key` ascending, and the query must mirror it + // or the test flakes rather than proving ordering. (date/timestamp have + // two samples across three rows; the numeric families have four.) + return q + .not(op.column, 'is', null) + .order(op.column, { ascending: op.direction === 'asc' }) + .order('row_key', { ascending: true }) } } diff --git a/packages/stack/integration/supabase/bigint.integration.test.ts b/packages/stack/integration/supabase/bigint.integration.test.ts deleted file mode 100644 index b0bd74ae9..000000000 --- a/packages/stack/integration/supabase/bigint.integration.test.ts +++ /dev/null @@ -1,4 +0,0 @@ -import { runFamilySuite } from '@cipherstash/test-kit/suite' -import { makeSupabaseAdapter } from './adapter' - -runFamilySuite('bigint', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/boolean.integration.test.ts b/packages/stack/integration/supabase/boolean.integration.test.ts deleted file mode 100644 index f12b0cbe2..000000000 --- a/packages/stack/integration/supabase/boolean.integration.test.ts +++ /dev/null @@ -1,4 +0,0 @@ -import { runFamilySuite } from '@cipherstash/test-kit/suite' -import { makeSupabaseAdapter } from './adapter' - -runFamilySuite('boolean', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/date.integration.test.ts b/packages/stack/integration/supabase/date.integration.test.ts deleted file mode 100644 index 971c4a141..000000000 --- a/packages/stack/integration/supabase/date.integration.test.ts +++ /dev/null @@ -1,4 +0,0 @@ -import { runFamilySuite } from '@cipherstash/test-kit/suite' -import { makeSupabaseAdapter } from './adapter' - -runFamilySuite('date', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/families.integration.test.ts b/packages/stack/integration/supabase/families.integration.test.ts new file mode 100644 index 000000000..b7329b36c --- /dev/null +++ b/packages/stack/integration/supabase/families.integration.test.ts @@ -0,0 +1,21 @@ +import { FAMILY_NAMES } from '@cipherstash/test-kit' +import { runFamilySuite } from '@cipherstash/test-kit/suite' +import { makeSupabaseAdapter } from './adapter' + +/** + * Every EQL v3 domain the SDK models, driven through the Supabase adapter + * against real ZeroKMS ciphertext. + * + * One file, not one per family. The driver derives the whole suite from the + * catalog, so a per-family file was three identical lines and bought nothing: + * `fileParallelism` is off (one database, shared tables), and vitest names each + * failure by its `describe` path — `v3 supabase — text > text_match > …` — so a + * failing family is just as identifiable from here. + * + * Iterating `FAMILY_NAMES` also closes a gap a hand-written file list cannot: a + * new family added to the kit is covered without touching this file, and a + * family removed from the kit cannot leave an orphaned suite behind. + */ +for (const family of FAMILY_NAMES) { + runFamilySuite(family, makeSupabaseAdapter) +} diff --git a/packages/stack/integration/supabase/integer.integration.test.ts b/packages/stack/integration/supabase/integer.integration.test.ts deleted file mode 100644 index a1463710a..000000000 --- a/packages/stack/integration/supabase/integer.integration.test.ts +++ /dev/null @@ -1,4 +0,0 @@ -import { runFamilySuite } from '@cipherstash/test-kit/suite' -import { makeSupabaseAdapter } from './adapter' - -runFamilySuite('integer', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/numeric.integration.test.ts b/packages/stack/integration/supabase/numeric.integration.test.ts deleted file mode 100644 index 914f03d3d..000000000 --- a/packages/stack/integration/supabase/numeric.integration.test.ts +++ /dev/null @@ -1,4 +0,0 @@ -import { runFamilySuite } from '@cipherstash/test-kit/suite' -import { makeSupabaseAdapter } from './adapter' - -runFamilySuite('numeric', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/real-double.integration.test.ts b/packages/stack/integration/supabase/real-double.integration.test.ts deleted file mode 100644 index 1797e67f2..000000000 --- a/packages/stack/integration/supabase/real-double.integration.test.ts +++ /dev/null @@ -1,4 +0,0 @@ -import { runFamilySuite } from '@cipherstash/test-kit/suite' -import { makeSupabaseAdapter } from './adapter' - -runFamilySuite('real-double', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/smallint.integration.test.ts b/packages/stack/integration/supabase/smallint.integration.test.ts deleted file mode 100644 index 38eaa23bc..000000000 --- a/packages/stack/integration/supabase/smallint.integration.test.ts +++ /dev/null @@ -1,4 +0,0 @@ -import { runFamilySuite } from '@cipherstash/test-kit/suite' -import { makeSupabaseAdapter } from './adapter' - -runFamilySuite('smallint', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/text.integration.test.ts b/packages/stack/integration/supabase/text.integration.test.ts deleted file mode 100644 index d25741d39..000000000 --- a/packages/stack/integration/supabase/text.integration.test.ts +++ /dev/null @@ -1,4 +0,0 @@ -import { runFamilySuite } from '@cipherstash/test-kit/suite' -import { makeSupabaseAdapter } from './adapter' - -runFamilySuite('text', makeSupabaseAdapter) diff --git a/packages/stack/integration/supabase/timestamp.integration.test.ts b/packages/stack/integration/supabase/timestamp.integration.test.ts deleted file mode 100644 index 33dbf0ce2..000000000 --- a/packages/stack/integration/supabase/timestamp.integration.test.ts +++ /dev/null @@ -1,4 +0,0 @@ -import { runFamilySuite } from '@cipherstash/test-kit/suite' -import { makeSupabaseAdapter } from './adapter' - -runFamilySuite('timestamp', makeSupabaseAdapter) diff --git a/packages/stack/src/supabase/query-builder-v3.ts b/packages/stack/src/supabase/query-builder-v3.ts index ccdf7364f..be119191a 100644 --- a/packages/stack/src/supabase/query-builder-v3.ts +++ b/packages/stack/src/supabase/query-builder-v3.ts @@ -222,28 +222,39 @@ export class EncryptedQueryBuilderV3Impl< } /** - * Ordering by ANY encrypted v3 column is rejected — including the ORE-capable - * ones, which is the non-obvious half. + * `ORDER BY` on an OPE-backed column is supported; on every other encrypted + * column it is rejected. * - * The `*_ord` domains are `CREATE DOMAIN … AS jsonb`, and the bundle declares - * NO btree operator class on any domain — it actively lints against one - * (`domain_opclass`), because an opclass on a domain bypasses operator - * resolution. So `ORDER BY col` does not reach `eql_v3`'s ORE comparisons at - * all: it resolves through jsonb's DEFAULT btree opclass, `jsonb_cmp`, and - * sorts by the envelope's byte structure — keys compare alphabetically, so the - * sort is effectively on the `bf` bloom array. No error, no warning, and a - * stable, plausible-looking, meaningless row order. + * A bare `ORDER BY col` IS wrong. The `*_ord` domains are + * `CREATE DOMAIN … AS jsonb`, and the bundle declares no btree operator class + * on any domain — it actively lints against one (`domain_opclass`), because an + * opclass on a domain bypasses operator resolution. So the sort resolves + * through jsonb's default `jsonb_cmp` and compares the envelope's keys in + * storage order, starting at the random ciphertext `c`. No error, and a + * stable, meaningless row order. (Measured: over 10 rows it returns + * `r00,r04,r08,r01,…` where the plaintext order is `r00..r09`.) * - * Correct ordering is `ORDER BY eql_v3.ord_term(col)`, which PostgREST's - * `order=` parameter cannot express. The v3 Drizzle integration emits exactly - * that (`sql-dialect.ts` `orderBy`), and proves it against live Postgres. + * But the correct sort key is reachable without a function call. `eql_v3.ord_term` + * returns the domain's `op` term, and OPE is order-preserving by construction: + * ordering by the term reproduces the plaintext order. PostgREST cannot emit + * `ORDER BY eql_v3.ord_term(col)`, but it CAN emit a jsonb path — + * `order=col->>op.asc` — which selects exactly that term. Measured against a + * live PostgREST: `order=amount->>op.asc` and `.desc` both reproduce the + * plaintext order for `integer_ord` and `text_search`, over 10 rows. * - * The `>=`/`<=` operators ARE declared on the ord domains, so `gte`/`lte` - * filters remain correct. Filtering and sorting resolve through different - * machinery; only sorting is broken. + * So the guard is on the ordering FLAVOUR, not on encryption: * - * A column absent from {@link v3Columns} is a plaintext passthrough, and - * orders normally. This runtime guard is the only protection the untyped + * - `ope` present → order by `col->>op`. Every plain `_ord` domain, plus + * `text_ord` and `text_search`. + * - `ore` present → reject. The `ob` term is an array of ORE blocks whose + * comparison needs the superuser-only opclass; a jsonb-path sort over it is + * meaningless. (Such a column cannot hold data on managed Postgres at all: + * its domain CHECK raises `ore_domain_unavailable`.) + * - neither → reject. Storage-only, equality-only and match-only columns + * carry no ordering term to sort by. + * + * A column absent from {@link v3Columns} is a plaintext passthrough and orders + * normally. This runtime guard is the only protection the untyped * (no-`schemas`) surface has. */ protected override validateTransforms(): void { @@ -251,12 +262,43 @@ export class EncryptedQueryBuilderV3Impl< if (t.kind !== 'order') continue const column = this.v3Columns[t.column] if (!column) continue + + const indexes = this.columnSchemas[column.getName()]?.indexes + if (indexes?.ope) continue + + const reason = indexes?.ore + ? 'its ORE ordering term (`ob`) needs the superuser-only ORE operator class, which PostgREST cannot reach through a jsonb path' + : 'it carries no ordering term to sort by' + throw new Error( - `[supabase v3]: cannot order by encrypted column "${column.getName()}" (${column.getEqlType()}) — PostgREST cannot emit \`ORDER BY eql_v3.ord_term("${column.getName()}")\`, and a bare \`ORDER BY\` sorts the raw ciphertext envelope, not the plaintext. Order by a plaintext column, expose \`eql_v3.ord_term()\` as a generated column or view and order by that, or use the EQL v3 Drizzle integration.`, + `[supabase v3]: cannot order by encrypted column "${column.getName()}" (${column.getEqlType()}) — ${reason}. ` + + 'Order by a plaintext column, or use an OPE-backed ordering domain ' + + '(`*_ord`, `text_ord`, `text_search`), or use the EQL v3 Drizzle integration.', ) } } + /** + * Encrypted ordering columns sort by their `op` term, not by the envelope. + * + * `order=col->>op` is the one ordering expression PostgREST can emit that + * reaches the OPE term. It must NOT leak into filters — those compare whole + * envelopes through the `eql_v3.*` operators — which is why this is its own + * seam rather than a change to `filterColumnName`. + * + * `validateTransforms` has already rejected every encrypted column that lacks + * an `ope` index, so reaching the jsonb path here implies the term exists. + */ + protected override orderColumnName(column: string): DbName { + const dbName = this.dbNameFor(column) + const encrypted = this.v3Columns[column] + if (!encrypted) return dbName as DbName + + return ( + this.columnSchemas[dbName]?.indexes?.ope ? `${dbName}->>op` : dbName + ) as DbName + } + /** * Resolve a raw `.filter()` operator to the capability it exercises. Unlike * v2, the v3 operand is always the full storage envelope, so `queryType` diff --git a/packages/stack/src/supabase/query-builder.ts b/packages/stack/src/supabase/query-builder.ts index ca604e3d8..1defbf8b7 100644 --- a/packages/stack/src/supabase/query-builder.ts +++ b/packages/stack/src/supabase/query-builder.ts @@ -831,10 +831,19 @@ export class EncryptedQueryBuilderImpl< return { kind: 'structured', conditions: of_.conditions.map(toDbCondition) } } + /** + * The column expression `order()` sends to PostgREST. Its own seam, separate + * from {@link filterColumnName}: v3 orders an encrypted column by a jsonb path + * into its ordering term, which must not leak into filters. + */ + protected orderColumnName(column: string): DbName { + return this.filterColumnName(column) + } + private transformToDbSpace(t: TransformOp): DbTransformOp { switch (t.kind) { case 'order': - return { ...t, column: this.filterColumnName(t.column) } + return { ...t, column: this.orderColumnName(t.column) } // `returns` is in the union but never pushed (`returns()` is a cast). case 'limit': case 'range': diff --git a/packages/stack/src/supabase/types.ts b/packages/stack/src/supabase/types.ts index 2c0761161..b65737ff0 100644 --- a/packages/stack/src/supabase/types.ts +++ b/packages/stack/src/supabase/types.ts @@ -188,15 +188,50 @@ export type V3ContainsValue< : string /** - * Row keys a v3 builder accepts in `order()`: every row key that is NOT an - * encrypted v3 column. `ORDER BY` on an EQL v3 domain sorts the raw ciphertext - * envelope — the bundle declares no btree operator class on any domain, so the - * sort resolves through jsonb's default `jsonb_cmp`. Correct ordering needs - * `eql_v3.ord_term(col)`, which PostgREST cannot emit. + * JS property names of a v3 table's columns that carry NO `orderAndRange` + * capability — storage-only, equality-only and match-only domains. They hold no + * ordering term, so there is nothing for `order()` to sort by. + */ +export type NonOrderableV3Keys = { + [K in Extract< + keyof V3ColumnsOfTable
, + string + >]: 'orderAndRange' extends QueryTypesForColumn[K]> + ? never + : K +}[Extract, string>] + +/** + * Row keys a v3 builder accepts in `order()`: every plaintext row key, plus the + * encrypted columns that carry an ordering term. + * + * A bare `ORDER BY col` on an EQL v3 domain IS wrong — the bundle declares no + * btree opclass on any domain, so the sort resolves through jsonb's default + * `jsonb_cmp` and compares the random ciphertext first. But the builder does not + * emit a bare `ORDER BY`: for an encrypted ordering column it emits the jsonb + * path `col->>op`, which selects the OPE term, and OPE is order-preserving. See + * `EncryptedQueryBuilderV3Impl.orderColumnName`. + * + * ORE-backed (`*_ord_ore`) columns are `orderAndRange`-capable and so pass this + * type, but are rejected at runtime: their `ob` term needs the superuser-only + * ORE opclass, which no jsonb path can reach. Encoding the ordering FLAVOUR in + * the type system would mean threading it through `QueryTypesForColumn`, and + * such a column cannot hold data on managed Postgres anyway (its domain CHECK + * raises `ore_domain_unavailable`), so the runtime guard is where it belongs. */ export type V3OrderableKeys< Table extends AnyV3Table, Row extends Record, +> = Exclude, NonOrderableV3Keys
> + +/** + * Row keys that are NOT encrypted v3 columns. Used where a method's operand is a + * SQL value rather than a ciphertext envelope — `is(col, true)` in particular, + * since an encrypted column holds jsonb and can never be `IS TRUE`. + */ +export type V3PlaintextKeys< + Table extends AnyV3Table, + Row extends Record, > = Exclude< Extract, Extract, string> @@ -220,10 +255,12 @@ export interface EncryptedQueryBuilderV3< V3FilterableKeys & StringKeyOf, EncryptedQueryBuilderV3, V3OrderableKeys & StringKeyOf, - // `is(col, true)` is legal only on a plaintext column. That is exactly the - // orderable set today — every encrypted column is excluded from both — but - // the two are threaded separately so they can diverge. - V3OrderableKeys & StringKeyOf + // `is(col, true)` is legal only on a PLAINTEXT column: an encrypted column + // holds a jsonb envelope, never a SQL boolean. The two axes were threaded + // separately "so they can diverge", and now they have — `order()` admits + // encrypted ordering columns (sorted by their `op` term), `is(col, true)` + // still admits none. + V3PlaintextKeys & StringKeyOf > { contains & StringKeyOf>( column: K, diff --git a/packages/test-kit/src/run-family-suite.ts b/packages/test-kit/src/run-family-suite.ts index 2a971c5a2..05e9f8675 100644 --- a/packages/test-kit/src/run-family-suite.ts +++ b/packages/test-kit/src/run-family-suite.ts @@ -13,6 +13,7 @@ import { containsPlain, expectedKeysFor, plainValue, + sortedKeysFor, } from './oracle.ts' import { planRows, planTable } from './rows.ts' @@ -308,6 +309,22 @@ export function runFamilySuite( }) } + if (positive.has('order')) { + it.each([ + 'asc', + 'desc', + ] as const)('order(%s) returns rows in plaintext order', async (direction) => { + // Order is the one operation whose ROW ORDER is the assertion, so + // it does not go through `expectRows` (which sorts both sides). + const rows = await adapter.run(table, { + kind: 'order', + column: slug, + direction, + }) + expect(rows).toEqual(sortedKeysFor(spec, rowKeys, direction)) + }) + } + // Structural, never capability-gated: a NULL plaintext is a SQL NULL on // every domain, storage-only ones included. it('isNull selects only the all-null row', async () => { From cd43b39f8abc9a817e874ad266bd099da496951a Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Fri, 10 Jul 2026 23:45:17 +1000 Subject: [PATCH 11/20] =?UTF-8?q?docs(test-kit):=20correct=20the=20ORE=20d?= =?UTF-8?q?eferral=20reason=20=E2=80=94=20unusable,=20not=20silently=20wro?= =?UTF-8?q?ng?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Measured on supabase/postgres as the non-superuser `postgres` role: the ORE domains are created, but their CHECK calls `ore_domain_unavailable()` and the first INSERT raises, with a hint naming the alternatives. The same insert succeeds on plain Postgres as a superuser. So ORE is loudly unusable on managed Postgres, not silently wrong. The bundle guards correctly. The earlier reason — 'ORDER BY silently mis-sorts' — described a type-level property (`ore_block_256` is a composite type, so with no opclass Postgres falls back to record comparison) that no value can reach, because the write is refused first. The deferral still stands, for a better reason: a matrix that must pass on both a superuser and a managed database cannot seed an ORE column at all. Corrects the claim in the plan doc, and in cipherstash/encrypt-query-language#395. --- .../2026-07-10-eql-v3-integration-tests.md | 7 +--- packages/test-kit/src/catalog.ts | 42 ++++++++----------- 2 files changed, 20 insertions(+), 29 deletions(-) diff --git a/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md b/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md index 4a30e788d..f972c7cac 100644 --- a/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md +++ b/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md @@ -26,11 +26,8 @@ - **v3 JSON is unimplemented, not merely untested.** The bundle ships `public.eql_v3_json` and `public.eql_v3_jsonb_entry` (51 `CREATE DOMAIN`s, 56 `eql_v3.{ste_vec,jsonb_path,selector,contained_by}` functions), but the SDK models 41 domains and `eql/v3/columns.ts` admits only `bigint | boolean | date | number | string | timestamp` as `cast_as`. There is no `'json'` kind, so a v3 JSON column cannot be declared. **The gap is in the core v3 schema; both adapters inherit it.** `eql/v3/drizzle/codec.ts:38` already carries defensive SteVec decode logic (`sv[0].c`) for documents nothing can currently create. - **`@cipherstash/stack/drizzle` is not `@cipherstash/drizzle`.** They are a fork: `@cipherstash/drizzle@3.0.3` peer-depends on the predecessor SDK `@cipherstash/protect@12` and exports `createProtectOperators` (2,038 lines); stack's in-tree copy exports `createEncryptionOperators` (1,945 lines); ~645 lines have diverged. There is no dependency between them — hence no cycle today, only duplication. Docs reference the stack subpath 13× and the package 2×. - **Ordering is safe on OPE.** EQL 3.0.0 pins `_ord` domains to `op` (CLLW-OPE), which orders via a native `bytea` btree, so `ORDER BY eql_v3.ord_term(col)` works on every provider without superuser. -- **`_ord_ore` is not "absent" on managed Postgres — it is silently wrong, and only for `ORDER BY`.** Measured against `supabase/postgres:17.4.1.048` after `stash eql install --eql-version 3 --supabase --direct` as the non-superuser `postgres` role: all 51 `public.eql_v3_*` domains install, and so do the 33 ORE comparison functions; only the btree opclass `eql_v3_internal.ore_block_256_operator_class` is skipped (`pg_opclass` count 0 vs 1 on plain Postgres). - - **Range filters still work everywhere.** `<`/`>`/`<=`/`>=` on `ore_block_256` are backed by `ore_block_256_lt` → the ORE comparator, and operators need no opclass. - - **`ORDER BY eql_v3.ord_term_ore(col)` does not error — it sorts, wrongly.** `ore_block_256` is a *composite* type, so with no opclass to resolve, Postgres falls back to built-in record comparison, which walks down to the raw `bytes` field and compares bytewise. That order is deterministic and stable but uncorrelated with plaintext: over 200 random well-formed terms it disagreed with the ORE comparator on **87**. Proof that the two variants take different paths: `ORDER BY` on a malformed 1-byte term returns rows on Supabase and raises `Malformed ORE term` on plain Postgres. - - A btree **index** on such a column cannot be created at all. - - That is why ORE gets its own suite rather than a skip. See `docs/eql-v3-ord-term-ordering-defect.md`. +- **`_ord_ore` columns cannot hold data on managed Postgres — and that is correct.** Measured against `supabase/postgres:17.4.1.048`, EQL 3.0.0, as the non-superuser `postgres` role: the nine ORE domains are created, but their CHECK calls `eql_v3_internal.ore_domain_unavailable()`, so the first `INSERT` raises with a hint naming `_eq` / `_ord` / `_ord_ope` as alternatives. The same insert succeeds on plain Postgres as a superuser. The bundle guards correctly; there is no silent wrongness. (An earlier draft of this plan claimed ORDER BY silently mis-sorts. It does at the *type* level — `ore_block_256` is a composite type, so with no opclass Postgres falls back to record comparison — but no value can reach a table to be ordered. `cipherstash/encrypt-query-language#395` records the correction.) A matrix that must pass on both databases cannot cover ORE, because the seed insert fails on one; ORE gets a superuser-only suite. +- **Supabase CAN order encrypted `_ord` columns.** A bare `ORDER BY col` sorts the ciphertext envelope through jsonb's default opclass (measured: `r00,r04,r08,r01,…` for plaintext `r00..r09`). But `eql_v3.ord_term` returns the `op` term, OPE is order-preserving, and PostgREST can emit the jsonb path `order=col->>op`, which reproduces plaintext order in both directions for `integer_ord` and `text_search`. The adapter now does exactly that, and rejects only ORE-backed and ordering-less columns. - **The CLI installs fine as a non-superuser.** `stash eql install --eql-version 3 --supabase --direct --database-url …` runs non-interactively as `postgres` on `supabase/postgres` and grants `anon` USAGE on **both** `eql_v3` and `eql_v3_internal`. No `SUPABASE_ADMIN_URL` is needed. (`--supabase` without `--direct`/`--migration` prompts, so pass `--direct`.) - **`postgres-eql:17-2.3.1` ships EQL v2, not v3.** Every DB variant must install v3 at setup. - **`./supabase` and `./drizzle` are published stack subpaths** (`npm view @cipherstash/stack exports`); `./eql/v3/drizzle` is not. diff --git a/packages/test-kit/src/catalog.ts b/packages/test-kit/src/catalog.ts index cdbfdfb97..93f085fd0 100644 --- a/packages/test-kit/src/catalog.ts +++ b/packages/test-kit/src/catalog.ts @@ -282,38 +282,32 @@ const BIGINT_ERR = [9223372036854775808n, -9223372036854775809n] as const // biome-ignore format: one row per domain reads as a table; keep it dense. /** * The `_ord_ore` domains are block-ORE, whose operator class is superuser-only: - * the eql-3.0.0 bundle self-skips those statements on `insufficient_privilege`. + * the eql-3.0.0 bundle self-skips that statement on `insufficient_privilege`. * * Measured against `supabase/postgres:17.4.1.048` after `stash eql install - * --eql-version 3 --supabase --direct`, connected as the non-superuser - * `postgres` role: all 51 `public.eql_v3_*` domains install, and so do the 33 - * ORE comparison functions. Only the btree OPCLASS - * (`eql_v3_internal.ore_block_256_operator_class`) is skipped. + * --eql-version 3 --supabase --direct`, as the non-superuser `postgres` role: + * the domains ARE created, but they cannot hold data. Their CHECK calls + * `eql_v3_internal.ore_domain_unavailable()`, so the first INSERT raises * - * What that does and does not break: + * ERROR: EQL: public.eql_v3_integer_ord_ore cannot be used on this platform: + * the EQL installer could not create the ORE operator class + * HINT: Use public.eql_v3_integer_eq (equality) or public.eql_v3_integer_ord + * (ordering) or public.eql_v3_integer_ord_ope (ordering) instead. * - * - RANGE FILTERS STILL WORK. The `<`/`>`/`<=`/`>=` operators on - * `ore_block_256` are backed by `ore_block_256_lt` → the ORE comparator, and - * operators need no opclass. `eql_v3.gt`/`lt`/`gte`/`lte` compare in true ORE - * order on every provider. - * - `ORDER BY eql_v3.ord_term_ore(col)` DOES NOT ERROR — it sorts, wrongly. - * `ore_block_256` is a COMPOSITE type, so with no opclass to resolve, Postgres - * falls back to its built-in record comparison, which walks down to the raw - * `bytes` field and compares bytewise. Bytewise order over ORE ciphertext is - * deterministic and stable but uncorrelated with plaintext order: measured - * over 200 random well-formed terms, it disagreed with the ORE comparator on - * 87 of them. A stable, plausible-looking, silently wrong ordering. - * - A btree INDEX on such a column cannot be created at all. + * The same INSERT succeeds on plain Postgres as a superuser. So ORE is not + * silently wrong on managed Postgres — it is loudly unusable, which is the right + * behaviour and means no query can ever return wrong rows from an ORE column. * - * A matrix whose ordering assertions must hold on every provider cannot straddle - * that, so ORE gets its own suite. + * A matrix that must pass on both a superuser and a managed database therefore + * cannot cover them: the seed INSERT fails on one of the two. ORE gets a + * superuser-only suite instead. * - * The OPE-backed `_ord` domains have no such split: they order via a native - * `bytea` btree and behave identically everywhere. See - * `docs/eql-v3-ord-term-ordering-defect.md`. + * (The OPE-backed `_ord` domains have no such split. `eql_v3_internal.ope_cllw` + * is a domain over `bytea`, so it orders via the native btree everywhere, and + * the Supabase adapter sorts by it through the jsonb path `col->>op`.) */ const ORE_DEFERRED = - 'block-ORE opclass is superuser-only, so ORDER BY silently mis-sorts on managed Postgres; covered by a follow-up suite' + 'block-ORE columns cannot hold data on managed Postgres (domain CHECK raises ore_domain_unavailable); covered by a superuser-only follow-up suite' export const V3_MATRIX = { // integer From 6043f9bb77fcb81eda884f58e0273a60facc61db Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 11 Jul 2026 00:02:14 +1000 Subject: [PATCH 12/20] fix(stack): order by `col->op`, and pin what makes it correct MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `->` (jsonb) rather than `->>` (text), matching the canonical EQL form `ORDER BY eql_v3.ord_term(col)` more closely — it keeps the comparison on the typed value. It does NOT avoid the database collation, which is worth saying plainly because it is the obvious reason to prefer it. Postgres compares jsonb strings with `varstr_cmp` under the default collation, exactly as it does text. Measured on `en_US.UTF-8`: `ORDER BY j->'op'` and `ORDER BY j->>'op'` both put `'a'` before `'B'`, while `COLLATE "C"` does the reverse. The two operators are equivalent for ordering. What actually makes the jsonb path agree with `ord_term()` is the term's encoding, so `ope-term.integration.test.ts` asserts it directly: - lowercase hex (`[0-9a-f]`), so every collation orders it identically — digits before letters, hex letters among themselves; - FIXED width on numeric and date domains (130 chars for `integer_ord`), where positional comparison equals numeric comparison only if the block count is constant; - per-character on text (16 hex chars each plus a 2-char header: `ada` -> 50, `adam` -> 66, `zebra` -> 82), so lexicographic order compares the plaintexts character by character and a prefix sorts first — the semantics of comparing the strings themselves. Change that encoding to variable-width numerics, uppercase, or base64, and ordering through PostgREST silently stops matching `ord_term()`. That file is the tripwire. The Drizzle adapter emits `ord_term()` directly and is unaffected. --- .changeset/supabase-v3-order-by-ope-term.md | 10 +- .../__tests__/supabase-v3-builder.test.ts | 2 +- .../__tests__/supabase-v3-matrix.test.ts | 2 +- .../integration/ope-term.integration.test.ts | 99 +++++++++++++++++++ .../stack/src/supabase/query-builder-v3.ts | 18 +++- 5 files changed, 126 insertions(+), 5 deletions(-) create mode 100644 packages/stack/integration/ope-term.integration.test.ts diff --git a/.changeset/supabase-v3-order-by-ope-term.md b/.changeset/supabase-v3-order-by-ope-term.md index 320216c6f..73e7e3977 100644 --- a/.changeset/supabase-v3-order-by-ope-term.md +++ b/.changeset/supabase-v3-order-by-ope-term.md @@ -16,7 +16,7 @@ But the correct sort key is reachable without a function call. `eql_v3.ord_term` returns the domain's `op` term, and OPE is order-preserving, so ordering by the term reproduces the plaintext order. PostgREST cannot emit `ORDER BY eql_v3.ord_term(col)`, but it can emit a jsonb path. The builder now -emits `order=col->>op` for an encrypted ordering column, verified against a live +emits `order=col->op` for an encrypted ordering column, verified against a live PostgREST for `integer_ord` and `text_search` in both directions. The guard is now on the ordering FLAVOUR, not on encryption: @@ -30,6 +30,14 @@ The guard is now on the ordering FLAVOUR, not on encryption: - **neither → rejected.** Storage-only, equality-only and match-only columns carry no ordering term. +The path is `col->op` (jsonb), not `col->>op` (text). Neither avoids the +database collation — Postgres compares jsonb strings with `varstr_cmp` under the +default collation, exactly as it does text. What makes the ordering +collation-independent is the term's encoding: lowercase hex, fixed-width for +numeric and date domains, and per-character (16 hex chars each) for text, so +lexicographic order reproduces plaintext order including the prefix case +(`ada` < `adam`). `ope-term.integration.test.ts` pins that shape. + `V3OrderableKeys` widens to match, so `order()` on an ordering column typechecks. `is(col, true)` is unaffected — it stays plaintext-only, and now has its own `V3PlaintextKeys` rather than borrowing the orderable set. diff --git a/packages/stack/__tests__/supabase-v3-builder.test.ts b/packages/stack/__tests__/supabase-v3-builder.test.ts index da561d618..c879f5b2e 100644 --- a/packages/stack/__tests__/supabase-v3-builder.test.ts +++ b/packages/stack/__tests__/supabase-v3-builder.test.ts @@ -362,7 +362,7 @@ describe('encryptedSupabaseV3 wire encoding', () => { expect(error).toBeNull() // The jsonb path, not a bare column: `ORDER BY created_at` would sort the // ciphertext envelope through jsonb's default opclass. - expect(supabase.callsFor('order')[0].args[0]).toBe('created_at->>op') + expect(supabase.callsFor('order')[0].args[0]).toBe('created_at->op') }) it('rejects order() on an equality-only encrypted column', async () => { diff --git a/packages/stack/__tests__/supabase-v3-matrix.test.ts b/packages/stack/__tests__/supabase-v3-matrix.test.ts index 7ecf052c5..876fe4ee3 100644 --- a/packages/stack/__tests__/supabase-v3-matrix.test.ts +++ b/packages/stack/__tests__/supabase-v3-matrix.test.ts @@ -172,7 +172,7 @@ describe('supabase v3 wire encoding, every domain', () => { expect(error).toBeNull() const [order] = supabase.callsFor('order') - expect(order.args[0]).toBe(`${name}->>op`) + expect(order.args[0]).toBe(`${name}->op`) }) } else { it('rejects order() even though gte() is supported', async () => { diff --git a/packages/stack/integration/ope-term.integration.test.ts b/packages/stack/integration/ope-term.integration.test.ts new file mode 100644 index 000000000..64c6afa0a --- /dev/null +++ b/packages/stack/integration/ope-term.integration.test.ts @@ -0,0 +1,99 @@ +import { expect, it } from 'vitest' +import { EncryptionV3 } from '@/encryption/v3' +import { encryptedTable, types } from '@/eql/v3' + +/** + * The property that makes `order=col->op` correct, asserted on the term itself. + * + * The Supabase adapter orders an encrypted column by its `op` term where it sits + * inside the envelope, because PostgREST cannot emit the canonical + * `ORDER BY eql_v3.ord_term(col)` — it cannot call a function. `ord_term()` + * returns `eql_v3_internal.ope_cllw`, a domain over `bytea`, ordered by the + * native btree. The jsonb path orders the same term as a STRING. + * + * Those agree only because of how the term is encoded. Postgres compares jsonb + * strings with `varstr_cmp` under the database's default collation — `->` does + * NOT avoid collation any more than `->>` does (measured: with `en_US.UTF-8`, + * both order `'a'` before `'B'`, while `COLLATE "C"` does the reverse). What + * saves us is that the term is FIXED-WIDTH LOWERCASE HEX: + * + * - fixed width, so lexicographic order is positional order — no prefix effects; + * - `[0-9a-f]` only, and every collation orders digits before letters and hex + * letters among themselves. + * + * Change the term's encoding — variable width, uppercase, base64 — and ordering + * through PostgREST silently stops matching `ord_term()`. This test is the + * tripwire. The Drizzle adapter emits `ord_term()` directly and is unaffected. + */ +const table = encryptedTable('ope_term_probe', { + amount: types.IntegerOrd('amount'), + when: types.TimestampOrd('when'), + name: types.TextSearch('name'), +}) + +const HEX = /^[0-9a-f]+$/ + +async function opTerms( + values: readonly unknown[], + column: 'amount' | 'when' | 'name', +) { + const client = await EncryptionV3({ schemas: [table] }) + const terms: string[] = [] + for (const value of values) { + const result = await client.encrypt(value as never, { + column: table[column], + table, + }) + if (result.failure) throw new Error(result.failure.message) + terms.push((result.data as { op: string }).op) + } + return terms +} + +it('integer_ord op terms are fixed-width lowercase hex', async () => { + const terms = await opTerms([-2147483648, -1, 0, 1, 2147483647], 'amount') + + expect(terms.every((term) => HEX.test(term))).toBe(true) + expect(new Set(terms.map((term) => term.length)).size).toBe(1) +}, 300_000) + +it('timestamp_ord op terms are fixed-width lowercase hex', async () => { + const stamps = await opTerms( + [new Date('1970-01-01T00:00:00Z'), new Date('2026-07-10T12:34:56Z')], + 'when', + ) + + expect(stamps.every((term) => HEX.test(term))).toBe(true) + expect(new Set(stamps.map((term) => term.length)).size).toBe(1) +}, 300_000) + +/** + * Text terms are VARIABLE width — 16 hex chars per character, plus a 2-char + * header (`'ada'` -> 50, `'zebra'` -> 82). That is not a defect: the blocks are + * positional, so comparing the terms as strings compares the plaintexts + * character by character, and a shorter term that is a prefix of a longer one + * sorts first — exactly the semantics of comparing the strings themselves. + * + * Numeric and date terms must stay FIXED width, because there positional + * comparison is only equivalent to numeric comparison when every term has the + * same number of blocks. + */ +it('text op terms are hex, per-character, and order like their plaintexts', async () => { + // Includes the prefix case (`ada` < `adam`), which is the one variable width + // could plausibly get wrong. + const ascending = ['ada', 'adam', 'zebra'] + const terms = await opTerms(ascending, 'name') + + expect(terms.every((term) => HEX.test(term))).toBe(true) + expect(terms.map((term) => term.length)).toEqual([50, 66, 82]) + expect([...terms].sort()).toEqual(terms) +}, 300_000) + +it('lexicographic order over the op terms reproduces plaintext order', async () => { + // The whole point, stated directly: sorting the terms as strings — which is + // what PostgREST does — must agree with sorting the plaintexts. + const ascending = [-2147483648, -1, 0, 1, 2147483647] + const terms = await opTerms(ascending, 'amount') + + expect([...terms].sort()).toEqual(terms) +}, 300_000) diff --git a/packages/stack/src/supabase/query-builder-v3.ts b/packages/stack/src/supabase/query-builder-v3.ts index be119191a..51bd7d230 100644 --- a/packages/stack/src/supabase/query-builder-v3.ts +++ b/packages/stack/src/supabase/query-builder-v3.ts @@ -281,11 +281,25 @@ export class EncryptedQueryBuilderV3Impl< /** * Encrypted ordering columns sort by their `op` term, not by the envelope. * - * `order=col->>op` is the one ordering expression PostgREST can emit that + * `order=col->op` is the one ordering expression PostgREST can emit that * reaches the OPE term. It must NOT leak into filters — those compare whole * envelopes through the `eql_v3.*` operators — which is why this is its own * seam rather than a change to `filterColumnName`. * + * The canonical EQL form is `ORDER BY eql_v3.ord_term(col)`, which returns + * `eql_v3_internal.ope_cllw` — a domain over `bytea`, ordered by the native + * btree. PostgREST cannot call a function, so it orders the `op` term where it + * sits, inside the envelope. The two agree because the term is what + * `ord_term()` returns. + * + * `->` (jsonb) rather than `->>` (text) keeps the comparison on the typed + * value. Note this does NOT avoid the database collation: Postgres compares + * jsonb strings with `varstr_cmp` under the default collation, exactly as it + * does text. What makes the ordering collation-independent is the term itself + * — fixed-width lowercase hex (`[0-9a-f]`, 130 chars for `integer_ord`, 82 for + * `text_search`) — and every collation orders digits before letters and hex + * letters among themselves. `match-bloom`'s sibling assertion pins that shape. + * * `validateTransforms` has already rejected every encrypted column that lacks * an `ope` index, so reaching the jsonb path here implies the term exists. */ @@ -295,7 +309,7 @@ export class EncryptedQueryBuilderV3Impl< if (!encrypted) return dbName as DbName return ( - this.columnSchemas[dbName]?.indexes?.ope ? `${dbName}->>op` : dbName + this.columnSchemas[dbName]?.indexes?.ope ? `${dbName}->op` : dbName ) as DbName } From a3152fbd513a795e04aa294fe2c994ec883d2a4e Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 11 Jul 2026 00:17:16 +1000 Subject: [PATCH 13/20] test(integration): port the Drizzle v3 suite onto the shared harness MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both adapters are now driven by one catalog, one plaintext oracle and one driver, so they cannot claim different coverage. Drizzle: 617 tests. Supabase: 665. Where they legitimately differ they differ in `supportedOps`, not in what a shared test asserts. Ordering is where they finally agree. Drizzle emits the canonical `ORDER BY eql_v3.ord_term(col)`; Supabase orders the same `op` term through the jsonb path `col->op`. Both satisfy the identical oracle, on every OPE-backed domain, in both directions. `operators-live-pg.test.ts` becomes `drizzle-v3/relational.integration.test.ts`, de-gated. Its per-domain `it.each` blocks — eq/ne/inArray/notInArray, ranges, between, asc/desc, contains, and the capability rejections — are deleted: the driver derives all of them from the catalog. What remains is what the driver cannot express, because it is about SQL shape rather than about a domain: `and`/`or`/`not` over disjoint predicates, `exists`/`notExists`, joins, pagination, the free-text needle guards, and a typed bigint round-trip. Two silent holes found while porting, both the failure this PR exists to kill: - The raw `in`-list variant was running against Drizzle, which has one `inArray` and no raw-filter path — 38 duplicate tests under a name that means nothing there. `IntegrationAdapter.hasRawInListPath` now declares it. - `'%s contains rejects a non-ASCII needle in the ORE term'` drove `matchDomains.filter(spec.indexes.ore)`. Since the OPE re-pin no match domain carries an `ore` index — `text_search` is `unique + ope + match` — so the `it.each` had ZERO cases and reported nothing. A vacuous test reads exactly like a passing one. Replaced with an assertion that the set is empty, so if an ORE-flavoured match domain ever returns, whoever adds it must restore the needle test with it. `integration-drizzle.yml` runs the suites on plain Postgres, fork-PR gated, `PGRST_URL` deliberately unset so a Supabase suite scoped here would throw. GitHub Actions does not support YAML anchors, so the path filters are repeated. --- .github/workflows/integration-drizzle.yml | 89 +++++ .../stack/integration/drizzle-v3/adapter.ts | 221 +++++++++++ .../drizzle-v3/families.integration.test.ts | 12 + .../relational.integration.test.ts} | 345 +++--------------- .../stack/integration/supabase/adapter.ts | 2 + packages/test-kit/src/adapter.ts | 11 + packages/test-kit/src/run-family-suite.ts | 8 +- 7 files changed, 382 insertions(+), 306 deletions(-) create mode 100644 .github/workflows/integration-drizzle.yml create mode 100644 packages/stack/integration/drizzle-v3/adapter.ts create mode 100644 packages/stack/integration/drizzle-v3/families.integration.test.ts rename packages/stack/{__tests__/drizzle-v3/operators-live-pg.test.ts => integration/drizzle-v3/relational.integration.test.ts} (68%) diff --git a/.github/workflows/integration-drizzle.yml b/.github/workflows/integration-drizzle.yml new file mode 100644 index 000000000..ac2d6c620 --- /dev/null +++ b/.github/workflows/integration-drizzle.yml @@ -0,0 +1,89 @@ +name: Integration — Drizzle (EQL v3) + +# Real ZeroKMS ciphertext against a real Postgres. The Drizzle adapter talks +# straight to the database, so no PostgREST here — that is the Supabase job. +# +# Separate from `tests.yml` on purpose: these suites need CipherStash credentials +# and a database, and they THROW rather than skip when unconfigured. Keeping them +# out of the unit job is what lets `pnpm test` stay runnable with neither. + +on: + push: + branches: [main] + paths: + - 'packages/stack/src/eql/v3/**' + - 'packages/stack/integration/**' + - 'packages/test-kit/**' + - 'packages/cli/src/installer/**' + - 'local/docker-compose.postgres.yml' + - '.github/workflows/integration-drizzle.yml' + - '.github/actions/integration-setup/**' + pull_request: + branches: ['**'] + # Repeated verbatim: GitHub Actions does not support YAML anchors/aliases. + paths: + - 'packages/stack/src/eql/v3/**' + - 'packages/stack/integration/**' + - 'packages/test-kit/**' + - 'packages/cli/src/installer/**' + - 'local/docker-compose.postgres.yml' + - '.github/workflows/integration-drizzle.yml' + - '.github/actions/integration-setup/**' + +jobs: + integration: + name: Drizzle v3 integration (db=${{ matrix.db }}) + runs-on: blacksmith-4vcpu-ubuntu-2404 + # Fork PRs have no secrets. Skip cleanly rather than fail on something the + # contributor cannot fix — `tests.yml` still gives them a green signal. + if: ${{ github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository }} + + strategy: + # A one-element matrix, not a cross-product: the Drizzle adapter only runs + # against plain Postgres. Kept as a matrix so adding a variant is one line. + matrix: + db: [postgres] + + env: + CS_WORKSPACE_CRN: ${{ secrets.CS_WORKSPACE_CRN }} + CS_CLIENT_ID: ${{ secrets.CS_CLIENT_ID }} + CS_CLIENT_KEY: ${{ secrets.CS_CLIENT_KEY }} + CS_CLIENT_ACCESS_KEY: ${{ secrets.CS_CLIENT_ACCESS_KEY }} + # Job-level env, not a `.env` file: `dotenv/config` does not override an + # already-set `process.env`, so these win and no secret is written to disk. + DATABASE_URL: postgres://cipherstash:password@localhost:55432/cipherstash + # Scoped to the suites this database serves. `PGRST_URL` is deliberately + # unset — the Supabase suites would throw here, which is the point. + CS_IT_SUITE: >- + integration/drizzle-v3/**/*.integration.test.ts, + integration/harness.integration.test.ts, + integration/ope-term.integration.test.ts, + integration/match-bloom.integration.test.ts + + steps: + - uses: actions/checkout@v6 + - uses: ./.github/actions/integration-setup + + # Fast pre-flight: fail in seconds if a secret was rotated or cleared, + # before the docker pull. The in-test `requireIntegrationEnv` is the + # correctness guarantee; this is the cheap one. + - name: Require CipherStash secrets + uses: ./.github/actions/require-cs-secrets + with: + workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }} + client-id: ${{ secrets.CS_CLIENT_ID }} + client-key: ${{ secrets.CS_CLIENT_KEY }} + client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }} + + - name: Start ${{ matrix.db }} + run: docker compose -f local/docker-compose.${{ matrix.db }}.yml up -d --wait + + # `globalSetup` installs EQL v3 by shelling out to the real + # `stash eql install --eql-version 3`, so an installer regression fails + # here rather than hiding behind a test-only SQL apply. + - name: Drizzle v3 integration suites + run: pnpm --filter @cipherstash/stack run test:integration + + - name: Stop ${{ matrix.db }} + if: always() + run: docker compose -f local/docker-compose.${{ matrix.db }}.yml down -v diff --git a/packages/stack/integration/drizzle-v3/adapter.ts b/packages/stack/integration/drizzle-v3/adapter.ts new file mode 100644 index 000000000..6cc35a172 --- /dev/null +++ b/packages/stack/integration/drizzle-v3/adapter.ts @@ -0,0 +1,221 @@ +import { + databaseUrl, + type IntegrationAdapter, + type PlainRow, + type QueryOp, + type QueryOpKind, + type TableSpec, +} from '@cipherstash/test-kit' +import { asc, type SQL } from 'drizzle-orm' +import { type PgTable, pgTable, text } from 'drizzle-orm/pg-core' +import { drizzle } from 'drizzle-orm/postgres-js' +import postgres from 'postgres' +import { EncryptionV3 } from '@/encryption/v3' +import { + createEncryptionOperatorsV3, + extractEncryptionSchemaV3, +} from '@/eql/v3/drizzle' +import { makeEqlV3Column } from '@/eql/v3/drizzle/column' + +/** + * The Drizzle v3 adapter under real ZeroKMS ciphertext. + * + * The counterpart to the Supabase adapter, driven by the same catalog, the same + * plaintext oracle and the same driver — so the two cannot quietly cover + * different operations while both read as "comprehensive". Where they legitimately + * differ, they differ in `supportedOps`, not in what a shared test asserts. + * + * Ordering is the interesting case. Drizzle emits the canonical EQL form, + * `ORDER BY eql_v3.ord_term(col)`, straight from `sql-dialect.ts`. Supabase + * cannot call a function through PostgREST and orders the same term through the + * jsonb path `col->op`. Both must satisfy the identical oracle. + */ + +const SUPPORTED_OPS: ReadonlySet = new Set([ + 'eq', + 'ne', + 'in', + 'notIn', + 'gt', + 'gte', + 'lt', + 'lte', + 'between', + 'notBetween', + 'contains', + 'order', + 'isNull', + 'isNotNull', +]) + +/** Drizzle talks straight to Postgres; nothing is refused adapter-wide. */ +const ALWAYS_REJECTED: ReadonlySet = new Set() + +// biome-ignore lint/suspicious/noExplicitAny: the table shape is built per family at runtime +type AnyTable = any + +export function makeDrizzleAdapter(): IntegrationAdapter { + let sqlClient: postgres.Sql + let db: ReturnType + let client: Awaited> + let ops: ReturnType + let table: AnyTable + let schema: ReturnType + let tableName: string + + const column = (slug: string) => table[slug] + + /** Every op except `order` resolves to a WHERE condition. */ + const conditionFor = async (op: QueryOp): Promise => { + const col = column(op.column) + switch (op.kind) { + case 'eq': + return ops.eq(col, op.value as never) + case 'ne': + return ops.ne(col, op.value as never) + case 'gt': + return ops.gt(col, op.value as never) + case 'gte': + return ops.gte(col, op.value as never) + case 'lt': + return ops.lt(col, op.value as never) + case 'lte': + return ops.lte(col, op.value as never) + case 'in': + // Drizzle has no raw-filter path; `asRawFilter` is a Supabase concern. + return ops.inArray(col, [...op.values] as never) + case 'notIn': + return ops.notInArray(col, [...op.values] as never) + case 'between': + return ops.between(col, op.lo as never, op.hi as never) + case 'notBetween': + return ops.notBetween(col, op.lo as never, op.hi as never) + case 'contains': + return ops.contains(col, op.needle as never) + case 'isNull': + return ops.isNull(col) + case 'isNotNull': + return ops.isNotNull(col) + case 'order': + throw new Error('order is a transform, not a condition') + } + } + + const selectRowKeys = async (where: SQL | undefined, orderBy: SQL[]) => { + const rows = (await db + .select({ rowKey: table.rowKey }) + .from(table as PgTable) + .where(where) + .orderBy(...orderBy)) as Array<{ rowKey: string }> + return rows.map((row) => row.rowKey) + } + + return { + name: 'drizzle', + supportedOps: SUPPORTED_OPS, + alwaysRejectedOps: ALWAYS_REJECTED, + + async setup() { + sqlClient = postgres(databaseUrl(), { prepare: false }) + db = drizzle({ client: sqlClient }) + // `client` and `ops` are built in `createTable`: the encryption client is + // constructed from the table's schema, which does not exist yet. + }, + + async teardown() { + if (tableName) await sqlClient.unsafe(`DROP TABLE IF EXISTS ${tableName}`) + await sqlClient.end() + }, + + async createTable(spec: TableSpec) { + tableName = spec.name + + const columns = Object.fromEntries( + spec.columns.map((c) => [ + c.slug, + makeEqlV3Column(c.spec.builder(c.slug)), + ]), + ) + table = pgTable(spec.name, { + rowKey: text('row_key').primaryKey(), + ...columns, + }) + schema = extractEncryptionSchemaV3(table) + + // The DDL comes from the column builders, not from a hand-written list, so + // a domain rename cannot silently desync the table from the schema. + const ddl = spec.columns + .map((c) => `"${c.slug}" ${c.eqlType}`) + .join(',\n ') + await sqlClient.unsafe(`DROP TABLE IF EXISTS ${spec.name}`) + await sqlClient.unsafe(` + CREATE TABLE ${spec.name} ( + row_key TEXT PRIMARY KEY, + ${ddl} + ) + `) + + // The client must know this table's schema to encrypt for it. Rebuilt per + // family, since each family has its own columns. + client = await EncryptionV3({ schemas: [schema as never] }) + ops = createEncryptionOperatorsV3(client) + }, + + async insertSingle(_spec: TableSpec, row: PlainRow) { + const result = await client.encryptModel( + { rowKey: row.rowKey, ...row.values } as never, + schema as never, + ) + if (result.failure) { + throw new Error( + `insertSingle(${row.rowKey}): ${result.failure.message}`, + ) + } + await db.insert(table as PgTable).values(result.data as never) + }, + + async insertBulk(_spec: TableSpec, rows: readonly PlainRow[]) { + const models = rows.map((row) => ({ rowKey: row.rowKey, ...row.values })) + const result = await client.bulkEncryptModels( + models as never, + schema as never, + ) + if (result.failure) + throw new Error(`insertBulk: ${result.failure.message}`) + await db.insert(table as PgTable).values(result.data as never) + }, + + async run(_spec: TableSpec, op: QueryOp): Promise { + if (op.kind === 'order') { + const col = column(op.column) + const term = op.direction === 'asc' ? ops.asc(col) : ops.desc(col) + // Exclude the all-NULL row, and break ties on `row_key` — the oracle + // does both. Domains with fewer distinct samples than rows give two rows + // the same plaintext, so the term alone does not determine their order. + return selectRowKeys(await ops.isNotNull(col), [ + term as SQL, + asc(table.rowKey), + ]) + } + + return selectRowKeys(await conditionFor(op), [asc(table.rowKey)]) + }, + + async expectRejected(_spec: TableSpec, op: QueryOp) { + try { + if (op.kind === 'order') { + const col = column(op.column) + // `asc`/`desc` throw synchronously on a column with no ordering term. + void (op.direction === 'asc' ? ops.asc(col) : ops.desc(col)) + } else { + await conditionFor(op) + } + } catch { + return // an `EncryptionOperatorError` is the rejection + } + throw new Error( + `Expected ${op.kind}("${op.column}") to be rejected, but it succeeded`, + ) + }, + } +} diff --git a/packages/stack/integration/drizzle-v3/families.integration.test.ts b/packages/stack/integration/drizzle-v3/families.integration.test.ts new file mode 100644 index 000000000..e04683011 --- /dev/null +++ b/packages/stack/integration/drizzle-v3/families.integration.test.ts @@ -0,0 +1,12 @@ +import { FAMILY_NAMES } from '@cipherstash/test-kit' +import { runFamilySuite } from '@cipherstash/test-kit/suite' +import { makeDrizzleAdapter } from './adapter' + +/** + * Every EQL v3 domain the SDK models, driven through the Drizzle adapter against + * real ZeroKMS ciphertext — the same catalog, oracle and driver as the Supabase + * suite, so the two cannot claim different coverage. + */ +for (const family of FAMILY_NAMES) { + runFamilySuite(family, makeDrizzleAdapter) +} diff --git a/packages/stack/__tests__/drizzle-v3/operators-live-pg.test.ts b/packages/stack/integration/drizzle-v3/relational.integration.test.ts similarity index 68% rename from packages/stack/__tests__/drizzle-v3/operators-live-pg.test.ts rename to packages/stack/integration/drizzle-v3/relational.integration.test.ts index 3283cb46b..dc90c6ffc 100644 --- a/packages/stack/__tests__/drizzle-v3/operators-live-pg.test.ts +++ b/packages/stack/integration/drizzle-v3/relational.integration.test.ts @@ -1,4 +1,32 @@ -import 'dotenv/config' +/** + * The Drizzle v3 tests the family driver cannot express. + * + * Per-domain equality, ordering, ranges, containment and capability rejections + * moved to `families.integration.test.ts`, where they are derived from the + * catalog and shared with the Supabase adapter. What remains is everything that + * is about SQL shape rather than about a domain: + * + * - boolean combinators (`and`/`or`/`not`) over disjoint encrypted predicates + * - `exists` / `notExists` correlated subqueries + * - joins against plain tables while filtering encrypted columns + * - `limit`/`offset` pagination over an encrypted ordering + * - the free-text needle guards: a needle below `token_length` blooms to + * nothing and `stored_bf @> '{}'` holds for EVERY row, so before the guard a + * short needle silently returned the whole table + * - a statically-typed bigint round-trip + * + * EQL v3 is installed once per run by `globalSetup`, via the real + * `stash eql install`. This suite throws rather than skips when unconfigured. + */ + +import { + type DomainSpec, + databaseUrl, + type EqlV3TypeName, + eqlTypeSlug as slug, + typedEntries, + V3_MATRIX, +} from '@cipherstash/test-kit' import { and, asc as drizzleAsc, @@ -9,7 +37,7 @@ import { import { integer, pgTable, text } from 'drizzle-orm/pg-core' import { drizzle } from 'drizzle-orm/postgres-js' import postgres from 'postgres' -import { afterAll, beforeAll, expect, it } from 'vitest' +import { afterAll, beforeAll, describe, expect, it } from 'vitest' import { EncryptionV3 } from '@/encryption/v3' import { createEncryptionOperatorsV3, @@ -18,20 +46,8 @@ import { types as v3drizzle, } from '@/eql/v3/drizzle' import { makeEqlV3Column } from '@/eql/v3/drizzle/column' -import { installEqlV3IfNeeded } from '../helpers/eql-v3' -import { describeLivePg, LIVE_EQL_V3_PG_ENABLED } from '../helpers/live-gate' -import { - type DomainSpec, - type EqlV3TypeName, - eqlTypeSlug as slug, - typedEntries, - V3_MATRIX, -} from '../v3-matrix/catalog' -const url = process.env.DATABASE_URL -const sqlClient = LIVE_EQL_V3_PG_ENABLED - ? postgres(url as string, { prepare: false }) - : (undefined as unknown as postgres.Sql) +const sqlClient = postgres(databaseUrl(), { prepare: false }) const TABLE_NAME = 'protect_ci_v3_drizzle_matrix' const ACCOUNT_TABLE_NAME = 'protect_ci_v3_drizzle_matrix_accounts' @@ -226,8 +242,6 @@ function encryptedInsertRows(): MatrixPlainRow[] { } beforeAll(async () => { - if (!LIVE_EQL_V3_PG_ENABLED) return - await installEqlV3IfNeeded(sqlClient) client = await EncryptionV3({ schemas: [schema, bigintSchema] }) ops = createEncryptionOperatorsV3(client) db = drizzle({ client: sqlClient }) @@ -302,254 +316,13 @@ beforeAll(async () => { }, 120000) afterAll(async () => { - if (!LIVE_EQL_V3_PG_ENABLED) return await sqlClient`DELETE FROM ${sqlClient(TABLE_NAME)} WHERE test_run_id = ${RUN}` await sqlClient`DELETE FROM ${sqlClient(ACCOUNT_TABLE_NAME)} WHERE test_run_id = ${RUN}` await sqlClient`DELETE FROM ${sqlClient(BIGINT_TABLE_NAME)} WHERE test_run_id = ${RUN}` await sqlClient.end() }, 30000) -describeLivePg('v3 drizzle operators (live pg matrix)', () => { - it.each(equalityDomains)( - '%s eq selects the exact row', - async (eqlType, spec) => { - const bound = plainValue(spec, ROW_A) - const rows = await selectRowKeys( - await ops.eq(matrixColumn(eqlType), bound), - ) - // Oracle, not `[ROW_A]`: ROW_C carries a near-miss value for domains with - // a third sample, so an `eq` that over-matches now shows up here. - expect(rows).toEqual( - expectedKeysFor(spec, (value) => comparePlain(value, bound) === 0), - ) - }, - 30000, - ) - - it.each(equalityDomains)( - '%s ne selects the complement rows', - async (eqlType, spec) => { - const bound = plainValue(spec, ROW_A) - const rows = await selectRowKeys( - await ops.ne(matrixColumn(eqlType), bound), - ) - expect(rows).toEqual( - expectedKeysFor(spec, (value) => comparePlain(value, bound) !== 0), - ) - }, - 30000, - ) - - it.each(equalityDomains)( - '%s inArray selects all listed rows', - async (eqlType, spec) => { - const listed = [plainValue(spec, ROW_A), plainValue(spec, ROW_B)] - const rows = await selectRowKeys( - await ops.inArray(matrixColumn(eqlType), listed), - ) - expect(rows).toEqual( - expectedKeysFor(spec, (value) => - listed.some((entry) => comparePlain(value, entry) === 0), - ), - ) - }, - 30000, - ) - - it.each(equalityDomains)( - '%s notInArray excludes listed rows', - async (eqlType, spec) => { - const bound = plainValue(spec, ROW_A) - const rows = await selectRowKeys( - await ops.notInArray(matrixColumn(eqlType), [bound]), - ) - expect(rows).toEqual( - expectedKeysFor(spec, (value) => comparePlain(value, bound) !== 0), - ) - }, - 30000, - ) - - it.each(comparisonDomains)( - '%s %s selects rows by encrypted ordering', - async (eqlType, spec, operator, predicate) => { - const bound = plainValue(spec, ROW_A) - const rows = await selectRowKeys( - await ops[operator](matrixColumn(eqlType), bound), - ) - expect(rows).toEqual( - expectedKeysFor(spec, (value) => predicate(comparePlain(value, bound))), - ) - }, - 30000, - ) - - // The bounds span ROW_A and ROW_B. ROW_C carries a third sample that may sit - // outside that span (most domains) or tie with ROW_B (date, timestamp), so - // membership is computed rather than assumed — with only A and B seeded these - // were `[ROW_A, ROW_B]` and `[]`, which no longer holds. - const spanBounds = (spec: DomainSpec): [PlainValue, PlainValue] => { - const sorted = [plainValue(spec, ROW_A), plainValue(spec, ROW_B)].sort( - comparePlain, - ) - return [sorted[0], sorted[1]] - } - const withinSpan = (spec: DomainSpec) => (value: PlainValue) => { - const [min, max] = spanBounds(spec) - return comparePlain(value, min) >= 0 && comparePlain(value, max) <= 0 - } - - it.each(orderDomains)( - '%s between selects the inclusive range', - async (eqlType, spec) => { - const [min, max] = spanBounds(spec) - const rows = await selectRowKeys( - await ops.between(matrixColumn(eqlType), min, max), - ) - expect(rows).toEqual(expectedKeysFor(spec, withinSpan(spec))) - }, - 30000, - ) - - it.each(orderDomains)( - '%s notBetween excludes the inclusive range', - async (eqlType, spec) => { - const [min, max] = spanBounds(spec) - const rows = await selectRowKeys( - await ops.notBetween(matrixColumn(eqlType), min, max), - ) - expect(rows).toEqual( - expectedKeysFor(spec, (value) => !withinSpan(spec)(value)), - ) - }, - 30000, - ) - - // The spanning cases above put ROW_A and ROW_B inside the range, so on most - // domains only ROW_C proves exclusion. These narrow cases use a single-point - // range at ROW_A's value to prove the operators EXCLUDE regardless of where - // ROW_C falls: `between` must drop ROW_B, `notBetween` must keep it. Without - // these, a `between` that matched everything (or a `notBetween` no-op) passes. - it.each(orderDomains)( - '%s between at a single point excludes the out-of-range row', - async (eqlType, spec) => { - const bound = plainValue(spec, ROW_A) - const rows = await selectRowKeys( - await ops.between(matrixColumn(eqlType), bound, bound), - ) - expect(rows).toEqual( - expectedKeysFor(spec, (value) => comparePlain(value, bound) === 0), - ) - }, - 30000, - ) - - it.each(orderDomains)( - '%s notBetween at a single point keeps the out-of-range row', - async (eqlType, spec) => { - const bound = plainValue(spec, ROW_A) - const rows = await selectRowKeys( - await ops.notBetween(matrixColumn(eqlType), bound, bound), - ) - expect(rows).toEqual( - expectedKeysFor(spec, (value) => comparePlain(value, bound) !== 0), - ) - }, - 30000, - ) - - // `between` returns a two-function conjunction; Drizzle's passthrough `not` - // prepends a bare `not` with no parentheses of its own. Postgres binds NOT - // tighter than AND, so an unparenthesised range would run as - // `(NOT gte(col, bound)) AND lte(col, bound)` — i.e. `col < bound` — instead - // of the range's complement, `col != bound`. - // - // The bound MUST be the SMALLER of the two seeded values. Anchored at the - // larger one, `col < bound` and `col != bound` both select the other row and - // the buggy SQL passes; anchored at the smaller, the buggy SQL selects - // nothing while the correct SQL selects the larger row. Several domains seed - // ROW_A with their maximum sample (`integer_ord` is `[0, -42]`), so keying - // off ROW_A directly would make this test vacuous for them. - it.each(orderDomains)( - '%s not(between(...)) negates the whole range', - async (eqlType, spec) => { - const bound = [plainValue(spec, ROW_A), plainValue(spec, ROW_B)].sort( - comparePlain, - )[0] - const rows = await selectRowKeys( - ops.not(await ops.between(matrixColumn(eqlType), bound, bound)), - ) - expect(rows).toEqual( - expectedKeysFor(spec, (value) => comparePlain(value, bound) !== 0), - ) - // Guards the guard: the expectation must be non-empty, or the buggy - // `col < bound` (which returns nothing) would satisfy it too. - expect(rows.length).toBeGreaterThan(0) - }, - 30000, - ) - - // The secondary `ORDER BY row_key` mirrors the oracle's tie-break. Domains - // with only two samples (date, timestamp) give ROW_B and ROW_C equal values, - // so without it the tied pair's order is arbitrary in Postgres. - it.each(orderDomains)( - '%s asc orders by encrypted order term', - async (eqlType, spec) => { - const rows = (await db - .select({ rowKey: matrixTable.rowKey }) - .from(matrixTable) - .where(drizzleEq(matrixTable.testRunId, RUN)) - .orderBy( - ops.asc(matrixColumn(eqlType)), - drizzleAsc(matrixTable.rowKey), - )) as SelectRow[] - expect(rows.map((row) => row.rowKey)).toEqual(sortedKeysFor(spec, 'asc')) - }, - 30000, - ) - - it.each(orderDomains)( - '%s desc orders by encrypted order term', - async (eqlType, spec) => { - const rows = (await db - .select({ rowKey: matrixTable.rowKey }) - .from(matrixTable) - .where(drizzleEq(matrixTable.testRunId, RUN)) - .orderBy( - ops.desc(matrixColumn(eqlType)), - drizzleAsc(matrixTable.rowKey), - )) as SelectRow[] - expect(rows.map((row) => row.rowKey)).toEqual(sortedKeysFor(spec, 'desc')) - }, - 30000, - ) - - // Needles are driven through the same substring oracle, so each domain gets - // the rows it should. `'ada'` is exactly `token_length`; `'lovelace'` and - // `'grace'` are longer (the suite previously had no `contains` proof above - // the token length at all); `'qqqzzz'` is present in no row, so a `contains` - // that matched everything would fail here rather than pass silently. - it.each( - matchDomains.flatMap(([eqlType, spec]) => - ['ada', 'lovelace', 'grace', 'qqqzzz'].map( - (needle) => [eqlType, spec, needle] as const, - ), - ), - )( - '%s contains %s matches exactly the rows holding it', - async (eqlType, spec, needle) => { - const rows = await selectRowKeys( - await ops.contains(matrixColumn(eqlType), needle), - ) - expect(rows).toEqual( - expectedKeysFor(spec, (value) => - String(value).toLowerCase().includes(needle), - ), - ) - }, - 30000, - ) - +describe('v3 drizzle — relational, needle guards, pagination', () => { // A needle below the tokenizer's `token_length` builds an EMPTY bloom filter, // and `stored_bf @> '{}'` holds for every row — so before the SDK guard this // silently returned the whole table. @@ -620,50 +393,18 @@ describeLivePg('v3 drizzle operators (live pg matrix)', () => { 30000, ) - // An `ore`-bearing match column rejects a non-ASCII needle inside ENCRYPTION, - // not in the needle guard. Pinned so the distinction stays visible: the guard - // is about tokenization, this is about the block-ORE term's ASCII-only - // ordering. OPE-backed match domains (text_search) encrypt non-ASCII fine — - // only the `ore`-flavoured ones carry the restriction. - it.each( - matchDomains.filter(([, spec]) => spec.indexes.ore), - )('%s contains rejects a non-ASCII needle in the ORE term, not the guard', async (eqlType) => { - const attempt = ops.contains( - matrixColumn(eqlType), - '\u{1F44D}\u{1F44D}\u{1F44D}', - ) - await expect(attempt).rejects.toThrow(/pure ASCII/) - await expect(attempt).rejects.not.toThrow(/free-text search needs/) - }, 30000) - - it.each( - noEqualityDomains, - )('%s eq rejects unsupported equality', async (eqlType, spec) => { - await expect( - ops.eq(matrixColumn(eqlType), plainValue(spec, ROW_A)), - ).rejects.toBeInstanceOf(EncryptionOperatorError) - }) - - it.each( - noOrderDomains, - )('%s gt rejects unsupported ordering', async (eqlType, spec) => { - await expect( - ops.gt(matrixColumn(eqlType), plainValue(spec, ROW_A)), - ).rejects.toBeInstanceOf(EncryptionOperatorError) - }) - - it.each(noOrderDomains)('%s asc rejects unsupported ordering', (eqlType) => { - expect(() => ops.asc(matrixColumn(eqlType))).toThrow( - EncryptionOperatorError, - ) - }) + // The non-ASCII ORE-needle test that used to live here drove + // `matchDomains.filter(([, spec]) => spec.indexes.ore)`. Since the eql-3.0.0 + // OPE re-pin, NO match domain carries an `ore` index — `text_search` is + // `unique + ope + match`, `text_match` is `match` — so that `it.each` had zero + // cases and reported nothing. A vacuous test reads exactly like a passing one. + // + // Assert the state instead. If an ORE-flavoured match domain ever returns, this + // fails and whoever adds it has to restore the needle test with it. + it('no match domain carries an ORE index, so no ASCII-only needle rule applies', () => { + const oreMatchDomains = matchDomains.filter(([, spec]) => spec.indexes.ore) - it.each( - noMatchDomains, - )('%s contains rejects unsupported match', async (eqlType, spec) => { - await expect( - ops.contains(matrixColumn(eqlType), plainValue(spec, ROW_A)), - ).rejects.toBeInstanceOf(EncryptionOperatorError) + expect(oreMatchDomains.map(([eqlType]) => eqlType)).toEqual([]) }) // The two predicates must be satisfied by DISJOINT row sets, or `and` and diff --git a/packages/stack/integration/supabase/adapter.ts b/packages/stack/integration/supabase/adapter.ts index 63037cdee..657f5885c 100644 --- a/packages/stack/integration/supabase/adapter.ts +++ b/packages/stack/integration/supabase/adapter.ts @@ -130,6 +130,8 @@ export function makeSupabaseAdapter(): IntegrationAdapter { name: 'supabase', supportedOps: SUPPORTED_OPS, alwaysRejectedOps: ALWAYS_REJECTED, + // `.in()` and the raw `.filter(col, 'in', […])` are different code paths. + hasRawInListPath: true, async setup() { sql = postgres(databaseUrl(), { prepare: false }) diff --git a/packages/test-kit/src/adapter.ts b/packages/test-kit/src/adapter.ts index aabc574d8..4380cee96 100644 --- a/packages/test-kit/src/adapter.ts +++ b/packages/test-kit/src/adapter.ts @@ -27,6 +27,17 @@ export interface IntegrationAdapter { */ readonly alwaysRejectedOps: ReadonlySet + /** + * Whether the adapter has a SECOND, distinct code path for `in`-lists that the + * driver should exercise alongside the primary one. + * + * Supabase does: `.in()` and the raw `.filter(col, 'in', […])` are different + * code paths, and the raw one encrypted the whole list as a single term until + * recently. Drizzle has one `inArray`, so running the variant there would add + * a duplicate test under a name (`filter(in)`) that means nothing to it. + */ + readonly hasRawInListPath?: boolean + setup(): Promise teardown(): Promise diff --git a/packages/test-kit/src/run-family-suite.ts b/packages/test-kit/src/run-family-suite.ts index 05e9f8675..65841362a 100644 --- a/packages/test-kit/src/run-family-suite.ts +++ b/packages/test-kit/src/run-family-suite.ts @@ -220,10 +220,10 @@ export function runFamilySuite( } if (positive.has('in')) { - // Both the `in()` method and the raw `filter(col, 'in', [...])` path: - // they are different code paths, and the raw one encrypted the whole - // list as a single term until recently. - for (const asRawFilter of [false, true]) { + // Adapters with a second `in`-list code path drive both against the + // same oracle. See `IntegrationAdapter.hasRawInListPath`. + const variants = adapter.hasRawInListPath ? [false, true] : [false] + for (const asRawFilter of variants) { const label = asRawFilter ? 'filter(in)' : 'in()' it(`${label} selects the union of the listed values`, async () => { From 6c03e73b74538bb07e1b2ce0fa0c34fef31d120c Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 11 Jul 2026 00:32:31 +1000 Subject: [PATCH 14/20] fix(test-kit): reject a partially configured CipherStash environment MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot on #616: `hasCipherStashCredentials()` returned true on the presence of `CS_WORKSPACE_CRN` alone, so a partial environment passed a gate whose entire purpose is to fail loudly. Reproduced against a live suite. With `CS_WORKSPACE_CRN` set and the three client variables missing, `globalSetup` waved the run through and every test then died with `[encryption]: Not authenticated` — four identical failures, none naming a variable, none naming a fix. Exactly the class of failure the gate exists to prevent, and the shape a rotated or cleared CI secret takes. Credentials now resolve from EITHER all four `CS_*` variables OR a `~/.cipherstash` profile. A partial environment is neither, and is refused: CipherStash credentials are PARTIALLY configured — missing CS_CLIENT_ID, CS_CLIENT_KEY, CS_CLIENT_ACCESS_KEY. A partial environment is rejected rather than half-used: it would otherwise fail later with "[encryption]: Not authenticated", which names nothing. Either set all four (...), or unset them all and run `stash auth login`. It is refused rather than half-used because it is ambiguous, not merely incomplete: with a profile on disk there is no way to tell whether the caller meant to override one field or to use the environment wholesale. Refusing to guess is the honest answer. `test-kit-env.test.ts` covers the gate, which had no test at all — the one thing standing between a rotated secret and a confusing failure. Three of its seven tests fail against the old check: the partial-environment regression, the "names exactly the missing variables" contract, and empty-string handling (a cleared GitHub secret expands to `''`, not to unset, so `!process.env[x]` is the correct predicate and `x in process.env` would not be). --- packages/stack/__tests__/test-kit-env.test.ts | 129 ++++++++++++++++++ packages/test-kit/src/env.ts | 65 +++++++-- 2 files changed, 179 insertions(+), 15 deletions(-) create mode 100644 packages/stack/__tests__/test-kit-env.test.ts diff --git a/packages/stack/__tests__/test-kit-env.test.ts b/packages/stack/__tests__/test-kit-env.test.ts new file mode 100644 index 000000000..191d80101 --- /dev/null +++ b/packages/stack/__tests__/test-kit-env.test.ts @@ -0,0 +1,129 @@ +import { mkdirSync, mkdtempSync, rmSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { requireIntegrationEnv } from '@cipherstash/test-kit' +import { afterEach, beforeEach, describe, expect, it } from 'vitest' + +/** + * The credential gate is the only thing between a rotated secret and a run that + * fails once per test with `[encryption]: Not authenticated`. It had no test. + * + * A partial environment — `CS_WORKSPACE_CRN` set, the client keys missing — used + * to pass, because the check was `if (CS_WORKSPACE_CRN) return true`. Reported by + * Copilot on #616, reproduced against a live suite, fixed here. + * + * `homedir()` reads `$HOME` on POSIX, so pointing `HOME` at a scratch directory + * controls whether a `~/.cipherstash` profile appears to exist. + */ +const CS_VARS = [ + 'CS_WORKSPACE_CRN', + 'CS_CLIENT_ID', + 'CS_CLIENT_KEY', + 'CS_CLIENT_ACCESS_KEY', +] as const + +let home: string +let saved: Record + +function setEnv(values: Partial>) { + for (const name of CS_VARS) delete process.env[name] + for (const [name, value] of Object.entries(values)) process.env[name] = value +} + +function withProfile() { + mkdirSync(join(home, '.cipherstash'), { recursive: true }) +} + +const ALL_FOUR = Object.fromEntries( + CS_VARS.map((name) => [name, `value-of-${name}`]), +) as Record<(typeof CS_VARS)[number], string> + +beforeEach(() => { + home = mkdtempSync(join(tmpdir(), 'cs-env-')) + saved = { HOME: process.env['HOME'] } + for (const name of CS_VARS) saved[name] = process.env[name] + process.env['HOME'] = home +}) + +afterEach(() => { + for (const [name, value] of Object.entries(saved)) { + if (value === undefined) delete process.env[name] + else process.env[name] = value + } + rmSync(home, { recursive: true, force: true }) +}) + +describe('requireIntegrationEnv — cipherstash credentials', () => { + it('accepts all four variables, with no profile', () => { + setEnv(ALL_FOUR) + + expect(() => requireIntegrationEnv(['cipherstash'])).not.toThrow() + }) + + it('accepts a profile with no variables', () => { + setEnv({}) + withProfile() + + expect(() => requireIntegrationEnv(['cipherstash'])).not.toThrow() + }) + + it('rejects a partial environment even when a profile exists', () => { + // The regression. `CS_WORKSPACE_CRN` alone used to satisfy the gate, and the + // run then died inside `encrypt()` rather than here. + setEnv({ CS_WORKSPACE_CRN: 'crn:ap-southeast-2.aws:ABC' }) + withProfile() + + expect(() => requireIntegrationEnv(['cipherstash'])).toThrow( + /PARTIALLY configured/, + ) + }) + + it('names exactly the missing variables', () => { + setEnv({ CS_WORKSPACE_CRN: 'crn', CS_CLIENT_ID: 'id' }) + + expect(() => requireIntegrationEnv(['cipherstash'])).toThrow( + /missing CS_CLIENT_KEY, CS_CLIENT_ACCESS_KEY/, + ) + }) + + it('rejects an empty environment with no profile, and offers both routes', () => { + setEnv({}) + + let message = '' + try { + requireIntegrationEnv(['cipherstash']) + } catch (cause) { + message = (cause as Error).message + } + + expect(message).toContain('none are configured') + expect(message).toContain('stash auth login') + expect(message).not.toContain('PARTIALLY') + }) + + it('treats an empty-string variable as missing, not as set', () => { + // A cleared GitHub secret expands to the empty string, not to unset. + setEnv({ ...ALL_FOUR, CS_CLIENT_KEY: '' }) + + expect(() => requireIntegrationEnv(['cipherstash'])).toThrow( + /missing CS_CLIENT_KEY/, + ) + }) + + it('reports every unmet requirement at once, not one per run', () => { + setEnv({}) + delete process.env['DATABASE_URL'] + delete process.env['PGRST_URL'] + + let message = '' + try { + requireIntegrationEnv(['cipherstash', 'database', 'pgrest']) + } catch (cause) { + message = (cause as Error).message + } + + expect(message).toContain('CipherStash credentials') + expect(message).toContain('DATABASE_URL') + expect(message).toContain('PGRST_URL') + }) +}) diff --git a/packages/test-kit/src/env.ts b/packages/test-kit/src/env.ts index c03d1ae62..cb68fe83d 100644 --- a/packages/test-kit/src/env.ts +++ b/packages/test-kit/src/env.ts @@ -14,27 +14,62 @@ import { join } from 'node:path' export type Requirement = 'cipherstash' | 'database' | 'pgrest' +/** Every variable protect-ffi's `AutoStrategy` needs to authenticate from the environment. */ +const CS_VARS = [ + 'CS_WORKSPACE_CRN', + 'CS_CLIENT_ID', + 'CS_CLIENT_KEY', + 'CS_CLIENT_ACCESS_KEY', +] as const + +/** True when `stash auth login` has written a device profile. */ +function hasProfile(): boolean { + return existsSync(join(homedir(), '.cipherstash')) +} + /** - * CipherStash credentials resolve from EITHER the four `CS_*` variables OR a - * local `~/.cipherstash` profile written by `stash auth login`. protect-ffi - * builds an `AutoStrategy` that consults both, so a developer who has logged in - * once needs no environment at all; CI, which has no profile, supplies the vars. + * CipherStash credentials resolve from EITHER all four `CS_*` variables OR a + * local `~/.cipherstash` profile written by `stash auth login`. A developer who + * has logged in once needs no environment; CI, which has no profile, supplies + * the vars. + * + * A PARTIALLY configured environment is neither, and is rejected. Accepting it + * (as an earlier version did, on the presence of `CS_WORKSPACE_CRN` alone) let a + * rotated or cleared secret through the gate, and the run then failed once per + * test with `[encryption]: Not authenticated` — which names nothing and points + * nowhere. That is precisely the "fail loudly, and say how to fix it" contract + * this module exists to keep. * - * Mirrors `examples/prisma/test/e2e/global-setup.ts`, which already resolves - * credentials this way. + * The partial case is not merely under-configured, it is ambiguous: with a + * profile on disk it is unclear whether the caller meant to override one field + * or to use the environment wholesale. Refusing to guess is the honest answer. */ -function hasCipherStashCredentials(): boolean { - if (process.env['CS_WORKSPACE_CRN']) return true - return existsSync(join(homedir(), '.cipherstash')) +function cipherstashHint(): string | null { + const missing = CS_VARS.filter((name) => !process.env[name]) + + // Fully configured from the environment. Any profile is irrelevant. + if (missing.length === 0) return null + + // Nothing in the environment: the profile is the intended source. + if (missing.length === CS_VARS.length && hasProfile()) return null + + const preamble = + missing.length === CS_VARS.length + ? 'CipherStash credentials: none are configured.' + : `CipherStash credentials are PARTIALLY configured — missing ${missing.join(', ')}.\n` + + ' A partial environment is rejected rather than half-used: it would otherwise\n' + + ' fail later with "[encryption]: Not authenticated", which names nothing.' + + return ( + `${preamble}\n` + + ` Either set all four (${CS_VARS.join(', ')}),\n` + + ' or unset them all and run `stash auth login` once (writes ~/.cipherstash,\n' + + ' which AutoStrategy picks up).' + ) } const HINTS: Record string | null> = { - cipherstash: () => - hasCipherStashCredentials() - ? null - : 'CipherStash credentials. Either set CS_WORKSPACE_CRN, CS_CLIENT_ID, ' + - 'CS_CLIENT_KEY and CS_CLIENT_ACCESS_KEY, or run `stash auth login` once ' + - '(writes ~/.cipherstash, which AutoStrategy picks up).', + cipherstash: cipherstashHint, database: () => process.env['DATABASE_URL'] From cd981e87e3c1260b93d8f351464ae25c612ad4f5 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 11 Jul 2026 00:48:55 +1000 Subject: [PATCH 15/20] test(integration): show test logs only for failing tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A green Supabase integration run printed 213 ERROR blocks with stack traces. All 213 came from PASSING tests, and the run reported 665 passed / 0 failed. The job was honestly green and looked broken. The capability-rejection tests are the bulk of the suite: for each domain, the driver derives the operations its capabilities forbid and asserts the adapter refuses them. `boolean` is storage-only, so it gets twelve — including `order`, which is why the log shows an adapter refusing to order a bool. That is the test working. `EncryptedQueryBuilderImpl.execute` logs `logger.error(...)` before returning its `Result` error, so every passing rejection emits an ERROR block. `silent: 'passed-only'` suppresses console output from passing tests and keeps it for failing ones — which is when it is worth reading. Measured on the Supabase family suite: all passing: 10 output lines, 0 ERROR blocks 212 failures: 3510 output lines, 212 ERROR blocks (all shown) Confined to the integration vitest config. The stack logger bottoms out at `error` (`STASH_STACK_LOG` has no silent level), so suppressing at the runner is the only way to do this without changing what the product logs. --- packages/stack/integration/vitest.config.ts | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/packages/stack/integration/vitest.config.ts b/packages/stack/integration/vitest.config.ts index a80207052..f0f111231 100644 --- a/packages/stack/integration/vitest.config.ts +++ b/packages/stack/integration/vitest.config.ts @@ -63,5 +63,22 @@ export default defineConfig({ // One database, shared tables. File-level parallelism would have two family // suites installing EQL and reloading the PostgREST schema cache at once. fileParallelism: false, + + /** + * Show console output only for tests that FAIL. + * + * The capability-rejection tests are the bulk of the suite, and each one + * makes the adapter refuse an operation. `EncryptedQueryBuilderImpl.execute` + * logs `logger.error(...)` before returning its `Result` error, so every + * passing rejection emits an ERROR block with a stack trace. A CI run + * printed 213 of them — all from passing tests — which is enough noise to + * bury a real failure and enough to make a green job look broken. + * + * `'passed-only'` suppresses those and keeps the logs of any test that + * actually fails, which is when they are worth reading. The stack logger has + * no silent level (`STASH_STACK_LOG` bottoms out at `error`), so this is the + * only place to do it without changing product logging behaviour. + */ + silent: 'passed-only', }, }) From c78b60f407f679e2e42acd8cbe786a050353fad4 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 11 Jul 2026 01:02:51 +1000 Subject: [PATCH 16/20] ci(integration): run Drizzle on both databases; Node 24; drop-before-create MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two review comments from @coderdan on #616. **Run the Drizzle suites against Supabase too.** The adapter talks straight to Postgres, but it still has to work on managed Postgres, where `postgres` is not a superuser, the EQL install takes its self-skipping path, and the ORE domains cannot hold data. A suite that passes on a superuser database can fail there. The job is now a two-cell matrix over `db: [postgres, supabase]`. Wiring it up found two real defects, neither visible on plain Postgres: - `relational.integration.test.ts` built its table from all 39 catalog rows, including the nine `deferred` `_ord_ore` domains. On managed Postgres the seed INSERT raised `ore_domain_unavailable`. It now filters on `isCovered`, which is what the field is for. - Worse, it created that table with `CREATE TABLE IF NOT EXISTS`. A table left by an earlier run keeps its old columns, so after the filter the leftover table still carried its nine ORE columns and every INSERT still raised — even leaving them NULL, because the domain CHECK calls a function that RAISEs rather than returning false. Silent schema drift across runs. Now DROP-then-CREATE. **Node 24 in the integration setup action**, was 22. It is the newest major `tests.yml` exercises and what the repo develops on; `engines` is `>=22` and the unit job still covers 22 in its matrix. Not 26 — nothing in the repo uses it yet, and a default should not be the first place a new major appears. Verified: the Drizzle suites now pass identically on both databases (617 passed, 10 skipped, 0 failed on each). The Supabase job is unchanged at 665 passed. --- .github/actions/integration-setup/action.yml | 4 ++- .github/workflows/integration-drizzle.yml | 34 ++++++++++++++----- .../drizzle-v3/relational.integration.test.ts | 31 ++++++++++++++--- 3 files changed, 56 insertions(+), 13 deletions(-) diff --git a/.github/actions/integration-setup/action.yml b/.github/actions/integration-setup/action.yml index d8ec179f7..171eddfc6 100644 --- a/.github/actions/integration-setup/action.yml +++ b/.github/actions/integration-setup/action.yml @@ -10,7 +10,9 @@ inputs: node-version: description: Node major to run the suites on. required: false - default: '22' + # The newest major `tests.yml` exercises, and what the repo develops on. + # `engines` is `>=22`; the unit job still covers 22 in its matrix. + default: '24' runs: using: composite diff --git a/.github/workflows/integration-drizzle.yml b/.github/workflows/integration-drizzle.yml index ac2d6c620..c46bd993b 100644 --- a/.github/workflows/integration-drizzle.yml +++ b/.github/workflows/integration-drizzle.yml @@ -1,7 +1,13 @@ name: Integration — Drizzle (EQL v3) -# Real ZeroKMS ciphertext against a real Postgres. The Drizzle adapter talks -# straight to the database, so no PostgREST here — that is the Supabase job. +# Real ZeroKMS ciphertext against a real Postgres, on BOTH database variants. +# +# The Drizzle adapter talks straight to the database, so it does not need +# PostgREST — but it does need to work on managed Postgres, where the `postgres` +# role is not a superuser, the EQL install takes its self-skipping path, and the +# ORE domains cannot hold data. The Supabase compose file brings up PostgREST +# too; this job simply ignores it, and leaves `PGRST_URL` unset so a Supabase +# suite scoped here would throw rather than silently skip. # # Separate from `tests.yml` on purpose: these suites need CipherStash credentials # and a database, and they THROW rather than skip when unconfigured. Keeping them @@ -16,6 +22,8 @@ on: - 'packages/test-kit/**' - 'packages/cli/src/installer/**' - 'local/docker-compose.postgres.yml' + - 'local/docker-compose.supabase.yml' + - 'local/supabase-init.sql' - '.github/workflows/integration-drizzle.yml' - '.github/actions/integration-setup/**' pull_request: @@ -27,6 +35,8 @@ on: - 'packages/test-kit/**' - 'packages/cli/src/installer/**' - 'local/docker-compose.postgres.yml' + - 'local/docker-compose.supabase.yml' + - 'local/supabase-init.sql' - '.github/workflows/integration-drizzle.yml' - '.github/actions/integration-setup/**' @@ -39,10 +49,18 @@ jobs: if: ${{ github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository }} strategy: - # A one-element matrix, not a cross-product: the Drizzle adapter only runs - # against plain Postgres. Kept as a matrix so adding a variant is one line. + fail-fast: false + # Drizzle talks straight to Postgres, so it runs against BOTH databases. + # The Supabase variant is not a formality: its `postgres` role is not a + # superuser, so the EQL install takes its self-skipping path and the ORE + # domains become unusable. A suite that passes on a superuser database can + # still fail there. matrix: - db: [postgres] + include: + - db: postgres + database-url: postgres://cipherstash:password@localhost:55432/cipherstash + - db: supabase + database-url: postgres://postgres:password@localhost:55433/postgres env: CS_WORKSPACE_CRN: ${{ secrets.CS_WORKSPACE_CRN }} @@ -51,9 +69,9 @@ jobs: CS_CLIENT_ACCESS_KEY: ${{ secrets.CS_CLIENT_ACCESS_KEY }} # Job-level env, not a `.env` file: `dotenv/config` does not override an # already-set `process.env`, so these win and no secret is written to disk. - DATABASE_URL: postgres://cipherstash:password@localhost:55432/cipherstash - # Scoped to the suites this database serves. `PGRST_URL` is deliberately - # unset — the Supabase suites would throw here, which is the point. + DATABASE_URL: ${{ matrix.database-url }} + # Scoped to the Drizzle suites plus the adapter-agnostic ones. `PGRST_URL` + # is deliberately unset: a Supabase suite scoped here would throw, not skip. CS_IT_SUITE: >- integration/drizzle-v3/**/*.integration.test.ts, integration/harness.integration.test.ts, diff --git a/packages/stack/integration/drizzle-v3/relational.integration.test.ts b/packages/stack/integration/drizzle-v3/relational.integration.test.ts index dc90c6ffc..5b349e1a1 100644 --- a/packages/stack/integration/drizzle-v3/relational.integration.test.ts +++ b/packages/stack/integration/drizzle-v3/relational.integration.test.ts @@ -23,6 +23,7 @@ import { type DomainSpec, databaseUrl, type EqlV3TypeName, + isCovered, eqlTypeSlug as slug, typedEntries, V3_MATRIX, @@ -63,7 +64,14 @@ const ROW_B = 'row-b' const ROW_C = 'row-c' const ROWS = [ROW_A, ROW_B, ROW_C] as const -const matrixEntries = typedEntries(V3_MATRIX) +// Covered domains only. The `_ord_ore` rows are `deferred`: their columns cannot +// hold data on managed Postgres — the domain CHECK calls `ore_domain_unavailable` +// and the seed INSERT raises — so a table built from every row can only ever run +// against a superuser database. Filtering here is what lets this suite run on +// both the plain-Postgres and Supabase variants. +const matrixEntries = typedEntries(V3_MATRIX).filter(([, spec]) => + isCovered(spec), +) const matrixColumns = Object.fromEntries( matrixEntries.map(([eqlType, spec]) => [ slug(eqlType), @@ -250,8 +258,17 @@ beforeAll(async () => { .map(([eqlType]) => `"${slug(eqlType)}" ${eqlType} NOT NULL`) .join(',\n ') + // DROP, not `CREATE TABLE IF NOT EXISTS`. A table left by an earlier run keeps + // its old columns, so a change to the catalog silently reuses the stale schema. + // That bit: after the `_ord_ore` domains were filtered out of `matrixEntries`, + // the leftover table still carried its nine ORE columns, and on managed + // Postgres every INSERT raised `ore_domain_unavailable` — even leaving those + // columns NULL, because the domain CHECK calls a function that RAISEs. + await sqlClient.unsafe(` + DROP TABLE IF EXISTS ${TABLE_NAME} + `) await sqlClient.unsafe(` - CREATE TABLE IF NOT EXISTS ${TABLE_NAME} ( + CREATE TABLE ${TABLE_NAME} ( id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, row_key TEXT NOT NULL, test_run_id TEXT NOT NULL, @@ -260,7 +277,10 @@ beforeAll(async () => { ) `) await sqlClient.unsafe(` - CREATE TABLE IF NOT EXISTS ${ACCOUNT_TABLE_NAME} ( + DROP TABLE IF EXISTS ${ACCOUNT_TABLE_NAME} + `) + await sqlClient.unsafe(` + CREATE TABLE ${ACCOUNT_TABLE_NAME} ( id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, row_key TEXT NOT NULL, label TEXT NOT NULL, @@ -268,7 +288,10 @@ beforeAll(async () => { ) `) await sqlClient.unsafe(` - CREATE TABLE IF NOT EXISTS ${BIGINT_TABLE_NAME} ( + DROP TABLE IF EXISTS ${BIGINT_TABLE_NAME} + `) + await sqlClient.unsafe(` + CREATE TABLE ${BIGINT_TABLE_NAME} ( id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, row_key TEXT NOT NULL, test_run_id TEXT NOT NULL, From bd25e0e42835c0f7247567197b78983bd0ff9948 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 11 Jul 2026 01:32:48 +1000 Subject: [PATCH 17/20] test(integration): no skipped tests, and fail the run if any appear MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A skipped test reads exactly like a passing one. Every silent hole this suite has found took that shape, and the skips it was itself carrying hid one more. **The bug the skips were hiding.** `dbVariant()` inferred the database from the presence of `PGRST_URL`. The new Drizzle `db=supabase` cell needs no PostgREST and left it unset, so the variant reported `postgres`: EQL installed WITHOUT `--supabase`, the role grants were never applied, and the grants test quietly did not run. The suite passed because Drizzle connects as `postgres`, never as `anon`. The variant is now explicit (`CS_IT_DB_VARIANT`) in both workflows, and the inference survives only as a documented fallback for a hand-run suite. **The skips are gone**, all ten: - The eight per-family "defers " notices were `it.skip`. They asserted nothing. `domainsForFamily` already excludes deferred domains, and `test-kit-families.test.ts` asserts — as a PASSING test — that the excluded set is exactly the nine block-ORE domains, each with a reason. - The two harness gates were `it.runIf`. They are now single tests that assert both branches: on Supabase the three roles exist and hold the `eql_v3` and `eql_v3_internal` grants; on plain Postgres those roles do not exist at all, which is a fact worth stating and makes the variant claim falsifiable. PostgREST is asserted on its own axis (`PGRST_URL` set) rather than on the variant — conflating "is this Supabase" with "is PostgREST up" is what made `dbVariant()` lie. **And a guard so they cannot come back.** `no-skips-reporter.ts` fails the integration run if any test is skipped, naming each one. Verified: adding a single `it.skip` exits 1. Scoped to the integration config. The unit suites still carry the `LIVE_*` gates and their 16 skips; working those out is a separate change. Verified on all three CI cells, each 0 skipped, 0 failed: drizzle / postgres 619 passed drizzle / supabase 619 passed supabase / supabase 665 passed --- .github/workflows/integration-drizzle.yml | 15 ++++- .github/workflows/integration-supabase.yml | 2 + .../integration/harness.integration.test.ts | 62 ++++++++++++++----- .../stack/integration/no-skips-reporter.ts | 47 ++++++++++++++ packages/stack/integration/vitest.config.ts | 4 ++ packages/test-kit/src/install.ts | 13 ++-- packages/test-kit/src/run-family-suite.ts | 11 +--- 7 files changed, 124 insertions(+), 30 deletions(-) create mode 100644 packages/stack/integration/no-skips-reporter.ts diff --git a/.github/workflows/integration-drizzle.yml b/.github/workflows/integration-drizzle.yml index c46bd993b..c8e3f0df6 100644 --- a/.github/workflows/integration-drizzle.yml +++ b/.github/workflows/integration-drizzle.yml @@ -59,8 +59,13 @@ jobs: include: - db: postgres database-url: postgres://cipherstash:password@localhost:55432/cipherstash + pgrest-url: '' - db: supabase database-url: postgres://postgres:password@localhost:55433/postgres + # The supabase compose file starts PostgREST anyway. Drizzle does not + # use it, but supplying the URL lets the harness assert the `anon` + # path on the database it is actually running against. + pgrest-url: http://localhost:55430 env: CS_WORKSPACE_CRN: ${{ secrets.CS_WORKSPACE_CRN }} @@ -70,8 +75,14 @@ jobs: # Job-level env, not a `.env` file: `dotenv/config` does not override an # already-set `process.env`, so these win and no secret is written to disk. DATABASE_URL: ${{ matrix.database-url }} - # Scoped to the Drizzle suites plus the adapter-agnostic ones. `PGRST_URL` - # is deliberately unset: a Supabase suite scoped here would throw, not skip. + PGRST_URL: ${{ matrix.pgrest-url }} + # EXPLICIT, never inferred. The variant decides whether EQL is installed + # with `--supabase` (and therefore whether the role grants are applied). + # Inferring it from `PGRST_URL` reported `postgres` for this job's Supabase + # cell and silently skipped the grants. + CS_IT_DB_VARIANT: ${{ matrix.db }} + # Scoped to the Drizzle suites plus the adapter-agnostic ones. The Supabase + # adapter suites are not run here; they have their own job. CS_IT_SUITE: >- integration/drizzle-v3/**/*.integration.test.ts, integration/harness.integration.test.ts, diff --git a/.github/workflows/integration-supabase.yml b/.github/workflows/integration-supabase.yml index 4c8bbeabf..f8aa5d320 100644 --- a/.github/workflows/integration-supabase.yml +++ b/.github/workflows/integration-supabase.yml @@ -62,6 +62,8 @@ jobs: # as they do on a real Supabase project. DATABASE_URL: postgres://postgres:password@localhost:55433/postgres PGRST_URL: http://localhost:55430 + # EXPLICIT, never inferred from `PGRST_URL` — see `dbVariant()`. + CS_IT_DB_VARIANT: ${{ matrix.db }} # Scoped to the suites this database serves. The Drizzle suites talk to # plain Postgres and get their own job; the harness and bloom suites are # adapter-agnostic and run here because a database is already up. diff --git a/packages/stack/integration/harness.integration.test.ts b/packages/stack/integration/harness.integration.test.ts index 7dc9d1a49..ba92ee617 100644 --- a/packages/stack/integration/harness.integration.test.ts +++ b/packages/stack/integration/harness.integration.test.ts @@ -66,26 +66,60 @@ it('installs the ORE opclass only when the connecting role is a superuser', asyn } }) -it.runIf(dbVariant() === 'supabase')( - 'grants anon USAGE on both eql_v3 and eql_v3_internal', - async () => { +/** + * Both database variants are asserted, neither is skipped. + * + * A skipped test reads exactly like a passing one, and the two skips that used to + * live here hid a real bug: `dbVariant()` inferred `postgres` for the Drizzle job + * running against `supabase/postgres`, so EQL installed without `--supabase`, the + * grants were never applied, and the grants test quietly did not run. + * + * So assert the whole truth: on a Supabase database the roles exist and hold the + * grants; on a plain one they do not exist at all. + */ +it('applies the anon grants on Supabase, and has no such roles on plain Postgres', async () => { + const [roles] = await sql<{ count: string }[]>` + SELECT count(*)::text AS count FROM pg_roles + WHERE rolname IN ('anon', 'authenticated', 'service_role') + ` + + if (dbVariant() === 'supabase') { + expect(Number(roles?.count)).toBe(3) + // `eql_v3_internal` is load-bearing: the SECURITY INVOKER extractors resolve // it with the CALLER's privileges, so without this grant every encrypted // filter fails for `anon` with "permission denied for schema". - const [row] = await sql<{ eql_v3: boolean; internal: boolean }[]>` + const [privs] = await sql<{ eql_v3: boolean; internal: boolean }[]>` SELECT has_schema_privilege('anon', 'eql_v3', 'USAGE') AS eql_v3, has_schema_privilege('anon', 'eql_v3_internal', 'USAGE') AS internal ` + expect(privs).toEqual({ eql_v3: true, internal: true }) + return + } - expect(row).toEqual({ eql_v3: true, internal: true }) - }, -) + // Plain Postgres: the Supabase roles do not exist, so there is nothing to + // grant. Asserting their ABSENCE is what makes the variant claim falsifiable — + // if this database ever grew them, the `--supabase` install path would have to + // run here too. + expect(Number(roles?.count)).toBe(0) +}) -it.runIf(dbVariant() === 'supabase')( - 'serves PostgREST as anon through the authenticator role', - async () => { - const response = await fetch(`${pgrestUrl()}/`) +/** + * PostgREST is asserted on its own axis, not on the variant. The Drizzle job runs + * against the Supabase database and does not need PostgREST; conflating "is this + * Supabase" with "is PostgREST up" is precisely what made `dbVariant()` lie. + */ +it('serves PostgREST when configured, and is not configured otherwise', async () => { + const url = process.env['PGRST_URL'] - expect(response.status).toBe(200) - }, -) + if (!url) { + // Only the plain-Postgres compose file omits PostgREST. A Supabase database + // with no `PGRST_URL` means the job forgot to pass it. + expect(dbVariant()).toBe('postgres') + return + } + + const response = await fetch(`${pgrestUrl()}/`) + + expect(response.status).toBe(200) +}) diff --git a/packages/stack/integration/no-skips-reporter.ts b/packages/stack/integration/no-skips-reporter.ts new file mode 100644 index 000000000..ee40f3e92 --- /dev/null +++ b/packages/stack/integration/no-skips-reporter.ts @@ -0,0 +1,47 @@ +import type { Reporter, TestModule } from 'vitest/node' + +/** + * Fail the integration run if any test was skipped. + * + * A skipped test reads exactly like a passing one. Every silent hole this suite + * has found took that shape: `contains` never ran on `text_match` because its + * needle was `''`; the non-ASCII ORE needle test had zero cases after the OPE + * re-pin; the Supabase grants check quietly did not run for the Drizzle job + * because `dbVariant()` mis-inferred the database. In each case a green run + * reported coverage it did not have. + * + * So the integration suites carry no `skip`, no `todo`, no `runIf` and no + * `skipIf`. Environmental differences are asserted rather than skipped: a plain + * Postgres has no `anon` role, and that is a fact worth stating. Where a + * behaviour genuinely cannot be exercised — the block-ORE domains cannot hold + * data on managed Postgres — the domain is excluded from the matrix by the + * catalog's `deferred` field, and a separate, PASSING test asserts that the + * excluded set is exactly what it should be. + * + * The unit suites still skip (the `LIVE_*` gates), so this is scoped to the + * integration config rather than imposed repo-wide. + */ +export default class NoSkipsReporter implements Reporter { + private readonly skipped: string[] = [] + + onTestModuleEnd(testModule: TestModule): void { + for (const test of Array.from(testModule.children.allTests())) { + const result = test.result() + if (result.state === 'skipped') this.skipped.push(test.fullName) + } + } + + onTestRunEnd(): void { + if (this.skipped.length === 0) return + + const list = this.skipped.map((name) => ` - ${name}`).join('\n') + console.error( + `\n${this.skipped.length} test(s) were SKIPPED. The integration suites must not skip:\n` + + `${list}\n\n` + + 'A skipped test reads exactly like a passing one. Assert the environmental\n' + + "difference instead, or exclude the domain via the catalog's `deferred`\n" + + 'field and assert the excluded set separately.\n', + ) + process.exitCode = 1 + } +} diff --git a/packages/stack/integration/vitest.config.ts b/packages/stack/integration/vitest.config.ts index f0f111231..3c226878a 100644 --- a/packages/stack/integration/vitest.config.ts +++ b/packages/stack/integration/vitest.config.ts @@ -80,5 +80,9 @@ export default defineConfig({ * only place to do it without changing product logging behaviour. */ silent: 'passed-only', + + // Fail the run if anything is skipped. A skipped test reads exactly like a + // passing one, and every silent hole this suite has found took that shape. + reporters: ['default', resolve(__dirname, 'no-skips-reporter.ts')], }, }) diff --git a/packages/test-kit/src/install.ts b/packages/test-kit/src/install.ts index 227372efd..cbb528fc0 100644 --- a/packages/test-kit/src/install.ts +++ b/packages/test-kit/src/install.ts @@ -15,10 +15,15 @@ export type DbVariant = 'postgres' | 'supabase' /** * Which database the integration suite is pointed at. * - * `PGRST_URL` is only ever set for the Supabase variant — the Drizzle suite - * talks straight to Postgres — so its presence identifies the variant without a - * second environment variable to keep in sync. `CS_IT_DB_VARIANT` overrides, - * for the case where that stops being true. + * Set `CS_IT_DB_VARIANT` explicitly. The `PGRST_URL` fallback exists only for a + * developer running one suite by hand, and it is a guess: it once inferred + * `postgres` for the Drizzle job running against `supabase/postgres`, because + * that job needs no PostgREST and leaves the variable unset. The consequence was + * silent — EQL installed without `--supabase`, so the role grants were never + * applied and the grants test skipped, while the suite passed because Drizzle + * connects as `postgres` rather than as `anon`. + * + * The variant decides how EQL is installed. It must not be a guess in CI. */ export function dbVariant(): DbVariant { const explicit = process.env['CS_IT_DB_VARIANT'] diff --git a/packages/test-kit/src/run-family-suite.ts b/packages/test-kit/src/run-family-suite.ts index 65841362a..f8016eaf4 100644 --- a/packages/test-kit/src/run-family-suite.ts +++ b/packages/test-kit/src/run-family-suite.ts @@ -1,12 +1,7 @@ import { afterAll, beforeAll, describe, expect, it } from 'vitest' import type { IntegrationAdapter } from './adapter.ts' import type { DomainSpec } from './catalog.ts' -import { - deferredForFamily, - domainsForFamily, - type FamilyDomain, - type FamilyName, -} from './families.ts' +import { domainsForFamily, type FamilyName } from './families.ts' import { negativeOps, type Plain, positiveOps, type QueryOp } from './ops.ts' import { comparePlain, @@ -148,7 +143,6 @@ export function runFamilySuite( ): void { const adapter = makeAdapter() const domains = domainsForFamily(family) - const deferred = deferredForFamily(family) // Unique per run so a crashed run never leaves a table that shadows the next. const runId = Math.random().toString(36).slice(2, 8) @@ -179,9 +173,6 @@ export function runFamilySuite( await adapter.teardown() }) - if (deferred.length > 0) { - it.skip(`defers ${deferred.map((d) => d.bare).join(', ')}: ${deferred[0]?.reason}`, () => {}) - } for (const domain of domains) { describe(domain.bare, () => { From f67f4488403a5ed51a003eb662f528420fc65546 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 11 Jul 2026 01:40:56 +1000 Subject: [PATCH 18/20] docs(plan): queue the skip-removal and live-suite reorg follow-ups Both are consequences of PR1 and both are about the same thing: a skipped test reads exactly like a passing one. The integration suites now carry zero skips and a reporter that fails the run if any appear; the unit suites still carry 16, behind the LIVE_* gates that live-coverage-guard.test.ts exists to police. --- .../plans/2026-07-10-eql-v3-integration-tests.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md b/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md index f972c7cac..9eb30fd2e 100644 --- a/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md +++ b/docs/superpowers/plans/2026-07-10-eql-v3-integration-tests.md @@ -290,6 +290,14 @@ Observed, not assumed. 6. **Ordering works on the managed-Postgres variant.** `asc`/`desc` on an `_ord` (OPE) column returns true plaintext order against `supabase/postgres`. 7. **CI shape.** `pnpm test` at the repo root runs zero integration tests; a fork PR shows the integration jobs as *skipped*, not failed. +## Follow-ups queued after PR1 + +Both are consequences of PR1, and both are about the same thing: a skipped test reads exactly like a passing one. + +**Remove the `LIVE_*` gates and the 16 unit-suite skips.** `__tests__/helpers/live-gate.ts` still turns `LIVE_CIPHERSTASH_ENABLED`, `LIVE_EQL_V3_PG_ENABLED`, `LIVE_PG_ENABLED` and `LIVE_LOCK_CONTEXT_ENABLED` into `describe.skip`. `live-coverage-guard.test.ts` exists *only* because a false gate is a silent whole-suite skip on a green job. Move those suites into the integration jobs — which throw rather than skip — then delete both files, and extend `no-skips-reporter.ts` to the unit config once it is clean. + +**Port the remaining live suites onto the shared harness.** Still bypassing it: `drizzle-v3/operators-null-live-pg`, `drizzle-v3/operators-lock-context-live-pg`, `v3-matrix/matrix-live*`, `schema-v3-pg`, `supabase-v3-grants-pg`, `supabase-v3-introspect-pg`. Each gets the shared env gate and no skips. This is also where the **superuser-only ORE suite** lands — the one the catalog's `deferred` field points at, covering the nine `_ord_ore` domains that cannot hold data on managed Postgres. + ## PR2 — type robustness (outline) Import the canonical per-domain types from `@cipherstash/eql` and thread them through. Stop typing `OperandEncryptionClient.encrypt`/`bulkEncrypt` as returning `unknown` (`eql/v3/drizzle/operators.ts:51,55`); stop collapsing `Result<…>` to `{ data?: unknown }` (`:94-101`); stop returning `Promise` from `encryptOperands` and from `encryptCollectedTerms`/`bulkEncryptGroup`/`encryptGroupPerTerm` (`supabase/query-builder-v3.ts:393,433,463`). Where a `JSON.stringify` is genuinely required at the SQL boundary, it should serialize a *typed* envelope, not an `unknown`. The PR1 suite is what makes this safe to attempt. From 02750c655b623a49204ca5206fefe051955b0a36 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sat, 11 Jul 2026 12:10:36 +1000 Subject: [PATCH 19/20] test(stack): address review findings on the v3 integration harness Follow-up to the code review of PR #616. - Comments: the Supabase v3 order path is `col->op` (jsonb), not `col->>op`. The emitted code always used `->`; three comment sites (query-builder-v3, types, the integration adapter) and the "measured against live PostgREST" note said `->>`, contradicting the code and the changeset on a correctness-critical seam. - test-kit IntegrationAdapter docstrings: `supportedOps`/`alwaysRejectedOps` said Supabase omits/rejects `order`; both adapters now support it, so the docs described the opposite of the shipped adapters. Rewritten to match. - needlesFrom: raise the discriminating-needle guard to length >= 7. At 6 the interior slice is itself a single trigram, so the needle is no more discriminating than the degenerate one and the #615 tripwire is toothless. - relational.integration.test.ts: drop the verbatim copies of the plaintext oracle (comparePlain/expectedKeysFor/sortedKeysFor) and use the single copy in @cipherstash/test-kit, binding this suite's fixed ROWS set. The bytewise-ordering rules now have one home. - CI suite selection: move the adapter-agnostic suites (harness, match-bloom, ope-term) into integration/shared/ and glob CS_IT_SUITE by directory only, never by named file, so a renamed suite cannot silently drop from a green CI run. ope-term is the tripwire for the Supabase `col->op` ordering path and now runs on the Supabase job too, not just Drizzle. Comment/test/CI-config only; no runtime source behaviour changes. Claude-Session: https://claude.ai/code/session_01MxTTPaPP16m6br7Hoab94w --- .github/workflows/integration-drizzle.yml | 12 ++-- .github/workflows/integration-supabase.yml | 13 +++-- .../drizzle-v3/relational.integration.test.ts | 56 +++---------------- .../{ => shared}/harness.integration.test.ts | 0 .../match-bloom.integration.test.ts | 0 .../{ => shared}/ope-term.integration.test.ts | 0 .../stack/integration/supabase/adapter.ts | 2 +- .../stack/src/supabase/query-builder-v3.ts | 6 +- packages/stack/src/supabase/types.ts | 2 +- packages/test-kit/src/adapter.ts | 19 ++++--- packages/test-kit/src/run-family-suite.ts | 6 +- 11 files changed, 42 insertions(+), 74 deletions(-) rename packages/stack/integration/{ => shared}/harness.integration.test.ts (100%) rename packages/stack/integration/{ => shared}/match-bloom.integration.test.ts (100%) rename packages/stack/integration/{ => shared}/ope-term.integration.test.ts (100%) diff --git a/.github/workflows/integration-drizzle.yml b/.github/workflows/integration-drizzle.yml index c8e3f0df6..acb2e0ccd 100644 --- a/.github/workflows/integration-drizzle.yml +++ b/.github/workflows/integration-drizzle.yml @@ -81,13 +81,13 @@ jobs: # Inferring it from `PGRST_URL` reported `postgres` for this job's Supabase # cell and silently skipped the grants. CS_IT_DB_VARIANT: ${{ matrix.db }} - # Scoped to the Drizzle suites plus the adapter-agnostic ones. The Supabase - # adapter suites are not run here; they have their own job. + # Scoped by directory, never by named file, so a renamed suite cannot + # silently drop from CI. `integration/shared/` holds the adapter-agnostic + # suites (harness, bloom, ope-term); the Supabase adapter suites are not + # run here — they have their own job. CS_IT_SUITE: >- - integration/drizzle-v3/**/*.integration.test.ts, - integration/harness.integration.test.ts, - integration/ope-term.integration.test.ts, - integration/match-bloom.integration.test.ts + integration/shared/**/*.integration.test.ts, + integration/drizzle-v3/**/*.integration.test.ts steps: - uses: actions/checkout@v6 diff --git a/.github/workflows/integration-supabase.yml b/.github/workflows/integration-supabase.yml index f8aa5d320..894b41165 100644 --- a/.github/workflows/integration-supabase.yml +++ b/.github/workflows/integration-supabase.yml @@ -64,13 +64,14 @@ jobs: PGRST_URL: http://localhost:55430 # EXPLICIT, never inferred from `PGRST_URL` — see `dbVariant()`. CS_IT_DB_VARIANT: ${{ matrix.db }} - # Scoped to the suites this database serves. The Drizzle suites talk to - # plain Postgres and get their own job; the harness and bloom suites are - # adapter-agnostic and run here because a database is already up. + # Scoped by directory, never by named file, so a renamed suite cannot + # silently drop from CI. `integration/shared/` holds the adapter-agnostic + # suites (harness, bloom, and the ope-term tripwire for THIS job's + # `col->op` ordering path); the Drizzle suites talk to plain Postgres and + # get their own job. CS_IT_SUITE: >- - integration/supabase/**/*.integration.test.ts, - integration/harness.integration.test.ts, - integration/match-bloom.integration.test.ts + integration/shared/**/*.integration.test.ts, + integration/supabase/**/*.integration.test.ts steps: - uses: actions/checkout@v6 diff --git a/packages/stack/integration/drizzle-v3/relational.integration.test.ts b/packages/stack/integration/drizzle-v3/relational.integration.test.ts index 5b349e1a1..e555b6a92 100644 --- a/packages/stack/integration/drizzle-v3/relational.integration.test.ts +++ b/packages/stack/integration/drizzle-v3/relational.integration.test.ts @@ -24,7 +24,9 @@ import { databaseUrl, type EqlV3TypeName, isCovered, + plainValue as plainValueFor, eqlTypeSlug as slug, + sortedKeysFor as sortedKeysForKit, typedEntries, V3_MATRIX, } from '@cipherstash/test-kit' @@ -169,56 +171,14 @@ const matrixColumn = (eqlType: EqlV3TypeName): SQLWrapper => const scoped = (cond: SQL | undefined): SQL | undefined => cond ? and(drizzleEq(matrixTable.testRunId, RUN), cond) : cond +// The plaintext oracle (`plainValue`, `comparePlain`, `sortedKeysFor`) lives in +// `@cipherstash/test-kit`, so the bytewise-ordering rules have a single home. +// These wrappers just bind this suite's fixed `ROWS` set. const plainValue = (spec: DomainSpec, rowKey: RowKey): PlainValue => - spec.samples[Math.min(ROWS.indexOf(rowKey), spec.samples.length - 1)] + plainValueFor(spec, ROWS, rowKey) -function comparePlain(left: PlainValue, right: PlainValue): number { - if (left instanceof Date && right instanceof Date) { - return left.getTime() - right.getTime() - } - if (typeof left === 'number' && typeof right === 'number') { - return left - right - } - // bigint order domains (`bigint_ord`/`bigint_ord_ore`) carry i64 samples - // beyond Number.MAX_SAFE_INTEGER, so they must be compared as bigints — the - // subtraction is narrowed to -1/0/1 because callers expect a `number`. - if (typeof left === 'bigint' && typeof right === 'bigint') { - return left < right ? -1 : left > right ? 1 : 0 - } - if (typeof left === 'string' && typeof right === 'string') { - // eql_v3 text ordering (ORE) is BYTEWISE, not locale-collated: the oracle - // must model codepoint order, not `localeCompare` (which folds case, - // reorders punctuation vs letters, and is locale-dependent). Text samples - // must stay ASCII/unambiguous so UTF-16 code-unit order == the byte order - // the DB actually sorts by. - return left < right ? -1 : left > right ? 1 : 0 - } - throw new Error( - `Unsupported ordered values: ${String(left)}, ${String(right)}`, - ) -} - -function expectedKeysFor( - spec: DomainSpec, - predicate: (value: PlainValue) => boolean, -): RowKey[] { - return ROWS.filter((rowKey) => predicate(plainValue(spec, rowKey))) -} - -/** - * Oracle for the encrypted ORDER BY. Domains with only two samples give ROW_B - * and ROW_C equal values, so the comparison alone does not determine the row - * order — ties break on `rowKey` ascending, which the query mirrors with a - * secondary `ORDER BY row_key`. Without that both sides would be arbitrary and - * the test would flake rather than prove ordering. - */ -function sortedKeysFor(spec: DomainSpec, direction: 'asc' | 'desc'): RowKey[] { - return [...ROWS].sort((left, right) => { - const cmp = comparePlain(plainValue(spec, left), plainValue(spec, right)) - if (cmp !== 0) return direction === 'asc' ? cmp : -cmp - return left < right ? -1 : left > right ? 1 : 0 - }) -} +const sortedKeysFor = (spec: DomainSpec, direction: 'asc' | 'desc'): string[] => + sortedKeysForKit(spec, ROWS, direction) async function selectRowKeys(condition: SQL | undefined): Promise { if (!condition) throw new Error('Expected query condition') diff --git a/packages/stack/integration/harness.integration.test.ts b/packages/stack/integration/shared/harness.integration.test.ts similarity index 100% rename from packages/stack/integration/harness.integration.test.ts rename to packages/stack/integration/shared/harness.integration.test.ts diff --git a/packages/stack/integration/match-bloom.integration.test.ts b/packages/stack/integration/shared/match-bloom.integration.test.ts similarity index 100% rename from packages/stack/integration/match-bloom.integration.test.ts rename to packages/stack/integration/shared/match-bloom.integration.test.ts diff --git a/packages/stack/integration/ope-term.integration.test.ts b/packages/stack/integration/shared/ope-term.integration.test.ts similarity index 100% rename from packages/stack/integration/ope-term.integration.test.ts rename to packages/stack/integration/shared/ope-term.integration.test.ts diff --git a/packages/stack/integration/supabase/adapter.ts b/packages/stack/integration/supabase/adapter.ts index 657f5885c..413c66ff0 100644 --- a/packages/stack/integration/supabase/adapter.ts +++ b/packages/stack/integration/supabase/adapter.ts @@ -25,7 +25,7 @@ import { makePostgrestClient, reloadSchemaCache } from '../helpers/pgrest' /** * `order` is supported. The builder rewrites an encrypted ordering column to the - * jsonb path `col->>op`, which selects the OPE term; OPE is order-preserving, so + * jsonb path `col->op`, which selects the OPE term; OPE is order-preserving, so * PostgREST reproduces the plaintext order. Columns with no ordering term are * rejected by the capability matrix, exactly like `gt`/`lt`. */ diff --git a/packages/stack/src/supabase/query-builder-v3.ts b/packages/stack/src/supabase/query-builder-v3.ts index 51bd7d230..05f0eff04 100644 --- a/packages/stack/src/supabase/query-builder-v3.ts +++ b/packages/stack/src/supabase/query-builder-v3.ts @@ -238,13 +238,13 @@ export class EncryptedQueryBuilderV3Impl< * returns the domain's `op` term, and OPE is order-preserving by construction: * ordering by the term reproduces the plaintext order. PostgREST cannot emit * `ORDER BY eql_v3.ord_term(col)`, but it CAN emit a jsonb path — - * `order=col->>op.asc` — which selects exactly that term. Measured against a - * live PostgREST: `order=amount->>op.asc` and `.desc` both reproduce the + * `order=col->op.asc` — which selects exactly that term. Measured against a + * live PostgREST: `order=amount->op.asc` and `.desc` both reproduce the * plaintext order for `integer_ord` and `text_search`, over 10 rows. * * So the guard is on the ordering FLAVOUR, not on encryption: * - * - `ope` present → order by `col->>op`. Every plain `_ord` domain, plus + * - `ope` present → order by `col->op`. Every plain `_ord` domain, plus * `text_ord` and `text_search`. * - `ore` present → reject. The `ob` term is an array of ORE blocks whose * comparison needs the superuser-only opclass; a jsonb-path sort over it is diff --git a/packages/stack/src/supabase/types.ts b/packages/stack/src/supabase/types.ts index b65737ff0..174ee4aca 100644 --- a/packages/stack/src/supabase/types.ts +++ b/packages/stack/src/supabase/types.ts @@ -209,7 +209,7 @@ export type NonOrderableV3Keys
= { * btree opclass on any domain, so the sort resolves through jsonb's default * `jsonb_cmp` and compares the random ciphertext first. But the builder does not * emit a bare `ORDER BY`: for an encrypted ordering column it emits the jsonb - * path `col->>op`, which selects the OPE term, and OPE is order-preserving. See + * path `col->op`, which selects the OPE term, and OPE is order-preserving. See * `EncryptedQueryBuilderV3Impl.orderColumnName`. * * ORE-backed (`*_ord_ore`) columns are `orderAndRange`-capable and so pass this diff --git a/packages/test-kit/src/adapter.ts b/packages/test-kit/src/adapter.ts index 4380cee96..52670479d 100644 --- a/packages/test-kit/src/adapter.ts +++ b/packages/test-kit/src/adapter.ts @@ -13,17 +13,22 @@ export interface IntegrationAdapter { readonly name: 'drizzle' | 'supabase' /** - * Operations this adapter can express at all, independent of any domain. - * Supabase omits `order`: PostgREST cannot emit `ORDER BY eql_v3.ord_term(col)`, - * and a bare `ORDER BY` would sort the raw ciphertext envelope. + * Operations this adapter can express at all, independent of any domain. Both + * adapters currently cover the full `QueryOpKind` union, `order` included: + * Drizzle emits `ORDER BY eql_v3.ord_term(col)`, and Supabase emits the jsonb + * path `col->op`, which selects the same order-preserving OPE term. (Ordering + * columns that lack an `ope` term are rejected by the capability matrix, not + * by this set.) */ readonly supportedOps: ReadonlySet /** - * Operations this adapter refuses on ANY encrypted column, whatever its - * capabilities. Supabase: `{'order'}`. This is the half of the contract the - * capability matrix cannot derive — an ORE-capable column still cannot be - * ordered through PostgREST — so it is stated once, here, and pinned by a test. + * Operations this adapter refuses on ANY encrypted column, regardless of its + * capabilities — the half of the contract the capability matrix cannot derive. + * Both adapters currently leave this empty: Supabase's one former entry, + * `order`, is now allowed on OPE columns and rejected on ORE/none by the + * ordering-flavour guard, which the capability matrix already drives. Kept as + * the seam for a future adapter with a genuinely unconditional refusal. */ readonly alwaysRejectedOps: ReadonlySet diff --git a/packages/test-kit/src/run-family-suite.ts b/packages/test-kit/src/run-family-suite.ts index f8016eaf4..f220e584a 100644 --- a/packages/test-kit/src/run-family-suite.ts +++ b/packages/test-kit/src/run-family-suite.ts @@ -95,7 +95,10 @@ function needlesFrom(values: readonly Plain[]): Needle[] { // The discriminating case: a strict substring longer than one trigram, drawn // from the interior. Fails the moment a whole-needle token enters the operand. - if (value.length >= 6) { + // Requires length >= 7: at 6 the interior slice `slice(2, 5)` is itself a + // single trigram, so the needle would be no more discriminating than the + // degenerate one above and the guard would be toothless. + if (value.length >= 7) { needles.splice(1, 0, { label: 'a strict interior substring longer than one trigram', needle: value.slice(2, Math.min(value.length - 1, 8)), @@ -173,7 +176,6 @@ export function runFamilySuite( await adapter.teardown() }) - for (const domain of domains) { describe(domain.bare, () => { const { slug, spec } = domain From 11fe4ec18a4c5c0963263ee8cde250411f587034 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Sun, 12 Jul 2026 20:47:27 +1000 Subject: [PATCH 20/20] fix(stack): address code-review findings on the v3 integration harness (round 2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - types.ts: exclude ORE (`*_ord_ore`) columns from `V3OrderableKeys` at COMPILE time (they are orderAndRange-capable but not sortable through a jsonb path), so `.order(oreCol)` is a type error matching the runtime rejection — no more green-typecheck-then-500. Pinned with a `@ts-expect-error` in the type tests. - relational: re-add the deleted `not(between())` precedence test — `between` emits a two-clause conjunction and the passthrough `not` relies on `v3Dialect.range` parenthesising it; the only surviving `not` test wrapped a single-clause `eq` and could not catch a paren regression. Asserts ROW_C is kept, which the buggy `value < bound` form would drop. - relational: run-scope the table names (`_${RID}`) and DROP them in teardown, like the family suites — a fixed name races a concurrent run's `beforeAll` DROP on a shared/persistent DB. - supabase adapter `expectRejected`: stop counting an arbitrary throw as a valid rejection (it matched its own sentinel by message substring). Now a thrown rejection must carry the `[supabase v3]` marker; a stray TypeError/network error rethrows and fails the test. - query-builder-v3: drop the cached `selectKeyToDb` field (a side effect of `buildSelectString` read later by `postprocessDecryptedRow`) — derive it inline from `this.selectColumns` so a reused builder can't postprocess with a stale select map. - vitest.shared.ts: add the bare `@cipherstash/stack` runtime alias (last, so the subpaths still win) — both sibling tsconfigs map it, and without it a bare- specifier importer would resolve to `dist/`, re-coupling `pnpm test` to a build. - integration workflows: broaden the path filters to the source layers the adapters' encoding rests on (`src/encryption`, `src/schema`, `packages/schema`) so a break there triggers the live wire/round-trip suites. Note: the `orderColumnName` bare-fallback finding was investigated and found to be a non-issue — `validateTransforms` runs before the query executes and rejects the column with a domain-specific message, so the bare name is only an intermediate value on an about-to-be-rejected request. Corrected the misleading comment instead of adding a throw (which regressed that message). Claude-Session: https://claude.ai/code/session_01MxTTPaPP16m6br7Hoab94w --- .changeset/supabase-v3-order-by-ope-term.md | 12 +++-- .github/workflows/integration-drizzle.yml | 12 +++++ .github/workflows/integration-supabase.yml | 12 +++++ .../stack/__tests__/supabase-v3.test-d.ts | 6 +++ .../drizzle-v3/relational.integration.test.ts | 54 ++++++++++++++----- .../stack/integration/supabase/adapter.ts | 33 ++++++------ .../stack/src/supabase/query-builder-v3.ts | 27 +++++----- packages/stack/src/supabase/types.ts | 36 +++++++++---- vitest.shared.ts | 6 +++ 9 files changed, 140 insertions(+), 58 deletions(-) diff --git a/.changeset/supabase-v3-order-by-ope-term.md b/.changeset/supabase-v3-order-by-ope-term.md index 73e7e3977..33d450239 100644 --- a/.changeset/supabase-v3-order-by-ope-term.md +++ b/.changeset/supabase-v3-order-by-ope-term.md @@ -26,7 +26,9 @@ The guard is now on the ordering FLAVOUR, not on encryption: - **`ore` present → rejected.** The `ob` term is an array of ORE blocks whose comparison needs the superuser-only operator class, which no jsonb path can reach. (Such a column cannot hold data on managed Postgres anyway: its domain - CHECK raises `ore_domain_unavailable`.) + CHECK raises `ore_domain_unavailable`.) ORE columns are now excluded from + `order()` at COMPILE time too, not only at runtime — `.order(oreColumn)` is a + type error, matching the rejection. - **neither → rejected.** Storage-only, equality-only and match-only columns carry no ordering term. @@ -38,6 +40,8 @@ numeric and date domains, and per-character (16 hex chars each) for text, so lexicographic order reproduces plaintext order including the prefix case (`ada` < `adam`). `ope-term.integration.test.ts` pins that shape. -`V3OrderableKeys` widens to match, so `order()` on an ordering column -typechecks. `is(col, true)` is unaffected — it stays plaintext-only, and now has -its own `V3PlaintextKeys` rather than borrowing the orderable set. +`V3OrderableKeys` widens to admit OPE-backed ordering columns (`*_ord`, +`text_ord`, `text_search`) while still excluding ORE (`*_ord_ore`) columns, so +`order()` typechecks exactly where it works. `is(col, true)` is unaffected — it +stays plaintext-only, and now has its own `V3PlaintextKeys` rather than +borrowing the orderable set. diff --git a/.github/workflows/integration-drizzle.yml b/.github/workflows/integration-drizzle.yml index acb2e0ccd..f04b2f19f 100644 --- a/.github/workflows/integration-drizzle.yml +++ b/.github/workflows/integration-drizzle.yml @@ -18,6 +18,12 @@ on: branches: [main] paths: - 'packages/stack/src/eql/v3/**' + # Source layers the adapter's encoding/round-trip rests on: a break here + # (not just under src/eql/v3) can produce wrong rows, so trigger the live + # suite that would catch it. + - 'packages/stack/src/encryption/**' + - 'packages/stack/src/schema/**' + - 'packages/schema/**' - 'packages/stack/integration/**' - 'packages/test-kit/**' - 'packages/cli/src/installer/**' @@ -31,6 +37,12 @@ on: # Repeated verbatim: GitHub Actions does not support YAML anchors/aliases. paths: - 'packages/stack/src/eql/v3/**' + # Source layers the adapter's encoding/round-trip rests on: a break here + # (not just under src/eql/v3) can produce wrong rows, so trigger the live + # suite that would catch it. + - 'packages/stack/src/encryption/**' + - 'packages/stack/src/schema/**' + - 'packages/schema/**' - 'packages/stack/integration/**' - 'packages/test-kit/**' - 'packages/cli/src/installer/**' diff --git a/.github/workflows/integration-supabase.yml b/.github/workflows/integration-supabase.yml index 894b41165..c730460e3 100644 --- a/.github/workflows/integration-supabase.yml +++ b/.github/workflows/integration-supabase.yml @@ -14,6 +14,12 @@ on: paths: - 'packages/stack/src/supabase/**' - 'packages/stack/src/eql/v3/**' + # Source layers the adapter's encoding/round-trip rests on: a break here + # (not just under src/supabase) can produce wrong wire output or rows, so + # trigger the live suite that would catch it. + - 'packages/stack/src/encryption/**' + - 'packages/stack/src/schema/**' + - 'packages/schema/**' - 'packages/stack/integration/**' - 'packages/test-kit/**' - 'packages/cli/src/installer/**' @@ -26,6 +32,12 @@ on: paths: - 'packages/stack/src/supabase/**' - 'packages/stack/src/eql/v3/**' + # Source layers the adapter's encoding/round-trip rests on: a break here + # (not just under src/supabase) can produce wrong wire output or rows, so + # trigger the live suite that would catch it. + - 'packages/stack/src/encryption/**' + - 'packages/stack/src/schema/**' + - 'packages/schema/**' - 'packages/stack/integration/**' - 'packages/test-kit/**' - 'packages/cli/src/installer/**' diff --git a/packages/stack/__tests__/supabase-v3.test-d.ts b/packages/stack/__tests__/supabase-v3.test-d.ts index 6b6337e30..32681f42e 100644 --- a/packages/stack/__tests__/supabase-v3.test-d.ts +++ b/packages/stack/__tests__/supabase-v3.test-d.ts @@ -18,6 +18,7 @@ const users = encryptedTable('users', { active: types.Boolean('active'), nickname: types.TextEq('nickname'), bio: types.TextMatch('bio'), + score: types.IntegerOrdOre('score'), }) type UserRow = InferPlaintext @@ -130,6 +131,11 @@ describe('encryptedSupabaseV3 typed surface (with schemas)', () => { builder.order('nickname') // @ts-expect-error — bio is public.eql_v3_text_match: match only builder.order('bio') + // @ts-expect-error — score is public.eql_v3_integer_ord_ore: ORE-backed, so + // orderAndRange-capable but NOT sortable through a jsonb path (its `ob` term + // needs the superuser-only ORE opclass). Excluded at compile time to match + // the runtime rejection. + builder.order('score') }) it('still allows order() on a plaintext row key', async () => { diff --git a/packages/stack/integration/drizzle-v3/relational.integration.test.ts b/packages/stack/integration/drizzle-v3/relational.integration.test.ts index e555b6a92..fd1c2aa01 100644 --- a/packages/stack/integration/drizzle-v3/relational.integration.test.ts +++ b/packages/stack/integration/drizzle-v3/relational.integration.test.ts @@ -52,9 +52,15 @@ import { makeEqlV3Column } from '@/eql/v3/drizzle/column' const sqlClient = postgres(databaseUrl(), { prepare: false }) -const TABLE_NAME = 'protect_ci_v3_drizzle_matrix' -const ACCOUNT_TABLE_NAME = 'protect_ci_v3_drizzle_matrix_accounts' -const RUN = `run-${Date.now()}-${Math.random().toString(36).slice(2, 8)}` +// Per-run table suffix so two runs sharing a database (a persistent/reused CI +// database, a developer's local DB, or re-enabled file parallelism) never +// operate on the same physical table — one run's `beforeAll` DROP would +// otherwise blow away a table another run is mid-query on. Mirrors the family +// suites' run-scoped naming (`rows.ts` `planTable`). +const RID = Math.random().toString(36).slice(2, 8) +const TABLE_NAME = `protect_ci_v3_drizzle_matrix_${RID}` +const ACCOUNT_TABLE_NAME = `protect_ci_v3_drizzle_matrix_accounts_${RID}` +const RUN = `run-${Date.now()}-${RID}` const ROW_A = 'row-a' const ROW_B = 'row-b' // A third row. With only two rows every predicate can return just [A], [B], @@ -103,7 +109,7 @@ const accountsTable = pgTable(ACCOUNT_TABLE_NAME, { // known at compile time, so it exercises A3 end-to-end with ZERO casts: the // insert takes envelope rows and the select yields `Encrypted` values ready for // decrypt. -const BIGINT_TABLE_NAME = 'protect_ci_v3_drizzle_bigint' +const BIGINT_TABLE_NAME = `protect_ci_v3_drizzle_bigint_${RID}` const bigintTable = pgTable(BIGINT_TABLE_NAME, { id: integer('id').primaryKey().generatedAlwaysAsIdentity(), rowKey: text('row_key').notNull(), @@ -218,12 +224,13 @@ beforeAll(async () => { .map(([eqlType]) => `"${slug(eqlType)}" ${eqlType} NOT NULL`) .join(',\n ') - // DROP, not `CREATE TABLE IF NOT EXISTS`. A table left by an earlier run keeps - // its old columns, so a change to the catalog silently reuses the stale schema. - // That bit: after the `_ord_ore` domains were filtered out of `matrixEntries`, - // the leftover table still carried its nine ORE columns, and on managed - // Postgres every INSERT raised `ore_domain_unavailable` — even leaving those - // columns NULL, because the domain CHECK calls a function that RAISEs. + // Table names are run-scoped (see RID), so DROP IF EXISTS is normally a no-op; + // it stays as a belt-and-braces guard against an id collision. Recreating from + // scratch each run also means a catalog change can never silently reuse a + // stale schema — the bug that bit once when the `_ord_ore` domains were + // filtered out of `matrixEntries` but a leftover fixed-name table kept its nine + // ORE columns, making every INSERT raise `ore_domain_unavailable` on managed + // Postgres (the domain CHECK RAISEs even for a NULL value). await sqlClient.unsafe(` DROP TABLE IF EXISTS ${TABLE_NAME} `) @@ -299,9 +306,11 @@ beforeAll(async () => { }, 120000) afterAll(async () => { - await sqlClient`DELETE FROM ${sqlClient(TABLE_NAME)} WHERE test_run_id = ${RUN}` - await sqlClient`DELETE FROM ${sqlClient(ACCOUNT_TABLE_NAME)} WHERE test_run_id = ${RUN}` - await sqlClient`DELETE FROM ${sqlClient(BIGINT_TABLE_NAME)} WHERE test_run_id = ${RUN}` + // Tables are run-scoped, so drop them outright rather than DELETE-ing rows — + // no other run shares them, and this leaves nothing behind on a persistent DB. + await sqlClient.unsafe(`DROP TABLE IF EXISTS ${TABLE_NAME}`) + await sqlClient.unsafe(`DROP TABLE IF EXISTS ${ACCOUNT_TABLE_NAME}`) + await sqlClient.unsafe(`DROP TABLE IF EXISTS ${BIGINT_TABLE_NAME}`) await sqlClient.end() }, 30000) @@ -449,6 +458,25 @@ describe('v3 drizzle — relational, needle guards, pagination', () => { expect(rows).toEqual([ROW_B, ROW_C]) }, 30000) + it('not(between()) negates the whole range, not just the lower bound', async () => { + // integer_ord: ROW_A=0, ROW_B=-42, ROW_C=2147483647. `not(between(0, 0))` + // must return every row whose value != 0 → ROW_B and ROW_C. + // + // `between` emits a TWO-clause conjunction, and Drizzle's passthrough `not` + // renders a bare `NOT `. Postgres binds NOT tighter than AND, so this + // only works because `v3Dialect.range` already parenthesises `(gte AND lte)`. + // Without those parens, `NOT gte(0) AND lte(0)` parses as + // `value < 0 AND value <= 0` = `value < 0` = ROW_B alone, silently dropping + // ROW_C. Asserting ROW_C is present is what discriminates the two — a + // single-bound complement would satisfy the buggy form too. + const rows = await selectRowKeys( + ops.not( + await ops.between(matrixColumn('public.eql_v3_integer_ord'), 0, 0), + ), + ) + expect(rows).toEqual([ROW_B, ROW_C]) + }, 30000) + it('isNull and isNotNull work on nullable encrypted columns', async () => { expect(await selectRowKeys(ops.isNull(matrixTable.nullableTextEq))).toEqual( [ROW_A], diff --git a/packages/stack/integration/supabase/adapter.ts b/packages/stack/integration/supabase/adapter.ts index 413c66ff0..cb549fd97 100644 --- a/packages/stack/integration/supabase/adapter.ts +++ b/packages/stack/integration/supabase/adapter.ts @@ -205,26 +205,27 @@ export function makeSupabaseAdapter(): IntegrationAdapter { }, async expectRejected(_table: TableSpec, op: QueryOp) { - // The guards fire in two places: capability checks throw synchronously at - // call time, while a term that reaches `execute` surfaces as a Result - // error. Accept either; the point is that the query never runs. + // The rejection fires in one of two places: a capability check throws + // synchronously while collecting terms (marked `[supabase v3]`), or a term + // that reaches `execute` comes back as a Result error. Accept either — but + // NOT an arbitrary throw. A `TypeError` while assembling the term, or a + // network failure, is not the refusal this test asserts; counting it would + // let the test pass for the wrong reason if the real capability guard were + // removed. The `try` wraps only `applyOp`, and the "not rejected" error is + // thrown OUTSIDE it, so it can never be swallowed as a rejection. + let error: { message: string } | null try { - const { error } = await applyOp(op) - if (!error) { - throw new Error( - `Expected ${op.kind}("${op.column}") to be rejected, but it succeeded`, - ) - } + ;({ error } = await applyOp(op)) } catch (cause) { - if ( - cause instanceof Error && - cause.message.startsWith('Expected ') && - cause.message.includes('to be rejected') - ) { - throw cause + if (cause instanceof Error && cause.message.includes('[supabase v3]')) { + return // a modeled capability rejection } - // A thrown guard is a rejection. + throw cause // an unexpected error — fail loudly } + if (error) return // an execute-time DB refusal (e.g. a PostgREST error) + throw new Error( + `Expected ${op.kind}("${op.column}") to be rejected, but it succeeded`, + ) }, } } diff --git a/packages/stack/src/supabase/query-builder-v3.ts b/packages/stack/src/supabase/query-builder-v3.ts index 05f0eff04..9dd28c645 100644 --- a/packages/stack/src/supabase/query-builder-v3.ts +++ b/packages/stack/src/supabase/query-builder-v3.ts @@ -146,15 +146,6 @@ export class EncryptedQueryBuilderV3Impl< private columnSchemas: Record /** Column builders keyed by BOTH property name and DB name. */ private v3Columns: Record - /** - * Result-row key → DB column name for the columns the current select - * produces, including caller-chosen PostgREST aliases (`ts:createdAt` keys - * rows by `ts`). Populated by {@link buildSelectString}; consumed by - * {@link postprocessDecryptedRow} so aliased date columns still get `Date` - * reconstruction. Empty on paths with no recorded select (an insert or update - * that returns rows), which fall back to the static property/DB names. - */ - private selectKeyToDb: Record = Object.create(null) constructor( tableName: string, @@ -300,8 +291,13 @@ export class EncryptedQueryBuilderV3Impl< * `text_search`) — and every collation orders digits before letters and hex * letters among themselves. `match-bloom`'s sibling assertion pins that shape. * - * `validateTransforms` has already rejected every encrypted column that lacks - * an `ope` index, so reaching the jsonb path here implies the term exists. + * This runs at column-name-mapping time (`transformToDbSpace`), BEFORE + * `buildAndExecuteQuery` calls `validateTransforms`. For an encrypted column + * with no `ope` index it therefore returns a bare `dbName` here — a name that + * would sort by `jsonb_cmp` over the ciphertext if it reached PostgREST — but + * it never does: `validateTransforms` throws (with a domain-specific reason) + * before the query executes, so the bare name is only ever an intermediate + * value on a request that is about to be rejected. */ protected override orderColumnName(column: string): DbName { const dbName = this.dbNameFor(column) @@ -348,7 +344,6 @@ export class EncryptedQueryBuilderV3Impl< protected override buildSelectString(): DbSelect | null { if (this.selectColumns === null) return null - this.selectKeyToDb = selectKeyToDbV3(this.selectColumns, this.propToDb) return addJsonbCastsV3(this.selectColumns, this.propToDb) } @@ -603,10 +598,14 @@ export class EncryptedQueryBuilderV3Impl< // Every key an encrypted column can appear under: the keys this select // actually produces (including caller-chosen aliases like `ts:createdAt`), // plus the static property and DB names as a fallback for paths that record - // no select. Aliases win — `selectKeyToDb` describes the row in hand. + // no select. Aliases win. Derived here from `this.selectColumns` (the row in + // hand) rather than cached from `buildSelectString`, so a reused builder can + // never postprocess a row with a previous operation's stale select map. const keyToDb: Record = Object.assign( Object.create(null), - this.selectKeyToDb, + this.selectColumns === null + ? undefined + : selectKeyToDbV3(this.selectColumns, this.propToDb), ) for (const [property, dbName] of Object.entries(this.propToDb)) { keyToDb[property] ??= dbName diff --git a/packages/stack/src/supabase/types.ts b/packages/stack/src/supabase/types.ts index 174ee4aca..ef78199be 100644 --- a/packages/stack/src/supabase/types.ts +++ b/packages/stack/src/supabase/types.ts @@ -1,6 +1,11 @@ import type { EncryptionClient } from '@/encryption' import type { AuditConfig } from '@/encryption/operations/base-operation' -import type { AnyV3Table, InferPlaintext, QueryTypesForColumn } from '@/eql/v3' +import type { + AnyV3Table, + EqlTypeForColumn, + InferPlaintext, + QueryTypesForColumn, +} from '@/eql/v3' import type { EncryptionError } from '@/errors' import type { LockContext } from '@/identity' import type { EncryptedTable, EncryptedTableColumn } from '@/schema' @@ -188,16 +193,26 @@ export type V3ContainsValue< : string /** - * JS property names of a v3 table's columns that carry NO `orderAndRange` - * capability — storage-only, equality-only and match-only domains. They hold no - * ordering term, so there is nothing for `order()` to sort by. + * JS property names of a v3 table's columns that `order()` cannot sort by. Two + * cases: + * + * 1. Columns with NO `orderAndRange` capability — storage-only, equality-only + * and match-only domains hold no ordering term. + * 2. ORE-backed (`*_ord_ore`) columns. They ARE `orderAndRange`-capable, but the + * builder sorts encrypted columns through a jsonb path (`col->op`), and the + * OPE term that path selects does not exist on an ORE domain — its `ob` term + * needs the superuser-only ORE opclass no jsonb path can reach. So they are + * excluded here to match the runtime rejection in `validateTransforms`, + * rather than type-checking clean and throwing at execute time. */ export type NonOrderableV3Keys
= { [K in Extract< keyof V3ColumnsOfTable
, string >]: 'orderAndRange' extends QueryTypesForColumn[K]> - ? never + ? EqlTypeForColumn[K]> extends `${string}_ord_ore` + ? K + : never : K }[Extract, string>] @@ -212,12 +227,11 @@ export type NonOrderableV3Keys
= { * path `col->op`, which selects the OPE term, and OPE is order-preserving. See * `EncryptedQueryBuilderV3Impl.orderColumnName`. * - * ORE-backed (`*_ord_ore`) columns are `orderAndRange`-capable and so pass this - * type, but are rejected at runtime: their `ob` term needs the superuser-only - * ORE opclass, which no jsonb path can reach. Encoding the ordering FLAVOUR in - * the type system would mean threading it through `QueryTypesForColumn`, and - * such a column cannot hold data on managed Postgres anyway (its domain CHECK - * raises `ore_domain_unavailable`), so the runtime guard is where it belongs. + * ORE-backed (`*_ord_ore`) columns are excluded at compile time by + * {@link NonOrderableV3Keys} — the builder sorts through a jsonb path that + * cannot reach their superuser-only ORE opclass, so `.order()` on one is a type + * error, matching the runtime rejection in `validateTransforms` (defense in + * depth for the untyped `.order(someString)` path). */ export type V3OrderableKeys< Table extends AnyV3Table, diff --git a/vitest.shared.ts b/vitest.shared.ts index 525bc12f4..7c66d8dfa 100644 --- a/vitest.shared.ts +++ b/vitest.shared.ts @@ -46,4 +46,10 @@ export const sharedAlias: Record = { repoRoot, 'packages/stack/src/supabase/index.ts', ), + // Bare entry LAST: it is a prefix of every subpath above, and Vite matches in + // order, so the subpaths must win first. Both sibling tsconfigs map bare + // `@cipherstash/stack` → `src/index.ts`; without the runtime match here a + // consumer importing the bare specifier would fall through to + // `packages/stack/dist`, re-coupling `pnpm test` to a prior `pnpm build`. + '@cipherstash/stack': resolve(repoRoot, 'packages/stack/src/index.ts'), }