Skip to content

feat: migrate ClickHouse adapter to utopia-php/query 0.3.x builder#120

Closed
lohanidamodar wants to merge 26 commits into
mainfrom
feat/utopia-query-0.3.x
Closed

feat: migrate ClickHouse adapter to utopia-php/query 0.3.x builder#120
lohanidamodar wants to merge 26 commits into
mainfrom
feat/utopia-query-0.3.x

Conversation

@lohanidamodar

@lohanidamodar lohanidamodar commented May 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Migrates the ClickHouse audit adapter off the legacy utopia-php/query 0.1.x
contract onto the 0.3.x Builder\ClickHouse / Schema\ClickHouse API.

Every SQL and DDL string the adapter emits now comes from the query library.
The audit side keeps only the HTTP transport, the column → ClickHouse type map
that turns positional ? bindings into typed {paramN:Type} placeholders, and
the tuple-cursor compilation.

This branch is a strict superset of main. origin/main is an ancestor of
this branch (0 commits behind), so everything shipped up to and past 2.9.1 is
included and re-expressed through the 0.3 builder where it touches ClickHouse.

Dependencies

"require": {
    "php": ">=8.5",
    "utopia-php/database": "^6.0.0",
    "utopia-php/fetch": "^1.1",
    "utopia-php/query": "0.3.*",
    "utopia-php/validators": "0.3.*"
}

The only line that differs from main is utopia-php/query: 0.1.*0.3.*.

  • utopia-php/query 0.3.3 is tagged and stable — no dev-branch pins
    anywhere, minimum-stability is back to stable, stability-flags is empty.
  • php >=8.5 and utopia-php/validators 0.3.* are taken from main.
  • utopia-php/fetch ^1.1 stays as the HTTP transport, matching main. (An
    earlier revision of this branch swapped in PSR-18 utopia-php/client; that
    has been reverted so the adapter constructor signature is byte-identical to
    main's and cloud call sites need no change.)
  • Lock resolves to utopia-php/query 0.3.3, utopia-php/validators 0.3.1,
    utopia-php/database 6.4.5, utopia-php/fetch 1.1.2.

What is compiled by the query library now

Adapter entry point 0.3 API used
setup() Schema\ClickHouse — columns, LowCardinality(...), Nullable(...), bloom-filter indexes, ENGINE, ORDER BY, PARTITION BY, SETTINGS
createBatch() Builder\ClickHouse::bulkInsert(Format::JSONEachRow, $rows, $columns) — emits the INSERT … FORMAT JSONEachRow envelope and serializes the body
find() Builder\ClickHousefilter(), sortAsc/sortDesc, sortRandom(), limit(), offset()
count() Builder\ClickHouse, with the $max variant wrapping an inner SELECT 1 … LIMIT
getById() Builder\ClickHouse::filter() + limit(1)
cleanup() Builder\ClickHouse::delete() with optional trailing SETTINGS lightweight_deletes_sync = 0

Typed placeholders come from useNamedBindings() + withParamTypes(), fed by a
column → type map derived from getAttributes() (DateTime columns get
DateTime64(3), tenant/limit/offset/max get UInt64, the rest
String).

Main features carried over and routed through the builder

Everything main added since this branch last synced now goes through
Schema\ClickHouse / Builder\ClickHouse:

  • Retention TTL (setRetention() / getRetention()) — MODIFY TTL … SETTINGS materialize_ttl_after_modify = 0, and the idempotent REMOVE TTL teardown
    that tolerates tables with no TTL.
  • sdk / sdkVersion columns and the _key_sdk bloom-filter index.
  • Premium geo columnscity, continentCode, subdivisions, isp,
    autonomousSystemNumber, autonomousSystemOrganization, connectionType,
    connectionUsageType, connectionOrganization.
  • Parsed user-agent columnsosCode, osName, osVersion, clientType,
    clientCode, clientName, clientVersion, clientEngine,
    clientEngineVersion, deviceName, deviceBrand, deviceModel.
  • The expanded LOW_CARDINALITY_COLUMNS set, emitted via the schema
    builder's lowCardinality(); nullable ones compile to
    LowCardinality(Nullable(String)).
  • contains / notContains substring semantics. Main replaced the old
    IN / NOT IN behaviour with LIKE '%needle%' / NOT LIKE '%needle%' plus
    manual wildcard escaping. The 0.3 ClickHouse builder compiles these natively
    to position(col, ?) > 0 and position(col, ?) = 0 — same results, same
    OR/AND combination across multiple needles, and no wildcard escaping needed,
    so escapeLikeWildcards() is gone.
  • cleanup() DateTime64(3) parameter type, readonly promoted constructor
    properties, and typed class constants.

Query 0.3 API notes

  • Query::getMethod() returns the Utopia\Query\Method enum. All method
    comparisons in Adapter\ClickHouse and Adapter\Database are now
    Method:: case comparisons, and VALUE_REQUIRED_METHODS holds enum cases.
  • Utopia\Audit\Query still exposes the legacy TYPE_* string constants
    mapping to the same string values, so external call sites keep working.
  • No public API change. Audit::log() and Audit::logBatch() signatures
    are byte-identical to main, and the ClickHouse constructor is unchanged.

Deferred / known gaps

  • Typed cursor comparisons. Keyset pagination still builds the
    (a > A) OR (a = A AND b > B) … cascade in the adapter and feeds it through
    whereRaw() with hand-typed placeholders; the base builder does not yet emit
    ClickHouse-flavoured tuple comparisons. Those raw bindings are merged with the
    builder's namedBindings at the call site.
  • Tenant filter still rides through whereRaw().
  • Utopia\Query\Query::contains() is deprecated upstream in 0.3.3 in favour of
    containsString() / containsAny(). Utopia\Audit\Query inherits it, so
    callers using Query::contains() will see a deprecation notice. The library
    itself no longer calls it (main already moved findByUserAndEvents() and
    friends to Query::equal()). Renaming the audit-side factory is out of scope
    for this PR.

Test plan

Run on PHP 8.5.8. Docker is unavailable on the machine used for this sync,
so the ClickHouse/MariaDB-backed suite could not be executed — see the
unverified section below.

Verified:

  • composer validate./composer.json is valid.
  • composer update resolves cleanly to stable utopia-php/query 0.3.3
    with no dev pins.
  • Pint (monorepo pint.json, per preset) — {"tool":"pint","result":"passed"}
    over src and tests.
  • PHPStan at the monorepo bar (level 5) — [OK] No errors.
  • PHPStan at level max — src/ is completely clean. Three findings remain,
    all in test files and all pre-existing on main
    (ClickHouseTest.php:630 offsetAccess.invalidOffset, and
    AuditBase.php:818 return.type in two class contexts). This branch also
    removes two argument.type findings main has at AuditBase.php:251.
  • phpunit --testsuite unit22 tests, 106 assertions, 0 failures
    (3 deprecations, all the upstream Query::contains() notice above).
    ClickHouseSqlSnapshotTest is now part of the unit suite since it
    needs no server, and is excluded from e2e to avoid running twice.
  • Adapter SQL generation exercised end-to-end against a recording HTTP
    stub, covering setup() (both retention branches, shared-tables on/off),
    find() (multi-value equal, contains, notContains, DateTime comparison,
    select projection, orderRandom, cursorBefore direction flip),
    count() with and without $max, getById(), cleanup() async, and
    createBatch(). The emitted CREATE TABLE contains every column main
    added, with the correct LowCardinality(Nullable(String)) wrapping, all
    12 bloom-filter indexes including _key_sdk, ORDER BY (tenant, time, id)
    and allow_nullable_key = 1 under shared tables. All of main's error
    messages are reproduced verbatim.

Not verified

  • phpunit --testsuite e2e did not run — no Docker daemon available on
    this machine
    , and no ClickHouse or MariaDB was reachable on any of the
    compose ports. Unrun classes:
    tests/Audit/Adapter/ClickHouseTest.php and
    tests/Audit/Adapter/DatabaseTest.php. These need
    docker compose up (ClickHouse 25.11 + MariaDB 10.11) before merge.

Note on repository layout

main has since been absorbed into utopia-php/monorepo and this repository is
now a mirror (.github/workflows/mirror.yml, phpstan.neon including
../../phpstan.neon, no require-dev, no pint.json, no Dockerfile). This
branch follows main's layout rather than restoring the removed files; the
tooling above was run with the monorepo's own pint.json and PHPStan level.

Pins to dev-feat/clickhouse-insert-delete-settings-mv as 0.3.2 — the branch
on utopia-php/query (PR #11) that adds the three ClickHouse builder
extras audit needs:

- Builder\ClickHouse::insertFormat(...) for INSERT ... FORMAT JSONEachRow
- Trailing SETTINGS clause on DELETE (for async cleanup)
- Schema\ClickHouse::create/dropMaterializedView (not used here)

TODO: flip to ^0.3.2 once PR #11 lands on utopia-php/query main and a
0.3.2 tag exists.
The 0.3.x utopia-php/query base class replaced the legacy `TYPE_*` string
constants with a `Method` enum. Audit's adapter switches and tests all
still compare against the string constants, so they're re-declared here
mapped to the same string values (`equal`, `lessThan`, ...).

Tests that compared `$query->getMethod()` (now returns a `Method` enum)
against the constants are updated to compare against `getMethod()->value`.
utopia-php/query 0.3.x uses PHP 8.4 asymmetric visibility
(`public protected(set)`) on schema and builder properties. PHPStan
1.12's bundled PhpParser can't tokenize that syntax — `composer check`
crashes with a Lexer internal error on any audit source file that
imports a query schema/builder class.

PHPStan 2.x ships an updated PhpParser that handles asymmetric
visibility, so bumping the dev dependency is the smallest change that
restores the static-analysis gate.
…sites

utopia-php/query 0.3.x's `Query::getMethod()` returns a `Method` enum
instead of a string. The ClickHouse adapter's parseQueries switch, and
the Database adapter's count-time filter, both compare against the
legacy string `TYPE_*` constants. Switch from `getMethod()` to
`getMethod()->value` at the two call sites so the existing comparisons
keep matching.
Bumping PHPStan to 2.x to handle PHP 8.4 syntax surfaces a handful of
pre-existing diagnostics in code paths the migration does not otherwise
touch:

- Database adapter: redundant `instanceof Utopia\Audit\Query` guards
  inside a method whose signature already constrains the parameter.
  Kept as runtime defense (real callers occasionally pass mixed
  arrays); annotated with @phpstan-ignore-next-line.
- Log::getData(): PHPStan can't widen the ArrayObject return without a
  local @var hint.
- AuditBase batch test: removed a duplicate `applyRequiredAttributesToBatch`
  call (typo; the second invocation already re-merged the same row).
Replace the hand-built CREATE TABLE in `setup()` with a `Schema\ClickHouse`
table builder. Engine, ORDER BY tuple, PARTITION BY expression, table
settings, columns, and data-skipping (bloom_filter) indexes are now
declared through the schema API; the resulting Statement is executed
verbatim through the existing HTTP layer.

Column definitions follow the audit attribute descriptors:
- `id String` (primary)
- `time DateTime64(3)` (NOT NULL — partition key)
- `<id> [Nullable(]String[)]` for every other attribute, nullability
  driven by the descriptor's `required` flag
- `tenant Nullable(UInt64)` when sharedTables is on

CREATE DATABASE IF NOT EXISTS still goes through a raw string — the
schema's `createDatabase()` helper has no IF NOT EXISTS form and we'd
otherwise break the second `setup()` call.

Also folds in a few PHPStan-driven cleanups in this file (collapsed
`!empty($inParams)` guards into unconditional emits since the upstream
VALUE_REQUIRED_METHODS guard already rejects empty value lists; dropped
the redundant is_array($row) inside parseJsonResults whose row type was
already array<string,mixed>).
…eBatch

Build the JSONEachRow INSERT statement through `Builder\ClickHouse::insertFormat`
instead of a hand-rolled string. Column list is derived from the audit
schema (id, time, the remaining attributes, optional tenant) in the
same insertion order as the row maps so the JSON keys line up against
ClickHouse's declared columns.

The JSONEachRow body itself still serializes in the adapter — that's an
HTTP-payload concern that stays in the runtime layer.
Compile the cleanup DELETE through `Builder\ClickHouse::delete()` with a
trailing SETTINGS clause emitted by the builder when async cleanup is
enabled. The DELETE WHERE expression is supplied via `whereRaw()` so the
ClickHouse-typed parameter syntax (`{datetime:DateTime64(3)}`) and the
existing tenant filter stay in the runtime layer — the builder still
emits generic `?` placeholders today (dry-run gap #7).

Behaviour note: the ClickHouse builder compiles DELETE as
`ALTER TABLE ... DELETE` (mutation) instead of the lightweight
`DELETE FROM ...` the previous string emitted. The async setting tracks
the same shift: `mutations_sync = 0` for mutations replaces the
previous `lightweight_deletes_sync = 0`. End-state row visibility is
the same; the storage path is different (mutations rewrite parts,
lightweight deletes mask rows).
Route the SELECT skeleton (FROM, raw projection, WHERE conjunction,
ORDER BY) for `find()`, `count()`, and `getById()` through
`Builder\ClickHouse`. Filter expressions emitted by `parseQueries()`
still carry ClickHouse-typed parameter hints (`{name:Type}`) and the
audit-side associative params map; both ride along via `whereRaw()` /
`orderByRaw()` so the typed-binding HTTP layer keeps working.

`LIMIT` / `OFFSET` and the trailing `FORMAT JSON` / `FORMAT TabSeparated`
are appended on the compiled SQL string — the base builder emits
positional `?` placeholders for limit/offset, which would collide with
the ClickHouse `{name:UInt64}` placeholders the runtime layer binds.
Cleaning that up belongs with the gap #7 (ClickHouse param hint) work
on utopia-php/query.

Cursor pagination still builds tuple WHERE fragments through the
existing `buildCursorWhere()` helper and feeds them in via `whereRaw()`
(gap #8, deferred).
Pin the SQL emitted by `Schema\ClickHouse` and `Builder\ClickHouse`
through the audit configurations the adapter relies on:

- `setup()` CREATE TABLE — engine, columns, indexes, ORDER BY,
  PARTITION BY, SETTINGS
- `createBatch()` `INSERT ... FORMAT JSONEachRow` with the audit
  column list
- `cleanup()` async DELETE with trailing SETTINGS clause
- `cleanup()` sync DELETE with no SETTINGS
- `find()` SELECT with whereRaw filters, cursor tuple comparison,
  ORDER BY tiebreaker, LIMIT and FORMAT JSON tail

These do not require a live ClickHouse so they run as fast unit tests
and prevent silent SQL drift from a query-lib upgrade.
@greptile-apps

greptile-apps Bot commented May 17, 2026

Copy link
Copy Markdown

Greptile Summary

The PR migrates ClickHouse SQL generation to the utopia-php/query 0.3 builder while preserving existing adapter behavior.

  • Replaces handwritten ClickHouse DDL, filtering, counting, insertion, and cleanup SQL with schema and query builders.
  • Adds typed named bindings for ClickHouse parameters and retains custom cursor and tenant predicates.
  • Updates query-method handling to the new enum-based API and expands SQL snapshot coverage.
  • Upgrades the query dependency to 0.3.x and refreshes the dependency lock.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported cleanup semantics, malformed-row guard, multi-value exclusion coverage, and cursor predicate grouping are corrected at current HEAD.

Important Files Changed

Filename Overview
src/Audit/Adapter/ClickHouse.php Migrates ClickHouse operations to the new builders; the previously reported cleanup, malformed-row, multi-value exclusion, and cursor-grouping concerns are addressed at current HEAD.
src/Audit/Adapter/Database.php Updates query-method comparisons for the enum-based query API without an identified behavioral regression.
src/Audit/Query.php Preserves legacy audit query constants while adapting to the upstream query contract.
tests/Audit/Adapter/ClickHouseSqlSnapshotTest.php Adds focused SQL-shape coverage, including lightweight cleanup, multi-value exclusions, typed bindings, and correctly grouped cursor predicates.
composer.json Updates utopia-php/query from 0.1.x to stable 0.3.x for the builder migration.
composer.lock Resolves the upgraded query package and associated transitive dependency updates.

Reviews (11): Last reviewed commit: "Merge origin/main into feat/utopia-query..." | Re-trigger Greptile

Comment thread src/Audit/Adapter/ClickHouse.php
Comment thread src/Audit/Adapter/ClickHouse.php
Bump utopia-php/query to the upstream branch HEAD that ships named-typed
{name:Type} placeholder support and the lightweight DELETE form, then add
two helpers to the adapter:

- getColumnTypeMap() derives a column → ClickHouse-type map from the
  schema attributes (DateTime → DateTime64(3), all other columns →
  String) plus id, tenant (when sharedTables is enabled) and the
  limit/offset/max pseudo-columns used by the count/find SQL wrappers.

- newBuilder() returns a Builder\ClickHouse with useNamedBindings() and
  withParamTypes() pre-applied, so positional `?` bindings flow through
  the typed-binding rewriter at Statement-emission time.

Every existing adapter call site is rerouted from `new ClickHouseBuilder()`
to `$this->newBuilder()`. The current call sites all hand-construct their
{name:Type} placeholders inside whereRaw fragments with zero `?` bindings,
so this change is a no-op for the existing SQL shape — it just preps the
infra that the find()/count()/getById() reads will switch to next.
…ter API

Drop every hand-built WHERE / ORDER BY fragment in find(), count() and
getById() and feed Query value objects straight to Builder\ClickHouse via
filter(), sortAsc/sortDesc/sortRandom, limit() and offset(). Positional
`?` bindings are rewritten to `{paramN:Type}` placeholders by the
typed-binding registration installed in newBuilder(), so HTTP params now
flow through $statement->namedBindings instead of a hand-maintained
$paramCounter dict.

parseQueries() is reduced to a list of Query objects plus auxiliary
metadata (orderAttributes, limit, offset, cursor, select). Two
audit-specific rewrites stay in this layer:

  - Contains / NotContains are remapped to Equal / NotEqual so they keep
    audit's historical IN / NOT IN semantics. The base builder compiles
    Contains to substring-match `position(x, ?) > 0`, which is not what
    callers like Audit::getByUserAndEvents() expect.
  - `time`-column DateTime values are stringified at parse time so they
    appear in namedBindings as `Y-m-d H:i:s.v` literals rather than raw
    objects the HTTP layer can't serialise.

Cursor pagination keeps its whereRaw escape hatch with explicit
{name:Type} placeholders — Builder\ClickHouse still has no tuple-compare
helper, so the existing buildCursorWhere() output is appended to the
builder and its params are merged into the final HTTP request alongside
$statement->namedBindings. The dead buildOrderBySql() helper is removed;
applyOrderBy() drives the builder directly.
Builder\ClickHouse now defaults to DELETE_MODE_LIGHTWEIGHT, so cleanup()
emits `DELETE FROM t WHERE …` again — matching audit's pre-migration
baseline. The previous mutation form (`ALTER TABLE t DELETE WHERE …`)
was a workaround forced by the older builder API and changed the
storage-path semantics: lightweight marks rows deleted via a mask and is
the right tool for row-level cleanup, while mutations rewrite parts on
disk and are heavier.

The async SETTINGS knob switches from `mutations_sync = 0` to
`lightweight_deletes_sync = 0` to match the new DELETE form. The public
setAsyncCleanup() docblock already referenced lightweight_deletes_sync,
so no docs change is needed.
…LETE

Pin the new SQL shape on the adapter's hot paths:

  - cleanup() emits `DELETE FROM t WHERE …` with an optional trailing
    `SETTINGS lightweight_deletes_sync=0` for the async path.
  - find() / count() filter via Builder\ClickHouse::filter() and
    sortAsc/sortDesc/limit, producing typed `{paramN:Type}` placeholders
    and a populated `namedBindings` map on the Statement.
  - The cursor whereRaw fragment with explicit `{name:Type}` placeholders
    composes cleanly with the typed positional bindings — both end up in
    the final HTTP params dict by the adapter merging them.
  - count(max) wraps the inner Statement in `SELECT COUNT(*) FROM (… LIMIT ?) sub`
    and reuses the inner builder's namedBindings unchanged.

A small newAuditBuilder() / auditTypeMap() helper mirrors the adapter's
own column → ClickHouse-type registration so the snapshot tests exercise
the same typed-binding flow callers will see in production.
The Dockerfile pinned php:8.3.3-cli-alpine3.19, but composer.json
requires php >=8.4. Once utopia-php/query: ^0.3.0 was adopted, the CI
Tests check failed because the library uses 8.4-only asymmetric
visibility syntax (public protected(set)). Bump the test image to
php:8.4.21-cli-alpine3.23 so the CI environment matches the package
requirement.
Comment thread src/Audit/Adapter/ClickHouse.php Outdated
ClickHouse JSON responses come from json_decode and aren't statically
guaranteed to satisfy any inner shape. The pre-migration code skipped
non-array rows defensively; restore that guard and loosen the @var on
\$data to array<int, mixed> so PHPStan max accepts the runtime check
instead of pruning it as already-narrowed.
Adds a snapshot test that locks the SQL emitted when TYPE_NOT_CONTAINS
is mapped to Method::NotEqual with multiple values. The base builder
compiles NotEqual with 2+ non-null values via compileNotIn(), producing
`attr NOT IN (?, ?)`; with named typed bindings on Builder\ClickHouse
this becomes `attr NOT IN ({param0:Type}, {param1:Type})`. Pinning the
shape guards against a query-lib drift silently degrading multi-value
notContains() to a single-value `!=` filter.
Comment on lines +2061 to +2063
if ($this->asyncCleanup) {
$builder->settings(['lightweight_deletes_sync' => '0']);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 The cleanup() builder uses lightweight_deletes_sync here, but the PR description states the setting was changed to mutations_sync = 0. DELETE FROM is ClickHouse's lightweight-delete syntax, and lightweight_deletes_sync is the correct setting for it — so the code and snapshot test are internally consistent. However, the PR description is factually wrong when it says this was switched to mutations_sync, which could mislead future maintainers into thinking the mutation semantics changed.

Suggested change
if ($this->asyncCleanup) {
$builder->settings(['lightweight_deletes_sync' => '0']);
}
if ($this->asyncCleanup) {
$builder->settings(['mutations_sync' => '0']);
}

Brings in actor-terminology rename (PR #122), location-column drop /
N-part resource parser (PR #118), and country / userAgent test refactors.

Conflict resolution (src/Audit/Adapter/ClickHouse.php parseQueries):
- Kept main's call to translateAttribute() right after reading
  $query->getAttribute() so callers passing legacy 'userId' /
  'userType' / 'userInternalId' Query filters are rewritten to the
  renamed 'actorId' / 'actorType' / 'actorInternalId' columns before
  the typed-bindings layer compiles them.
- The column type map registered via withParamTypes() is derived from
  getAttributes() and already uses the actor* names, so once the
  attribute is translated the map binds it as String correctly.
- Public Audit API (Audit::log, getLogsByUser, countLogsByUser,
  getLogsByUserAndEvents, countLogsByUserAndEvents) is unchanged --
  the ClickHouse adapter's getByUser / countByUser implementations
  already build internal Query::equal('actorId', $userId) filters.

ClickHouseSqlSnapshotTest:
- Updated the synthetic audit type map and DDL snapshot to reflect
  actorId / actorType / actorInternalId columns, idx_actorId_event,
  and the dropped location column. Find / count / cursor snapshots
  now filter on actorId.
utopia-php/query 0.3.2 is now tagged and published with all ClickHouse
builder/schema features this PR depends on. Switch from the temporary
dev-branch alias to the stable semver constraint.
Re-introduces the dev-branch pin (b26baf9) for the duration of
utopia-php/query PR #13's lifecycle so audit can adopt the new typed
Builder\ClickHouse::bulkInsert(Format, rows, columns) bulk-ingest entry
point ahead of the 0.3.3 tag. Relaxes composer's minimum-stability to
dev with prefer-stable=true so the alias resolves.

TODO: flip composer.json back to "utopia-php/query": "^0.3.3" and
minimum-stability: "stable" once 0.3.3 is tagged.
…nual JSONEachRow body assembly

Switches createBatch() to the typed Builder\ClickHouse::bulkInsert(
Format::JSONEachRow, $rows, $columns) entry point so the INSERT envelope
and the serialized row body are produced together. The HTTP helper now
takes a pre-serialized $rawBody string instead of a $jsonRows array,
deleting the inline array_map(json_encode, ...) + implode("\n", ...)
body assembly loop — that serialization now lives in Format::serialize()
on the builder side, with the JSON flags (JSON_THROW_ON_ERROR,
JSON_UNESCAPED_SLASHES, JSON_UNESCAPED_UNICODE) baked into Format::JSONEachRow.

The SQL envelope shape is unchanged (`INSERT INTO ... (cols) FORMAT
JSONEachRow`); the snapshot test is updated to drive bulkInsert and now
pins the JSONEachRow body serialization with a fixture row, so an
upstream regression in Format::serialize() would be caught by CI.
Adopt utopia-php/database ^6.0.0 from main and express main's shared-table
sort key (tenant, time, id), allow_nullable_key and LowCardinality columns
through the Schema builder. LowCardinality is applied to required columns
only because the query builder emits Nullable(LowCardinality(String)) for
nullable columns, which ClickHouse rejects.
… utopia-php/client

Swap the Fetch client for the PSR-18 utopia-php/client cURL adapter with
connection reuse so the TCP handshake is paid once across queries, matching
the transport used by utopia-php/usage. The constructor accepts an optional
injected ClientInterface for custom transports.
…inality on country

Query 0.3.3 compiles lowCardinality()->nullable() to LowCardinality(Nullable(T)),
so the nullable country column again matches main's DDL.
Syncs the query 0.3.x migration onto audit main (post-2.9.1), keeping
everything main added since the previous sync and re-expressing the
ClickHouse-adapter parts through the utopia-php/query 0.3 builder.

Dependencies
- utopia-php/query 0.1.* -> 0.3.* (locked 0.3.3, stable; no dev pins)
- php >=8.5 and utopia-php/validators 0.3.* from main
- keeps main's utopia-php/fetch ^1.1 HTTP transport
- minimum-stability back to "stable"

Carried over from main and routed through the 0.3 builder
- retention TTL (MODIFY TTL / REMOVE TTL) in setup()
- sdk / sdkVersion columns and the _key_sdk index
- premium geo columns (city, continentCode, subdivisions, isp,
  autonomousSystemNumber, autonomousSystemOrganization, connectionType,
  connectionUsageType, connectionOrganization)
- parsed user-agent OS / client / device columns
- LowCardinality column set, emitted via Schema\ClickHouse::lowCardinality()
- contains/notContains substring semantics, now compiled by the builder to
  position(col, ?) > 0 / = 0 instead of hand-written LIKE '%needle%'
- cleanup() DateTime64(3) parameter type
- readonly promoted constructor properties and typed class constants

Query 0.3 API
- Query::getMethod() returns the Utopia\Query\Method enum; all method
  comparisons in ClickHouse and Database adapters use Method:: cases
- Utopia\Audit\Query still exposes the legacy TYPE_* string constants
@lohanidamodar

Copy link
Copy Markdown
Contributor Author

Moving this to the monorepo: utopia-php/monorepo#91

utopia-php/audit is now a read-only subtree-split mirror of utopia-php/monorepo, so development happens in packages/audit there. The monorepo PR carries the full net diff of this branch, reconciled with the monorepo's pint/PHPStan/Rector configuration, with utopia-php/query pinned to the tagged 0.3.*.

The feat/utopia-query-0.3.x branch stays in place here — appwrite-labs/cloud#3965 still resolves dev-feat/utopia-query-0.3.x at c1aefab. Closing the PR only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant