feat: migrate ClickHouse adapter to utopia-php/query 0.3.x builder - #4
feat: migrate ClickHouse adapter to utopia-php/query 0.3.x builder#4lohanidamodar wants to merge 1 commit into
Conversation
Greptile SummaryMigrates the ClickHouse adapter to the query 0.3.x schema and builder APIs.
Confidence Score: 3/5The PR is not yet safe to merge because getTimeSeries still rejects intervals supported by the library’s grouped-query API. The stable dependency and static-analysis cleanup are in place, but the time-series path still validates and compiles through a map containing only 1h and 1d while the shared interval API supports additional minute, week, and month buckets. Files Needing Attention: src/Usage/Adapter/ClickHouse.php Important Files Changed
Reviews (7): Last reviewed commit: "feat!: migrate ClickHouse adapter to uto..." | Re-trigger Greptile |
| @@ -172,49 +166,24 @@ public function setTimeout(int $milliseconds): self | |||
| return $this; | |||
| } | |||
|
|
|||
| /** | |||
| * Enable or disable query logging for debugging. | |||
| * | |||
| * @param bool $enable Whether to enable query logging | |||
| * @return self | |||
| */ | |||
| public function enableQueryLogging(bool $enable = true): self | |||
| { | |||
There was a problem hiding this comment.
Let's keep the docs on public methods
There was a problem hiding this comment.
Restored docblocks on the public setter / lifecycle methods that lost them during the migration rewrite (__construct, setTimeout, enableQueryLogging, setCompression, setKeepAlive, setMaxRetries, setRetryDelay, setAsyncInserts, clearQueryLog, getName, healthCheck, setNamespace, setDatabase, setSecure, plus the namespace/tenant/sharedTables getters). Done in 0e0bbd6.
| 'integer' => $table->addColumn($id, ColumnType::BigInteger), | ||
| 'float' => $table->addColumn($id, ColumnType::Float), | ||
| 'boolean' => $table->addColumn($id, ColumnType::Boolean), | ||
| 'datetime' => $table->datetime($id, 3), |
There was a problem hiding this comment.
There should be shorthand methods we can use for integer, float and boolean too
There was a problem hiding this comment.
Swapped the addColumn(..., ColumnType::BigInteger|Float|Boolean) calls in the match block to the dedicated bigInteger() / float() / boolean() shorthand, plus the value BigInteger column in createDailyTable(). ColumnType import is no longer needed. SQL snapshot tests still green. Done in c2a261e.
4530574 to
8fc13b3
Compare
Bump utopia-php/query from 0.1.* to 0.3.* and compile the ClickHouse adapter's SQL through the shared builder and schema layer instead of hand-assembled strings: - Reads (find, count, sum, totals, time series, daily, hybrid daily+raw union) build through the ClickHouse query builder with typed named bindings; every schema column is pinned to its ClickHouse parameter type so placeholder types never fall back to PHP value inference. - CREATE TABLE for the events/gauges/daily tables and the daily materialized view go through the schema layer. Column shapes the typed API cannot express (DateTime64 timezone argument, LowCardinality(Nullable(...)), hyphenated skip-index names) are emitted as raw definitions so deployed DDL is byte-for-byte unchanged. Projections, ALTER-based dim backfills, the retention TTL ALTERs and the MV body stay raw SQL, which the schema layer cannot express yet. - INSERT envelopes compile via insertFormat(); the JSONEachRow body assembly and transport are unchanged. - Query 0.3 API: getMethod() returns the Method enum, so all string TYPE_* comparisons are gone. UsageQuery keeps its groupByInterval() and groupBy() factories, now mapped to Method::GroupByTimeBucket and Method::GroupBy; groupBy() also accepts an array of columns to stay signature-compatible with the base class. - contains()/containsAny()/notContains() pass straight through to the builder, which compiles them to position(column, needle) on ClickHouse. That keeps the utopia-php/database substring semantics while matching needles literally, so the adapter no longer has to escape LIKE wildcards itself.
8fc13b3 to
597c4fa
Compare
Summary
Migrates the ClickHouse adapter to
utopia-php/query0.3.x (stable pin0.3.*, resolving to the tagged 0.3.3 release — no dev-branch pins). Every table CREATE, materialized view, INSERT envelope, SELECT and DELETE now compiles throughSchema\ClickHouse/Builder\ClickHouse; the adapter keeps only its HTTP transport, routing logic, and the few DDL shapes the library cannot express yet.Rebased onto current
main(7c3fbf1) as a single commit, so the migration now also covers everything that landed while this PR was open:events_daily, including theREMOVE TTLguard anchored on error code 36 (feat(clickhouse): optional retention TTL on the raw events table #24, feat(clickhouse): extend retention TTL to the events_daily table #27)sdk/sdkVersiondimension columns (Add sdk and sdkVersion dimension columns to usage events #25)ordinalgauge dimension column (feat: add ordinal gauge dimension column #26)containsaligned with utopia-php/database substring semantics (Align contains query with utopia-php/database semantics #28)What changed
Schema (DDL via
Schema\ClickHouse)createTable()emits the events/gauges tables throughTable\ClickHouse: typed columns withcodec(),Engine::MergeTree,partitionBy,orderBy,settings.createDailyTable()usesEngine::SummingMergeTree;createDailyMaterializedView()goes throughSchema\ClickHouse::createMaterializedView(the MV body remains a hand SELECT — subquery-aggregation MV bodies do not round-trip through the builder yet).rawColumn()so the deployed DDL stays byte-for-byte identical:DateTime64(3, 'UTC')(timezone argument),LowCardinality(Nullable(String))(the typed API emits the invalidNullable(LowCardinality(...))nesting), and the hyphenated skip-index names (index-path), which the typed index API rejects.ADD PROJECTION),MODIFY SETTING, theADD COLUMN IF NOT EXISTSdim backfills and the retentionMODIFY TTL/REMOVE TTLALTERs stay raw SQL — no schema-layer support yet. The new geo /sdk/sdkVersion/ordinalcolumns flow into both the schema-layer CREATE and the raw backfill path automatically, since both read fromMetric::getEventSchema()/getGaugeSchema().Reads (SELECT via
Builder\ClickHouse)useNamedBindings()+withParamTypes(); all schema columns are pinned to their ClickHouse parameter types (DateTime64(3, 'UTC'),Int64,String) so placeholders never fall back to PHP value inference.find/findFromTable,findAggregatedFromTable(interval buckets + dim group-bys),count(including the boundedLIMIT maxsubquery),sumand the routed daily/hybrid paths,findDaily,sumDaily,sumDailyBatch,getTotal*,getTimeSeries.SUM(...) FROM (... UNION ALL ...), prefixing one side's named bindings to avoid placeholder collisions.whereRawfragment with its own named bindings (upstream builder helper deferred).contains()/containsAny()/notContains()now pass straight through to the builder. The 0.3 ClickHouse builder compiles them toposition(column, ?), which is the same substring semantics Align contains query with utopia-php/database semantics #28 introduced, and matches needles literally — so the adapter no longer needs its ownescapeLikeWildcards()helper for%/_. The adapter still validates that every needle is a string and still rejects empty value lists.Writes
addBatch()compiles theINSERT INTO t (...) FORMAT JSONEachRowenvelope viainsertFormat(); JSONEachRow body assembly and the non-retrying POST transport are unchanged.Deletes
purge()/purgeDaily()emit lightweightDELETE FROMthrough the builder.Query 0.3 API surface
Query::getMethod()returns theMethodenum, so all stringTYPE_*comparisons are gone (ClickHouse and Database adapters).UsageQuery::TYPE_GROUP_BY_INTERVAL/TYPE_GROUP_BYstring constants are removed.UsageQuery::groupByInterval()andUsageQuery::groupBy()remain and now map toMethod::GroupByTimeBucket/Method::GroupBy;groupBy()also accepts an array of columns to stay signature-compatible with the 0.3 base class.composer.lockmovesutopia-php/query0.1.1 -> 0.3.3and nothing else;utopia-php/validatorsandutopia-php/databasestay onmain's pins.Test plan
composer check(PHPStan level max oversrc+tests) — passdocker-compose.ymlstack (ClickHouse 25.11 + MariaDB 10.7) — OK (264 tests, 1305 assertions)Locally the rebase was additionally verified without Docker, against a natively installed ClickHouse 26.8.1 + MariaDB 11.8.8: 264 tests, 1293 assertions, 2 failures, 0 errors, and an
origin/main(7c3fbf1) run on the exact same servers produced an identical result — same two tests, same assertion count. Those two,ClickHouseSchemaTest::testEventsTableSwapsBloomForSetOnLowCardinalityand::testGaugesTableSwapsBloomForSetOnLowCardinality, are a ClickHouse-version artefact: 26.x renders skip indexes asINDEX `index-status` (status) TYPE set(0)(parenthesised column list) where 25.11 rendersINDEX `index-status` status TYPE set(0), and the assertions expect the un-parenthesised form. They pass on CI's 25.11 image.Green coverage includes the
SHOW CREATE TABLEschema assertions (codecs,LowCardinality(Nullable(String)),DateTime64(3, 'UTC'), hyphenatedindex-*names), projection/dim-routing, cursor pagination, purge propagation, the retention-TTL tests, and thecontainswildcard-escaping e2e test added in #28.Still deferred upstream (follow-up PRs)
whereRaw)LowCardinality(Nullable(...))column type and timezone argument forDateTime64MODIFY SETTING/ADD COLUMN IF NOT EXISTS/MODIFY TTLin the schema layer