Skip to content

Generate SDK from the Omni OpenAPI spec#20

Open
jdavidson wants to merge 11 commits into
mainfrom
jamie/generated-sdk
Open

Generate SDK from the Omni OpenAPI spec#20
jdavidson wants to merge 11 commits into
mainfrom
jamie/generated-sdk

Conversation

@jdavidson

@jdavidson jdavidson commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Replaces the hand-written OmniAPI class (~30 endpoints, drifted from the API) with a client generated from the official OpenAPI 3.1 spec — 195 operations across 22 resource groups, 597 typed models, sync + async — plus a small hand-written helper layer for the query → pyarrow flow.

How it works

  • spec/openapi.json is vendored from the omni repo (also served at <instance>/openapi.json).
  • scripts/preprocess_spec.py collapses zod-to-openapi allOf description wrappers and stubs ~15 recursive document-layout/vis schemas as free-form dicts (the only spec constructs openapi-python-client can't model).
  • scripts/generate.sh regenerates omni_python_sdk/ from the spec (deterministic via pinned PYTHONHASHSEED), preserving the hand-written omni_python_sdk/helpers.py.
  • CI runs the test suite on Python 3.10/3.12 and a drift check that regenerates from the checked-in spec and fails on any diff.

Breaking changes (v1.0.0)

  • OmniAPI is gone. client_from_env() + run_query_blocking() in omni_python_sdk.helpers cover the old query flow; everything else is omni_python_sdk.api.<tag>.<operation>.sync(client=client, ...). Migration table in the README.
  • Errors now raise (httpx exceptions / typed responses) instead of printing and returning None.
  • Python floor moves 3.9 → 3.10; packaging moves to pyproject.toml (hatchling), and the broken dotenv dependency name is fixed (python-dotenv).
  • Known gap: the PNG response of GET /api/v1/ai/jobs/{jobId}/vis is omitted (generator doesn't support image/png); the old /embed/sso/generate-url endpoint isn't in the public spec — the embed example calls it via the raw httpx escape hatch.

Examples & tests

  • All examples ported; examples/model.py previously called a method that didn't exist (update_field) and now works via models_update_field.
  • New pytest suite: client smoke tests, an all-195-endpoints import check, and mocked-transport tests for the NDJSON + Arrow IPC query decode. Existing semantic-view example tests still pass.

Verification

  • pytest — 15 passed (fresh venv, editable install)
  • examples/tests unittest suite — 6 passed
  • scripts/generate.sh run 3× → identical output (drift check is stable)
  • All examples compile; no OmniAPI references remain
  • Live smoke test against a real instance — passed: whoami, models_list (20 models), topic fetch, and a LIMIT-1 query decoded to a 1×1 pyarrow.Table

🤖 Generated with Claude Code

jdavidson and others added 6 commits July 16, 2026 12:34
Adds the Omni OpenAPI 3.1 spec (from the omni repo, also served at
<instance>/openapi.json), a preprocessing script that collapses
zod-to-openapi allOf-description wrappers and stubs recursive layout/vis
schemas, an openapi-python-client config, and scripts/generate.sh to
regenerate the package.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generated by openapi-python-client 0.29.0 from spec/openapi.json via
scripts/generate.sh: 195 operations across 22 tag modules, 597 typed
models, sync + async. Removes the hand-written api.py (~30 endpoints).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- omni_python_sdk/helpers.py (hand-written, preserved by generate.sh):
  client_from_env plus run/wait_query_blocking that decode the query
  API's NDJSON + base64 Arrow IPC payload into a pyarrow.Table
- pyproject.toml (hatchling, v1.0.0, Python >=3.10) replacing
  setup.py/requirements.txt; fixes the broken dotenv dependency name
- pytest suite: client smoke tests, all-endpoints import check, and
  mocked-transport tests for the query decode path
- README rewritten for the generated client with 0.x migration notes
- CI workflow: tests on 3.10/3.12 plus a generated-code drift check

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mixed-case enum values (e.g. Literal["Remove", "remove"]) were emitted
in set-iteration order, which varies with Python's hash randomization
and would make the CI drift check flaky. Pin the seed in generate.sh
and commit the canonical seed-0 output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- query.py / example_usage.py: AuthenticatedClient + run_query_blocking
- content_migration.py: unstable_documents_export/import modules
- model.py: fixes the broken update_field call using models_update_field
- generate_embed_url.py: raw httpx call (endpoint not in the public spec)
- user_management: new scim_helpers.py reimplementing upsert/group
  membership on the generated SCIM modules
- semantic-view examples: models_get_topic for topic fetch

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Mark generated files linguist-generated so GitHub collapses them in
  PR diffs (helpers.py stays reviewable)
- Stop tracking spec/openapi.processed.json (derived; rebuilt by
  generate.sh)
- Record spec provenance (source, omni commit SHA, sha256, timestamp)
  in spec/provenance.json on each spec sync
- Document the generated-code review contract and SDK versioning
  policy in the README

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jdavidson

Copy link
Copy Markdown
Collaborator Author

For reviewers — how to review this PR: generated files are now marked linguist-generated (collapsed by default in the Files tab). The reviewable surface is just: spec/openapi.json provenance, omni_python_sdk/helpers.py, scripts/, tests/, examples/, pyproject.toml, workflows, and the README — roughly 1,500 lines. The generated ~85k lines don't need eyes: CI's drift check regenerates from the checked-in spec and fails on any difference, so the generated code is provably a pure function of the spec.

Also added since the PR opened: spec/provenance.json (records the omni repo commit each spec sync came from), dropped the derived openapi.processed.json from version control, and documented the generated-code review contract + semver policy in the README.

jdavidson and others added 4 commits July 16, 2026 12:56
VERSIONING.md covers the semver contract (independent of the API's
info.version), how to classify a spec sync as major/minor/patch
including oasdiff breaking-change detection, the generator-upgrade
policy, and the release process through the existing PyPI trusted
publishing workflow. CHANGELOG.md seeds the 1.0.0 entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
openapi-python-client 0.29 requires Python >=3.11, which broke
'pip install -e .[dev]' on 3.10. The generator is codegen-only tooling;
the SDK itself still supports 3.10.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A FAILED job (e.g. a bad view name) previously raised the generic 'No
result found in the response'; now the error type and message from the
job status line are included. Found during live smoke testing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Spec synced from omni repo main @ c3fe793 (fixes exploreomni/omni#57144):
- query/run and query/wait now declare their real content types
  (text/ndjson et al) with typed stream-line models (QueryStreamJobLine,
  QueryStreamFooterLine, ...); the JSON-only QueryRun/WaitResponse
  models are gone. The generated modules still can't consume a
  multi-line stream at runtime, so helpers.run_query_blocking remains
  the query path (README note updated).
- New endpoints: AI credit-control entity groups, AI model suggestions,
  documents_v2_remove_dashboard (128 paths / 201 operations total)
- Assorted model updates (documents v2, eval runs, git settings)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant