Skip to content

release(ai): gate v1.28.7 on curated local-model and runtime qualification #743

Description

@qnbs

Context

v1.28.7 is intentionally not being cut immediately after the current cloud-AI/provider convergence work.

Desktop-download volume is presently low enough that the release can wait until the most important current AI work has converged, especially the local-model surface.

This issue is the durable AI / Local-AI pre-release gate for v1.28.7.

It is a qualification/tracking authority, not a replacement implementation owner.

Implementation ownership remains in the existing issues, especially:

Do not expand an implementation PR merely to close this tracker. A checkbox here is satisfied only from terminal implementation/qualification evidence.

The current #742 convergence must remain bounded to its existing OpenAI request-builder scope. This tracker does not authorize #742 scope expansion.


Release invariant

v1.28.7 may ship without completing every future AI roadmap item, but it must not knowingly ship:

  • an uncurated local-model surface;
  • an active default pointing at an unavailable/unusable model;
  • a broken advertised local runtime;
  • destructive persisted-model migration;
  • uncontrolled local-AI cancellation/lifecycle behavior;
  • corrupt/incomplete model-artifact admission;
  • a privacy/local-only boundary that can silently send work to the cloud;
  • a materially false local/offline capability claim.

Target sequence:

current AI convergence
        ↓
local-model/runtime curation + critical issue convergence
        ↓
AI / LOCAL-AI PRE-RELEASE GATE = PASS
        ↓
final resulting-main stabilization
        ↓
release-readiness audit
        ↓
v1.28.7 release cut / PR
        ↓
pre-tag desktop/Tauri qualification
        ↓
deliberate signed v1.28.7 tag
        ↓
artifact / updater verification

Gate A — canonical model authority / #704

PASS only when the current production model surface is explicit and internally consistent.

  • Every production model ID that is selectable, defaulted, persisted or used as a runtime fallback has an explicit authority/classification.
  • Recommended defaults are distinguished from current-supported, specialized, preview/experimental, legacy-compatible, deprecated, removed and unknown/custom persisted values.
  • No known deprecated/removed/unusable model remains an accidental active default.
  • Settings, runtime routing, fallbacks and provider adapters do not expose contradictory model/default truth.
  • OpenAI o-series catalogue/type reachability discovered during fix(ai): adapt OpenAI reasoning parameters #742 receives an explicit durable disposition under ai: requalify current model catalog, defaults and end-to-end AI functionality across all providers #704.
  • Capability admission prevents known model/provider-operation mismatches.
  • Valid historical user selections are preserved where still supported.
  • Unknown/custom IDs remain preservable where the configured provider/runtime legitimately supports arbitrary IDs.
  • Removed/unusable persisted models have an explicit non-destructive migration/error policy.
  • LOCAL / ECO / HYBRID / CLOUD routing semantics are explicit and test-backed.
  • Local-only/privacy policy cannot silently escalate to cloud.

Automatic FAIL examples:

  • active default references a removed/unusable model;
  • runtime and UI disagree on effective model authority;
  • persisted configuration is silently rewritten to an unrelated default without policy;
  • known capability mismatch remains routable;
  • local-only behavior can contact cloud without explicit user policy.

Gate B — curated local model/runtime catalogue

Every currently advertised/selectable local model/runtime must receive one explicit disposition:

ADMITTED_DEFAULT
ADMITTED_SUPPORTED
SPECIALIZED
EXPERIMENTAL_OPT_IN
LEGACY_COMPAT
UNSUPPORTED
REMOVE

WebLLM

  • Every admitted model currently exists and is loadable with the supported runtime/library version.
  • Model/quantization choice is explicit.
  • Approximate download size and practical memory class are known.
  • WebGPU/runtime requirements are truthful.
  • Successful real local inference has been demonstrated.
  • Cancellation/Stop has been exercised.
  • WebGPU-unavailable/OOM/device-loss behavior is controlled.
  • No hidden cloud fallback exists.
  • A realistic lightweight local default exists if browser generative WebLLM is presented as a normal product capability.

ONNX Runtime Web

  • Every production ONNX model/artifact is identified by current source/version.
  • Current runtime compatibility is verified.
  • WASM/WebGPU requirements are explicit.
  • Model download/cache behavior is deterministic.
  • Real local execution succeeds for the task for which the model is presented.
  • Failure/resource-boundary behavior is controlled.

Transformers.js

  • Current model/revision identifiers are verified.
  • Download and reload work.
  • Offline-after-download works where claimed.
  • The admitted model remains practically useful for its advertised WorldScript task.
  • Historical/obsolete models are not normal recommendations.

Ollama

  • Local endpoint/connection semantics are current and intentional.
  • Model enumeration/selection works.
  • Real local inference has been exercised.
  • Local-server-unavailable behavior is actionable.
  • Timeout/cancellation semantics are defined.
  • Ollama failure cannot silently trigger cloud fallback under local-only policy.

LM Studio / vLLM / custom OpenAI-compatible endpoints

Where WorldScript continues to claim support:

  • Endpoint normalization is correct.
  • Custom model IDs are preserved where legitimate.
  • Request shape is compatible with the admitted runtime contract.
  • Streaming works where claimed.
  • Auth/no-auth behavior is explicit.
  • Timeout/cancellation behavior is defined.
  • Connection testing proves meaningful usability rather than mere endpoint reachability.

Do not equate OpenAI-compatible with complete OpenAI behavioral parity automatically.


Gate C — #556 critical Local-AI / Voice lifecycle

The entirety of #556 is not automatically a v1.28.7 blocker. Its currently product-relevant correctness subset is.

  • Current Whisper model/runtime inventory is explicit.
  • Current Kokoro model/runtime inventory is explicit.
  • Local model artifact sources/versions are known.
  • Incomplete/corrupt downloads cannot be promoted as valid models.
  • Failed initialization does not blindly destroy valid downloaded artifacts.
  • Concurrent model init/download storms are prevented or bounded.
  • User cancellation has deterministic semantics.
  • Timeout is distinct from user cancellation.
  • Resource exhaustion/backend-unavailable states are actionable.
  • Offline-after-download works wherever product copy promises it.
  • Eco Mode has a concrete resource/model policy rather than a cosmetic label.
  • Local inference does not make editor/autosave behavior unusable.
  • UI truthfully distinguishes on-device/browser-local, local-network and cloud-provider execution.

Release-blocking findings include corrupt-artifact loops, uncontrolled init/download storms, hidden cloud fallback, persistent editor/autosave starvation, destructive cache/model recovery, materially misleading local/cloud labels, or reproducible critical OOM/crash without safe recovery.


Gate D — #711 Browser-Ollama privacy/admission

This subset is P1 and release-blocking.

OFF-state

With enableBrowserOllama = false:

  • Opening/mounting AI Settings/Ollama UI causes no automatic localhost:11434 request.
  • Reloading the Settings surface causes no local-network probe.
  • A saved local endpoint is not contacted merely because the UI mounted.
  • OFF remains network-inert until explicit admitted user intent.

ON-state

  • User explicitly enables Browser-Ollama.
  • Explicit Load Models / Test Connection or equivalent action produces the expected request.
  • Deterministic success produces correct UX.
  • Unreachable/CORS/local-network failure produces actionable classification.
  • CORS/CSP protections are not weakened to make the feature work.
  • Required CI does not depend on a real Ollama daemon accidentally present on a runner.

Any unsolicited local-network request while Browser-Ollama is disabled is an automatic FAIL.


Gate E — #714 local-runtime lifecycle subset

For every local runtime admitted into v1.28.7:

start
→ active/streaming
→ cancel/stop/supersede
→ terminal
→ no stale resurrection
  • Caller cancellation reaches the lowest practical runtime/transport boundary.
  • USER_CANCELLED is not treated as ordinary provider failure.
  • TIMEOUT is distinct from user cancellation.
  • Cancellation stops retry/fallback escalation where policy requires.
  • Stop cannot accidentally trigger a cloud fallback.
  • Late stream chunks after cancel/supersession are rejected.
  • Late completion cannot overwrite a newer request.
  • Repeated Stop → Generate cycles remain functional.
  • Request-specific listeners/readers/workers/timers are cleaned up.
  • Cancellation does not unnecessarily unload/delete reusable local model state.
  • OOM/device-lost/runtime-terminated states are differentiated sufficiently for safe recovery.

Gate F — conditional promotion from #483

#483 as a whole does not block v1.28.7.

Promote only a finding affecting an already advertised/reachable production capability, including:

  • advertised offline local AI does not actually work offline;
  • corrupt/incomplete artifacts can be treated as valid;
  • missing WebGPU produces a crash instead of controlled fallback;
  • local execution silently escalates to cloud;
  • currently recommended browser-local model is no longer loadable;
  • local inference materially breaks editor/autosave usability;
  • product/runtime capability claims are materially false.

Long-term OPFS/model-manager/multi-tab/browser-AI productization may remain post-release unless promoted by the rules above.


Gate G — conditional promotion from #717

#717 as a whole does not block v1.28.7.

Promote only confirmed production-impacting findings such as:

  • production GPU path returns materially incorrect results versus its reference path;
  • WebGPU failure crashes functionality instead of safely falling back;
  • device/buffer/resource leak has realistic user impact;
  • GPU capability detection causes an unusable local-AI default;
  • default-on Compute Shader policy causes incorrect product behavior;
  • privacy/routing truth depends on an incorrect GPU/runtime decision.

Pure experimental WGSL expansion, benchmark refinement or roadmap cleanup remains post-release unless promoted by current production evidence.


Gate H — real local-runtime qualification

Mocks alone are insufficient for final release qualification.

For every runtime actually admitted into the release, exercise real supported environments where applicable, including the currently admitted subset of:

  • WebLLM/browser-local generative model;
  • ONNX Runtime Web task(s);
  • Transformers.js task(s);
  • Ollama;
  • LM Studio / vLLM if retained as explicitly supported presets;
  • Whisper;
  • Kokoro.

Minimum real qualification contract:

  • initialization/download succeeds;
  • inference succeeds;
  • result is structurally/usefully valid;
  • cancellation succeeds, or runtime limitations are explicitly documented with safe late-result rejection;
  • a second inference after cancellation still works;
  • offline operation works where promised;
  • unsupported hardware/runtime fails safely;
  • no unexpected cloud request occurs.

Do not assert exact creative prose. Assert stable properties such as successful admission, non-empty/valid result, expected schema/task result, correct execution backend and privacy-boundary preservation.

Paid/live cloud-provider qualification that requires purchasing or supplying provider credentials remains deferred unless separately authorized. Mandatory CI must remain secret-free and deterministic.


Gate I — hardware/resource qualification

A full benchmark laboratory is not required.

For recommended local models/runtimes, capture enough evidence to avoid obviously unsuitable defaults:

  • approximate download size;
  • practical memory class;
  • execution backend;
  • cold/warm initialization class;
  • representative responsiveness;
  • known unsupported environment/fallback.

Where practical, distinguish at least:

  • lower-end/integrated hardware;
  • ordinary current laptop/desktop;
  • WebGPU unavailable/disabled;
  • bounded memory-pressure/OOM failure behavior.

Gate J — unconditional privacy/data-integrity invariants

  • Local-only means no cloud-provider contact.
  • No qualification path puts real API keys into repository or logs.
  • Model-download requests contain no manuscript/prompt data.
  • Local diagnostics do not expose manuscript/prompt/result contents by default.
  • Persisted-model migration is preserve-first.
  • Unknown/custom configuration is not destructively normalized without an explicit compatibility rule.
  • Local artifact cleanup cannot delete unrelated project/user data.
  • Failure recovery cannot weaken encryption/storage/privacy authority.

Any confirmed violation is an automatic release FAIL.


Finding promotion rule

Must be fixed before v1.28.7:

  • every P0/P1 finding;
  • any P2 affecting data integrity, privacy/local-only guarantees, security, production crash, advertised core AI functionality, active-default usability, destructive persisted-config migration, severe resource exhaustion, hidden network/cloud fallback, or a materially false release/product claim.

May remain post-release with a durable owner:

  • future model additions;
  • optional extra runtimes;
  • benchmark refinement;
  • experimental GPU expansion;
  • architecture cleanup without current correctness impact;
  • long-term PWA model-manager expansion;
  • non-critical UX polish.

Do not create scope cascade merely to make this tracker numerically complete.


Per-PR terminality

A contributing implementation slice counts as complete only when normal repository governance is satisfied:

  • source correctness established;
  • focused tests green;
  • pnpm run pr:budget -- --base <ref> within governance;
  • pnpm run ci:prepush green before push;
  • all current review channels inspected;
  • every actionable finding fixed, disproved or durably deferred to an explicit owner;
  • no unresolved actionable current-head thread remains;
  • exact-head CI/CD terminal green;
  • exact-head CodeQL terminal green;
  • required external quality gates terminal or explicitly classified as non-authoritative/transient;
  • PR merged without bypass;
  • resulting-main CI/CD terminal green;
  • resulting-main CodeQL terminal green;
  • durable owning issue updated with resulting-main evidence.

Final release decision

PASS

Only when all hard gates above are satisfied and no unresolved release-blocking AI/local-AI finding remains.

FAIL

If any hard gate remains incomplete or a known P0/P1/release-critical P2 remains unresolved.

CONDITIONAL PASS is not sufficient for the release cut.

Non-release-critical work may be deferred only with explicit scope, owner, reason and post-release disposition.

Once this tracker reaches PASS:

AI / LOCAL-AI PRE-RELEASE GATE = PASS
        ↓
final resulting-main stabilization
        ↓
release-readiness audit
        ↓
version synchronization
        ↓
CHANGELOG [Unreleased] → [1.28.7]
        ↓
fresh empty [Unreleased]
        ↓
v1.28.7 release PR convergence
        ↓
resulting-main qualification
        ↓
pre-tag desktop/Tauri qualification
        ↓
deliberate signed v1.28.7 tag
        ↓
tag-triggered desktop builds
        ↓
artifact / signature / updater-metadata verification
        ↓
RELEASE TERMINAL

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions