[M13 baseline] Pin the OSTW oracle and acquire the initial real-world corpus

[Teakowa]

Parent: #90
Depends on: #113

Goal

Turn the accepted OSTW investigation into reproducible evidence by pinning the initial OSTW reference/oracle, adding a deterministic headless oracle runner, acquiring the first licensed real-world corpus, and recording the observed S/D/N support surface before any Wright OSTW frontend implementation is decomposed.

This is the final evidence baseline before parser/type-system/emitter implementation planning.

Context

#113 established:

• upstream reference candidate: ItsDeltin/Overwatch-Script-To-Workshop stable tag v3.4.0;
• the rolling latest build is not a valid pin;
• the compiler/wiki are treated as unlicensed for source-copying purposes, while observable reference behavior may be studied through the oracle boundary;
• the default CLI compile path is interactive/clipboard-bound;
• stdio --langserver is the viable deterministic headless S/D/N oracle path;
• pharingWell/MOBAwatch (BSD-2-Clause) and GrandeurHammers/protect-ban (MIT) are the approved initial corpus candidates.

The accepted planning records are docs/compatibility/upstream-references.md and docs/ostw/compatibility-baseline.md.

Scope

• Pin OSTW v3.4.0 by immutable content identity and record acquisition/integrity metadata consistent with Wright's ADR-0007 reference policy.
• Add a reference-only OSTW oracle runner under the compatibility harness that communicates with upstream through stdio --langserver and can deterministically capture at least:
  • diagnostics / accept-reject status;
  • emitted Workshop text from the workshopCode notification;
  • element count where useful evidence exists.
• Keep the upstream runtime/tooling entirely outside Wright's production Rust dependency graph and supported standalone runtime path.
• Add a minimal oracle smoke check (--ping) and deterministic failure reporting for missing/incompatible reference artifacts.
• Acquire the approved initial corpus at immutable commits:
  • MOBAwatch;
  • protect-ban.
• Record repository/license/provenance, complete relevant source/include closures, per-file SHA-256, project settings (ds.toml) and source-kind metadata using the existing corpus tooling where practical.
• Run the pinned oracle over the acquired projects/files and produce a machine-readable observed inventory of:
  • oracle accept/reject;
  • first diagnostic/failure where applicable;
  • emitted Workshop availability;
  • source constructs/categories exercised;
  • project/include relationships relevant to frontend planning.
• Refresh docs/ostw/compatibility-baseline.md with measured corpus distribution and classify which semantic categories are actually required for the first Wright-owned frontend milestone.
• Produce a PM-ready decomposition recommendation from the measured corpus, but do not implement or pre-create parser/type-system/emitter issues inside this task.

Licensing / provenance guardrails

• Do not copy, mechanically translate, or import source/data from the unlicensed OSTW compiler/wiki into Wright core.
• Do not import OSTW Elements.json, LobbySettings.json, Maps.json, or other game-derived data into Wright's catalog.
• Only commit third-party corpus material whose license permits repository inclusion and whose provenance is recorded.
• Reference artifacts may be downloaded/used by compatibility tooling under the documented oracle boundary; they are not shipped as Wright runtime dependencies.

Non-goals

• Implementing wright-ostw, an OSTW parser, semantic/type system, emitter, decompiler, or language-service integration.
• Supporting the entire OSTW language because upstream exposes it.
• Byte-identical optimizer/compiler output.
• Direct OPY <-> OSTW conversion.
• E-level timing/runtime claims; upstream emulator limitations remain documented.
• Multi-locale support.

Acceptance criteria

• The OSTW reference is version-exact/content-pinned and reproducible; rolling latest is never used as evidence identity.
• A deterministic headless oracle harness can obtain diagnostics and Workshop output through the upstream stdio language-server path without clipboard/X-server interaction.
• MOBAwatch and protect-ban are acquired at immutable revisions with license/provenance and per-file integrity records.
• The pinned oracle has been executed over the initial corpus and results are persisted in a reviewable machine-readable form.
• The resulting inventory identifies the actual construct/category distribution and first blockers needed to define Wright's initial OSTW support matrix.
• Production Wright remains independent of .NET/upstream OSTW runtime dependencies.
• No unlicensed upstream implementation/data is copied into Wright core.
• Current Wright CI remains green.
• PM has enough measured corpus evidence to create the first bounded OSTW frontend implementation issues under [M13] Source-language interoperability and OSTW compatibility #90.

Relationships

• Implements the approved post-[M13 investigation] Establish OSTW reference, corpus, licensing, and semantic support baseline #113 phase A.
• Must complete before [M13] Source-language interoperability and OSTW compatibility #90 is decomposed into OSTW parser/type-system/emitter implementation tasks.
• Uses the compatibility methodology established by Establish a proactive OPY compatibility baseline from the pinned upstream language surface #106 but does not depend on completing all remaining OPY compatibility work.

[Teakowa]

PM acceptance recheck — reopened for reproducibility drift

The substantive #115 implementation is now on main (1fc870c, followed by e4b0a06 to run the reference through the pinned runtime boundary): the OSTW v3.4.0 identity is pinned, the stdio --langserver harness exists, MOBAwatch/protect-ban are committed with provenance/integrity metadata, and machine-readable corpus.json / results.json exist.

Acceptance is temporarily blocked by the new CI gate itself. On e4b0a06, the OSTW reference baseline job fails with:

OSTW_ORACLE_DRIFT: results.json differs; review with --update

That means the committed evidence is not yet reproducible under the runtime path CI now treats as canonical. This directly violates #115's reproducibility/current-CI acceptance criteria, so the issue is reopened rather than treating the failed gate as advisory.

The concurrent Rust quality (1.85.0) failure on the same commit is unrelated to #115: it is a clippy format_collect failure in crates/wright-cli/tests/update.rs from #116.

Before PM decomposes #90 into OSTW frontend implementation issues:

1. reproduce the CI oracle drift locally/in the same pinned container/runtime;
2. determine whether the difference is deterministic environment/path/protocol normalization or a real reference-output change;
3. normalize only environment-dependent evidence fields, or update results.json only after confirming the new result is the canonical deterministic observation;
4. rerun the OSTW baseline twice from clean acquisition and prove byte-stable machine-readable results;
5. obtain a green OSTW reference baseline CI job.

Do not start wright-ostw parser/type/emitter implementation until this gate is green. The measured corpus evidence already suggests project settings/imports/rules/macros/arrays/Workshop calls are first-order categories, while classes remain evidence-prioritized; the exact first frontend boundary will be decided from the stabilized results.

[Teakowa]

Implementation evidence: OSTW reference baseline now reproducible

Root cause of the OSTW_ORACLE_DRIFT

compatibility/ostw/run_oracle.py broke out of its receive loop on the FIRST
workshopCode notification. OSTW's --langserver (pinned v3.4.0,
769ce7aab097178cfe905bf21f0326d8e0d12e6b) debounces compiles ~50 ms after the
last didOpen and publishes one coherent workshopCode/elementCount/
publishDiagnostics triple per compile; which triple arrives first is
timing-dependent. The committed results.json therefore recorded transient
first notifications (protect-ban: a lone "\n"; MOBAwatch: a partial error
fragment) and misclassified both projects as accepted. Reproduced locally:
the drift failed 1-in-5 runs on the same pinned artifact/runtime/corpus.

Fix (final commit cba9faa)

The harness now drains the langserver until 3 s of quiet and records only the
LAST compile triple after every file is open. accept/reject derives from
the final elementCount (>= 0 = the reference produced Workshop code; -1
= errors) instead of accumulated diagnostics. Only genuinely
environment-dependent/timing fields are excluded (no message counts); all
diagnostics and the full Workshop-output identity are preserved.

Did corpus observations change?

Yes, and this is the meaningful correction (no corpus files were re-acquired):

• MOBAwatch (b9b1ac3b…): rejects under the pinned Linux reference —
  first error Operator '==' cannot be applied to the types 'T' and 'T'.
  (tests.ostw:8), plus missing include-closure files (Windows backslash
  includes ..\entity.del/..\structures.del, absent customGameSettings.lobby
  and .resources/meshes/core.obj); elementCount -1, error-log SHA-256
  b0a8d959….
• protect-ban (f8c2353e…): accepts — 79-element Workshop output
  (SHA-256 75722f0a…) with only Unable to infer type hints. The previously
  recorded "empty workshopCode" blocker was the harness race and is resolved.

Clean repeated-run evidence

• 10 consecutive runs at the default reference root: zero drift
  (results.json SHA-256 f3d2e637… every run).
• Two full runs from a clean reference re-acquisition (fresh root, pinned asset
  size+SHA-256 re-verified, re-extracted): byte-identical results.
• The final compile triple (accept, elementCount, workshop SHA, firstDiagnostic)
  was additionally stable across 21 instrumented runs, including a slow-send
  variant that forces inter-compile debounce races.

CI status (run 31875281084, commit cba9faa)

• OSTW reference baseline: PASS (acquire + ping + baseline drift check).
• v1 gates, differential, adapter, oracle, dist (ubuntu + macos), Rust quality
  (stable): PASS.
• Rust quality (1.85.0): FAIL — pre-existing Add wright update self-update support #116 clippy (format_collect in
  crates/wright-cli/src/update.rs), unrelated to this task and intentionally
  not fixed here.

Files changed

• compatibility/ostw/run_oracle.py — deterministic last-compile capture.
• compatibility/ostw/results.json — corrected S/D/N observation.
• compatibility/ostw/README.md — capture semantics.
• docs/ostw/compatibility-baseline.md — measured baseline table + resolved
  empty-output blocker.

Reference identity unchanged: OSTW v3.4.0, asset SHA-256
1ae882898961eac8ac25234a18fa3b130a02836651f7f936b9ece85f181e3a88.

[Teakowa]

PM acceptance — baseline accepted

#115 is accepted on the stabilized evidence at cba9faa.

The reproducibility blocker is resolved: the oracle now captures the final debounced compile result rather than a timing-dependent first notification; repeated runs and clean re-acquisition are byte-stable, and the dedicated OSTW reference baseline CI job passes.

The corrected corpus boundary materially changes M13 planning:

• protect-ban is the first accepted end-to-end reference corpus: 79-element Workshop output is available and stable.
• MOBAwatch is reference-rejected under pinned OSTW v3.4.0 on the canonical Linux runtime. It remains valuable structural/corpus evidence, but MUST NOT be used as a compile-success acceptance gate unless its reference/project-closure situation is separately resolved.
• Classes and other advanced surfaces remain evidence-prioritized; they are not pulled into the first native frontend milestone merely because MOBAwatch contains them.

The unrelated Rust 1.85 clippy failure from #116 is not attributed to #115 and does not invalidate the now-green OSTW baseline gate.

With #106/#113/#115 complete, the evidence gate that previously blocked decomposition of #90 is satisfied. PM may now create bounded native OSTW implementation issues.

[Teakowa]

Implementation evidence — #115 persisted on main

The #115 scope is committed and pushed on main; the working tree is clean (nothing left to persist). No re-investigation performed.

Commit SHAs: final cba9faa (fix(compatibility): make the OSTW reference baseline deterministic), preceded by 1fc870c (corpus baseline) and e4b0a06 (pinned-runtime boundary). Current main HEAD is a125ffc.

Oracle identity: OSTW v3.4.0 from ItsDeltin/Overwatch-Script-To-Workshop at content commit 769ce7aab097178cfe905bf21f0326d8e0d12e6b; asset v3.4.0-linux-x64.zip (3,332,180 bytes, SHA-256 1ae882898961eac8ac25234a18fa3b130a02836651f7f936b9ece85f181e3a88); headless stdio --langserver; --ping returns the pinned Hello! response.

Corpus revisions (immutable; per-file SHA-256 in corpus.json): MOBAwatch b9b1ac3b77a484256e89aca6be8c27470803f665 (BSD-2-Clause, 71 files); protect-ban f8c2353ed8447f13038fbf6b9938031cced5796f (MIT, 18 files); full source/include closures under compatibility/ostw/corpus/.

Measured results (results.json, deterministic final-compile capture):

• MOBAwatch — reject: first diagnostic Operator '==' cannot be applied to the types 'T' and 'T'. (tests.ostw:8), elementCount −1, error-log SHA-256 b0a8d959…, no Workshop output.
• protect-ban — accept: 79-element Workshop output (SHA-256 75722f0a…), only Unable to infer type hints; entry main.ostw.

Files: compatibility/ostw/reference.json, compatibility/ostw/run_oracle.py, compatibility/ostw/corpus.json, compatibility/ostw/results.json, compatibility/ostw/README.md, compatibility/ostw/corpus/{mobawatch,protect-ban}/, and refreshed docs/ostw/compatibility-baseline.md.

CI status: cba9faa run 31875281084 — OSTW reference baseline PASS, all other jobs PASS except Rust quality (1.85.0) (pre-existing #116 clippy, fixed later in f717c66). Current main (a125ffc) run 31883046967 — OSTW reference baseline PASS and all other jobs PASS except Distribution validation (windows-latest), a pre-existing failure from concurrent dist commit 631e556 and unrelated to #115.

No OSTW frontend/parser/type-system/emitter issues created.