Establish a proactive OPY compatibility baseline from the pinned upstream language surface

[Teakowa]

Goal

Move Wright's OPY frontend from purely corpus-reactive compatibility toward a contract-first, evidence-prioritized compatibility baseline.

Wright should proactively implement high-leverage/common OverPy language and metadata surfaces when the upstream semantics are discoverable and reference-testable, while continuing to use real consumers/corpus evidence to prioritize sequencing, validate value, and avoid spending disproportionate effort on legacy/rare quirks.

Context

M11/#82 correctly stopped open-ended reference-output parity work and established semantic compatibility as the governing criterion. Its closing planning note, however, is intentionally conservative: future compatibility work was to be created only when real corpus/tooling evidence demonstrated a blocker.

The first real wrightkit/agent-lab integration (#102/#103/#104/#105) now shows the downside of applying that rule too literally: some predictable, generic compatibility surfaces are discovered serially only after consumers fall back.

OverPy itself exposes substantial structured language/content metadata rather than hiding all compatibility semantics inside compiler control flow. The upstream source contains large centralized data surfaces for actions, values, constants, custom-game settings, heroes/maps, plus OPY-specific metadata for functions, member functions, keywords, modules, preprocessing, annotations, and related constructs. This makes at least part of the compatibility surface suitable for systematic inventory/generation instead of one-symbol-at-a-time fixes.

Wright's compatibility implementation is expected to read and study upstream OverPy source code and later OSTW source code where necessary. Reimplementing compatible language semantics without inspecting the reference implementation is neither required nor desirable. The project should keep this transparent through one centralized upstream-reference document rather than scattering source/provenance notes through every implementation issue or compatibility entry.

This issue refines future planning; it does not retroactively change M11's historical acceptance decision.

Compatibility strategy

Use a hybrid model:

A. Proactive baseline: grammar and semantic primitives

Inventory and proactively support mainstream structural constructs whose semantics are stable, compositional, and high fan-out across real code, for example:

• expression/postfix/member/call structure;
• assignment/control-flow forms;
• declarations and rule directives;
• common preprocessing/include/macro forms;
• modules/enums/member functions as language categories;
• source identity and diagnostics required for tooling.

The unit of work should normally be a semantic/grammar category, not a single encountered spelling.

B. Proactive baseline: structured builtin/catalog surface

Where upstream/reference metadata is structured, prefer a reproducible catalog/import/generation path over manually maintaining a tiny corpus-only table.

Candidate surfaces include:

• builtin functions/actions/values;
• member-function metadata;
• builtin enums/constants;
• modules/annotations;
• Workshop settings/content identities where they belong to language compatibility rather than dynamic Workshop content tracking.

Wright may inspect and reuse/reference upstream definitions as needed to implement compatibility. Prefer a Wright-owned normalized representation or generated manifest that fits Wright's semantic architecture and can be reference-validated, rather than preserving upstream implementation structure for its own sake.

C. Evidence-prioritized advanced features

Implement more complex language features proactively when they have clear tooling value or broad expected use, but let corpus/consumer evidence determine ordering. Examples may include named arguments, richer settings expressions, advanced preprocessing, decompiler-related source constructs, and optimizer-sensitive semantics.

D. Demand-driven legacy/quirk compatibility

Keep rare historical quirks, upstream bugs, obsolete aliases, scripting hooks, and implementation-specific behavior demand-driven unless they become necessary for the declared compatibility target.

Compatibility mode may preserve observable upstream quirks when required; strict/fixed behavior remains a separate semantic decision.

Upstream/reference inventory

• Pin the upstream OverPy version/reference identity used for the baseline (initially align with Wright's existing pinned compatibility oracle unless deliberately upgraded).
• Inventory the upstream language surface by category, including grammar, directives, builtins, member functions, enums/constants, modules, settings behavior, and known special cases.
• For each category, classify it as:
  • baseline-supported;
  • baseline-planned;
  • evidence-prioritized;
  • legacy-quirk/demand-driven;
  • reference-limited/inconclusive.
• Keep observable semantic compatibility, not generated Workshop text identity, as the correctness criterion.

Centralized upstream-reference documentation

Maintain one repository document (for example docs/compatibility/upstream-references.md) covering upstream implementations Wright studies or derives compatibility knowledge from.

For each upstream project, record only durable project-level facts such as:

• project/repository identity;
• license;
• pinned/reference version or commit policy;
• which Wright compatibility surfaces use it as a reference/oracle;
• the distinction between reference semantics and Wright-owned implementation architecture;
• any important compatibility limitations or intentional divergences.

Do not require per-symbol or per-file provenance annotations throughout the codebase when the upstream relationship is already covered by this document. Extend the same document for OSTW when M13 begins.

Machine-readable compatibility contract

Where practical, make the support inventory machine-readable so documentation, differential tests, agents, and future release metadata can consume the same declared boundary.

The contract should distinguish at least:

• syntax accepted/rejected;
• semantic resolution support;
• standalone compilation support;
• analysis/tooling support;
• reference/oracle coverage;
• intentional limitations/quirks.

Avoid claiming a construct is supported merely because it parses.

Validation model

• Continue using real-world corpus and consumers as regression/priority evidence.
• Add systematic reference/differential cases for proactively implemented categories.
• Prefer generated table tests for large data-driven surfaces rather than hand-authoring one fixture per symbol when the semantics are uniform.
• Use focused real-project fixtures to prove that generated/systematic coverage actually composes through the frontend and tooling pipeline.

Relationship to tooling-first direction

This does not make Wright language-first.

Compiler-grade semantic frontends are enabling infrastructure for check, lint, analyze, inspect, source editing, agents, CI, and interoperability. Proactive compatibility work is justified when it reduces predictable fallback and expands the semantic surface those tools can operate on.

M12/#89 remains the active product-value milestone. Baseline compatibility work may proceed in parallel and becomes M12-critical only when a concrete tooling/consumer blocker requires it.

Non-goals

• Byte-for-byte or formatting parity with OverPy output.
• Reimplementing OverPy's internal compiler architecture for its own sake.
• Pretending Wright's compatible implementation must be developed without reading upstream source.
• Per-symbol/per-file provenance bureaucracy when project-level upstream attribution is sufficient.
• Declaring every historical OverPy bug/quirk part of the default language semantics.
• Wright-only OPY syntax or language evolution.
• Completing every compatibility category inside this single issue.
• Blocking M12 until the full baseline is implemented.

Deliverables

• A pinned upstream/reference surface inventory.
• A documented tiered compatibility policy (baseline vs evidence-prioritized vs legacy/quirk).
• One centralized upstream-reference document covering OverPy now and extensible to OSTW later.
• A machine-readable support/compatibility manifest or an explicit decision why a smaller representation is preferable.
• A prioritized set of implementation child issues grouped by semantic category rather than isolated encountered symbols.
• A plan for systematic builtin/enum/member metadata ingestion or generation.
• Differential/regression strategy for the proactive baseline.

Acceptance criteria

• Wright has an explicit forward-looking OPY compatibility baseline that is broader than the current accidental corpus subset but remains bounded by semantic/product value.
• The upstream language surface has been inventoried by category against a pinned reference identity.
• Common structural grammar/semantic categories are not intentionally left to one-bug-at-a-time discovery when their contract can be implemented/tested systematically.
• Large structured builtin/catalog surfaces have an explicit systematic strategy rather than indefinite manual enumeration.
• Wright's use of upstream OverPy implementation as a compatibility reference is documented centrally rather than treated as something to avoid or repeatedly justify.
• Real consumer/corpus evidence determines priority and confidence, not whether predictable baseline categories are allowed to exist.
• Legacy quirks and rare advanced behavior remain explicitly separable from mainstream compatibility.
• The resulting policy does not alter the no-language-fork rule or semantic-compatibility criterion established by [Architecture] Rebaseline Wright as a tooling-first semantic platform #88.

Relationships

• Refines future planning after closed M11/[M11] Rebaseline real-world compatibility from expanded corpus #82; does not rewrite its historical acceptance.
• Follows the tooling-first architecture contract from [Architecture] Rebaseline Wright as a tooling-first semantic platform #88.
• Fix real-world OPY receiver/member call compatibility (eventPlayer.setMoveSpeed and peers) #104 is an example of a baseline structural/member-call correctness gap.
• Add corpus-backed OPY builtin enum coverage for ChaseTimeReeval.NONE and nearby real-world gaps #105 is an example where a systematic builtin/enum catalog strategy may be preferable to serial one-member fixes.
• The same methodology and centralized upstream-reference document should later cover M13/OSTW compatibility after OSTW-specific investigation.

[Teakowa]

PM priority note (post-M12 reassessment, see #100)

Per the post-M12 reassessment recorded on #100: this issue takes priority over #90 and is a prerequisite for #90's decomposition/implementation. Its open children #104/#105 are the immediate next items to close. Scope guardrails stand: child issues are prioritized by corpus/consumer evidence (tier C), legacy quirks stay demand-driven (tier D), and the catalog path uses generation + reference validation rather than manual enumeration. The catalog surface here is language-compatibility metadata, distinct from #96's runtime content registry, which remains deferred.

[Teakowa]

#106 investigation & planning results (durable)

Findings are persisted in the repository, not only here: docs/compatibility/upstream-references.md (pinned reference provenance), docs/opy/compatibility-baseline.md (tiered baseline + residual evidence), docs/opy/compat-manifest-spec.md (machine-readable manifest spec). This comment is the concise planning record.

Current reality

• Reference identity confirmed by byte-verification: overpy@9.7.10, content commit 889d974 == tag v9.7.10; recorded gitHead 1e268895 lags content (v9.7.9 tag), as ADR-0007 warns.
• Wright's supported surface is an S-level corpus subset: 8 synthetic + 1 real-world differential parity cases; the emission catalog is a manually transcribed corpus-only table (14 actions / 31 values / 11 enum domains) and KNOWN_ENUMS in wright-opy/lower.rs is a separate hardcoded 8-domain table.
• Wright's parse surface already exceeds its semantic/compile surface: several constructs parse/lower but cannot compile because the emission catalog lacks generic action/value/member entries.

Key findings (verified oracle-vs-Wright, main@582c269)

• chaseOverTime(...) and isGameInProgress() : oracle accept; Wright check ✅, compile ❌ (unknown-action / unknown-value at emission) — generic catalog gap (residual of Add corpus-backed OPY builtin enum coverage for ChaseTimeReeval.NONE and nearby real-world gaps #105).
• Bare playervar receiver: A = B.C (playervar member on player-valued receiver) oracle accept, Wright rejects unsupported-member; B.isAlive() as a bare statement oracle rejects (value in action position), Wright accepts — an intentional-divergence candidate.
• getPlayersInRadius(...).setStatusEffect(eventPlayer, 30): oracle rejects on arity (needs player/assister/status/duration); Wright parses — signature metadata must encode arity.
• Enum-gated members (setInvisibility/Invis, getThrottle, worldVector/Transform): oracle accept, Wright rejects — member metadata + enum domains.
• chase(A, 10, rate=2, ChaseReeval.NONE): oracle accept via contextual alias; Wright rejects (named args unsupported). ChaseReeval stays out of the enum table until keyword-argument chase exists.
• Ambiguous bare None (ChaseTime/ChaseRate reeval domains): documented Workshop round-trip boundary; emitted semantic value is reference-equivalent.
• globalvar A = 0 vs = 5/= 0.0 vs A 0: canonicalization matches the oracle only under --profile compat; default off drops initializers.
• Diagnostic provenance: generic unresolved action/value errors are structured but surface at emission; the manifest moves resolution earlier.

Architectural decisions / recommendations

1. Machine-readable manifest justified — Wright-owned OPY semantic compatibility manifest (actions/values, member functions, signatures + parameter enum domains, enum members, source aliases), reference-validated; absorbs KNOWN_ENUMS; separate from catalog.json (emission/localization) and from [v0.2 workshop-rs] Define Workshop catalog, locale, extension, and version boundaries #96 (runtime content registry — no trigger found to reopen [v0.2 workshop-rs] Define Workshop catalog, locale, extension, and version boundaries #96).
2. Provenance rule: manifest entries are Wright-authored + oracle-validated probes — never mechanical extraction from OverPy's GPL-3.0 data files (ADR-0004 / docs/licensing.md).
3. No per-symbol issues; work is grouped by semantic category. No OSTW work starts here.

Proposed bounded child issue categories (pending PM approval — not created)

• C1 — Builtin/member/signature metadata baseline (categories 6+7+8): manifest schema + generator + reference-validation harness; closes the unknown-action/unknown-value/unsupported-member emission gaps.
• C2 — Call argument semantics (category 11): named/keyword arguments (rate=/duration=), enabling chase special form + ChaseReeval contextual alias, raycast include/exclude forms, macro defaults.
• C3 — Contextual Workshop enum resolution (category 8b): expected-type-driven enum domain resolution + None ambiguity policy.
• C4 — Remaining structural grammar (evidence-prioritized): switch/case, do…while, del, not in, 0x hex, rule-annotation surface — only if corpus/consumer evidence justifies each.
• C5 — Centralized upstream-reference doc + manifest wiring (doc now exists in-repo; wiring = support-matrix refresh when implementation lands).

Ordering / dependencies

C5 ∥ C1 → C3 → C2 → C4. C1 is the keystone (C2/C3 consume its signature/enum data); C4 is independent and lowest priority. C3 before C2 makes enum-typed named args testable; C2/C3 may swap.

Unresolved PM decisions

1. Manifest provenance path: confirm Wright-authored data + oracle-validated probes (recommended) vs. mechanical GPL data conversion (needs licensing ADR + legal review).
2. Default compile profile: make compat the effective default so initializer emission matches the oracle?
3. Divergence ledger: record oracle-rejects/Wright-accepts cases (e.g. value member as statement) as reviewed intentional differences (ADR-0008)?
4. Approve C1–C5 creation as sub-issues (not created yet, per scope).

[Teakowa]

PM acceptance and handoff

#106 is accepted on current main at 462a998d.

Acceptance basis

• The pinned OverPy reference identity and durable upstream-reference boundary are recorded in docs/compatibility/upstream-references.md.
• The OPY surface is inventoried by semantic category and by parse / semantic resolution / compile / tooling / reference dimensions in docs/opy/compatibility-baseline.md.
• A bounded machine-readable semantic manifest contract is specified in docs/opy/compat-manifest-spec.md.
• Fix real-world OPY receiver/member call compatibility (eventPlayer.setMoveSpeed and peers) #104/Add corpus-backed OPY builtin enum coverage for ChaseTimeReeval.NONE and nearby real-world gaps #105 residuals are classified as category-level evidence rather than one-symbol implementation requests.
• The plan remains tooling-first, does not fork OPY syntax, and does not trigger [v0.2 workshop-rs] Define Workshop catalog, locale, extension, and version boundaries #96.
• GitHub checks on 462a998d are green.

PM decisions on the investigation's unresolved items

1. Manifest provenance: approve the Wright-authored + pinned-oracle-validated data path. Do not mechanically convert/import OverPy implementation/data files into Wright core. The centralized upstream-reference record is sufficient project-level provenance; avoid per-symbol bureaucracy beyond reproducible validation evidence.

2. Compile profile: do not make compat the default merely to hide initializer semantic loss. If off drops observable initializer behavior on a declared supported surface, that is a profile/lowering ownership correctness defect. Track separately in [Correctness] Preserve OPY declaration initializer semantics independently of optimization profile #112.

3. Oracle-rejects / Wright-accepts cases: do not default these to intentional divergences. For OPY source-language semantics, no-language-fork remains authoritative. [OPY baseline] Implement semantic compatibility manifest and systematic builtin/member/signature coverage #109 must enforce action/value/member/signature contexts where the pinned reference contract is clear. A deliberate divergence requires an explicit future contract, not accidental permissiveness.

4. Implementation decomposition: approved, but reduced to the categories that are currently justified:

   • [OPY baseline] Implement semantic compatibility manifest and systematic builtin/member/signature coverage #109 — semantic compatibility manifest + systematic builtin/member/signature baseline (C1, keystone)
   • [OPY baseline] Implement named/keyword argument binding and contextual call semantics #110 — named/keyword argument binding + contextual call semantics (C2; depends on [OPY baseline] Implement semantic compatibility manifest and systematic builtin/member/signature coverage #109)
   • [Workshop semantics] Resolve ambiguous enum members using expected parameter domains #111 — expected-domain Workshop enum resolution (C3; depends on [OPY baseline] Implement semantic compatibility manifest and systematic builtin/member/signature coverage #109)
   • C4 structural grammar remains evidence-prioritized and is not pre-created.
   • C5 durable reference/baseline/spec documentation is already satisfied by 462a998d.

Additional correctness finding

• [Correctness] Preserve OPY declaration initializer semantics independently of optimization profile #112 tracks the initializer/profile semantic-ownership issue exposed by this investigation. It is correctness work, not output-parity work.

M13 handoff

• [M13 investigation] Establish OSTW reference, corpus, licensing, and semantic support baseline #113 starts the read-only OSTW reference/corpus/licensing/semantic investigation required by [M13] Source-language interoperability and OSTW compatibility #90. It may run in parallel with [OPY baseline] Implement semantic compatibility manifest and systematic builtin/member/signature coverage #109/[Correctness] Preserve OPY declaration initializer semantics independently of optimization profile #112, but no OSTW parser/type-system/emitter implementation decomposition is authorized until [M13 investigation] Establish OSTW reference, corpus, licensing, and semantic support baseline #113 is accepted and PM reassesses [M13] Source-language interoperability and OSTW compatibility #90.

Execution order

1. [Correctness] Preserve OPY declaration initializer semantics independently of optimization profile #112 correctness investigation/fix and [OPY baseline] Implement semantic compatibility manifest and systematic builtin/member/signature coverage #109 manifest implementation may proceed in parallel.
2. After [OPY baseline] Implement semantic compatibility manifest and systematic builtin/member/signature coverage #109 stabilizes the shared signature/enum contract, [OPY baseline] Implement named/keyword argument binding and contextual call semantics #110 and [Workshop semantics] Resolve ambiguous enum members using expected parameter domains #111 may proceed in parallel.
3. [M13 investigation] Establish OSTW reference, corpus, licensing, and semantic support baseline #113 OSTW investigation may proceed concurrently as a read-only Architect/QA workstream.
4. Reassess [M13] Source-language interoperability and OSTW compatibility #90 after [M13 investigation] Establish OSTW reference, corpus, licensing, and semantic support baseline #113; do not mechanically start M13 implementation.

With this handoff, #106 has satisfied its planning and decomposition purpose and can close.