Goal
Simplify OPY compatibility verification into conventional differential/regression tests and test data, removing evidence-specific metadata and historical task bookkeeping that are not needed to execute or understand the tests.
Context
The current OPY differential suite is meaningful and should remain: it runs OPY inputs through the native implementation and compares supported observable behavior with the pinned OverPy reference.
However, current main layers additional evidence-specific state around those tests:
crates/opy-rs/tests/differential.rs contains a large declared-corpus table with Issue-specific rationale/history in test expectations;
differential-expectations.json defines an evidencePolicy and per-case evidence arrays in addition to native expectations;
- each corpus case carries
fixture.json plus oracle.json, with overlapping status/reference/provenance information;
- current docs/ADR-0002 describe an "evidence-first" compatibility architecture rather than ordinary differential tests;
- support/current documentation routes developers through corpus/oracle/evidence terminology as a distinct model.
The desired model from wrightkit/.github#60 is: OPY compatibility is a test requirement; OverPy is the reference implementation; committed expected outputs/reference snapshots are test data; third-party provenance is minimal and practical.
Scope
- Keep the pinned OverPy reference runner and differential comparison where they materially protect OPY behavior.
- Make the differential test itself the primary executable compatibility contract.
- Collapse duplicated expectation/status metadata so each test case has one clear expected native/reference relationship.
- Remove generic
evidencePolicy, evidence arrays, Issue/PR/task references, and historical implementation narrative from committed test expectations.
- Simplify
fixture.json / oracle.json or replace them with more direct test-data forms where the current schema does not serve an active test requirement.
- Preserve only the provenance needed for redistributed upstream/third-party fixtures, licensing, immutable identity, and reproducible reference comparison.
- Keep generated/run reports under
target/ or CI artifacts rather than committing point-in-time execution results.
- Reconcile AGENTS, language-support, upstream-reference, compatibility-manifest, architecture, and corpus documentation with the tests-first model.
- Supersede or partially supersede ADR-0002 where it establishes evidence-first architecture; preserve the historical ADR rather than rewriting history.
- Reconcile open parent/support contracts that still require "compatibility evidence" as a distinct subsystem.
Non-goals
- Removing meaningful compiler, semantic, differential, real-project, or negative compatibility tests.
- Removing the pinned OverPy reference as a behavioral oracle/reference implementation.
- Requiring byte-identical Workshop output where observable compatibility does not require it.
- Removing licensing/redistribution metadata required for imported fixtures.
- Moving OPY compatibility ownership to Wright or workshop-rs.
- Replacing the current metadata layers with another generic test framework.
Acceptance Criteria
- A contributor can understand OPY compatibility as input + native execution + reference/expected result + assertion.
- Differential tests retain the same meaningful failure detection after metadata simplification.
- There is one authoritative expected relationship per test case rather than overlapping expectation tables and evidence records.
- Current test data contains no Issue/PR/task history unless that identifier is itself part of the tested public behavior.
- Third-party fixture metadata is limited to attribution/licensing/immutable identity/reproducibility needs.
- Current docs no longer require an evidence-first mental model.
- ADR history remains truthful while current contracts point to tests-first compatibility verification.
- Wright does not gain duplicate OPY compatibility ownership as a result of the cleanup.
Dependencies / Ownership
ADR cleanup requirement
ADR cleanup is part of this Issue, not a later documentation task.
Current ADR audit found:
- ADR-0002 explicitly establishes an "evidence-first" compatibility model and must be superseded or partially superseded by the tests-first decision;
- ADR-0001/0003/0004/0005 and the ADR registry still use "historical evidence", "compatibility evidence", "conformance evidence", or similar terminology even where the durable architecture decision is otherwise valid.
Required treatment:
- add a current tests-first compatibility/testing ADR/decision when needed to replace ADR-0002's evidence-first contract;
- mark ADR-0002 Superseded/Partially Superseded and remove it from current routing authority;
- preserve the historical ADR body, but make the supersession explicit at the top and in the ADR registry;
- update every remaining Accepted ADR so current decision language refers directly to tests, differential/reference comparison, source provenance, diagnostics/provenance, licensing, or reproducibility rather than generic evidence;
- rename "Historical evidence" fields/references to related/decision history;
- keep OverPy reference pinning and source provenance where they remain real compatibility/licensing contracts.
Acceptance is not satisfied while any Accepted/current OPY ADR declares an evidence-first model or uses generic evidence as an architectural authority.
Terminology requirement: no generic provenance abstraction
This cleanup also removes provenance as a generic WrightKit domain concept.
Use the concrete concern instead:
- source mapping / source location for diagnostics, edits, or generated-to-authored attribution;
- source attribution for imported or redistributed test/source data;
- license / licensing metadata for legal and redistribution requirements;
- pinned revision / version for immutable external references;
- checksum / hash only where integrity or reproducibility requires it;
- reproducibility for the commands/configuration/version information needed to recreate a current test/reference result.
Do not preserve generic concepts such as provenance-aware fixture, provenance-linked regression, diagnostic provenance, catalog provenance, artifact provenance, or a generic provenance schema merely to group unrelated metadata. If a field or abstraction cannot be mapped to a concrete continuing requirement above, remove it.
This does not authorize removing source spans/mappings, third-party attribution, licensing metadata, or immutable reference identity where those are actually required.
Goal
Simplify OPY compatibility verification into conventional differential/regression tests and test data, removing evidence-specific metadata and historical task bookkeeping that are not needed to execute or understand the tests.
Context
The current OPY differential suite is meaningful and should remain: it runs OPY inputs through the native implementation and compares supported observable behavior with the pinned OverPy reference.
However, current
mainlayers additional evidence-specific state around those tests:crates/opy-rs/tests/differential.rscontains a large declared-corpus table with Issue-specific rationale/history in test expectations;differential-expectations.jsondefines anevidencePolicyand per-caseevidencearrays in addition to native expectations;fixture.jsonplusoracle.json, with overlapping status/reference/provenance information;The desired model from wrightkit/.github#60 is: OPY compatibility is a test requirement; OverPy is the reference implementation; committed expected outputs/reference snapshots are test data; third-party provenance is minimal and practical.
Scope
evidencePolicy, evidence arrays, Issue/PR/task references, and historical implementation narrative from committed test expectations.fixture.json/oracle.jsonor replace them with more direct test-data forms where the current schema does not serve an active test requirement.target/or CI artifacts rather than committing point-in-time execution results.Non-goals
Acceptance Criteria
Dependencies / Ownership
opy-rsowns OverPy source/compiler compatibility tests and the pinned OverPy reference workflow.workshop-rsremains the owner of canonical Workshop semantics and emission.ADR cleanup requirement
ADR cleanup is part of this Issue, not a later documentation task.
Current ADR audit found:
Required treatment:
Acceptance is not satisfied while any Accepted/current OPY ADR declares an evidence-first model or uses generic evidence as an architectural authority.
Terminology requirement: no generic provenance abstraction
This cleanup also removes
provenanceas a generic WrightKit domain concept.Use the concrete concern instead:
Do not preserve generic concepts such as
provenance-aware fixture,provenance-linked regression,diagnostic provenance,catalog provenance,artifact provenance, or a generic provenance schema merely to group unrelated metadata. If a field or abstraction cannot be mapped to a concrete continuing requirement above, remove it.This does not authorize removing source spans/mappings, third-party attribution, licensing metadata, or immutable reference identity where those are actually required.