Establish a tests-first Workshop corpus and seasonal feature census

[Teakowa]

Goal

Establish a durable tests-first Workshop corpus and seasonal feature census
that exercises real projects, minimized regressions, the canonical Workshop
surface, and recorded client captures for workshop-rs and downstream
language implementations.

Context

workshop-rs owns canonical Workshop feature identities and the test/result
contracts used by this corpus. Source-language repositories remain responsible
for their own syntax, semantics, and source-level fixtures.

The durable contract is ordinary tests, test data, source attribution, and
machine-readable results. It must not introduce a parallel evidence taxonomy
or treat implementation output as an independent expected result.

Scope

• Keep focused unit/property tests, minimized regressions, complete real-project fixtures, and seasonal client workflows as separate test/data workflows without generic evidence classes.
• Retain repository, revision, path, digest, license, locale, catalog, game, season, and capture-time metadata where it supports attribution, reproducibility, or runtime interpretation.
• Require an actual expected-versus-observed comparison before reporting matched; when only a pinned source identity is available, report inconclusive rather than fabricating a comparison.
• Preserve minimized reproductions for real-project semantic/compatibility defects where practical, together with the original full-project source input.
• Define stable Workshop feature identities for machine-readable results across the declared semantic/catalog surface.
• Maintain a canonical Workshop feature-census fixture/mode that exercises the declared surface and reports failures by feature identity rather than only as a monolithic script diff.
• Use semantic/normalized equivalence when text identity is not part of the contract.
• Provide machine-readable results for matched comparisons, unsupported surface, known gaps, unexpected regressions, and inconclusive tests; known gaps and inconclusive tests must not count as successful parity.
• Define a repeatable seasonal validation workflow in which census artifacts can be round-tripped through the live Overwatch client with game/season/client-locale/catalog and raw-capture metadata.
• Use English client output as the primary semantic baseline where appropriate, with explicit locale sweeps for localization/catalog changes.
• Keep corpus/result contracts reusable by opy-rs and deltin-rs without transferring canonical Workshop ownership out of this repository.

Non-goals

• Claiming exhaustive live-runtime/gameplay correctness solely from import/export tests.
• Requiring byte-identical or formatting-identical output.
• Building a network service or automatically downloading unreviewed catalog data at runtime.
• Moving OPY or DEL/OSTW source-language fixtures or semantics into workshop-rs.
• Treating one giant census script as a replacement for focused unit/property tests or real-world projects.
• Freezing a separate cross-repository test framework before concrete corpus integration requires one.
• Maintaining child-issue completion state or execution sequencing in this parent issue body.

Acceptance criteria

• The repository documents and enforces a tests-first contract: direct tests and test data are primary, with concrete source attribution retained where needed.
• Matched results require an actual expected-versus-observed comparison; source identity alone produces an explicit inconclusive result.
• Real-project failures retain project-level source attribution and, where valuable, minimized source-attributed regression fixtures.
• A canonical feature-census fixture covers the declared Workshop semantic/catalog surface with stable machine-readable feature identities.
• Census failures identify affected features rather than reducing the result to a single whole-file pass/fail.
• parse -> canonical semantics -> emit and supported locale conversion are gated against the corpus using semantic/normalized contracts.
• Known gaps and inconclusive tests are represented explicitly and are not reported as successful parity/conformance.
• A documented seasonal client-validation procedure records client/game/catalog/locale and raw-capture metadata and supports structured comparison with prior captures.
• The resulting corpus/result contracts can be consumed by opy-rs and deltin-rs without transferring canonical Workshop ownership out of this repository.

Relationships

• Extends Establish the canonical multi-locale Workshop core #2 rather than replacing its canonical multi-locale Workshop core scope.
• Builds on Define Workshop catalog, locale, provenance, and version boundaries #1 catalog/locale/source-attribution/version boundaries.
• Executable work is decomposed in related issues such as Define the Workshop conformance result contract and stable feature identities #18-Establish the seasonal live-client Workshop validation and structured-diff workflow #21; their current state and ordering should be derived from GitHub rather than mirrored here.
• Downstream source-language adoption remains tracked in the owning opy-rs and deltin-rs repositories.

[Teakowa]

Execution update for #18–#21:

• Define the Workshop conformance result contract and stable feature identities #18: Draft PR feat(conformance): define Workshop evidence contract #26 pushed, Architect contract review completed, CI green.
• Build the canonical sharded Workshop feature census #19: Draft PR feat(census): add canonical sharded Workshop feature census #27 pushed on Define the Workshop conformance result contract and stable feature identities #18; initial independent QA found an oracle-boundary defect, which was corrected. Fresh QA now PASS/VERIFIED on 6a23b3e; CI stable/Rust 1.85 green.
• Add provenance-linked real-project and minimized Workshop regression evidence #20: Draft PR feat: add provenance-linked real-project evidence #28 pushed in parallel on Define the Workshop conformance result contract and stable feature identities #18; independent QA VERIFIED on fd21367; full CI green. The complete project remains an explicit KnownGap for ambiguous bare Up, while the minimized regression is separately matched.
• Establish the seasonal live-client Workshop validation and structured-diff workflow #21: Draft PR feat: add seasonal Workshop client evidence workflow #30 pushed on Build the canonical sharded Workshop feature census #19; fresh QA PASS/VERIFIED on 72ff506; CI stable/Rust 1.85 green. Offline capture schema, structured diff, CLI and ADR are complete for repository-owned automation.

No PR has been merged and no release published. No real Overwatch client capture is claimed; #21 retains the manual en-US/zh-CN client-validation boundary for maintainer action.

[Teakowa]

Execution update after final provenance/evidence review:

• Define the Workshop conformance result contract and stable feature identities #18 / PR feat(conformance): define Workshop evidence contract #26: final head 58832ff; Architect review PASS and independent QA VERIFIED. ConformanceResult::validate_against now requires exact evidence.catalog == supplied Catalog::identity(), with a format-valid mismatched-catalog regression test. CI stable and Rust 1.85 green.
• Build the canonical sharded Workshop feature census #19 / PR feat(census): add canonical sharded Workshop feature census #27: final head f9a520b; independent QA PASS and CI green. Public CensusIdentity/SHA-256 report identity is authoritative; every Inconclusive probe executes offline parse/validate/emit/round-trip/locale gates first. Current census truthfully exposes 298 UnexpectedRegression, 18 Unsupported, and 169 Inconclusive cases; the two newly authored localization fixtures remain Inconclusive and are not independent evidence.
• Add provenance-linked real-project and minimized Workshop regression evidence #20 / PR feat: add provenance-linked real-project evidence #28: final head 86d8753; independent QA VERIFIED and CI green. Full and minimized fixtures have separate pinned workshop-rs source tuples; the expected artifact is a separately pinned Wright/OverPy oracle with a reviewed digest. The full project remains KnownGap for ambiguous bare Up; the minimized case remains Matched.
• Establish the seasonal live-client Workshop validation and structured-diff workflow #21 / PR feat: add seasonal Workshop client evidence workflow #30: final head a7215dc; independent QA VERIFIED and CI green. It consumes the canonical feat(census): add canonical sharded Workshop feature census #27 CensusIdentity, provides the offline schema/diff workflow, and contains no fabricated client capture. All commits and the PR body use Refs #21; Issue Establish the seasonal live-client Workshop validation and structured-diff workflow #21 remains OPEN for real en-US/zh-CN client validation and runtime acceptance.

All four PRs remain OPEN Draft PRs, stacked as #26 -> #27/#28 -> #30. No PR was merged, main was not pushed, and no release was published. The remaining evidence boundary is maintainer review/merge plus real-client validation for #21; #27 also records current regression/unsupported gaps for separate follow-up rather than hiding them.

[Teakowa]

Fresh independent QA update for the final heads:

• PR feat(conformance): define Workshop evidence contract #26 58832ff: VERIFIED/PASS; exact catalog evidence binding and negative regression confirmed.
• PR feat(census): add canonical sharded Workshop feature census #27 f9a520b: VERIFIED/PASS; canonical census identity, probes-before-Inconclusive execution, and explicit surfaced failures confirmed.
• PR feat: add provenance-linked real-project evidence #28 86d8753: BLOCKED/FAIL; provenance tuples and digests pass, but the corpus runner records the pinned oracle only as metadata and does not compare oracle content against observed canonical WIR, so a divergent implementation can still report Matched.
• PR feat: add seasonal Workshop client evidence workflow #30 a7215dc: VERIFIED/PASS; canonical CensusIdentity consumption, offline/no-fabrication boundary, and non-closing Refs #21 history confirmed. Issue Establish the seasonal live-client Workshop validation and structured-diff workflow #21 remains OPEN.

No implementation changes were made in this QA pass. Merge order remains #26 first, then #27/#28 only after #28 resolves its oracle-comparison blocker, then #30 after #27. #21 stays open for real en-US and zh-CN client capture review.