Skip to content

Define the Workshop conformance result contract and stable feature identities #18

Description

@Teakowa

Parent: #10

Goal

Define the canonical machine-readable conformance result model and stable Workshop feature identities that later census, regression, seasonal, and downstream-provider evidence can share.

Context

workshop-rs now provides the released canonical Workshop semantic core, but current tests are primarily repository-local focused gates. #10 requires failures and compatibility evidence to be attributable to concrete Workshop capabilities rather than only files, snapshots, or whole-script pass/fail results.

This issue establishes the contract first so later corpus work does not invent incompatible schemas independently.

Scope

  • Define stable, locale-independent feature identities for the currently declared Workshop surface, including events, actions, values, enums/content IDs, settings, variables, subroutines, control flow, strings/localization, and other catalog/WIR-backed constructs where evidence exists.
  • Define a machine-readable conformance result model that distinguishes at least matched behavior, unsupported surface, known gap, unexpected regression/divergence, and inconclusive evidence.
  • Define evidence/provenance fields sufficient to identify the fixture/source, evidence class, catalog identity, implementation version/revision where relevant, locale, and external/live-client evidence where applicable.
  • Keep feature identity separate from localized spelling and from source-language provider identities.
  • Define semantic/normalized equivalence as the default comparison contract where textual identity is not observable semantics.
  • Add focused tests for schema stability, serialization, invalid states, and representative feature identities.
  • Document the public/reusable parts of the contract that opy-rs and del-rs may consume without transferring Workshop ownership.

Non-goals

  • Building the complete feature census fixture.
  • Importing real projects or live-client captures.
  • Designing OPY/DEL-specific compatibility result schemas.
  • Freezing a general cross-repository testing framework or plugin ABI.
  • Treating every catalog entry as a separate implementation issue.

Acceptance criteria

  • Conformance results can report the required states without counting known gaps or unsupported cases as passes.
  • Representative Workshop capabilities have stable machine-readable feature IDs independent of locale and provider naming.
  • Evidence/provenance metadata can distinguish synthetic, regression, real-project, and live-client evidence without relying on implementation-generated expectations as the oracle.
  • The contract has serialization/validation tests and concise documentation.
  • Later Establish a tests-first Workshop corpus and seasonal feature census #10 work can build census and corpus evidence on this contract without redefining the identity/result model.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions