Skip to content

feat(conformance): define Workshop evidence contract - #26

Merged
Teakowa merged 6 commits into
mainfrom
feat/workshop-rs-18-conformance
Aug 18, 2026
Merged

Teakowa merged 6 commits into
mainfrom
feat/workshop-rs-18-conformance

Conversation

@Teakowa

@Teakowa Teakowa commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Implements #18 by adding the canonical workshop_rs::conformance public contract.

  • Adds namespace-qualified, locale-independent FeatureId values derived from canonical catalog identities or workshop-rs-owned namespaces.
  • Adds machine-readable result states: matched, unsupported, known-gap, unexpected-regression, and inconclusive.
  • Separates executed fixture provenance from independent expectation provenance and records catalog, locale, implementation, and live-client metadata.
  • Adds structured comparison artifacts/reason codes, SHA-256 validation, and a validated JSON entry point.
  • Documents the public ownership and downstream-consumer boundary in ADR-0002.

Evidence and checks

The contract has focused positive/negative serialization and validation tests, including duplicate features, status/reason mismatch, missing live-client provenance, missing artifact digests, implementation metadata not serving as an oracle, and a format-valid mismatched catalog-digest regression.

ConformanceResult::validate_against requires the complete evidence.catalog == supplied Catalog::identity() before accepting catalog-backed feature identities.

  • cargo fmt --all --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace --all-targets
  • cargo run -p workshop-rs --bin workshop-catalog-gen -- check
  • git diff --check

This PR establishes the offline contract only. It does not claim census coverage, real-project evidence, or live-client captures; those are the dependent #19–#21 workflows.

Define stable locale-independent feature identities and machine-readable conformance results with explicit evidence provenance, comparison states, and validation rules. Document the public contract for future census, corpus, and live-client workflows.
Separate fixture and expectation provenance, namespace feature identities, add structured comparison and reason data, and validate artifacts through the JSON entry point. Fixes #18
@Teakowa

Teakowa commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

Architect review\n\nIndependent Architect review identified four contract risks before dependent work: unscoped feature names, fixture/expectation provenance conflation, free-text-only result semantics, and raw JSON deserialization bypassing validation. Commit 0bf6917 addresses these by adding namespace-qualified identities (including domain-qualified enum members), separate fixture/expectation artifacts, structured comparison/reason codes with digest checks, and ConformanceResult::from_json validation. The review boundary remains explicit: this PR establishes the contract only; #19/#20/#21 must consume it without redefining it. No Architect approval is being claimed here; maintainer review remains required.

Require matched expected and observed artifacts to be distinct from the fixture and recorded implementation output, and cover the false-positive path with a regression test. Fixes #18
@Teakowa

Teakowa commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

Fresh QA found an artifact-aliasing false-positive risk. Commit 68a4340 now rejects matched expected/observed artifacts that are identical to each other, the executed fixture, or the recorded implementation artifact, with a regression test. This was revalidated locally with focused conformance tests, clippy -D warnings, catalog check, and diff-check; full CI is required again at the new head.

Add validate_against for canonical catalog and enum-member identities, keep non-materialized comparison artifacts valid, and document that #19 owns the independent semantic comparison gate. Fixes #18
@Teakowa

Teakowa commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

QA follow-up addressed in commit 0e2ee3d: validate_against(Catalog) now rejects fabricated catalog and enum-member identities; comparison artifacts remain valid when non-materialized (optional SHA-256); ADR-0002 explicitly assigns execution of semantic/normalized comparison to the independent #19 gate. Local full workspace tests, clippy, catalog check, fmt, and diff-check pass. Awaiting fresh CI and QA review at this head.

Reject expected artifacts that alias the implementation output while preserving the valid expectation-source relationship, with explicit negative coverage. Fixes #18
@Teakowa

Teakowa commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

Fresh QA blocker resolved in commit 9a1f753: expected artifacts are now also rejected when they alias the implementation artifact, while expected == expectation.artifact remains the valid independent-oracle relationship. Added regression coverage. Current local conformance tests/clippy/catalog check pass; awaiting CI and final independent QA.

@Teakowa

Teakowa commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

Fresh independent QA — VERIFIED / PASS

Exact head reviewed: 58832ff74326ac438f121895d54d3c791b6c4dfd.

  • ConformanceResult::validate_against requires exact evidence.catalog == catalog.identity().
  • The format-valid mismatched-catalog digest regression rejects the wrong identity.
  • Define the Workshop conformance result contract and stable feature identities #18 status/provenance/serialization and provider-neutral ownership boundaries are covered.
  • Local fmt, Clippy, workspace/all-targets tests, catalog check, and diff check passed.
  • Stable and Rust 1.85 CI passed on this exact head.
  • Worktree is clean and HEAD matches the PR head.

The previous QA on 9a1f753 is superseded by this exact-head review. No findings. #18 is ready for maintainer merge review.

@Teakowa
Teakowa marked this pull request as ready for review August 18, 2026 08:38
@Teakowa
Teakowa merged commit 51faf02 into main Aug 18, 2026
2 checks passed
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Make Workshop Great Again Aug 18, 2026
@Teakowa
Teakowa deleted the feat/workshop-rs-18-conformance branch August 18, 2026 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

1 participant