You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add an exhaustive, data-driven Workshop conformance suite that verifies the audited native language surface by category and by individual feature, using deterministic offline input -> expected semantic/output behavior.
The suite should make it possible to answer both:
does an entire Workshop category remain conformant? and
which exact Action / Value / Event / Constant / operator / structural feature regressed?
Context
#19 established sharded census coverage for the repository's declared surface, but that census is derived from existing catalog/WIR capabilities. #86 adds the missing external documentation-driven inventory audit.
Once the documented inventory is authoritative, testing should stop relying mainly on representative examples and real-project incidental coverage. Every supported leaf capability should have an attributable conformance case where the language contract can be exercised offline.
This is a language/tooling correctness suite, not a live-game integration suite.
Scope
Build category-level conformance sets for at least:
Actions;
Values;
Events;
Constants / enum domains and members;
operators and variable-modification operations;
structure/control flow;
settings;
strings/localization where applicable.
Give every individual documented feature a stable, human-readable test identity, for example:
Prefer data-driven cases/generators over hundreds of unrelated hand-written Rust test functions.
For each supported capability, test the offline language contract that is applicable to that feature, including as relevant:
documented source spelling/category resolves to the expected canonical identity;
documented parameter count/order is accepted;
documented parameter types/domains and defaults are validated;
documented Value return contract is represented;
a minimal valid Workshop input parses/validates successfully;
representative invalid arity/domain/category input is rejected with the expected structured diagnostic class;
canonical emission uses the expected Workshop identity/spelling and argument ordering;
emitted output reparses to the same semantics;
supported locale conversion preserves the same canonical semantics.
Where deterministic presentation is part of the declared emitter contract, test normalized/canonical output presentation. Do not turn irrelevant whitespace or upstream formatter identity into a compatibility requirement.
Produce category summaries suitable for CI diagnostics, for example Actions, Values, Events, and Constants, while retaining exact per-feature failure identities.
Ensure the suite fails when an audited ✅ Supported feature has no conformance case, so support documentation and test coverage cannot silently diverge.
Keep 🚧 Coming soon and ❌ Unsupported entries explicit without fabricating passing tests for behavior that is not implemented.
Correctness boundary
The test oracle is the reviewed Workshop language/semantic contract and repository-declared canonical behavior.
The suite should verify that known inputs produce the expected parse/validation/semantic/emission results and that the documented support matrix, semantic model, and observable compiler/tooling behavior remain consistent.
No test should require execution inside Overwatch. Gameplay consequences such as actual damage dealt, movement, physics, timing, hero state, or engine-side effects are outside this suite because workshop-rs cannot integration-test them reliably.
A documented runtime note should only affect this suite when it constrains an offline semantic operation such as validation, static analysis, constant folding, or a source transformation. The suite must not introduce a game simulator to test such notes.
Non-goals
Live client/game integration tests.
Simulating Workshop runtime execution or gameplay effects.
Byte-identical comparison with OverPy, OSTW, or Blizzard-generated formatting.
Treating a large all-features fixture as a substitute for individually attributable cases.
Fixed-count assertions such as 219/219 that can stay green while the external inventory changes; completeness must be derived from the audited feature inventory.
Duplicating OPY/DEL source-language test suites.
Weakening validation or changing expected output merely to make the suite pass.
Acceptance criteria
Every audited ✅ Supported Workshop leaf capability that can be exercised offline has an attributable conformance case.
CI can report pass/fail by major category and exact feature identity.
Actions, Values, Events, and Constants each have dedicated category coverage rather than being represented only incidentally by corpus projects.
Same-name features in different semantic categories have independent cases; values/raise-to-power and the variable-modification Raise To Power operation cannot satisfy each other's coverage.
Valid test inputs resolve/validate/emit to the expected semantic result and supported canonical presentation.
Representative invalid inputs verify documented arity/domain/category rejection where applicable.
Extends rather than replaces Build the canonical sharded Workshop feature census #19: the existing census remains useful for sharded integration/corpus coverage, while this issue provides exhaustive per-category/per-feature language conformance.
docs/language-support.md remains the human-facing support truth; this suite is the executable guard that supported rows remain true.
Goal
Add an exhaustive, data-driven Workshop conformance suite that verifies the audited native language surface by category and by individual feature, using deterministic offline input -> expected semantic/output behavior.
The suite should make it possible to answer both:
Context
#19 established sharded census coverage for the repository's declared surface, but that census is derived from existing catalog/WIR capabilities. #86 adds the missing external documentation-driven inventory audit.
Once the documented inventory is authoritative, testing should stop relying mainly on representative examples and real-project incidental coverage. Every supported leaf capability should have an attributable conformance case where the language contract can be exercised offline.
This is a language/tooling correctness suite, not a live-game integration suite.
Scope
actions/damage;values/raise-to-power;events/player-dealt-damage;constants/team/team-1;operations/raise-to-power.docs/language-support.md, not only from the implementation catalog under test.Actions,Values,Events, andConstants, while retaining exact per-feature failure identities.✅ Supportedfeature has no conformance case, so support documentation and test coverage cannot silently diverge.🚧 Coming soonand❌ Unsupportedentries explicit without fabricating passing tests for behavior that is not implemented.Correctness boundary
The test oracle is the reviewed Workshop language/semantic contract and repository-declared canonical behavior.
The suite should verify that known inputs produce the expected parse/validation/semantic/emission results and that the documented support matrix, semantic model, and observable compiler/tooling behavior remain consistent.
No test should require execution inside Overwatch. Gameplay consequences such as actual damage dealt, movement, physics, timing, hero state, or engine-side effects are outside this suite because
workshop-rscannot integration-test them reliably.A documented runtime note should only affect this suite when it constrains an offline semantic operation such as validation, static analysis, constant folding, or a source transformation. The suite must not introduce a game simulator to test such notes.
Non-goals
219/219that can stay green while the external inventory changes; completeness must be derived from the audited feature inventory.Acceptance criteria
✅ SupportedWorkshop leaf capability that can be exercised offline has an attributable conformance case.values/raise-to-powerand the variable-modificationRaise To Poweroperation cannot satisfy each other's coverage.Dependencies / relationships
docs/language-support.mdremains the human-facing support truth; this suite is the executable guard that supported rows remain true.