Prove the JustDummies generator invariants with a property-based suite#306
Merged
Merged
Conversation
The example-based suite pins hand-picked constraint arguments -- Between(10, 20), WithLength(12) -- and can only prove a generator right for those. The sampled loops it wraps them in are property tests in spirit, but the constraint space itself stays fixed, so a bound that overflows for one interval in a million is never drawn. Issue #206 was exactly that: a decimal interval whose draws never left the lower half, found by hand and pinned as a seeded regression. JustDummies.PropertyTests inverts the split. FsCheck generates the CONSTRAINTS -- the bounds, lengths, counts and seeds a caller declares -- while JustDummies generates the value that must satisfy them, and the property asserts the invariant over the whole space. A failure shrinks to its minimal counter-example instead of surfacing as an opaque sample. Drawing the constraints from FsCheck also breaks a circularity, since the suite no longer uses the component under test to decide what to test it with. The project mirrors its siblings: JustDummies only, so the library's standalone boundary stays visible in its own test bed, and the net472 floor leg so the invariants are proven against the netstandard2.0 asset .NET Framework consumers actually load. This commit carries the scaffold, the shared constraint generators, and the Int32 interval family as the exemplar the remaining families follow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019v9nNWTjPcLdv3K54adsWr
JustDummies claims that every value it builds satisfies every constraint declared on it, for every legal combination of constraint arguments -- a universal quantification. The example suite proves that claim by sampling: it pins the constraint arguments and varies only the drawn values, so it instantiates the claim at a handful of hand-picked points. Issue #206 lived in the dimension that leaves unexplored, the relation between an arbitrary bound and the produced value. Adding a sibling *.PropertyTests project is established practice here and needed no record on its own -- neither existing one has an ADR. What is new and lasting is the BOUNDARY, and the fact that tests are actively moved out of the example suite: properties own invariants quantified over generated constraint arguments, examples own message content, argument validation, structural conventions and dated regressions. That rule is what a future maintainer will question when the same bound appears tested in two places, and it survives any reorganization of the files themselves. Drafted as Proposed and indexed; only the maintainer accepts an ADR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019v9nNWTjPcLdv3K54adsWr
Four families following the Int32 exemplar, each quantifying over generated constraint arguments rather than hand-picked ones. Scalars cover the integer widths other than Int32, with the invariant set spread so every one is proven on at least one signed and one unsigned width. Continuous covers Double, Single and Decimal, including the both-halves reachability that issue #206 violated -- now asserted over arbitrary intervals instead of the single [0, 100] the seeded regression pins. Lattice covers MultipleOf, WithScale and WithGranularity, whose per-type anchor (zero, DateTime.MinValue, DateTimeOffset.MinValue) is the classic place to get the grid wrong. String covers the length, affix, charset and casing algebra, including the value-dependent legality ADR-0035 describes: WithLength(n) with StartingWith(prefix) is valid exactly when n is at least the prefix length, and the property branches on that relation rather than assuming either outcome. 87 properties pass on net10.0; the project also compiles on the net472 floor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019v9nNWTjPcLdv3K54adsWr
Completes the property suite at 210 properties across twelve families. The regular-expression round trip is the centrepiece: patterns are generated recursively over the supported subset -- nested alternation, named and non-capturing groups, negated classes, ranges, lazy and bounded quantifiers, \x and \u escapes -- and every value drawn from one is checked against the real .NET engine as oracle. Sampling the generator shows 100 distinct patterns per run averaging ~82 characters, so this fuzzes the home-grown engine rather than re-pinning a corpus. ADR-0025's safety net is now literally a property test, which is what the July audit asked for. Seeds are quantified rather than hand-picked: the example suite pins 12345, 777 and 31415, which says nothing about seeds in general. Collections quantify the count algebra including the eager cardinality conflict, where a distinct count exceeding what the element generator advertises must be refused at declaration time. Composition covers As, OrNull, Combine and the explicit pools, including that ElementOf materializes a lazy source exactly once. Temporal covers the offset dimension over arbitrary whole-minute offsets within the legal range. Modern types cover the net8-only generators and are excluded from the net472 leg by the project file. Two draws assembled a 64-bit word by or-ing shifted fields, which the compiler reads as or-ing a sign-extended operand (CS0675). The fields are disjoint, so they are summed instead: same word, no warning. 210 properties pass on net10.0 with no warnings; the suite also compiles on the net472 floor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019v9nNWTjPcLdv3K54adsWr
Sixteen tests asserted an invariant that holds for every legal constraint argument while pinning exactly one: Between containment and pinning, bound strictness and inclusiveness, OneOf and Except membership, recipe immutability, crossed-argument rejection, non-finite argument rejection, and seed agreement pinned to 12345, 777 and 31415. Each is now quantified over generated arguments in JustDummies.PropertyTests, which proves strictly more, so keeping the pinned version would assert a special case of a property one file away (ADR-0040). Removed only what a property strictly subsumes. Everything whose subject is a specific named case stays: conflict messages, which name both offending constraints in direction-aware wording; null and empty argument guards; the named domain extremes; that a bounded range is actually reached, which is statistical rather than universal; and the seeded regression for issue #206, which pins the interval where the defect occurred and is not retired by the property covering the class. Nothing was removed from the signed, unsigned or remaining suites: their methods interleave a sampled invariant with a conflict-message assertion, so removing the loop would take the assertion with it. Each pruned class now carries a note saying which half of its contract it keeps, so the absence reads as a decision rather than an oversight. JustDummies.UnitTests 535 -> 519; JustDummies.PropertyTests adds 210. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019v9nNWTjPcLdv3K54adsWr
ADR-0040 records WHY the test bed is split; a contributor adding a constraint or fixing a defect needs the HOW. The guide reduces the boundary to one question -- does the assertion have an input space? -- and applies it: bounds, lengths, counts, seeds and patterns are input spaces and belong to a property; message wording, null arguments, named domain extremes, structural conventions and dated regressions are not, and stay examples. It also carries the two disciplines that keep such a suite honest and that no ADR would hold: pin a seed for anything statistical, since "both halves are reached" is probabilistic and flakes without one; and break each property on purpose once to confirm it can go red, since a property returning true for the wrong reason passes silently and proves nothing. Indexed in the maintainer README and pointed to from CONTRIBUTING, CLAUDE and AGENTS, so it is found before a test is written rather than after -- the same reason the solution-nesting reminder was added to those files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019v9nNWTjPcLdv3K54adsWr
ADR-0040 ("Split the JustDummies test bed between an example suite and a
property suite") records the boundary the property suite and the pruning of the
example suite implement. The maintainer accepts the decision; the Date stays
2026-07-26, the day it took effect.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019v9nNWTjPcLdv3K54adsWr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
JustDummiesclaims that every value it builds satisfies every constraint declared on it, for every legal combination of constraint arguments — a universal quantification. The example suite proved that claim by sampling: it pinned the constraint arguments (Between(10, 20),WithLength(12)) and varied only the drawn values, instantiating the claim at a handful of hand-picked points. This addsJustDummies.PropertyTests, which generates the constraint arguments too and quantifies over them, and moves the invariants that belong there out of the example suite.Type of change
Changes
JustDummies.PropertyTests(FsCheck + xUnit v3), nested under thetestssolution folder and added to the .NET Framework 4.7.2 floor job inci.yml.MultipleOf,WithScale,WithGranularity— whose anchor differs per type), string shape, collections, composition, explicit pools, temporal types with the offset dimension, seeded determinism, URIs, and the net8-only generators.\x/\uescapes — and every value drawn from one is checked against the real .NET engine as oracle. Sampling the generator shows 100 distinct patterns per run averaging ~82 characters, so this fuzzes the home-grown engine rather than re-pinning a corpus.PropertyTestSupport.cs: ordered bound pairs, edge-biased numeric generators (FsCheck's own cluster near zero, so the domain ends would almost never be drawn), and per-case multi-draw helpers.JustDummies.UnitTestsfrom 535 to 519 tests, removing only what a property strictly subsumes. Each pruned class carries a note saying which half of its contract it keeps.Testing
dotnet build FirstClassErrors.slndotnet test FirstClassErrors.slnFirstClassErrors.Analyzers.UnitTests)Additionally run locally, since a plain
dotnet testdoes not cover them:dotnet build JustDummies.PropertyTests -c Release -f net472 -p:EnableNet472Floor=true— succeeds, as does the same forJustDummies.UnitTests.Whole solution green, no warnings.
JustDummies.UnitTests519,JustDummies.PropertyTests210.Documentation
doc/updateddoc/handwritten/for-users/README.fr.md) updated if user-facing behavior changedNo public API changed and no user-facing behavior changed, so the user-facing README and its French translation are untouched. New maintainer documentation:
doc/handwritten/for-maintainers/WritingJustDummiesTests.{en,fr}.md— where a new test belongs and how to write it, reduced to one question: does the assertion have an input space? Indexed in the maintainer README.CONTRIBUTING.md,CLAUDE.mdandAGENTS.md, so the rule is found before a test is written rather than after.Architecture decisions
Drafted as
Proposed; the maintainer then explicitly instructed acceptance, applied inbee8c2c(status flipped in both language versions and in the index, Date unchanged).Two notes for the reviewer:
*.PropertyTestsproject is not what the ADR records — neither existing one has an ADR. What it records is the boundary, and the fact that tests are actively moved out of the example suite.Reviewer notes — deliberate limits
OneOfcombined with an interval (the conflict surface there is value-dependent), and strict bounds ondecimalnearMaxValue(the1E-28exclusion nudge vanishes in rounding there — documented behavior, not a defect).Related issues
Relates to #206 — the decimal interval defect whose class is now quantified rather than pinned to one interval.
Generated by Claude Code