Put the scaling numbers where the objection is raised - #386
Merged
Conversation
The README's only performance figure was 7.4 seconds to index 768 commits, from a field report. That answers "is this cheap to try" and leaves the question anyone with a real repository asks next: what happens at a hundred times that size. The answer is measured and it was sitting in docs/evidence.md. An indexed context query at 100,000 commits is 496 ms p50; the commit-msg hook is 185.85 ms p50 and the injection hook 102.40 ms. Those are the numbers that decide whether this stays installed rather than whether it gets tried once, and the second question is the one that costs a user something to answer for themselves. The paragraph also carries the figure from that run that looks bad: the same query without an index takes 86,673 ms at 100,000 commits. Publishing the good number while leaving that one in a linked document would be choosing which half of a single measurement a reader sees. It also explains a design choice rather than excusing it -- the index is not a speedup on a query that already worked, it is what makes the query possible at that size, which is why init builds one and doctor checks it. Record-Id: r-scaleproof Limit: the 100,000-commit figures come from a synthetic repository built by the deterministic harness, not from a real codebase of that size, so they describe the index's shape rather than any particular project Ruled-out: Quoting only the 496 ms | it is one arm of a run whose other arm is 86,673 ms, and a reader who found that later would be right to distrust the first number Ruled-out: A latency table | three figures do not need one, and the README just had its reference material moved into docs/ Certainty: firm Blast: local Undo: easy Verified: readme, readme-order, readme-numbers, readme-positioning and compatibility-matrix pass at 83 across all four language files; check-readme-numbers.mjs exit 0 with the BENCH block byte-identical; spec/verify.sh OK at 26 fixtures; the beautify skill's audit_readme.py reports no issues; every figure quoted was read out of docs/evidence.md rather than recomputed Unverified: how the indexed query behaves on a real repository at that size, which no run in this project has measured
CommitLore — record lintTrailers: clean — 1 commit in Active constraints for the paths this PR touchesLimits (49)
Ruled out (117)
Warnings (24)
Trailer violations fail this check. Active constraints are informational — they are what the repository already decided, not a verdict on this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The README's only performance figure was 7.4 seconds to index 768 commits, from a field report. That answers is this cheap to try and leaves the question anyone with a real repository asks next: what happens at a hundred times that size.
The answer was measured and sitting in
docs/evidence.md.contextquery at 100,000 commitscommit-msghookThe last row is in the paragraph too. Publishing the good number while leaving that one in a linked document would be choosing which half of a single measurement a reader sees.
It also explains a design choice instead of excusing it: the index is not a speedup on a query that already worked — it is what makes the query possible at that size, which is why
initbuilds one anddoctorchecks it.Verified
readme,readme-order,readme-numbers,readme-positioning,compatibility-matrixcheck-readme-numbers.mjsspec/verify.shbeautify-github-readmeauditdocs/evidence.md, not recomputedStated limit, in the commit: the 100,000-commit figures come from a synthetic repository built by the deterministic harness, not a real codebase of that size. They describe the index's shape, not any particular project.