Skip to content

PRDCT-565: docs quality gates - #1002

Draft
Iamfle4ka wants to merge 9 commits into
mainfrom
chore/docs-quality-gates
Draft

PRDCT-565: docs quality gates#1002
Iamfle4ka wants to merge 9 commits into
mainfrom
chore/docs-quality-gates

Conversation

@Iamfle4ka

@Iamfle4ka Iamfle4ka commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Adds automated quality gates to the PR build so content and rendering regressions block merge — prompted by the wide-table display:block bug (PRDCT-379) reaching main unnoticed.

Note on the force-push (4 Aug). This branch had sat unmerged since 8 Jul while main moved 118 commits, and two parallel branches had independently rebuilt the same gates. The history was rewritten to consolidate them into this one PR, fix three defects found while verifying, and — see the last section — restore a documentation section that a bad merge on the superseded branch had silently deleted.

What runs on every PR

.github/workflows/branch.yml builds the PR and then runs four checks. Two are ratchets, not absolute gates: the docs carry real inherited debt (broken links into pages that still live on developers.keboola.com), so they compare against a committed baseline and fail only on breakage the branch introduced.

Check Fails when Baseline
npm run lint a PR adds a broken link or anchor, a missing/placeholder image, frontmatter without title/slug, an unclosed fence, a raw HTML table, a Liquid leftover, a deprecated term scripts/lint-baseline.json
npm run check:redirects a PR orphans a dev URL that used to resolve — a page moved, renamed or deleted without a redirect _data/redirects/not-yet-migrated.txt
npm run check:cli docs reference kbagent commands or flags the CLI no longer has
npm run check:render a table or layout regresses across viewports

Both baselines shrinking is the point — they double as the remaining cleanup list. AGENTS.md documents how to rebaseline and why growing one needs justification.

Three defects fixed in the gate itself

1. A deleted page read as healthy. resolves() in lint-docs.mjs (and audit-phase2.mjs, and inDist() in check-redirects.mjs) accepted dist/<route>/ as proof a page exists. But a page directory outlives its index.html — the page's images sit in it. So the one breakage these gates exist to catch was invisible. Every candidate must now be a file. Measured: deleting dist/storage/tables/index.html produced 0 findings before, 390 after.

2. A CI step with no script behind it. main had gained check:cli; this branch replaced that line in package.json, so the Check CLI reference freshness step that branch.yml already called had nothing to run. Both sets of scripts are kept.

3. A platform-dependent sample. Cherry-picked 6966b5b1, which existed only on the pushed ref: check-render sorts before sampling, because readdir order differs between macOS and Linux and an unsorted sample means local pass / CI fail.

The URL contract now actually runs

check-redirects.mjs and the dev→help map come from PRDCT-572, which wires neither to CI — so the conservation check had never run on a PR. It runs here, tolerating the 119 dev URLs never yet migrated and failing only on ones this branch orphans. Two soft skips keep merge order from mattering: a missing map skips the check; a present map with no baseline reports instead of failing. PRDCT-572 landing therefore cannot redden main.

Baseline regenerated after fixing the detector, not to silence it

630 findings, up from 605 measured against a month-old tree. The 9 that main's newer content added are now listed as known debt — including one real broken link, /404 → /overview/environment/, worth a follow-up.

A documentation section came back

The superseded branch's merge commit b38f12f8 had dropped 20 lines from src/content/docs/workspace/index.md — the whole Bringing Your Own Key Pair section (BYO Snowflake key pair, needed by tools like SAS CI360) plus two cross-references to it. Merging that state would have deleted the section from production. This branch's copy of the file is byte-identical to main.

Worth noting what it implies: no gate here would have caught it. Deleted prose is not a broken link.

Verification

All ten CI steps pass on this commit. Locally, each gate was verified by breaking the build on purpose, not by inspection:

  • remove a linked page → lint exits 1 with 390 new findings; restore → exits 0
  • remove a contract-mapped page → check:redirects exits 1 naming the orphaned dev URL; restore → exits 0
  • move the map away → skip, exits 0; map present, baseline absent → report, exits 0

@vercel

vercel Bot commented Jul 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
connection-docs Ready Ready Preview Aug 8, 2026 12:08am

Request Review

@Iamfle4ka Iamfle4ka changed the title chore(docs): content lint + render/layout regression gates PRDCT-469: content lint + render/layout regression gates Jul 8, 2026
@linear-code

linear-code Bot commented Jul 8, 2026

Copy link
Copy Markdown

PRDCT-469

PRDCT-378

PRDCT-565

Nikita and others added 2 commits July 9, 2026 01:29
Nothing checked rendering, so the wide-table display:block regression
(PRDCT-379) shipped to main. Add two gates wired into the PR build:

- scripts/lint-docs.mjs — content lint (broken links/anchors, bare code
  fences, missing title/slug/description, deprecated terms, raw HTML tables,
  Liquid leftovers, placeholder images). Gated via scripts/lint-baseline.json:
  fails only on findings not already in the baseline, so it catches regressions
  today without first clearing the existing backlog (1155 known findings).
- scripts/check-render.mjs — headless Playwright over `astro preview`: asserts
  every .beacon-table-scroll > table computes display:table (catches the
  Starlight display:block collapse), wrapper overflow-x:auto, no horizontal body
  scroll at 375/768/1280, all images load, headings have ids.
- package.json: `lint`, `check:render`, `audit` scripts + playwright devDep.
- branch.yml: run lint + render checks after build on every PR.

Verified: check-render currently flags the still-unmerged table bug on main
(105 failures, all the display:block table); applying the PRDCT-379 fix locally
makes it pass. Merge order: PRDCT-379 (#989) first, then this.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- branch.yml: fail-fast step — PR title must match ^PRDCT-[0-9]+ (auto-links
  the PR to its Linear issue).
- Document the rule in AGENTS.md (Workflow) and CLAUDE.md (notes).

Folded into the quality-gates PR since both touch branch.yml. Closes PRDCT-378.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Iamfle4ka

Copy link
Copy Markdown
Collaborator Author

Refreshed status (local run, 2026-07-17): branch is already up to date with main (0 behind). Gates verified locally:

  • PRDCT-378 title check — in place (branch.yml), matches the convention the whole PR queue now follows.
  • Content lint — ✅ green: OK — no new findings vs baseline on a fresh build.
  • Render/layout check (playwright vs preview) — ❌ red with ~33 real findings, all one class: /components/extractors/other/telemetry-data/ tables render display:block (the known wide-table clipping — exactly what PRDCT-379: fix wide tables clipping under the TOC instead of scrolling #989 fixes, currently CHANGES_REQUESTED).

Recommendation: merge order #989 → this PR (gate turns on green), or land this first with the telemetry-data findings baselined. Either way this is the cheapest way to get quality gates on every PR — relevant right now with the phase-1 migration PR #1027 in flight.

@Iamfle4ka

Copy link
Copy Markdown
Collaborator Author

Rebased onto mainCONFLICTINGMERGEABLE.

Left as a draft on purpose — tooling sits behind the content work in Michal's ordering. Ready to un-draft whenever you want it in the queue.

Nikita Zverev and others added 4 commits August 4, 2026 02:36
The gate has sat unmerged since 8 Jul while main moved 118 commits. Two
detector bugs would have failed every PR on noise, and rebasing would have
silently dropped the check:cli step main gained in the meantime.

- lint-docs: parse frontmatter with \r?\n. 56 pages close theirs with
  '---\r\n', so an LF-only regex read the whole block as absent and reported
  112 phantom missing-title/missing-slug findings.
- lint-docs: skip /_astro/ in the placeholder-image rule. Those are
  build-hashed theme assets (the ~2 KB Starlight logo on every page) —
  623 findings collapse to 11 real ones.
- branch.yml: re-add 'Check CLI reference freshness'; this branch predates it.
- lint-baseline.json: regenerated against today's main content — 627 known
  findings (was 1151 from a month-old tree). Verified: lint is green on the
  current build, so the gate starts catching regressions the moment it merges.

Still real and NOT baselined away as unknown: 44 broken internal links from
just 12 unique targets, plus 85 broken anchors, mostly in the migrated
extend/common-interface pages. Some targets only exist on the unmerged
PRDCT-552 / PRDCT-554 branches.
readdir order differs between macOS and Linux, so the TABLE_CAP sample tested
different pages locally vs in CI (local pass, CI fail on telemetry-data). Sort
the walk and log how many table pages the cap skips.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two defects in the gate itself, plus the redirect checker that was never
connected to CI.

A dead route read as healthy. `resolves()` in lint-docs.mjs (and audit-phase2.mjs,
and `inDist()` in check-redirects.mjs) accepted `dist/<route>/` as proof a page
exists, but a page directory outlives its index.html because the page's images sit
in it. So the one breakage the gate exists to catch — a page that stopped being
built — was invisible. Measured: deleting dist/storage/tables/index.html produced
0 findings before, 390 after requiring every candidate to be a file.

check-redirects.mjs and the dev→help map came from PRDCT-572, which wires neither
to CI, so the conservation check has never run on a PR. It runs now, as a ratchet:
it fails when a dev URL that used to resolve resolves nowhere (a page moved or
deleted without a redirect) and tolerates the 119 URLs never yet migrated, listed
in _data/redirects/not-yet-migrated.txt. Two soft skips keep merge order from
mattering — a missing map skips the check, a present map with no baseline reports
instead of failing, so PRDCT-572 landing cannot redden main.

Also merged main (the branch was 118 commits behind, base 8 Jul) and cherry-picked
6966b5b, which existed only on the pushed ref: check-render sorts before sampling,
since readdir order differs between macOS and Linux and a platform-dependent
sample means local pass / CI fail.

The merge resolved a package.json conflict that mattered: main had gained
`check:cli`, this branch replaced that line, so `npm run check:cli` — a step
branch.yml already calls — had no script behind it. Both sets of scripts kept.

Baseline regenerated after fixing the detector, never to silence it: 630 findings
(was 605 against a month-old tree). The 9 that main's newer content added are now
listed as known debt, including one real broken link, /404 → /overview/environment/,
worth a follow-up.

Verified in this worktree: all four gates green on a clean build; lint exits 1 with
390 new findings when a linked page is removed and 0 when restored; the URL contract
exits 1 naming the orphaned dev URL and 0 when restored; both soft skips exit 0.
check:render is unverified locally — playwright cannot install without network; CI
installs it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nikita added 2 commits August 7, 2026 01:30
…gates

# Conflicts:
#	AGENTS.md
#	CLAUDE.md
The five new findings are inherited from pages that landed on main after this
branch's baseline was captured (Data Streams tutorial/reference from PRDCT-541,
the deprecated 'orchestration' term on transformations/variables). None is
introduced here — this branch touches no content pages.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant