Skip to content

docs(rfc): add RFC 0043, GQ logic tests - #584

Merged
azimafroozeh merged 3 commits into
ModernRelay:mainfrom
azimafroozeh:gq-fixtures-rfc
Sep 2, 2026
Merged

azimafroozeh merged 3 commits into
ModernRelay:mainfrom
azimafroozeh:gq-fixtures-rfc

Conversation

@azimafroozeh

@azimafroozeh azimafroozeh commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

What & why

RFC 0043 makes query-behavior tests one self-contained text file per case (.gqt): a header (issue anchor, red_on: provenance), a .pg schema, JSONL seed rows, then steps (--- query / --- mutate / --- restart, loopable), each with params and an expected outcome (rows, affected counts, or an error substring). One test target, crates/omnigraph/tests/gq_logic_tests.rs, runs each case against a fresh temporary store through public engine surfaces only; no second toolchain. An enforcement ladder makes the medium stick: AGENTS.md contract sentences, a CI gate holding issue-closing fix PRs to a matching regression, and a no-repro waiver label.

Why now: review of the #563 fix surfaced a pure input-to-output defect (aggregates computed over the capped BM25 scan window) a twenty-row logic test case would have caught before review. The format adopts the sqllogictest lineage (SQLite, DuckDB, DataFusion) at its mature state and omits its documented mistakes (result hashing, type strings, bare any-error expectations).

Backing issue / RFC

  • Is an RFC PR: docs/rfcs/0043-gq-logic-tests.md

Checklist

  • Change is focused (one RFC plus its registry row)
  • Tests added/updated for behavior changes (N/A: proposal only)
  • Public docs updated if user-facing surface changed (the RFC and its registry row are the docs; no engine surface changes)
  • Reviewed against docs/dev/invariants.md — no Hard Invariant weakened, no deny-list item hit

Local verification

  • python3 scripts/check-docs.py — green (front matter, registry row, links)
  • cargo test --workspace — not run: docs-only diff

Notes for reviewers

Greptile Summary

This documentation-only PR proposes RFC 0045, defining a self-contained .gqt format and an enforcement model for query-behavior regression tests.

  • Specifies schema, seed, query, mutation, restart, loop, comparison, and bless semantics.
  • Defines CI execution, fix-regression gating, waiver handling, and heavy-reproduction tiers.
  • Registers RFC 0045 in the RFC index.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename Overview
docs/rfcs/0045-gq-logic-tests.md Adds the RFC describing the .gqt format, runner semantics, CI enforcement, compatibility boundaries, evidence plan, and rollout.
docs/rfcs/README.md Adds the RFC 0045 registry entry with frontmatter-consistent metadata.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Case[.gqt case] --> Parse[Fail-closed parser]
  Parse --> Init[Fresh temporary store]
  Init --> Seed[Load JSONL seed]
  Seed --> Index[Ensure required indexes]
  Index --> Steps[Run query / mutate / restart steps]
  Steps --> Compare[Normalize and compare outcomes]
  Compare --> Result[Report case result]
  PR[Issue-closing PR] --> Gate[Fix Regression Gate]
  Gate -->|matching regression| CI[GQ Logic Tests]
  Gate -->|no-repro label| Waiver[Maintainer waiver]
  CI --> Result
Loading

Reviews (5): Last reviewed commit: "docs(rfc): address review: gate guarante..." | Re-trigger Greptile

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@aaltshuler aaltshuler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three RFC-level comments: the regression-gate guarantee, ranked ordering semantics, and bounded runner concurrency.

Comment thread docs/rfcs/0045-gq-logic-tests.md Outdated
Comment thread docs/rfcs/0045-gq-logic-tests.md Outdated
Comment thread docs/rfcs/0045-gq-logic-tests.md
@azimafroozeh
azimafroozeh merged commit d7b709e into ModernRelay:main Sep 2, 2026
21 checks passed
@azimafroozeh
azimafroozeh deleted the gq-fixtures-rfc branch September 2, 2026 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants