Conversation
* feat(autopilot:postgres-support): phase 0 complete — gate PASSED
Introduces the Database backend abstraction (Sqlite/Postgres enum, URL-scheme
dispatch via connect()), enables sqlx's postgres feature, documents the
selection convention in config.yaml, and records the decision in ADR-033. The
struct-to-enum change required ~250 mechanical db.pool -> db.pool() renames
across 27 files (plus 3 Database{pool} -> Database::Sqlite(pool) test-helper
fixes) to keep the crate compiling — purely syntactic, no behavior change,
confirmed by the full test suite passing unchanged (1177 tests). SQLite path
is provably unchanged; Postgres connects but isn't usable end to end until
later phases (migrations, call-site migration, CI).
Gate: cargo fmt/clippy/build/test all green. Tier-3 adversarial review (4
parallel agents: CLAUDE.md compliance, bug scan, git-blame history, comment
compliance) found no blocking issues; two ADR doc-accuracy nits and one
doc-comment precision gap were found and fixed.
* fix(autopilot:postgres-support): ci attempt #1 — config.yaml comment exceeded yamllint's 200-char line limit
…y to new phase 5 Researched (RuvNet brain + web) how to reach BM25 parity with SQLite FTS5 on PostgreSQL. ruvector-postgres (already vendored via ADR-003's RuVector commitment, MIT-licensed) beats ParadeDB's pg_search (AGPL-3.0, new dependency) and vanilla tsvector/ts_rank (not BM25) on both license and zero-new-dependency grounds. Recorded as ADR-034. Since adopting it needs a custom Postgres image + new index-maintenance code - a real, independently-reviewable slice of work - it becomes its own phase 5 (depends_on [1, 3], added to risk_phases) rather than riding on phase 3's CI/infra scope. Phase 1 now explicitly excludes the 3 FTS5-only migrations and commits to the parallel-per-backend-migration-directories strategy; phase 4 documents the interim gap.
* feat(autopilot:postgres-support): phase 1 complete — gate PASSED
Splits backend/migrations/ into per-backend directories: sqlite/ (the existing
29 files, moved verbatim) and postgres/ (26 files -- every migration except
the 3 SQLite FTS5-only ones, which have no Postgres equivalent per ADR-034).
Database::run_migrations() now runs the matching sqlx::migrate! embedded
migrator per backend.
Dialect port: AUTOINCREMENT -> GENERATED ALWAYS AS IDENTITY, BLOB -> BYTEA,
DATETIME -> TIMESTAMPTZ, datetime('now') -> now() (or a to_char(...) expression
for columns that stay TEXT, matching what downstream Rust string-parsing
expects). Two fixes only surfaced by actually running the ported migrations
against a live postgres:16-alpine container: Postgres's real BOOLEAN type
rejects an integer literal default (migration 027), and two unix-millis
columns needed BIGINT instead of INTEGER since Postgres's INTEGER is a real
4-byte type unlike SQLite's dynamically-8-byte one (migrations 024/025).
~13 include_str!() call sites (all SQLite-only test helpers) updated to the
new sqlite/ path.
Gate: cargo fmt/clippy/build/test all green (1177 tests, 0 failed -- same
count as before this phase). All 26 Postgres migrations verified end-to-end
against a live postgres:16-alpine container (docker run), including spot-
checking the resulting schema via psql. Tier-3 adversarial review (dedicated
reviewer pass + 3 parallel agents: CLAUDE.md compliance, bug scan, comment
compliance) found no blocking issues; one real comment inaccuracy (a
copy-pasted column reference in 025's dialect note) found and fixed.
* fix(autopilot:postgres-support): ci attempt #1 — rustfmt after lengthened include_str! paths
SeaORM 2.0 dialect layer (ADR-036): entities, sea_orm() accessor, plan/job exemplar repo ports, spike graduated. 1205 backend tests + live postgres:16-alpine verification. qe-court SHIP (36 charges, 0 surviving, 2 overturn rounds). See PR #238 for full evidence.
…coveries into the pipeline Phase 3 amended (grep-driven .pool() sweep, write-side TIMESTAMP TZ fix, two-owner scoping tests mandatory); phase 4 amended (CI exports EMAILIBRIUM_TEST_PG_URL); NEW phase 7 (apply-lifecycle: begin_apply CAS + terminal-persist handling; added to risk_phases); NEW phase 8 (five cleanup-domain latent-bug fixes as deliberate behavior changes). Nine parking-lot records promoted with status transitions; pl-plan-id-only-authz stays open, deferred to the ADR-031 multi-tenancy track.
Full SeaORM port (ADR-036): every backend call site on the single dialect code path; Database::adapt(), audited_sql(), sqlite_placeholders_to_postgres() and Database::pool() deleted. 1300 backend tests + clippy green; DoD greps clean; live postgres:16-alpine 15/15 repo-layer verification (caught + fixed 2 PG decode bugs). qe-court SHIP (39 charges, 0 surviving, 0 overturn rounds; 26 remediated incl. 8 mutation kill-pins). 7 new parking-lot records. See PR #259 for full evidence.
CI proves PostgreSQL; SQLite vs PostgreSQL is one flag per deploy mode. qe-court SHIP (upheld on the 4th overturn pass). Jury binding condition: phase 5 must merge before any promotion to main, release, or documentation publication from develop.
…phase 5, re-home 6/7/8 by severity The parking lot had 40 open items across four lineages, and the two with a scheduled phase were not the two most severe. postgres-support had folded its phases 7 and 8 in opportunistically — because the phase-2 court happened to find those bugs during the port — so they were queued to run FIRST, ahead of a confirmed-broken GDPR erasure path that had no phase at all. This re-plan fixes the ordering rather than the symptom. Active plan now ends at phase 5 (docs), which is also what the phase-4 court's binding condition requires to precede the integration PR. Removed phases are re-homed, not dropped: * phase 6 (Postgres FTS) -> queued/postgres-fulltext-search, moved intact. It is a capability addition, not "make Postgres work" — the pipeline's goal is met at phase 5, which already documents the gap citing ADR-034. Holding phases 0-5 unmerged behind a new compiled Postgres extension, a third-party image of unproven availability, and hand-written TF/DF queries served nobody. * phases 7/8 -> queued/correctness-debt as its phases 2 and 4, moved intact. correctness-debt orders the backlog by severity instead of discovery order: erasure-that-does-not-erase, then the AppState lib-export (deliberately second — api_integration.rs REPLICATES production handlers, so nothing after it is honestly testable until it lands), then apply-lifecycle races, concurrent-writer races, cleanup latent bugs, config knobs that silently do nothing, and a final sweep whose DoD is a machine-checkable zero un-adjudicated items. Also re-opened six discovered items that were stamped `promoted` into phases 7/8. Those phases never ran, and `promoted` on an unrun phase reads as fixed — the precise mislabel that let the erasure bug sit below them. Their truthful status is open until correctness-debt is promoted and the owning phase gate-PASSES. Stale numbering corrected: phase 5's FTS-gap text said the gap lasts "until phase 5", left over from RE-PLAN 2's renumbering — it now names the queued pipeline.
…seline ahead of everything A user-requested audit of the MCP implementation found two CRITICAL exposures that are live in the SHIPPED production config, not hypothetical: * No authentication exists anywhere. backend/src/middleware/ has no auth module; the only JWT references are log-scrubbing patterns and an aspirational comment at rate_limit.rs:386. The MCP transport is nest_service'd at main.rs:905 as a SIBLING of api::routes(), so auth added inside the API router would not cover it — the obvious fix silently misses the endpoint that most needs it. * config.development.yaml and config.production.yaml BOTH set host: 0.0.0.0, overriding backend/config.yaml's own safe 127.0.0.1 default, while compose publishes 8080. Together: an unauthenticated MCP endpoint serving get_email / search_emails / list_attachments — full message bodies — to anyone who can reach the port. The tools being read-only does not mitigate this. Read-only limits damage; it does nothing about disclosure, and for an email application disclosure is the larger harm. The read-only posture was being counted as a security control it is not. security-baseline is deliberately a tiny two-phase lineage so it can run FIRST, ahead of correctness-debt. Making a one-line bind fix and a small auth layer wait behind seven phases is the wrong trade. It is explicitly NOT the identity substrate — scoping every tool by user is the larger mcp-identity-authz work, which is also the ADR-031 multi-tenancy foundation. Two further findings recorded, not fixed: the tool rate limiter is keyed by tool name only, so every MCP client and the chat assistant share one budget (pl-tool-rate-limit-not-per-caller); and docs/plan/mcp-maturation.md — the document ADR-028 sends readers to for "the lived state" — is stale on four points verified against the tree, including claiming a chat/MCP tool duplication that no longer exists (pl-mcp-maturation-doc-stale). Audit conclusions worth keeping: MCP is transport-only over a shared tools layer, NOT a parallel implementation and NOT layered on REST — one registry serves the MCP transport, the chat orchestrator and tests, with CallSource keeping them separable in the audit trail. That is a good foundation to expand onto.
Operator docs describe the PostgreSQL support that now exists, closing the loop docs-accuracy-audit opened. DoD 5/5; Tier-3 review verified all 8 factual claims against the code. Final phase of the pipeline.
Appended after the merge rather than with it: the squash-merge landed before the firing record was written, so the ledger briefly had markers for phases 0-5 and firing records for 0-4 only. Marker points at 62419a9, the real gate-PASSED squash commit on develop. Records the phase honestly including the CI story: 0 fix attempts, because both apparent failures were GitHub runner starvation (steps:[] and 'Set up job' only — nothing of ours executed) and the same commit passed in 33s and 7s once real runners were available. No repair commit was manufactured for a defect that did not exist.
Owner
Author
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
…skipped by decision Records the skip explicitly rather than leaving a gap. qe-court was AVAILABLE with 3 vendors — this was a maintainer decision to treat the release-level review as optional, not an accelerator-absence degrade, and the two read differently to anyone auditing later. Per-phase courts all convened and shipped: 95 charges filed across phases 2/3/4, zero surviving. develop -> main is human-gated; autopilot does not merge trunk.
…mit trips the marker grep Self-inflicted and worth writing down because it recurs. orchestrate derives the done-set by grepping commit subjects for 'phase .* gate PASSED'; the phase-5 ledger commit's own subject matches, so develop shows 7 marker-looking commits for 6 phases. Harmless today — the done-set is a set, not a count, and orchestrate never counts matches — but it is a false positive against a load-bearing convention, and it is not fixable retroactively without force-pushing the base branch, which the invariants forbid. Recorded with the convention fix for future pipelines: keep that phrase out of bookkeeping commit SUBJECTS. Phase 3 did the same thing and only escaped notice because the squash-merge collapsed it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ready for human review; do NOT auto-merge. Autopilot never merges
main— this PR is the handoff.171 files, +17,014 / −5,509, across 11 commits. Every phase carries a
gate PASSEDmarker ondevelop.What this delivers
PostgreSQL was a
docker-composeservice the backend never talked to, plus a doc claim. It is now a working second backend: URL-scheme dispatch (ADR-033), per-dialect migrations, every call site on a single SeaORM code path (ADR-036), a CI job running the suite against a live server, and a smoke workflow that proves a compose deployment connects to and writes to PostgreSQL.Phases
Databaseenum + URL-scheme dispatch + ADR-033risk_phases)Court records are committed at
.autopilot/court/postgres-support/phase-{2,3,4}.md; the full firing history is.autopilot/runs/postgres-support.jsonl.The three defects the pipeline found in its own premise
The feature was believed to already work. Phase 4 found three independent breaks, each alone enough to keep a
postgres://deployment silently on SQLite:Env::prefixed("EMAILIBRIUM_").split("_")mappedEMAILIBRIUM_DATABASE_URLto nesteddatabase.url— a key no field matches, so serde discarded it. The documented one-flag switch was a no-op. Proven with a failing test before the fix./run/secrets/database_urlbecameDATABASE_URL; nothing underbackend/srcreads that.Strongest case FOR merging
Every claim here is machine-verified, not asserted.
Rust Tests (PostgreSQL)runs the real suite against a livepostgres:16-alpineand is guarded so skipped tests cannot pass as green — it asserts against the database that migrations actually ran (printed 26). The smoke workflow runs both legs: SQLite with no profile (asserting no postgres container exists) and PostgreSQL via the profile, each checking the backend's own report of the connection it opened plus a real write. Three courts across two vendors filed 95 charges; zero survived.Strongest case AGAINST merging
0.0.0.0. The docs merged here tell operators how to run PostgreSQL; they do not make the deployment safe to expose.security-baselineis queued and unrun, and should go first.postgres-fulltext-search.wipe_user_dataerrors partway on any migrated database). Not introduced here — found by these courts and recorded rather than absorbed.correctness-debtphase 0 fixes it.Accelerators
ruflo, agentic-qe, qe-court (per-phase), code-review (Tier-3 floor every phase).
Integration court: SKIPPED by explicit maintainer decision — treated as optional for this release. Stated rather than omitted, per the court's own degrade rule. Phases 2, 3 and 4 each had a full two-vendor court; phase 5 is docs-only.
After this merges
Queued lineages, in recommended order:
security-baseline→correctness-debt→mcp-identity-authz→adr-implementation-gaps→db-schema-modernization→mcp-write-parity→postgres-fulltext-search.