feat(selfhost): self-hostable Docker stack — gittensory's full Worker on Node (#979/#980) - #1157
Merged
Conversation
JSONbored
marked this pull request as ready for review
June 24, 2026 03:52
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1157 +/- ##
==========================================
+ Coverage 94.82% 94.92% +0.10%
==========================================
Files 158 173 +15
Lines 19157 19616 +459
Branches 6939 7050 +111
==========================================
+ Hits 18165 18620 +455
Misses 396 396
- Partials 596 600 +4
🚀 New features to boost your workflow:
|
|
Caution 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 🛑 Gittensory review — blocked
🛑 Blocked Review summary Blockers
Nits (5)
CI checks failing
Nits — 1 non-blocking
Review context
Contributor next steps
Signal definitions
Review detailsGenerated from public PR metadata and the diff. Advisory only; deterministic signals remain authoritative. The diff adds a self-hostable Docker stack for the gittensory Worker, enabling operators to run the same review engine on a plain Node container. The changes include new modules for AI providers, database adapters, queue implementations, and a setup wizard. The Dockerfile and docker-compose.yml are configured for easy deployment. The code is well-structured and follows existing conventions, with comprehensive tests and documentation. Blockers
Nits (5)
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
JSONbored
force-pushed
the
feat/self-host-docker
branch
from
June 24, 2026 08:10
b3e8bbb to
7b2a48b
Compare
JSONbored
added a commit
that referenced
this pull request
Jun 24, 2026
…ch coverage Completes the self-host feature parity gaps identified in PR #1157: • MCP Node transport (mcp-server-node.ts): replaces the CF Agents SDK createMcpHandler (Durable-Object-only) with WebStandardStreamableHTTP from the MCP SDK — stateless, per-request, Node 18+ compatible. agents-mcp stub re-exports it so existing MCP routes work unchanged. • pgvector RAG (pg-vectorize.ts): Vectorize adapter backed by pgvector's <=> cosine distance operator. initPgVectorize() issues DDL at startup; PGVECTOR_ENABLED=true env flag gates it (skips on plain Postgres). docker-compose.yml updated to pgvector/pgvector:pg16 image. • Visual review (puppeteer stub): connects to an external Chrome sidecar via BROWSER_WS_ENDPOINT (e.g. browserless/chrome); puppeteer-core is an optional runtime dep installed with INSTALL_VISUAL_REVIEW=true build-arg. Dockerfile wires the optional install; server.ts injects BROWSER binding. • Release workflow hardening: all action SHAs pinned to exact versions. • restart: unless-stopped added to docker-compose gittensory service. • 100% branch coverage on all new src/ files (mcp-server-node.ts, pg-vectorize.ts); stubs/** and server.ts remain Codecov-excluded. All 3709 tests pass; npm audit clean.
- Remove unit-test and typecheck steps (covered by main CI validate job) - Add npm dependency caching via setup-node cache: 'npm' - Add docker/setup-buildx-action + GHA layer caching: node:24-slim layer is served from cache on subsequent runs, avoiding Docker Hub transient failures - Wrap Docker build in a 3-attempt retry loop for cold-cache runs - Add test/integration/selfhost-pg* to path triggers
…te limits Switch FROM node:24-slim to public.ecr.aws/docker/library/node:24-slim in the Dockerfile (both build and runtime stages). ECR Public Gallery mirrors Docker Official Images with no rate limits and no auth, eliminating 503s in CI and operator builds. GHA BuildKit layer cache still applies for fast reruns.
… deploy options Docker Compose profile system (#1199): 9 optional profiles activated via `--profile` — postgres, pgbouncer, redis, ollama, litestream, caddy, observability, tailscale, runners. Operators compose exactly the stack they need; the core service always runs profile-free. Caddy (#1203): HTTPS reverse proxy with auto-TLS via Let's Encrypt; zstd/gzip compression; security headers (HSTS, X-Frame-Options, X-Content-Type-Options, Referrer-Policy). Prometheus + Grafana (#1206): pre-wired observability at --profile observability; 15s scrape interval; pre-provisioned datasource and a Grafana dashboard (queue depth, dead-letter count, HTTP request rate, job throughput by status). Tailscale sidecar (#1204): --profile tailscale exposes the instance on an operator's tailnet without any public firewall rules; TS_AUTHKEY + persistent state volume. Self-hosted GitHub Actions runner (#1205): --profile runners registers a runner against any repo/org; mounts /var/run/docker.sock for in-runner Docker builds. Terraform (Hetzner) (#1209): cx22 VPS + 20 GB volume + firewall; cloud-init installs Docker from the official apt repo; one terraform apply to a ready-to-clone host. Railway template (#1210): railway.json with Dockerfile builder, /health check, ON_FAILURE restart policy — one-click deploy, eligible for Railway Template Marketplace creator revenue. Worker concurrency (#1201): QUEUE_CONCURRENCY env var (default 1) controls how many concurrent pump() loops run per instance; both SQLite and Postgres queues updated; stop()/drain() wait for active === 0. Structured audit log (#1202): logAudit() writes one JSON line per job lifecycle event (job_complete, job_dead, job_error) to stdout — level, ts, job_id, payload_type, latency_ms, attempts, error — captured by Docker's json-file driver with zero operator configuration. Coverage: new selfhost-audit.test.ts + selfhost-pg-queue.test.ts (fills the pre-existing gap referenced by selfhost-pg-vectorize.test.ts); concurrency branch tests in selfhost-sqlite-queue.test.ts; all gates green at 97.06% line / 94.81% branch.
JSONbored
force-pushed
the
feat/self-host-docker
branch
from
June 24, 2026 09:08
7a77b8f to
3ec6239
Compare
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
gittensory-ui | 3ec6239 | Commit Preview URL Branch Preview URL |
Jun 24 2026, 09:10 AM |
- Pin puppeteer-core to 22.13.1 in Dockerfile (was unpinned) - Add CSRF state nonce to /setup → /setup/callback flow: generate a UUID per-visit, embed it in the manifest redirect_url, bind it to the browser via an HttpOnly SameSite=Lax cookie, and validate both match in the callback (prevents manifest code injection) - Remove /var/run/docker.sock mount from runner service; add inline guidance for DinD alternative - Pin runner image from :latest to :ubuntu-22.04 - Require GRAFANA_ADMIN_PASSWORD explicitly (fail-fast :? expansion)
…ostgres readiness wait (#1217) - Redis cache (redis-cache.ts): webhook delivery dedup — marks delivery IDs after a successful response so GitHub retries are handled idempotently; rate limiter now shares the same client connection as the cache - Qdrant adapter (qdrant-vectorize.ts): optional RAG vector backend behind --profile qdrant; deterministic SHA-1 → UUID ID mapping, QDRANT_API_KEY auth, labeled error counters per operation, graceful-degrade on network errors - waitForPostgres: exponential-backoff retry prevents crash loops when gittensory starts before Postgres is ready in compose stacks - Grafana dashboard: full datasource-UID fix, System Health row, Qdrant panels, webhook dedup counter; all counter metrics pre-initialized to 0 at startup
14 tasks
… tests All mock.calls[0] casts now use `as unknown as [...]` to satisfy the strict mock type — was causing typecheck (lint CI job) to fail on the Docker PR.
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
…ost-header redirect attack The setup wizard derived the App manifest origin from `process.env.PUBLIC_API_ORIGIN ?? new URL(request.url).origin`. When PUBLIC_API_ORIGIN is unset, the fallback uses the Host header from the incoming request. An attacker who can craft a request with a spoofed Host header (e.g. `evil.com`) causes the manifest's redirect_url to point at an attacker-controlled domain. GitHub follows that redirect on App creation, allowing the attacker to exchange the one-time code for the App's private key and webhook secret. Fix: remove the request.url fallback entirely. If PUBLIC_API_ORIGIN is not set the wizard now returns 400 with a clear operator message. The origin used to build the manifest is always the operator-configured value, never derived from the request.
This was
linked to
issues
Jun 24, 2026
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runs the same gittensory review engine on a plain Node container so operators can self-host next to their own GitHub App.
docker compose up→ webhooks, the deterministic gate, AI summaries, the maintain/sweep cron, and (optionally) full maintainer autonomy. The Cloudflare Worker (src/index.ts, wrangler) is untouched — this is a parallel Node entry.Foundation — runs the Worker on Node
D1 →
node:sqliteshim (Drizzle + raw + all 56 migrations + atomic batch, byte-identical); Queue → durable queue; cron → timer; Hono via@hono/node-server;*_FILEsecrets. Workers-only deps build-stubbed (visual off,/mcp→501).Tier 0 — reliability
Durable queue (jobs persist; restart re-claims in-flight);
/ready+/metrics(Prometheus); graceful SIGTERM; AI model-id fix (never leaks@cf/…to Ollama/claude/codex); CI Docker build+boot smoke test.Tier 1 — AI power
BYOK fallback chain (
AI_PROVIDER=anthropic,ollama); providers: ollama / openai-compatible / openai / native anthropic / claude-code / codex (subscription CLIs, scrubbed keys, fail-safe); CLI-in-image build arg; local RAG (SQLite vector store +/embeddings→ no Vectorize needed).Tier 2 — scale / HA (validated against real Postgres + Redis)
Postgres backend (
DATABASE_URL) via a tested SQLite→PG dialect translator + async D1 adapter (all 56 migrations apply); Postgres queue withFOR UPDATE SKIP LOCKED(multi-instance-safe); Redis rate limiter (REDIS_URL,x-ratelimit-*headers). Real-infra testing caught 2 bugs (DO-stub fetch signature; int8→string). PG is beta (migrations + exercised paths validated; RAG is SQLite-only).Tier 3 — packaging & onboarding
254 MB image (was 1.33 GB; self-contained bundle); GHCR multi-arch release pipeline (
selfhost-v*tag → amd64+arm64 + provenance/SBOM + Release); Litestream continuous backup; GitHub App Manifest/setupwizard (one-click App creation, gated to first-run).Tier 4 — production-grade compose profiles, observability, and 1-click deploy
Docker Compose profiles (#1199): 9 opt-in profiles via
--profile—postgres,pgbouncer,redis,ollama,litestream,caddy,observability,tailscale,runners. Operators layer exactly the services they need.Caddy reverse proxy (#1203):
--profile caddy→ HTTPS + HTTP/3 with auto-renewing Let's Encrypt certificates; zstd/gzip; HSTS + frame + content-type security headers.Prometheus + Grafana (#1206):
--profile observability→ pre-provisioned datasource + dashboard (queue depth, dead-letter count, HTTP request rate, job throughput). Zero Grafana config needed.Tailscale sidecar (#1204):
--profile tailscale→ instance visible on the operator's tailnet with no public firewall holes; persistent node identity volume.Self-hosted GitHub Actions runner (#1205):
--profile runners→ registers a runner against any repo/org; mounts/var/run/docker.sockfor in-runner Docker builds.Worker concurrency (#1201):
QUEUE_CONCURRENCYenv var (default 1); both SQLite and Postgres queues support N concurrent pump loops;stop()/drain()awaitactive === 0.Structured audit log (#1202):
logAudit()writes one JSON line per job lifecycle event (job_complete / job_dead / job_error) to stdout —level,ts,job_id,payload_type,latency_ms,attempts,error— captured by Docker's json-file driver with zero operator config.Terraform (Hetzner) (#1209):
terraform/provisions a cx22 VPS + 20 GB volume + firewall; cloud-init installs Docker;terraform apply→ SSH into a ready-to-clone host.Railway 1-click template (#1210):
railway.json— Dockerfile builder,/healthcheck, ON_FAILURE restart policy — eligible for Railway Template Marketplace creator revenue.Validated
selfhost-audit.test.ts+selfhost-pg-queue.test.ts+ concurrency branch tests) + a real-Postgres integration test (in CI via a postgres service) + a Docker build+boot smoke test. Typecheck clean, full gate green.Config = samples only
Every
.env.example/ compose / Dockerfile / litestream value is a placeholder; real.envgitignored. Full guide:docs/self-hosting.md.Known limitations (documented)
claude-code/codexsubscription paths are fail-safe + unit-tested but not run against a real authenticated CLI in CI.Advances #979 #980 #981 #982 #1029 #1199 #1201 #1202 #1203 #1204 #1205 #1206 #1209 #1210