-
-
Notifications
You must be signed in to change notification settings - Fork 87
feat(selfhost): self-hostable Docker stack — gittensory's full Worker on Node (#979/#980) #1157
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
0400411
feat(selfhost): D1-over-SQLite adapter — gittensory's data layer runs…
JSONbored 408d43e
feat(selfhost): Node entry — gittensory's full Worker stack boots on …
JSONbored 957ed9b
feat(selfhost): Dockerfile + docker-compose — one-command self-host (…
JSONbored e12eb19
feat(selfhost): AI provider (#979) + self-hosting guide
JSONbored ed94864
feat(selfhost): Tier 0 reliability — durable queue, model-id fix, /re…
JSONbored 1372c01
feat(selfhost): Tier 1 — multi-provider BYOK + fallback chain + nativ…
JSONbored d85917a
feat(selfhost): Tier 1 — local RAG via SQLite vector store + embeddin…
JSONbored 07ff98c
perf(selfhost): self-contained bundle → 254MB image (was 1.33GB) (#980)
JSONbored 9bc1d6a
feat(selfhost): GHCR multi-arch release pipeline (#980)
JSONbored a96f43b
feat(selfhost): optional Litestream continuous SQLite backup (#982)
JSONbored 5f13a40
test(selfhost): raise patch coverage to ~99% + drop dead in-process q…
JSONbored 25dc57d
feat(selfhost): Tier 2 — Postgres backend + Redis rate limiter (multi…
JSONbored 2fe96ab
feat(selfhost): GitHub App Manifest one-click setup wizard (#981)
JSONbored f3562dd
ci(selfhost): fix shellcheck SC2034 (unused loop var) in the smoke-te…
JSONbored 819e350
fix(selfhost): valid codecov.yml (ignores now apply) + branch coverage
JSONbored 6664ddc
test(selfhost): cover remaining branch partials (→ 99% statements)
JSONbored 24afb3e
feat(selfhost): MCP Node port, pgvector RAG, visual review + 100% pat…
JSONbored c903e86
fix(selfhost): address Superagent security findings in workflows + Do…
JSONbored 531db6d
ci(selfhost): optimize build+boot workflow — cache, deduplicate, retry
JSONbored 09ce007
fix(selfhost): pull base image from ECR Public to avoid Docker Hub ra…
JSONbored 3ec6239
feat(selfhost): production-grade compose profiles, observability, and…
JSONbored 5da5d62
fix(selfhost): harden security for Docker self-host configs (#1157)
JSONbored 73e9dbd
feat(selfhost): add Redis webhook dedup cache, Qdrant vector store, P…
JSONbored 2b98edd
Merge branch 'main' into feat/self-host-docker
JSONbored d50d4b3
test(selfhost): fix TS2352 mock.calls cast errors in qdrant-vectorize…
JSONbored 5f0a784
fix(selfhost): require PUBLIC_API_ORIGIN for setup wizard — prevent H…
JSONbored File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Keep the build context lean — deps are installed fresh (npm ci) and the bundle is built in the image. | ||
| node_modules | ||
| **/node_modules | ||
| dist | ||
| dist-ssr | ||
| .output | ||
| .nitro | ||
| .tanstack | ||
| .wrangler | ||
| .playwright-cli | ||
| coverage | ||
| .git | ||
| .claude | ||
| .DS_Store | ||
| *.tsbuildinfo | ||
| # Never ship secrets into the build context | ||
| .env | ||
| .env.* | ||
| .dev.vars | ||
| !.env.example |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| # Self-host image releases (#980). Cutting a `selfhost-v<semver>` tag builds the multi-arch image, pushes it | ||
| # to GHCR with version + latest + sha tags (with provenance + SBOM), and opens a GitHub Release. | ||
| # | ||
| # git tag selfhost-v0.1.0 && git push origin selfhost-v0.1.0 | ||
| # | ||
| # Pull: docker pull ghcr.io/<owner>/gittensory-selfhost:0.1.0 | ||
| name: release-selfhost | ||
|
|
||
| on: | ||
| push: | ||
| tags: | ||
| - "selfhost-v*" | ||
| workflow_dispatch: | ||
| inputs: | ||
| version: | ||
| description: "Version to publish (e.g. 0.1.0)" | ||
| required: true | ||
|
|
||
| permissions: | ||
| contents: write # create the GitHub Release | ||
| packages: write # push to GHCR | ||
|
|
||
| jobs: | ||
| release: | ||
|
JSONbored marked this conversation as resolved.
|
||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 40 | ||
| # Environment gate — requires reviewer approval before a release runs (configure under repo Settings > Environments). | ||
| environment: release | ||
| steps: | ||
| - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
|
JSONbored marked this conversation as resolved.
|
||
| - name: Resolve version | ||
| id: version | ||
| env: | ||
| INPUT_VERSION: ${{ github.event.inputs.version }} | ||
| run: | | ||
| if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then | ||
| echo "v=${INPUT_VERSION}" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "v=${GITHUB_REF_NAME#selfhost-v}" >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| - uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3 | ||
| - uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3 | ||
|
|
||
| - name: Log in to GHCR | ||
| uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3 | ||
| with: | ||
| registry: ghcr.io | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Image metadata | ||
| id: meta | ||
| uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5 | ||
| with: | ||
| images: ghcr.io/${{ github.repository_owner }}/gittensory-selfhost | ||
| tags: | | ||
| type=raw,value=${{ steps.version.outputs.v }} | ||
| type=raw,value=latest | ||
| type=sha,format=short | ||
| labels: | | ||
| org.opencontainers.image.title=gittensory-selfhost | ||
| org.opencontainers.image.description=Self-hostable Gittensory review engine | ||
| org.opencontainers.image.version=${{ steps.version.outputs.v }} | ||
|
|
||
| - name: Build + push (linux/amd64 + linux/arm64) | ||
| uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6 | ||
| with: | ||
| context: . | ||
| platforms: linux/amd64,linux/arm64 | ||
| push: true | ||
|
JSONbored marked this conversation as resolved.
|
||
| tags: ${{ steps.meta.outputs.tags }} | ||
| labels: ${{ steps.meta.outputs.labels }} | ||
| provenance: true | ||
| sbom: true | ||
|
|
||
| - name: GitHub Release | ||
| if: github.event_name == 'push' | ||
| uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2 | ||
| with: | ||
| generate_release_notes: true | ||
| body: | | ||
| Self-host container image: | ||
|
|
||
| ```bash | ||
| docker pull ghcr.io/${{ github.repository_owner }}/gittensory-selfhost:${{ steps.version.outputs.v }} | ||
| ``` | ||
|
|
||
| Multi-arch (linux/amd64 + linux/arm64). See [docs/self-hosting.md](docs/self-hosting.md) for setup. | ||
| To include the Claude Code / Codex subscription CLIs, build locally with | ||
| `--build-arg INSTALL_AI_CLIS=true`. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| # Self-host stack CI (#980/#982). Provides integration coverage the main CI can't: | ||
| # 1. Postgres integration test — needs a real PG service container | ||
| # 2. Self-host bundle build validation (build-selfhost.mjs) | ||
| # 3. Docker image build + container smoke test (/health, /ready, /metrics) | ||
| # Unit tests and typecheck are NOT duplicated here — the main CI validate job covers them. | ||
| name: self-host | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - "src/selfhost/**" | ||
| - "src/server.ts" | ||
| - "scripts/build-selfhost.mjs" | ||
| - "scripts/register-selfhost.mjs" | ||
| - "Dockerfile" | ||
| - "docker-compose.yml" | ||
| - "migrations/**" | ||
| - "test/unit/selfhost-*" | ||
| - "test/integration/selfhost-pg*" | ||
| - ".github/workflows/selfhost.yml" | ||
| pull_request: | ||
| paths: | ||
| - "src/selfhost/**" | ||
| - "src/server.ts" | ||
| - "scripts/build-selfhost.mjs" | ||
| - "scripts/register-selfhost.mjs" | ||
| - "Dockerfile" | ||
| - "docker-compose.yml" | ||
| - "migrations/**" | ||
| - "test/unit/selfhost-*" | ||
| - "test/integration/selfhost-pg*" | ||
| - ".github/workflows/selfhost.yml" | ||
|
|
||
| # Least privilege — the smoke test only reads the repo; no writes, no packages. | ||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| build-boot: | ||
| name: build + boot smoke test | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 20 | ||
|
JSONbored marked this conversation as resolved.
JSONbored marked this conversation as resolved.
|
||
| services: | ||
| postgres: | ||
| image: postgres:16-alpine | ||
| env: | ||
| POSTGRES_PASSWORD: devpw | ||
| POSTGRES_DB: gittensory | ||
| ports: | ||
| - 5432:5432 | ||
| options: >- | ||
| --health-cmd "pg_isready -U postgres" --health-interval 5s --health-timeout 5s --health-retries 10 | ||
| steps: | ||
| - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6 | ||
| with: | ||
| node-version: "24" | ||
| cache: "npm" | ||
|
|
||
| - name: Install deps | ||
| run: npm ci --ignore-scripts | ||
|
|
||
| - name: Postgres integration test (real PG) | ||
| run: PG_TEST_URL=postgres://postgres:devpw@localhost:5432/gittensory npx vitest run test/integration/selfhost-pg.test.ts | ||
|
|
||
| - name: Build the self-host bundle | ||
| run: node scripts/build-selfhost.mjs | ||
|
|
||
| - uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3 | ||
|
|
||
| - name: Build the Docker image | ||
| run: | | ||
| docker buildx build \ | ||
| --cache-from type=gha \ | ||
| --cache-to type=gha,mode=max \ | ||
| --load \ | ||
| -t gittensory:selfhost-ci . | ||
|
|
||
| - name: Boot the container + smoke-test /health, /ready, /metrics, migrations | ||
| run: | | ||
| docker run -d --name gt -p 8787:8787 gittensory:selfhost-ci | ||
| ok=0 | ||
| for _ in $(seq 1 30); do | ||
| if curl -sf http://127.0.0.1:8787/health >/dev/null; then ok=1; break; fi | ||
| sleep 2 | ||
| done | ||
| if [ "$ok" != "1" ]; then echo "::error::container did not become healthy"; docker logs gt; exit 1; fi | ||
| curl -sf http://127.0.0.1:8787/health | grep -q '"status":"ok"' | ||
| curl -sf http://127.0.0.1:8787/ready | grep -q '"ok":true' | ||
| curl -sf http://127.0.0.1:8787/metrics | grep -q 'gittensory_uptime_seconds' | ||
| docker logs gt 2>&1 | grep -q 'selfhost_migrations_applied' | ||
| echo "self-host smoke test passed" | ||
| docker rm -f gt | ||
|
github-advanced-security[bot] marked this conversation as resolved.
Fixed
github-advanced-security[bot] marked this conversation as resolved.
Fixed
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,6 +9,9 @@ dist-ssr/ | |
| .playwright-cli/ | ||
| output/ | ||
| .dev.vars | ||
| .env | ||
| .env.* | ||
| !.env.example | ||
| *.local | ||
| .DS_Store | ||
| coverage/ | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| # Self-host image for gittensory-api (#980). Runs the SAME Worker handlers on Node via src/server.ts — | ||
| # the Cloudflare bindings become self-host adapters (D1 -> node:sqlite, Queue -> in-process). The hosted | ||
| # Cloudflare Worker (wrangler) deploy is unaffected. SECRETS ARE NEVER BAKED: supply them at run time via | ||
| # the .env file or mounted *_FILE secrets (see docker-compose.yml + .env.example). | ||
|
|
||
| # --- build: install deps + bundle the Node entry -------------------------------------------------------- | ||
| # ECR Public Gallery mirrors Docker Official Images with no rate limits and no auth. | ||
| FROM public.ecr.aws/docker/library/node:24-slim AS build | ||
| WORKDIR /app | ||
| COPY package*.json ./ | ||
| # --ignore-scripts: no native builds are needed (SQLite is the built-in node:sqlite; @hono/node-server is | ||
| # pure JS; esbuild ships its binary as an optional dependency, not a script). | ||
| RUN npm ci --ignore-scripts | ||
| COPY . . | ||
| # --all: bundle every dependency into one self-contained dist/server.mjs, so the runtime image needs no | ||
| # node_modules (≈10× smaller). The bundle has zero `cloudflare:*` imports (stubbed at build), so no loader. | ||
| RUN node scripts/build-selfhost.mjs --all | ||
|
|
||
| # --- runtime: slim, non-root ---------------------------------------------------------------------------- | ||
| FROM public.ecr.aws/docker/library/node:24-slim AS runtime | ||
| WORKDIR /app | ||
| ENV NODE_ENV=production \ | ||
| PLATFORM=self-hosted \ | ||
| PORT=8787 \ | ||
| DATABASE_PATH=/data/gittensory.sqlite \ | ||
| MIGRATIONS_DIR=/app/migrations | ||
| COPY --from=build /app/dist ./dist | ||
| COPY --from=build /app/migrations ./migrations | ||
| # Optional: bake the Claude Code / Codex CLIs so the `claude-code` / `codex` subscription providers (#979) | ||
| # work in-image. Build with `--build-arg INSTALL_AI_CLIS=true`. No credentials are baked — operators mint | ||
| # CLAUDE_CODE_OAUTH_TOKEN (`claude setup-token`) / codex auth at run time and pass it via the env. | ||
| ARG INSTALL_AI_CLIS=false | ||
| RUN if [ "$INSTALL_AI_CLIS" = "true" ]; then npm install -g @anthropic-ai/claude-code@2.1.187 @openai/codex@0.142.0 --ignore-scripts; fi | ||
| # Optional: enable visual review via an external Chrome sidecar (e.g. `browserless/chrome:latest`). | ||
|
JSONbored marked this conversation as resolved.
|
||
| # Build with `--build-arg INSTALL_VISUAL_REVIEW=true` then set BROWSER_WS_ENDPOINT=<ws-url> at runtime. | ||
| ARG INSTALL_VISUAL_REVIEW=false | ||
| COPY --from=build /app/package*.json ./ | ||
| RUN if [ "$INSTALL_VISUAL_REVIEW" = "true" ]; then npm install puppeteer-core@22.13.1 --ignore-scripts; fi | ||
| # Data dir (the SQLite file) — owned by the unprivileged node user; mount a volume here to persist. | ||
| RUN mkdir -p /data && chown -R node:node /data /app | ||
| USER node | ||
| EXPOSE 8787 | ||
| HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \ | ||
| CMD node -e "fetch('http://127.0.0.1:'+(process.env.PORT||8787)+'/health').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))" | ||
| CMD ["node", "dist/server.mjs"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| # Caddy reverse proxy for gittensory (#980 self-host). | ||
| # Activated via: docker compose --profile caddy up | ||
| # | ||
| # DOMAIN is injected from the DOMAIN env var in docker-compose.yml. | ||
| # Set DOMAIN=reviews.yourcompany.com in .env — Caddy fetches a TLS cert from Let's Encrypt automatically. | ||
| # For local testing without a domain, set DOMAIN=localhost (self-signed cert, browser will warn). | ||
| # | ||
| # When using this profile, remove the `ports:` entry from the gittensory service in docker-compose.yml | ||
| # so port 8787 is NOT exposed publicly — all traffic should flow through Caddy on 443. | ||
|
|
||
| {$DOMAIN} { | ||
| reverse_proxy gittensory:8787 { | ||
| # Surface the real client IP to the app (logged in access events). | ||
| header_up X-Forwarded-For {remote_host} | ||
| header_up X-Real-IP {remote_host} | ||
| } | ||
|
|
||
| # Compress responses. | ||
| encode zstd gzip | ||
|
|
||
| # Security headers. | ||
| header { | ||
| Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" | ||
| X-Content-Type-Options "nosniff" | ||
| X-Frame-Options "DENY" | ||
| Referrer-Policy "strict-origin-when-cross-origin" | ||
| -Server | ||
| } | ||
|
|
||
| log { | ||
| output stderr | ||
| format json | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.