Skip to content

feat(ai): wire optional deterministic-summary rewrite layer into public output - #204

Merged
JSONbored merged 1 commit into
JSONbored:mainfrom
Khaostica:feat/ai-deterministic-summary-rewrite
Jun 1, 2026
Merged

feat(ai): wire optional deterministic-summary rewrite layer into public output#204
JSONbored merged 1 commit into
JSONbored:mainfrom
Khaostica:feat/ai-deterministic-summary-rewrite

Conversation

@Khaostica

Copy link
Copy Markdown
Contributor

Summary

Completes #151 by turning the optional AI summary scaffold (added in #14) into a complete, reusable, public-safe deterministic-summary rewrite layer, and wiring it into the public GitHub PR intelligence comment.

Optional AI now rewrites deterministic signal bundles into clearer prose — it never decides facts. Deterministic templates stay authoritative, and the layer safely falls back whenever AI is disabled, unavailable, over quota, or produces unsafe output.

What changed

  • src/services/ai-summaries.ts
    • Added rewriteSignalBundleWithAi() — a generic rewrite layer whose returned text is always safe to use: disabled, unavailable, quota_exceeded, unsafe, and error paths all return the caller-supplied deterministic template.
    • Routed every public AI output through the canonical public/private sanitizer (sanitizePublicComment / FORBIDDEN_PUBLIC_COMMENT_WORDS) instead of the previous bespoke regex, keeping a stricter local net as defense in depth.
    • Added rewritePublicPrIntelligenceComment() — public-surface wrapper that preserves the sticky-comment marker and returns the deterministic body on any non-ok outcome.
  • src/signals/engine.ts
    • Added buildPublicCommentSignalBundle() — a pure, source-free compact signal bundle (counts, levels, booleans, role context, public finding titles only). It deliberately excludes PR title/body, diffs, and finding detail so source contents can never reach the AI provider.
  • src/queue/processors.ts
    • Wired the optional rewrite into the GitHub App public PR comment path: build the deterministic body, then optionally rewrite it, then post. Disabled-by-default behavior is unchanged.

Issue linkage

Closes #151.

Validation

Run from repo root:

  • npm run typecheck — passes.
  • npx vitest run test/unit/ai-summaries.test.ts test/unit/signals.test.ts — 48 passed (new rewrite-layer + bundle-builder tests).
  • npx vitest run test/unit — 558 passed. The only failures are in test/unit/mcp-cli.test.ts, which fails with the Windows-only libuv Assertion failed: !(handle->flags & UV_HANDLE_CLOSING) when spawning the MCP CLI child process; this is environment-only and reproduces on the unmodified base branch. The full npm run test:ci coverage gate runs on CI.

New behavior is covered by: rewrite-layer fallbacks (disabled / public-disabled / unavailable / quota / error / empty), an unsafe-output fallback for every forbidden public term, a no-source-contents prompt invariant, a source-free + forbidden-language bundle invariant, and sticky-marker preservation on both fallback and ok. test/unit/queue.test.ts exercises the wired comment path with AI disabled, proving the deterministic fallback is posted unchanged.

Contract notes

No API / OpenAPI / MCP schema changes. The PR intelligence comment marker (<!-- gittensory-pr-intelligence -->) is preserved so sticky-comment updates keep working.

Operational details

No new configuration. Reuses the existing, already-false-by-default flags: AI_SUMMARIES_ENABLED, AI_PUBLIC_COMMENTS_ENABLED, WORKERS_AI_SUMMARY_MODEL, AI_DAILY_NEURON_BUDGET, AI_MAX_OUTPUT_TOKENS, and the AI Workers binding. Public rewrites require both AI_SUMMARIES_ENABLED and AI_PUBLIC_COMMENTS_ENABLED.

Security / privacy assessment

  • AI stays disabled by default; public rewrites need an explicit second opt-in.
  • No source contents sent — only a compact, source-free deterministic bundle.
  • Quota-limited via the existing daily neuron budget; over-budget falls back to the template.
  • All public AI output runs through the canonical public/private sanitizer; unsafe output falls back to the deterministic body.
  • Public output is tested against the full forbidden-language list (wallet, hotkey, raw trust score, payout, reward estimate, farming, private reviewability, public score estimate); no wallets/hotkeys/scores/payouts/farming language can be emitted.
  • Backend-only change; no changelog edits (handled at release prep).

@Khaostica
Khaostica requested a review from JSONbored as a code owner June 1, 2026 20:02
@dosubot dosubot Bot added the size:L label Jun 1, 2026
@Khaostica
Khaostica force-pushed the feat/ai-deterministic-summary-rewrite branch from 62093a4 to 4c537ac Compare June 1, 2026 20:04
@dosubot

dosubot Bot commented Jun 1, 2026

Copy link
Copy Markdown

Related Knowledge

1 document with suggested updates is ready for review.

gittensory

Gittensory Architecture
View Suggested Changes
@@ -360,6 +360,29 @@
 
 Gittensory includes an optional AI layer using [Cloudflare Workers AI](https://github.com/JSONbored/gittensory/blob/40c019c82d87568c232fbe7fabf50212e983075a/wrangler.jsonc#L47-L49) with the [@cf/meta/llama-3.1-8b-instruct-fp8-fast](https://github.com/JSONbored/gittensory/blob/40c019c82d87568c232fbe7fabf50212e983075a/wrangler.jsonc#L36) model. Both [AI summaries and public comments are disabled by default](https://github.com/JSONbored/gittensory/blob/40c019c82d87568c232fbe7fabf50212e983075a/wrangler.jsonc#L34-L35), preserving the system's deterministic core while offering opt-in AI-assisted features.
 
+#### Deterministic-Summary Rewrite Layer
+
+The optional AI layer implements a rewrite pattern that clarifies deterministic signal bundles into friendlier prose when enabled. This approach maintains the deterministic-first philosophy: AI never decides facts—it only restates compact structured signals that the system has already computed.
+
+**Core functions:**
+
+- `rewriteSignalBundleWithAi()` — Generic rewrite function accepting signal bundles and fallback text. The returned `text` is always safe to use: disabled, unavailable, quota-exceeded, unsafe, and error paths return the caller-supplied deterministic template.
+- `rewritePublicPrIntelligenceComment()` — Public-surface wrapper used by the GitHub App PR intelligence comment. Preserves the sticky-comment marker and guarantees the deterministic body is posted whenever AI produces unsafe output or is unavailable.
+- `buildPublicCommentSignalBundle()` — Builds compact, source-free signal bundles (counts, levels, booleans, role context, public finding titles) that deliberately exclude PR titles, bodies, diffs, and source contents to prevent repository source from reaching the AI provider.
+
+**Safety guarantees:**
+
+- Deterministic templates remain authoritative; AI rewrites are optional enhancements
+- Falls back to templates when AI is disabled, unavailable, over quota, or produces unsafe output
+- All public AI output routes through the canonical public/private sanitizer (`sanitizePublicComment`)
+- Signal bundles carry only public-safe structured data—no source code, diff contents, or sensitive scoring context
+- Uses existing `AI_SUMMARIES_ENABLED` and `AI_PUBLIC_COMMENTS_ENABLED` flags (both default to false)
+- Quota-limited via the existing `AI_DAILY_NEURON_BUDGET` configuration
+
+**Integration:**
+
+Wired into the GitHub App's public PR intelligence comment workflow in `src/queue/processors.ts`. The deterministic comment is built first, then optionally rewritten by passing a source-free signal bundle to the rewrite layer. On any non-ok outcome, the unchanged deterministic body is posted to GitHub.
+
 ### Frontend Applications
 
 **Web UI:** The [gittensory-ui](https://github.com/JSONbored/gittensory/blob/40c019c82d87568c232fbe7fabf50212e983075a/apps/gittensory-ui) application uses TypeScript and Vite for build tooling and deploys as a Cloudflare Worker at gittensory.aethereal.dev.

✅ Accepted

How did I do? Any feedback?  Join Discord

@Khaostica
Khaostica force-pushed the feat/ai-deterministic-summary-rewrite branch 4 times, most recently from 9aee67e to b3eae9d Compare June 1, 2026 23:22
…ic output

Completes JSONbored#151 by turning the existing AI summary scaffold into a complete,
reusable, public-safe rewrite layer and wiring it into the public PR
intelligence comment.

- Add rewriteSignalBundleWithAi(): a generic rewrite layer whose returned text
  is always safe to use - disabled, unavailable, quota-exceeded, unsafe, and
  error paths all fall back to the caller-supplied deterministic template.
- Route every public AI output through the canonical public/private sanitizer
  (sanitizePublicComment / FORBIDDEN_PUBLIC_COMMENT_WORDS) instead of a bespoke
  regex, with a stricter local net as defense in depth.
- Add buildPublicCommentSignalBundle(): a pure, source-free compact signal
  bundle (counts, levels, booleans, role, finding titles only) - never PR
  title/body/diff or finding detail.
- Add rewritePublicPrIntelligenceComment() and wire it into the GitHub App PR
  comment path; preserves the sticky-comment marker and posts the deterministic
  body on any non-ok outcome.
- Keep AI disabled by default and quota-limited (unchanged env flags).
- Tests: rewrite-layer fallbacks (disabled/public-disabled/unavailable/quota/
  error/empty), an unsafe-output fallback per forbidden public term, a
  no-source-contents prompt invariant, a source-free/forbidden-language bundle
  invariant, and sticky-marker preservation on fallback and ok.
@Khaostica
Khaostica force-pushed the feat/ai-deterministic-summary-rewrite branch from b3eae9d to 84f8c67 Compare June 1, 2026 23:26

@JSONbored JSONbored left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Khaostica this looks merge-ready from this pass.

A few notes:

  • The deterministic comment body remains the source of truth, with AI rewrite only used when explicitly enabled.
  • The fallback paths are covered for disabled config, public-comment disabled config, quota/error/unavailable responses, and unsafe output.
  • The prompt bundle avoids source/diff content, and the sticky marker is preserved.

No code changes requested.

@dosubot dosubot Bot added the lgtm label Jun 1, 2026
@JSONbored
JSONbored merged commit 5be78a9 into JSONbored:main Jun 1, 2026
3 checks passed
@JSONbored JSONbored added gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. and removed feature labels Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(ai): add optional deterministic-summary rewrite layer

2 participants