Skip to content

Improve knowledge curation grounding and staleness detection - #15

Merged
gering merged 4 commits into
mainfrom
task/improve-curate-grounding
Jun 22, 2026
Merged

Improve knowledge curation grounding and staleness detection#15
gering merged 4 commits into
mainfrom
task/improve-curate-grounding

Conversation

@gering

@gering gering commented Jun 21, 2026

Copy link
Copy Markdown
Owner

Summary

  • Reduce the factual-error / duplication / staleness rate of curated knowledge — systemic follow-up to PR Dogfood knowledge-system: init + curate plugin-design learnings #13, where ~10 content defects slipped into newly curated files.
  • Root cause: /curate stored generated prose without verifying it against sources or deduping against always-loaded surfaces. This PR adds a grounding gate, broader dedup, a staleness net, and a documented link convention.

Changes

  • /curate: grounding gate — every concrete claim (paths, counts, flag/skill names, cascades) must come from a file read this run, not memory; prefer linking mutable specifics over copying them. Dedup extended to always-loaded surfaces (CLAUDE.md, rules, memory). pluginVersion read fresh from plugin.json at write time so it can't go stale mid-session.
  • /backfill-knowledge: agent must produce source-grounded learnings, flag inference vs. fact, and reject unsourced significance (insufficient-context).
  • /reindex: new staleness net (stale-vs-source via git dates, verbatim-restatement flagging); validates link convention (stray [[wikilinks]], cross-layer dangling) and dedups against always-loaded; report + log extended.
  • Link convention decision: markdown links, knowledge↔knowledge only — no wikilinks. Documented in /curate and /reindex.
  • Staleness decision: extend /reindex rather than build a new /audit skill.
  • Bump knowledge-system 1.8.0 → 1.8.1 (plugin.json + marketplace.json).

Readiness

  • ✅ README updated (curate grounding + reindex staleness)
  • ✅ Version bumped 1.8.1 (both files in sync)
  • ✅ Structure check: 0 errors
  • ➖ No changelog / build / lint step in this repo

Test plan

  • python3 scripts/check-structure.py stays green
  • Spot-run /reindex --dry-run on a knowledge base and confirm the new stale/link findings surface in the report
  • Spot-run /curate and confirm it grounds claims + dedups against CLAUDE.md

🤖 Generated with Claude Code

@gering @claude
Improve knowledge curation grounding and staleness detection
0a58931 
Reduce factual-error / duplication / staleness rate of curated knowledge
(systemic follow-up to PR #13 dogfooding defects).

- /curate: add grounding gate (claims must come from a file read this run,
  not memory), prefer linking mutable specifics over copying them
- /curate: extend dedup to always-loaded surfaces (CLAUDE.md, rules, memory)
- /curate: read pluginVersion fresh from plugin.json so it can't go stale
- /curate + /reindex: document link convention (markdown, knowledge<->knowledge,
  no wikilinks)
- /backfill-knowledge: require source-grounded learnings, flag inference vs
  fact, reject unsourced claims
- /reindex: add staleness net (stale-vs-source, verbatim restatement),
  validate link convention and cross-layer dangling, dedup vs always-loaded
- Bump knowledge-system 1.8.0 -> 1.8.1

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gering
gering force-pushed the task/improve-curate-grounding branch from 5a5d247 to 0a58931 Compare June 22, 2026 13:10
gering and others added 3 commits June 22, 2026 15:12
@gering @claude
Fix code-review findings in curation grounding
0aa252b 
- /reindex task C: stop flagging legitimate always-loaded pointers as
  cross-layer defects; only dead links and stray wikilinks are flagged
  (resolves contradiction with /curate's "store a pointer" guidance)
- /reindex scope: clarify the write-only restriction permits reading
  CLAUDE.md / .claude/rules for the new dedup and link-convention checks
- /reindex task F: normalize link-relative source paths to repo-root
  before git lookup; surface unresolved refs instead of dropping silently
- /curate link format: add source-code / pointer carve-out and reference
  /reindex task C as the authoritative convention (de-duplicate)
- /curate dedup: describe the memory index as loaded context, not a path
- README: match the softened cross-reference validation wording

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gering @claude
Address second-pass review findings
3f2f41d 
- /curate: reference reindex's section as "task B" (not "step B") to
  match reindex's "## Tasks" naming and the "task C" reference
- /reindex log template: add unresolved-source count so it is persisted
  in the run log, symmetric with the Final report
- /reindex scope: attribute reads accurately (task E reads CLAUDE.md/rules,
  task F reads source files) instead of bundling all three tasks

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gering @claude
Address third-pass review findings
601221f 
- README roadmap: /audit no longer claims duplicate detection and
  stale-content flagging as future work — both now ship in /reindex
- /curate step 8: cover the no-write branch (already fully covered
  always-loaded) so the report doesn't invent a file path

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gering
gering merged commit dc93f3b into main Jun 22, 2026
1 check passed
@gering
gering deleted the task/improve-curate-grounding branch June 22, 2026 14:24
gering added a commit that referenced this pull request Jul 16, 2026
@gering @claude
Address --max swarm self-review findings
005246f 
Second dogfooding round at the deepest profile (/swarm:review --max --fix:
per-lens split live — gate pruned style, 10 xhigh finders + codex
gpt-5.6-sol@xhigh + grok-4.5 + composer; 24 findings, 10 consensus).
Fixes the agreed findings:

- Never auto-accept an all-untagged consensus cluster: no tagged lens
  backs it, so it is verified like a solo. Verify/auto-accept now derive
  from ONE needsVerify predicate (structural exactly-once partition);
  kind vote in clearer every-form (#1, consensus critical; #21, #22)
- Design verifier sees the finding's recommendation — the proposal the
  applicability rubric actually tests — and carries an escape hatch:
  a genuine defect mis-filed under a design lens is not refuted away
  (#12, #3); "solo" dropped from verifier prompts (#4)
- Untagged findings from multi-lens cluster finders fall back to
  'unspecified' (safe defect bucket), not lenses[0] (#7); merge-agent
  free-text lens validated, majority-member fallback (#9)
- Improvement invitation scoped to design finder units — defect-lens
  finders stay defect-only (#8); merge prompt clusters by issue, not
  only defect (#13); schema descriptions generalized for design
  findings (#14)
- LENS_BRIEF startup assertion (#6); gate prompt interpolates
  LENS_CLUSTERS.design (#19); new test_lens_sync.py guards all lens
  mirrors: SKILL HDR prompt, LENS_BRIEF, pr-post DESIGN_LENSES (#5)
- Workflow assigns stable finding num (defects first, shared sequence);
  presenter/pr-post render it verbatim (#20)
- pr-post.py: design lens is the backup kind signal when the handoff
  drops kind (explicit defect still wins); single-pass partition (#17, #24)
- Balance: REFUTED is its own segment (refuted ⊄ solo since design
  consensus can be refuted) (#2); LOCKED design-table column precedence
  in --loop rounds clarified (#11); "verifies solos" doc sweep across
  manifests/README/knowledge (#4); cluster failure-isolation trade-off
  documented (#10); knowledge index line trimmed (#15)

Declined: #16 (JS test harness for sandbox code; lens-sync test covers
the drift class), #18 (gate-fail under --max runs all lenses — the
documented never-silently-narrower degrade), #23 (micro-opt vs readability).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0126cGxsBYxqgEFH4AcpriNk
gering added a commit that referenced this pull request Jul 17, 2026
@gering @claude
Unjail the grok probe; fold _build_jail back into sandboxed
df700bd 
Swarm loop round 3 — 3 findings, all in the probe. Rather than patch a probe
that had a security jail bolted on (rounds 1-3 all fixed jail-adjacent bugs),
this removes the jail: a readiness check passes no untrusted diff, so it never
needed one — the sibling `codex login status` in ready_check is unjailed too.

- Run `grok models` directly, not through sandboxed(). This roots out the
  round-3 CRITICAL (#18): the jail's `_init_sandbox` builds its deny profile
  with python3, so routing the probe through it made python3 a hard dependency
  of the formerly-local `ready`/`list` paths — and a missing python3 then
  misreported as "grok models failed (rc=65)". No jail, no python3 dep, honest
  reasons.
- Bound with `timeout -k 3` (#17): plain `timeout` only SIGTERMs, so a grok
  that ignores SIGTERM (or forks a stdout-inheriting child) kept the command
  substitution blocking past the deadline — the "must never hang" hole. `-k`
  SIGKILLs after a grace period. rc 124 (SIGTERM at the deadline) reports as
  "timed out"; rc 137 (SIGKILL) reports as "killed" — almost always our own
  `-k`, but an OOM/external kill shares the code, so the message doesn't assert
  a timeout that may not have happened.
- Take only the FIRST id per bullet line (#19): scanning the whole line also
  matched a grok-4.5 mentioned in prose on another model's line
  ("* grok-5 (successor to grok-4.5)"), reporting a retired model as offered.
- Fold _build_jail back into sandboxed(). It existed ONLY to share the jail
  with the probe; with the probe unjailed, the split — and the shared-warning
  (#12) and per-backend-memo (#15) fixes it forced last round — are moot.
  sandboxed() is back to its pre-split shape; _init_sandbox keeps the backend
  key (harmless, strictly more correct).
- Fix a stale swarm-review.js input-contract comment ("subset of the three" →
  codex, grok).
- Docs (header/README/CHANGELOG/knowledge) drop the jail claim. The knowledge
  entry records the real lesson: five rounds of patching a jailed probe ended
  the moment the jail was deleted — a feature added to be safe that generates
  every round's bugs is a shape problem; cut it, don't harden it. The composer
  removal (the PR's subject) drew zero findings across all five rounds.

Verified: live grok ready; SIGTERM-ignoring grok bounded by -k in ~5s, reported
as killed; SIGTERM-obeying grok reported as timed out; model-gone → honest hint,
no warning; probe rc=1 → warn + trust-auth; NO python3 invoked on the ready
path; review path still fully jailed (sandbox-exec + AWS/GH secret stripping);
end-to-end review returns a schema-valid finding.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gering added a commit that referenced this pull request Jul 17, 2026
@gering @claude
Address --max swarm self-review findings
9ff85f3 
Second dogfooding round at the deepest profile (/swarm:review --max --fix:
per-lens split live — gate pruned style, 10 xhigh finders + codex
gpt-5.6-sol@xhigh + grok-4.5 + composer; 24 findings, 10 consensus).
Fixes the agreed findings:

- Never auto-accept an all-untagged consensus cluster: no tagged lens
  backs it, so it is verified like a solo. Verify/auto-accept now derive
  from ONE needsVerify predicate (structural exactly-once partition);
  kind vote in clearer every-form (#1, consensus critical; #21, #22)
- Design verifier sees the finding's recommendation — the proposal the
  applicability rubric actually tests — and carries an escape hatch:
  a genuine defect mis-filed under a design lens is not refuted away
  (#12, #3); "solo" dropped from verifier prompts (#4)
- Untagged findings from multi-lens cluster finders fall back to
  'unspecified' (safe defect bucket), not lenses[0] (#7); merge-agent
  free-text lens validated, majority-member fallback (#9)
- Improvement invitation scoped to design finder units — defect-lens
  finders stay defect-only (#8); merge prompt clusters by issue, not
  only defect (#13); schema descriptions generalized for design
  findings (#14)
- LENS_BRIEF startup assertion (#6); gate prompt interpolates
  LENS_CLUSTERS.design (#19); new test_lens_sync.py guards all lens
  mirrors: SKILL HDR prompt, LENS_BRIEF, pr-post DESIGN_LENSES (#5)
- Workflow assigns stable finding num (defects first, shared sequence);
  presenter/pr-post render it verbatim (#20)
- pr-post.py: design lens is the backup kind signal when the handoff
  drops kind (explicit defect still wins); single-pass partition (#17, #24)
- Balance: REFUTED is its own segment (refuted ⊄ solo since design
  consensus can be refuted) (#2); LOCKED design-table column precedence
  in --loop rounds clarified (#11); "verifies solos" doc sweep across
  manifests/README/knowledge (#4); cluster failure-isolation trade-off
  documented (#10); knowledge index line trimmed (#15)

Declined: #16 (JS test harness for sandbox code; lens-sync test covers
the drift class), #18 (gate-fail under --max runs all lenses — the
documented never-silently-narrower degrade), #23 (micro-opt vs readability).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0126cGxsBYxqgEFH4AcpriNk
gering added a commit that referenced this pull request Jul 17, 2026
@gering @claude
Address --max swarm self-review findings
2be83aa 
Second dogfooding round at the deepest profile (/swarm:review --max --fix:
per-lens split live — gate pruned style, 10 xhigh finders + codex
gpt-5.6-sol@xhigh + grok-4.5 + composer; 24 findings, 10 consensus).
Fixes the agreed findings:

- Never auto-accept an all-untagged consensus cluster: no tagged lens
  backs it, so it is verified like a solo. Verify/auto-accept now derive
  from ONE needsVerify predicate (structural exactly-once partition);
  kind vote in clearer every-form (#1, consensus critical; #21, #22)
- Design verifier sees the finding's recommendation — the proposal the
  applicability rubric actually tests — and carries an escape hatch:
  a genuine defect mis-filed under a design lens is not refuted away
  (#12, #3); "solo" dropped from verifier prompts (#4)
- Untagged findings from multi-lens cluster finders fall back to
  'unspecified' (safe defect bucket), not lenses[0] (#7); merge-agent
  free-text lens validated, majority-member fallback (#9)
- Improvement invitation scoped to design finder units — defect-lens
  finders stay defect-only (#8); merge prompt clusters by issue, not
  only defect (#13); schema descriptions generalized for design
  findings (#14)
- LENS_BRIEF startup assertion (#6); gate prompt interpolates
  LENS_CLUSTERS.design (#19); new test_lens_sync.py guards all lens
  mirrors: SKILL HDR prompt, LENS_BRIEF, pr-post DESIGN_LENSES (#5)
- Workflow assigns stable finding num (defects first, shared sequence);
  presenter/pr-post render it verbatim (#20)
- pr-post.py: design lens is the backup kind signal when the handoff
  drops kind (explicit defect still wins); single-pass partition (#17, #24)
- Balance: REFUTED is its own segment (refuted ⊄ solo since design
  consensus can be refuted) (#2); LOCKED design-table column precedence
  in --loop rounds clarified (#11); "verifies solos" doc sweep across
  manifests/README/knowledge (#4); cluster failure-isolation trade-off
  documented (#10); knowledge index line trimmed (#15)

Declined: #16 (JS test harness for sandbox code; lens-sync test covers
the drift class), #18 (gate-fail under --max runs all lenses — the
documented never-silently-narrower degrade), #23 (micro-opt vs readability).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0126cGxsBYxqgEFH4AcpriNk
gering added a commit that referenced this pull request Jul 25, 2026
@gering @claude
Address swarm review findings on the lane registry
4f29afb 
Apply the agreed findings from a local swarm review (11 ✅ + 1 🟨; the 3 ❌ —
a feature version bump, a README inventory line, and a per-subcommand python3
guard — were deliberately left, see the PR discussion).

herdr-agent.sh:
- Bound herdr list/get/read with a wall-clock timeout (_ha_bounded: timeout →
  gtimeout → perl alarm), honouring the header's "never a hang" promise. wait
  stays governed by its own --timeout. (#6)
- ha_wait now detects the --timeout=MS form too, so a caller's explicit bound in
  either spelling is honoured and no duplicate flag is appended. (#7)
- A missing <target> returns usage code 2, not 4 (server-unreachable), so a
  programmer error is not mistaken for a transient outage. (#12)
- set -u is enabled only on the executed-CLI path, never at source time, so
  sourcing for the prelude/helpers no longer mutates the caller's shell. (#13)
- classify_cwd returns the resolved path as a third tuple element, so a caller
  keying by full path reuses it instead of a second realpath. (#14)

lanes.sh:
- Scrub tab/CR/LF from every TSV cell: agent-derived fields are untrusted, and an
  embedded tab/newline would forge columns/rows. Mirrors herdr-tab-glyph. (#1)
- Guard the agent loop against non-dict (null) elements → never crash, always
  exit 0. (#2)
- flush() now calls the shared classify_cwd instead of re-open-coding the task
  rule, so the classification can't drift between the two consumers. (#15)
- Resolve SCRIPT_DIR via BASH_SOURCE (robust to bare-name invocation). (#5)
- Header wording: exit-0 scope clarified; --json emits [] when no lanes. (#3, #8)

herdr-tab-glyph.sh: consume classify_cwd's new 3-tuple; same BASH_SOURCE fix (#5).
herdr-tab-glyphs.md: point at $HERDR_MATCH_PRELUDE as the shared match SoT. (#10)
Tests extended: null-element + TSV-injection (lanes); --timeout= + missing-target
exit 2 (herdr-agent). Regression: herdr-tab-glyph output byte-identical vs a live
snapshot.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R8crYp6tzRKMiwW63Zt3GG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@gering