Skip to content

Add tab and approval flow for server -> client sampling - #28

Merged
jspahrsummers merged 4 commits into
mainfrom
justin/sampling
Oct 28, 2024
Merged

Add tab and approval flow for server -> client sampling#28
jspahrsummers merged 4 commits into
mainfrom
justin/sampling

Conversation

@jspahrsummers

Copy link
Copy Markdown
Member
CleanShot.2024-10-25.at.14.48.41.mp4

@jspahrsummers

Copy link
Copy Markdown
Member Author

@dsp-ant @ashwin-ant 🙏

@jspahrsummers
jspahrsummers merged commit a5606f7 into main Oct 28, 2024
@jspahrsummers
jspahrsummers deleted the justin/sampling branch October 28, 2024 15:24
cliffhall added a commit that referenced this pull request Jul 1, 2026
…d option IDs

Adding the new status option regenerated every Status option ID on board
#28, so update the whole option-ID table (not just the new row) plus the
inline references to the Done and In progress option IDs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cb5xVMkUUgpxwBsmZSQyRm
cliffhall added a commit that referenced this pull request Jul 1, 2026
…d V1.5 board (#1555)

* docs(AGENTS): document board membership + status workflow; drop closed V1.5 board (closes #1554)

The Issue-driven Work Style section described board interaction in prose
but only gave concrete gh commands for the version label, so agents
labeled issues/PRs without adding them to board #28 or setting a Status.

- Add explicit gh recipes for `project item-add` + `project item-edit`,
  including the V2 board's project/field/Status-option IDs.
- Correct Status names to the real board values (In progress / In review,
  plus Backlog / Building / Done).
- State that a v2 item isn't "created" until labeled AND on the board
  with a Status set.
- Remove the closed V1.5 board (project 39) and its base-branch/label
  references.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(AGENTS): require real issues (no drafts) and assign issues to their creator

Per maintainer guidance: every board item must be an actual GitHub issue,
not a draft card, and each issue must be assigned to the user it is created
for. Replaces the draft-items-vs-issues bullet.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(AGENTS): clarify PR-to-issue linking limits on non-default branch

Closing keywords (Closes #N) only auto-link and auto-close for PRs that
target the repo's default branch. v2 PRs target v2/main, so the keyword is
only a cross-reference there. Document that:
- the PR body's first line must still be `Closes #<ISSUE_NUMBER>`,
- there is no gh flag to manually link (no `gh pr edit --add-issue`),
- on merge, the issue must be closed manually and its board item moved to
  Done, since auto-close won't fire on v2/main.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(AGENTS): drop V1.5 board mention; annotate v1 board scope

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(AGENTS): drop stale "v1.5-ported" qualifier from integration-test notes

v1.5/main is fully merged into v2/main, so the integration suite is just
the v2 integration suite — the provenance tag no longer distinguishes
anything.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(AGENTS): mark v1.5/main as a frozen reference branch (do not delete)

v1.5/main is merged and takes no new work, but the branch is kept as a
reference point (e.g. to track down a regression introduced by the merge
into v2/main). Note that it should not be deleted.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(AGENTS): add MCP Apps Extension status; refresh regenerated board option IDs

Adding the new status option regenerated every Status option ID on board
#28, so update the whole option-ID table (not just the new row) plus the
inline references to the Done and In progress option IDs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cb5xVMkUUgpxwBsmZSQyRm

* docs(AGENTS): use gh --jq in board one-liner; tie building statuses to workflow

Address PR review: drop the python3 pipe in favor of gh's built-in --jq
flag (no python3-on-PATH assumption), and add a one-line pointer mapping
the Building / MCP Apps Extension statuses to the workflow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cb5xVMkUUgpxwBsmZSQyRm

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cliffhall added a commit that referenced this pull request Jul 1, 2026
* docs(AGENTS): board tracks issues only, never PRs

Reconcile a contradiction in the board workflow: line 109 said every board
item is a real issue, but the "created" callout and the add-to-board bullet
said "issue or PR." Make it consistent — only issues go on board #28; a PR
still gets the v2 label but is tracked through its linked issue's card via
Closes #N.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cb5xVMkUUgpxwBsmZSQyRm

* docs(AGENTS): use <issue-url> placeholder in board one-liner recipe

Address PR review nit: make the one-liner's placeholder <issue-url> to
match the two-step recipe, reinforcing that only issues go on the board.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cb5xVMkUUgpxwBsmZSQyRm

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cliffhall added a commit that referenced this pull request Aug 1, 2026
- Align the v1 maintenance policy on "security and bug fixes only" in
  AGENTS.md and CONTRIBUTORS.md; both said "security fixes only" while the
  README callout, the branch table, and the board description said
  "security and bug fixes".
- Drop the stale "or V2 Go Live" instruction from the work-begins step —
  that column no longer exists on board #28 (verified via
  `gh project field-list 28`), so the option id it implied would be
  rejected. Restore the removed-column id list the same edit dropped, now
  including V2 Go Live, and re-date the table to today's verification.
- Fix "contains the the new version" typo.
- Strip a stray trailing blank line at the end of AGENTS.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU
cliffhall added a commit that referenced this pull request Aug 1, 2026
…review instructions (#1866)

* docs: correct post-swap branch model, PR policy, and stale gate/tree entries

Part of the go-live phase-8 docs hygiene: everything that silently inverted
when `main` became v2.

Branch/release model — documented canonically in AGENTS.md and corrected
everywhere it was restated:
- AGENTS.md: replace the one-line "Base Branches" bullet with a table giving
  each branch its role (v2/main = develop, main = release, v1/main =
  maintenance), whether PRs target it, and which npm dist-tag it publishes.
- README.md: the "Repo status" callout had it backwards, claiming `main` was
  the legacy v1 implementation. Also note that a v2 release is cut from `main`
  after the milestone merge, not from v2/main.
- CONTRIBUTORS.md: the version table listed v2's base branch as `main`.

PR policy:
- Every PR must reference an issue, from anyone, no exceptions.
- PRs are opened by repo maintainers only — org write access is not
  authorization. Everyone else files a detailed issue.
- Branch names start with the target version segment (v2/…, v1/…).
- Issues and PRs carry exactly one of `v1`/`v2` at creation; default `v2`.
- Screenshots proving UI/TUI changes go in the gitignored pr-screenshots/.

Corrections to stale content found while in there:
- The TUI coverage bullet still described the Ink/App.tsx/hooks exclusion as
  interim pending #1501, which closed 2026-06-29; the gate now covers all of
  src/** with only tui-servers.ts excluded.
- The completion checklist said `npm run validate` was the gate and covered
  "e2e tests", contradicting the mandatory-pre-push-gate section (npm run ci).
- v1 PR base was given as `main`; it is `v1/main`.
- Project tree omitted core/client, docs/, and scripts/ (README: core/client,
  docs/). core/client is under the coverage gate and imported by both App.tsx
  trees.

Also adds the launcher and `npm run test:scripts` to the "run the tests" list
and a tiers overview, since neither was mentioned.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU

* docs: require a milestone on every new issue, defaulting to the current one

Adds the rule alongside the version-label rule, since they fail the same way:
an unmilestoned issue drops out of release planning as silently as an
unlabeled one drops out of version filtering.

- Set it at create time (`gh issue create --milestone <title> ...`).
- If the user didn't specify one, default to the current milestone — the open
  milestone with the nearest due date — rather than leaving it blank pending a
  decision. Includes the `gh api .../milestones` one-liner that identifies it.
- Milestones are release buckets, so pick by when the work ships, not by size;
  sub-issues normally inherit their parent's milestone.

Also widens the section's opening callout: an issue is not "created" until it
is labeled, milestoned, AND boarded with a Status — four distinct steps, not
three.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU

* docs: add .github/copilot-instructions.md and require mirroring into it

GitHub Copilot reads .github/copilot-instructions.md when reviewing a PR. The
repo had no such file, so Copilot reviewed without any of the conventions that
actually govern this codebase — the ban on `any` and on unjustified double
casts, the Mantine-first styling rules, the `.withProps()` extraction rule, the
lib/utils split, test placement, and the per-file >=90% four-dimension gate.

The new file is a review-focused distillation of AGENTS.md: the rules a
reviewer would cite against a diff, plus a short "what to prioritize" list that
puts correctness and security first (this backend spawns processes and proxies
outbound requests). Deliberately omitted: board recipes and IDs, milestone and
branch mechanics, release procedure, and the project tree — no reviewer cites
those, and copying them would double the maintenance surface.

AGENTS.md's "Keep documentation files up to date" section now requires
mirroring review-relevant changes into it in the same PR, states that AGENTS.md
stays the source of truth, and spells out what counts as review-relevant. There
is no generation step and nothing detects drift, so the rule is the only thing
keeping the two in sync.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU

* docs: address Copilot review — v1 policy wording, stale V2 Go Live, typo

- Align the v1 maintenance policy on "security and bug fixes only" in
  AGENTS.md and CONTRIBUTORS.md; both said "security fixes only" while the
  README callout, the branch table, and the board description said
  "security and bug fixes".
- Drop the stale "or V2 Go Live" instruction from the work-begins step —
  that column no longer exists on board #28 (verified via
  `gh project field-list 28`), so the option id it implied would be
  rejected. Restore the removed-column id list the same edit dropped, now
  including V2 Go Live, and re-date the table to today's verification.
- Fix "contains the the new version" typo.
- Strip a stray trailing blank line at the end of AGENTS.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU

* docs: restore `inspector/` as the Project Structure tree root

The tree is a filesystem layout — every child is a real directory
(`clients/`, `core/`, `test-servers/`). Labeling its root `v2/main/` named
a branch instead, which reads as a path that does not exist in a checkout
and diverged from the identical tree in README.md, which still says
`inspector/`. The branch model is documented in its own table under
Repository & Project Boards; the tree does not need to restate it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU

* docs: settle the v1 policy wording as "security fixes only"

Reverses the direction the Copilot round took in 33cc036. That commit
normalized the six v1-policy sites onto "security and bug fixes" because that
was the majority wording — but majority is the wrong tiebreak here. The policy
is set by what we told contributors: the #1819 backlog close note, posted to
120 closed PRs, says "v1 will receive security fixes only", and the five PRs
retained for 1.0.5 are all security fixes.

So the two sites that already said "security fixes only" were the correct ones,
and the four that said otherwise are now aligned to them:

- AGENTS.md: the `v1` label description, the branch-role table, and the v1
  board line.
- README.md: the "Repo status" callout.

CONTRIBUTORS.md and AGENTS.md's Project Status bullet already read correctly
and are unchanged.

This is the substance of #1813, which is why that issue's PR is now reduced to
its PR-template-link half — the wording work landed here to avoid two PRs
editing the same lines in opposite directions.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU

* docs: point the AGENTS.md policy link at CONTRIBUTING.md

This PR rewrites the paragraph holding the only in-repo link to the
contributing policy, so it owns that line. #1884 renames the file
CONTRIBUTORS.md -> CONTRIBUTING.md; having #1884 also edit this line would
guarantee a one-line conflict between the two.

Writing the new filename here instead makes #1884 a pure `git mv` with no
overlapping line, so neither PR conflicts with the other.

Between this merging and #1884 merging, the link points at a filename that
doesn't exist yet. That window is deliberate and bounded — this PR is first in
the merge order (#1821) and #1884 is third.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
cliffhall added a commit that referenced this pull request Aug 1, 2026
…ce (#1891)

The Done Status option was deleted from board #28, blanking the Status of
all 247 cards that held it. Restored from a pre-change item-list snapshot;
verified all 247 are Done again and no other card was disturbed.

A recreated option never regains its old id, so Done is now 259d6aab rather
than 248a3910 — updated in the option table and the merge step.

The hazard callout recommended the web UI as the safe path, which is what
made this look safe. It is not: deleting an option blanks its cards in the
UI exactly as it does via the API, with no undo. Corrects that, adds a
snapshot-first rule, and documents the recovery recipe used here — including
the grouping check that proves the orphaned set is exactly the deleted
option's cards before re-applying.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU
cliffhall added a commit that referenced this pull request Aug 1, 2026
Board #11 had the same gap as #28 — no state for "filed but not yet
triaged" — so a v1 security-fix issue had nowhere to sit that didn't
assert maintainer approval. Adds Incoming there too.

AGENTS.md carried no v1 board recipes at all, only a link, so anyone
boarding a v1 issue had to discover the project and field ids by hand
and was liable to reach for #28's (which are rejected, but only after
the attempt). Adds a V1 board section with its ids, states that Incoming
is the default status for new items on both boards, and notes that
Priority is v2-only — #11 has no such field.

The Incoming option was added with all four existing option ids echoed
back, and a before/after item-list diff confirms all 74 cards kept their
Status. Three cards on #11 carry no Status; that predates this change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU
cliffhall added a commit that referenced this pull request Aug 29, 2026
- skill-manifest: the closing `---` fence must occupy a whole line. `---oops`
  was accepted as a terminator, silently truncating the frontmatter and passing
  validation — the exact failure the guard exists to catch. Regression cases
  added (prefix, trailing whitespace, EOF, CRLF).
- skill-eval: reject a nonzero `claude -p` exit instead of resolving it as an
  empty observation. An auth error or rate limit was scoring as "no skill
  invoked", which passes every negative case and reads as a trigger miss on
  every positive one.
- skill-eval: extract `collectSkillInvocations` / `sampleHit` behind a `main()`
  guard and cover them in `skill-eval.test.mjs` — the stream parsing and the
  process-outcome handling are unreachable from an eval run's happy path.
- docs/test-servers: annotate the era on every catalogue row. Six legacy rows
  carried no annotation under a "unless noted, Modern" default, so the doc sent
  readers to the wrong era for the feature they were trying to see.
- docs/quality-gate: `validate` runs four guards, not three; add the
  `verify:skills` row and the two new `test:scripts` suites; fix the `#publishing`
  fragment left dangling by the split.
- issue-create / pr-flow: delegate the card steps to `/board-ops` rather than
  restating its option IDs, which are explicitly unstable.
- issue-triage: the audit counted presence, not cardinality — an issue carrying
  both `v1` and `v2` passed a check that claims "exactly one". Same for the five
  type labels. Also scope the two milestone checks to board #28.
- AGENTS.md: codify the v1 milestone exception. Every milestone is a v2 release
  bucket, so a v1 issue has none to take and the Incoming-milestone invariant
  cannot apply to board #11.
- clients/launcher/README: its pointer promised an explanation of why the cli/tui
  manifests ship, which the extracted guide does not make.

Signed-off-by: cliffhall <cliff@futurescale.com>
cliffhall added a commit that referenced this pull request Sep 1, 2026
Fixes an unquoted `#` in `board-ops`'s description. YAML reads ` #28`
onward as a comment, so the parser — and Claude Code, which parses the
same way — only ever saw the description up to `Covers board`. Both
board numbers and the option-deletion hazard were absent from the
listing that decides whether the skill fires, and every check stayed
green because they look for an *empty* description and a truncated one
is still non-empty. Quoting restores all 271 characters, which is the
whole of the 3,144 vs 3,234 discrepancy the review flagged.

`parseSkill` now compares the raw scalar against what YAML kept and
errors on any loss, so the next one fails the gate rather than shipping.
Tests cover the truncated case and a legitimately quoted `#`.

Re-aims the `project-structure` Node-backend case, which `AGENTS.md`
answers via its tree and which had oscillated 80 -> 100% across runs —
the partial-overlap signature this PR documents. The replacement names
no internals and probed 4/4.

Makes the documented probe reproduce the harness: the same
`--disallowedTools` flags as `runPrompt` rather than `--allowedTools`,
`printf` instead of zsh's `print -r`, and `jq` over `tool_use` blocks
instead of a `grep` that also matched MCP server names in the init
event. The first-tool-call table is relabelled as an unrestricted probe,
since the eval cannot reach for Bash at all.

RUNS=5 skills:eval: 63/63, 62 at 100%; restoring those 90 characters
displaced nothing.

Signed-off-by: cliffhall <cliff@futurescale.com>
cliffhall added a commit that referenced this pull request Sep 1, 2026
The truncation guard read only the first physical line of the scalar, so
a multiline plain scalar slipped past it:

    description: Covers boards
      #28 and #11

YAML parses that as `Covers boards`, and the first-line-only comparison
saw `Covers boards` on both sides and reported no loss — passing the
exact silent truncation the guard exists to reject.

It now walks the whole plain scalar: the value line plus every indented
continuation, whitespace-collapsed to match YAML's folding of newlines to
spaces. Quoted and block scalars are still skipped, since neither loses
text to a comment.

Two regression cases: the multiline truncation, and a multiline scalar
with no `#`, which must stay clean — the fix cannot be "reject anything
multiline".

No skill description or eval case changed, so skills:eval is unchanged
from round 4's 63/63. local:gate passes, 34/34. 390/390 script tests.

Signed-off-by: cliffhall <cliff@futurescale.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants