[safeoutputs] Generalize the noop prohibition: agents pass `details` (a sibling-tool field) alongside a valid message

[github-actions]

Improve the noop tool description to prevent recurring cross-tool field borrowing

Summary

Analysis of the last 24 h of agentic workflow runs (69 runs, 8 safe-output validator rejections) surfaced one new actionable tool-description gap: agents call noop with a correctly-serialized message string plus an extra details field. noop accepts only message/secrecy/integrity (additionalProperties: false), so the call is rejected.

This pattern has now recurred on 2 consecutive days across 2 different workflows, both Copilot, both reaching for details — meeting the escalation threshold recorded during the 2026-06-14 audit. The workflow prompts are correct (they ask for a run summary, not for a details field), so this is a tool-description gap, not a prompt issue.

🔍 Error analysis details

Error pattern: noop + details (cross-tool parity borrow)

Occurrences: 2 (2026-06-14, 2026-06-15) across 2 workflows.

What agents did wrong: passed a details field in addition to a valid message.

Example 1 — Slide Deck Maintainer (Run §27565130559, copilot, 2026-06-15):

{
  "message": "Slide deck maintenance complete - no changes needed",
  "details": {
    "content_errors_found": 0,
    "layout_issues_found": 0,
    "slides_reviewed": 0,
    "next_recommended_focus": "agentic-workflows",
    "sources_checked": ["source-code"],
    "notes": "slides/index.md is in Git LFS; LFS download blocked (403)..."
  }
}

Rejected with: Invalid arguments: unknown parameter 'details'. Supported parameters for this tool: 'message', 'secrecy', 'integrity'. Recovered ~4 s later by folding everything into message.

Example 2 — PR Sous Chef (Run §27507600670, copilot, 2026-06-14): same tool, same details field, but carrying narrative prose instead of a structured object. Recovered ~3 s later.

Expected:

{ "message": "Slide deck maintenance complete - no changes needed. <metrics and notes folded into this single string>" }

Why this happened: details is a real field on the sibling safe-output tool report_incomplete (verified present in both runs' live tools/list). This is the same cross-tool parity-borrow failure mode already documented for submit_pull_request_review (agents borrowed item_number) and create_check_run (agents borrowed pull_request_number). The current noop description enumerates only two banned example fields (processed, skipped); the agent dutifully avoids those by serializing metrics into message, then reaches for the next plausible sibling field (details) for the extended/structured half of its summary. Enumerating specific examples shifts the agent to the next field rather than closing the gap.

Current tool description

From pkg/workflow/js/safe_outputs_tools.json (noop)

The message property currently reads: "... This is the only content field noop accepts. Serialize any metrics or structured run summary into this single string; additional named fields (for example processed, skipped) are not accepted and cause a missing-message error."

additionalProperties: false; required: message. Both the source copy (pkg/workflow/js/safe_outputs_tools.json) and the runtime copy (actions/setup/js/safe_outputs_tools.json) are in sync — no deployment gap.

Root cause analysis

1. Enumerated-examples prohibition is insufficient. Naming processed, skipped as the banned examples does not stop the agent from using a different plausible field (details). This is the identical failure mode that required generalizing the prohibitions on submit_pull_request_review and create_check_run.
2. details is borrowed from a sibling tool. report_incomplete genuinely accepts details, so the agent assumes parity.
3. The error-message phrasing is inaccurate. The current text says additional fields "cause a missing-message error", but in both occurrences message WAS present, so the real rejection was unknown parameter 'details'. The description over-promises one failure mode and misses the actual one.

Recommended improvements

Apply the same generalize-the-prohibition pattern used for submit_pull_request_review and create_check_run.

1. Lead with the prohibition at the tool-description level (not buried in the message property). Prepend to the noop description: "noop accepts ONLY a single message string (plus optional secrecy/integrity). It does NOT accept any other field — including details, reason, context, or any metric/summary name. Fold all metrics and structured run summaries into the message string."

2. Generalize and correct the message field description:

   • Current: "... additional named fields (for example processed, skipped) are not accepted and cause a missing-message error."
   • Suggested: "... message is the ONLY content field. Any other named field (e.g. details, reason, context, processed, skipped, or any metric name) is rejected with an unknown parameter error — even when message is present. Serialize all metrics/structured summaries into this single string."
   • Why: names the actually-observed offender (details), generalizes beyond two examples, and fixes the inaccurate "missing-message error" claim.

3. Optionally add a one-line cross-reference distinguishing noop from report_incomplete (which DOES take details), since that is the source of the borrowed field.

Affected workflows

• Slide Deck Maintainer — 1 (2026-06-15, structured-object details)
• PR Sous Chef — 1 (2026-06-14, narrative-prose details)

Both recovered automatically within a few seconds, so no downstream output was lost — but the rejection wastes a turn and the pattern is spreading across workflows.

Implementation checklist

• Update noop description + message field description in pkg/workflow/js/safe_outputs_tools.json
• Propagate the same change to the runtime copy actions/setup/js/safe_outputs_tools.json (the 2026-05-30 deployment-gap lesson — keep both in sync)
• make build and make recompile
• make test (ensure TestSafeOutputsToolsJSONInSync still passes)
• Monitor logs 2-3 days for noop + non-message field rejections

References

• Tool schema: pkg/workflow/js/safe_outputs_tools.json + runtime copy actions/setup/js/safe_outputs_tools.json
• Sibling tool with a real details field: report_incomplete
• Prior generalize-the-prohibition precedents: submit_pull_request_review (item_number), create_check_run (pull_request_number)

Run IDs with errors: §27565130559, §27507600670

Generated by ⚡ Daily Safe Output Tool Optimizer · 450.6 AIC · ⌖ 32.5 AIC · ⊞ 9.7K · ◷

• expires on Jun 17, 2026, 2:00 PM UTC-08:00

[copilot-swe-agent]

The agent encountered an error and was unable to start working on this issue: Please try again later, or contact support if the problem persists. (Request id: 09C0:453C4:93BC:22DFD:6A3082D1)

[github-actions]

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · 54.6 AIC · ⌖ 6.77 AIC · ⊞ 24.6K · ◷

[github-actions]

This issue was automatically closed because it expired on 2026-06-17T22:00:54.043Z.

Closed by Workflow