feat(conclude): distinguish threat detection engine failures from real findings

[davidslater]

Created by GitHub Ace · View Session

Closes #732

Parity follow-up for #732 (gh-aw v0.84.3).

What gh-aw changed

gh-aw PR	Change	Applies here?
#49527	Tooling failures (agent_failure, parse_error) get a distinct <!-- gh-aw-threat-engine-error --> marker instead of <!-- gh-aw-threat-detected -->	✅ this repo renders its own verdict step summary and job-log diagnostics with the same reason codes
#49497	Engine-error banner retitled Threat Detection Engine Failure with progressive disclosure	✅ same surfaces
#49586	Detection model defaults to the detection alias	⚪️ no code change needed — the model reaches threat-detect through COPILOT_MODEL/GH_AW_MODEL_DETECTION_* and is forwarded verbatim; aliases are resolved by the AWF API proxy. Documented only.

Changes

• pkg/detector/reason.go (new) — shared host-side reason constants (ReasonThreatDetected, ReasonAgentFailure, ReasonParseError), the ThreatDetectedMarker / ThreatEngineErrorMarker markers mirrored from gh-aw, plus IsToolingFailureReason, ThreatMarker, and ThreatHeadline.
• pkg/detector/summary.go — FormatVerdictSummary now emits the marker matching the reason, titles the block Threat Detection Engine Failure for tooling failures (Threat Detection Verdict otherwise), and adds the one-line headline. Clean outcomes (success, skipped) carry no marker.
• cmd/threat-detect/conclude.go — echoes the same headline into the job log, and reuses detector.IsToolingFailureReason plus the reason constants instead of duplicating string literals.
• specs/threat-detection-spec.md — new normative rule TD-20i defining the reason → marker/headline mapping.
• README.md — documents the marker/title table and that --model may be a gh-aw alias.
• Tests for both the detector helpers and the two conclude paths (tooling failure vs real verdict).

Rendered output for a missing verdict:

<!-- gh-aw-threat-engine-error -->

<details>
<summary>Threat Detection Engine Failure</summary>

**Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.**
...

Validation

make fmt lint build test — all green; manually smoke-tested both conclude paths.

[Copilot]

Pull request overview

Aligns standalone conclusion reporting with gh-aw by distinguishing engine failures from genuine security findings.

Changes:

• Adds shared reason, marker, and headline helpers.
• Updates summaries and logs with failure-specific messaging.
• Documents and tests the revised contract.

Show a summary per file

File	Description
README.md	Documents markers and model aliases.
specs/threat-detection-spec.md	Defines normative reason-to-marker mapping.
pkg/detector/reason.go	Adds reason classification and rendering helpers.
pkg/detector/reason_test.go	Tests reason helpers.
pkg/detector/summary.go	Renders differentiated verdict summaries.
pkg/detector/summary_test.go	Tests markers, titles, and headlines.
cmd/threat-detect/conclude.go	Applies shared reasons and log headlines.
cmd/threat-detect/conclude_test.go	Tests tooling-failure and threat paths.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 8/8 changed files
• Comments generated: 0
• Review effort level: Balanced