Support wildcard cached logs files by Copilot · Pull Request #60702 · github/gh-aw

Copilot · 2026-09-13T23:11:01Z

gh aw logs --cached-logs now supports prefix wildcard cache inputs, allowing runs to reuse all matching JSONL cache shards and write fresh data to a collision-resistant new shard.

Wildcard cache input
- Accepts trailing prefix wildcards such as logs-*.
- Loads all matching *.jsonl files as the starting cache.
- Rejects non-trailing wildcard patterns.
Unique cache output
- Writes newly downloaded records to a new file using the same prefix.
- File names include Unix time plus random crypto bytes.

gh aw logs --cached-logs .github/aw/logs/runs-*
# reads:  .github/aw/logs/runs-*.jsonl
# writes: .github/aw/logs/runs-<unix>-<random>.jsonl

Date-range pruning
- In wildcard mode, removes source cache files that contain only dated run records outside the requested range.
- Preserves files with metadata-only or otherwise non-prunable records.
Cache merge behavior
- Merges wildcard sources by modification time, with generated timestamp tie-breaking.
- Warns when newer cache shards replace duplicate run or workflow-list records.
CLI surface
- Adds --cached-logs as an alias for cached JSONL logs.
- Updates help text and docs for wildcard cache behavior.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2026-09-13T23:24:10Z

@copilot add tests for expired shards deletion.

Copilot

🟡 Changes recommended

Quote wildcard examples and align help/documentation with actual pruning behavior.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds wildcard cached JSONL shard support to gh aw logs, including merging, pruning, and unique output files.

Changes:

Adds wildcard cache loading, merging, duplicate warnings, and date-range pruning.
Integrates cache handling across standard, stdin, and multi-target flows.
Adds the --cached-logs alias, tests, help text, and documentation.

File summaries

File	Description
`pkg/cli/logs_orchestrator.go`	Cache lifecycle integration
`pkg/cli/logs_orchestrator_types.go`	Internal cache state fields
`pkg/cli/logs_orchestrator_stdin.go`	Stdin cache integration
`pkg/cli/logs_multi.go`	Multi-target cache finalization
`pkg/cli/logs_command.go`	CLI flag, help, and example updates
`pkg/cli/logs_command_test.go`	CLI flag tests
`pkg/cli/logs_cached_json.go`	Wildcard loading, merging, output, and pruning
`pkg/cli/logs_cached_json_test.go`	Wildcard and pruning tests
`docs/src/content/docs/setup/cli.md`	CLI documentation updates

Review details

Suppressed comments (2)

docs/src/content/docs/setup/cli.md:520

The documentation both shows an unquoted shell wildcard and overstates pruning: matching files are not all deleted when they have metadata-only or other non-prunable records. Quote the example and state that only shards containing exclusively out-of-range dated run records are removed.

Pass a trailing wildcard prefix such as `--cached-logs logs-*` to load all matching `logs-*.jsonl` files as the starting cache and write newly downloaded data to a unique `logs-<unix-time>-<random>.jsonl` file. With `--start-date` or `--end-date`, wildcard cache files that contain no in-range run records are deleted.

pkg/cli/logs_command.go:160

This wording says every wildcard cache file with no in-range run is deleted, but pruneCachedLogsJSONLWildcardSources preserves metadata-only, unknown-schema, undated, and other non-prunable records. Please describe deletion as applying only to files containing exclusively dated run records outside the requested range, so the help text matches the implementation and the PR contract.

cached files with no run records in the requested date range are deleted.

Files reviewed: 9/9 changed files
Comments generated: 2
Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.

Copilot · 2026-09-14T00:12:21Z

-`--cached-jsonl` reuses compatible, schema-versioned run records and workflow-run discovery responses. It writes exactly one JSON value per line, appending every complete `gh run list` payload before downloading artifacts and available GitHub API rate-limit reports after collection. Each enriched `run` record includes job execution data, sanitized MCP tool-call metadata, and available engine, model, runtime, and component versions for downstream dashboards. Raw tool errors, arguments, responses, and artifact bodies are excluded. Discovered runs therefore remain available when a timeout or API limit interrupts processing. Records from incompatible schema versions are ignored. Use `gh aw json-schema logs-jsonl` to generate the schema for each JSON Lines item.
+`--cached-jsonl` and its `--cached-logs` alias reuse compatible, schema-versioned run records and workflow-run discovery responses. They write exactly one JSON value per line, appending every complete `gh run list` payload before downloading artifacts and available GitHub API rate-limit reports after collection. Each enriched `run` record includes job execution data, sanitized MCP tool-call metadata, and available engine, model, runtime, and component versions for downstream dashboards. Raw tool errors, arguments, responses, and artifact bodies are excluded. Discovered runs therefore remain available when a timeout or API limit interrupts processing. Records from incompatible schema versions are ignored. Use `gh aw json-schema logs-jsonl` to generate the schema for each JSON Lines item.
+
+Pass a trailing wildcard prefix such as `--cached-logs logs-*` to load all matching `logs-*.jsonl` files as the starting cache and write newly downloaded data to a unique `logs-<unix-time>-<random>.jsonl` file. With `--start-date` or `--end-date`, wildcard cache files that contain no in-range run records are deleted.


Quoted the wildcard in the setup documentation as --cached-logs 'logs-*' in ab5f905.

Copilot · 2026-09-14T00:12:21Z

  %[1]s logs --json                    # JSON format (compact by default, use -v for full)
  %[1]s logs --json -v                 # Full JSON with audit metadata
  %[1]s logs --cached-jsonl logs.jsonl # Reuse matching records and append new results immediately
+  %[1]s logs --cached-logs logs-*      # Reuse logs-*.jsonl files and write new data to a unique logs-*.jsonl file


Quoted the wildcard in the CLI example and matching setup documentation in ab5f905.

github-actions · 2026-09-13T23:40:41Z

⚠️ PR Code Quality Reviewer failed during code quality review.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🔎 Code quality review by PR Code Quality Reviewer

github-actions · 2026-09-13T23:40:42Z

✅ Ponytail Reviewer completed successfully!

Lean already. Ship.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

ab.chatgpt.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"

See Network Configuration for more information.

Generated by Ponytail Reviewer for #60702

github-actions · 2026-09-13T23:40:42Z

🧠 Matt Pocock Skills Reviewer failed during the skills-based review.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

github-actions · 2026-09-13T23:40:45Z

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

github-actions · 2026-09-13T23:40:48Z

⚠️ Security scanning failed for Design Decision Gate 🏗️. Review the logs for details.

🏗️ ADR gate enforced by Design Decision Gate 🏗️

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-09-13T23:41:43Z

@copilot add tests for expired shards deletion.

Added end-to-end coverage for deleting both pre-range and post-range wildcard shards while preserving in-range and newly written shards in be67a11.

github-actions · 2026-09-13T23:45:40Z

ADR required for this implementation

I generated a draft ADR at docs/adr/60702-support-wildcard-cached-logs-files.md and pushed it to this PR branch.

Evidence used

The PR adds 387 new lines in default business logic directories, so ADR enforcement is required by the design gate.
The implementation changes pkg/cli/logs_cached_json.go and related orchestration code to support wildcard cache shards, unique shard output names, duplicate-resolution behavior, and date-range pruning.
The PR description and docs updates describe a new --cached-logs alias and wildcard cache semantics for gh aw logs.

Inferred architectural decision

Treat cached logs as either a single JSONL file or a trailing-wildcard shard prefix, loading all matching shards as cache input while writing new data to a uniquely named shard and pruning fully out-of-range wildcard shards.

Next action

Please review and refine the draft ADR, especially the rationale and trade-offs, before merge.

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · pi · gpt54 · 20.8 AIC · ⊞ 9.7K · ◷
_{Comment /review to run again}

github-actions

Reviewed with a normal correctness-focused pass (this PR is CLI/backend logic, not UI, so Impeccable's UX/accessibility modes don't apply — used inline mode guidance's mixed_unclear fallback).

The wildcard cache implementation itself (path resolution, merge, pruning, unique-file naming with crypto/rand) looks correct and is well covered by new tests; build and go test ./pkg/cli/... -run 'CachedLogs|CachedJSONL' pass.

One recurring issue found in three places: the --cached-logs logs-* examples in help text and docs are unquoted, so a user's shell will glob-expand them against files in the current directory before the CLI ever sees the argument, silently breaking the wildcard-cache feature. Left inline comments with the fix (quote the pattern, e.g. 'logs-*').

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · copilot · sonnet50 · 63.7 AIC · ⌖ 13.7 AIC · ⊞ 8.4K

github-actions · 2026-09-13T23:50:15Z

  %[1]s logs --json                    # JSON format (compact by default, use -v for full)
  %[1]s logs --json -v                 # Full JSON with audit metadata
  %[1]s logs --cached-jsonl logs.jsonl # Reuse matching records and append new results immediately
+  %[1]s logs --cached-logs logs-*      # Reuse logs-*.jsonl files and write new data to a unique logs-*.jsonl file


This example uses an unquoted shell glob (logs-*). In an interactive shell with any matching files in the CWD, this argument will be expanded by the shell before the program sees it, silently defeating the wildcard cache feature (and potentially passing an unexpected literal filename to --cached-logs). The example should be quoted, e.g. --cached-logs 'logs-*', to correctly demonstrate the intended usage.

@copilot please address this.

Updated the CLI example to use --cached-logs 'logs-*' in ab5f905.

github-actions · 2026-09-13T23:50:15Z

+Use --cached-jsonl (or --cached-logs) to reuse matching run records without downloading and processing their
 artifacts again. New results are appended immediately as JSON Lines. When a date range is specified,
 cached run records outside that range are removed after collection; other record types are retained.
+Pass a trailing wildcard prefix such as --cached-logs logs-* to load all matching .jsonl files


Same unquoted-glob issue as the example above: --cached-logs logs-* will be shell-expanded before reaching the program in typical usage. Please quote the pattern in this help text (e.g. --cached-logs 'logs-*') so users don't unintentionally pass an already-expanded filename or file list.

@copilot please address this.

Updated the detailed help text to quote 'logs-*' and precisely describe deletion of exclusively dated, out-of-range shards in ab5f905.

github-actions · 2026-09-13T23:50:16Z

-`--cached-jsonl` reuses compatible, schema-versioned run records and workflow-run discovery responses. It writes exactly one JSON value per line, appending every complete `gh run list` payload before downloading artifacts and available GitHub API rate-limit reports after collection. Each enriched `run` record includes job execution data, sanitized MCP tool-call metadata, and available engine, model, runtime, and component versions for downstream dashboards. Raw tool errors, arguments, responses, and artifact bodies are excluded. Discovered runs therefore remain available when a timeout or API limit interrupts processing. Records from incompatible schema versions are ignored. Use `gh aw json-schema logs-jsonl` to generate the schema for each JSON Lines item.
+`--cached-jsonl` and its `--cached-logs` alias reuse compatible, schema-versioned run records and workflow-run discovery responses. They write exactly one JSON value per line, appending every complete `gh run list` payload before downloading artifacts and available GitHub API rate-limit reports after collection. Each enriched `run` record includes job execution data, sanitized MCP tool-call metadata, and available engine, model, runtime, and component versions for downstream dashboards. Raw tool errors, arguments, responses, and artifact bodies are excluded. Discovered runs therefore remain available when a timeout or API limit interrupts processing. Records from incompatible schema versions are ignored. Use `gh aw json-schema logs-jsonl` to generate the schema for each JSON Lines item.
+
+Pass a trailing wildcard prefix such as `--cached-logs logs-*` to load all matching `logs-*.jsonl` files as the starting cache and write newly downloaded data to a unique `logs-<unix-time>-<random>.jsonl` file. With `--start-date` or `--end-date`, wildcard cache files that contain no in-range run records are deleted.


This doc example also has the unquoted-glob problem: `--cached-logs logs-*` will be expanded by the user's shell before the CLI parses it (unless there happen to be zero matching files, or the user manually quotes it themselves). Please update the doc to show the quoted form, `--cached-logs 'logs-*'`, so users copying this example get the intended wildcard-cache behavior instead of a shell-expanded argument.

@copilot please address this.

Updated the setup documentation to quote 'logs-*' and precisely describe deletion of exclusively dated, out-of-range shards in ab5f905.

gh-aw-bot · 2026-09-14T00:02:31Z

@copilot reviewers left actionable feedback on the wildcard cache docs/help text for this PR.

Please address these items, then run the pr-finisher skill:

Quote wildcard examples as --cached-logs 'logs-*' so shells do not expand them before gh aw receives the argument.
Align the help/docs wording with actual pruning behavior: only wildcard shard files containing exclusively out-of-range dated run records should be described as deleted.
Review the existing comments on pkg/cli/logs_command.go and docs/src/content/docs/setup/cli.md, apply the fixes, and push an update.

Recent review links:

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 20.1 AIC · ⊞ 9.1K · ◷
_{Comment /souschef to run again}

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-09-14T00:12:21Z

@copilot reviewers left actionable feedback on the wildcard cache docs/help text for this PR.

Please address these items, then run the pr-finisher skill:...

Addressed all linked help/docs feedback and pushed ab5f905. Wildcards are now quoted, pruning language matches the implementation, validation passed, and the pr-finisher skill was run.

github-actions · 2026-09-14T07:38:15Z

🎉 This pull request is included in a new release.

Release: v0.89.13

Copilot AI and others added 4 commits September 13, 2026 22:51

Add cached logs wildcard support

f7416d7

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Address cached logs wildcard review feedback

4a747c3

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Harden cached logs wildcard merging

fa72974

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Stabilize cached logs wildcard ordering

b812be0

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI assigned Copilot and pelikhan Sep 13, 2026

Copilot finished work on behalf of pelikhan September 13, 2026 23:11

Copilot AI requested a review from pelikhan September 13, 2026 23:11

Copilot created this pull request from a session on behalf of pelikhan September 13, 2026 23:11 View session

pelikhan marked this pull request as ready for review September 13, 2026 23:23

Copilot AI balanced review requested due to automatic review settings September 13, 2026 23:23

Copilot started reviewing on behalf of pelikhan September 13, 2026 23:24 View session

Copilot AI reviewed Sep 13, 2026

View reviewed changes

Copilot started work on behalf of pelikhan September 13, 2026 23:33 View session

Test expired wildcard shard deletion

be67a11

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan September 13, 2026 23:42

github-actions Bot added the needs-review label Sep 13, 2026

Add ADR for wildcard cached logs files

39773d8

github-actions Bot mentioned this pull request Sep 13, 2026

[pr-sous-chef] pr-sous-chef run report #60656

Closed

github-actions Bot reviewed Sep 13, 2026

View reviewed changes

This was referenced Sep 13, 2026

[aw] Matt Pocock Skills Reviewer timed out #60707

Closed

[pr-sous-chef] pr-sous-chef run report #60708

Closed

Copilot started work on behalf of gh-aw-bot September 14, 2026 00:03 View session

Clarify wildcard cache help

ab5f905

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot September 14, 2026 00:21

Copilot AI requested a review from gh-aw-bot September 14, 2026 00:21

pelikhan merged commit c3d7c9e into main Sep 14, 2026
34 checks passed

pelikhan deleted the copilot/update-logs-command-support branch September 14, 2026 00:28

github-actions Bot mentioned this pull request Sep 14, 2026

[blog] Weekly blog post – 2026-09-14 #60729

Merged

This was referenced Sep 14, 2026

[instructions] Sync instruction files with release v0.88.7 (assign-to-agent reasoning-effort) #60800

Merged

Remove logs cached-logs alias #60933

Merged

Conversation

Copilot AI commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pelikhan commented Sep 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (2)

Uh oh!

Copilot AI Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI commented Sep 13, 2026

Uh oh!

github-actions Bot commented Sep 13, 2026

ADR required for this implementation

Evidence used

Inferred architectural decision

Next action

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Sep 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

gh-aw-bot commented Sep 14, 2026

Uh oh!

Copilot AI commented Sep 14, 2026

Uh oh!

Uh oh!

github-actions Bot commented Sep 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Sep 13, 2026 •

edited

Loading

github-actions Bot commented Sep 13, 2026 •

edited

Loading

github-actions Bot commented Sep 13, 2026 •

edited

Loading

github-actions Bot commented Sep 13, 2026 •

edited

Loading

github-actions Bot commented Sep 13, 2026 •

edited

Loading

github-actions Bot commented Sep 13, 2026 •

edited

Loading