Skip to content

fix(cli): preserve ordinary scrape and search output - #248

Draft
developersdigest wants to merge 8 commits into
mainfrom
fix/scrape-search-output
Draft

developersdigest wants to merge 8 commits into
mainfrom
fix/scrape-search-output

Conversation

@developersdigest

@developersdigest developersdigest commented Sep 17, 2026

Copy link
Copy Markdown
Member

Summary

Preserve ordinary scrape/search output and diagnostics: honor multi-URL JSON/file output, add shared receipts, save structured failures before nonzero exit, keep empty search output, expose search --pretty, and report available retry/cache metadata. Add validated millisecond scrape --timeout and update CLI guidance.

Deferred from #245 so Alexandria can be reviewed separately. This draft is stacked on #245 for its shared receipt/error helper; merge that first and retarget this PR to main.

Validation

TypeScript build and 140 tests passed across six affected suites, including compiled CLI calls against a local HTTP fixture. Reduced redundant new coverage; retained order/partial-failure/file-output checks and command-level pretty/timeout/error regressions. No paid API calls.

Boundaries

Related: Exchange #548, #556, #547, #567. No API changes; server billing/cache policy is unchanged. All-command normalization beyond search/scrape remains separate.

Explicit multi-URL output is JSON regardless of extension. Failed single calls write JSON to requested files, including non-JSON filenames; successful raw text remains pipeable. Missing charge metadata means unknown. Retry hints are preserved only when supplied by the transport/SDK.


Summary by cubic

Preserves ordinary scrape and search output while making structured output and failures consistent. Multi-URL scrape now honors --json and -o with an ordered JSON array, while failed requests write structured errors before the command exits nonzero; raw single-URL output remains pipeable.

CLI behavior

  • Adds receipts with reported credits, request and operation IDs, plus cache and retry diagnostics on stderr.
  • Keeps empty search responses and metadata in JSON and file output, and adds search --pretty.
  • Adds validated scrape --timeout <ms> support and disables automatic resume when it is set.
  • Keeps default multi-URL scraping as per-URL files, while reporting failed URLs without creating files for them.
  • Updates the README and skill guides with output, billing, retry, timeout, and recovery guidance.

Written for commit 0811001. Summary will update on new commits.

Review in cubic

Base automatically changed from fix/multi-scrape-explicit-output to main September 17, 2026 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant