feat(cli): add the agents command to teach evlog to AI agents - #492
Merged
Conversation
🦋 Changeset detectedLatest commit: b287962 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
Contributor
|
Thank you for following the naming conventions! 🙏 |
commit: |
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughThe CLI adds ChangesAgent setup workflow
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant User
participant AgentsCommand
participant RunAgents
participant ProjectFiles
participant SkillsCLI
User->>AgentsCommand: run evlog agents
AgentsCommand->>RunAgents: pass command options
RunAgents->>ProjectFiles: plan and write AGENTS.md and CLAUDE.md
RunAgents->>SkillsCLI: run npx skills add
SkillsCLI-->>RunAgents: return installation status
RunAgents-->>AgentsCommand: return and format result
AgentsCommand-->>User: display human or JSON output
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 14
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/docs/content/3.cli/8.agents.md`:
- Around line 67-70: Keep the agent-file documentation consistent across all
listed sites: in apps/docs/content/3.cli/8.agents.md lines 67-70, clarify that
--no-skills still writes both AGENTS.md and CLAUDE.md while skipping only skill
installation; in apps/docs/content/3.cli/1.init.md lines 87-88, name both files
in the init agent step; in apps/docs/content/3.cli/1.init.md line 332, state
that --no-agents skips both files and skills; and in packages/cli/README.md
lines 54-57, apply the same both-file wording to both skip flags.
- Around line 40-44: Update the documentation around the agent skills
installation flow to specify a reviewed, pinned version and integrity or trust
validation for the npx skills add command invoked by evlog agents. Include the
--source path in the documented command and clearly describe the accepted trust
model before shipping the command.
In `@packages/cli/src/commands/agents.ts`:
- Around line 90-106: Move the shared fail(error, { args, log, ui }) helper from
agents.ts into packages/cli/src/lib/command.ts, preserving its EvlogError
handling, catalog logging, UI output, and EXIT_FAIL behavior. Export it from
command.ts, then remove the local duplicate and import the shared fail helper in
both agents.ts and init.ts.
- Line 38: Validate the source option before passing it to skillsCommand: add a
parseSourceArg helper that accepts only non-empty absolute http: or https: URLs,
normalizes valid values, and throws cliErrors.AGENTS_BAD_SOURCE for invalid
input. Apply it in the run option-parsing flow so runSkills receives only the
validated source.
- Around line 11-15: Update parseSkillsArg so both false and an empty string
return { skills: [], noSkills: true }, while preserving trimming and filtering
for valid comma-separated input. Add unit coverage for false, '', and valid
comma-separated values, verifying each parsed result.
In `@packages/cli/src/lib/agents/plan.ts`:
- Around line 32-34: Update read to catch readFileSync failures for existing
paths and raise the established catalog error, including the AGENTS.md path and
its fix metadata, instead of propagating the raw exception. Add the
corresponding catalog entry and imports using the existing cliErrors helper
conventions; preserve the null result for paths that do not exist.
In `@packages/cli/src/lib/agents/report.ts`:
- Around line 36-52: Extract the skills-status branch logic from the agent
report into a shared skillsReportLines helper in the agents report module,
accepting the existing paint function and skills outcome fields. Preserve all
four statuses, glyphs, wording, error rendering, and refresh/retry alignment
exactly, then replace the duplicated rendering in both report modules with
lines.push(...skillsReportLines(paint, skills)).
In `@packages/cli/src/lib/agents/run.ts`:
- Around line 241-268: Update recordAgentsRun to construct the nine-field
telemetry payload once, pass that payload to telemetry.set, and derive
agentsTelemetryFieldNames from the payload’s keys. Remove the duplicated
hard-coded name list while preserving all existing emitted fields and values.
- Around line 119-129: Ensure the generated agents guidance does not claim
skills are available before installation succeeds. Update the flow around
installing, planAgents, and the later skills installation so planning/writing
occurs after the skills outcome when possible, or rewrite the block with
hasSkills: false when skills.status is 'failed' and not in dry-run mode,
allowing the existing documentation fallback in renderBlock to be used.
In `@packages/cli/src/lib/agents/skills.ts`:
- Around line 84-119: Prevent shell injection in the command construction and
execution flow centered on skillsCommand and runSkills. Ensure user-controlled
source and skill values are safely passed when Windows uses shell execution,
using a batch-safe argument mechanism or strict validation/escaping rather than
treating raw values as commands; preserve the existing command behavior and
display output for valid inputs.
In `@packages/cli/src/lib/init/run.ts`:
- Around line 282-295: Update the installing and addingSkills conditions used to
build runs so they no longer require !dryRun, allowing the interactive dry-run
preview in showPlan to include both commands. Preserve the separate dryRun
checks in the execution flow so these commands remain preview-only and are not
actually run during dry runs.
In `@packages/cli/test/agents.test.ts`:
- Around line 105-106: Update the findInstalledSkills test setup to derive
NO_HOME from a fresh, suite-owned mkdtemp directory before each test instead of
using the fixed tmpdir path. Add the required beforeEach import from vitest and
ensure the temporary directory is created and assigned before assertions run,
matching the existing fakeContext isolation pattern.
- Around line 214-299: Add a runAgents test that mocks or spies on the imported
runSkills function to return a failed result, while allowing the normal skills
path to execute. Assert that the returned skills object has status 'failed' and
a populated error field, covering the failure branch used by the agents command.
In `@packages/cli/test/init.test.ts`:
- Around line 242-290: The agent-guidelines tests in runInit lack coverage for
the default agentGuide path when no skills are installed. Add a test alongside
the existing cases that uses nuxtProject() without skill files, mocks the skills
execution seam identified by the agentGuide/SkillsStatus pending transition in
runInit, and verifies the result status changes from pending to the mock’s
installed or failed outcome. Also assert the generated AGENTS.md/CLAUDE.md
writes occur in the expected order relative to skills execution, while
preserving the existing no-skill-file behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: afc9ead3-92b6-4409-96af-58dbc123e768
📒 Files selected for processing (26)
.changeset/cli-agents-command.mdapps/docs/content/3.cli/0.overview.mdapps/docs/content/3.cli/1.init.mdapps/docs/content/3.cli/8.agents.mdapps/docs/content/7.reference/6.agent-skills.mdapps/docs/skills/review-logging-patterns/SKILL.mdapps/telemetry/server/utils/allowed-tools.tsapps/telemetry/test/allowed-tools.test.tspackages/cli/README.mdpackages/cli/src/commands/agents.tspackages/cli/src/commands/index.tspackages/cli/src/commands/init.tspackages/cli/src/core/context.tspackages/cli/src/lib/agents/block.tspackages/cli/src/lib/agents/plan.tspackages/cli/src/lib/agents/report.tspackages/cli/src/lib/agents/run.tspackages/cli/src/lib/agents/skills.tspackages/cli/src/lib/init/prompts.tspackages/cli/src/lib/init/report.tspackages/cli/src/lib/init/resolve.tspackages/cli/src/lib/init/run.tspackages/cli/src/lib/init/telemetry.tspackages/cli/test/agents.test.tspackages/cli/test/init.resolve.test.tspackages/cli/test/init.test.ts
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/cli/src/lib/init/run.ts (1)
252-280: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
runInitdoes not correct AGENTS.md guidance after a failed skill install.
agentGuideis planned withhasSkills: trueat line 263, and this block is written to disk before the skill install (runSkills) runs at line 353. When that install fails,agentGuide.statusis set to'failed'(line 356), but nothing rewrites AGENTS.md — the file keeps telling the agent to read thereview-logging-patternsskill, which is not on disk.
packages/cli/src/lib/agents/run.tsalready fixes this exact scenario forevlog agents(lines 190-206): after a failed install, it re-plans withhasSkills: falseand rewrites the guidance files. Port the same correction here soevlog initgives consistent behavior withevlog agents.The test
packages/cli/test/init.test.tsat lines 326-335 confirms this gap — it only checks that AGENTS.md exists, not that it stops promising the missing skill.🐛 Suggested fix — mirror the correction from agents/run.ts
if (agentGuide) { if (agentGuide.found.length > 0) { agentGuide.status = 'already' } else if (dryRun) { agentGuide.status = 'pending' } else { /* Runs after the writes: the block is the part we own, and it should be on disk whatever a subprocess we do not control decides to do. */ if (interactive) noteSkillsStarting(ctx, agentGuide.command) const outcome = await log.step( 'skills', () => runSkills(skillsCommand({ interactive }), project.packageDir, interactive), r => ({ installed: r.ok }), ) agentGuide.status = outcome.ok ? 'installed' : 'failed' - if (!outcome.ok) agentGuide.error = outcome.error + if (!outcome.ok) { + agentGuide.error = outcome.error + /* The block was written on the assumption the install would succeed. + Rewrite it so it stops naming a skill that is not on disk. */ + const corrected = planAgents({ + root: project.packageDir, + projectName: project.packageName ?? 'This project', + framework: answers.framework, + hasSkills: false, + }) + await log.step('rewriteAgentGuide', async () => { + for (const action of corrected.actions) { + await writeFile(action.path, action.contents, 'utf8') + } + return corrected.actions.length + }) + } } }Consider extracting this correction into a shared helper (e.g. in
packages/cli/src/lib/agents/plan.ts) soagents/run.tsandinit/run.tsdo not each carry their own copy.Also applies to: 342-360
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cli/src/lib/init/run.ts` around lines 252 - 280, Update runInit’s failed runSkills path to re-plan agent guidance with hasSkills: false and rewrite the guidance files after setting agentGuide.status to failed, matching the existing correction in agents/run.ts. Prefer extracting and reusing a shared helper from the agent planning code if practical, while preserving successful-install behavior and the existing status/reporting flow.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/docs/content/3.cli/8.agents.md`:
- Around line 61-65: The documented command and trust statement must match the
command produced and displayed by skillsCommand. Update the relevant
documentation or skillsCommand display construction to include the leading --yes
and any non-interactive or option-specific arguments, while preserving the exact
command shown before execution.
In `@packages/cli/src/lib/agents/skills.ts`:
- Around line 82-102: Update checkSource to reject valid HTTP(S) URLs containing
cmd.exe shell metacharacters such as &, |, ;, and related command separators
before returning the raw value, while preserving existing URL and protocol
validation. Keep the change scoped to the source validation used by runSkills.
---
Outside diff comments:
In `@packages/cli/src/lib/init/run.ts`:
- Around line 252-280: Update runInit’s failed runSkills path to re-plan agent
guidance with hasSkills: false and rewrite the guidance files after setting
agentGuide.status to failed, matching the existing correction in agents/run.ts.
Prefer extracting and reusing a shared helper from the agent planning code if
practical, while preserving successful-install behavior and the existing
status/reporting flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: bb88f1f8-14b2-407d-9264-9e9d22bcfb56
📒 Files selected for processing (16)
.changeset/cli-agents-command.mdapps/docs/content/3.cli/1.init.mdapps/docs/content/3.cli/8.agents.mdpackages/cli/README.mdpackages/cli/src/commands/agents.tspackages/cli/src/commands/init.tspackages/cli/src/lib/agents/plan.tspackages/cli/src/lib/agents/report.tspackages/cli/src/lib/agents/run.tspackages/cli/src/lib/agents/skills.tspackages/cli/src/lib/command.tspackages/cli/src/lib/errors.tspackages/cli/src/lib/init/report.tspackages/cli/src/lib/init/run.tspackages/cli/test/agents.test.tspackages/cli/test/init.test.ts
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/docs/content/3.cli/8.agents.md`:
- Line 63: The “What runs” documentation overstates command visibility by saying
it is always printed before execution. Update that sentence to distinguish the
interactive plan from non-interactive modes, stating that the exact command is
shown in the interactive plan and included in the resulting report/output
otherwise, while preserving the command composition details.
In `@packages/cli/README.md`:
- Around line 59-60: Update the agent-specific options table in the README to
document the supported evlog agents --yes / -y flag, noting that it skips
confirmation for scripted and CI usage.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 31cb2ff1-1dfb-4be4-b575-48e5758f96c4
📒 Files selected for processing (5)
apps/docs/content/3.cli/8.agents.mdpackages/cli/README.mdpackages/cli/src/lib/agents/skills.tspackages/cli/test/agents.test.tspackages/cli/test/init.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔗 Linked issue
📚 Description
📝 Checklist
Summary by CodeRabbit
evlog agentsto generate or updateAGENTS.mdandCLAUDE.mdguidance.evlog init, with an option to skip it and continued execution after skill failures.