Skip to content

chore: make automated reviews complexity-aware - #3172

Merged
Astro-Han merged 3 commits into
mainfrom
chore/review-delete-first
Aug 18, 2026
Merged

chore: make automated reviews complexity-aware#3172
Astro-Han merged 3 commits into
mainfrom
chore/review-delete-first

Conversation

@Astro-Han

@Astro-Han Astro-Han commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • version the same deletion-first review policy for CodeRabbit and Qodo
  • separate concrete findings from their proposed remedies and classify them as fix-now, follow-up, or optional
  • require additive remedies to justify why deletion, consolidation, or reuse is insufficient
  • keep automatic incremental reviews enabled while routing only Qodo action-required findings inline

Qodo operational settings

The Qodo Portal remains the authority for controls that are not reliably repository-scoped:

  • Review every push: enabled
  • High findings: inline and summary
  • Medium and Low findings: summary only
  • Suggest committable fixes: disabled

The repository-level .pr_agent.toml becomes effective for new PRs after merge and owns the Qodo review guidance and inline severity threshold.

Verification

  • git diff --check
  • parsed .coderabbit.yaml with Ruby YAML
  • parsed .pr_agent.toml with Python tomllib
  • confirmed the configured fields against the current CodeRabbit and Qodo documentation
  • reloaded Qodo Portal and verified the saved workflow, routing, remediation, and prompt values
  • product tests were not run because this changes review configuration only

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Codex researched the supported review controls and authored the CodeRabbit and Qodo review guidance in this PR. Both commits include the required Generated-by: Codex trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

Review focus

Please verify that both tools preserve full review coverage while making deletion, consolidation, and reuse the default remediation order. The human contributor owns the final diff, configuration policy, and merge decision.

Classify findings by disposition, require additive remedies to justify their complexity, and keep incremental review explicit.

Generated-by: Codex
@coderabbitai

coderabbitai Bot commented Aug 17, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f28382e8-f5da-43f7-a34f-bea927e26ca6

📥 Commits

Reviewing files that changed from the base of the PR and between d97224b and 4f418c0.

📒 Files selected for processing (1)
  • .coderabbit.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .coderabbit.yaml

Included review availability: Your plan includes up to 3 reviews per rolling hour; 0 remain after this review.


📝 Walkthrough

Summary

This PR improves automated review guidance. It separates findings from remedies, classifies findings by urgency, and requires justification for additive changes. It enables automatic incremental reviews and adds a complexity-delta summary.

The PR extends the existing review configuration. It does not add an application execution path or change source-code behavior.

The change is the smallest coherent solution for the stated review goals. It adds configuration guidance only. No source code, public entities, or product tests require changes.

No code or tests can be deleted without weakening review behavior or validation coverage.

Validation included:

  • git diff --check
  • Ruby YAML parsing of .coderabbit.yaml
  • Python tomllib parsing of .pr_agent.toml
  • Verification of configured review settings against current CodeRabbit and Qodo documentation

Product tests were not run because the PR changes review configuration only. Final required-check status remains unverified.

Complexity delta

  • Authorities: Adds explicit finding dispositions and remedy-priority rules for CodeRabbit and Qodo.
  • States: Adds fix-now, follow-up, and optional classifications.
  • Branches: Adds guidance for additive remedies, resolved-decision carryover, inline-comment severity, and Qodo action-required routing.
  • Configuration: Enables automatic incremental reviews and adds the Complexity delta summary section.
  • Public surface: Adds no exported or public entities.
  • Test-maintenance burden: Adds no product-test burden. Configuration parsing and documentation checks remain necessary.

The added rules make review outcomes more explicit and preserve full review coverage, including security and protected-area findings. They do not add an application execution path. Total maintenance complexity stays justified.

Optional and follow-up findings are non-blocking. They are not implementation requests for this PR.

Review-relevant risks

The current diff changes automated review configuration only. It may affect review comments, finding classification, incremental review behavior, and inline-comment routing. Material changes to security controls, protected areas, governance, releases, licensing, user-visible behavior, or public contracts require independent human review under repository policy.

Final required-check status remains unverified. The person performing the merge reviews the final diff. A maintainer makes the final determination.

Walkthrough

The pull request expands CodeRabbit and PR Agent review instructions with finding dispositions, remedy prioritization, risk classification, decision persistence, Complexity delta reporting, and automatic incremental reviews.

Changes

Review policy configuration

Layer / File(s) Summary
Finding, remedy, and complexity guidance
.coderabbit.yaml, .pr_agent.toml
The configurations define finding dispositions, ordered remedies, additive-change justification, risk categories, decision persistence, and Complexity delta reporting.
Automatic incremental reviews
.coderabbit.yaml
The configuration enables automatic incremental reviews.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 4f418

This PR changes review configuration and guidance only; no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: making automated review guidance complexity-aware.
Description check ✅ Passed The description covers the change, verification, AI use, checklist context, behavior change, and review focus.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Ai Use Disclosure ✅ Passed The PR selects only substantive generative use, names Codex and its scope, and all three introduced commits contain standalone Generated-by: Codex trailers.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/review-delete-first

Comment @coderabbitai help to get the list of available commands.

@Astro-Han
Astro-Han marked this pull request as ready for review August 18, 2026 02:39
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Make automated reviews complexity-aware

⚙️ Configuration changes ✨ Enhancement 🕐 10-20 Minutes

Grey Divider

AI Description

• Classifies concrete findings as fix-now, follow-up, or optional.
• Prioritizes deletion, consolidation, and reuse before additive remedies.
• Adds complexity-delta summaries while preserving automatic incremental reviews.
Diagram

graph TD
  A["Pull request"] --> B["Review policy"] --> C{"Concrete risk?"}
  C -->|Yes| D["Finding disposition"] --> E["Remedy priority"] --> F["Complexity delta"] --> H["Incremental review"] --> B
  C -->|No| G["Omit finding"]
Loading
High-Level Assessment

Keeping the behavior in the existing CodeRabbit configuration is the most direct approach because it changes review output at its source of truth without adding scripts, services, or parallel policy documents. Built-in controls alone cannot express the required finding dispositions and remediation order, while disabling incremental reviews would conflict with the stated goal of preserving resolved decisions.

Files changed (1) +15 / -0

Other (1) +15 / -0
.coderabbit.yamlAdd complexity-aware automated review policy +15/-0

Add complexity-aware automated review policy

• Extends CodeRabbit guidance to separate risks from remedies, classify findings by disposition, and prefer deletion, consolidation, or reuse before additive changes. Requires a complexity-delta summary and explicitly enables automatic incremental reviews that retain resolved or dismissed decisions.

.coderabbit.yaml

@qodo-code-review

qodo-code-review Bot commented Aug 18, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Security findings are suppressed ✓ Resolved 🐞 Bug ≡ Correctness
Description
The new instruction permits only correctness or maintenance findings, excluding concrete security
risks despite the configuration explicitly requiring security effects to be identified. This
violates the PR's stated goal of preserving full review coverage and can cause actionable
vulnerabilities to be omitted from automated reviews.
Code

.coderabbit.yaml[20]

+        Report only concrete, reproducible correctness or maintenance risks, and give each finding one disposition:
Relevance

●●● Strong

PR explicitly requires full review coverage; restricting reports to correctness or maintenance can
suppress concrete security risks.

PR-#1661
PR-#2961

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The added category restriction conflicts with the same configuration's requirement to identify
security, public-contract, licensing, release, and governance effects. Repository policy separately
establishes security and the other listed areas as protected review concerns, proving that
correctness and maintenance alone are not complete coverage.

.coderabbit.yaml[20-23]
.coderabbit.yaml[47-50]
CONTRIBUTING.md[61-67]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The path instruction restricts reportable findings to correctness and maintenance risks, unintentionally excluding security and other protected-area risks. Preserve the evidence threshold without limiting the categories reviewed.

## Issue Context
Deletion is sufficient: remove the category-specific wording and retain a general requirement to report only concrete, reproducible risks. No new state, branch, configuration, authority, public surface, or test-maintenance burden is needed.

## Fix Focus Areas
- .coderabbit.yaml[20-23]
- .coderabbit.yaml[47-50]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context
Review mode: ⚖️ Balanced: This is a behavioral review-configuration change that alters finding dispositions, remediation guidance, incremental-review state, and required output, so it warrants a careful single-pass review despite the small localized diff.

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more'

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread .coderabbit.yaml Outdated
Version the repository-level Qodo review guidance and route only action-required findings inline.

Generated-by: Codex
Remove the category restriction that could suppress concrete security and protected-area findings.

Generated-by: Codex
@Astro-Han

Copy link
Copy Markdown
Contributor Author

Fast-path rationale:\n\n- This PR changes automated review configuration only. It does not alter runtime behavior, user-visible behavior, public contracts, security controls, licensing, releases, or project governance.\n- The change is localized and easy to reverse.\n- All required checks pass. CodeRabbit completed its incremental review without actionable findings, and Qodo resolved its reported High finding after the correction.\n- Astro-Han reviewed the final diff, accepts responsibility for its accuracy and configuration policy, and chose the self-merge fast path.\n\nAI assistance disclosure: Codex inspected the final diff and check status and prepared this comment. The human contributor verified the final diff and made the merge decision.

@Astro-Han
Astro-Han merged commit e685853 into main Aug 18, 2026
12 checks passed
@Astro-Han
Astro-Han deleted the chore/review-delete-first branch August 18, 2026 03:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant