Add dedicated BC PR Review evaluation runner - #800
Open
Wenjie Fan (gggdttt) wants to merge 3 commits into
Open
Conversation
Keep code-review available to generic Copilot and Claude runners while adding a production-fidelity runner backed by the BC PR Review engine. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Wenjie Fan (gggdttt)
force-pushed
the
gggdttt-merge-pr-795
branch
from
August 17, 2026 12:15
ae0dce5 to
f758e35
Compare
Keep the workflow dispatch choice aligned with the Copilot CLI model values introduced by #796. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sun Haoran (haoranpb)
left a comment
Collaborator
There was a problem hiding this comment.
Try to test by running this locally, since it's a brand new path
Pin the engine, run its full non-posting production pipeline, collect cost and provenance metrics, and simplify the runner configuration and CLI. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 003aa3eb-6da6-49f0-bb28-49c8bb0e0d3f
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reapply the decoupled code-review workflow on top of
revert-774-bench/engine-review-runnerso the complete feature can be reviewed as one focused diff.code-reviewavailable to the generic Copilot and Claude runners.pr-reviewrunner backed by the production BC PR Review engine and BCQuality.al-code-review-findings.jsonartifact.f2ac8704bf8d39000f8002bcf2d287f1f5b5e9ba, default togpt-5.6-luna, simplify the CLI, and bump BC-Bench to0.9.0.Dependency
CI authentication for local engine reviews depends on microsoft/BC-ALAgents#57, whose exact commit is pinned by this PR.
Validation
pre-commit run --all-filespytest: 750 passed, 2 skippedsynthetic__style-018with the pinned engine completed through the production non-posting path, emitted normalized output, and parsed 5.9M prompt tokens, 51K completion tokens, and 50.6 AI Credits.Context
The target branch reverts #774. Using it as the base makes this PR show the full feature rather than only the follow-up decoupling changes from #795.