Uptake bc-eval 0.3.12 and drop the evaluator CAPI taxonomy shim - #771
Closed
Sun Haoran (haoranpb) with Copilot wants to merge 1 commit into
Closed
Uptake bc-eval 0.3.12 and drop the evaluator CAPI taxonomy shim#771Sun Haoran (haoranpb) with Copilot wants to merge 1 commit into
Sun Haoran (haoranpb) with Copilot wants to merge 1 commit into
Conversation
Co-authored-by: haoranpb <27280733+haoranpb@users.noreply.github.com>
Copilot created this pull request from a session on behalf of
Sun Haoran (haoranpb)
August 5, 2026 09:24
View session
Sun Haoran (haoranpb)
marked this pull request as ready for review
August 5, 2026 10:32
Contributor
There was a problem hiding this comment.
Pull request overview
Updates bc-eval to use its native CAPI taxonomy headers and removes the obsolete evaluator shim.
Changes:
- Bumps bc-eval to 0.3.12 in evaluation workflows.
- Removes the evaluator-side taxonomy patch and tests.
- Retains the separate bcal bridge taxonomy handling.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/summarize-results.yml |
Updates bc-eval to 0.3.12. |
.github/workflows/bcal-evaluation.yml |
Updates the bcal bridge environment dependency. |
evaluator/scores.py |
Removes shim initialization. |
evaluator/_capi_taxonomy.py |
Deletes the obsolete taxonomy shim. |
tests/test_capi_taxonomy_evaluator.py |
Deletes tests for the removed shim. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bc-eval 0.3.12 sends the M365 LLM API taxonomy/CoS headers itself, so the workaround added in #756 for the evaluator/judge path is no longer needed.
Version bumps
.github/workflows/summarize-results.yml:bc-eval[capi]==0.3.8→0.3.12.github/workflows/bcal-evaluation.yml:bc-eval[capi]==0.3.7→0.3.12Cleanup
evaluator/scores.py: removed thesys.pathhack and_install_capi_taxonomy()call; the file is now just the evaluator classes.evaluator/_capi_taxonomy.pyandtests/test_capi_taxonomy_evaluator.py.Unchanged
src/bcbench/agent/bcal/bc_eval_capi_bridge.pyand theCAPI_TAXONOMY_*env vars inbcal-evaluation.ymlare kept — the bridge patchesCapiModelfor the bcal target-model path and still needs them.