Skip to content

Uptake bc-eval 0.3.12 and drop the evaluator CAPI taxonomy shim - #771

Closed
Sun Haoran (haoranpb) with Copilot wants to merge 1 commit into
mainfrom
copilot/update-bc-eval-version
Closed

Uptake bc-eval 0.3.12 and drop the evaluator CAPI taxonomy shim#771
Sun Haoran (haoranpb) with Copilot wants to merge 1 commit into
mainfrom
copilot/update-bc-eval-version

Conversation

Copilot AI commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

bc-eval 0.3.12 sends the M365 LLM API taxonomy/CoS headers itself, so the workaround added in #756 for the evaluator/judge path is no longer needed.

Version bumps

  • .github/workflows/summarize-results.yml: bc-eval[capi]==0.3.80.3.12
  • .github/workflows/bcal-evaluation.yml: bc-eval[capi]==0.3.70.3.12

Cleanup

  • evaluator/scores.py: removed the sys.path hack and _install_capi_taxonomy() call; the file is now just the evaluator classes.
  • Deleted evaluator/_capi_taxonomy.py and tests/test_capi_taxonomy_evaluator.py.

Unchanged

  • The taxonomy shim in src/bcbench/agent/bcal/bc_eval_capi_bridge.py and the CAPI_TAXONOMY_* env vars in bcal-evaluation.yml are kept — the bridge patches CapiModel for the bcal target-model path and still needs them.

Co-authored-by: haoranpb <27280733+haoranpb@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings August 5, 2026 09:24
Copilot AI removed the request for review from Copilot August 5, 2026 09:24
@haoranpb
Sun Haoran (haoranpb) marked this pull request as ready for review August 5, 2026 10:32
Copilot AI balanced review requested due to automatic review settings August 5, 2026 10:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates bc-eval to use its native CAPI taxonomy headers and removes the obsolete evaluator shim.

Changes:

  • Bumps bc-eval to 0.3.12 in evaluation workflows.
  • Removes the evaluator-side taxonomy patch and tests.
  • Retains the separate bcal bridge taxonomy handling.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
.github/workflows/summarize-results.yml Updates bc-eval to 0.3.12.
.github/workflows/bcal-evaluation.yml Updates the bcal bridge environment dependency.
evaluator/scores.py Removes shim initialization.
evaluator/_capi_taxonomy.py Deletes the obsolete taxonomy shim.
tests/test_capi_taxonomy_evaluator.py Deletes tests for the removed shim.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants