Skip to content

Uptake capi headers wave 2 - #772

Merged
Sun Haoran (haoranpb) merged 3 commits into
mainfrom
chore/uptake-CAPI-headers-wave-2
Aug 6, 2026
Merged

Uptake capi headers wave 2#772
Sun Haoran (haoranpb) merged 3 commits into
mainfrom
chore/uptake-CAPI-headers-wave-2

Conversation

@haoranpb

@haoranpb Sun Haoran (haoranpb) commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Follow up #756

Instead of handling the headers in BC-Bench, moved the logic upstream into bc-eval, and uptake, bc-eval now have those headers out of box. (see https://microsoft.ghe.com/bic/BC-LanguageModelTools/pull/69)

The implementation of those headers belongs in bc-eval, so the downstream (e.g. BC-Bench) does not need to handle them.

Copilot AI balanced review requested due to automatic review settings August 6, 2026 10:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Moves CAPI header handling into bc-eval 0.3.13 and removes BC-Bench’s local shims.

Changes:

  • Upgrades CAPI evaluation workflows to bc-eval 0.3.13.
  • Removes local taxonomy-header injection and related tests/configuration.
  • Simplifies the CAPI bridge to call the client directly.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
.github/workflows/bcal-evaluation.yml Upgrades bc-eval and removes taxonomy environment variables.
.github/workflows/summarize-results.yml Upgrades the evaluation tool dependency.
evaluator/_capi_taxonomy.py Removes the local evaluator header shim.
evaluator/scores.py Removes shim initialization.
src/bcbench/agent/bcal/bc_eval_capi_bridge.py Removes header injection and retry code.
tests/test_bcal_capi_bridge.py Removes tests for deleted bridge behavior.
tests/test_capi_taxonomy_evaluator.py Removes tests for the deleted evaluator shim.

Comment thread src/bcbench/agent/bcal/bc_eval_capi_bridge.py Outdated
Copilot AI review requested due to automatic review settings August 6, 2026 11:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/bcbench/agent/bcal/bc_eval_capi_bridge.py:137

  • This bypasses the bridge's existing retry policy, which was independent of the taxonomy shim. A transient CAPI HttpResponseError such as DependencyFailure/server_error will now abort the entire agent turn on the first failure instead of retrying up to three times. Please retain _create_with_retry (and its tests) and remove only the taxonomy-specific patching.
    response = client.chat.completions.create(**kwargs)

@haoranpb
Sun Haoran (haoranpb) marked this pull request as ready for review August 6, 2026 11:08
@haoranpb
Sun Haoran (haoranpb) enabled auto-merge (squash) August 6, 2026 11:12
@haoranpb
Sun Haoran (haoranpb) merged commit 0c04201 into main Aug 6, 2026
15 checks passed
@haoranpb
Sun Haoran (haoranpb) deleted the chore/uptake-CAPI-headers-wave-2 branch August 6, 2026 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants