Skip to content

Remove deprecated version field from schema and delete Evaluate.ps1 - #10

Merged
Sun Haoran (haoranpb) merged 3 commits into
features/evaluate-minifrom
copilot/cleanup-schema-and-scripts
Oct 9, 2025
Merged

Remove deprecated version field from schema and delete Evaluate.ps1#10
Sun Haoran (haoranpb) merged 3 commits into
features/evaluate-minifrom
copilot/cleanup-schema-and-scripts

Conversation

Copilot AI commented Oct 9, 2025

Copy link
Copy Markdown
Contributor

Overview

This PR performs cleanup tasks to remove obsolete code and simplify the dataset schema:

  1. Removes the version field from the dataset schema and all related code
  2. Deletes scripts/powershell/Evaluate.ps1 which is no longer needed

Changes

Removed version field

The version field in the dataset schema was redundant with environment_setup_version. This PR removes it from:

  • dataset/schema.json: Removed from properties and required fields
  • Python code: Removed from DatasetEntry dataclass, from_json(), and to_dict() methods
  • PowerShell code: Removed from DatasetEntry class and constructor
  • CLI: Removed from the dataset view command output
  • README.md: Removed TODO comment about investigating the version field

The environment_setup_version field remains and continues to serve its purpose of specifying the BC Artifact version for environment setup (e.g., "26.5").

Deleted Evaluate.ps1

The scripts/powershell/Evaluate.ps1 script (110 lines) has been removed as its functionality is now fully handled by the Python package's evaluation capabilities.

Impact

  • 122 lines of code removed across 6 files
  • Cleaner, more maintainable codebase
  • Schema now correctly enforces that new dataset entries should not include the version field
  • No breaking changes to the Python or PowerShell APIs - the code gracefully handles existing dataset entries

Notes

Existing dataset entries in bcbench_nav.jsonl still contain the version field (with empty string values). The updated schema will reject new entries with this field due to additionalProperties: false. These existing entries can be regenerated or cleaned up in a future update if needed.

Original prompt

There are some clean up could be done:

  1. the version from the schema is not needed. Related logic from python pckage and Powershell can be safely removed. README should also be updated. version and environment_setup_commit is combined into envrionment_setup_version.

  2. the Evaluate.ps1 is no longer needed now that we have the python package doing that
    The following is a snapshot of a chat conversation between a user and an AI coding assistant. Prioritize later messages in the conversation.
    User: There are some clean up could be done:

  3. the version from the schema is not needed. Related logic from python pckage and Powershell can be safely removed. README should also be updated. version and environment_setup_commit is combined into envrionment_setup_version.

  4. the Evaluate.ps1 is no longer needed now that we have the python package doing that
    User: There are some clean up could be done:

  5. the version from the schema is not needed. Related logic from python pckage and Powershell can be safely removed. README should also be updated. version and environment_setup_commit is combined into envrionment_setup_version.

  6. the Evaluate.ps1 is no longer needed now that we have the python package doing that

TITLE: Cleanup Tasks for Project Codebase

USER INTENT: The user aims to clean up the project codebase by removing unnecessary components and updating documentation.

TASK DESCRIPTION: The user has identified specific areas in the project that require cleanup, including the removal of outdated logic and files, as well as updates to the README documentation.

EXISTING: The user has noted that the version from the schema is no longer needed, and related logic in the Python package and PowerShell scripts can be removed. The README documentation needs to reflect these changes. Additionally, the Evaluate.ps1 script is deemed unnecessary due to the functionality being handled by the Python package.

PENDING:

  1. Remove the version from the schema and related logic from the Python package and PowerShell scripts.
  2. Update the README documentation to reflect the changes, specifically regarding the combined environment_setup_version.
  3. Delete the Evaluate.ps1 script.

CODE STATE:

  • Python package: Logic related to the version from the schema needs to be identified and removed.
  • PowerShell scripts: Any references to the schema version should be eliminated.
  • README.md: Requires updates to mention the new environment_setup_version and the removal of the Evaluate.ps1 script.

RELEVANT CODE/DOCUMENTATION SNIPPETS:

  • The user did not provide specific code snippets, but the focus is on the schema version and the Evaluate.ps1 script.

OTHER NOTES: The user has reiterated the cleanup tasks, indicating their importance and urgency in the project.

Created from VS Code via the [GitHub Pull Request](https://marketplace.visualstudio.com/items?itemName=GitHub.vscode-pull-request-github) extension.

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Co-authored-by: haoranpb <27280733+haoranpb@users.noreply.github.com>
Copilot AI changed the title [WIP] Clean up schema version and remove unnecessary scripts Remove deprecated version field from schema and delete Evaluate.ps1 Oct 9, 2025
Comment thread README.md Outdated
@haoranpb
Sun Haoran (haoranpb) marked this pull request as ready for review October 9, 2025 06:45
@haoranpb
Sun Haoran (haoranpb) merged commit 461d5e5 into features/evaluate-mini Oct 9, 2025
1 check passed
@haoranpb
Sun Haoran (haoranpb) deleted the copilot/cleanup-schema-and-scripts branch October 9, 2025 06:45
Marko Aleksandric (AleksandricMarko) added a commit that referenced this pull request Jul 31, 2026
…ding

Applies Haoran's review feedback for the extensibility-request categories:

- Rename to explicit `extensibility-request-implement` / `extensibility-request-triage`
  across category ids, enum members, classes, dataset files and module files (#1/#2/#4).
- Slim both prompt templates to reflect real-world usage; category-specific rules now
  live in the ai-ext-fix / argus-triage skills (#3).
- Merge duplicate `bcbench.results` import in commands/evaluate.py (#5).
- Uptake #761: both ext entries subclass `RepoGroundedEntry` (#6/#7).
- Triage `patch` defaults to None; stripped from the dataset (#8).
- Constrain triage label fields with a `ManagedLabel` Literal to catch dataset typos (#9).
- Move the ext dataset classes into dataset/extensibility_request.py, like code review (#10).
- Drop `fetch_commit_if_missing` from both ext pipelines to match the other pipelines (#11).
- Keep ext-implement judge-only for now; container build/publish noted as future work (#12).
- Grade triage with the NL2AL LMChecklist instead of the code-review-style hybrid:
  `ExtRequestTriageResult` removed in favour of `JudgeBasedEvaluationResult`, expected_*
  fields replaced by an `expected` checklist, evaluators `lm_checklist`, core score
  `test_passed`; 17 triage entries migrated (#13/#14/#15/#16).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fb712366-b026-470c-9d09-1bd9e9f52a31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants