Skip to content

Add facilities for unit and functional tests#165

Merged
mergify[bot] merged 1 commit into
instructlab:mainfrom
danmcp:unittests
Nov 5, 2024
Merged

Add facilities for unit and functional tests#165
mergify[bot] merged 1 commit into
instructlab:mainfrom
danmcp:unittests

Conversation

@danmcp

@danmcp danmcp commented Oct 31, 2024

Copy link
Copy Markdown
Contributor

There was already 1 unit test in place. The existing functional tests are being moved to scripts for consistency with other projects.

One new functional test has been added to test the mt_bench_branch generator as a starting example

I've included basic tests for mt_bench and mmlu. Which already found two issues:

  • mmlu_branch was using self.tasks instead of the number of results calculating the wrong average
  • Not specifying max_workers for mt_bench resulted in an error when it should have defaulted

@mergify mergify Bot added CI/CD Affects CI/CD configuration documentation Improvements or additions to documentation testing Relates to testing ci-failure and removed ci-failure labels Oct 31, 2024
This was referenced Oct 31, 2024
@mergify mergify Bot added ci-failure and removed ci-failure labels Oct 31, 2024
@mergify

mergify Bot commented Oct 31, 2024

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. @danmcp please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@nathan-weinberg nathan-weinberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looked over all these changes and honestly everything LGTM - thank you for this @danmcp!

Two nits - one is in the comment, the other would be it'd be nice to add a badge for this new workflow similar to what is done for the CLI here: https://github.com/instructlab/instructlab/blob/main/README.md?plain=1#L4

Comment thread .github/workflows/test.yml
@mergify mergify Bot added the one-approval label Nov 1, 2024
@danmcp

danmcp commented Nov 1, 2024

Copy link
Copy Markdown
Contributor Author

the other would be it'd be nice to add a badge for this new workflow similar to what is done for the CLI here: https://github.com/instructlab/instructlab/blob/main/README.md?plain=1#L4

Done!

@mergify mergify Bot added the ci-failure label Nov 1, 2024
There was one unit test already existing and this commit adds a couple of starter unit tests for mt_bench and mmlu.

Signed-off-by: Dan McPherson <dmcphers@redhat.com>
@mergify mergify Bot removed the ci-failure label Nov 1, 2024
@mergify
mergify Bot merged commit 4c76340 into instructlab:main Nov 5, 2024
@mergify mergify Bot removed the one-approval label Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD Affects CI/CD configuration documentation Improvements or additions to documentation testing Relates to testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants