Skip to content

Add missing comment for error_rate return #141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mergify merged 1 commit into from
Oct 2, 2024
Merged

Add missing comment for error_rate return #141

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions src/instructlab/eval/mt_bench.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ def judge_answers(
overall_score MT-Bench score for the overall model evaluation
qa_pairs Question and answer pairs (with scores) from the evaluation
turn_scores A list of indexed turn scores
error_rate Percentage of questions dropped due to errors during evaluation
"""
logger.debug(locals())
return mt_bench_judgment.generate_judgment(
Expand Down Expand Up @@ -246,9 +247,9 @@ def judge_answers(
serving_gpus Number of gpus allocated for serving. Used to tune with max_workers=auto. None indicates to use value specified in constructor.

Returns:
overall_score overall score from the evaluation
overall_score Overall score from the evaluation
qa_pairs Question and answer pairs (with scores) from the evaluation
error_rate percentage of questions dropped due to errors during evaluation
error_rate Percentage of questions dropped due to errors during evaluation
"""
logger.debug(locals())
overall_score, qa_pairs, _, error_rate = mt_bench_judgment.generate_judgment(
Expand Down