Skip to content

No skills in your taxonomy results in stack trace for mt_bench_branch #150

Description

@danmcp

DEBUG 2024-10-14 16:42:16,696 instructlab.model.backends.vllm:563: Successful sample recorded, (stable count 6, free 23368695808, last free 23368695808)
Traceback (most recent call last):
File "/home/ec2-user/instructlab/venv/bin/ilab", line 8, in
sys.exit(ilab())
^^^^^^
File "/home/ec2-user/instructlab/venv/lib64/python3.11/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ec2-user/instructlab/venv/lib64/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/ec2-user/instructlab/venv/lib64/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ec2-user/instructlab/venv/lib64/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ec2-user/instructlab/venv/lib64/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ec2-user/instructlab/venv/lib64/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ec2-user/instructlab/venv/lib64/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ec2-user/instructlab/src/instructlab/clickext.py", line 319, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/home/ec2-user/instructlab/src/instructlab/model/evaluate.py", line 668, in evaluate
evaluator.gen_answers(
File "/home/ec2-user/eval/src/instructlab/eval/mt_bench.py", line 221, in gen_answers
mt_bench_answers.generate_answers(
File "/home/ec2-user/eval/src/instructlab/eval/mt_bench_answers.py", line 160, in generate_answers
reorg_answer_file(answer_file)
File "/home/ec2-user/eval/src/instructlab/eval/mt_bench_answers.py", line 30, in reorg_answer_file
with open(answer_file, "r", encoding="utf-8") as fin:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/ec2-user/.local/share/instructlab/internal/eval_data/mt_bench_branch/rc/model_answer/granite-7b-lab.jsonl'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions