Add magic generics to ai.ops.evaluate

[anbuzin]

Here's one option for how we can tie together FooQuestions and FooAnswers without having to pass FooAnswers somewhere twice.

class TicketAnswers(ai.ops.BaseAnswerModel):
    queue: ai.ops.ChoiceAnswer
    urgency: ai.ops.ScoreAnswer
    refund_warranted: ai.ops.BooleanAnswer


class TicketQuestions(ai.ops.BaseQuestionModel[TicketAnswers]):
    queue: ai.ops.ChoiceQuestion
    urgency: ai.ops.ScoreQuestion
    refund_warranted: ai.ops.BooleanQuestion

See evaluation.py:213 for why we might not want to do this.

    if isinstance(questions, BaseQuestionModel):
        # Pydantic stores concrete generic arguments on the specialized base,
        # not on a question subclass. Walk the MRO for indirect subclasses.
        for base in type(questions).__mro__:
            if issubclass(base, BaseQuestionModel):
                args = getattr(base, "__pydantic_generic_metadata__", {}).get(
                    "args", ()
                )
                if args:
                    answer_model = args[0]
                    break

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
ai-python	Ready	Preview, v0	Sep 28, 2026 4:14pm UTC