agent-evals

https://github.com/kigster/agent-evals

Runs a suite of cases against one or more models through an adapter the consumer writes, scores each reply with a scorer the consumer writes, and prints leaderboards, per-case breakdowns and Braintrust experiments. The gem knows nothing about what is being evaluated: it owns discovery, the model lineup, providers, parallel execution, reporting and the command line.