Choose for the Task, Not the Leaderboard

A model leaderboard can help discover candidates, but it is not a decision by itself. General benchmark scores may not reflect your language, terminology, response format, input length, or tolerance for mistakes. A small model that follows your narrow classification rules reliably may be more useful than a larger model that performs well on unrelated benchmarks.…