That is the speed half. Agents add a second thing to measure, in dollars, and everything above about measuring applies to it too.
Cost per task
What it really costs to get one piece of work finished: every try, every check and the failures you paid for and threw away. The price per token is only a price list. This is the number you can compare.
Comparing models on price per token is like comparing cars by the cost of a litre of fuel. A cheap model that takes three tries can lose to a dear one that takes one, and a wrong result that nobody catches can cost more than all of them. Measure it on your own tasks.
“Run these 10 tasks from git log on each candidate with claude -p --output-format json. Record total_cost_usd, the number of tries, and whether our test suite passed. Give me a table of cost per passing task, not cost per run.”