Chapter 18: The proof

Those four ideas are enough to read the study an interviewer is most likely to bring up.

METR's studies

Studies by METR, a non-profit, timing experienced open-source developers on their own real tasks, with chance deciding whether AI was allowed. The 2025 one found them slower. The 2026 follow-up says it couldn't produce a reliable new number.

If you say you build with agents, this is the study they will ask about, and you can get it wrong in two directions. "AI makes developers slower" overstates a trial of 16 people on early-2025 tools. "METR now shows AI speeds you up" overstates a follow-up METR itself distrusts.

Say it to your agent

“Open metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ and metr.org/blog/2026-02-24-uplift-update/. Draft two sentences I can say in an interview about what they do and don't show, with the sample, the date of the tools and the intervals. Mark any number you didn't read on those pages.”

The workshop

Name a word, or describe what you mean in everyday language.