Chapter 5: The model

Jagged frontier

Models do some tasks very well and others badly, and the boundary is not where you would guess: two tasks that seem equally hard can fall on opposite sides. The output looks equally polished on both.

A general feeling that "AI is good at this" doesn't transfer safely from one task to the next. Keep a record of where your agent has been wrong on your code, and check hardest there.

Say it to your agent

“You have got concurrency wrong in this repo before. Write a failing test that reproduces the race before changing any code, and I will review this diff line by line.”

The workshop

Name a word, or describe what you mean in everyday language.