Chapter 17: Many agents

Evaluator-optimiser

One side writes and the other judges, in a loop. The judge's notes go back to the writer until it passes or the tries run out. It works when the judge has clear criteria, and the result ends up as good as what the judge can see.

Agents grade their own work too kindly. Splitting the writer from the judge fixes more than a stricter prompt does, and an automatic loop saves you from carrying the notes yourself. A blind spot in the judge stays a blind spot, however many rounds you run.

Say it to your agent

“Write this, run ./check.sh, and read what it says. Fix what it reports, and run it again. Stop after four tries, or when it passes, and show me the full output of the last run either way.”

The workshop

Name a word, or describe what you mean in everyday language.