Chapter 5: The model

Token

The unit a model reads and writes: a chunk of text such as a short word, part of a long word, a space plus a word, or a single symbol. Limits are counted in tokens, and so is what you pay.

A 3,000-line log that an agent reads costs real money, and it takes up room that the code could have had. Thinking in tokens is how you see that before it happens.

Say it to your agent

“Before you read that log into context, count it with wc -c. If it is over 100 KB, grep for the failing test name instead and read only the lines around it.”

The workshop

Name a word, or describe what you mean in everyday language.