Chapter 15: Performance and cost

Spending tokens well

Deciding what is worth putting in front of the model. Every turn sends the whole conversation again, so a big log read early isn't paid for once. You pay for it again on each turn until the session ends.

The agent decides what to read, and it has no reason to care what that costs you. Long test output, whole files and old conversation all ride along on every later turn. Trimming these saves money, and leaves room for the code. Chapter 7 is about what that does to quality. This is the bill.

Say it to your agent

“Don't read the whole test log. Run bun test 2>&1 | tail -30, and if a test failed, grep for its name. Use a subagent for the codebase search and bring back only the file paths. Then /usage, and tell me what used the most.”

The workshop

Name a word, or describe what you mean in everyday language.