The model

A program that takes text and outputs a probability for each piece of text that could come next. One piece gets picked and added to the text, and the step repeats. Everything it appears to do is built out of that step.

An agent's code, plans and progress reports are all output of this step. Reading well only shows that the text is likely, so whether it is correct has to be settled by something else.

The unit a model reads and writes: a chunk of text such as a short word, part of a long word, a space plus a word, or a single symbol. Limits are counted in tokens, and so is what you pay.

A 3,000-line log that an agent reads costs real money, and it takes up room that the code could have had. Thinking in tokens is how you see that before it happens.

The program that cuts text into tokens before the model sees it and joins tokens back into text afterwards. Its vocabulary is fixed in advance, so the same text can count differently on different models.

It explains costs that look strange: deep indentation, long names in code, encoded data such as base64, and languages that were rare in its training text all use tokens differently from the English you estimated with.

Adjusting a model's numbers over a huge amount of text until it gets better at predicting what comes next. When training ends the numbers are frozen and shipped. Using the model doesn't change them.

Correcting an agent teaches the model nothing. Your correction is text in one conversation, and when that ends it is gone. For a lesson to reach the next session, it has to be written where the next session will read it.

The date after which a model's training text has little or nothing. Nothing tells it that time has passed, so it describes old versions and old APIs in the present tense unless the prompt says otherwise.

Agents reach for the version of a library that was common in their training text. The fix is cheap: make it read the installed version's types or docs instead of recalling them.

Running a trained model to get output. Each request is worked on separately, and the model itself carries nothing over from one to the next, so it starts every request knowing only what that request contains.

The whole conversation is sent again on every turn, each round of asking and answering, because it is the only way the model can see it. So long sessions cost more per turn, and a fresh session has no idea what you did this morning.

The most tokens one request can involve: instructions, the conversation so far, files read, and the reply. What is outside it can't affect the output, any more than a program can use a file it never opened.

Agents fill windows quickly by reading files and logs. What happens next is decided by the program running the model, and the first thing to go may be the rule you set at the start.

The part of a request set by whoever built the tool rather than typed by the user: the role, the rules, how to use tools. It is still text in the window, read as tokens like everything else.

It makes an instruction more likely to be followed, and that is all. The model can still do the other thing, so a rule that must always hold belongs in code that checks it.

Everything sent to the model in one request: system prompt, tool definitions, earlier messages, file contents and your latest words. In everyday speech it means only what you just typed, which is usually a small part.

Most of what shapes an agent's output isn't what you typed this turn. When output goes wrong, look at the whole of what was sent before you reword your sentence.

Choosing the next token from the model's odds, by weighted dice. Temperature stretches or flattens the odds: low favours the likeliest token and high gives rare ones a chance. Top-p and top-k leave unlikely tokens out of the draw.

It is one reason a retry sometimes works, and why one success proves little. Advice to "set temperature to 0 for code" is also out of date on Claude's current API version, where the setting is gone.

The same input can give different output. Dice are the obvious cause. Even with the dice removed, a hosted model can vary, because of details such as how requests get batched together on the server.

An agent's success at a task is a rate, not a yes or a no, so one good run shows little and so does one bad run. Whatever must hold every time has to be checked by something that gives the same answer every time.

Output that reads as fact and isn't: a package, function, flag or citation that doesn't exist. It is produced by the same step as correct output, so nothing in the text marks it, and confident wording tells you nothing.

Agents act on it. They import packages and call methods that were never there. Type checks and installs catch a lot of this, but reading alone won't. Invented package names also give attackers something to register.

A tendency to produce what the person seems to want rather than what is true: backing down when challenged, praising weak work, confirming your guess about a bug.

Your question steers the answer. "That's the bug, right?" invites a yes, and a correct fix can get abandoned because you pushed back. Ask in a way that leaves room for you to be wrong.

An account a model gives of its own behaviour, written as new text rather than read from a record. It may be right or wrong, and it reads the same either way. Asking "why did you do that" gets you a plausible story.

When an agent breaks something, its explanation isn't a log. Read what it actually did: the actions it asked for (tool calls, coming next), the results and the diff. Treat "why" as a source of hypotheses, then test them.

Structured output asking for a named action with arguments, such as read_file on config.json. The model runs nothing. The program around it, which the next chapter calls the harness, decides whether to run it and sends back the result as text.

All real power sits in that program, so that is where the safety controls go, as the next chapter shows. It also means the model works from whatever comes back, including instructions hidden in a file it read.

Tokens a model produces before its answer, working a problem through, which tends to help on hard tasks. Effort is the setting that controls how much of this happens. More effort costs more and takes longer.

Effort is a dial to try before changing model. Thinking is billed at the output price, whether you can see it or not, and it isn't a trustworthy record of why the model did something.

The provider keeps the processed start of your request for a few minutes. If your next request begins with exactly the same bytes, that part costs a fraction of the price. One changed byte invalidates everything after it.

For long agent sessions this usually matters more than any wording trick. A timestamp at the top of a system prompt breaks every hit, and because saving to the cache costs extra, you then pay more than with no caching at all.

Providers sell a ladder of models at very different prices. Higher rungs cost more per token, tend to be slower and tend to do better on hard work. The names change often, so check the current list rather than remembering one.

Using the top rung to rename a variable is waste, and the cheapest rung on a hard task can cost more in retries than getting it right once. Judge a tier by its cost per finished task, on your tasks.

What a request costs: the tokens of each kind multiplied by that kind's price per million. Input, cache writes, cache reads and output are all priced separately, and output is the most expensive, five times input on current Claude models.

An agent's bill grows with the length of the session, not only the size of the task, since every turn carries everything before it. Starting a fresh session is a cost decision as well as a quality one.

A list of numbers standing for a piece of text, produced so that texts with related meanings get similar numbers. Compare two lists and you get a measure of how related the texts are, even if they share no words.

It is what makes search by meaning possible, and you will meet it in tools that search documents on a model's behalf. It isn't automatically better than plain search: for an exact function name, a text search is faster, cheaper and right.

A fixed set of tasks that models are scored on so they can be compared. SWE-bench is the best known for coding: given a real bug report from GitHub and the project's code, the model has to produce a change, and tests decide whether it worked.

A score tells you how a model did on those tasks under those tests. The tests can be wrong, the answers may have been in the training text, and your code base isn't in it. Treat scores as a shortlist, then test on your own work.

Models do some tasks very well and others badly, and the boundary is not where you would guess: two tasks that seem equally hard can fall on opposite sides. The output looks equally polished on both.

A general feeling that "AI is good at this" doesn't transfer safely from one task to the next. Keep a record of where your agent has been wrong on your code, and check hardest there.

Training a copy of a model further on your own examples, so that it leans towards your format or style. It changes tendencies. It isn't a way of giving a model your code base, which changes daily anyway.

People reach for it to teach an agent about their project. Instructions and search almost always do that better: they cost little, can be edited in a minute, and are in the repo where you can review them.

A field on each response saying why the model stopped: it finished, hit your token limit, wants a tool run, or declined. It arrives on a success, so code that only reads the text can treat half an answer as a whole one.

An agent that saves a file from a cut-off response saves half a file, with status 200 and no error. Whatever drives a model has to check this field before it uses the content.