# Harness: the vocabulary of building software with agents

526 terms in 18 chapters. Each has a plain definition, what it is confused with, where it really shows up, and a sentence you could say to an agent. The working instruments are at https://dev.dqnotes.com.

## 1. The machine

What a program is and what happens when one runs, from the first line of code to the terminal.

### Source code
The text of a program, written by a person or an agent and saved in ordinary files. It is a precise set of instructions. The computer does what the text says, which is not always what its author meant.
Source code is what an agent hands you and what you answer for. It can read perfectly sensibly and still be wrong, so you judge it by what it does when it runs, not by how confident it sounds.
Not Program: Source code is the text. A program is that text in a form a computer can carry out.
Seen in: On any project on GitHub, the files ending in .ts, .py or .go are source code. They are plain text, and you can open them in any text editor.
Say it: "For each file you changed, show me the lines that changed and tell me what they do when they run. Don't summarise what you meant them to do."
Described as: "the text a program is written in", "the actual code behind an app", "what the agent is writing when it writes code".
https://dev.dqnotes.com/machine/source-code

### Program
A set of instructions a computer can carry out. Your browser is a program, and so is the agent you talk to. A program sits in a file doing nothing until something starts it.
What an agent can really do is start programs: ones that run tests, ones that install things, ones that delete. Each does exactly what it does, whatever the agent intended, so a command deserves more of your attention than a sentence.
Say it: "Before you run that program, tell me in one sentence what it does and what it changes on my machine. If it deletes or overwrites anything, stop and ask me."
Described as: "software, an application", "the thing that runs on the computer", "what is the difference between code and an app".
https://dev.dqnotes.com/machine/program

### Files and folders
Also: Directory.
A file is a named chunk of data kept on a computer. A folder, also called a directory, is a named container for files and other folders, so everything on the machine forms one big tree.
A code project is only a folder of files, and nearly everything an agent does is read, change, create or delete them. It can touch dozens in a minute, so you need to know the tree well enough to notice when it strays outside the job.
Not File extension: The .txt or .png on the end is part of the name: a hint for people and programs, not what the file is. Rename photo.png to photo.txt and the contents are still a picture.
Seen in: A name starting with a dot is hidden from a plain ls. Projects keep a lot of their settings that way, in files and folders such as .gitignore, .env and .claude. To see them, use ls -a.
Say it: "List the folders you plan to change before you start. Stay inside src/checkout/, and tell me first if you need to touch anything outside it."
Described as: "where things are kept on the computer", "what is a directory", "hidden files I can't see", "what is the .env file and why can't I see it".
https://dev.dqnotes.com/machine/file-and-folder

### Code editor
Also: Text editor, Editor, IDE, VS Code.
A program for opening and changing plain text files, made for code. It colours the text so the structure stands out, shows the project's folders down one side, and marks some mistakes as you type. VS Code is a common one.
You need somewhere to read what an agent wrote that isn't the agent's own account of it. Open the file it says it changed and you are looking at the thing itself, in the same folders it is working in.
Not Word processor: Word and Google Docs save fonts and layout along with your text, and a program can't run that. A code editor saves exactly the characters you see.
Seen in: Microsoft says Visual Studio Code "combines the simplicity of a source code editor with powerful developer tooling", and that it "supports macOS, Linux, and Windows". An IDE, or integrated development environment, is an editor with more of those tools built in. Many editors can also show a terminal in a panel beside your code.
Source: VS Code docs, Why VS Code, https://code.visualstudio.com/docs/editor/whyvscode
Say it: "List every file you changed as a path I can open in my editor. Don't reformat anything you weren't asked to change, so that I can see the real edit."
Described as: "where do I open code", "what do programmers type code into", "can I use Word to write code", "what is VS Code", "what is an IDE".
https://dev.dqnotes.com/machine/code-editor

### CPU and memory
Also: RAM, Processor.
The CPU is the part that carries out instructions, one tiny step at a time, very fast. Memory, or RAM, is the quick short-term space where a running program keeps its instructions and the values it is working on.
Agents test on small examples, so they often write code that loads a whole file into memory because that worked on the sample. At real size it runs out and gets killed. You will notice it as a loud fan, a crash, or a process that vanished.
Not Disk: Memory is fast and is wiped when the power goes. Disk is slower and keeps things. People say memory for both, which causes real confusion.
Say it: "This reads the whole log file into memory, and say our logs reach 4 GB. Rewrite it to read one line at a time, then run it under /usr/bin/time -l and show me the peak memory."
Described as: "my laptop fan spins up", "it ran out of memory", "what is RAM", "the part that makes a computer fast".
https://dev.dqnotes.com/machine/cpu-and-memory

### Disk
Also: Storage, SSD.
Long-term storage, where files stay when the power goes off. It is much slower than memory. A program that only keeps data in memory loses it when it stops, crashes or the machine restarts.
An agent reaches for whatever makes the demo work, and that can be a list in memory. It passes every test. Then your first restart wipes all your users. So always ask where each piece of data lives.
Not Saved: Saving usually hands data to the operating system, which may hold it for a while before it reaches the disk. Software that must not lose data, such as a database (a program whose job is keeping data safe), forces the write.
Seen in: The Linux manual on write, the call behind saving: "A successful return from write() does not make any guarantee that data has been committed to disk." A separate call, fsync, exists so that "all changed information can be retrieved even if the system crashes or is rebooted."
Source: Linux man-pages, write(2), https://man7.org/linux/man-pages/man2/write.2.html
Source: Linux man-pages, fsync(2), https://man7.org/linux/man-pages/man2/fsync.2.html
Say it: "You stored sessions in a module-level Map, so they vanish on every restart. Store them in the database, then prove it: log in, restart the server, and show me that the session still works."
Described as: "everything disappeared when I restarted", "I saved it but it was gone", "what survives a reboot", "storage versus memory".
https://dev.dqnotes.com/machine/disk

### Process
Also: PID.
One running copy of a program, with its own private memory and an ID number called a PID. Start a program twice and you have two processes that know nothing about each other. Each runs until it finishes or something stops it.
Agents start servers and leave them running, then start them again. The leftovers keep hold of resources and go on serving old code, so you can lose an hour debugging a change that isn't even the thing running.
Not Program: The program is the file on disk. A process is one run of it. One program can have many processes at once.
Seen in: This site's own AGENTS.md warns agents that its dev server may already be running and not to start a second one. The script passes --strictPort, so a second copy fails loudly instead of quietly moving to another port. Control-C stops whatever is running in the terminal, and kill stops a process by its PID.
Say it: "Before you start the dev server, run lsof -i :3366 to see if one is already running. If it is, tell me its PID and don't start another. Stop anything you started when you finish."
Described as: "port already in use", "address already in use", "it is still running in the background", "my change is not showing up", "how do I stop a program that is stuck".
https://dev.dqnotes.com/machine/process

### Terminal
Also: Command line, Console, CLI.
A window where you control a computer by typing instead of clicking, and it answers in text. It looks old-fashioned, but nearly every tool a programmer uses can be driven from it.
Text is what an agent reads and writes, so the terminal is its natural home: commands go in, results come out. Anything you can do here, it can do. Reading its session is how you find out what it really did.
Not Shell: The terminal is the window. The shell is the program inside it that understands what you type.
Seen in: To open one: on a Mac, press Command and Space and type Terminal. On Windows, open Windows Terminal. On Linux, it is usually called Terminal too. Type pwd and press Return. Claude Code is itself a program that runs in a terminal, and its Bash tool runs each command in a separate process.
Source: Claude Code docs, Tools reference, https://code.claude.com/docs/en/tools-reference
Say it: "Run these in the terminal one at a time: pwd, ls, git status. Paste exactly what each one printed, with nothing summarised."
Described as: "the black window with text in it", "where you type commands", "the hacker screen", "what is a CLI", "the command line".
https://dev.dqnotes.com/machine/terminal

### Shell
Also: bash, zsh.
The program inside the terminal that reads your line, works out what you mean, starts the right programs and shows you the result. Common ones are bash and zsh. It can also join programs together, and that is most of its power.
An agent's command goes to a shell first, with all its rules, and a stray space or a * means something to it. Those rules are why one wrong character can delete the wrong thing, and why a short line can do far more than it appears to.
Not bash and zsh: Two shells, nearly the same for daily use. A script that runs under one can still behave differently under the other, so say which one you mean.
Seen in: Apple says that since macOS 10.15 "your Mac uses zsh as the default login shell and interactive shell". GitHub Actions runs scripts on Linux and macOS with bash by default. So the same line may meet two different shells on its way to production.
Source: Apple Support, Use zsh as the default shell on your Mac, https://support.apple.com/en-us/102360
Source: GitHub Actions docs, Workflow syntax, https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax
Say it: "This has to work on my Mac and in CI. Start the script with #!/usr/bin/env bash, write it for bash, and run it once with bash explicitly before you tell me it works."
Described as: "what actually reads what I type", "what is bash", "what is zsh", "difference between shell and terminal".
https://dev.dqnotes.com/machine/shell

### Command
Also: Flag, Argument, Option.
One line you give the shell: the name of a program, then flags that change how it behaves, then arguments saying what to work on. In rm -r build, rm is the program, -r is a flag and build is the argument. Spaces separate the pieces.
When an agent asks permission to run something, the command is what you are approving. Read it left to right: which program, which flags, which targets. Slow down for -f or --force, and for a * among the targets.
Not Prompt: A prompt is read by a model, which interprets it. A command is read by a shell, which does what it says without wondering whether you meant it.
Seen in: Agent permission rules follow this shape. In Claude Code, Bash(rm *) names a program and allows any arguments after it. To find out what a flag means, man rm opens the manual for rm.
Source: Claude Code docs, Hooks, https://code.claude.com/docs/en/hooks
Say it: "You want to run rm -rf dist build node_modules. Before I approve it, tell me what each of those folders holds, whether it can be rebuilt, and whether anything outside this project is affected."
Described as: "what do the dashes mean", "what does -rf mean", "my file name has a space and it broke", "the options after a command".
https://dev.dqnotes.com/machine/command

### Path
The written address of a file or folder. An absolute path starts from the top, as in /home/you/notes.txt, and means the same everywhere. A relative path, as in src/cart.js, starts from where you are. .. means the folder above and ~ means your home folder.
A relative path is right from one place only, and agents get lost in big projects. No such file or directory often means a correct path from the wrong folder. And an absolute path that works on your machine, an agent's favourite shortcut, works nowhere else.
Not The PATH variable: A path is one address. PATH, in capitals, is a list of folders the shell searches for programs. It comes up shortly.
Say it: "That path only works on my machine. Replace /Users/me/shop/data with a path relative to the project root, then run it from a different folder to prove it still works."
Described as: "no such file or directory", "file not found but it is right there", "what does ../ mean", "what does the squiggle ~ mean", "it only works on my machine".
https://dev.dqnotes.com/machine/path

### Working directory
Also: Current directory, cwd, pwd, cd.
The folder a shell is standing in right now. Relative paths start from here. pwd prints it and cd changes it. Every process has its own, so a cd inside a script doesn't move the shell you started it from.
A lot of agent trouble is the right command in the wrong folder: things installed at the wrong level of a project, or the wrong folder deleted. Agent tools also differ on whether a cd sticks between commands, so have it print where it is.
Not Project root: The root is the top folder of the project, and it stays put. The working directory is wherever you happen to be, which may be somewhere else.
Seen in: In Claude Code's Bash tool, as documented in November 2026, a cd in the main session carries over to later commands while it stays inside the project, and "Subagent sessions never carry over working directory changes."
Source: Claude Code docs, Tools reference, https://code.claude.com/docs/en/tools-reference
Say it: "Run this from apps/web, not the project root. Write it as cd apps/web && pwd && bun test, and paste all of the output."
Described as: "it ran in the wrong folder", "which folder am I in", "cd did not stick", "it installed in the wrong place".
https://dev.dqnotes.com/machine/working-directory

### Environment variable
Also: Env var, export.
A named setting a process carries with it, such as TZ=Europe/London for its time zone. A program can read it without it being in the code, and passes a copy to any program it starts. A child can't change its parent's copy.
This is how the same code uses a practice database on your laptop and the real one in production, the live copy that real users rely on. It is also where secrets such as passwords belong. And it is a trap for agents: "I set it" may mean it was set in a shell that has already ended.
Not .env file: A .env file is only a text file of these settings. Nothing happens until some program reads it, so having the file doesn't set anything in your shell.
Seen in: Claude Code's documentation on its Bash tool: "Environment variables don't persist. An `export` in one command won't be available in the next." It also sets CLAUDECODE to 1 in the programs it starts, so a script can tell when an agent is running it.
Source: Claude Code docs, Tools reference, https://code.claude.com/docs/en/tools-reference
Source: Claude Code docs, Environment variables, https://code.claude.com/docs/en/env-vars
Say it: "Read the key from process.env.PAYMENT_API_KEY and fail at startup with a clear message if it is missing. Add the name to .env.example with no value, and never print it in a log."
Described as: "where does the API key go", "works on my machine but not on the server", "I set it but the program can't see it", "what is a .env file", "what does export do".
https://dev.dqnotes.com/machine/environment-variable

### The PATH variable
Also: $PATH.
An environment variable holding a list of folders. When you type a program's name, the shell looks through those folders in order and runs the first match. If none has it, you get command not found, even when the program is on the machine.
Command not found is what an agent often meets straight after installing something, and the usual cause is a folder missing from this list. It also explains a command running the wrong version: another copy sits earlier in the list.
Not Not installed: Command not found only means it isn't in those folders. It may be installed somewhere else. which node shows you what is being found, if anything.
Seen in: The Bash manual: "Bash searches each element of `$PATH` for a directory containing an executable file by that name." npm puts a project's own tools in node_modules/.bin and adds them to PATH while running its scripts, which is why a tool can work through npm run and not when typed.
Source: GNU Bash manual, https://www.gnu.org/software/bash/manual/bash.html
Source: npm docs, Scripts, https://docs.npmjs.com/cli/v11/using-npm/scripts
Say it: "You got command not found for tsc. Don't install it globally. Run which tsc and echo $PATH, then use bunx tsc or a script in package.json, so that it runs this project's version."
Described as: "command not found", "I installed it but it says not found", "it runs the wrong version", "what does adding something to my path mean".
https://dev.dqnotes.com/machine/path-variable

### Exit code
Also: Exit status, Return code, $?.
The number a program hands back when it ends. 0 means it went fine, and anything else means it didn't. You rarely see it, but the shell keeps the latest one in $?. Automatic checks in software rest on this.
An agent, or any automatic check, decides whether to carry on from this number, not from the words on screen. A program can print success and exit 1, or print nothing and exit 0. When you want proof that something passed, ask for the code.
Not Error message: A message is words for people and can say anything. The exit code is what a check acts on. When the two disagree, believe the code.
Seen in: In Bash, 127 means the command wasn't found, 126 means it was found but isn't executable, and a program killed by signal N gives 128+N. Not every non-zero is a disaster: grep exits 1 when no lines match and 2 on a real error, and Claude Code's Bash tool treats exit 1 from grep, find, diff and test as a valid result. In Claude Code hooks, only 2 blocks.
Source: GNU Bash manual, https://www.gnu.org/software/bash/manual/bash.html
Source: GNU grep manual, Exit status, https://www.gnu.org/software/grep/manual/grep.html
Source: Claude Code docs, Tools reference, https://code.claude.com/docs/en/tools-reference
Source: Claude Code docs, Hooks, https://code.claude.com/docs/en/hooks
Say it: "Run bun test, then echo $? on its own line, with nothing piped. Show me the number. Don't tell me the tests passed unless it is 0."
Described as: "how does it know a command failed", "exit 1", "it said success but it failed", "error code 127", "what does dollar question mark mean".
https://dev.dqnotes.com/machine/exit-code

### stdin, stdout and stderr
Also: Standard streams, Standard output, Standard error.
Three channels every process starts with. Stdin is what comes in. Stdout is its results. Stderr is for errors and remarks. On screen, stdout and stderr look alike, but they are separate and can each be sent somewhere different.
An agent reads these the way you read a screen. The split lets a program pass clean results onward while its complaints still reach someone. Agent hooks, scripts that run at set moments in an agent's work (chapter 6), speak this too: they get their data on stdin and answer on stdout or stderr.
Seen in: A Claude Code hook receives JSON on stdin. If it exits 0, its stdout is parsed for instructions. If it exits 2, its stderr is fed back to the agent as the reason it was blocked.
Source: Claude Code docs, Hooks, https://code.claude.com/docs/en/hooks
Say it: "Send only the JSON result to stdout, and every progress and error message to stderr, so that mytool | jq . keeps working when something goes wrong."
Described as: "where does the error text go", "I saved the output to a file but errors still showed", "the input and output of a program", "difference between output and error output".
https://dev.dqnotes.com/machine/standard-streams

### Redirection
Also: 2>&1, /dev/null.
Sending a stream somewhere other than the screen. > writes stdout to a file, replacing what was there, and >> adds to the end. 2> does the same for stderr. 2>&1 sends stderr wherever stdout is going. /dev/null throws away whatever reaches it.
Agent commands are full of this. 2>&1 is there so errors get captured too. > /dev/null 2>&1 means nobody will see anything, including failures. Read the end of a long command first, because that is where output gets hidden.
Not Pipe: A redirection sends a stream to a file. A pipe sends it to another program.
Seen in: The Bash manual: "The order of redirections is significant." Its example: ls > dirlist 2>&1 sends both streams to the file, while ls 2>&1 > dirlist "directs only the standard output to file dirlist."
Source: GNU Bash manual, https://www.gnu.org/software/bash/manual/bash.html
Say it: "Don't send that to /dev/null. Run it with > build.log 2>&1, then show me the exit code and the last 30 lines with tail -30 build.log."
Described as: "what does 2>&1 mean", "save the output to a file", "what is dev null", "the greater-than sign in a command", "hide the output".
https://dev.dqnotes.com/machine/redirection

### Pipe
Also: Pipeline, pipefail.
The | character. It feeds one program's stdout into the next program's stdin, so small tools combine: grep ERROR app.log | sort | uniq -c finds lines, sorts them and counts repeats. By default, the whole line reports only the last program's exit code.
It is how an agent reads without drowning: tail keeps only the last lines, so a long test run costs it far less to read. But bun test | tail -20 reports how tail went, so failing tests look fine unless a setting called pipefail is on. An agent can fool itself this way, and so can you.
Not &&: && runs the next command only if the last one worked, and passes nothing along. A pipe runs them together and passes output along, whether or not the first one worked.
Seen in: With pipefail, the Bash manual says a pipeline returns "the value of the last (rightmost) command to exit with a non-zero status". GitHub Actions shows why it pays to check: a step marked shell: bash runs with -eo pipefail, and a step with no shell set runs with only -e.
Source: GNU Bash manual, https://www.gnu.org/software/bash/manual/bash.html
Source: GitHub Actions docs, Workflow syntax, https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax
Say it: "Run this as set -o pipefail; bun test 2>&1 | tail -30. I want the last 30 lines and the real exit code of the tests, not of tail."
Described as: "the vertical bar in a command", "feed the output into another command", "tests failed but it says success", "keep only the last few lines".
https://dev.dqnotes.com/machine/pipe

### File permissions
Also: rwx, chmod, sudo, root.
Rules saying who may read, write and run each file. ls -l shows them as -rwxr-xr--: three letters for the owner, three for a group, three for everyone else. chmod changes them. root, the administrator account, ignores most of them.
A script an agent writes can't run until it is marked executable, so Permission denied usually calls for chmod +x, not sudo. Be wary when an agent reaches for sudo or chmod 777. They don't fix the problem. They remove the protection that reported it.
Not Agent permissions: File permissions are enforced by the operating system on every program. An agent's permission settings are the agent's own software deciding which actions to allow. Either can refuse, separately.
Seen in: The example hook script in Claude Code's docs is made executable with chmod +x before use. Bash returns 126 when "a command is found but is not executable". sudo's own manual describes it as "execute a command as another user", and that user is usually root.
Source: Claude Code docs, Hooks, https://code.claude.com/docs/en/hooks
Source: GNU Bash manual, https://www.gnu.org/software/bash/manual/bash.html
Source: sudo manual, https://www.sudo.ws/docs/man/sudo.man/
Say it: "Permission denied on scripts/deploy.sh means it isn't executable. Run chmod +x scripts/deploy.sh and commit that. Don't use sudo or chmod 777. If you think you need to, stop and ask me."
Described as: "permission denied", "what does chmod do", "what does rwx mean", "what does sudo mean", "should I use sudo", "the file is there but it won't run".
https://dev.dqnotes.com/machine/file-permissions

### Programming language
A strict, made-up notation for writing instructions. Python, JavaScript, TypeScript, Go and Rust are all programming languages. Each has exact rules, so a missing bracket is not a typo to read past. It is a program that won't run.
An agent can write any of them, so you choose by how cheaply its mistakes get caught. A language with strict checks gives an agent an automatic no. If you don't say which language, the agent will pick, and you will own that choice.
Not Framework: A language is the notation. A framework is a large body of code, written in a language, that you build on. Django is written in Python.
Say it: "Write this in TypeScript with strict mode on, not JavaScript. Match the language and versions already used in this repo, and tell me before adding a new language to it."
Described as: "Python or JavaScript", "what language should I use", "what is this written in", "coding language".
https://dev.dqnotes.com/machine/programming-language

### Compiler
Also: Compile, tsc.
A program that reads all your source code and translates it into a form a machine can run. If anything is broken, it refuses and produces nothing. Go, Rust and C are compiled. TypeScript's compiler, tsc, checks each value is the right kind and writes JavaScript.
A compiler is a free no. It arrives in seconds, before anything has run, in a form an agent can read and fix, so a compile step is one of the cheapest checks you can put in an agent's way.
Not Interpreter: A compiler finds problems before anything runs. An interpreter finds them when it gets there.
Seen in: tsc --noEmit runs TypeScript's compiler only for its checks and writes no files. This site's typecheck script uses exactly that, and it has to stay clean.
Say it: "Run bun run typecheck. Fix every error by changing the code, not by adding any or @ts-ignore. Paste the last lines of output and the exit code."
Described as: "a build error before it even runs", "translating code into what the machine runs", "it won't compile", "what does tsc do".
https://dev.dqnotes.com/machine/compiler

### Interpreter and runtime
Also: Runtime, Node.js.
An interpreter runs source code directly, working through it as it goes, so many problems only appear when reached. A runtime is what a program needs around it while running. Python, Node.js and Bun are examples. The line with compilers is blurry.
With no compile step, code that never ran is code nothing has checked. An agent can show you a working demo while a misspelt name waits on a path through the code it never took. So running the script once proves that one path. Tests have to go down the others.
Not Compiler: Real systems mix the two. Python compiles to an internal form first, and Node.js can run TypeScript by removing the types, without checking them.
Seen in: The Python glossary: "Python is an interpreted language, as opposed to a compiled one, though the distinction can be blurry because of the presence of the bytecode compiler." Node.js has run TypeScript by default since v23.6.0 and v22.18.0, and its docs say "no type checking is performed", so node app.ts running tells you nothing about its types.
Source: Python docs, Glossary, https://docs.python.org/3/glossary.html
Source: Node.js docs, Modules: TypeScript, https://nodejs.org/api/typescript.html
Say it: "This is Python, so a bad name in that error branch only fails when the branch runs. Add a test that forces the payment to fail and goes through that branch, and show me it passing."
Described as: "it only crashed when it got to that line", "it worked in the demo then broke later", "what is Node", "what does runtime mean", "runtime error".
https://dev.dqnotes.com/machine/interpreter

### Package manager
Also: npm, Registry.
A program that downloads code other people have published, from a public store called a registry, and keeps track of what you use. npm, pnpm and Bun do this for JavaScript, and pip and uv for Python. You run it in the terminal like anything else.
An install puts strangers' code on your machine, and some package managers let it run scripts as it installs. An agent types install as easily as ls, and it can name a package that doesn't exist, which an attacker could then register.
Seen in: Bun's docs: "Bun does not execute arbitrary lifecycle scripts by default, unlike other `npm` clients." In a USENIX Security 2025 paper, Spracklen and colleagues found that code from commercial models named non-existent packages at an average rate of at least 5.2%. That measures made-up names, not victims.
Source: Bun docs, Lifecycle scripts, https://bun.com/docs/pm/lifecycle
Source: Spracklen et al., We Have a Package for You, 2025, https://arxiv.org/abs/2406.10279
Say it: "Before you add a package, show me its registry page, its last publish date and why we need it. Say whether we could write it ourselves in under 20 lines. Install with bun add, with no --force and no sudo."
Described as: "npm install", "how do I get a library", "bun add", "pip install", "where does code I didn't write come from".
https://dev.dqnotes.com/machine/package-manager

### Dependency
Also: Transitive dependency, node_modules, Package.
Code your project relies on but didn't write. The ones you name, in a file called package.json for JavaScript, are direct. They have dependencies of their own, and those have theirs, so these transitive ones are usually most of what you install, into a folder called node_modules.
Each one is code you now run without having read it, from people you don't know. Agents add them readily, because that solves today's problem. The cost arrives later, as updates, size and exposure. So ask for a reason each time.
Not Dev dependency: A dev dependency is only needed while building and testing, such as a tool that runs tests. It isn't shipped to users, but it still runs on your machine.
Seen in: In March 2016 an author unpublished 273 packages from npm, one of them left-pad. npm wrote that this "impacted many thousands of projects", and that "The duration of the disruption was 2.5 hours." On a smaller scale, this site's package.json names 21 packages and, when this was written, its lockfile had about 290 entries, including builds for other platforms.
Source: npm blog, kik, left-pad, and npm, 2016, https://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm
Say it: "Don't add a dependency for this yet. Tell me what it would do for us and how many packages it brings, using bun pm ls --all before and after. If the job is under 20 lines, write it here with a test."
Described as: "why is node_modules so huge", "I installed one thing and it added hundreds", "what is a transitive dependency", "a library my app needs".
https://dev.dqnotes.com/machine/dependency

### Lockfile
Also: package-lock.json, bun.lock.
A file the package manager writes, recording the exact version of every dependency it installed and a fingerprint of its contents. It means everyone, and every automated run, installs the same code. Don't write it by hand, but do keep it in the project.
Without one, an unchanged project can install different code next week and break for no visible reason. Agents change it in passing, so look at what changed: a one-package request shouldn't rewrite most of the file. And make automated runs refuse a mismatch.
Not package.json: package.json says what you accept, often a range such as ^2.1.0. The lockfile says what you got, exactly, such as 2.1.3.
Seen in: npm says package-lock.json "describes the exact tree that was generated". Of npm ci it says: "It will never write to package.json or any of the package-locks: installs are essentially frozen." It exits with an error if the two disagree. Bun has written a text bun.lock by default since v1.2, and bun install --frozen-lockfile installs from it without changing it.
Source: npm docs, package-lock.json, https://docs.npmjs.com/cli/v11/configuring-npm/package-lock-json
Source: npm docs, npm ci, https://docs.npmjs.com/cli/v11/commands/npm-ci
Source: Bun docs, Lockfile, https://bun.com/docs/pm/lockfile
Source: node-semver, range rules, https://github.com/npm/node-semver/blob/main/README.md
Say it: "Only bun.lock should change for this, and only for that one package. Show me the lockfile diff, commit it, and make CI run bun install --frozen-lockfile so a mismatch fails the build."
Described as: "it worked yesterday and nothing changed", "works on my machine but not on the build server", "a huge diff in package-lock.json", "should I commit the lock file", "different versions on different machines".
https://dev.dqnotes.com/machine/lockfile

### Script
Also: Shell script, set -e, bun run.
A text file of commands that run in order, so what you would type by hand happens the same way every time. Projects also give short ones names in package.json, which is what bun run check means. A shell script has to be told to stop at a failure.
Scripts are how a fix outlives a working session: an agent runs the same check you did, or the same deploy (putting new code live for users), without improvising. But a plain shell script carries on after a failure and reports only its last line, so without set -e it can't really say no.
Not Program: The line is soft. A script is a program, usually a short one, run by an interpreter: here, the shell.
Seen in: GitHub Actions runs a step with no shell set as bash -e {0}, and with shell: bash as bash --noprofile --norc -eo pipefail {0}. Its docs warn: "Note that this runs a different command to when `bash` is specified explicitly." The same lines can fail differently because of one setting.
Source: GitHub Actions docs, Workflow syntax, https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax
Say it: "Put those steps in scripts/release.sh. Start it with #!/usr/bin/env bash and set -euo pipefail, and make it executable. Then make one step fail on purpose, so we can both see it stop."
Described as: "automate the commands I keep typing", "what is bun run", "what does set -e do", "it kept going after an error", "the hash and exclamation mark on the first line".
https://dev.dqnotes.com/machine/script

### Operating system
Also: OS, macOS, Linux, Kernel.
The program that runs the machine: it shares out the CPU and memory, keeps files on the disk, and starts and stops processes. macOS, Linux and Windows are operating systems. Programs ask it for everything, so a program built for one often won't run on another.
Your agent may run on a Mac while the code ships to Linux, and the two differ in small ways that pass on your laptop and then fail on a server. Agents write commands for one system out of habit, so tell yours what the target is.
Not Kernel: The kernel is the core that talks to the hardware. Operating system usually means the kernel plus the tools around it, such as the shell.
Seen in: We ran sed -i s/a/b/ x.txt on macOS 27.0.1 and got an error and exit 1, because the Mac's BSD sed expects an argument after -i, where the suffix is optional in GNU sed on Linux. touch CaseTest; ls casetest also found the file: Apple lists APFS (Case-sensitive) as a separate format. So an import of ./Button can find button.tsx on your Mac and fail on a Linux server.
Source: FreeBSD manual, sed, https://man.freebsd.org/cgi/man.cgi?query=sed
Source: GNU sed manual, https://www.gnu.org/software/sed/manual/sed.html
Source: Apple, Disk Utility file system formats, https://support.apple.com/guide/disk-utility/dsku19ed921c
Say it: "CI runs on Ubuntu and I am on a Mac. Don't use sed -i or anything else that differs between them: do this edit in a small Node script, and check that every import matches its file name's exact capitals."
Described as: "works on my Mac but not on the server", "Mac versus Linux", "what does the OS actually do", "file name capitals broke the build", "sed gives an error on my Mac".
https://dev.dqnotes.com/machine/operating-system

### Binary and bytes
Also: Bit, Byte, Executable.
Computers store everything as bits, each one 0 or 1. Eight bits make a byte, which can hold 256 different values. Text, pictures and programs are all runs of bytes, and only the reader decides what they mean.
Agents and people both read text far more easily than anything else, and neither of you can readily check a compiled program. So you review and keep the source, and build the rest from it.
Not A binary: Confusingly, "a binary" also means a compiled program ready to run. Base two is the idea, and a binary is the file. Same word, and context tells you which.
Seen in: Files often announce themselves in their first bytes. Every PNG starts 89 50 4E 47 0D 0A 1A 0A. Linux programs start with 7F and then the letters ELF. We checked a Node.js program on a Mac with xxd, and it starts CF FA ED FE.
Source: W3C, PNG specification, https://www.w3.org/TR/png-3/
Source: Linux man-pages, elf(5), https://man7.org/linux/man-pages/man5/elf.5.html
Say it: "Don't commit the built bundle or any other binary. Add dist/ to .gitignore, and commit the source and the command that builds it, so the result can be reproduced and reviewed."
Described as: "ones and zeros", "what is a byte", "the file is gibberish when I open it", "what is a binary", "kilobytes and megabytes".
https://dev.dqnotes.com/machine/binary

### Text encoding
Also: UTF-8, Unicode, Mojibake.
The rule for turning characters into bytes and back. UTF-8 is the usual one today: English letters take one byte, é takes two and 日 takes three. Read with the wrong rule, the same bytes come out as garbage, such as café for café.
Agents tend to test with plain English, so these bugs wait for your first customer called José. Bytes and characters also differ, so a limit of 100 bytes can cut a character in half. Ask for a test with accents in it.
Not Unicode: Unicode is the numbered list of characters: é is U+00E9. UTF-8 is one way of writing those numbers as bytes.
Seen in: RFC 3629 defines UTF-8: characters "are encoded using sequences of 1 to 4 octets", an octet being a byte, and the first 128 characters keep their old single-byte values. That compatibility is why English-only tests can't reveal a mistake.
Source: RFC 3629, UTF-8, https://www.rfc-editor.org/rfc/rfc3629
Say it: "Add a test that stores and reads back the name José Ñandú 日本 through the whole path: form, API, database and page. Set the database and the HTTP charset to UTF-8, and show me that it comes back identical."
Described as: "weird characters like é in my text", "accents are broken", "question marks in diamonds", "what is UTF-8", "an emoji broke my text", "garbled text".
https://dev.dqnotes.com/machine/encoding

### Your first agent
Also: Installing an agent, Getting started.
A coding agent is a program you install and then start in a terminal, from inside a project's folder. It reads and changes what is there and runs commands, usually asking you first. A project is only a folder, so a new empty one will do.
From here on, the course assumes you have one, because its drills, the exercises at the end of each chapter, happen on your machine and not on this page. Start it in a folder holding nothing you would miss, and read each command before you approve it.
Not A chat window: A chat in a browser tab hands you text, and you decide what to do with it. An agent on your machine acts on your files itself, so what it can reach matters from the first minute.
Seen in: Claude Code's quickstart, read 5 November 2026: on a Mac or Linux, paste curl -fsSL https://claude.ai/install.sh | bash into a terminal, open a new terminal, and check it with claude --version. Windows has its own line on that page. It needs a paid Claude plan or a Claude Console account: the free plan doesn't include it, and on 6 November 2026 the cheapest that does, Pro, was $20 a month. Then cd into your folder and type claude. That install line downloads a script and pipes it into bash to run, so take it from the vendor's page and nowhere else. Codex and Cursor have their own install pages. Chapter 3 needs git too: on a Mac, git --version offers to install it if it is missing.
Source: Claude Code docs, Quickstart, https://code.claude.com/docs/en/quickstart
Source: Claude, Pricing, https://claude.com/pricing
Source: Pro Git, Installing Git, https://git-scm.com/book/en/v2/Getting-Started-Installing-Git
Say it: "This is an empty practice folder. Build a one-page to-do list I can open in my browser, with no packages. Before each command, tell me in one sentence what it does. When you finish, list every file you made and what it is for."
Described as: "how do I actually start", "what do I need to install", "how do I get an agent on my computer", "where do I type to the AI", "do I have to pay for this".
https://dev.dqnotes.com/machine/first-agent

## 2. The network

How a web app works: browsers, servers, requests, databases, and what happens between them.

### Client and server
A client asks and a server answers. A server is only a program on a computer somewhere, waiting to be asked. Your browser is a client. So is a phone app, or a script you run in a terminal.
Agents will happily put code on the wrong side of this line, and the result still runs. When you read what one built, the first question is which machine each piece runs on, because that decides who can see it and who can change it.
Not Frontend and backend: Client and server are roles in one conversation, so your server becomes a client the moment it asks another company's server for something. Frontend and backend, coming up next, describe where your code lives.
Seen in: MDN puts the rule bluntly: "The browser is always the entity initiating the request. It is never the server (though some mechanisms have been added over the years to simulate server-initiated messages)." Those mechanisms turn up later in this chapter.
Source: MDN, An overview of HTTP, https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Overview
Say it: "Before you change anything, list every file you created in a table with three columns: the path, whether it runs on the server or in the browser, and which secrets or data it can reach."
Described as: "which computer does this code run on", "the thing that asks and the thing that answers", "does this run on my machine or theirs", "what is a server, actually".
https://dev.dqnotes.com/network/client-server

### Browser
The program that turns an address into a page. It asks servers for files, draws what comes back, and runs the code that came with it. Chrome, Safari and Firefox are browsers.
An agent reading your code sees what should happen. The browser keeps a log of what did. When something looks broken, that log settles the argument, so ask for its contents instead of a theory about them.
Seen in: Chrome's DevTools open with Command+Option+J on a Mac or Control+Shift+J elsewhere. The Network tab lists every request with its status, type, size and time, and it only records while DevTools is open.
Source: Chrome for Developers, Inspect network activity, https://developer.chrome.com/docs/devtools/network
Say it: "The save button does nothing. Open the page in Playwright, click save, and show me the request it sent and the status it got back, from the network log. Don't work it out from reading the code."
Described as: "I clicked and nothing happened", "how do I see what the page is loading", "the panel that shows all the requests", "it works on your side but not in my browser".
https://dev.dqnotes.com/network/browser

### Frontend and backend
Also: Client side and server side.
The frontend is the code that runs in the user's browser. The backend is the code that runs on your server. Anyone can read the frontend and change what it sends. Nobody outside can read the backend.
Agents often put a rule where the button is, because that's where the feature is. A check in the frontend is only a courtesy to honest people, so anything that protects money or data has to be repeated on the backend.
Seen in: Build tools draw this line with a name. In Vite, variables starting with VITE_ are "exposed in client-side source code", and in Next.js the same goes for NEXT_PUBLIC_, which is "inlined" into the JavaScript sent to the browser. An agent that adds the prefix to fix an undefined value has just published that value.
Source: Vite docs, Env variables and modes, https://vite.dev/guide/env-and-mode
Source: Next.js docs, Environment variables, https://nextjs.org/docs/app/guides/environment-variables
Say it: "Work out the order total on the server from the product ids, and ignore any price the browser sends. Add a test that posts total: 1 and checks the real price is charged. Then grep the build output in dist/ for any secret key."
Described as: "can users change the price", "hiding the button should be enough", "where does the secret key go", "is this code visible to everyone", "someone could just edit the request".
https://dev.dqnotes.com/network/frontend-backend

### URL
Also: Web address.
The address of one thing on the web. https://shop.example.com:443/cart?coupon=SAVE#total has a scheme, a host, a port, a path, a query and a fragment. Each part is read by something different.
Naming the part gets you the right change. "Read the page number from the query string" and "put it in the path" are different pieces of work. Agents that build or compare URLs by gluing text together also produce a family of bugs that send people to the wrong place, and you can now spot them.
Not File path: A URL path looks like a path on disk, and in the early web it often was one. Now it's just text the server decides how to answer, and there may be no file with that name anywhere.
Seen in: MDN notes that the part after the # "is never sent to the server with the request", and that a path is now "mostly an abstraction handled by Web servers without any physical reality."
Source: MDN, What is a URL?, https://developer.mozilla.org/en-US/docs/Learn_web_development/Howto/Web_mechanics/What_is_a_URL
Say it: "Read the page number from the query string as ?page=2, default it to 1, and keep it when the user changes the filter. Build the URL with new URL() and searchParams, not by joining strings. For redirects, compare the parsed hostname to an allowlist."
Described as: "the bit after the question mark", "what does the # do in an address", "the link looked like my bank but went somewhere else", "the address has weird percent signs in it".
https://dev.dqnotes.com/network/url

### DNS
Also: Domain Name System.
The internet's address book. It turns a name like shop.example.com into the numeric address of a computer. Answers get saved along the way, and each one says how long it may be kept, which is called its TTL.
Agents can edit a DNS record in seconds and then tell you it's done, while people keep arriving at the old server. The waiting is decided before the change, by the TTL already out there, so a move needs planning days ahead.
Not Propagation: People say a DNS change is propagating, as if it spreads outwards. Nothing is sent anywhere. Each place holding a saved copy, called a cache, keeps the old answer until its time runs out, then asks again.
Seen in: RFC 1035 defines the TTL as a time in seconds for which a record may be cached, and zero means it should not be cached at all. Running dig shows the seconds left beside each answer. Run it twice and the number can jump around, because you may be asking a different cache each time.
Source: RFC 1035, Domain names, 1987, https://www.rfc-editor.org/rfc/rfc1035.html
Say it: "We're moving servers on Friday. Today, lower the TTL on the A record for shop.example.com to 300. On Friday, change the record, keep the old server running for at least the old TTL, and show me dig shop.example.com +noall +answer before and after."
Described as: "I changed the domain but it still shows the old site", "waiting for the domain to update", "how does a name turn into a server", "it works for me but not for my friend after the move".
https://dev.dqnotes.com/network/dns

### IP address and port
An IP address identifies a computer on a network, like 203.0.113.10. A port number picks out one program on that computer, because many can be listening at once. Together they say exactly where a message goes.
The errors an agent hits while running your app are often about this and nothing else. "Address already in use" means two programs want one port. "Connection refused" means nothing is listening there. Neither is a bug in the code you just changed.
Seen in: HTTP defaults to port 80 and HTTPS to port 443, which is why you don't normally see them in an address. A port is a 16-bit number. PostgreSQL listens on 5432 unless told otherwise.
Source: RFC 9110, HTTP Semantics, 2022, https://www.rfc-editor.org/rfc/rfc9110.html
Source: RFC 9293, TCP, 2022, https://www.rfc-editor.org/rfc/rfc9293.html
Source: PostgreSQL docs, Connections and authentication, https://www.postgresql.org/docs/current/runtime-config-connection.html
Say it: "The dev server won't start because port 3000 is in use. Run lsof -i :3000, tell me which process holds it and what started it, and ask me before you kill anything."
Described as: "address already in use", "the number after the colon", "connection refused", "two things fighting over the same port".
https://dev.dqnotes.com/network/ip-and-port

### Localhost
Also: 127.0.0.1, Loopback.
A name that always means this same computer. A server on localhost:5173 answers you and nobody else, because for anyone else the same name means their own machine.
An agent saying "it's running on localhost" has only told you about its own machine. If it runs in a container (a sealed-off environment on your machine) or in the cloud (on someone else's computer), its localhost isn't yours and you won't see a thing. Working on localhost also says little about production, where there's real distance, real encryption and a different server.
Seen in: RFC 6761 says localhost names always resolve to the loopback address. Vite defaults to host localhost on port 5173, tries the next port if that one is taken, and setting host to true makes it listen on all addresses, "including LAN and public addresses."
Source: RFC 6761, Special-use domain names, 2013, https://www.rfc-editor.org/rfc/rfc6761.html
Source: Vite docs, Server options, https://vite.dev/config/server-options
Say it: "Start the dev server and give me the exact URL it printed, because Vite picks another port if 5173 is taken. Then curl that URL and show me the status code before you tell me it works. Don't bind to 0.0.0.0."
Described as: "it says localhost but the page will not open", "can my friend see my local site", "the site only works on my computer", "why can't I open what the agent started".
https://dev.dqnotes.com/network/localhost

### TCP connection
Also: TCP.
A reliable conversation between two programs across a network. Before any data moves, the two sides exchange messages to open the connection, and that costs one full trip there and back.
Opening a connection is expensive, and you pay for it per connection, which agents rarely think about. Code that opens a new database connection for every request can look perfectly clean and still fall over when many people use it at once.
Not HTTP: TCP carries bytes and has no idea what they mean. HTTP is the agreement about what those bytes say. Most web requests are HTTP travelling inside TCP.
Seen in: Ilya Grigorik: "each new connection will have a full roundtrip of latency before any application data can be transferred", which is why he calls connection reuse "a critical optimization".
Source: Grigorik, High Performance Browser Networking, ch. 2, https://hpbn.co/building-blocks-of-tcp/
Say it: "Don't create a new database client inside the request handler. Create one at startup and reuse it through a pool. Tell me the pool size you chose and how it compares to the database's connection limit."
Described as: "too many connections", "the first request is slow and the next ones are fast", "the database ran out of connections", "what happens before the data is sent".
https://dev.dqnotes.com/network/tcp

### HTTP
The agreed format browsers and servers use to talk. One side sends a request, the other sends back a response. It remembers nothing between requests, so each one has to carry everything the server needs to know.
Almost everything your agent builds on the web goes through it, including the agent's own calls to its model. Reading one raw exchange is the fastest way to find out which side is wrong, and it replaces a lot of guessing by either of you.
Not HTTPS: Same conversation. HTTPS wraps it in encryption so that nobody on the way can read or change it.
Seen in: MDN: "HTTP is stateless: there is no link between two requests being successively carried out on the same connection." Newer versions send binary frames rather than text, but mean the same thing. Agent tooling runs on it too: MCP's Streamable HTTP transport sends each message as an HTTP POST.
Source: MDN, An overview of HTTP, https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Overview
Source: Model Context Protocol, Specification, https://modelcontextprotocol.io/specification/latest
Say it: "Show me the raw HTTP exchange: run curl -i against the endpoint and paste the status line, the headers and the body. I want the output itself, not your description of it."
Described as: "what is actually being sent", "how do the browser and server talk", "why does the server forget me between clicks", "what does the http in the address mean".
https://dev.dqnotes.com/network/http

### Request and response
A request is a method (a verb such as GET), a path, some headers (labelled lines of extra information) and maybe a body (the content). A response is a status code (a number saying how it went), some headers and maybe a body. Every feature on the web is built from those two shapes.
Saying "it's broken" gives an agent nothing to hold. Splitting the bug in two does: if the request was wrong, the problem is in the frontend, and if the response was wrong, it's in the backend. That cuts the search in half before anyone reads a line of code.
Seen in: A real exchange is short enough to read. Press send on the bench and every line you see is produced by a real function in the page, which is also how a route handler looks in a framework.
Say it: "Before you change anything, log the exact request the page sends and the exact response it gets back, and paste both. Then tell me whether the bug is in the request or in the response, and why."
Described as: "the API is broken but I do not know which side", "what gets sent when I click", "what comes back from the server", "the two halves of a web call".
https://dev.dqnotes.com/network/request-response

### curl
A command that sends one HTTP request from the terminal and prints what comes back. It takes the browser out of the question, so it tells you whether the server itself works.
It gives an agent a cheap way to prove a claim, and it gives you output you can read. One trap: curl exits 0 even when the server answers with an error, because the transfer itself worked. Use it in a script without --fail and your check can never fail.
Seen in: Anthropic's API docs use curl -sS -i to show a response with its headers. The curl manual says --fail makes it return error 22 on server errors, instead of quietly printing the error page and exiting 0.
Source: Claude API docs, Errors, https://platform.claude.com/docs/en/api/errors
Source: curl manual, https://curl.se/docs/manpage.html
Say it: "Prove it with curl. Run curl -i -X POST localhost:3000/api/notes -H 'Content-Type: application/json' -d '{"text":"hi"}' and paste all of it. Then run it again without the cookie and show me the 401."
Described as: "test the server without the website", "send a request from the terminal", "the check passes even though the server returned an error", "how can the agent prove the endpoint works".
https://dev.dqnotes.com/network/curl

### HTTP method
Also: HTTP verb, GET and POST.
The verb at the start of a request. GET reads, POST creates or submits, PUT replaces, DELETE removes. The method also tells everything between you and the server whether it's safe to send the same request again.
Requests get repeated, by double taps, flaky networks and helpful retry code. An agent that uses GET for an action can have it fired by a link preview, and one that writes a plain POST checkout can charge someone twice. Neither shows up in a test where everything goes right.
Not Safe and idempotent: Safe means it changes nothing, like GET. Idempotent means repeating it has the same effect as doing it once, like PUT or DELETE. POST is neither, unless you make it so.
Seen in: RFC 9110: "the GET, HEAD, OPTIONS, and TRACE methods are defined to be safe" and "PUT, DELETE, and safe request methods are idempotent." Stripe accepts an Idempotency-Key header on POST so that you can retry a payment "without accidentally performing the same operation twice".
Source: RFC 9110, HTTP Semantics, 2022, https://www.rfc-editor.org/rfc/rfc9110.html
Source: Stripe API docs, Idempotent requests, https://docs.stripe.com/api/idempotent_requests
Say it: "This checkout POST can run twice if the user double-taps. Require an Idempotency-Key header, store the first response against that key for 24 hours, and add a test that sends the same key twice and gets one order and one charge."
Described as: "I clicked twice and got charged twice", "the difference between GET and POST", "is it safe to retry this", "a link deleted something by itself".
https://dev.dqnotes.com/network/http-method

### Status code
Also: HTTP status.
A three-digit number at the top of every response. 2xx means it worked, 3xx means look elsewhere, 4xx means the request was wrong, and 5xx means the server failed. It works like an exit code for the web.
This is the word that sharpens your instructions most. Left alone, agents often catch an error and return 200 with an error message inside, which reads as success to every client, cache and graph. Asking for the exact code fixes all of them at once.
Not 401 and 403: 401 means the server doesn't know who you are, so signing in might help. 403 means it knows exactly who you are and the answer is no.
Seen in: fetch does not reject on a 404 or 500. It resolves normally, so you must check response.ok. RFC 6585 made Retry-After optional on a 429 ("MAY"), so you have to ask for it. The Claude API returns 429 for rate limits and 529 when overloaded, and its SDK honours retry-after (docs read November 2026). GitHub uses 403 or 429 for rate limits.
Source: RFC 9110, HTTP Semantics, 2022, https://www.rfc-editor.org/rfc/rfc9110.html
Source: RFC 6585, Additional HTTP status codes, 2012, https://www.rfc-editor.org/rfc/rfc6585.html
Source: MDN, Using the Fetch API, https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch
Source: Claude API docs, Errors, https://platform.claude.com/docs/en/api/errors
Source: GitHub docs, Rate limits for the REST API, https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api
Say it: "Add a 429 with a Retry-After header to POST /login after 5 failed attempts per IP address per minute. Return 401 for a wrong password and 422 for a missing email, never a 200 with an error in the body, and add a test for each status."
Described as: "it says 200 but it failed", "the page says 404", "what does this error number mean", "tell the client to slow down", "the difference between 401 and 403".
https://dev.dqnotes.com/network/status-code

### Header
Also: HTTP header.
A labelled line of extra information on a request or response, such as Content-Type: application/json. Names ignore upper and lower case. Many travel with every request, and you never see them unless you look.
One header is often the whole fix, and it can save an agent writing a page of code: caching, content type, a redirect, a login challenge. Ask for it by name. Code running in the page can't set them all either, so some header bugs can't be fixed in the frontend at all.
Seen in: RFC 9110: "Field names are case-insensitive". Browsers keep some to themselves, including Cookie and Host, because, in MDN's words, "the user agent retains full control over them."
Source: RFC 9110, HTTP Semantics, 2022, https://www.rfc-editor.org/rfc/rfc9110.html
Source: MDN, Forbidden request header, https://developer.mozilla.org/en-US/docs/Glossary/Forbidden_request_header
Say it: "Add Cache-Control: no-store and Content-Type: application/json to every response from /api/account, and add a test that makes a real request and fails if either header is missing."
Described as: "the hidden information on a request", "where does the content type go", "the extra lines above the body", "I set the cookie in my code and it was ignored".
https://dev.dqnotes.com/network/header

### JSON
A plain text format for data: named values in curly braces (called objects), lists in square brackets, text in double quotes, plus numbers, true, false and null, which means nothing. Most programs on the web send it. Nothing else is allowed, including comments.
Your agent reads and writes JSON all day, including when it asks to use its tools, and its output can be almost right. The strictness is useful: one stray comma fails loudly. Numbers are the quiet exception, because a big id can come back changed with no error.
Not JavaScript object: They look alike. JavaScript allows single quotes, unquoted names, comments and trailing commas. JSON allows none of them.
Seen in: RFC 8259 calls integers interoperable only within -(2^53)+1 to (2^53)-1, says that names "SHOULD be unique", and warns that when they aren't, the behaviour of the software receiving them is "unpredictable".
Source: RFC 8259, JSON, 2017, https://www.rfc-editor.org/rfc/rfc8259.html
Say it: "Send order ids as strings in the JSON, because they go past 2^53. Parse every incoming body with a schema at the edge of the route, and return a 400 with a helpful message when it's malformed."
Described as: "the curly bracket format", "unexpected token in JSON", "the id came back with different digits", "package.json stopped working after I edited it".
https://dev.dqnotes.com/network/json

### API
Also: Web API, Endpoint.
The set of requests a program agrees to answer, and the answers it promises to give. It's a contract between two pieces of software, so that neither side has to read the other's code.
Two ways this goes wrong with agents. They invent endpoints (the separate addresses an API answers at) and fields that sound right but don't exist, so point them at real docs. And they rename a field on your server without knowing who else reads it, and nothing fails until a client shows nonsense.
Not Library: A library is code you run inside your own program. A web API is someone else's running program that you send messages to, and it can change or go down without asking you.
Seen in: The OpenAPI Specification, at v3.2.1 as of September 2026, is the common way of writing this contract down. A description "formally describes the surface of an API and its semantics", in JSON or YAML, which suits an agent very well.
Source: OpenAPI Specification, https://spec.openapis.org/oas/latest.html
Say it: "Don't change the shape of GET /api/orders, because other clients use it. Only add fields. Update openapi.yaml in the same commit, and add a contract test that fails if a documented field disappears or changes type."
Described as: "the agent made up a function that does not exist", "how two programs talk to each other", "I changed the backend and the app broke", "the docs for what the server accepts".
https://dev.dqnotes.com/network/api

### REST
Also: RESTful.
A style of designing web APIs around things rather than actions: GET /orders/42, not POST /getOrder. The word comes from Roy Fielding's 2000 dissertation, and most APIs called REST follow only part of it.
It's a convention an agent already knows, so naming it gets you predictable addresses. Don't stop there, though. "Make it RESTful" leaves plenty open, so spell out the addresses and the status codes you want, and argue about the design before the code exists.
Not JSON over HTTP: What the trade means by REST is usually this: nouns in paths and methods as verbs. Fielding's definition is much stricter, says "REST does not restrict communication to a particular protocol", and never mentions JSON. Whether most so-called REST APIs deserve the name is a standing argument.
Source: Fielding, Architectural Styles, ch. 5, 2000, https://ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm
Say it: "Use these routes: GET /orders, POST /orders, GET /orders/:id, PATCH /orders/:id, DELETE /orders/:id. Return 201 with a Location header on create, no verbs in paths, and show me the route list before you build it."
Described as: "how should I name the routes", "should it be getUser or users", "what does RESTful mean", "the agent made a mess of the URLs".
https://dev.dqnotes.com/network/rest

### Database, briefly
A program whose whole job is keeping data safe on disk and finding it again quickly. Your server sends it questions over a connection. When your server restarts, its memory is gone, and the database still has everything.
Agents building a quick demo often keep data in a variable, because it runs first time and every check passes. Then a deploy wipes it, or a second copy of the server gives different answers. The "Data and failure" chapter goes much deeper.
Not Memory: A variable lives in one running process and dies with it. A database is a separate program, shared by every copy of your server, as chapter 1 said of memory and disk.
Seen in: PostgreSQL, a common choice, listens on TCP port 5432 by default, and by default it "allows only local TCP/IP" connections from the same machine, which is why a fresh install can work locally and be unreachable from your server.
Source: PostgreSQL docs, Connections and authentication, https://www.postgresql.org/docs/current/runtime-config-connection.html
Say it: "You stored signups in an array, so they vanish on restart. Move them to a users table in Postgres, read the connection string from DATABASE_URL, restart the server, and show me the row is still there."
Described as: "my data disappeared when I restarted", "it works until I deploy", "where should this be saved", "different users see different lists".
https://dev.dqnotes.com/network/database-intro

### Cookie
A small piece of text a server asks the browser to keep. The browser then attaches it to later requests to that site, by itself. It's how HTTP, which has no memory, comes to know you.
The security of a login rests on a few words after the value, and they're easy to leave out, because a cookie without them works fine in every test. Asking for them by name gets them. They also explain "works on localhost, not in production".
Not localStorage: localStorage is a browser store that page code can read and that goes nowhere unless the code sends it. A cookie goes out with requests automatically, and with HttpOnly page code can't read it at all.
Seen in: MDN: HttpOnly stops JavaScript reading it, Secure limits it to https except on localhost, and SameSite=None "must" be combined with Secure. OWASP asks session cookies to set SameSite explicitly, to Strict or Lax.
Source: MDN, Set-Cookie, https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Set-Cookie
Source: OWASP, Session Management Cheat Sheet, https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html
Say it: "Set the session cookie with HttpOnly; Secure; SameSite=Lax; Path=/ and no Domain. Add a test that reads the Set-Cookie header from POST /login and fails if any of those is missing."
Described as: "it keeps logging me out", "login works locally but not in production", "how does the site remember me", "can JavaScript read my login", "a form on another site acted as me".
https://dev.dqnotes.com/network/cookie

### Login session
Also: Session, Server-side session.
The server's memory of a signed-in user. At login it creates a record and gives the browser a long random id in a cookie. Each later request carries the id, so the server can look up who it belongs to.
An agent asked for "login" will produce something that lets the right person in, and you won't see the gaps by logging in. The questions to ask are whether logging out really ends it, and what happens to a stolen copy.
Not Signed token: A session record lives on the server, so deleting it ends the login. A signed token, a tamper-proof pass the client carries, needs no lookup and stays valid until it expires even after the user logs out, unless you build a way to cancel it.
Seen in: OWASP asks for at least 64 bits of entropy in a session id, and for a new id after any privilege change such as logging in. On logout, it says "the web application must invalidate the session at least on server side."
Source: OWASP, Session Management Cheat Sheet, https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html
Say it: "On login, generate a new session id of at least 64 bits from a secure random source and store the session row on the server. On logout, delete the row, then show me that replaying the old cookie with curl gets a 401."
Described as: "I logged out but the old link still works", "how does the server know it is me", "someone copied my cookie", "should I use sessions or tokens".
https://dev.dqnotes.com/network/session

### HTTPS and TLS
Also: SSL.
HTTPS is HTTP sent through TLS, an encrypted tunnel. TLS proves you reached the real site, hides what you send, and detects any change on the way. A padlock means the line is private, and says nothing about whether the site is honest.
A certificate is a site's proof of who it is. When a certificate error blocks an agent, the quick way past is to turn checking off, and it may just do that. The app then works and is open to anyone on the path. Treat a disabled check as a failed task, however many other checks pass.
Not Encrypted everything: TLS protects data while it travels. It doesn't protect what's in your database, and it doesn't hide everything: the IP address, the size of the traffic and usually the site name remain visible.
Seen in: RFC 8446 gives TLS 1.3 three properties, authentication, confidentiality and integrity, and says it "does not hide the length of the data". In Python's Requests, verify=False "will make your application vulnerable to man-in-the-middle (MitM) attacks". curl's equivalent is -k.
Source: RFC 8446, TLS 1.3, 2018, https://www.rfc-editor.org/rfc/rfc8446.html
Source: Requests docs, SSL cert verification, https://requests.readthedocs.io/en/latest/user/advanced/
Source: IETF draft, TLS Encrypted Client Hello, https://datatracker.ietf.org/doc/html/draft-ietf-tls-esni
Say it: "Never turn off certificate verification to get this working, so no verify=False and no curl -k. Find out why it fails, show me the output of curl -v https://api.example.com, and redirect all plain http traffic to https with a 308."
Described as: "the padlock in the address bar", "certificate error", "can people on the wifi see my password", "the agent turned off the security check to make it work".
https://dev.dqnotes.com/network/https-tls

### The life of a request
Also: Latency budget.
One click travels through a DNS lookup, a connection, encryption, your server, your database and back. Each step takes time, and they add up. A latency budget says how long the whole trip may take, so you can see which step broke it.
Told to "make it faster", an agent will optimise what it can see, which is the code. The time may well be somewhere else: distance, a new connection for every request, or sixty database queries. Measure first and point it at the step that matters.
Not Fast code: A request can run in 2 ms on the server and still take half a second to reach the user. Light in fibre doesn't get any faster, so past a point the only fix is being closer.
Seen in: Grigorik gives light in fibre as "around 200,000,000 meters per second", so New York to Sydney can't beat a 160 ms round trip, and he reports real ones at 200 to 300 ms. A new connection pays for that trip once for TCP, once for TLS 1.3, and once more for the request.
Source: Grigorik, High Performance Browser Networking, ch. 1, https://hpbn.co/primer-on-latency-and-bandwidth/
Source: RFC 8446, TLS 1.3, 2018, https://www.rfc-editor.org/rfc/rfc8446.html
Say it: "Measure this before changing anything: curl -o /dev/null -s -w 'dns %{time_namelookup} connect %{time_connect} tls %{time_appconnect} first byte %{time_starttransfer} total %{time_total}\n' $URL. Tell me which step uses most of our 500 ms budget, then fix that one."
Described as: "the page is slow but I do not know why", "where does the time go", "it is fast for me and slow in Australia", "how fast could this possibly be".
https://dev.dqnotes.com/network/request-journey

### Caching in the browser and on the way
Also: HTTP cache, Cache-Control.
A saved copy of a response, kept by the browser or by something between you and the server, so the same request needn't travel again. The Cache-Control header sets the rules. It makes sites fast, and it's why people see old versions.
Agents add caching when asked to speed things up, and rarely work out how the new version will reach people, which is the harder half. "I deployed and nothing changed" is often this. So is a private page served to the wrong person.
Not no-cache: It does not mean "don't cache". It means keep it, and check with the server before using it. The directive that means don't keep it at all is no-store.
Seen in: MDN's recommended pattern: files with a content hash in the name get max-age=31536000, immutable, because a new version gets a new name. HTML gets no-cache, so it always points at the newest names. RFC 9111 defines how freshness and validation work.
Source: MDN, Cache-Control, https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cache-Control
Source: RFC 9111, HTTP Caching, 2022, https://www.rfc-editor.org/rfc/rfc9111.html
Source: RFC 5861, Stale content extensions, 2010, https://www.rfc-editor.org/rfc/rfc5861.html
Say it: "Serve files in /assets/ with Cache-Control: max-age=31536000, immutable, and only if their names contain a content hash. Serve index.html with no-cache and /api/account with private, no-store. Show me the response headers for all three."
Described as: "I deployed but I still see the old version", "it works after a hard refresh", "people are seeing stale data", "why is it downloading the same file again".
https://dev.dqnotes.com/network/cache-http

### CDN
Also: Content delivery network, Edge.
A network of servers around the world that keep copies of your files close to users. Requests go to the nearest one, and only a miss has to travel on to your real server, which is called the origin.
It's the only real fix for distance, and it takes load off your server. It also means a mistake in your cache headers is now served to the world. Personal pages must never be shared, so have the agent prove it with the response headers.
Not Browser cache: A browser cache serves one person. A CDN is shared, so one person's miss becomes everyone's hit, and one person's private page, cached by mistake, becomes everyone's too.
Seen in: Cloudflare's docs (read November 2026): "The Cloudflare CDN does not cache HTML or JSON by default", and it skips caching when the response has a Set-Cookie header. Its CF-Cache-Status header reports HIT, MISS, EXPIRED and others.
Source: Cloudflare docs, Default cache behaviour, https://developers.cloudflare.com/cache/concepts/default-cache-behavior/
Source: Cloudflare docs, Cloudflare cache responses, https://developers.cloudflare.com/cache/concepts/cache-responses/
Say it: "Run curl -I on /assets/app.js twice and show me CF-Cache-Status, which should say HIT the second time. Give /account Cache-Control: private, no-store, and show me that header too. Tell me what happens to our server when the cache is purged."
Described as: "it is slow for people far away", "one user saw another user's page", "the copy near the user", "the site fell over when we cleared the cache".
https://dev.dqnotes.com/network/cdn

### Reverse proxy
Also: Load balancer, Gateway.
A server that stands in front of your real server, takes requests from outside and passes them on. It can handle encryption, share load between copies and cut off slow replies, so "the server" is usually several programs.
When an agent looks for an error in your app's logs and finds nothing, it may be looking in the wrong program. A 502 or 504 often comes from the proxy, and your code may never have seen the request. Naming the layer sends it to the right place.
Seen in: RFC 9110 defines 502 and 504 by this role: a server "acting as a gateway or proxy" that got a bad reply, or no timely one, from upstream. In nginx, proxy_read_timeout defaults to 60 seconds, so a slower reply gets cut off however long your own code would wait.
Source: RFC 9110, HTTP Semantics, 2022, https://www.rfc-editor.org/rfc/rfc9110.html
Source: nginx docs, ngx_http_proxy_module, https://nginx.org/en/docs/http/ngx_http_proxy_module.html
Say it: "We're getting 504s from nginx with nothing in the app log. Compare proxy_read_timeout with how long /report really takes and show me both numbers. Then make /report faster or move it to a background job. Don't just raise the timeout."
Described as: "502 bad gateway", "the error is not in my logs", "nginx is in the way", "something between the internet and my app".
https://dev.dqnotes.com/network/reverse-proxy

### WebSocket
A connection that stays open, so either side can send at any moment. It starts as an ordinary HTTP request, and then both sides agree to switch. Plain HTTP can't do this, because there the server only ever answers.
Agents reach for this whenever they hear "real time", and it's a lot to run: connections to hold open, reconnection, and proxies that drop them. Asking again every ten seconds, or a one-way stream, is often all you need, so ask which one it chose and why.
Not Server-sent events: Server-sent events are a one-way stream from server to browser over normal HTTP, and browsers reconnect by themselves. A WebSocket goes both ways. AI models send their replies to you word by word using server-sent events.
Seen in: RFC 6455: the browser sends an HTTP Upgrade request and the server replies "HTTP/1.1 101 Switching Protocols". MDN on server-sent events: "This is a one-way connection". The Claude API streams responses that way when you set stream to true.
Source: RFC 6455, The WebSocket Protocol, 2011, https://www.rfc-editor.org/rfc/rfc6455.html
Source: MDN, Using server-sent events, https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
Source: Claude API docs, Streaming, https://platform.claude.com/docs/en/build-with-claude/streaming
Say it: "We only need the server to send progress one way, so use server-sent events and not a WebSocket. Reconnect after a dropped connection without losing progress, and show me that working by restarting the server during a job."
Described as: "live updates without refreshing", "how does the chat update by itself", "the page keeps asking the server every second", "the words appear one at a time".
https://dev.dqnotes.com/network/websocket

### Webhook
A request another company's server sends to yours when something happens, such as a payment clearing. It's an API call the other way round: you give them a URL and wait. Anyone who finds that URL can send to it too.
An agent's first version usually trusts the request, handles it once and does all the work before replying. All three are wrong: anyone can send to that URL, real senders repeat themselves, and they give up if you are slow. A demo sends one honest event, so spell out what you need.
Not Polling: Polling is you asking again and again whether anything has happened. A webhook is them telling you once, and retrying if you don't answer.
Seen in: Stripe signs each delivery with HMAC-SHA256 in a Stripe-Signature header, over the timestamp and the raw body. It retries "for up to three days" in live mode, and warns: "Webhook endpoints might occasionally receive the same event more than once." Its libraries reject timestamps over 5 minutes old by default.
Source: Stripe docs, Webhooks, https://docs.stripe.com/webhooks
Say it: "Verify the Stripe-Signature against the raw request body before doing anything, and reject events over 5 minutes old. Log each event id and skip ones already handled. Return 200 straight away and queue the work. Test it with stripe listen --forward-to localhost:4242/webhook."
Described as: "the payment company tells my app when something happens", "the customer got the order twice", "how do I know it is really from them", "I can't receive it on localhost".
https://dev.dqnotes.com/network/webhook

### Framework
A large body of code that makes most of the structural decisions for you, such as how a URL finds its code and where files go. You fill in the gaps. Next.js, Rails, Django and Laravel are frameworks.
A framework is why an agent can build quickly, and also a place it goes wrong. Models learned conventions from older versions, so it may write the old way with full confidence. Make it check the installed version and read those docs.
Not Library: You call a library when you want it. A framework calls you: it owns the program's structure and runs your code at the moments it decides.
Seen in: In the Next.js App Router (docs updated July 2026), "A route becomes public when a page or route file exists", app/blog/[slug]/page.tsx answers /blog/my-first-post, and a folder in brackets like (shop) stays out of the URL.
Source: Next.js docs, Project structure, https://nextjs.org/docs/app/getting-started/project-structure
Say it: "Check the Next.js version in package.json and read the docs for that version before writing any routes. Put the page at app/orders/[id]/page.tsx and the API at app/api/orders/[id]/route.ts, then list every file you created."
Described as: "the agent used an old way of doing it", "where do I put this file", "why does this folder name have brackets", "the code it wrote does not match the docs".
https://dev.dqnotes.com/network/framework

### HTML, CSS and JavaScript
The three languages a browser understands. HTML says what's on the page, CSS says how it looks, and JavaScript says what it does. Whatever tools an agent uses, these are what finally reach the browser.
Naming the layer narrows the change. "This is a CSS problem" stops an agent rewriting working logic to fix a margin. It also tells you what a screenshot can prove: it shows HTML and CSS, and says nothing about whether JavaScript is working.
Not React: React, a popular tool for building pages, and others like it run on top of these three and compile down to them. A browser has never heard of a .tsx file, which is where React code is usually written.
Seen in: Open any site, turn off JavaScript in DevTools, and reload. How much of it is left tells you a lot about how it was built. The next term explains why.
Say it: "This is a CSS problem, so don't touch the JavaScript: the button overflows at 390px wide. Fix it in CSS, use a real <button> rather than a <div> with a click handler, and send me screenshots at 390 and 1440."
Described as: "the page looks right but nothing happens", "the styling is wrong", "which language does what", "it is only a visual problem".
https://dev.dqnotes.com/network/html-css-js

### Server and client rendering
Also: SSR, CSR, Hydration.
Rendering means turning data into a page. Server rendering does it before sending, so the HTML arrives ready to read. Client rendering sends JavaScript that builds the page in the browser. Static rendering does it once, when the site is built.
Your agent will pick one or inherit one from the framework, and the choice decides what breaks: a blank screen, a page that looks ready and isn't, or data that's as old as the last build. Decide on purpose, page by page, and ask how to tell which you got.
Not Hydration: Server rendering produces the HTML. Hydration comes after, when JavaScript in the browser attaches itself to that HTML so the buttons work. Until it finishes, a page can look ready and do nothing.
Seen in: web.dev (updated January 2026) describes hydration as running client-side scripts to add state and interactivity to server-rendered HTML. Its most important reason for caution is the moment "when a page looks ready but none of its interactive features work."
Source: web.dev, Rendering on the Web, 2026, https://web.dev/articles/rendering-on-the-web
Say it: "Render the product page on the server so the content is in the HTML. Prove it: curl the URL and show me the product name in the output. Keep add-to-basket as client code, and make it look disabled until it can really respond."
Described as: "the page is blank and then pops in", "I click and nothing happens until it finishes loading", "view source shows almost nothing", "search engines cannot see my content".
https://dev.dqnotes.com/network/rendering

### Timeout
A limit on how long to wait for something before giving up and reporting failure. Without one, a program waiting on a server that never replies can wait for ever, holding on to everything it was using.
Agents rarely add timeouts unless asked, because the code works without them, right up to the day a service you depend on goes slow. Then requests pile up waiting, and pages that never touch that service go down too. Asking is one sentence.
Not Retry: A timeout decides when to stop waiting. A retry decides whether to try again. Retrying with no timeout, or retrying a POST that isn't idempotent, makes things worse.
Seen in: The Python Requests docs: "Nearly all production code should use this parameter in nearly all requests. Failure to do so can cause your program to hang indefinitely", and "If no timeout is specified explicitly, requests do not time out." In JavaScript, the tool is AbortSignal.timeout().
Source: Requests docs, Quickstart, https://requests.readthedocs.io/en/latest/user/quickstart/
Source: MDN, AbortSignal.timeout(), https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal/timeout_static
Say it: "Every outbound call gets a timeout: fetch(url, { signal: AbortSignal.timeout(3000) }). On a timeout, return 504 and log which upstream it was. Add a test with a fake upstream that never replies, and check that the route answers within about 3 seconds."
Described as: "it just hangs forever", "one slow service took the whole site down", "the spinner never stops", "how long should it wait".
https://dev.dqnotes.com/network/timeout

## 3. Version control

Commits, branches, diffs and pull requests: the undo button and the paper trail for everything an agent does.

### Version control
Also: VCS, Source control.
A system that keeps every saved version of a project, with who changed what and when, so you can compare any two versions or go back to one. Git is the one this course uses, and the one you will keep meeting.
An agent can be confidently wrong across a whole project in a minute. Version control is what makes that cheap: you can see precisely what it did, throw away what you don't want, and return to the last state that worked. Without it, every agent mistake is a manual repair job.
Not Agent checkpoints: Claude Code's /rewind restores its own file edits from a session. Its docs say a file changed only by shell commands is not tracked, and head that section "Not a replacement for version control". Use both.
Seen in: Git dates from 2005, when the Linux kernel community lost free use of the tool it had been using, which prompted that community, and Linus Torvalds in particular, to write their own. Anthropic's long-running agent harness asks the model to commit as it goes because, in its words, "This allowed the model to use git to revert bad code changes and recover working states of the code base."
Source: Pro Git, A Short History of Git, https://git-scm.com/book/en/v2/Getting-Started-A-Short-History-of-Git
Source: Young, Effective harnesses for long-running agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
Source: Claude Code docs, Checkpointing, https://code.claude.com/docs/en/checkpointing
Say it: "Before you change anything, run git status and tell me if the working tree is clean. If it isn't, stop and ask me. Commit each finished step, so that I can go back to any of them."
Described as: "how do I undo what the AI just did", "keeping old versions of my code", "final_v2_really_final.zip", "seeing what changed and when", "going back to when it worked".
https://dev.dqnotes.com/git/version-control

### Repository
Also: Repo.
One project under version control: your files, plus a hidden folder called .git that holds the entire history. Run git init in a folder and it becomes a repository. Delete .git and the history is gone.
The whole history sits on your machine, in a folder an agent can reach. Reading it is how an agent catches up at the start of a session, and damaging it is one of the few ways it can really hurt you, which is why agent tools treat that folder as special.
Seen in: Claude Code lists .git as a protected directory, and its docs say writes there are never auto-approved, apart from narrow exceptions around bypassPermissions mode. OpenAI's Codex keeps .git read-only in its default sandbox, and in a folder that isn't under version control it starts read-only altogether. Both as of November 2026.
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Source: OpenAI, Codex approvals and security, https://learn.chatgpt.com/docs/agent-approvals-security
Say it: "Run git init, add a .gitignore covering node_modules and .env, and make a first commit called "chore: initial commit" before you write any code, so every change after this can be seen and undone."
Described as: "what is that hidden .git folder", "turning this folder into a git project", "where is the history actually stored", "is my project in git".
https://dev.dqnotes.com/git/repository

### Commit
A saved snapshot of the whole project, with a message, an author, a time, and a link to the commit before it. Its name is a hash: a long fingerprint worked out from all of that, so identical contents always get the same name.
Committed work is the only agent work that is safe, because anything uncommitted can be overwritten by one command. And since each hash includes the hash before it, nobody can alter the past without every later hash changing, so a history can serve as evidence.
Not Saving a file: Saving writes a file to disk. Committing records the project's state in the history. A file you saved and never committed has no past, and git can't bring it back.
Seen in: Pro Git: "Git is a content-addressable filesystem." A commit object holds the hash of a snapshot, its parents, the author and committer with timestamps, a blank line and the message. Run git cat-file -p HEAD in any repository to read one. The bench builds these same objects and hashes them with SHA-1, and we checked its hashes against real git.
Source: Pro Git, Git Objects, https://git-scm.com/book/en/v2/Git-Internals-Git-Objects
Say it: "Commit now, with only the files for this change: git add src/cart.ts src/cart.test.ts, then git commit. Show me the hash and the output of git show --stat HEAD."
Described as: "saving my progress in git", "a checkpoint I can go back to", "what are those long random codes", "a save point for the code".
https://dev.dqnotes.com/git/commit

### Diff
A list of exactly what changed between two versions: the lines removed, the lines added, and a few around them for context. Removed lines start with a minus sign and added lines with a plus.
An agent's summary is a story it tells afterwards. The diff is the thing itself, and it shows what the summary left out: a deleted test, a changed setting, a file you never asked it to touch. Learning to read one is the first skill of supervising an agent.
Not The agent's summary: A summary is what the agent believes, or says, it did. A diff is what it did. When they disagree, the diff is right.
Seen in: git diff shows what isn't staged yet, git diff --staged shows what the next commit will contain, and git diff HEAD shows both. Git's default is the Myers algorithm: "The basic greedy diff algorithm." Elicit, writing about its interviews, says "Rapidly skimming a big diff and saying 'looks good'" is more common than it would like, and calls the ability to review AI code an increasingly important skill.
Source: Git docs, git-diff, https://git-scm.com/docs/git-diff
Source: Elicit, Engineering interviews in the era of agents, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Say it: "Show me git diff --stat main...HEAD first, then the full diff. If you reformatted anything, split it out: I want one commit with spacing changes only, and another with the real change."
Described as: "seeing what the AI actually changed", "the red and green lines", "comparing two versions of a file", "it says it only changed one thing".
https://dev.dqnotes.com/git/diff

### Staging area
Also: Index, The cache.
A holding area between your files and the next commit. git add puts a copy of a file there, exactly as it is at that moment, and git commit records only what is there. This lets you commit one change while others wait.
Agents like sweeping commands. git add -A stages whatever is in the folder, including the .env it just created and the debug file it forgot about. Naming each file keeps the commit to what you meant, and it is much the easiest way to keep a secret out of history.
Seen in: Pro Git: "Git stages a file exactly as it is when you run the git add command." So a file can be both staged and changed at once, and the commit takes the staged version. Of commit -a, which stages every tracked file for you, it warns: "sometimes this flag will cause you to include unwanted changes."
Source: Pro Git, Recording Changes to the Repository, https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository
Say it: "Don't use git add -A or git add . in this repo. Stage files by name, then run git diff --staged and check that nothing is in there I didn't ask for before you commit."
Described as: "it committed files it should not have", "only saving some of my changes", "what does git add actually do", "my commit has random stuff in it".
https://dev.dqnotes.com/git/staging-area

### Branch
A name that points at one commit, and moves forward each time you commit on it. It lets you work on one thing without touching the main line. HEAD is git's marker for which branch you are on.
A branch is how you give an agent room to be wrong. If the work goes badly, you switch back and delete the branch, and the main line never knew. Since branches cost almost nothing, one for each task is a sensible default.
Not A copy of the code: People picture a branch as a copy of the folder. Nothing is copied. The commits are shared, and the branch is one small file naming one of them.
Seen in: Pro Git: "A branch in Git is simply a lightweight movable pointer to one of these commits." Making one "is as quick and simple as writing 41 bytes to a file": 40 characters of hash and a newline.
Source: Pro Git, Branches in a Nutshell, https://git-scm.com/book/en/v2/Git-Branching-Branches-in-a-Nutshell
Say it: "Create a branch called fix/cart-rounding from main before you change anything. Work only on that branch. Don't merge it, and don't push to main."
Described as: "trying something without breaking the working version", "a separate copy to experiment on", "keeping this work apart from the rest", "what does HEAD mean".
https://dev.dqnotes.com/git/branch

### Main branch
Also: main, master, Default branch, Trunk.
The branch everyone treats as the real version of the project. Work happens elsewhere and joins it when it is ready. The name is only a convention: many older projects call it master.
An agent working on main is one bad commit away from breaking what everyone depends on. Don't assume your tools protect it. Claude Code's auto mode, as of November 2026, allows pushes to the default branch (sending your commits to a shared server, coming shortly) unless your own rules or the server stop them.
Seen in: git init still names the first branch master: its docs say "this will change to main when Git 3.0 is released". GitHub already uses main for new repositories. Claude Code's docs list among things allowed by default "Pushing to any branch of the repository you're working in, including the default branch."
Source: Git docs, git-init, https://git-scm.com/docs/git-init
Source: GitHub, renaming, https://github.com/github/renaming
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Say it: "Never commit directly to main. If git status says you are on main, stop, run git switch -c with a name for this task, and carry on there."
Described as: "the real version of the code", "it pushed straight to production branch", "master or main, which is it", "where finished work ends up".
https://dev.dqnotes.com/git/main-branch

### Merge
Bringing one branch's work into another. If only one side has moved, git simply slides the branch forward. If both have, it finds the commit they share, combines the two sets of changes, and makes a new commit with two parents.
Once several agents are working, the merge is where their work meets, and where two individually correct changes can stop being correct together. Merge a branch that has passed its checks, then run those checks again on the result.
Not Fast-forward: A fast-forward isn't really a merge of anything. No new commit is made, because there is nothing to combine. Pass --no-ff if you want the branch to stay visible in the history.
Seen in: Git's docs on the fast-forward case: "a new commit is not needed to store the combined history". For everything else the default strategy is ort, which they call a 3-way merge algorithm: it compares both tips with their common ancestor.
Source: Git docs, git-merge, https://git-scm.com/docs/git-merge
Say it: "Merge feature/search into main with --no-ff so the branch stays visible in the history, then run bun test on the result before pushing. If anything conflicts, stop and show me."
Described as: "combining two versions of the code", "bringing my branch back into main", "what does fast-forward mean", "getting the agent's work into the main code".
https://dev.dqnotes.com/git/merge

### Merge conflict
What happens when two branches changed the same lines. Git can't choose, so it stops and writes both versions into the file between markers, and someone has to decide what the file should really say.
Agents resolve conflicts quickly and often pick wrong, because making the markers go away is easy and looks finished. So ask what each side was trying to do. And the opposite danger is real: a merge with no conflict can still be broken, since git compares text and knows nothing about code.
Not Semantic conflict: A merge conflict stops and tells you. A semantic conflict merges cleanly and breaks when it runs: one side renames a function while the other adds a call to it. Only tests and type checks catch those.
Seen in: Git's docs: "The part before the ======= is typically your side, and the part afterwards is typically their side." Setting merge.conflictStyle to diff3 or zdiff3 also shows the original text between the two, and git merge --abort backs out of the whole attempt. The bench's merge matches real git merge on every case we tried.
Source: Git docs, git-merge, https://git-scm.com/docs/git-merge
Say it: "Run git config merge.conflictStyle zdiff3, then read both commits with git log -p and tell me what each side was trying to do. Keep both intents, and run the tests. If you can't tell which is right, stop and ask. Don't guess."
Described as: "the weird <<<<<<< lines in my file", "git says it can't merge", "two people changed the same thing", "the AI deleted my change when it fixed the conflict".
https://dev.dqnotes.com/git/merge-conflict

### Rebase
Moving your commits so they start from a newer point, by replaying them one at a time. The result looks as if you began today and worked in a straight line. But each replayed commit is a new commit, with a new hash.
It lets an agent's messy branch become a clean series before anyone reads it. The risk comes from the same place as the benefit: it rewrites history, so it is only safe on commits nobody else has. An agent won't know who else has them unless you say.
Not Merge: Both end with the same files. A merge records what happened, with a commit that has two parents. A rebase rewrites it as a straight line. Which one a team should prefer is an unsettled, real argument.
Seen in: Pro Git's rule: "Do not rebase commits that exist outside your repository and that people may have based work on." pstack's workflow has the agent verify its work, then rebase into "small, ordered commits", and only then open a pull request.
Source: Pro Git, Rebasing, https://git-scm.com/book/en/v2/Git-Branching-Rebasing
Source: Tan, pstack, opening-a-pr playbook, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/opening-a-pr.md
Say it: "Rebase this branch onto the latest main, and squash your fixups into small, ordered commits that each pass bun test. Only I am using this branch, so that is safe. Don't rebase anything already merged or pulled by anyone else."
Described as: "making my branch look clean before review", "updating my branch with the latest main", "the commit hashes all changed", "squashing my messy commits".
https://dev.dqnotes.com/git/rebase

### Remote
Also: origin.
Another copy of the repository, usually on a server, which yours knows by name. The usual name is origin. Your machine also keeps its last-known note of where the server's branches were, called origin/main.
Work you haven't sent to a remote exists on one disk only. And origin/main is a memory, not a fact: it was true the last time you checked with the server. An agent reasoning about what is on the server from an out-of-date origin/main can be confidently wrong, so have it fetch first.
Not origin/main: origin is the other repository. origin/main is your local note of where its main branch was. Yours can be hours out of date without anything telling you.
Seen in: Git's docs on clone.defaultRemoteName: "Defaults to origin". git clone creates "references to the remote branch heads under refs/remotes/origin", which is where origin/main lives.
Source: Git docs, git-clone, https://git-scm.com/docs/git-clone
Say it: "Run git fetch origin, then git log --oneline main..origin/main to see what the server has that we don't. Tell me what you find before you do anything about it."
Described as: "the copy on GitHub", "what does origin mean", "it says up to date but it isn't", "is my code on the server or just my laptop".
https://dev.dqnotes.com/git/remote

### Push and pull
Also: Fetch.
Push sends your commits to the remote. Pull fetches the remote's commits and brings them into your branch. Fetch only downloads, and changes none of your work. Nothing travels until you do one of these.
A push is where a local mistake becomes everyone's, so it deserves more care than anything else an agent runs in a normal day. You can't quietly take back what others have already pulled. Decide in advance whether your agent may push at all.
Not Fetch: Fetch updates your notes about the server and touches nothing else, so it is always safe. Pull is a fetch followed by changing your branch.
Seen in: Git's docs on the default: "git pull --ff-only will only do 'fast-forward' updates: it fails if your local branch has diverged from the remote branch. This is the default." We ran it on git 2.54, and a plain pull on diverged branches refuses and asks you to choose.
Source: Git docs, git-pull, https://git-scm.com/docs/git-pull
Say it: "Push this branch to origin and open a draft pull request. Don't push to main, and don't use any form of force. If the push is refused, stop and tell me what it said."
Described as: "getting my code onto GitHub", "getting other people's changes", "it says divergent branches", "my commit is not showing on the website".
https://dev.dqnotes.com/git/push-pull

### Pull request
Also: PR, Merge request, MR.
A request, on a hosting site such as GitHub, to merge one branch into another. It shows the diff, collects comments and check results, and keeps the whole discussion attached to the change.
This is the gate between an agent and the real project. Checks run, someone reads it, and the decision goes on the record. It is also where you write what you checked and why you believe it, which makes it proof that a person was in charge.
Not Merge: A merge is the git operation. The page with the diff, the comments and the approvals belongs to the hosting site, wrapped around that operation.
Seen in: GitHub: "Pull requests are proposals to merge code changes into a project." GitLab calls the same idea a merge request. As of November 2026, Claude Code's auto mode blocks by default "Merging a pull request no human has approved, approving Claude's own pull request, or disabling CI checks".
Source: GitHub docs, About pull requests, https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests
Source: GitLab docs, Merge requests, https://docs.gitlab.com/user/project/merge_requests/
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Say it: "Open a draft with gh pr create --draft. In the body, write what changed and why, how you tested it with the real command output, and anything you are unsure about. Don't mark it ready, and don't merge it."
Described as: "asking to get my code merged", "the page where people review changes", "what is a PR", "getting someone to check the agent's work".
https://dev.dqnotes.com/git/pull-request

### Clone and fork
A clone is a full copy of a repository on your machine, history included. A fork is your own copy on the hosting site, which you can change without asking anyone and later propose changes from.
Cloning is also how someone else's instructions reach your agent: a repository can hold files an agent reads and obeys, such as AGENTS.md or tool settings. Read those before you let it run anything. And an agent with a fork, and no write access to the original, is limited in what it can break.
Not Branch: A branch is a pointer inside one repository. A clone or a fork is a whole other repository.
Seen in: GitHub: "Forks are repositories that start as copies of another repository, called the upstream repository." On the risk: Check Point's MCPoison showed that someone with write access to a shared repository could commit a harmless tool config to Cursor, wait for approval, then change what it ran. Disclosed and fixed in Cursor 1.3 in July 2025.
Source: GitHub docs, About forks, https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/about-forks
Source: Check Point Research, MCPoison, 2025, https://research.checkpoint.com/2025/cursor-vulnerability-mcpoison/
Say it: "Clone this repository into a new folder, but before running anything, list what would execute: package.json scripts, any .claude, .cursor or .mcp.json config, and AGENTS.md. Show me those and wait."
Described as: "downloading a project from GitHub", "making my own copy of someone's repo", "contributing to a project I don't own", "is it safe to run this repo".
https://dev.dqnotes.com/git/clone-fork

### .gitignore
Also: Ignore file.
A file listing the names and patterns git should not track, such as dependencies, build output and secrets. Git then behaves as if those files weren't there. It only applies to files git isn't already tracking.
Without one, git add -A commits node_modules and your keys. One line here is a small permanent rule, and better than asking an agent to remember. But it can't undo anything: a secret already committed stays in the history.
Not Removing a secret: Ignoring a file stops it being added in future, and leaves every past commit alone. If a key was committed, GitHub's advice starts with revoking or rotating it, because rewriting history can't reach other people's clones.
Seen in: Git's docs: "Files already tracked by Git are not affected", and "It is not possible to re-include a file if a parent directory of that file is excluded." That second one catches almost everyone once. The bench's matcher gave the same answers as git check-ignore across 25 pattern files and 21 paths.
Source: Git docs, gitignore, https://git-scm.com/docs/gitignore
Source: GitHub docs, Removing sensitive data, https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository
Say it: "Add .env, dist/ and node_modules/ to .gitignore, and run git check-ignore -v .env to prove it matches. If git ls-files shows .env is already tracked, run git rm --cached .env and tell me, because that key needs replacing."
Described as: "stop committing node_modules", "it committed my .env file", "I added it to gitignore but git still tracks it", "keeping secrets out of git".
https://dev.dqnotes.com/git/gitignore

### Commit message
Also: Conventional Commits.
The note attached to a commit. Its first line says what changed, short enough to read in a list, and the body says why. Conventional Commits is a common format for that first line: fix(cart): stop charging VAT twice.
The next session's agent starts knowing nothing and reads git log to catch up, so this is the memory you leave it. It also matters for you: a message giving a real reason shows you understood the change, and "fix stuff" shows nothing. An agent will write vague ones unless told otherwise.
Seen in: Anthropic's long-running harness asks the model to commit "with descriptive commit messages", and starts each session with "Read the git logs and progress files to get up to speed". Chris Beams's 2014 rules include "Use the body to explain what and why vs. how". On interviews, be careful: we found no employer saying commit history is what persuades them, so that is our reasoning. What Canva does say is that it expects engineers to "take full ownership of any code they produce", and a history is where you can show that.
Source: Young, Effective harnesses for long-running agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
Source: Beams, How to Write a Git Commit Message, 2014, https://cbea.ms/git-commit/
Source: Conventional Commits 1.0.0, https://www.conventionalcommits.org/en/v1.0.0/
Source: Newton, Yes, you can use AI in our interviews, Canva, 2025, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Say it: "Write the message as type(scope): subject. In the body, say why this change was needed and what you ruled out, not what the diff already shows. Never write "fix bug" or "update code"."
Described as: "what should I write when I commit", "the AI writes useless commit messages", "feat and fix at the start of commits", "nobody knows why this was changed".
https://dev.dqnotes.com/git/commit-message

### Small commit
Also: Atomic commit.
A commit that does one thing and leaves the project working. If you need the word "and" to describe it, it is probably two commits. A rename, a bug fix and a reformat are three, even if they happened in the same hour.
Left alone, an agent makes one huge commit at the end, which can't be reviewed or undone in part, and which hides the moment any bug arrived. Small commits fix all of that. A series also shows the order you thought in, such as failing test, fix, tidy, which is the kind of evidence a single large commit can't give.
Not Small pull request: A small pull request is one reviewable unit of work, and it can hold several small commits. They help each other, but neither guarantees the other.
Seen in: pstack says that sequencing small units into a delivery a reviewer can replay turns a claim into "watch it go red, then green". Kent Beck's published prompt for working with an agent says "Never mix structural and behavioral changes in the same commit". No employer we could find has said it looks for this. It is our reasoning, and you should present it that way.
Source: Tan, pstack, principle-sequence-verifiable-units, https://github.com/cursor/plugins/blob/main/pstack/skills/principle-sequence-verifiable-units/SKILL.md
Source: Beck, Augmented Coding: Beyond the Vibes, 2025, https://newsletter.kentbeck.com/p/augmented-coding-beyond-the-vibes
Say it: "Commit in this order, as separate commits: the failing test first, then the smallest fix that makes it pass, then any tidying. Every commit after the first must pass bun test, and the first must say in its message that the test is expected to fail."
Described as: "the agent made one giant commit", "too many changes in one commit", "I can't undo just the bad part", "splitting my work up".
https://dev.dqnotes.com/git/small-commit

### Revert
Undoing a commit by adding a new commit that does the opposite. The mistake stays in the history along with its undoing, and nobody else's copy is disturbed.
This is the undo for anything already pushed. It needs no force, it can't lose anyone's work, and it leaves a record of what went wrong and that someone dealt with it. Make it your agent's default, because left to itself it may reach for something worse.
Not Reset: Revert adds a commit and moves forward. Reset moves the branch backward and acts as if later commits never happened. On anything shared, use revert.
Seen in: Git's docs: revert is "used to record some new commits to reverse the effect of some earlier commits". One trap: "Reverting a merge commit declares that you will never want the tree changes brought in by the merge", so merging that branch again later won't bring those changes back.
Source: Git docs, git-revert, https://git-scm.com/docs/git-revert
Say it: "Commit a1b2c3d broke checkout and it is on main. Undo it with git revert a1b2c3d, on a new branch, through a pull request. Don't use reset or force push. Say in the message what broke and how we know."
Described as: "undoing a commit that is already on GitHub", "taking one bad change back out", "undo without rewriting history", "the safe way to undo".
https://dev.dqnotes.com/git/revert

### Reset
Also: git reset --hard.
Moving the current branch to a different commit. Soft leaves your files and staged work alone. Mixed, the default, also clears what is staged. Hard also overwrites your files to match that commit, which destroys uncommitted work.
git reset --hard is the fastest way for an agent to destroy hours of work, because it looks like tidying up. Commits it drops can usually be rescued. Uncommitted changes can't, because git never saved them. Never let an agent run it without asking first.
Not Checkout: git checkout of an old commit moves HEAD so you can look around, and the branch stays where it is. Reset moves the branch. People say "go back" for both, and only one of them loses anything.
Seen in: In January 2026 a user reported that Claude Code, told to "ROLL BACK to the last push", ran git reset --hard where a checkout would have done, and that "several hours" of uncommitted work was lost (issue 17190, closed as a duplicate). Stories need checking, though: a widely shared report that it reset repositories every 10 minutes (issue 40710) was withdrawn by its author, who found their own tool doing it. As of November 2026, auto mode blocks reset --hard by default.
Source: Git docs, git-reset, https://git-scm.com/docs/git-reset
Source: anthropics/claude-code issue 17190, https://github.com/anthropics/claude-code/issues/17190
Source: anthropics/claude-code issue 40710, https://github.com/anthropics/claude-code/issues/40710
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Say it: "Don't run git reset --hard, git checkout -- ., git restore . or git clean -f. If you need a clean tree, tell me what would be lost, and use git stash push -u -m "before this task" instead."
Described as: "the AI wiped my changes", "go back to the last commit and throw this away", "hard or soft reset, which one", "undo my last commit but keep the code".
https://dev.dqnotes.com/git/reset

### Force push
Also: git push --force, --force-with-lease.
Telling the server to replace a branch with yours, even when that throws away commits it has. A normal push can only add. This one can delete other people's work from the server, and it barely mentions it.
This is the git command whose damage lands on other people. An agent reaches for it when a normal push is refused, and a refused push often means someone else's work is in the way. On your own branch after a rebase it is routine. On a shared one, it needs a person.
Not --force-with-lease: The lease refuses if the server has moved since you last looked. That is safer, and it still isn't safe: a background fetch updates the thing it compares against. Add --force-if-includes to close that gap.
Seen in: Git's docs on --force: "It can cause the remote repository to lose commits; use it with care." GitHub: "By default, GitHub blocks force pushes on all protected branches", though admins are exempt unless you turn that off. Claude Code's auto mode blocks force push by default (November 2026), and pstack names "force-push to shared branches" among the irreversible writes an agent must pause for.
Source: Git docs, git-push, https://git-scm.com/docs/git-push
Source: GitHub docs, About protected branches, https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Source: Tan, pstack, poteto-mode SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/SKILL.md
Say it: "If a push is refused, stop and tell me why. Don't use --force, -f or a + refspec. If this is your own branch and you rebased it, use git push --force-with-lease --force-if-includes on that branch only, and never on main."
Described as: "the push got rejected so it forced it", "my colleague's commit disappeared", "overwriting what is on GitHub", "updates were rejected, what now".
https://dev.dqnotes.com/git/force-push

### Stash
A shelf for unfinished work. git stash puts your changes to tracked files aside and returns your folder to the last commit, and git stash pop brings them back. New files git isn't tracking yet stay where they are unless you add -u.
A stash has no branch and no reminder, and agents don't remember across sessions, so work stashed "to get a clean tree" can sit there unnoticed for ever. Anything you care about belongs in a commit on a branch, which costs no more and shows up in the history.
Not Commit: A commit is part of a branch's history and can be pushed. A stash is a private list on one machine. It doesn't get pushed, and a dropped stash has no normal way back.
Seen in: Git's docs: "If you mistakenly drop or clear stash entries, they cannot be recovered through the normal safety mechanisms." Claude Code's auto mode blocks git stash drop and git stash clear by default (November 2026).
Source: Git docs, git-stash, https://git-scm.com/docs/git-stash
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Say it: "If you stash, use git stash push -u -m "why", and pop it before your last step. Run git stash list at the end and show me that it is empty. If it isn't, tell me what is in there."
Described as: "putting changes aside for a moment", "switch branches without committing", "where did my changes go", "save this but not as a commit".
https://dev.dqnotes.com/git/stash

### Tag
A permanent name for one commit, usually a release such as v1.2.0. Unlike a branch, it doesn't move when you commit. An annotated tag also records who made it, when, and a message.
A tag gives you and your agent a fixed point: compare against v1.2.0, go back to the last release. An agent that moves a published tag breaks that promise for everyone who already has it, so treat tags as write-once.
Not Branch: A branch moves forward with each commit. A tag stays where you put it, so use one for a release.
Seen in: Git's docs: "Annotated tags are meant for release while lightweight tags are meant for private or temporary object labels." On moving one you have published, the sane option is "Just admit you screwed up, and use a different name." We confirmed on git 2.54 that a plain git push doesn't send tags.
Source: Git docs, git-tag, https://git-scm.com/docs/git-tag
Say it: "Tag this release with git tag -a v1.3.0 -m "Adds CSV export", then push it with git push origin v1.3.0. If the tag turns out wrong, make v1.3.1. Never move or delete a tag that is already pushed."
Described as: "marking a release", "naming a version of the code", "the last version that worked", "v1.0 in git".
https://dev.dqnotes.com/git/tag

### Blame
Also: Annotate.
A view of a file showing, for each line, the last commit that changed it and who made it. The name is unfortunate: you use it to find out why a line exists, not whose fault it is.
Before an agent changes a strange line, it should read why that line is there, since it may be preventing something invisible. That only works if the history is good, and an agent that reformats a file while fixing it makes itself the author of every line.
Not git log: git log lists commits. Blame answers a question about one line. Neither one tells you about a line that has been deleted, for which you search the history with git log -S.
Seen in: Git's docs: blame "does not tell you anything about lines which have been deleted or replaced". Pass -w to ignore spacing changes, and list formatting commits in a file for --ignore-revs-file, so that they stop stealing the credit.
Source: Git docs, git-blame, https://git-scm.com/docs/git-blame
Say it: "Before you change line 42 of src/cart.ts, run git blame -w -L 42,42 src/cart.ts, read that commit with git show, and tell me why the line is there. If you still want to change it, say what would break."
Described as: "who wrote this and why", "finding out why this weird line exists", "when did this line change", "every line says the formatter changed it".
https://dev.dqnotes.com/git/blame

### Bisect
A way to find the commit that introduced a bug by halving the range. You name a good commit and a bad one, git checks out the middle, you say good or bad, and it halves again. A thousand commits takes about ten tests.
When an agent is stuck guessing at a bug, bisect stops the guessing, because its answer is one commit. It can run unattended too: give it a script, and it finds the commit while you do something else. It depends on every commit building, and on the commit it names being small enough to read.
Not Debugging: Bisect tells you when the bug arrived. It doesn't tell you why. It also believes whatever it is told, so a flaky test sends it to the wrong commit with complete confidence.
Seen in: Git's docs: "This command uses a binary search algorithm to find which commit in your project's history introduced a bug." Their example shows "675 revisions left to test after this (roughly 10 steps)". With git bisect run, exit 0 means good, 125 means skip this commit, and any other code from 1 to 127 means bad.
Source: Git docs, git-bisect, https://git-scm.com/docs/git-bisect
Say it: "v1.2.0 worked and HEAD is broken. Write scripts/repro.sh so it exits 0 without the bug, 1 with it, and 125 if the commit won't build. Then run git bisect start HEAD v1.2.0 and git bisect run scripts/repro.sh, and show me the commit it names."
Described as: "it worked last week and now it doesn't", "which commit broke this", "finding when the bug started", "too many commits to check by hand".
https://dev.dqnotes.com/git/bisect

### Worktree
Also: git worktree.
A second working folder attached to the same repository, on its own branch. The folders share one history, but each has its own files, its own staged changes, and its own idea of which branch is current.
Two agents in one folder interfere without any error: one switches branch under the other, and a commit picks up files it shouldn't have. A worktree for each agent stops that, and it is much lighter than a full clone. Files git ignores, such as .env and node_modules, aren't there until you add them.
Not Clone: A clone is a separate repository with its own copy of the history and its own remote setup. Worktrees share one set of commits, so a commit in one is visible at once in the others.
Seen in: Git's docs: a repository "can support multiple working trees, allowing you to check out more than one branch at a time". Try the same branch twice and git refuses; we saw "is already used by worktree" on git 2.54. claude --worktree <name> makes one per session, and SwarmForge "coordinates AI agents in isolated git worktrees and tmux sessions."
Source: Git docs, git-worktree, https://git-scm.com/docs/git-worktree
Source: Git source, the message in branch.c, https://github.com/git/git/blob/master/branch.c
Source: Claude Code docs, Worktrees, https://code.claude.com/docs/en/worktrees
Source: Martin, swarm-forge, https://github.com/unclebob/swarm-forge
Say it: "Make a worktree for this task with git worktree add ../app-search -b search, install dependencies there, and work only in that folder. When it is merged, remove it with git worktree remove ../app-search."
Described as: "running two agents at once on one repo", "agents keep overwriting each other", "working on two branches at the same time", "it committed the other agent's files".
https://dev.dqnotes.com/git/git-worktree

### Git hook
A script git runs by itself at certain moments, such as just before a commit or a push. If the script exits with anything other than 0, git stops. Hooks live in your own copy of the repository and aren't part of its history.
A hook turns "please run the linter", a tool that checks code for known bad habits, into something that happens every time, and its error goes straight to the agent. But it only guards one machine, and a single flag skips it, which an agent under pressure may well use. Treat it as a fast helper, and run the same checks on the server.
Not Continuous integration: A git hook runs on your machine, and whoever is at the keyboard can skip it. CI runs on a server after the push, where the person pushing can't.
Seen in: Git's docs say the pre-commit hook "can be bypassed with the --no-verify option", and "Hooks that don't have the executable bit set are ignored." Pro Git adds that client-side hooks are not copied when you clone. So teams keep the config in the repository with a tool such as lefthook, which each person then has to install.
Source: Git docs, githooks, https://git-scm.com/docs/githooks
Source: Pro Git, Git Hooks, https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
Say it: "Add lefthook with a pre-commit job running gitleaks git --pre-commit --staged --redact, and a pre-push job running bun run typecheck. Show me each one refusing something. Never use --no-verify: if a hook fails, fix the cause or ask me."
Described as: "running checks automatically before a commit", "it skipped my pre-commit check", "my hook isn't running", "stop bad code getting committed".
https://dev.dqnotes.com/git/git-hook

### GitHub
Also: Forge, GitLab, Hosting site.
A website that hosts git repositories and adds what git lacks: pull requests, reviews, issues, permissions and automated checks on every push. Git is the tool. GitHub is one company's service built around it, and GitLab is another.
Everything on your machine can be skipped by whoever is at the keyboard, and that includes your agent. The server is where rules really hold: branch protection, required checks, required review. An agent with limited access can't talk its way past any of them.
Not Git: Git runs on your machine and needs no account. GitHub is a service that hosts git repositories. Plenty of people use the first word when they mean the second, and in an interview it is worth saying it precisely.
Seen in: GitHub's docs describe Git as "a version control system that tracks changes to files", and GitHub as "based on the open-source software, Git." Its repository rulesets include Block force pushes, Require a pull request before merging, and Require status checks to pass before merging.
Source: GitHub docs, About GitHub and Git, https://docs.github.com/en/get-started/start-your-journey/about-github-and-git
Source: GitHub docs, About rulesets, https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets
Say it: "Using gh, add a ruleset for main: require a pull request, require the ci check to pass, and block force pushes and deletions. Then try a direct push to main and show me the refusal."
Described as: "is git the same as GitHub", "the website where my code lives", "stopping people pushing to main", "where the checks and reviews happen".
https://dev.dqnotes.com/git/github

### Reflog
Git's private diary of everywhere HEAD has been: each commit, checkout, reset and rebase. When a commit seems lost, it is usually listed here, and you can walk straight back to it.
This is the real undo, and what makes it reasonable to let an agent move fast: anything it committed can usually be recovered, even after a bad reset or rebase. Its limits matter as much. It is local to one machine, it expires, and it knows nothing about work that was never committed.
Not git log: git log shows what you can reach from where you are now. The reflog shows where you have been, including commits no branch points at any more. So log can look as if your work has vanished when it hasn't.
Seen in: Git's docs: reflogs "record when the tips of branches and other references were updated in the local repository." Defaults: entries expire after 90 days, or 30 if the commit is no longer reachable from where you are, and git gc then prunes unreachable objects older than 2 weeks.
Source: Git docs, git-reflog, https://git-scm.com/docs/git-reflog
Source: Git docs, git-gc, https://git-scm.com/docs/git-gc
Say it: "You reset past my work. Run git reflog, find the last entry before the reset, tell me its hash and message, and make a branch there with git branch rescue and that hash. Don't run gc, and don't reset anything else."
Described as: "I lost my commits", "the AI reset my branch, can I get it back", "git log doesn't show my work any more", "undoing a bad rebase".
https://dev.dqnotes.com/git/reflog

### Git guardrails
Rules at three levels that stop dangerous git commands: a deny rule in the agent's settings, a script that inspects each command before it runs, and protection on the server. Each one catches things the others miss.
A line in an instructions file is a request, and an agent trying hard to finish will do what looks like progress. Local rules catch local damage, such as reset and clean, which no server ever sees. The server catches whatever reaches it, however it was spelled. You need both, and each needs testing.
Not A line in AGENTS.md: That line shapes what the agent tries. A rule enforced outside the model decides what it can do. Write the line as well, because it saves a refused attempt, but don't rely on it.
Seen in: Claude Code's docs: "Permission rules are enforced by Claude Code, not by the model." Deny rules apply in every permission mode, but a rule like Bash(git push *) doesn't match git -C . push origin main, and the docs say such a rule "covers the invocation Claude usually produces and isn't a security boundary". For full-text checks they point to a PreToolUse hook. All as of November 2026.
Source: Claude Code docs, Configure permissions, https://code.claude.com/docs/en/permissions
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Say it: "In .claude/settings.json add these to permissions.deny: Bash(git push --force *), Bash(git push -f *), Bash(git reset --hard *), Bash(git clean *). Add a PreToolUse hook that reads the full command and exits 2 on any force push, reset --hard or --no-verify. Then try each command and show me it being refused."
Described as: "how do I stop it force pushing", "stopping the AI deleting my work", "it ran a dangerous git command", "rules the agent can't get around", "what permissions should it have for git".
https://dev.dqnotes.com/git/git-guardrails

## 4. Reading and debugging

How to read code you did not write, and how to find a bug when the agent has run out of ideas.

### Reading code
Working out what code does by following it. You rarely read a file from top to bottom. You start where the program starts, or where the problem showed up, then follow the calls, each place one piece of code starts another, and ignore everything that path never touches.
An agent can write more in a minute than you can read in an hour, so reading all of it is off the table. What you can do is follow one path through a change and ask about any step you can't explain. That is supervision, and it is what an interviewer is testing when they point at a file and say walk me through it.
Not Reading a diff: A diff shows what changed. Reading code shows what it does now, including all the parts the diff didn't touch.
Seen in: In a randomised trial Anthropic published in January 2026, 52 mostly junior engineers learned a new library with or without AI help. The AI group averaged 50% on a quiz afterwards, against 67% by hand, and Anthropic reports: "The largest gap in scores between the two groups was on debugging questions." It was a small study, with the quiz taken straight after the task, and Anthropic calls the findings preliminary.
Source: Anthropic, How AI assistance impacts the formation of coding skills, 2026, https://www.anthropic.com/research/AI-assistance-coding-skills
Say it: "Before you change anything, trace what happens when POST /api/checkout is called. List each function in the order it runs, with its file and line, and stop at the first database call. I'll read along."
Described as: "where do I even start reading this code", "the file is huge and I am lost", "how do people understand code they did not write", "which parts of this actually run".
https://dev.dqnotes.com/reading/reading-code

### Variable
A name for a value the program is holding, like a label on a box. In JavaScript, let means the name can be given a new value later, and const means it can't.
Most bugs come down to a variable not holding what someone assumed it held. When you read an agent's code, the question to keep asking is: what is in this name, right now, on this line?
Not Spreadsheet cell: A spreadsheet cell recalculates when its inputs change. A variable holds whatever it was last given, and stays that way until something assigns to it again.
Say it: "This function reassigns result five times. Give each step its own const, with a name that says what it holds, and change no behaviour. Show me the tests still pass."
Described as: "I changed the price but the total stayed the same", "what is let and const", "a name that stores a value", "why did this not update".
https://dev.dqnotes.com/reading/variable

### Function
A named piece of code you run by writing its name followed by brackets. It lets one idea be written once and used in many places, and it lets a reader take in the name rather than the details.
An agent that hasn't seen the function you already have will happily write a second one. Now one rule lives in two places, and the next fix only reaches one of them. So it pays to ask whether something exists before anything gets added.
Not Method: A method is a function that belongs to an object and is called through it, as in cart.total(). More on those below.
Say it: "Before adding a helper, run rg -n "formatMoney|formatPrice" src/ and tell me what exists. Use it if it fits. If you add a new one, put it in src/lib/money.ts and say why the old one didn't fit."
Described as: "the same code is copied in three places", "what does the word function mean", "a named block of code", "what are the brackets after the name for".
https://dev.dqnotes.com/reading/function

### Parameters and return values
Also: Arguments.
Parameters are the inputs a function asks for, named in its brackets. The values you pass in are the arguments. return hands one value back to the caller and ends the function, and a function with no return hands back undefined, JavaScript's word for nothing.
A function's parameters and return value are its promise to the rest of the code. When an agent changes them, every caller is affected, including the ones it didn't read.
Not Printing: console.log shows a value to a person. return gives a value to the code that called. A function can print the right answer and still return nothing.
Say it: "You changed the parameters of createOrder. Run rg -n "createOrder\(" src/ and list every call site, then confirm each one passes the new arguments in the right order. Show me the list."
Described as: "it works out the answer but I get undefined", "it prints the right thing but the result is empty", "what goes in the brackets", "NaN shows up further down".
https://dev.dqnotes.com/reading/parameter-return

### Type
What kind of value something is: a number, some text (called a string), true or false (a boolean), and so on. The type decides what an operation means, so + adds two numbers but glues two strings together.
Whatever arrives from a form or an API is often text, even when it looks like a number. In plain JavaScript that gives a wrong answer with no error. TypeScript, which is JavaScript with types written in, turns it into an error before the code runs, and an agent can read that error and fix it by itself.
Not Type checker: A type is a property of a value. A type checker is a tool that reads your code and reports where the types can't line up.
Seen in: typeof null gives "object" in JavaScript. MDN explains this as a leftover from the first implementation, and notes that a proposed fix was rejected. Run it yourself on the instrument. The TypeScript errors shown there were recorded from TypeScript 5.9.3.
Source: MDN, typeof, https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/typeof
Say it: "The price comes from a form, so it is a string. Convert it once, where it enters, using Number(). Reject NaN with a clear error, and type it as number everywhere after that. No any."
Described as: "adding two numbers gives 205", "it says NaN", "the number is actually text", "why is this a string".
https://dev.dqnotes.com/reading/type

### Conditions and loops
Also: If statement, For loop.
A condition (if) runs some code only when a test is true. A loop (for, while) repeats some code until its test turns false. Between them, they decide which lines run and how many times.
Reading an agent's code, every if is a question: what happens in the other case? A branch nobody wrote and a loop that stops a step early are where a lot of almost-right code hides.
Not Recursion: Recursion is a function that calls itself, which repeats without a loop. Each call waits on the stack for the next one to finish.
Say it: "This function has four nested ifs. Rewrite it with early returns so the main path is not indented, keep the behaviour identical, and add a test for each branch, including the one with no else."
Described as: "the page froze", "it never stops running", "do this for every item", "only do this if something is true".
https://dev.dqnotes.com/reading/conditional-loop

### Lists and objects
Also: Array, Dictionary.
A list, called an array, holds values in order, numbered from 0. An object holds values under names, like a small form. Both live in a box in memory, and a variable holds a pointer to the box, not what is in it.
That pointer is behind a whole family of bugs where you change a "copy" and the original changes too. Spot them by asking if the code made a new box or only a new name. JSON, from the network chapter, is these same two shapes written out as text.
Not Shallow copy: { ...cart } makes a new outer box and shares everything inside it. A deep copy makes new boxes all the way down. Most copies you will read are shallow.
Say it: "Don't mutate the cart argument in applyDiscount. Return a new object with a new items array, and add a test that checks the original cart is unchanged after the call."
Described as: "I changed the copy and the original changed too", "two variables seem to be linked", "why does the list start at 0", "a list of things with names".
https://dev.dqnotes.com/reading/array-object

### Modules and imports
A module is a file that shares some of what it defines using export. Another file pulls those parts in using import. Anything not exported stays private, so a module's insides can change without breaking the files that use it.
The import lines at the top are the quickest map of a file: they say what it depends on. They are also where agents go wrong in a way you can check, by importing a function that doesn't exist or a package nobody ever published.
Not Dependency: A module is a file in your code. A dependency is someone else's code, installed by a package manager and imported in the same way.
Seen in: Package names that models make up are common enough to measure. A 2026 preprint by Aleksandr Churilov, not peer reviewed, tested five models on 199,845 prompts and found invented package names in 4.62% to 6.10% of cases, depending on the model.
Source: Churilov, arXiv 2605.17062, 2026, https://arxiv.org/abs/2605.17062
Say it: "Run bun run typecheck after adding imports. For every new package in package.json, show me its page on npmjs.com and how many weekly downloads it has before you install it."
Described as: "cannot find module", "does not provide an export named", "how does one file use another", "the agent imported something that does not exist".
https://dev.dqnotes.com/reading/module-import

### Null and undefined
Two ways JavaScript says "nothing here". undefined usually means nothing was ever set, and null usually means someone set it to nothing on purpose. Reading a property of either one crashes the program, with an error you will see often.
Agents like ?., a shortcut that skips the read when the value is missing, because it makes the crash go away. But it swaps an error you would notice for a wrong answer you might not. The real question is what should happen when the value is missing, so ask that, and then have it written down in the code.
Not Zero or empty text: Zero and "" are real values. null means no value at all. Code that writes value || "default" replaces 0 and "" as well, where ?? only replaces null and undefined.
Seen in: Tony Hoare added the null reference to ALGOL W in 1965, and at QCon London in 2009 said: "I call it my billion-dollar mistake." The figure is his own rough estimate of the damage over forty years, not a measurement.
Source: Hoare, Null References: The Billion Dollar Mistake, QCon London 2009, https://qconlondon.com/london-2009/qconlondon.com/london-2009/presentation/Null%2BReferences_%2BThe%2BBillion%2BDollar%2BMistake.html
Say it: "user.address can be missing for guest checkouts. Don't add ?. to silence it. Show a "no address yet" message on the page, and add a test with a guest user that checks that message appears."
Described as: "Cannot read properties of undefined", "undefined is not an object", "it says undefined on the page", "it crashes when a field is missing".
https://dev.dqnotes.com/reading/null

### Exception
Also: Thrown error.
An error that stops the current function and climbs back through whoever called it, until some code catches it using try and catch. If nothing does, the program ends.
An empty catch hides the failure from you and from every automatic check, which makes it a very easy way to turn a failure into a pass. Question any catch that does not log, retry, rethrow or tell someone.
Not Exit code: An exception lives inside a running program. An exit code is the one number a finished program gives the outside world. In Node, an exception nothing catches ends the process with a non-zero exit code.
Say it: "Search src/ for catch blocks that do nothing. For each one, rethrow it, log it with the request id, or explain to me why ignoring it is right. Then add a lint rule that bans empty catch blocks."
Described as: "it failed but nothing showed up", "the error just disappears", "what is try and catch", "it fails silently".
https://dev.dqnotes.com/reading/exception

### Stack trace
Also: Backtrace, Traceback.
What a crash leaves behind: the error message, then every function that was running at that moment, newest first, each with a file and a line. It is a map from where it blew up back to where it started.
It is the most precise bug report you will ever get, and it's free. Give an agent the whole thing, not your summary of it. Then check its story against the trace: if it blames a function that isn't there, it is guessing.
Not Log: A log is what your program chose to write down as it went. A stack trace is written by the runtime at the moment of failure, whether or not anyone planned for it.
Seen in: The stack property is not part of any JavaScript standard, though every major engine has it. MDN shows the same call as "at baz (filename.js:10:15)" in V8 and "baz@filename.js:10:24" in JavaScriptCore. Turn on Raw text on the instrument to see what your own browser writes.
Source: MDN, Error.prototype.stack, https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error/stack
Say it: "Here is the full stack trace. Start from the top frame that is in our code, open that file at that line, and tell me which value was wrong there and where it came from, before you propose any fix."
Described as: "a wall of red text", "what are all those lines starting with at", "which line actually broke", "the error points at a file I did not write".
https://dev.dqnotes.com/reading/stack-trace

### Classes and objects
A class is a blueprint. Each object made from it with new gets its own data and shares the functions, called methods. Inside a method, the word this means the object it was called on.
Classes gather data and the rules for changing it in one place, and you will read plenty even if you never write one. When an agent's class grows long, ask what it holds and what changes it, because a class with many reasons to change is hard to review.
Not Plain object: In JavaScript, any { name: value } is already an object, with no class involved. A class is one way of making objects that share methods.
Say it: "This class has 14 methods and 9 fields. List which methods use which fields, and propose how to split it so each part has one reason to change. Do not refactor anything yet."
Described as: "what does this mean in code", "this is undefined", "what is the word new doing", "difference between a method and a function".
https://dev.dqnotes.com/reading/class-object

### Async and await
Also: Promise.
Slow work, such as a network call, doesn't hold everything up. The program gets a promise of a result and moves on. await pauses one function until its promise finishes, and other code gets to run in the meantime.
A missing await is a classic almost-right bug: the code runs, nothing errors, and it reports success before the work has happened. Look for it in anything an agent writes that saves, sends or charges.
Not Running in parallel: JavaScript still runs one line at a time. await lets other code take a turn while something is waiting. It does not make two things run at once.
Seen in: The order on the instrument is specified behaviour, not luck. MDN on the event loop: "tasks and microtasks are kept on separate queues, and that microtasks run first." A promise's callback is a microtask, and a timer's is a task.
Source: MDN, Using microtasks in JavaScript with queueMicrotask(), https://developer.mozilla.org/en-US/docs/Web/API/HTML_DOM_API/Microtask_guide
Say it: "Turn on the no-floating-promises rule from typescript-eslint, fix everything it reports, and list each place where a promise was not being awaited, saying what could have gone wrong there."
Described as: "it says done before it has finished", "it prints in the wrong order", "I see Promise object where the value should be", "why does this log 1 3 2".
https://dev.dqnotes.com/reading/async-await

### State
Everything a program remembers at a given moment: its variables, what is in its database, what you have selected. The same function can behave differently depending on state, so a bug can depend on what happened before, not only on what you did just now.
Agents read code, and state is mostly not in the code. It only exists while the program runs, which is why an agent can stare at correct-looking code and find nothing. So hand it the sequence of steps, not only the symptom.
Not Configuration: Configuration is set before the program starts, and stays put. State changes while it runs.
Say it: "This only fails on a second checkout in the same session. Write a test that does add, pay, new cart, add, pay in that order and checks the customer is charged twice. It should fail. Then fix it."
Described as: "it only breaks the second time", "it works after a refresh", "it depends on what I did before", "turning it off and on again fixes it".
https://dev.dqnotes.com/reading/state

### Side effect
Anything a function does beyond handing back its answer: writing to a database, sending an email, changing something outside itself, even printing. A program's useful work is mostly side effects, and so is most of its risk.
An agent retrying a failed step reruns its side effects, so the email goes out twice and the card is charged twice. When you read its code, find the lines that change the outside world. Those are the ones you can't take back.
Not Return value: The return value is what a function says. A side effect is what it does. A test (a small program that runs your code and checks the answer) can't see the email if it only looks at the return value.
Say it: "Keep the calculation and the charge apart. Put the total in a plain function with no I/O and test that, and make chargeCard the only place that talks to Stripe, using an idempotency key."
Described as: "it sent the email twice", "calling this changes something somewhere else", "it does more than its name says", "a function that writes to the database".
https://dev.dqnotes.com/reading/side-effect

### Pure function
A function whose answer depends only on its inputs, and which changes nothing else. Same input, same output, every time. Code that is not pure is called impure, which is no insult: impure code does the useful work.
Pure functions are the easiest code to review, because you can understand one without knowing about the rest of the program. Asking an agent to push its logic into them is one of the cheapest ways of getting code you can check.
Not Idempotent: Idempotent means doing it twice has the same effect as doing it once, and it can still write to a database. Pure means no effects at all.
Seen in: JavaScript has both versions of the same idea. sort() rearranges the list it was given. MDN describes toSorted() as "the copying version of the sort() method", which leaves the original alone. Try both on the instrument.
Source: MDN, Array.prototype.toSorted(), https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/toSorted
Say it: "Split out calculateDiscount(cart, coupon) as a pure function: no database, no clock, no global state. Pass today's date in as a parameter. Test it with a table of inputs and expected outputs."
Described as: "it is hard to test because it needs the database", "the same input gives different answers", "a function with no surprises", "it depends on the time of day".
https://dev.dqnotes.com/reading/pure-function

### Scope and closures
Scope is which names a line of code can see. A name made inside a function or a block can only be seen there. A closure is a function that keeps hold of the variables around where it was created, even after that code has finished.
Agents write functions that get called later all the time, and closures explain a whole family of bugs where one of them uses an old value. If a button acts on out-of-date data, this is the first thing to suspect.
Not Global variable: A global can be seen and changed from anywhere. A closure's variable can only be reached by the functions created beside it, so nothing else can change it.
Seen in: MDN: "In JavaScript, closures are created every time a function is created, at function creation time." The same page walks through the var-in-a-loop mistake on the instrument, where several closures share "the same single lexical environment".
Source: MDN, Closures, https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Closures
Say it: "This click handler logs an old count. I think its closure captured the value from an earlier render. Confirm that with a log first, then fix it using setCount((c) => c + 1). Do not add a ref."
Described as: "it says the variable is not defined", "it is using an old value", "all the buttons do the same thing", "how does this function still know that".
https://dev.dqnotes.com/reading/scope

### Logging
Also: Print debugging.
Making the program write down what it is doing as it runs, with something like console.log. It lets you see inside a run without stopping it, and what comes back is evidence, not a theory about what happened.
Logging suits agents better than almost any other debugging tool, because an agent works by running a command and reading the output. When one starts theorising about a bug, tell it to add logs and report what they said. Then make sure it takes them out.
Not Debugger: A debugger pauses one run so you can look around by hand. Logs record the whole run, including on machines you can't pause.
Seen in: Cursor's Debug Mode, added in Cursor 2.2 in December 2025, is built around this. The agent proposes several possible causes, adds logging to test them, and has you reproduce the bug. It reads the logs, proposes a fix, and removes the logging once you confirm the bug has gone.
Source: Cursor, Introducing Debug Mode, December 2025, https://cursor.com/blog/debug-mode
Say it: "Don't guess yet. Add a labelled console.log at each step of calculateTotal, printing the values. Run it on the failing input, paste the output, and tell me the first line where a value is wrong. Then remove the logs."
Described as: "print statements everywhere", "how do I see the value at this point", "I cannot tell what the code is doing", "add some logging".
https://dev.dqnotes.com/reading/log-line

### Debugger
A tool that runs your program under your control. It pauses on a line you chose, shows you every variable and the call stack, and moves on one step at a time. Browsers have one built in, and so do most code editors.
Coding agents mostly work by running commands and reading what comes back, so pausing a live program and looking around is awkward for them. For a person it is one of the fastest ways of finding out what is really true, and being able to drive one with the agent off is worth showing.
Not Debugging: Debugging is the whole job of finding and fixing a fault. A debugger is one tool for doing it.
Seen in: In Chrome DevTools, the docs say: "Call debugger from your code to pause on that line." In Node, node --inspect-brk starts the program paused, before any of your code has run, so you can attach a debugger from the first line.
Source: Chrome DevTools docs, Pause your code with breakpoints, https://developer.chrome.com/docs/devtools/javascript/breakpoints
Source: Node.js docs, Debugging, https://nodejs.org/en/learn/getting-started/debugging
Say it: "Give me the exact command to run this test under the debugger, paused at the start, using node --inspect-brk. I will step through parseOrder myself and tell you what I find."
Described as: "pause the program and look around", "go through it line by line", "step through the code", "see every variable at once".
https://dev.dqnotes.com/reading/debugger

### Breakpoint
A mark on a line that tells the debugger to pause there, so the program can run at full speed up to the part you care about. A conditional breakpoint only pauses when a test you wrote is true.
You can't press Step sixty thousand times. A condition such as price < 0 makes the machine do the searching. Working out what would only be true at the moment things go wrong is most of the skill, and that same question makes your instructions to an agent sharper.
Not Logpoint: A logpoint writes a message and carries on without pausing. It is a log line you add from the debugger, with no change to the code.
Seen in: Chrome DevTools: "Use a conditional line-of-code breakpoint when you want to stop the execution but only when some condition is true."
Source: Chrome DevTools docs, Pause your code with breakpoints, https://developer.chrome.com/docs/devtools/javascript/breakpoints
Say it: "I paused at line 88 and order.total is already wrong there, but the inputs were right at line 40. Read lines 40 to 88 and list every place that changes total."
Described as: "stop only when this value is wrong", "pause on that line", "stepping through takes forever", "it only goes wrong on the 500th time round".
https://dev.dqnotes.com/reading/breakpoint

### Reproduction
Also: Repro, Steps to reproduce.
A set of steps that makes the bug happen, ideally every time. Without one, you can't tell if a change fixed the bug or if it just didn't happen this time. A bug you can't reproduce isn't fixed, only quiet.
Agents like to jump straight to a fix, and an agent that never saw the failure can report success with nothing behind it. Make it produce the failure first, and show you. The best form of this is a failing test.
Not Bug report: A report says what someone saw. A reproduction is something you can run, to make it happen again.
Seen in: pstack's bug-fix playbook asks for the reply to "Paste failing-then-passing repro output verbatim." The QA role in SwarmForge has the instruction "Reproduce failures before changing code."
Source: Tan, pstack, bug-fix playbook, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/bug-fix.md
Source: Martin, SwarmForge, QA.prompt on the six-pack branch, https://github.com/unclebob/swarm-forge/blob/six-pack/swarmforge/roles/QA.prompt
Say it: "Reproduce this before you touch the code. Write a failing test in tests/checkout.test.ts using this exact input, run it, and paste the red output. Then fix it and paste the green output."
Described as: "it only happens sometimes", "it works on my machine", "it happened once and went away", "I cannot make it fail again".
https://dev.dqnotes.com/reading/reproduce

### Minimal reproduction
Also: Reduced test case, Delta debugging.
The smallest input or program that still shows the bug. You get there by removing things until the failure goes away, then putting the last thing back. What remains is often close to an explanation.
A small reproduction is cheap to hand to an agent and leaves it nothing irrelevant to chase, so you get a better answer for less.
Not Regression test: A minimal reproduction is how you find out. A regression test is what you keep afterwards. One often becomes the other.
Seen in: Andreas Zeller and Ralf Hildebrandt automated this in 2002 and called it delta debugging. In their case study, a Mozilla crash that took 95 user actions came down to 3, and 896 lines of HTML came down to one. The instrument runs their ddmin algorithm.
Source: Zeller and Hildebrandt, Simplifying and Isolating Failure-Inducing Input, IEEE TSE, 2002, https://doi.org/10.1109/32.988498
Source: The same paper, a copy on a Purdue course page (PDF), https://www.cs.purdue.edu/homes/xyzhang/fall07/Papers/delta-debugging.pdf
Say it: "This 4,000 row CSV crashes the import. Write a script that deletes rows and reruns the import until it has the fewest rows that still crash, and save them as tests/fixtures/import-crash.csv."
Described as: "cut it down to the smallest thing that breaks", "which part of this input matters", "the file is too big to share", "strip it back until it still fails".
https://dev.dqnotes.com/reading/minimal-repro

### Hypothesis
One specific guess about the cause that a single check could prove wrong, such as "the price is already a string by line 40". Debugging goes fastest when you test one at a time and change one thing at a time.
If an agent changes five things and the bug goes away, nobody knows which one did it, and the other four stay in your code as superstition. Ask for the guess and the check before any change, because a guess you can disprove in one command is cheap.
Not Fix: A hypothesis predicts what you will see. A fix changes the code. A change made because it might help is still a hypothesis, however confident it sounds.
Seen in: pstack's bug-fix playbook: "Belt-and-suspenders that 'might help' is a hypothesis, not a fix. It does not ship." Cursor's Debug Mode starts the same way, with an agent that "generates multiple hypotheses" before it changes anything.
Source: Tan, pstack, bug-fix playbook, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/bug-fix.md
Source: Cursor, Introducing Debug Mode, December 2025, https://cursor.com/blog/debug-mode
Say it: "Give me three hypotheses for why this fails, each with one command or log that would rule it out. Run those checks and report what they showed. Change no code until one has survived."
Described as: "changing things at random until it works", "the agent keeps guessing", "I changed five things and now it works", "how do I narrow this down".
https://dev.dqnotes.com/reading/hypothesis

### Debugging by halving
Also: Binary search.
Finding a fault by checking the middle. If things are still right there, the bug is in the later half, and if not, the earlier half. Each check rules out half of what is left, so a thousand places takes about ten checks.
It needs no understanding of the code, only a way of telling good from bad, so it still works when you are lost and the agent is too. Checking in order is the natural thing to try, and the slow one.
Not git bisect: git bisect halves your history to find the commit that brought a bug in. Halving works on anything in an order: steps in a run, rows in a file, lines in a config.
Seen in: It is built into git. The docs for bisect open: "This command uses a binary search algorithm to find which commit in your project’s history introduced a bug."
Source: Git docs, git-bisect, https://git-scm.com/docs/git-bisect
Say it: "Run git bisect between v1.4 and HEAD, using git bisect run bun test tests/checkout.test.ts as the check. Tell me which commit it finds and show me its diff."
Described as: "it worked last week and I do not know what changed", "which of these broke it", "cut the problem in half", "too many places it could be".
https://dev.dqnotes.com/reading/binary-search-debug

### Explain it out loud
Also: Rubber duck debugging.
Explaining the problem line by line to anything at all: a colleague, a pet, a rubber duck. You often spot the mistake part way through, because explaining makes you say what you had only been assuming.
Writing a careful bug report for an agent does the same job, and you will often find the answer before you send it. It works the other way too: making an agent explain code back to you shows what it assumed, though its account of its own reasons can be a made-up story.
Not Asking for help: Nobody has to answer. The duck's only job is to make you be precise.
Seen in: The name comes from an anecdote in The Pragmatic Programmer by Andy Hunt and Dave Thomas, about someone who carried a rubber duck around and explained code to it.
Source: Wikipedia, Rubber duck debugging, https://en.wikipedia.org/wiki/Rubber_duck_debugging
Say it: "Don't fix anything yet. Explain this function to me line by line, saying what you expect each variable to hold at that point. Mark any line you are not sure about with UNSURE."
Described as: "I worked it out while I was explaining it", "talking it through", "as soon as I asked, I saw the answer", "explaining it to someone who does not know the code".
https://dev.dqnotes.com/reading/rubber-duck

### Root cause
Also: Symptom.
The symptom is what you can see going wrong. The root cause is the earliest wrong thing that produced it, and the place where a fix ends it for good. A fix at the symptom only hides it, and it comes back somewhere else.
An agent told to "fix the NaN" will do exactly that, possibly by hiding the NaN, and the test it writes will agree. Keep asking why the value was wrong until you reach the place it was created.
Not Workaround: A workaround is a symptom fix you chose on purpose and wrote down. That is fine. The trouble starts when nobody knows it is one.
Seen in: In Stack Overflow's 2025 survey, developers' top frustration, at 66%, was "AI solutions that are almost right, but not quite", and 45.2% said "Debugging AI-generated code is more time-consuming". pstack's bug-fix reply format starts with: "what was broken, root cause, fix, how you verified."
Source: Stack Overflow Developer Survey 2025, AI, https://survey.stackoverflow.co/2025/ai
Source: Tan, pstack, bug-fix playbook, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/bug-fix.md
Say it: "You removed the NaN, but why was it NaN? Trace the value back from the screen to where it first went wrong, tell me the file and line, and fix it there. Take out the || 0."
Described as: "the fix just moved the problem", "it keeps coming back", "patching over it", "why was it wrong in the first place".
https://dev.dqnotes.com/reading/root-cause

### Off-by-one
Also: Fencepost error.
Being wrong by exactly one at an edge: stopping a step too early or too late, or miscounting because lists number from 0. A list of 5 has positions 0 to 4, so position 5 is off the end.
These get past casual checking, because the middle of the data is fine and only the edges break: the first item, the last item, the empty list. Agents make them too. So ask for tests at the edges, because reading the code carefully is not a reliable way of catching one.
Seen in: Martin Fowler's page on Phil Karlton's saying about hard things records Leon Bambrick's variation: "There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors." On why lists count from 0, the classic argument is Edsger Dijkstra's EWD 831, from August 1982.
Source: Fowler, TwoHardThings, https://martinfowler.com/bliki/TwoHardThings.html
Source: Dijkstra, Why numbering should start at zero, 1982, https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD831.html
Say it: "Add tests for getPage at the edges: page 1, the last page, one past the last page, an empty list, and a list with exactly one full page. Run them before you change the code."
Described as: "the last item is missing", "there is one extra blank row", "undefined at the end of the list", "wrong by exactly one".
https://dev.dqnotes.com/reading/off-by-one

### Heisenbug
A bug that goes away or changes when you try to look at it, because looking changes the timing or the conditions. The name is a joke on Heisenberg's uncertainty principle.
Logs and debuggers change timing, so "I added a log and now it works" is a finding, not a fix. An agent can easily report that run as a success. Look for something that depends on timing, which usually means two things sharing state.
Not Flaky test: A flaky test passes and fails with no change at all. A heisenbug reacts to being observed. The same timing problem can cause both.
Seen in: Wikipedia traces the earliest known use to the proceedings of a 1983 ACM symposium on high-level debugging, which credit an unnamed participant. Jim Gray used the word in a 1985 paper, and is sometimes mistakenly credited with it. Who coined it is not settled.
Source: Wikipedia, Heisenbug, https://en.wikipedia.org/wiki/Heisenbug
Say it: "The test passes with the log and fails without it, so this is timing. Don't keep the log. Find which two things run at once and share balance, write a test that forces the bad order, and then fix it."
Described as: "it goes away when I add a print", "it only fails in production", "it works in the debugger", "adding logging fixed it somehow".
https://dev.dqnotes.com/reading/heisenbug

### Comments
Text in a file meant for people, which the program ignores. In JavaScript it starts with //. The useful ones say why the code is the way it is, which the code itself can't tell you.
Because the program ignores comments, nothing keeps them honest, and agents read them as fact. A stale comment misleads the next agent as well as you. Cut the ones that only repeat the line below, and keep the reasons.
Not Documentation: Documentation tells people how to use code from the outside. A comment explains one decision, at the place it matters.
Seen in: The Python style guide, PEP 8, opens its section on comments with: "Comments that contradict the code are worse than no comments."
Source: PEP 8, Comments, https://peps.python.org/pep-0008/
Say it: "Delete comments that only restate the code. Keep one wherever a decision would surprise a reader, and write why: for example, // capped at 3 retries because the provider limits us to 5 calls a second. Change nothing else."
Described as: "the comment says one thing and the code does another", "comments everywhere saying what each line does", "the grey text in the code", "should I ask for more comments".
https://dev.dqnotes.com/reading/code-comment

### Grep and code search
Also: ripgrep, Regular expression.
Searching every file for some text or a pattern. grep is the old command, and ripgrep (rg) a fast modern one. A regular expression is a small language for patterns: \bprice\b finds the word price, but not priced.
It is how agents find their way around a codebase, and how you check what they tell you. "Nothing else calls this" is a claim you can test with one command, so test it.
Not Language server: Search matches letters, so it finds comments and strings too, and it works on any file. A language server understands the code, so it finds what a name really refers to.
Seen in: ripgrep: "By default, ripgrep will respect gitignore rules and automatically skip hidden files/directories and binary files." As of November 2026, Claude Code's docs say that on macOS and Linux it searches using grep through its shell, which runs an embedded ugrep, while its separate Grep tool, in the default set only on Windows, is built on ripgrep.
Source: ripgrep, https://github.com/BurntSushi/ripgrep
Source: Claude Code docs, Tools reference, https://code.claude.com/docs/en/tools-reference
Say it: "Before you delete formatPrice, run rg -n "formatPrice" . and paste the output. Also look for names built at runtime, such as "format" + name. Then tell me why it is safe."
Described as: "find everywhere this is used", "where is this in the code", "search the whole project", "is anything still using this".
https://dev.dqnotes.com/reading/code-search

### Language server
Also: Go to definition, LSP.
A program running beside your editor that understands your code's structure. It powers go to definition, find references, rename and the red underlines. Because it follows scope and imports, it knows which of five things called total you mean.
Search shows every place a word occurs. A language server shows the places that mean this thing, and that difference matters when something gets renamed. Agents can use one as well, and it is often worth setting up for them.
Not Language Server Protocol: A server knows one language. The protocol is the shared way that editors talk to any server, so one editor can support many languages. MCP, later in the course, was inspired by it.
Seen in: The protocol's site says it is used "between an editor or IDE and a language server". As of November 2026, Claude Code has an LSP tool that can jump to definitions and find references, which stays inactive until you install a code intelligence plugin for your language.
Source: Language Server Protocol, https://microsoft.github.io/language-server-protocol/
Source: Claude Code docs, Tools reference, https://code.claude.com/docs/en/tools-reference
Say it: "Rename calculateTotal to orderTotal using the language server's rename, not find and replace. Then run bun run typecheck and show me that it is clean."
Described as: "jump to where this is defined", "right click, go to definition", "rename it everywhere safely", "which total does this refer to".
https://dev.dqnotes.com/reading/go-to-definition

### Fix loop
Also: Going round in circles.
When an agent tries fix after fix for the same bug, each one plausible, none of them working, some undoing the last. More attempts rarely help when its idea of the problem is wrong. This is the point at which you step in.
Stop it, go back to the last good commit, get a reproduction, halve the search, and hand back something specific you have observed, not "still broken". This is what "I can debug without the agent" means in practice, and it is a skill employers now ask about directly.
Not Backpressure: With backpressure, which chapter 9 covers, the agent corrects itself against a check and gets closer each time. A fix loop has the same machinery and no progress: same failure, new guess.
Seen in: Assembled's CTO lists candidates who "get stuck on bugs that AI couldn’t solve out of the box" among the weak patterns they saw in AI-allowed interviews. Canva's interviewers ask themselves: "Can they identify and fix issues in AI-generated code?"
Source: Wang, How we learned to stop worrying and love the AI in coding interviews, Assembled, 2025, https://www.assembled.com/blog/how-we-learned-to-stop-worrying-and-love-the-ai-in-coding-interviews
Source: Newton, Yes, You Can Use AI in Our Interviews, Canva, 2025, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Say it: "Stop changing code. Three fixes have failed, so your idea of the cause is wrong. Go back to the last commit, write a failing test for this bug, add labelled logs at each step, and report only what you observed."
Described as: "it keeps trying the same thing", "going round in circles", "each fix breaks something else", "the agent is stuck", "tried four times and it is still broken".
https://dev.dqnotes.com/reading/fix-loop

## 5. The model

What a language model is and is not: tokens, context, sampling, tools, cost, and why it sounds sure when it is wrong.

### Language model
Also: LLM, Large language model.
A program that takes text and outputs a probability for each piece of text that could come next. One piece gets picked and added to the text, and the step repeats. Everything it appears to do is built out of that step.
An agent's code, plans and progress reports are all output of this step. Reading well only shows that the text is likely, so whether it is correct has to be settled by something else.
Not A database: A database returns what was stored, or nothing. A language model generates likely text whether or not anything like it was ever stored, and has no built-in way of returning nothing.
Seen in: The one-step description covers how output is produced, and is not a full account of what goes on inside. Anthropic's interpretability team found Claude choosing a rhyme before writing the line that ends on it: "even though models are trained to output one word at a time, they may think on much longer horizons to do so."
Source: Anthropic, Tracing the thoughts of a large language model, 27 March 2025, https://www.anthropic.com/research/tracing-thoughts-language-model
Say it: "Don't tell me the migration worked. Run bun run db:migrate and then bun test, paste the output, and point at the line that shows it."
Described as: "what is an AI model actually doing", "how does ChatGPT work", "is it just fancy autocomplete", "why does it sound so convincing".
https://dev.dqnotes.com/model/llm

### Token
The unit a model reads and writes: a chunk of text such as a short word, part of a long word, a space plus a word, or a single symbol. Limits are counted in tokens, and so is what you pay.
A 3,000-line log that an agent reads costs real money, and it takes up room that the code could have had. Thinking in tokens is how you see that before it happens.
Not Word: On current Claude models 1M tokens is about 555k words, or roughly 1.8 tokens per word. That is an average for ordinary text. Code, whitespace and other languages land somewhere else.
Seen in: Anthropic says 1M tokens is "roughly 555k words or 2.5M Unicode characters" on the tokeniser introduced with Claude Opus 4.7, where earlier models fit about 750k words. So moving to a newer model can raise your token count for identical text, whatever the price per token does.
Source: Anthropic docs, Models overview, read 5 November 2026, https://platform.claude.com/docs/en/models/overview
Say it: "Before you read that log into context, count it with wc -c. If it is over 100 KB, grep for the failing test name instead and read only the lines around it."
Described as: "why isn't the limit in words", "what counts as one token", "how much text fits", "what am I paying for per million".
https://dev.dqnotes.com/model/token

### Tokeniser
Also: Tokenizer, Byte pair encoding, BPE.
The program that cuts text into tokens before the model sees it and joins tokens back into text afterwards. Its vocabulary is fixed in advance, so the same text can count differently on different models.
It explains costs that look strange: deep indentation, long names in code, encoded data such as base64, and languages that were rare in its training text all use tokens differently from the English you estimated with.
Not Compiler: A compiler works out what code means, and refuses what it can't understand. A tokeniser knows nothing about meaning. It will cut a variable name in half and never refuses anything.
Seen in: Sennrich, Haddow and Birch brought byte pair encoding to language work in 2015. OpenAI's tiktoken calls itself "a fast BPE tokeniser for use with OpenAI's models" and notes that BPE "works on arbitrary text, even text that is not in the tokeniser's training data". For Claude, Anthropic's count_tokens endpoint gives the count, which its docs describe as an estimate.
Source: Sennrich, Haddow and Birch, Neural Machine Translation of Rare Words with Subword Units, 2015, https://arxiv.org/abs/1508.07909
Source: OpenAI, tiktoken, https://github.com/openai/tiktoken
Source: Anthropic docs, Token counting, read 5 November 2026, https://platform.claude.com/docs/en/build-with-claude/token-counting
Say it: "Don't estimate by dividing characters by 4. Call the count_tokens endpoint for this model with the real system prompt, tools and messages, and tell me input_tokens."
Described as: "why does this short text use so many tokens", "why does Japanese cost more than English", "how does it split text up", "does whitespace cost anything", "minified code token count".
https://dev.dqnotes.com/model/tokeniser

### Training
Adjusting a model's numbers over a huge amount of text until it gets better at predicting what comes next. When training ends the numbers are frozen and shipped. Using the model doesn't change them.
Correcting an agent teaches the model nothing. Your correction is text in one conversation, and when that ends it is gone. For a lesson to reach the next session, it has to be written where the next session will read it.
Not Agent memory: Training changes the model's numbers, and happens before you get it. Agent memory is text put back into the prompt, so it is really just more reading.
Say it: "You won't remember this correction next session. Add it to AGENTS.md as a rule with one example from this repo, and show me the diff."
Described as: "does it learn from what I tell it", "I told it this yesterday", "why does it keep making the same mistake", "how did it learn to code".
https://dev.dqnotes.com/model/training

### Knowledge cutoff
Also: Training cutoff.
The date after which a model's training text has little or nothing. Nothing tells it that time has passed, so it describes old versions and old APIs in the present tense unless the prompt says otherwise.
Agents reach for the version of a library that was common in their training text. The fix is cheap: make it read the installed version's types or docs instead of recalling them.
Not Training data cutoff: Anthropic publishes both. The reliable knowledge cutoff is when knowledge is "most extensive and reliable", and the training data cutoff is the broader range used. They can differ, and the reliable date is the one to plan around.
Seen in: Anthropic lists both dates as June 2026 for Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5, as read on 5 November 2026. Anything from after that date needs to reach the model through the prompt.
Source: Anthropic docs, Models overview, read 5 November 2026, https://platform.claude.com/docs/en/models/overview
Say it: "You may be remembering an older version of this library. Read node_modules/zod/package.json and its type definitions before you write the schema, and name the file you checked."
Described as: "it used an old version of the library", "it says that function exists but it was removed", "it doesn't know about the new release", "it thinks it is still last year".
https://dev.dqnotes.com/model/knowledge-cutoff

### Inference
Running a trained model to get output. Each request is worked on separately, and the model itself carries nothing over from one to the next, so it starts every request knowing only what that request contains.
The whole conversation is sent again on every turn, each round of asking and answering, because it is the only way the model can see it. So long sessions cost more per turn, and a fresh session has no idea what you did this morning.
Not Training: Training produces the numbers and is slow and rare. Inference uses them, and happens on every request. If you are using a model, you are doing inference.
Say it: "This is a new session, so you know nothing about yesterday. Read docs/handoff.md and run git log -10, then tell me where the work stopped before you change anything."
Described as: "why doesn't it remember what we did earlier", "it forgot everything in the new chat", "what happens when I send a message", "does each message start from scratch".
https://dev.dqnotes.com/model/inference

### Context window
The most tokens one request can involve: instructions, the conversation so far, files read, and the reply. What is outside it can't affect the output, any more than a program can use a file it never opened.
Agents fill windows quickly by reading files and logs. What happens next is decided by the program running the model, and the first thing to go may be the rule you set at the start.
Not Memory: The window is what this one request contains. It isn't storage: nothing in it survives unless a program sends it again.
Seen in: Current Claude models have a 1M-token window and a 128K-token limit on output, as read on 5 November 2026. If the input alone is over the limit, the API returns a 400 invalid_request_error, "prompt is too long". Anthropic notes that chat interfaces can instead manage the window on a rolling "first in, first out" basis.
Source: Anthropic docs, Models overview, read 5 November 2026, https://platform.claude.com/docs/en/models/overview
Source: Anthropic docs, Context windows, read 5 November 2026, https://platform.claude.com/docs/en/build-with-claude/context-windows
Say it: "Don't read the whole log. Run grep -n "FAIL" test.log | head -50 and read only around those lines, so the window stays free for the code."
Described as: "it forgot what I said at the start", "conversation too long error", "it stopped following my instructions after a while", "how much can it read at once", "prompt is too long".
https://dev.dqnotes.com/model/context-window

### System prompt
The part of a request set by whoever built the tool rather than typed by the user: the role, the rules, how to use tools. It is still text in the window, read as tokens like everything else.
It makes an instruction more likely to be followed, and that is all. The model can still do the other thing, so a rule that must always hold belongs in code that checks it.
Not A setting: A setting is enforced by the program that reads it. A system prompt is text a model weighs when producing output. Claude Code's docs make this distinction for CLAUDE.md, its instructions file: it is context, and a hook, a script the tool runs whatever the model wants, is what enforces.
Seen in: OWASP lists System Prompt Leakage as LLM07 in its 2025 Top 10 for LLM applications, so treat a system prompt as readable. API keys don't go in it.
Source: OWASP Top 10 for LLM Applications 2025, https://genai.owasp.org/llm-top-10/
Source: Claude Code docs, Memory, https://code.claude.com/docs/en/memory
Say it: "Put "database access only through src/db/" in AGENTS.md, and also add a test that fails if any other file imports the database client, because the line alone can be ignored."
Described as: "the hidden instructions it starts with", "how do I give it permanent rules", "can I make it always do something", "it ignored my instructions".
https://dev.dqnotes.com/model/system-prompt

### Prompt
Everything sent to the model in one request: system prompt, tool definitions, earlier messages, file contents and your latest words. In everyday speech it means only what you just typed, which is usually a small part.
Most of what shapes an agent's output isn't what you typed this turn. When output goes wrong, look at the whole of what was sent before you reword your sentence.
Not Context: People use these loosely. Prompt usually means the text of a request, and context means everything available to the model while it works, including what a tool brings in later. Chapter 7 is all about context.
Say it: "Before you start, tell me which files you have read for this task and which you haven't. Then read src/cart.ts and src/cart.test.ts, and nothing else yet."
Described as: "what does the AI actually see", "how do I write a better prompt", "what gets sent when I press enter", "why did my short message cost so much".
https://dev.dqnotes.com/model/prompt

### Sampling
Also: Temperature, Top-p, Top-k.
Choosing the next token from the model's odds, by weighted dice. Temperature stretches or flattens the odds: low favours the likeliest token and high gives rare ones a chance. Top-p and top-k leave unlikely tokens out of the draw.
It is one reason a retry sometimes works, and why one success proves little. Advice to "set temperature to 0 for code" is also out of date on Claude's current API version, where the setting is gone.
Not Effort: Temperature changes how the dice are weighted at each token. Effort changes how much reasoning the model does before it answers. They are unrelated dials.
Seen in: Anthropic's API version 2026-10-21 removes the settings: "A request that sends any of them returns a 400 error on every model." We haven't checked how other vendors' current models treat them.
Source: Anthropic docs, API versions, read 5 November 2026, https://platform.claude.com/docs/en/api/versioning
Say it: "Remove temperature, top_p and top_k from this request: API version 2026-10-21 returns a 400 for them. If we need the same structure every time, use output_config.format with a JSON schema and validate the result."
Described as: "how do I make it less random", "what does temperature do", "set temperature to zero", "creative versus precise setting".
https://dev.dqnotes.com/model/sampling

### Non-determinism
The same input can give different output. Dice are the obvious cause. Even with the dice removed, a hosted model can vary, because of details such as how requests get batched together on the server.
An agent's success at a task is a rate, not a yes or a no, so one good run shows little and so does one bad run. Whatever must hold every time has to be checked by something that gives the same answer every time.
Not Flaky test: A flaky test should give the same answer every time and doesn't, which makes it a bug to fix. Model variation is how the thing works, so you measure it. If a test calls a real model, expect it to vary.
Seen in: Horace He at Thinking Machines sampled 1,000 completions at temperature 0 from one open model and got 80 different ones, and traced the cause to batch size. With batch-invariant kernels, all 1,000 matched, at some cost in speed. In our own recording, Claude Haiku 5.5 reviewing the same correct code with the same prompt approved it 33 times out of 40.
Source: He, Defeating Nondeterminism in LLM Inference, Thinking Machines, 2025, https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
Say it: "Run this eval 20 times and report the pass count, not one result. If it is under 18 of 20, list the cases that failed, and don't call it fixed."
Described as: "it worked yesterday and nothing changed", "same prompt, different answer", "why does it give a different result every time", "it passes sometimes and fails other times".
https://dev.dqnotes.com/model/non-determinism

### Hallucination
Output that reads as fact and isn't: a package, function, flag or citation that doesn't exist. It is produced by the same step as correct output, so nothing in the text marks it, and confident wording tells you nothing.
Agents act on it. They import packages and call methods that were never there. Type checks and installs catch a lot of this, but reading alone won't. Invented package names also give attackers something to register.
Not Lying: Lying means knowing the truth and hiding it. The better description is that a model produces likely text, and a likely false sentence is still likely. How much it internally separates known from unknown is an active research question.
Seen in: Spracklen and colleagues (USENIX Security 2025) found at least 5.2% of packages hallucinated by commercial models and 21.7% by open-source ones, across 576,000 code samples. A 2026 preprint by Churilov, not peer reviewed, found 4.62% to 6.10% on five newer models. Kalai and colleagues argue that training and scoring reward guessing. Anthropic found a circuit in Claude that is "on" by default and declines to answer. A familiar-looking name can suppress it, and if little is really known, "it proceeds to confabulate".
Source: Spracklen et al., We Have a Package for You, USENIX Security 2025, https://arxiv.org/abs/2406.10279
Source: Churilov, 2026 replication preprint, https://arxiv.org/abs/2605.17062
Source: Kalai et al., Why Language Models Hallucinate, 2025, https://arxiv.org/abs/2509.04664
Source: Anthropic, Tracing the thoughts of a large language model, 27 March 2025, https://www.anthropic.com/research/tracing-thoughts-language-model
Say it: "Before you add any dependency, run npm view <name> version repository.url and show me the output. If it doesn't exist or the repository looks wrong, stop and tell me."
Described as: "it made up a package that doesn't exist", "that function isn't real", "it cited something that doesn't exist", "it sounds so sure but it is wrong", "why does AI make things up".
https://dev.dqnotes.com/model/hallucination

### Sycophancy
A tendency to produce what the person seems to want rather than what is true: backing down when challenged, praising weak work, confirming your guess about a bug.
Your question steers the answer. "That's the bug, right?" invites a yes, and a correct fix can get abandoned because you pushed back. Ask in a way that leaves room for you to be wrong.
Not Being polite: Polite wording is style. Sycophancy changes the content: the claim itself moves towards what you appear to believe.
Seen in: Sharma and colleagues found it in five assistants and found that human preference data favoured answers matching the person's views. OpenAI rolled back a GPT-4o update in April 2025, saying it had "focused too much on short-term feedback" and the model "skewed towards responses that were overly supportive but disingenuous". A related effect in our recording: told to find a defect, Claude Haiku 5.5 approved correct code 1 time in 40, against 33 in 40 with a neutral prompt.
Source: Sharma et al., Towards Understanding Sycophancy in Language Models, 2023, https://arxiv.org/abs/2310.13548
Source: OpenAI, Sycophancy in GPT-4o, 29 April 2025, https://openai.com/index/sycophancy-in-gpt-4o/
Say it: "I may be wrong about the cause. Give me the two likeliest causes of this failure, the file and line behind each, and what would rule each one out. Don't assume my guess is one of them."
Described as: "it always says I am right", "you're absolutely right", "it changes its answer when I push back", "it told me my bad idea was great".
https://dev.dqnotes.com/model/sycophancy

### Confabulated explanation
An account a model gives of its own behaviour, written as new text rather than read from a record. It may be right or wrong, and it reads the same either way. Asking "why did you do that" gets you a plausible story.
When an agent breaks something, its explanation isn't a log. Read what it actually did: the actions it asked for (tool calls, coming next), the results and the diff. Treat "why" as a source of hypotheses, then test them.
Not Log: A log is written at the moment something happens, by the thing doing it. An explanation is written afterwards, from whatever is in the window. In an agent, the tool calls and results are the record.
Seen in: Ask Claude how it added 36 and 59 and, Anthropic reports, "it describes the standard algorithm involving carrying the 1", while its internal paths work differently. Turpin and colleagues biased models towards wrong answers and got step-by-step reasoning that justified those answers without mentioning the bias. After the 2025 Replit database deletion, the agent's claim that a rollback would not work turned out wrong, according to Fortune's account: its owner recovered the data by hand.
Source: Anthropic, Tracing the thoughts of a large language model, 27 March 2025, https://www.anthropic.com/research/tracing-thoughts-language-model
Source: Turpin et al., Language Models Don't Always Say What They Think, 2023, https://arxiv.org/abs/2305.04388
Source: Fortune, 23 July 2025, https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure
Say it: "Don't explain from memory. List the exact commands you ran this session that touched the database, in order, copied from the tool calls. Then we'll check them against the database logs."
Described as: "I asked it why and the answer made no sense", "it gave a different reason each time", "it admitted the mistake, but was that real", "can I trust its explanation of what it did".
https://dev.dqnotes.com/model/confabulation

### Tool call
Also: Function calling.
Structured output asking for a named action with arguments, such as read_file on config.json. The model runs nothing. The program around it, which the next chapter calls the harness, decides whether to run it and sends back the result as text.
All real power sits in that program, so that is where the safety controls go, as the next chapter shows. It also means the model works from whatever comes back, including instructions hidden in a file it read.
Not Running code: The model produces a request, and a program does the running. With "server tools" such as Anthropic's web search, the provider's machines run it instead, but it is still not the model.
Seen in: In Claude's API, a response with stop_reason "tool_use" contains a tool_use block with an id, name and input, and your next request answers with a tool_result carrying that tool_use_id. Anthropic: Claude "returns a structured call that your application runs (client tools) or that Anthropic runs (server tools)". Dex Horthy's 12-Factor Agents puts it as "Tools are just structured outputs". Tool definitions are billed as input tokens.
Source: Anthropic docs, Tool use, read 5 November 2026, https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview
Source: Anthropic docs, Handle tool calls, read 5 November 2026, https://platform.claude.com/docs/en/agents-and-tools/tool-use/handle-tool-calls
Source: Horthy, 12-Factor Agents, https://github.com/humanlayer/12-factor-agents
Say it: "Expose read_file and run_tests as tools, not a general shell. Set strict: true on each one, return failures as a tool_result with is_error: true, and log every call with its arguments."
Described as: "how does the AI run commands", "can it actually touch my files", "how does it use tools", "what stops it deleting things", "function calling".
https://dev.dqnotes.com/model/tool-call

### Thinking and effort
Also: Reasoning, Extended thinking, Effort level.
Tokens a model produces before its answer, working a problem through, which tends to help on hard tasks. Effort is the setting that controls how much of this happens. More effort costs more and takes longer.
Effort is a dial to try before changing model. Thinking is billed at the output price, whether you can see it or not, and it isn't a trustworthy record of why the model did something.
Not A record of its reasoning: Thinking text is more output. When Anthropic planted hints in questions, Claude 3.7 Sonnet mentioned the hint it had used 25% of the time, and DeepSeek R1 39%. Anthropic: "we can't always rely on what they tell us about their reasoning".
Seen in: On Claude, output_config.effort takes low, medium, high, xhigh or max. It defaults to medium on Opus 5.5 and Haiku 5.5 and high on most other models. Thinking tokens are billed as output, and usage.output_tokens_details.thinking_tokens reports them, even when the thinking text isn't shown to you. Read 5 November 2026.
Source: Anthropic docs, Steering thinking, read 5 November 2026, https://platform.claude.com/docs/en/build-with-claude/thinking-steering-and-cost
Source: Anthropic, Reasoning models don't always say what they think, 3 April 2025, https://www.anthropic.com/research/reasoning-models-dont-say-think
Say it: "Run the codebase search at effort low and the migration design at effort high. Show me usage.output_tokens_details.thinking_tokens for both before we change anything else."
Described as: "the thinking dots", "how do I make it try harder", "why is my output token count higher than the text", "can I read its reasoning", "it took ages to answer".
https://dev.dqnotes.com/model/reasoning

### Prompt caching
The provider keeps the processed start of your request for a few minutes. If your next request begins with exactly the same bytes, that part costs a fraction of the price. One changed byte invalidates everything after it.
For long agent sessions this usually matters more than any wording trick. A timestamp at the top of a system prompt breaks every hit, and because saving to the cache costs extra, you then pay more than with no caching at all.
Not HTTP cache: An HTTP cache gives back a stored response. A prompt cache only skips reprocessing the input. The model still generates new output, and you still pay full price for that.
Seen in: On Claude, the order is tools, then system, then messages, up to a cache_control breakpoint. The default lifetime is 5 minutes, refreshed free on each hit. Writes cost 1.25x the input price. Reads cost 0.05x on Opus 5.5 and Sonnet 5.5, 0.025x on Fable 5.1, and 0.1x on Haiku 5.5. The minimum is 512 tokens on those models, and anything shorter silently isn't cached. Read 5 November 2026.
Source: Anthropic docs, Prompt caching, read 5 November 2026, https://platform.claude.com/docs/en/build-with-claude/prompt-caching
Source: Anthropic docs, Pricing, read 5 November 2026, https://platform.claude.com/docs/en/about-claude/pricing
Say it: "Move the date out of the system prompt and into the last user message. Then send two identical requests and show me usage.cache_read_input_tokens from the second: it must be above zero."
Described as: "why is my bill so high for long chats", "it costs less if I keep the start the same", "cache_read_input_tokens is always zero", "repeated context is expensive".
https://dev.dqnotes.com/model/prompt-caching

### Model tiers
Providers sell a ladder of models at very different prices. Higher rungs cost more per token, tend to be slower and tend to do better on hard work. The names change often, so check the current list rather than remembering one.
Using the top rung to rename a variable is waste, and the cheapest rung on a hard task can cost more in retries than getting it right once. Judge a tier by its cost per finished task, on your tasks.
Not Effort: A tier is a different model. Effort is how hard a given model works. Trying a lower effort on the same model is usually the simpler experiment.
Seen in: Claude, per 1M input and output tokens as read on 5 November 2026: Fable 5.1 at $10 and $50, Opus 5.5 at $4 and $20, Sonnet 5.5 at $2 and $10, Haiku 5.5 from $0.10 and $0.50. That is a factor of 100 from top to bottom. OpenAI's ladder on the same day: gpt-6-astra $10 and $50, gpt-6.1-sol $2 and $10, gpt-6.1-terra $1 and $5, gpt-6-luna $0.10 and $0.50.
Source: Anthropic docs, Pricing, read 5 November 2026, https://platform.claude.com/docs/en/about-claude/pricing
Source: OpenAI API pricing, read 5 November 2026, https://developers.openai.com/api/docs/pricing
Say it: "Use claude-haiku-5-5 for the bulk file summaries and claude-opus-5-5 for the plan. Log cost and pass or fail per task for this week, and show me the table before we change the split."
Described as: "which model should I use", "is the expensive one worth it", "Opus or Sonnet", "cheap model for simple tasks".
https://dev.dqnotes.com/model/model-tier

### Token cost
What a request costs: the tokens of each kind multiplied by that kind's price per million. Input, cache writes, cache reads and output are all priced separately, and output is the most expensive, five times input on current Claude models.
An agent's bill grows with the length of the session, not only the size of the task, since every turn carries everything before it. Starting a fresh session is a cost decision as well as a quality one.
Not Cost per task: Cost per token is a price list. Cost per task is what you actually spend to finish something, retries included. A cheap model that needs three attempts can lose to one that needs one.
Seen in: Anthropic's own example: on Opus 5.5, 50,000 input and 15,000 output tokens cost $0.20 and $0.30. With 40,000 of that input as cache reads, the input falls to $0.048. Batch processing is half price, if you can wait. At the other extreme, StrongDM's Justin McCarthy wrote: "If you haven't spent at least $1,000 on tokens today per human engineer, your software factory has room for improvement." That is one company's position, and not a norm.
Source: Anthropic docs, Pricing, read 5 November 2026, https://platform.claude.com/docs/en/about-claude/pricing
Source: McCarthy, Software Factories And The Agentic Moment, 2026, https://factory.strongdm.ai/
Say it: "Run this with claude -p --output-format json and read total_cost_usd. Do that for three real tasks, then tell me the cost per task and which step used the most tokens."
Described as: "how much will this cost", "why was my bill so high", "what does an agent session cost", "input versus output price".
https://dev.dqnotes.com/model/token-cost

### Embedding
Also: Vector embedding.
A list of numbers standing for a piece of text, produced so that texts with related meanings get similar numbers. Compare two lists and you get a measure of how related the texts are, even if they share no words.
It is what makes search by meaning possible, and you will meet it in tools that search documents on a model's behalf. It isn't automatically better than plain search: for an exact function name, a text search is faster, cheaper and right.
Not Text search: Text search finds exact characters and misses synonyms. Embeddings find related meaning and can miss the exact identifier you wanted. Search for handleLogin by name. Use embeddings for "where do we deal with sign-in".
Seen in: Anthropic's docs say: "Anthropic does not offer its own embedding model." They point to Voyage AI, whose models include voyage-code-4, optimised for code retrieval. Read 5 November 2026.
Source: Anthropic docs, Embeddings, read 5 November 2026, https://platform.claude.com/docs/en/build-with-claude/embeddings
Say it: "Search for handleLogin with rg first. If that finds nothing, tell me before you build an embedding index, and if we do build one, measure it on 20 real questions against plain rg."
Described as: "search by meaning not keywords", "how does semantic search work", "find related documents", "what is a vector database for".
https://dev.dqnotes.com/model/embedding

### Model benchmarks
Also: SWE-bench, Leaderboard.
A fixed set of tasks that models are scored on so they can be compared. SWE-bench is the best known for coding: given a real bug report from GitHub and the project's code, the model has to produce a change, and tests decide whether it worked.
A score tells you how a model did on those tasks under those tests. The tests can be wrong, the answers may have been in the training text, and your code base isn't in it. Treat scores as a shortlist, then test on your own work.
Not Your own eval: A public benchmark compares models on someone else's tasks. An eval runs them on yours, and is the only one of the two that tells you about your work.
Seen in: The original SWE-bench had 2,294 problems from 12 Python repositories. In February 2026 OpenAI stopped reporting the Verified subset. Its audit of a hard 27.6% of the set found "at least 59.4% of the audited problems have flawed test cases that reject functionally correct submissions", and it reported that models had been exposed to the answers in training. OpenAI sells models and recommends a replacement, so it isn't a neutral party, but the audit is checkable.
Source: Jimenez et al., SWE-bench, 2023, https://arxiv.org/abs/2310.06770
Source: OpenAI, Why SWE-bench Verified no longer measures frontier coding capabilities, 23 February 2026, https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/
Say it: "Don't pick the model from a leaderboard. Take 10 tasks we have already solved from git log, run each candidate on them in its own worktree, score with our test suite plus a test it can't see, and show me pass count and cost."
Described as: "which AI is best at coding", "the leaderboard says this one is best", "how do they measure model quality", "is the benchmark score real".
https://dev.dqnotes.com/model/benchmark-model

### Jagged frontier
Models do some tasks very well and others badly, and the boundary is not where you would guess: two tasks that seem equally hard can fall on opposite sides. The output looks equally polished on both.
A general feeling that "AI is good at this" doesn't transfer safely from one task to the next. Keep a record of where your agent has been wrong on your code, and check hardest there.
Not Model tier: A tier says how capable a model is overall. The frontier is about where that capability runs out, which is uneven within every tier.
Seen in: Dell'Acqua and colleagues ran a preregistered experiment with 758 knowledge workers at Boston Consulting Group, using GPT-4. On 18 tasks inside the frontier, people with AI completed 12.2% more tasks, 25.1% more quickly, at higher quality. On a task chosen to be outside it, they were "19% less likely to produce correct solutions", in the abstract's words, and the article itself says 19 percentage points, so quote it with care. These were consulting tasks with one older model, so the boundary has moved, and it says nothing direct about code. Published in Organization Science, 2026.
Source: Dell'Acqua et al., Navigating the Jagged Technological Frontier, Organization Science, 2026, https://doi.org/10.1287/orsc.2025.21838
Say it: "You have got concurrency wrong in this repo before. Write a failing test that reproduces the race before changing any code, and I will review this diff line by line."
Described as: "it nailed the hard thing and failed the easy one", "brilliant at one task and useless at the next", "how do I know what it's bad at", "when can I trust it".
https://dev.dqnotes.com/model/jagged-frontier

### Fine-tuning
Training a copy of a model further on your own examples, so that it leans towards your format or style. It changes tendencies. It isn't a way of giving a model your code base, which changes daily anyway.
People reach for it to teach an agent about their project. Instructions and search almost always do that better: they cost little, can be edited in a minute, and are in the repo where you can review them.
Not AGENTS.md: An instructions file is read fresh on each run, and a commit can change it. A fine-tuned model is a new frozen copy, which needs retraining to change anything.
Seen in: OpenAI's docs now say: "OpenAI is winding down the fine-tuning platform" and it "is no longer accessible to new users" (read 5 November 2026). Fine-tuning for Claude 3 Haiku became generally available in Amazon Bedrock in November 2024. We couldn't confirm whether any current Claude model offers it.
Source: OpenAI docs, Model optimization, read 5 November 2026, https://developers.openai.com/api/docs/guides/model-optimization
Source: AWS, Fine-tuning for Claude 3 Haiku, 1 November 2024, https://aws.amazon.com/about-aws/whats-new/2024/11/fine-tuning-anthropics-claude-3-haiku-amazon-bedrock
Say it: "Don't fine-tune for this. Put our naming conventions in AGENTS.md with three real examples from src/, then run the same task twice and show me whether the output now matches."
Described as: "can I train it on my code", "teach it our style permanently", "custom model for my company", "make it know my codebase".
https://dev.dqnotes.com/model/fine-tuning

### Stop reason
Also: stop_reason, Truncated output, max_tokens.
A field on each response saying why the model stopped: it finished, hit your token limit, wants a tool run, or declined. It arrives on a success, so code that only reads the text can treat half an answer as a whole one.
An agent that saves a file from a cut-off response saves half a file, with status 200 and no error. Whatever drives a model has to check this field before it uses the content.
Not Error: An error means the request failed, with a status such as 400 or 429. A stop reason comes with a 200. A cut-off answer and a refusal are both "successful" responses.
Seen in: Claude's values: end_turn, max_tokens, stop_sequence, tool_use, pause_turn, refusal and model_context_window_exceeded, as read on 5 November 2026. Thinking counts towards max_tokens, so a deep think can use up the limit before the answer gets written.
Source: Anthropic docs, Handling stop reasons, read 5 November 2026, https://platform.claude.com/docs/en/build-with-claude/handling-stop-reasons
Source: Anthropic docs, Steering thinking, read 5 November 2026, https://platform.claude.com/docs/en/build-with-claude/thinking-steering-and-cost
Say it: "Check response.stop_reason before using the content. On max_tokens, save nothing and retry with a higher limit. Add a test that feeds in a truncated response and asserts that no file gets written."
Described as: "the answer got cut off", "it stopped halfway through the file", "response ends mid sentence", "the JSON from the model won't parse", "it wrote half a file".
https://dev.dqnotes.com/model/stop-reason

## 6. The agent

A model plus a loop plus tools. Everything around the model is the harness, and that part is yours.

### Agent
Also: AI agent.
A model, a loop and some tools. The model asks for a tool, a program runs it and hands back the result, and this repeats until the model says it has finished.
The loop is what changes your job. A chat reply is one answer you can read before anything happens. An agent takes many actions before you look, so your attention moves from what it said to what it did.
Not Workflow: A workflow follows a path someone wrote in advance in code. An agent picks its own next step from what it just saw. Anthropic draws the line in exactly this place.
Seen in: Simon Willison's definition is the shortest: "Agents run tools in a loop to achieve a goal". Anthropic, in December 2024: "Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage."
Source: Willison, What is agentic engineering?, https://simonwillison.net/guides/agentic-engineering-patterns/what-is-agentic-engineering/
Source: Anthropic, Building effective agents, 2024, https://www.anthropic.com/engineering/building-effective-agents
Say it: "Work on this as an agent, not a chat. Run bun test yourself, read the failures, fix them, and keep going until it exits 0. Do not paste code for me to apply."
Described as: "AI that actually does things instead of just answering", "it ran commands by itself", "a chatbot versus something that edits my files", "why does it keep going without me".
https://dev.dqnotes.com/agent/agent

### Agent loop
Also: Agentic loop, Tool loop.
The cycle inside every agent: send the context to the model, read its reply, run any tools it asked for, add the results to the context, and send it all again. It ends when the model stops asking for tools.
The whole context goes back on every turn, so it only grows, and a wrong assumption from turn two is still being read on turn twenty. Slow late turns, rising cost, and an agent stuck on an early mistake all come from this.
Seen in: In Anthropic's API a reply with stop_reason tool_use contains tool_use blocks. Your code runs them, sends tool_result blocks back with the whole conversation, and loops until stop_reason is end_turn. Claude Code's docs describe the same loop as gathering context, taking action and verifying results.
Source: Anthropic docs, Tool use overview, https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview
Source: Claude Code docs, How Claude Code works, https://code.claude.com/docs/en/how-claude-code-works
Say it: "Before you edit anything, tell me the next three tool calls you expect to make and what result would change your mind. If you have made 10 tool calls without a passing test, stop and tell me where you are."
Described as: "it goes round and round until it is finished", "what is it actually doing between my messages", "why does each step take longer and cost more", "the cycle of ask, run, read the result".
https://dev.dqnotes.com/agent/agent-loop

### Harness
Also: Agent harness, Agentic harness.
Everything in an agent except the model: the loop, the tools, the instructions, the permissions, the hooks, the sandbox. It decides what the model sees and what happens to what it asks for.
You can't change how the model thinks, but you can change nearly everything it sees and can do. The same model in a different harness does different work, and the harness is the only part of that you own.
Not Harness engineering: The harness is the thing itself. Harness engineering is the practice of improving it each time the agent goes wrong.
Seen in: "Agent = Model + Harness" and "If you're not the model, you're the harness." are from Vivek Trivedy at LangChain, 10 March 2026, the earliest dated use we found, though we can't rule out an earlier one. It is not in Mitchell Hashimoto's post or in OpenAI's, and people often credit it to them. Claude Code's own docs use the term "agentic harness" for the layer around the model.
Source: Trivedy, The Anatomy of an Agent Harness, LangChain, 2026, https://www.langchain.com/blog/the-anatomy-of-an-agent-harness
Source: Böckeler, Harness engineering for coding agent users, 2026, https://martinfowler.com/articles/harness-engineering.html
Source: Claude Code docs, How Claude Code works, https://code.claude.com/docs/en/how-claude-code-works
Say it: "Don't just fix this. Tell me which part of our setup let it through: an instruction that was missing, a check that doesn't exist, or a permission that is too wide. Then propose one change to that part."
Described as: "everything around the AI that is not the AI", "the setup that makes the agent work", "why does the same model behave differently in different tools", "the part that runs the model".
https://dev.dqnotes.com/agent/harness

### Tool
A function the harness offers the model, described by a name, a purpose and the inputs it takes. The model can only write a request to use it. The harness checks the request, runs the function and returns the result.
Tools set the limit of what an agent can do, and their output is how it learns what happened. An error that names the fix makes it better without touching the model. A 5,000-line dump makes it worse and costs you for the privilege.
Not Tool call: The tool is the function on offer. A tool call is one request to use it, written by the model, and chapter 5 showed what that looks like. This chapter is about the side that answers.
Seen in: Anthropic's guide to writing tools warns "More tools don't always lead to better outcomes" and asks for error messages that tell the agent how to fix the problem. Claude Code's built-ins include Read, Edit, Write and Bash.
Source: Anthropic, Writing effective tools for agents, 2025, https://www.anthropic.com/engineering/writing-tools-for-agents
Source: Claude Code docs, Tools reference, https://code.claude.com/docs/en/tools-reference
Say it: "Write a search_orders tool with inputs customer_id (string) and status (open or shipped). Return at most 20 rows. If customer_id is missing, the error must say what to pass, not just "invalid input"."
Described as: "how does the AI actually run commands", "the list of things it is allowed to call", "giving the agent a new ability", "it can search the web but not my database".
https://dev.dqnotes.com/agent/tool

### Coding agent
An agent whose tools can read and edit files and run commands, so it can write code, run it, see what broke and try again. Claude Code, Codex and Cursor's agent are coding agents.
Running what it wrote is the difference. Chat hands you code you have to test yourself. A coding agent can run it, which means a check you set up can reach the agent while you are somewhere else. Everything in chapter 9 depends on this.
Not Autocomplete: Autocomplete suggests the next few lines and waits for you. A coding agent takes a goal and works through many steps on its own.
Seen in: Willison: "They're agents that can both write and execute code." As of November 2026 Claude Code runs in the terminal, IDE extensions, a desktop app and the web. Codex has a CLI. Cursor is an editor that also has a CLI, started with agent -p.
Source: Willison, What is agentic engineering?, https://simonwillison.net/guides/agentic-engineering-patterns/what-is-agentic-engineering/
Source: Claude Code docs, How Claude Code works, https://code.claude.com/docs/en/how-claude-code-works
Source: Cursor docs, Headless CLI, https://cursor.com/docs/cli/headless
Say it: "Start the server with bun run dev, curl localhost:3000/api/health, and paste the response. Don't tell me it works until you have run it."
Described as: "an AI that writes and runs code by itself", "Claude Code, Codex, that kind of thing", "something better than copy pasting from a chatbot", "it tests its own code".
https://dev.dqnotes.com/agent/coding-agent

### Vibe coding
Building with a model and not reading the code it writes. You accept every change, paste errors back in, and judge only by whether it seems to work. It describes how you work, not which tool you use, and it is fine for throwaway projects.
Agents make it easy to go far past the point where you understand the code, and it feels the same all the way. So ask what happens if this is wrong. For a weekend toy, nothing: vibe code it and enjoy the afternoon. For anything holding other people's data, that answer changes.
Not Agentic engineering: Both use agents. In vibe coding you don't check the code, and you have accepted that. In agentic engineering you stay answerable for it. Using an AI to write code does not by itself make it vibe coding.
Seen in: Andrej Karpathy named it on 2 February 2025: "There's a new kind of coding I call 'vibe coding', where you fully give in to the vibes, embrace exponentials, and forget that the code even exists." Also: "I 'Accept All' always, I don't read the diffs anymore." He set the scope himself: "It's not too bad for throwaway weekend projects". Simon Willison's definition: "building software with an LLM without reviewing the code it writes", and on using it: "For low stakes projects and prototypes why not just let it rip?"
Source: Karpathy on X, 2 Feb 2025 (read through the fxtwitter mirror of this status), https://x.com/karpathy/status/1886192184808149383
Source: Willison, Not all AI-assisted programming is vibe coding, 2025, https://simonwillison.net/2025/Mar/19/vibe-coding/
Say it: "This is a throwaway demo for Saturday and it touches no real data, so don't stop to explain and I won't read the diffs. If we decide to keep it, we stop there and write tests first."
Described as: "I just keep prompting and never read the code", "it mostly works and I do not know why", "accept all and hope", "built a whole app without looking at the code", "is it ok to not review a quick prototype".
https://dev.dqnotes.com/agent/vibe-coding

### Agentic engineering
Also: Vibe engineering, Augmented coding.
Building software with coding agents while staying answerable for the result. You set the goal, decide what the checks are, and can explain what shipped. The agent types, and you are still the engineer.
This is what an employer pays for and what a sceptic will test. Typing stopped being scarce. Knowing what to build, and being able to prove it works, did not. Agents make the gap between the two people much wider, because the output looks alike until something breaks.
Not Vibe engineering: The same idea under another name. Willison proposed it in October 2025 for professionals who stay "proudly and confidently accountable", and later noted that agentic engineering was gaining more traction. Kent Beck's "augmented coding" draws a similar line.
Seen in: Karpathy, a year after naming vibe coding, 4 February 2026: "Many people have tried to come up with a better name for this", and his favourite was agentic engineering: "you are not writing the code directly 99% of the time, you are orchestrating agents who do and acting as oversight", and "there is an art & science and expertise to it." Canva's interviews expect engineers to "take full ownership of any code they produce, whether they wrote it themselves or with AI assistance."
Source: Karpathy on X, 4 Feb 2026 (read through the fxtwitter mirror of this status), https://x.com/karpathy/status/2019137879310836075
Source: Willison, Vibe engineering, 2025, https://simonwillison.net/2025/Oct/7/vibe-engineering/
Source: Beck, Augmented Coding: Beyond the Vibes, 2025, https://newsletter.kentbeck.com/p/augmented-coding-beyond-the-vibes
Source: Newton, Yes, you can use AI in our interviews, Canva, 2025, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Say it: "Before you start, write the acceptance checks for this change in docs/plan.md. When you finish, show me the diff, the test output and anything you are unsure about. I will read it, because I have to be able to explain it."
Described as: "using AI seriously at work", "the grown-up version of vibe coding", "I direct the agent but I own the code", "what do I call it when I actually check the work".
https://dev.dqnotes.com/agent/agentic-engineering

### AGENTS.md and CLAUDE.md
Also: Instructions file, Memory file.
A plain text file in your repo that the agent loads when it starts: build commands, a map of the code, things not to do. AGENTS.md is the name shared across tools. CLAUDE.md is Claude Code's own.
It goes into the context of every request, and it is only advice. Claude Code's docs say these files are "context, not enforced configuration". So short and specific beats long and thorough, and anything that must always hold belongs in a hook or a deny rule, both coming shortly.
Not README: A README is for a person deciding how to use a project. AGENTS.md is for an agent working inside it, so it holds commands and prohibitions, not an introduction.
Seen in: November 2026: Codex reads AGENTS.md from the project root down to your directory and stops adding files at 32 KiB by default. Cursor reads it too. Claude Code reads it only when there is no CLAUDE.md (since v2.1.277), so to share one file you put @AGENTS.md in CLAUDE.md. Size advice: OpenAI used "roughly 100 lines", Claude Code's docs say under 200. Hashimoto on his: "Each line in that file is based on a bad agent behavior."
Source: AGENTS.md, the format, https://agents.md
Source: Claude Code docs, Memory, https://code.claude.com/docs/en/memory
Source: OpenAI Codex docs, AGENTS.md, https://learn.chatgpt.com/docs/agent-configuration/agents-md
Source: Lopopolo, Harness engineering, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: Hashimoto, My AI Adoption Journey, 2026, https://mitchellh.com/writing/my-ai-adoption-journey
Say it: "Add to AGENTS.md: "Run bun run typecheck and bun test before saying you are done. All database access goes through src/db/; copy src/db/users.ts." Keep the file under 100 lines, and delete anything you could work out by reading the code."
Described as: "a file that tells the AI about my project", "I keep telling it the same things every session", "where do I put the instructions", "it keeps using npm when we use bun", "what should go in CLAUDE.md".
https://dev.dqnotes.com/agent/agents-md

### Rules file
Also: Cursor rules, Path-scoped rules.
Instructions that load only when they apply, usually when the agent touches certain files. A rule for database code stays out of the context until the agent opens database code.
One big instructions file is paid for on every request, even when a tenth of it matters. Scoped rules keep that file short. The risk is the opposite one: if the agent never opens a matching file, it never sees the rule.
Not AGENTS.md: AGENTS.md loads whatever you are doing. A rules file loads when its condition is met, so it can afford to be longer and narrower.
Seen in: Cursor (November 2026): .mdc files in .cursor/rules with the fields description, globs and alwaysApply, and a plain .md file in that folder is ignored. It advises "Keep rules under 500 lines." Claude Code: .claude/rules/*.md with a paths field.
Source: Cursor docs, Rules, https://cursor.com/docs/rules
Source: Claude Code docs, Memory, https://code.claude.com/docs/en/memory
Say it: "Move the SQL conventions out of AGENTS.md into .cursor/rules/sql.mdc with globs: src/db/**. Then open src/db/users.ts and confirm the rule attached."
Described as: "instructions only for some folders", "different rules for the frontend and backend", "the cursorrules thing", "my instructions file is too long".
https://dev.dqnotes.com/agent/rules-file

### Skill
Also: Agent Skill, SKILL.md.
A folder holding a SKILL.md file: a name, a one-line description and steps to follow, sometimes with scripts beside it. Only the name and description load at first. The agent reads the rest when the description matches what it is doing.
You can keep fifty procedures available for roughly the price of fifty one-liners. That makes the description the part that matters most: write it vaguely and the skill never fires, and you will think the agent ignored you.
Not Hook: A skill is instructions the agent chooses to follow. A hook is a script the harness runs whether the agent wants it or not.
Seen in: Agent Skills came from Anthropic in October 2025 and became an open standard that December. Anthropic says more than 70 platforms now use it, which we have not checked. As of November 2026 Claude Code reads .claude/skills, Codex reads .agents/skills, and Cursor reads .agents/skills, .cursor/skills and the Claude and Codex folders. Only some frontmatter is portable: name, description, license, compatibility, metadata and allowed-tools.
Source: Anthropic, Equipping agents for the real world with Agent Skills, 2025, https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills
Source: Anthropic, Donating Agent Skills to the AAIF, https://www.anthropic.com/news/donating-agent-skills-to-the-agentic-ai-foundation
Source: Claude Code docs, Skills, https://code.claude.com/docs/en/skills
Source: OpenAI Codex docs, Build skills, https://developers.openai.com/codex/skills
Source: Cursor docs, Skills, https://cursor.com/docs/skills
Say it: "Turn our release steps into a skill at .claude/skills/release/SKILL.md. Description: "Use when cutting a release or bumping the version." Keep it under 100 lines and put the changelog script next to it."
Described as: "a saved procedure the agent can follow", "teach it how we do releases once", "it only loads when it is needed", "a reusable prompt with scripts", "it never uses the thing I wrote for it".
https://dev.dqnotes.com/agent/skill

### Slash command
A word starting with / that you type to make the harness do something right now: clear the context, switch mode, or run a saved prompt. You type it, not the agent.
It turns a long instruction you repeat into one word, so the careful version becomes the easy version. If you type the same review prompt every day, it should be a command the whole team shares.
Not Skill: In Claude Code the two have merged: .claude/commands/deploy.md and .claude/skills/deploy/SKILL.md both create /deploy. What a skill adds is that the agent can also load it by itself when it looks relevant.
Seen in: Claude Code (November 2026): /clear starts a new conversation, /permissions edits your rules. Codex: /plan, /hooks and /memories. In Cursor, typing / in Agent chat searches your skills.
Source: Claude Code docs, Commands, https://code.claude.com/docs/en/commands
Source: Claude Code docs, Skills, https://code.claude.com/docs/en/skills
Source: OpenAI Codex docs, Developer commands, https://learn.chatgpt.com/docs/developer-commands
Source: Cursor docs, Skills, https://cursor.com/docs/skills
Say it: "Make a /review-diff command in .claude/commands/review-diff.md. It runs git diff main, and for each changed file says what could break, with the line number. Commit it so the team gets it."
Described as: "the commands that start with a slash", "a shortcut for a prompt I keep typing", "how do I clear the chat", "save this prompt so I can reuse it".
https://dev.dqnotes.com/agent/slash-command

### Hook
Also: Agent hook, PreToolUse hook.
A script the harness runs at a set moment, such as just before a tool runs or when the agent tries to stop. It can block the action. The model has no say in whether it runs.
A line in AGENTS.md saying "never run rm -rf" can be ignored. A hook that blocks it can't, as long as the hook recognises the command. A script looking for rm -rf will wave through rm -r -f, and in Claude Code, by default, the only exit code that blocks is 2.
Not Git hook: A git hook belongs to git and fires on events like a commit. An agent hook belongs to the harness and fires on events like a tool call. You often want both.
Seen in: All three tools have them (November 2026). Claude Code: PreToolUse in settings.json, where exit 2 blocks and by default any other non-zero exit lets the action proceed. Since v2.1.295 a command or http hook can set onFailure to block, so a crash, a timeout or an unexpected exit code blocks too. That setting has no effect on Stop hooks: exit 2 there sends the agent back to work, and a Stop hook that can't run doesn't hold it. Codex: hooks.json or config.toml, with the same event name and exit 2. Cursor: .cursor/hooks.json with preToolUse and beforeShellExecution, where exit 2 blocks and other failures fail open unless you set failClosed.
Source: Claude Code docs, Hooks, https://code.claude.com/docs/en/hooks
Source: OpenAI Codex docs, Hooks, https://learn.chatgpt.com/docs/hooks
Source: Cursor docs, Hooks, https://cursor.com/docs/hooks
Say it: "Add a PreToolUse hook on Bash in .claude/settings.json. It reads tool_input.command from stdin and exits 2, with the reason on stderr, if the command is rm with both -r and -f in any form. Test it on rm -rf, rm -r -f and rm -fr."
Described as: "run a script before the agent does something", "block a command no matter what the AI thinks", "make it run the formatter after every edit", "an automatic check on every action".
https://dev.dqnotes.com/agent/hook

### Subagent
Also: Sub-agent.
A second agent the main one starts for a side job. It gets its own context window and its own instructions, and hands back a summary. The thousands of lines it read never enter the main conversation.
Searching and reading are noisy, and a subagent keeps that noise away from the agent doing the real work. A fresh one also hasn't seen the reasoning it is checking. The cost: it starts knowing nothing, so your brief to it is all it has.
Not Parallel agents: A subagent is a helper inside one session that reports back to it. Parallel agents are separate sessions you run side by side, usually each on its own branch.
Seen in: Claude Code: Markdown files in .claude/agents; each subagent "runs in its own context window", with Explore and Plan built in. Codex: TOML files in .codex/agents. Cursor: .cursor/agents, and it also reads the .claude and .codex folders. Docs read November 2026.
Source: Claude Code docs, Subagents, https://code.claude.com/docs/en/sub-agents
Source: OpenAI Codex docs, Subagents, https://developers.openai.com/codex/subagents/
Source: Cursor docs, Subagents, https://cursor.com/docs/subagents
Say it: "Use a subagent to find every place we read the session cookie. It must return only a list of file:line with one sentence each, not the files. I don't want its search results in this conversation."
Described as: "get another AI to do the research", "the main chat is filling up with search output", "a helper agent that reports back", "hand off a side task".
https://dev.dqnotes.com/agent/subagent

### MCP
Also: Model Context Protocol.
Model Context Protocol: a standard way for an agent to use tools that live outside it. A server offers tools such as "search issues", and any agent that speaks the protocol can list and call them.
One integration works across every agent, so you don't rebuild the link to, say, your bug tracker for each one. But each tool definition costs tokens on every request, and a server's tools run real code. Treat installing one like adding a dependency that has your agent's access.
Not Tool: A tool is one function. MCP is how a tool on someone else's server reaches your agent. Built-in tools like Read don't use it.
Seen in: Anthropic gave it to the Agentic AI Foundation, under the Linux Foundation, on 9 December 2025. It is JSON-RPC 2.0 with methods such as tools/list and tools/call. The latest spec when we read it, 2026-07-28, has no protocol-level session: every request carries what it needs. In Claude Code: claude mcp add --transport http <name> <url>.
Source: MCP specification, Tools, https://modelcontextprotocol.io/specification/latest/server/tools
Source: Anthropic, Donating MCP and establishing the AAIF, 2025, https://www.anthropic.com/news/donating-the-model-context-protocol-and-establishing-of-the-agentic-ai-foundation
Source: Claude Code docs, MCP, https://code.claude.com/docs/en/mcp
Say it: "Connect our issue tracker as an MCP server in .mcp.json. List every tool it exposes with its description in full, tell me how many tokens the definitions add to each request, and remove any tool we won't use."
Described as: "connect the agent to GitHub or my database", "a plugin for the AI", "how does it talk to other services", "the agent needs to see our issue tracker".
https://dev.dqnotes.com/agent/mcp

### Code registry
Also: shadcn registry, Copy-in registry.
A list of code you can pull into your repo as files you own, with a command that works out what else each piece needs. You end up with source in your tree that you can edit, and not a dependency someone else updates for you.
An agent can read, change and test code in your repo, which is harder with what sits inside a package. You pay in upkeep: nothing tells you when the original gets a fix, and code you copied without reading is still a stranger's code. Nobody we found has measured whether agents do better this way.
Not Package registry: npm is a registry of published packages: a version in node_modules and a line in a lockfile. A code registry gives you files to commit. Same word, opposite ownership, and one registry item can still pull npm packages in.
Seen in: shadcn, on X, 5 November 2026: "Agents love registries. You can put pretty much anything in it. Seriously, try building one." That is one person's view, unmeasured, about their own project. The docs: "You can use the shadcn CLI to run your own code registry." An item lists npm packages under dependencies, and other items under registryDependencies, "Used for registry dependencies. Each entry is an item address." A file can name its own target, so it can land anywhere: "Use ~ to refer to the root of the project". For agents, the docs offer an MCP server that "works out of the box with any shadcn-compatible registry". The registry health page is plain about trust: "Always review third-party code before installing it." Reported use: OrcDev, whose post shadcn was quoting, runs one for a recording app: "made it so much easier to control every component in the app". A registry item is a supply-chain decision too, and nothing pins or audits it the way a lockfile can.
Source: shadcn on X, 5 November 2026, https://x.com/shadcn/status/2118355007003058503
Source: The same post, through the fxtwitter mirror, https://api.fxtwitter.com/shadcn/status/2118355007003058503
Source: shadcn/ui docs, Registry, https://ui.shadcn.com/docs/registry
Source: shadcn/ui docs, registry-item.json, https://ui.shadcn.com/docs/registry/registry-item-json
Source: shadcn/ui docs, Registry MCP, https://ui.shadcn.com/docs/registry/mcp
Source: shadcn/ui docs, Registry health, https://ui.shadcn.com/docs/registry/health
Source: OrcDev on X, 5 November 2026, through the fxtwitter mirror, https://api.fxtwitter.com/orcdev/status/2118303406007714068
Say it: "Run bunx shadcn@latest view @acme/login-form, then add it with --dry-run and show me every file it would write and every package it would add. Read those files for network calls and env reads, and tell me what you find. If they are clean, add it, commit it alone, and note the source and date in docs/vendored.md."
Described as: "copy the component into my repo", "should this be a package or a registry", "my agent cannot change the library code", "share components across my projects", "what is a shadcn registry".
https://dev.dqnotes.com/agent/registry

### Permission modes
Also: Approval mode, Run mode, YOLO mode.
A setting for how much the agent may do before it has to stop and ask you. Modes run from asking about nearly everything to asking about almost nothing.
With an agent running for an hour you can't read every prompt, so the honest question is which actions you want stopped. Choosing a mode is choosing what you will no longer see. The names mislead too: Claude Code's acceptEdits also approves rm inside your project.
Not Sandbox: A mode decides what the agent may try without asking. A sandbox decides what the operating system lets a command do once it runs.
Seen in: Claude Code: default, acceptEdits, plan, auto, dontAsk, bypassPermissions, and deny rules hold in all of them. Codex splits it in two: --sandbox and --ask-for-approval. Cursor 3.6 and later: Auto-review (the default), Allowlist, Run Everything, and its docs say "Auto-review is not a security boundary." Elicit's interviewers name candidates who "yolo" with --dangerously-skip-permissions as a bad sign. All as of November 2026.
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Source: OpenAI Codex docs, Approvals and security, https://learn.chatgpt.com/docs/agent-approvals-security
Source: Cursor docs, Run modes, https://cursor.com/docs/agent/run-modes
Source: Elicit, Engineering interviews in the era of agents, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Say it: "Start in default mode. Add deny rules for Bash(git push *) and Read(./.env), an ask rule for Bash(rm *), and allow rules for Bash(bun test) and Bash(bun run typecheck). Then open /permissions and show me what is there."
Described as: "it keeps asking me to approve everything", "how do I stop all the prompts", "let it run without asking", "yolo mode", "can I make it ask before it deletes things".
https://dev.dqnotes.com/agent/permission-mode

### Allowlist
Also: Allow rules, Permission rules.
A list of actions the agent may take without asking. Anything not on it still stops for approval. Rules like Bash(bun test) are how you cut prompts without turning them off.
It is the cheap way to fewer prompts, but a pattern allows more than it looks like. Bash(git *) allows git push --force. Rules also match the text of a command, so a deny rule stops the usual spelling and can miss the same program written another way.
Not Denylist: An allowlist says what may run, and everything else stops. A denylist says what may not, and everything else carries on, so a dangerous command nobody thought of gets through.
Seen in: Claude Code checks rules in the order deny, ask, allow, and "An allow rule can't carve an exception out of a deny rule." Its docs say a deny rule on a command "isn't a security boundary around the program": Bash(rm *) stops rm -rf build/ but not bash -c 'rm -rf build/'. Cursor keeps a similar allowlist under Approvals & Execution. Read November 2026.
Source: Claude Code docs, Permissions, https://code.claude.com/docs/en/permissions
Source: Cursor docs, Run modes, https://cursor.com/docs/agent/run-modes
Say it: "Add to permissions.allow: Bash(bun test), Bash(bun run typecheck), Bash(git diff *), Bash(git status). Do not add Bash(git *) or Bash(bun *), because those also allow git push and bun publish."
Described as: "stop asking me about bun test", "approve this command forever", "pre-approve safe commands", "the list of commands it can run", "always allow".
https://dev.dqnotes.com/agent/allowlist

### Sandbox
Also: Agent sandbox.
A boundary enforced by the operating system that limits what a running command can touch: which folders it can write to, which it can read, and where it can connect. It holds even when the agent has been tricked.
Prompts depend on you paying attention, and rules match text. A sandbox doesn't care how a command is spelled, which is what makes long unattended runs reasonable. But read the defaults before you rely on one, because they are not what most people assume.
Not Container: A container isolates a whole environment and is one way to build a boundary. Agent sandboxes often use lighter operating system features, such as Seatbelt on a Mac, applied to each command.
Seen in: November 2026. Claude Code: off by default, turned on with /sandbox, shell commands only. A sandboxed command can still read "Most of the machine, including credential files", and the network starts with an empty allowed list. Codex: on by default as workspace-write, with network off and .git read-only. Cursor: writes limited to the workspace, network "Blocked by default". Claude Code and Codex both use Seatbelt on macOS.
Source: Claude Code docs, Sandboxing, https://code.claude.com/docs/en/sandboxing
Source: OpenAI Codex docs, Approvals and security, https://learn.chatgpt.com/docs/agent-approvals-security
Source: Cursor docs, Run modes, https://cursor.com/docs/agent/run-modes
Say it: "Turn on the sandbox with /sandbox. Add ~/.ssh and ~/.aws to sandbox.filesystem.denyRead, allow only registry.npmjs.org in sandbox.network.allowedDomains, set allowUnsandboxedCommands to false, then run cat ~/.ssh/config and show me it fails."
Described as: "stop it touching anything outside the project", "what if it runs something bad", "lock the agent in a box", "can it read my SSH keys".
https://dev.dqnotes.com/agent/sandbox

### Plan mode
A mode where the agent can read and explore but can't edit your source files. It writes a plan, you read and change it, and only then does the work start.
A wrong plan takes a minute to fix, and a wrong diff can take an hour to unpick. It is also the cheapest moment to find out the agent misunderstood you. People do skip it: Elicit's interviewers report that "Plan-first thinking is rarer than we expected."
Not Plan: A plan is the document. Plan mode is the setting that stops the agent editing while it writes one.
Seen in: Claude Code: Shift+Tab or /plan, and edits "stay blocked until you approve the plan". Codex: /plan or Shift+Tab. Cursor: Shift+Tab, and plans save to your home directory unless you move them to the workspace. Read November 2026. Not everyone agrees it is worth it: Lauren Tan's pstack README says "i don't believe in planning. the best spec is code."
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Source: OpenAI Codex docs, Best practices, https://learn.chatgpt.com/guides/best-practices
Source: Cursor docs, Plan mode, https://cursor.com/docs/agent/plan-mode
Source: Tan, pstack README, https://github.com/cursor/plugins/blob/main/pstack/README.md
Source: Elicit, Engineering interviews in the era of agents, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Say it: "/plan Add rate limiting to /api/login. Read the existing middleware first. Give me the files you would change, the test you would write first, and what you are unsure about. Do not edit anything."
Described as: "make it think before it touches anything", "read only until I say go", "it started coding before I had explained", "show me what you will do first".
https://dev.dqnotes.com/agent/plan-mode

### Agent session
Also: Thread, Conversation.
One run of an agent from its first message to its last, with its own growing context. When it ends, that context is gone unless you resume it, or the agent wrote something to a file.
An agent doesn't remember yesterday. Whatever a session decided and didn't write into the repo is lost to the next one, so a good session leaves files behind. Keeping to one task per session also keeps one job's clutter out of the next.
Not Web session: A web session, from chapter 2, is how a server remembers who is logged in. An agent session is one run of an agent. Same word, unrelated thing.
Seen in: Justin Young at Anthropic: "The core challenge of long-running agents is that they must work in discrete sessions". In Claude Code, /clear starts a new conversation with empty context and /resume returns to an earlier one.
Source: Young, Effective harnesses for long-running agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
Source: Claude Code docs, Commands, https://code.claude.com/docs/en/commands
Say it: "Before we finish this session, write what you learned, what is left and the next step to docs/progress.md, and commit it. The next session will start from that file and nothing else."
Described as: "it forgot what we did yesterday", "start a new chat or keep going", "one very long conversation", "pick up where I left off".
https://dev.dqnotes.com/agent/session

### Agent memory
Also: Auto memory, Memories.
Whatever carries over from one session to the next. There is nothing magic in it: it is files, put back into the context when a session starts. Some you write yourself. Some the tool writes for itself.
Without it, every session pays again to find out what the last one learned. With it, a wrong note gets loaded into every session too, possibly one you have never read. Treat memory like code: look at it, and delete what is wrong.
Not Context: Context is what the model can see on this request. Memory is what gets put back into the context when a new session starts.
Seen in: Claude Code's auto memory (on by default in local sessions, November 2026): Claude writes its own notes, and the first 200 lines or 25KB load every session. Codex has memories, with /memories to control whether a chat uses them. Steve Yegge's Beads takes another route, an issue tracker billed as "A memory upgrade for your coding agent".
Source: Claude Code docs, Memory, https://code.claude.com/docs/en/memory
Source: OpenAI Codex docs, Memories, https://developers.openai.com/codex/memories
Source: Yegge, Beads, https://github.com/steveyegge/beads
Say it: "Show me your auto memory files. Delete anything out of date, and move anything the whole team needs into CLAUDE.md, where it gets reviewed like code."
Described as: "make it remember this for next time", "it learned this last week and forgot", "where does it store what it knows about me", "it keeps believing something out of date".
https://dev.dqnotes.com/agent/memory-agent

### Headless mode
Also: Print mode, Non-interactive mode.
Running an agent with nobody at the keyboard: one command, one prompt, and output a script can read. It is how an agent runs on a schedule, or in continuous integration, the automatic checks on a server that chapter 9 covers.
There is no one to answer a prompt, so whatever would have asked must be decided in advance. Allow too little and the run achieves nothing. Allow too much and nothing is watching. And never trust its final message: have the script run the checks itself.
Not Background agent: A background agent runs while you do something else, and you can still step in. Headless means there is nobody to step in.
Seen in: Claude Code: claude -p with --allowedTools, --output-format json and --max-budget-usd. Since v2.1.302, calls needing approval are denied unless you allowed them in advance. Its docs warn that without --bare, -p runs a project's hooks and .mcp.json servers "even in a folder you've never trusted". Codex: codex exec. Cursor: agent -p, where changes are only proposed until you add --force. All as of November 2026.
Source: Claude Code docs, Headless, https://code.claude.com/docs/en/headless
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Source: OpenAI Codex docs, Non-interactive mode, https://learn.chatgpt.com/docs/non-interactive-mode
Source: Cursor docs, Headless CLI, https://cursor.com/docs/cli/headless
Say it: "In CI run: claude -p "Fix the lint errors in src/" --allowedTools "Read,Edit,Bash(bun run lint)" --max-budget-usd 2 --output-format json. Then have the job run bun run lint itself and fail on a non-zero exit, whatever the agent said."
Described as: "run it from a script", "an agent in CI", "no one there to click approve", "run it overnight without me", "automate it on a schedule".
https://dev.dqnotes.com/agent/headless

### Levels of autonomy
Also: Autonomy slider, The five levels.
A scale for how much of the work an agent does before a person looks at it. At one end you approve every line. At the other you write a spec and judge only the result.
Each step up removes a check you were doing, and it is only safe if something else now does that check. People talk about the levels as if higher were better. The honest question is what replaced you at each one.
Seen in: Dan Shapiro's levels, January 2026: 0 spicy autocomplete, 1 coding intern, 2 junior developer, 3 developer ("You spend your days reviewing code"), 4 engineering team, 5 dark software factory ("It's a black box that turns specs into software"). StrongDM says it works at the top, where code "must not be" reviewed by humans. Steve Yegge has a different scale with 8 stages. These are opinions, not measurements.
Source: Shapiro, The Five Levels, 2026, https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/
Source: McCarthy, Software Factories And The Agentic Moment, StrongDM, 2026, https://factory.strongdm.ai/
Source: Yegge, Welcome to Gas Town, 2026, https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16dd04
Say it: "In this repo you write and I review every diff, except docs/, which you can merge once CI is green. Put that in AGENTS.md. Then tell me which checks would have to exist before I could stop reading src/payments/."
Described as: "how much should I let it do alone", "do I still need to read the code", "should I review everything", "how far can I trust it", "no human looks at the code".
https://dev.dqnotes.com/agent/autonomy-slider

### Human in the loop
Also: Approval prompt, Prompt fatigue.
A person has to approve at a chosen point before the work continues. That point could be each command, each plan or each merge.
An approval only counts if it gets read. Ask too often and people click yes without looking, and then there is a human in the loop but no check. A few approvals on things that matter beat fifty on things that don't.
Not Human on the loop: In the loop, the work waits for you. On the loop, it carries on while you watch and can step in. Many long agent runs are really the second, whatever they get called.
Seen in: The MCP spec: "there SHOULD always be a human in the loop with the ability to deny tool invocations." Claude Code's docs offer auto mode for "reducing prompt fatigue". Birgitta Böckeler says a good harness should not aim to remove human input but "direct it to where our input is most important."
Source: MCP specification, Tools, https://modelcontextprotocol.io/specification/latest/server/tools
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Source: Böckeler, Harness engineering for coding agent users, 2026, https://martinfowler.com/articles/harness-engineering.html
Say it: "Don't stop to ask about edits or test runs. Do stop and ask me before any git push, any migration, any new dependency, and anything that writes outside this repo. List those back to me before you start."
Described as: "I just click yes without reading", "approval fatigue", "where should I actually be checking", "it asks me about everything".
https://dev.dqnotes.com/agent/human-in-the-loop

### Destructive actions
Also: Irreversible actions.
Anything that can't easily be undone: deleting data, force-pushing, dropping a table, sending an email, spending money. An ordinary mistake gets another attempt. These may not.
An agent is as confident about these as about anything else, and the loop means it can reach one many steps after you stopped watching. So keep the undo out of its reach: commit first, keep a backup it can't delete, and don't give it production credentials.
Seen in: July 2025: Jason Lemkin reported that Replit's agent deleted a live database during a code freeze, and then said a rollback wouldn't work. He recovered the data by hand. Fortune reports data for more than 1,200 executives was wiped; one critic disputes that it was really production. Replit's CEO called it "Unacceptable and should never be possible." Claude Code (November 2026) won't auto-approve rm of paths it recognises as critical, such as your home directory, but warns that "A command the check misses gets no special handling".
Source: Fortune, AI-powered coding tool wiped out a software company's database, 2025, https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure
Source: Claude Code docs, Permission modes, https://code.claude.com/docs/en/permission-modes
Say it: "Add deny rules: Bash(git push --force*), Bash(git push -f*), Bash(git reset --hard*), Bash(*DROP TABLE*). Commit before you start so everything can be restored. You don't get production credentials: use the staging database in .env.staging."
Described as: "it deleted my files", "it wiped the database", "can it push over my work", "what should it never be allowed to do", "commands you can not undo".
https://dev.dqnotes.com/agent/lethal-operations

### What the agent can reach
Also: Blast radius of an agent, Agent trust boundary.
Everything the agent could touch if it wanted to, or if someone tricked it: your files, your environment variables, your logins, your network. That reach, and not the task you gave it, is the size of what could go wrong.
Anything it reads can contain instructions, and a model can't reliably tell your words from an attacker's. So you judge a setup by its reach. If private data, outside content and a way to send data out are all in one agent, you have a problem whatever the task is.
Not Permissions: Permissions are what you said it may do. Reach is what it could do with what it already holds. An agent running as you has your SSH keys, the files that log you in to other machines, in reach unless something blocks them.
Seen in: Simon Willison's lethal trifecta (June 2025): "Access to your private data", "Exposure to untrusted content", "The ability to externally communicate". In CVE-2025-53773, text planted in a source file or GitHub issue could make Copilot's agent switch on its own auto-approve setting (patched August 2025). Repo content became instructions.
Source: Willison, The lethal trifecta for AI agents, 2025, https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
Source: Rehberger, GitHub Copilot remote code execution via prompt injection, 2025, https://embracethered.com/blog/posts/2025/github-copilot-remote-code-execution-via-prompt-injection/
Say it: "List everything you can reach from here: environment variable names, which credential files exist, which MCP tools you have, and which hosts you can connect to. Print names only, never values."
Described as: "what can it get at on my machine", "could a README trick my agent", "it has all my logins", "is it safe to point it at a random repo", "what is the worst it could do".
https://dev.dqnotes.com/agent/trust-boundary-agent

## 7. Context

What the agent can see decides what it can do. How to feed it, keep it clean, and carry knowledge between sessions.

### Context engineering
Choosing what the model can see on each request: which instructions, files, tool results and history go in, and which stay out. The prompt is what you say. Context is everything else it reads before it answers.
An agent gathers most of its context itself by calling tools, so your sentence is a small part of what it reads. Better wording can't supply a file it never opened. In chat you pasted everything in by hand. With an agent, you set up what it can reach and what it reads first.
Not Prompt engineering: Prompt engineering is choosing the words of a request. Context engineering decides what else is in the request, turn after turn, as the agent adds to it. Anthropic treats the first as the earlier stage of the second.
Seen in: The term spread in June 2025. Tobi Lütke: "the art of providing all the context for the task to be plausibly solvable by the LLM." Andrej Karpathy: "+1 for 'context engineering' over 'prompt engineering'", calling it "the delicate art and science of filling the context window". Both as quoted by Simon Willison. Anthropic's version: "finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome." HumanLayer's guide goes further: "the contents of your context window are the ONLY lever you have to affect the quality of your output". That overstates it, since checks and tools also change the result, as later chapters show.
Source: Willison, Context engineering, 27 June 2025, https://simonwillison.net/2025/Jun/27/context-engineering/
Source: Rajasekaran et al., Effective context engineering for AI agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
Source: HumanLayer, Getting AI to Work in Complex Codebases (no byline or date on the page), https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/ace-fca.md
Say it: "Before you change anything, list the files you have read for this task, what you still need to see, and any name you are assuming exists. Read src/api/checkout.ts and src/discount.ts first, then stop and tell me."
Described as: "what does the agent actually know about my project", "it guessed instead of looking", "getting it the right information", "prompt engineering but for agents", "why better wording did not help".
https://dev.dqnotes.com/context/context-engineering

### Context budget
Also: Attention budget.
Treating the window as money you spend. Each token that goes in stays in, gets sent again on every later turn, and leaves less room for what comes next. Test logs and whole files are the big purchases.
An agent spends this without asking. A test log nobody needed is read again on each turn after it, and billed each time. Larger windows moved the limit but not the logic. Claude Code's docs say you will "rarely run out", and still advise clearing between tasks, because "a focused context gets you better answers and costs less."
Not Token cost: Token cost is the price in dollars. The budget also counts what a token takes from later work, which caching doesn't discount: it can be almost free to resend and still be in the way.
Seen in: Anthropic: "LLMs have an 'attention budget' that they draw on when parsing large volumes of context." HumanLayer lists what eats it: file searches, code-flow exploration, edits, test and build logs, and big JSON from tools. In Claude Code, /context shows a live breakdown by category. Claude Code's docs note: "Every request sends the whole conversation".
Source: Rajasekaran et al., Effective context engineering for AI agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
Source: HumanLayer, Getting AI to Work in Complex Codebases (no byline or date on the page), https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/ace-fca.md
Source: Claude Code docs, Explore the context window, read 5 November 2026, https://code.claude.com/docs/en/context-window
Source: Claude Code docs, Best practices, read 5 November 2026, https://code.claude.com/docs/en/best-practices
Say it: "Here is my /context output. Name the five biggest items and say whether this task needs each one. Take any MCP server we haven't used this week out of .mcp.json. From now on, search with rg -n and read line ranges, not whole files."
Described as: "what is using up all the space", "why does each message cost more than the last", "how much of the context is gone already", "it has read half the repo", "where did my tokens go".
https://dev.dqnotes.com/context/context-budget

### The dumb zone
Also: Smart zone, Context rot.
The informal claim that a model gets worse past some point in its window, so you should stay below it. That reliability falls as context grows has support. Any specific percentage is a working rule, because no one has shown us a measured line.
The habit it teaches is good: clear early, and keep noise out. But don't repeat a number as fact, least of all in an interview. Windows went from 200K to 1M, and a percentage and a token count now give very different answers. Watch for symptoms in your own sessions: rereading files, forgetting rules.
Not Context rot: Context rot is Anthropic's name for a tendency: recall falls as tokens grow, more gently on some models. The dumb zone is a practitioner's name for a threshold. The first has benchmarks behind it, and the second has experience.
Seen in: Dex Horthy of HumanLayer, December 2025, is the earliest dated text we read that uses the words: "stay in the ~75k token 'smart zone' for claude models", and "an agent in the dumb zone". The roughly 40% figure people attach to it comes from summaries of his AI Engineer talk, and we could not check those against a transcript, so treat it as attributed. HumanLayer's guide says "keeping utilization in the 40%-60% range". Neither shows data. Anthropic has no threshold either: "as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases." Its API docs say "more context isn't automatically better". The research is real but narrow, on 2023 models. Liu and colleagues: recall "significantly degrades when models must access relevant information in the middle of long contexts". Shi and colleagues: arithmetic accuracy "dramatically decreased when irrelevant information is included". Meanwhile Claude Code by default compacts native 1M sessions at about 967K, so the vendor's default and the rule of thumb disagree.
Source: Horthy, Context-Efficient Backpressure for Coding Agents, HumanLayer, 2025, https://www.humanlayer.dev/blog/context-efficient-backpressure
Source: HumanLayer, Getting AI to Work in Complex Codebases (no byline or date on the page), https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/ace-fca.md
Source: Rajasekaran et al., Effective context engineering for AI agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
Source: Anthropic docs, Context windows, read 5 November 2026, https://platform.claude.com/docs/en/build-with-claude/context-windows
Source: Liu et al., Lost in the Middle, TACL, 2023, https://arxiv.org/abs/2307.03172
Source: Shi et al., Large Language Models Can Be Easily Distracted by Irrelevant Context, ICML, 2023, https://arxiv.org/abs/2302.00093
Source: Claude Code docs, Model configuration, read 5 November 2026, https://code.claude.com/docs/en/model-config
Say it: "If you reread a file you already read, or break a rule in AGENTS.md, stop and tell me. When I see /context pass 100k, I will ask you to write the state to docs/handoff.md and start a new session. That number is our rule of thumb, not a known limit."
Described as: "it gets worse the longer the session goes", "it got dumber after a while", "how full is too full", "is 40 percent a real number", "the smart zone thing", "it forgets things late in a session".
https://dev.dqnotes.com/context/dumb-zone

### Instruction budget
Also: Short AGENTS.md.
The limit you set on instructions that always load. AGENTS.md goes into every request, so each line has to earn its place on every task, including the many it has nothing to do with. Short and specific beats long and thorough.
Caching makes the dollars small, so that is not the main cost. Agents tend to do what these files say, so each needless requirement makes them do needless work, and a long file buries the rules that matter. Keep what the agent can't work out from the code, and cut the rest.
Not Documentation: Documentation can be as long as it needs to be, because it loads only when something points to it. An instructions file loads every time, so length is a cost it pays on every turn.
Seen in: OpenAI: "give Codex a map, not a 1,000-page instruction manual". One big file failed because "When everything is 'important,' nothing is." Claude Code's docs target under 200 lines ("Longer files consume more context and reduce adherence") and warn that @path imports don't reduce the cost. The most careful evidence is Gloaguen and colleagues (ETH Zurich and LogicStar, 2026): context files do "not generally improve task success rates, while increasing inference cost by over 20% on average", and "repository overviews" are "not helpful", though the files are "useful for specifying non-standard coding practices". Their benchmark measures issue-fixing, so it says nothing about rules that protect against rare disasters.
Source: Lopopolo, Harness engineering, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: Claude Code docs, Memory, read 5 November 2026, https://code.claude.com/docs/en/memory
Source: Gloaguen et al., Evaluating AGENTS.md, 2026, https://arxiv.org/abs/2602.11988
Say it: "Go through AGENTS.md line by line. Delete anything you could learn by reading the code, and any overview of the repo. Keep commands and rules the code can't show. Move rules that only apply to src/db/ into .claude/rules/db.md with a paths field. Get it under 100 lines and show me the diff."
Described as: "my instructions file keeps growing", "how long should CLAUDE.md be", "the AI generated a huge AGENTS.md", "it ignores half of what I wrote", "does AGENTS.md even help".
https://dev.dqnotes.com/context/instruction-budget

### Progressive disclosure
Starting the agent with a small entry point and showing it where to look next: a short index now, details when they are needed. A skill's description is the index entry, and its body is the detail.
You keep a whole shelf of procedures for the price of their labels. But the risk moves. The detail arrives only if the label matches, so a vague description means a procedure that never loads, and nothing tells you. Test the label, not just the procedure.
Not Just-in-time retrieval: Progressive disclosure is how you arrange knowledge, so there is a path to follow. Just-in-time is the agent following that path, or searching, at the moment it needs something.
Seen in: OpenAI: "agents start with a small, stable entry point and are taught where to look next, rather than being overwhelmed up front." Anthropic: "Progressive disclosure is the core design principle that makes Agent Skills flexible and scalable." Anthropic reported the same effect for tools in November 2025: about 77K tokens used by tool definitions before any work, down to about 8.7K with tool search, a vendor's own test. Claude Code (November 2026) gives the skill listing 1% of the window and drops descriptions of your least-used skills when it overflows, and each description is cut at 1,536 characters.
Source: Lopopolo, Harness engineering, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: Anthropic, Equipping agents for the real world with Agent Skills, 2025, https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills
Source: Anthropic, Introducing advanced tool use, 2025, https://www.anthropic.com/engineering/advanced-tool-use
Source: Claude Code docs, Skills, read 5 November 2026, https://code.claude.com/docs/en/skills
Say it: "Move the 400-line release procedure out of AGENTS.md into .claude/skills/release/SKILL.md, with the description "Use when cutting a release or bumping the version." Leave one line in AGENTS.md. Start a new session, ask it to cut a release, and show me that the skill loaded."
Described as: "only load the instructions when they are needed", "it never uses the skill I wrote", "too many tools are loaded", "a table of contents for the agent", "keep the start small".
https://dev.dqnotes.com/context/progressive-disclosure

### Just-in-time retrieval
Also: Just-in-time context, Agentic search.
Keeping lightweight pointers, such as file paths and search patterns, and loading what they point to only when it is needed. The agent searches, reads twenty lines and moves on, and never holds the whole repo in view.
You pay only for what the task touches. And what it reads is current, whereas a file loaded an hour ago may have changed since. The price is more tool calls, and an agent that searches for the wrong word may not learn what it missed.
Not Loading up front: Up front is quick and predictable, and it goes stale and costs tokens on every turn. Claude Code does both: its instructions file loads at the start, and everything else is found as it goes.
Seen in: Anthropic: agents "maintain lightweight identifiers (file paths, stored queries, web links, etc.)", and in Claude Code "CLAUDE.md files are naively dropped into context up front, while primitives like glob and grep allow it to navigate its environment". Long command output works this way too. In Claude Code (November 2026), a successful command past roughly 30,000 characters arrives as a file path plus a preview of up to 2,000 characters, and the agent reads the rest if it wants to.
Source: Rajasekaran et al., Effective context engineering for AI agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
Source: Claude Code docs, Tools reference, read 5 November 2026, https://code.claude.com/docs/en/tools-reference
Say it: "Don't read the whole of src/. Run rg -n "SESSION_TTL" src, open only those lines with 20 lines either side, and tell me the value on disk right now. If the search finds nothing, say so. Don't guess."
Described as: "read only the part it needs", "stop reading every file", "it uses an old version of the file", "search first, then read", "look it up when you need it".
https://dev.dqnotes.com/context/just-in-time

### Codebase map
Also: Repo map.
A short description of how a repo is laid out: which folder does what, and where to begin. It gives the agent a picture of the project without reading it, so its first search is aimed at something.
Without one, an agent can spend its first tokens wandering. But this is the most disputed advice in the chapter. Measured evidence says overviews often don't help, because agents find their way with ls and search anyway. So write down only what those can't show: the real entry point, what is generated, what owns what.
Not Architecture document: An architecture document explains why the system is shaped as it is, and can be long. A map says where things are, and stays short enough to load every time.
Seen in: OpenAI treats AGENTS.md "as the table of contents". Aider builds a "concise map of your whole git repository" from code symbols, ranks it with a graph, and fits it to a token budget that defaults to 1k. Ghostty's short AGENTS.md has a directory map. On the other side, Gloaguen and colleagues (2026) found that "repository overviews, although popular and recommended by model providers, are not helpful" on their benchmark. Both can be true: a benchmark of one-off fixes may not reward what a map gives you over months.
Source: Lopopolo, Harness engineering, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: Aider docs, Repository map, https://aider.chat/docs/repomap.html
Source: Ghostty, AGENTS.md, https://github.com/ghostty-org/ghostty/blob/main/AGENTS.md
Source: Gloaguen et al., Evaluating AGENTS.md, 2026, https://arxiv.org/abs/2602.11988
Say it: "Add at most 10 lines to AGENTS.md: the entry point, which folders own the database and auth, and which folder is generated and must not be edited. Anything ls would tell you stays out. Then start a new session and ask it where login is handled."
Described as: "it does not know where anything is", "a summary of how the repo is laid out", "it keeps searching in the wrong place", "should I describe the folders", "where do I tell it to start".
https://dev.dqnotes.com/context/codebase-map

### Pointing at examples
Also: Prior art.
Telling the agent to copy something that already exists: "follow the pattern in src/db/users.ts". One path, and the code does the explaining. It is about the cheapest useful context there is.
Left alone, an agent writes what was common in its training text and not what is common in your repo. A pointer settles naming, errors and test layout in one line. It also copies the flaws, so point at the file you would want more of.
Not Examples in a brief: An example in a brief shows an input and the output you want. Prior art is existing code in the repo, so it also teaches conventions nobody thought to write down.
Seen in: Claude Code's docs replace "add a calendar widget" with "follow the pattern in HotDogWidget.php and build a calendar widget that lets you pick a month and page between years. no new libraries". Anthropic prefers examples to rule lists: "For an LLM, examples are the 'pictures' worth a thousand words."
Source: Claude Code docs, Best practices, read 5 November 2026, https://code.claude.com/docs/en/best-practices
Source: Rajasekaran et al., Effective context engineering for AI agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
Say it: "Add a GET /api/orders route. Copy the structure of src/api/users.ts: same validation, same error shape, same test layout as src/api/users.test.ts. Tell me anything you had to do differently, and why."
Described as: "do it the way we already do it", "it invented its own style", "it ignores how the rest of the code looks", "copy this file as a template", "it wrote a new helper when we already had one".
https://dev.dqnotes.com/context/prior-art

### Retrieval (RAG)
Also: Retrieval-augmented generation, BM25.
Searching a body of text, taking the top matches and putting them in the prompt so the model answers from them. Keyword search (BM25 is the standard scoring formula) finds exact words. Embeddings find related meaning. Each misses what the other finds.
The model sees only what came back, so a missed page may as well not exist, and the answer still sounds sure. For code, the leading agents have moved towards searching live. For big piles of documents nobody can grep usefully, retrieval still earns its place, if you measure it.
Not Just-in-time retrieval: In classic RAG a search system runs first and its results land in the prompt. In just-in-time retrieval the agent picks the search, reads what came back, and can search again.
Seen in: Lewis and colleagues named it in 2020. Anthropic's 2024 write-up on contextual retrieval reports failed retrievals falling from 5.7% to 1.9% by combining embeddings, BM25 and reranking on its tests, and says that under 200,000 tokens "you can just include the entire knowledge base in the prompt". For code, Boris Cherny, February 2026: "Early versions of Claude Code used RAG + a local vector db, but we found pretty quickly that agentic search generally works better." Cursor's search docs (read November 2026) say it "does not store embeddings of your codebase for search". These are vendors describing their own products, not independent studies.
Source: Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, 2020, https://arxiv.org/abs/2005.11401
Source: Robertson and Zaragoza, The Probabilistic Relevance Framework: BM25 and Beyond, 2009, https://doi.org/10.1561/1500000019
Source: Anthropic, Introducing Contextual Retrieval, 2024, https://www.anthropic.com/news/contextual-retrieval
Source: Cherny on X, 1 Feb 2026 (read through the fxtwitter mirror of this status), https://x.com/bcherny/status/2017824286489383315
Source: Cursor docs, Search, https://cursor.com/docs/agent/tools/search
Say it: "Search docs/ with rg first. If we build retrieval, measure it: write 20 real questions, each with the document that answers it, and report how many of those documents appear in the top 5, for BM25 alone and for BM25 with embeddings."
Described as: "make it answer from our documents", "chat with my docs", "do I need a vector database", "it did not find the page that was right there", "search then answer".
https://dev.dqnotes.com/context/rag

### Context-efficient output
Also: Context-efficient backpressure.
Making your tools print only what an agent can use: one tick when a check passes, and the full output only when it fails. The check still runs in full. Only the reading gets shorter.
A test run that prints 200 lines to say "all good" spends window on nothing, then gets read again each turn. Agents also trim noisy output themselves with head, which keeps only the first lines and can cut off the one failure and force a second full run. Let the exit code decide what gets shown, and not the model.
Not Truncation: Cutting output with head or tail discards by position, so it may discard the failure. A wrapper discards by outcome, because the exit code says which case you are in.
Seen in: Dex Horthy named it at HumanLayer in December 2025 in "Context-Efficient Backpressure for Coding Agents" (backpressure is chapter 9's word for checks that push back on an agent when its work is wrong): "swallow all test/build/lint output and replace it with a single ✓ if the stage passes", and "Success = ✓. Failure = full output." He warns against running a five-minute suite through head -n 50. Claude Code's docs show a hook that filters test output, "reducing context from tens of thousands of tokens to hundreds". bun test has --only-failures and --bail for the same job (checked with bun test --help on Bun in this repo).
Source: Horthy, Context-Efficient Backpressure for Coding Agents, HumanLayer, 2025, https://www.humanlayer.dev/blog/context-efficient-backpressure
Source: Claude Code docs, Costs, read 5 November 2026, https://code.claude.com/docs/en/costs
Say it: "Write scripts/quiet.sh. It runs a command, saves the output to a temp file, prints one ✓ line on exit 0, and otherwise prints the whole file and exits with the same code. Use it for bun test, bun run typecheck and lint, and put it in AGENTS.md."
Described as: "the test output is enormous", "it wastes all its context on logs", "it ran the slow tests twice", "only show me what failed", "it piped the tests through head".
https://dev.dqnotes.com/context/context-efficient-output

### Subagents as context hygiene
Sending noisy work, such as searching or reading big files, to a subagent, so the reading happens in its window and only a short summary comes back to yours. What you are buying is the clean window, not a specialist.
A search through 30 files shouldn't crowd the agent doing the careful work. But it isn't free: it costs more tokens in total, it may be slower because it starts knowing nothing, and you see only its summary, which can be wrong. Now that windows are big, keep it for work that is truly noisy.
Not Agent roles: Roles split work by job: a planner, a coder, a reviewer. Hygiene splits it by noise. Plenty of "role" subagents save no context at all, because their whole output comes back.
Seen in: HumanLayer: "Subagents are about context control." Anthropic: each "might explore extensively, using tens of thousands of tokens or more, but returns only a condensed, distilled summary of its work (often 1,000-2,000 tokens)". pstack: "Route bulk to subagents; keep summaries in the main thread, not raw payloads." Claude Code's docs now push back on the older advice: "Running out of context is rarely the problem anymore", and each of the remaining uses "costs extra tokens".
Source: HumanLayer, Getting AI to Work in Complex Codebases (no byline or date on the page), https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/ace-fca.md
Source: Rajasekaran et al., Effective context engineering for AI agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
Source: Tan, pstack, principle-guard-the-context-window, https://github.com/cursor/plugins/blob/main/pstack/skills/principle-guard-the-context-window/SKILL.md
Source: Claude Code docs, Best practices, read 5 November 2026, https://code.claude.com/docs/en/best-practices
Say it: "Use a subagent to read src/auth/ and find where sessions expire. Return at most 10 lines: file:line and one sentence each, with no code. Also say what you did not check."
Described as: "the search results filled up the chat", "keep the main session clean", "get a helper to do the reading", "are subagents worth it", "it read too many files".
https://dev.dqnotes.com/context/subagent-context

### Context poisoning
A mistake that gets into the context and stays: a guess stated as fact, an invented function, a wrong summary. Every later turn reads it again and builds on it, so the agent's own earlier words keep arguing for the error.
A correction is one line against all the lines already written in favour of the mistake. So a third correction rarely works, and starting over with what you learned often beats arguing. Memory files can carry the same error into every future session.
Not Prompt injection: Prompt injection is someone putting instructions in on purpose. Poisoning is usually the agent's own honest mistake, repeated. A bad memory file or a planted document can do either.
Seen in: Google DeepMind described it in the Gemini 2.5 report, after watching an agent play Pokémon: many parts of the context are "'poisoned' with misinformation about the game state, which can often take a very long time to undo." Drew Breunig's June 2025 list of how contexts fail defines it: "when a hallucination or other error makes it into the context, where it is repeatedly referenced." Claude Code's docs give the working rule: "a third correction mostly adds to a conversation that already contains two failed attempts for Claude to keep re-reading."
Source: Google DeepMind, Gemini 2.5 technical report, p. 70, https://storage.googleapis.com/deepmind-media/gemini/gemini_v2_5_report.pdf
Source: Breunig, How Long Contexts Fail, 2025, https://dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html
Source: Claude Code docs, Best practices, read 5 November 2026, https://code.claude.com/docs/en/best-practices
Say it: "Stop. You have been assuming issuedAt is in seconds, and nothing has shown that. Print one real value from the test fixture. Then write what you found to docs/handoff.md, and I will start a new session from that file."
Described as: "it keeps coming back to the same wrong idea", "I corrected it three times", "it is stuck on a false assumption", "it built everything on a mistake", "it will not let go of it".
https://dev.dqnotes.com/context/context-poisoning

### Drift
Also: Session drift.
A session slowly moving away from what you asked: the scope grows, an early rule stops being followed, the goal changes shape. No single turn looks wrong. You notice when the diff has things in it you never asked for.
Your first message becomes a smaller share of what the model reads, compaction may have removed it, and you are probably not watching closely by then. The defence is short sessions, rules kept on disk and not in chat, and comparing the diff with the original request.
Not Entropy: Entropy is the whole codebase wandering from its standards over months. Session drift happens inside one conversation and ends with it, which makes it much cheaper to fix.
Seen in: Anthropic reports that "models tend to lose coherence on lengthy tasks as the context window fills". Claude Code's docs: "detailed instructions from early in the conversation may be lost. Put persistent rules in CLAUDE.md rather than relying on conversation history", and "Watch the first minute, press Esc the moment Claude drifts". These are vendor observations, not measurements. Drift here is an ordinary word that nobody owns.
Source: Rajasekaran, Harness design for long-running application development, Anthropic, 2026, https://www.anthropic.com/engineering/harness-design-long-running-apps
Source: Claude Code docs, How Claude Code works, read 5 November 2026, https://code.claude.com/docs/en/how-claude-code-works
Source: Claude Code docs, Best practices, read 5 November 2026, https://code.claude.com/docs/en/best-practices
Say it: "Re-read the task at the top of this session, then run git diff. List anything in the diff I did not ask for, and anything I asked for that is missing. Don't fix either until I have answered."
Described as: "it wandered off from what I asked", "it stopped following my rule after a while", "the change got much bigger than I wanted", "it forgot the point of the task", "it started doing other things".
https://dev.dqnotes.com/context/conversation-drift

### Starting fresh
Also: /clear, Context reset.
Ending the conversation and beginning a new one with empty context, then giving it only what it needs. Your files and commits stay. What it read and said is gone, unless it was written down.
It costs nothing, and it removes every wrong turn at once. People resist because it feels like losing progress, but anything worth keeping should be in a file anyway. Do it between unrelated tasks, and after two failed corrections.
Not Compaction: Compaction keeps a summary, so the same conversation carries on, mistakes included. Starting fresh keeps only what you wrote down, which you chose and can read.
Seen in: Claude Code (November 2026): /clear starts "a new conversation with empty context". Its docs: "It costs nothing, and it means the next task doesn't keep paying for the previous one's history", and "/clear plus one new prompt that folds in what you learned works better almost every time." Codex has /clear and /new. Anthropic used resets for a different reason: Sonnet 4.5 showed "context anxiety", wrapping up early near its limit, and then "Opus 4.5 largely removed that behavior on its own, so I was able to drop context resets", which is a reminder that this advice ages with the models.
Source: Claude Code docs, Commands, read 5 November 2026, https://code.claude.com/docs/en/commands
Source: Claude Code docs, Best practices, read 5 November 2026, https://code.claude.com/docs/en/best-practices
Source: OpenAI Codex docs, Slash commands, read 5 November 2026, https://learn.chatgpt.com/docs/developer-commands
Source: Rajasekaran, Harness design for long-running application development, Anthropic, 2026, https://www.anthropic.com/engineering/harness-design-long-running-apps
Say it: "We are moving to something unrelated. Commit what is done and write anything undecided to docs/handoff.md. I will run /clear, then point you at that file."
Described as: "should I start a new chat", "is it ok to throw the conversation away", "it has gone off the rails", "new task, same chat", "start over but keep what we learned".
https://dev.dqnotes.com/context/clear-session

### Compaction
Also: /compact, Tool result clearing.
Replacing older turns with a summary, so a session can continue in less space. Most tools do it automatically near the limit, or when you ask. Whatever the summary leaves out is gone, and nothing flags the loss.
It is a summary you didn't write, and it loses things. A rule you typed in chat can vanish, and a wrong assumption can survive as fact. So keep rules on disk, where they get reloaded, and if you do compact, say what to keep. Doing it yourself, early, beats waiting for the automatic one.
Not Tool result clearing: Clearing removes old raw tool output and leaves the conversation word for word. Compaction rewrites the conversation. Clearing loses less and frees less.
Seen in: Claude Code (November 2026): after compaction, project-root CLAUDE.md is "Re-injected from disk", but path-scoped rules and nested CLAUDE.md files are summarised away until they load again. It re-reads up to five recent files, and a file over 5,000 tokens comes back as a path only. On a long session /compact "is itself a big request". Anthropic: "overly aggressive compaction can result in the loss of subtle but critical context whose importance only becomes apparent later", and of long runs, "compaction isn't sufficient". HumanLayer's "frequent intentional compaction" means doing it on purpose, throughout.
Source: Claude Code docs, Explore the context window, read 5 November 2026, https://code.claude.com/docs/en/context-window
Source: Claude Code docs, Best practices, read 5 November 2026, https://code.claude.com/docs/en/best-practices
Source: Rajasekaran et al., Effective context engineering for AI agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
Source: Young, Effective harnesses for long-running agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
Source: HumanLayer, Getting AI to Work in Complex Codebases (no byline or date on the page), https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/ace-fca.md
Say it: "/compact keep the decision not to add a date library, the failing test name, the exact error text, what we ruled out and the next step. Drop old test logs and file contents."
Described as: "it forgot my rule after a long session", "the conversation got summarised", "context left until auto-compact", "it lost the error message", "what survives when it compresses".
https://dev.dqnotes.com/context/compaction

### Handoff note
Also: Structured note-taking.
A short file one session writes for the next: what is done, what failed, what was ruled out, what to do next and how to check it. It is all that crosses from one empty context to the next, so it has to stand alone.
The next session will otherwise pay to rediscover everything, or walk into the same dead end. What was ruled out is the part people forget, and it is the most expensive to find twice. A cheerful status report answers none of this, so write it for a stranger.
Not Compaction: Compaction is a summary the tool writes so that it can carry on. A handoff is a file you can read, correct and commit, and a fresh agent starts from it with nothing else.
Seen in: Justin Young at Anthropic had the first session create "a claude-progress.txt file that keeps a log of what agents have done", and each later one begin: "Read the git logs and progress files to get up to speed on what was recently worked on." The feature list stayed in JSON, "as the model is less likely to inappropriately change or overwrite JSON files compared to Markdown files." Prithvi Rajasekaran states the cost: a reset gives a clean slate, "at the cost of the handoff artifact having enough state for the next agent to pick up the work cleanly."
Source: Young, Effective harnesses for long-running agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
Source: Rajasekaran, Harness design for long-running application development, Anthropic, 2026, https://www.anthropic.com/engineering/harness-design-long-running-apps
Say it: "Write docs/handoff.md before we stop: branch name, what is done with commit hashes, the exact failing test and error, what you ruled out and how, the next step, and the command that checks it. A new agent must be able to carry on from this file alone."
Described as: "pick up tomorrow where we left off", "notes for the next session", "it starts from scratch every time", "how do I carry on in a new chat", "it tried the same dead end again".
https://dev.dqnotes.com/context/handoff

### Research, plan, implement
Also: RPI, Frequent intentional compaction.
Splitting work into three phases. Research how the code works now and write it down. Turn that into a plan file. Implement from the plan. Each phase starts from the last one's file with a small window, and you review the files.
A research note and a plan are short enough to read properly, which a 2,000-line diff isn't. So you catch a wrong idea when it is one sentence and not a hundred files. For a small change it is overhead, so skip it. Nobody has measured this properly that we could find.
Not Plan mode: Plan mode is a setting that stops edits. Research, plan, implement is a way of working across sessions, whose outputs are files you can review and hand on.
Seen in: From HumanLayer's guide: "A bad line of code is… a bad line of code", a bad line of a plan "could lead to hundreds of bad lines of code", and a bad line of research "could land you with thousands of bad lines of code." Its success stories are its own. Not everyone agrees with the premise. Lauren Tan's pstack README: "i don't believe in planning. the best spec is code."
Source: HumanLayer, Getting AI to Work in Complex Codebases (no byline or date on the page), https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/ace-fca.md
Source: Tan, pstack README, https://github.com/cursor/plugins/blob/main/pstack/README.md
Say it: "Phase 1 only: research how checkout applies discounts today. Read code and change nothing. Write docs/research/discounts.md, under 60 lines, with file:line for every claim, then stop so that I can read it. We plan in a new session, from that file."
Described as: "it works on small things but fails in a big codebase", "review the plan and not the code", "break the work into stages", "it charged ahead without understanding the code", "RPI".
https://dev.dqnotes.com/context/research-plan-implement

### Docs in the repo
Keeping decisions, plans and how-tos as plain files next to the code, not in a wiki or a chat. The agent can search them with the same tools it uses on code, they change in the same commit as the code, and review covers them.
For an agent, a stale doc is worse than none: it believes what it reads, and states it with confidence. So docs in the repo need the same discipline as code, and the discipline can be a check. An agent can also keep them current, if you ask it to in the same commit.
Not AGENTS.md: AGENTS.md loads every time and has to stay short. Files in docs/ load only when something points to them, so they can be as long as they need to be.
Seen in: OpenAI keeps a docs/ directory as its record, and says "Dedicated linters and CI jobs validate that the knowledge base is up to date, cross-linked, and structured correctly", with a recurring "doc-gardening" agent opening fix-up pull requests. That is one team's account of one repo. It warns that its results "should not be assumed to generalize without similar investment".
Source: Lopopolo, Harness engineering, OpenAI, 2026, https://openai.com/index/harness-engineering/
Say it: "Move the deploy steps out of the wiki into docs/deploy.md, with the commands as they really run today. Then add a CI step that fails if a file path mentioned in docs/ no longer exists."
Described as: "where should the docs live", "our wiki is out of date", "it believed an old readme", "keep docs next to the code", "the agent can not see our notion".
https://dev.dqnotes.com/context/docs-in-repo

### The repo as the system of record
Also: System of record.
The rule that anything the work depends on lives in the repo, in a form the agent can read. A decision in a meeting, a chat thread or your head doesn't exist for the agent until someone writes it where the agent looks.
It changes what counts as finishing a discussion: a decision isn't made until it is committed. It helps new people as much as agents. It has limits. Peter Naur argued in 1985 that part of a program's theory never gets written down, and a repo can't hold what nobody can say.
Not Agent memory: Memory files belong to one tool and often one person. The repo is shared, reviewed and versioned, so the next person and the next tool get the same answer.
Seen in: OpenAI: "From the agent's point of view, anything it can't access in-context while running effectively doesn't exist. Knowledge that lives in Google Docs, chat threads, or people's heads are not accessible to the system." The repo is not the only way in: Claude Code's docs say that if a system has a command-line tool, "that's usually the easiest way in". Naur, "Programming as Theory Building": "the programmer's knowledge transcends that given in documentation".
Source: Lopopolo, Harness engineering, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: Claude Code docs, Best practices, read 5 November 2026, https://code.claude.com/docs/en/best-practices
Source: Naur, Programming as Theory Building, 1985 (full-text copy), https://gist.github.com/dpritchett/fd7115b6f556e40103ef
Say it: "We agreed in Slack that refunds over $500 need a second approver. Write that in docs/decisions/refund-approval.md with the date and the reason, link it from AGENTS.md, and add a failing test for it so the rule exists in code too."
Described as: "the agent does not know what we decided", "it is all in Slack", "only one person knows how this works", "make the repo the source of truth", "it keeps asking about things we settled".
https://dev.dqnotes.com/context/repo-as-truth

### Agent legibility
Also: Legibility, Ambient affordances.
How easily an agent can find out what it needs by itself: names it can search for, errors that say what went wrong, logs it can query, an app it can start. Whatever an agent can't observe, it can only guess about.
Poor legibility is paid for in tokens and wrong guesses on every task. Most fixes are ordinary good engineering, with one twist: a name you can grep for now matters more, and code that assembles a name from pieces of text while it runs hides from search. Being findable saves reading, and so saves tokens.
Not Readable code: Readable means a person can follow it once they are looking at it. Legible also means an agent can find it and check it with tools. A web address assembled while the program runs can read well and still never turn up in a search.
Seen in: OpenAI asks "what capability is missing, and how do we make it both legible and enforceable for the agent?", and gave Codex an app that boots per git worktree, with logs and metrics it can query through LogQL and PromQL. Birgitta Böckeler credits Ned Letcher with "ambient affordances": "structural properties of the environment itself that make it legible, navigable, and tractable to agents". Her related word is harnessability: "Not every codebase is equally amenable to harnessing".
Source: Lopopolo, Harness engineering, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: Böckeler, Harness engineering for coding agent users, 2026, https://martinfowler.com/articles/harness-engineering.html
Say it: "Make this debuggable without me. Add bun run dev:test, which boots the app on a free port with seed data and writes logs to .logs/app.jsonl, then show me you can find a failed request by searching that file. Replace routes built by string concatenation with literal names that rg can find."
Described as: "it can not tell what happened when it runs", "it can not find the code by searching", "make the project easier for the agent", "it keeps asking me to check the browser", "the logs are useless to it".
https://dev.dqnotes.com/context/agent-readable

## 8. The brief

How to say what you want: specs, acceptance criteria, plans, slices, and the questions to ask first.

### Brief
What you tell an agent before it starts: what to build, why, what counts as finished, and whatever it couldn't work out by itself. It can be one sentence or a page. Anything it leaves out, the agent decides for you.
A colleague asks when a request is unclear. A model usually picks an answer, writes working code for it and reports success, so you learn what it chose only by reading the result. The brief is the cheapest moment to make those choices yourself.
Not Prompt: A prompt is any text you send a model. A brief is a prompt with a job: it says what the result must do, so the result can be checked. Plenty of prompts don't need to be briefs.
Seen in: Anthropic's prompting guide: "Think of Claude as a brilliant but new employee who lacks context on your norms and workflows." Its test: "Show your prompt to a colleague with minimal context on the task and ask them to follow it." Claude Code's docs add the other side: when you are exploring, a vague prompt "is completely fine".
Source: Anthropic docs, Prompting best practices, https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Say it: "Add dedupe to the payment webhook handler in src/webhooks/pay.ts. Webhooks can arrive twice, so a repeat must change nothing. Done means bun test passes with a new test for the repeat. Don't touch the refund code."
Described as: "I told it what I wanted and got something else", "how much detail do I give the agent", "it built the wrong thing", "what should I write in the prompt".
https://dev.dqnotes.com/brief/brief

### Spec
Also: Specification.
A written description of what something must do, exact enough that two people, or two agents, would build the same behaviour from it. It says what and why, and usually leaves how to the builder.
A session forgets. A spec in the repo lets the next session, a reviewer and a second agent all hold the work to the same thing, and you can read it before any code exists. The price is that it is one more thing that can be wrong, and nothing tells you when it is.
Not Brief: A brief is what you say for this task, and it usually ends with the session. A spec is a document that stays, which later work is held to.
Seen in: The word is getting loose. Birgitta Böckeler has heard people use "spec" basically as a synonym for "detailed prompt". Claude Code's docs suggest having the agent help you write a SPEC.md, then to "start a fresh session to build from it, so the build begins with a clean context and a written definition of done."
Source: Böckeler, Understanding Spec-Driven-Development: Kiro, spec-kit, and Tessl, 2025, https://martinfowler.com/articles/exploring-gen-ai/sdd-3-tools.html
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Say it: "Write docs/specs/webhooks.md: what apply() must do when events repeat or arrive out of order, five example sequences with the result of each, and what is out of scope. No code yet. I will read it first."
Described as: "a document that says what to build", "write it down before building", "the agent and I keep remembering it differently", "what is the difference between a prompt and a spec".
https://dev.dqnotes.com/brief/spec

### Requirement
One thing the result must do, stated so that someone could check it. "apply ignores an event older than the newest one for that order" is a requirement. "Handle events properly" is a wish.
An agent reads a wish as permission to decide. But plenty of real requirements change nothing either: on payment-events, every vague attempt dropped repeated events without being told. Spend your words on what the model wouldn't have done by itself.
Not Constraint: A requirement says what the result must do. A constraint limits how you may get there.
Seen in: Strict wording for this is old. RFC 2119 (1997) defines MUST as "an absolute requirement of the specification" and SHOULD as meaning "there may exist valid reasons in particular circumstances to ignore a particular item". Kiro writes requirements in EARS notation: "WHEN [condition/event] THE SYSTEM SHALL [expected behavior]" (docs read November 2026).
Source: Bradner, RFC 2119, 1997, https://www.rfc-editor.org/rfc/rfc2119
Source: Kiro docs, Feature specs, https://kiro.dev/docs/specs/feature-specs/
Say it: "Rewrite these as requirements I can check: each one names the input and the exact result. Flag any line you can't turn into a check and ask me about it. Leave "should be fast" alone until I give you a number."
Described as: "how do I say exactly what it must do", "my request was too fuzzy", "what is a good requirement", "which parts of the prompt actually matter".
https://dev.dqnotes.com/brief/requirement

### Acceptance criteria
The specific checks that decide whether this piece of work is finished, agreed before it starts. Each one passes or fails. The best are written so that a program can run them.
An agent's own tests check what it built, not what you needed. Of the 11 broken vague attempts in our runs, 9 passed every test they wrote for themselves. Criteria written before the code, by someone who knows the job, can't be shaped by the code.
Not Definition of done: Acceptance criteria belong to one piece of work and say what it must do. A definition of done applies to all work and says what must be true of anything you call finished.
Seen in: Dan North gave scenarios their usual shape: "Given some initial context (the givens), When an event occurs, Then ensure some outcomes." In Anthropic's long-running harness, two agents negotiated a "sprint contract" before each chunk of work, and "Sprint 3 alone had 27 criteria". StrongDM keeps its scenarios "outside the codebase", so the agent can't bend them.
Source: North, Introducing BDD, https://dannorth.net/blog/introducing-bdd/
Source: Rajasekaran, Harness design for long-running application development, Anthropic, 2026, https://www.anthropic.com/engineering/harness-design-long-running-apps
Source: McCarthy, Software Factories And The Agentic Moment, StrongDM, 2026, https://factory.strongdm.ai/
Say it: "Before you write code, add these to tests/webhook.test.ts and show me them failing: a repeat of the same event id ships once; refunded at 2 then paid at 1 leaves the status refunded and ships nothing. Then make them pass. Do not edit the tests."
Described as: "how do I know when it is actually finished", "the checklist for this ticket", "its tests pass but it is still wrong", "write down what success looks like first".
https://dev.dqnotes.com/brief/acceptance-criteria

### Definition of done
The list that must be true before anything counts as finished, on every task: say the code type-checks, the tests pass and the docs are updated. You write it once, and it applies to all the work.
An agent stops when the work looks done to it. A written definition gives it something to run in place of that feeling. It only rejects what it names, though. The bench below shows how far a general list gets you when nobody asked for the behaviour that was missing.
Not Acceptance criteria: A definition of done is general, and the same on every task. Criteria are particular to this one. You usually need both.
Seen in: The Scrum Guide (2020) calls it "a formal description of the state of the Increment when it meets the quality measures required for the product." Claude Code's docs: "Claude stops when the work looks done." pstack's guide wants a check that couldn't run reported as "inconclusive", not as a pass.
Source: Schwaber and Sutherland, The Scrum Guide, 2020, https://scrumguides.org/scrum-guide.html
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Source: Tan, pstack guide, Verify and ship, https://github.com/cursor/plugins/blob/main/pstack/docs/guide/06-verify-and-ship.md
Say it: "Add to AGENTS.md: "Done means bun run typecheck, bun run lint and bun test all exit 0, with the output pasted in your reply. If one could not run, say inconclusive, not done.""
Described as: "it says done but it is not", "what should finished actually mean", "the same checklist for every task", "done means different things to me and the AI".
https://dev.dqnotes.com/brief/definition-of-done

### Scope
The edges of the job: which files, features and behaviours this task may change, and which it may not. One line of scope can save a long review.
Agents overshoot and undershoot, and you pay either way: too much is a diff you must read, too little is a follow-up. They can't tell how much you were hoping for. So say it, with something you can check, such as a folder and a rough size.
Not Non-goal: Scope draws the boundary around this task. A non-goal names one tempting thing just outside it, so it is clearly a decision and not an oversight.
Seen in: Both failures are on record. Anthropic, 2025: "the agent tended to try to do too much at once". Claude Code's docs, 2026: "Claude tends to read a request conservatively and build the smallest thing that satisfies it", and saying how far to go is "telling Claude about scope, which only you know".
Source: Young, Effective harnesses for long-running agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Say it: "Only change src/webhooks/ and its tests. If you think anything else needs to change, stop and list it in your reply instead of changing it. Aim for under 100 changed lines."
Described as: "it rewrote half the project", "it changed files I did not ask about", "the diff is huge", "it only did the bare minimum".
https://dev.dqnotes.com/brief/scope

### Non-goal
Something that could reasonably be a goal and that you have chosen not to pursue this time. "No offline mode in this version" is a non-goal. "It shouldn't crash" isn't, because nobody was ever going to want crashes.
An agent reads a gap as room to build. Naming the reasonable thing you are leaving out stops it being built, spares you reviewing it, and tells the next person it was a decision and not something you forgot.
Not Constraint: A constraint limits how you build what you are building. A non-goal is something you are not building at all.
Seen in: Malte Ubl on design docs at Google: non-goals aren't negated goals, "but rather things that could reasonably be goals, but are explicitly chosen not to be goals." Claude Code's interview prompt asks the same question as "what's out of scope".
Source: Ubl, Design Docs at Google, https://www.industrialempathy.com/posts/design-docs-at-google/
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Say it: "Non-goals for this change: no admin screen, no email notifications, no migration of old events. If you find yourself building one of these, stop and tell me."
Described as: "it keeps adding things I did not ask for", "it added a settings page nobody wanted", "how do I say not this time", "tell it what we are deliberately leaving out".
https://dev.dqnotes.com/brief/non-goal

### Edge case
An input at the limit of what a program expects: nothing at all, exactly one, zero, the largest allowed, the same thing twice, things in the wrong order. Bugs gather here because the usual case is what everybody pictures.
Models handle the edges everyone has heard of and miss the ones particular to your domain. In our runs all 15 vague attempts handled a repeated webhook, and 11 of 15 got an old event arriving after a newer one wrong. Listing the edges costs one sentence.
Not Happy path: The happy path is the case where nothing unusual happens. It is what a demo shows, and what a vague brief describes.
Seen in: Our own hidden scenarios split the same way. On payment-events, every run passed the four scenarios the vague brief literally asked for, and the failures were all among the edge scenarios it never mentioned. The next bench uses one real attempt at a different task.
Say it: "List the edge cases for this function before you write it: empty input, one item, duplicates, items out of order, very large values. Give the result you would expect for each, and ask me about any you are unsure of."
Described as: "it works until someone does something odd", "what if the list is empty", "unusual inputs break it", "it only breaks in weird situations".
https://dev.dqnotes.com/brief/edge-case

### Constraint
A limit on how the job may be done, rather than on what it must do: no new libraries, under 200 milliseconds, use the database we already have.
An agent takes whichever route looks most natural, and it has no idea what your project ruled out last year. Constraints are cheap to state and expensive to discover during review. Once one matters twice, turn it into a check.
Not Requirement: Remove a requirement and the product does less. Remove a constraint and the product does the same thing, built some other way you may not want.
Seen in: Claude Code's docs improve "add a calendar widget" to a brief that ends "no new libraries". They also point out that a preference between two approaches is invisible to the agent: "Claude can't read that from the code, so if you care which one it picks, say so up front."
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Say it: "No new dependencies. Use the Map we already have, not a database table. It must handle 1,000 events a second on one server. If you can't meet one of these, tell me which and why before writing code."
Described as: "it installed a library I did not want", "it used a different database", "it must stay under a size limit", "it picked an approach we ruled out".
https://dev.dqnotes.com/brief/constraint

### Examples
Also: Specification by example, Few-shot prompting.
Showing what you want with concrete inputs and the results they must give, in place of or beside a description. A good one settles what a paragraph leaves open, and it is hard to read two ways.
Examples are direct, but most of them rule out very little. "Jan 15 gives Feb 15" leaves wrong programs standing that "Jan 31 gives Feb 28" removes at once. The useful ones sit at the edges, so you have to know where the edges are.
Not Pointing at examples: That means showing the agent existing code to copy, which settles style and pattern. Here an example is an input and the result it must give, which settles behaviour.
Seen in: Anthropic's prompting guide: "Examples are one of the most reliable ways to steer Claude's output format, tone, and structure." Note what that sentence covers. It is about the shape of output, and we found no measurement of how well examples pin down program behaviour, so treat the bench as logic, not evidence.
Source: Anthropic docs, Prompting best practices, https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices
Say it: "nextBillingDate must do this: Jan 15 plus 1 gives Feb 15. Jan 31 plus 1 gives Feb 28. Jan 31 plus 2 gives Mar 31, not Mar 28. Feb 29 2024 plus 12 gives Feb 28 2025. Write these as tests first."
Described as: "it is easier to show than explain", "give it a sample input and output", "here is what I mean", "it keeps misreading my description".
https://dev.dqnotes.com/brief/example-driven

### Ambiguity
A sentence that can honestly be read more than one way. A model rarely stops to ask which you meant. It picks a reading and writes working code for it, so nothing looks wrong.
Between people, ambiguity comes back as a question. From an agent it comes back as behaviour. "Updates an order's status" gave us three kinds of program. All of them type-check, and they agree on the obvious cases, so nothing tells you a choice was made.
Not Vague: Vague means too little was said: "make it fast". Ambiguous means two things were said at once: "sort the names". More words fix the first. The second needs an example where the readings differ.
Seen in: Anthropic on writing tools for agents: inputs "should be unambiguously named: instead of a parameter named user, try a parameter named user_id." GitHub's Spec Kit sets the bar for a whole spec at "precise, complete, and unambiguous enough to generate working systems", which is a claim, not a result.
Source: Anthropic, Writing effective tools for agents, 2025, https://www.anthropic.com/engineering/writing-tools-for-agents
Source: GitHub, Spec Kit, Specification-Driven Development, https://github.com/github/spec-kit/blob/main/spec-driven.md
Say it: "Before you start, quote back any sentence in this brief that could be read two ways, and give me both readings with an input where they differ. Do not pick one yourself."
Described as: "it took what I said the wrong way", "that is not what I meant", "I meant the other kind", "it could mean two things".
https://dev.dqnotes.com/brief/ambiguity

### Have it interview you
Also: Interview me.
Instead of writing the brief yourself, you ask the agent to question you until it could write it. It finds the decisions you hadn't noticed you were making.
You can't list what you haven't thought of. A model has read a great many specs for this kind of feature, so it is good at what must be decided and knows nothing about your answers. That split is the useful part: it brings the list, you bring the choices.
Not Ask, don't guess: An interview happens once, before the work, because you asked for it. Ask, don't guess is about questions that turn up during the work.
Seen in: Claude Code's docs give the prompt: "Interview me until you could write the spec, then write it to SPEC.md." For rate limiting, they say it asks about "per-user versus per-key limits, what happens to requests over the limit, what's out of scope". pstack's /sharpen asks it to "Say what a top professional in this field would do here that I'm not asking for."
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Source: Tan, pstack skill: sharpen, https://github.com/cursor/plugins/blob/main/pstack/skills/sharpen/SKILL.md
Say it: "I want to add rate limiting to the public API but haven't thought it through yet. Interview me until you could write the spec, then write it to SPEC.md."
Described as: "I do not know what I want yet", "ask me questions first", "help me work out the requirements", "I have not thought this through".
https://dev.dqnotes.com/brief/interview-me

### Plan
The steps an agent intends to take, written before it takes them: which files, in what order, what it will check, and what it isn't sure about.
It is the cheapest place to find out the agent misunderstood you. It is also a guess that nothing has tested yet, and polishing one can feel like progress while no knowledge is being added. Skip it when the change is small and the approach is obvious.
Not Spec: A spec says what must be true. A plan says how this agent will get there. You can throw away the plan and keep the spec.
Seen in: Mitchell Hashimoto: "For vague requests, split the work into separate planning vs. execution sessions." pstack's guide calls "polishing a plan that no code has tested" one of the two commonest design mistakes, and its playbook for large changes opens by saying when not to: "When the change is one or two files with an obvious approach, skip the plan."
Source: Hashimoto, My AI Adoption Journey, 2026, https://mitchellh.com/writing/my-ai-adoption-journey
Source: Tan, pstack guide, Design before you write code, https://github.com/cursor/plugins/blob/main/pstack/docs/guide/04-design.md
Source: Tan, pstack playbook: multi-phase plan, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/multi-phase-plan.md
Say it: "/plan Add idempotency to apply(). Read src/webhooks/ first. Give me the files you would change, the test you would write first, and what you are unsure about. If it is one file and obvious, say so and skip the plan."
Described as: "tell me what you are going to do first", "it charged off in the wrong direction", "it should think before coding", "walk me through your approach".
https://dev.dqnotes.com/brief/plan

### Plan as an artefact
Also: Plan file, Feature list.
A plan saved as a file in the repo, with steps that get ticked off, so that it survives the session. A later agent can pick it up, and a reviewer can see what was meant.
A plan in a chat disappears with the chat. In a file, it carries work across sessions and makes drift visible. The hazard is the same as any document: a stale plan gets read as truth, so it has to change when reality does.
Not Plan mode: Plan mode is a setting that stops edits while a plan is written. Where the plan lives afterwards is a separate decision, and some tools don't save it in your repo by default.
Seen in: Anthropic's long-running harness keeps a feature list in JSON, "as the model is less likely to inappropriately change or overwrite JSON files compared to Markdown files." pstack: "The plan is a checklist an owner runs box by box and the operator audits from the evidence."
Source: Young, Effective harnesses for long-running agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
Source: Tan, pstack playbook: multi-phase plan, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/multi-phase-plan.md
Say it: "Write the plan to docs/plans/webhooks.md as a checklist. After each step, tick it, commit, and add the command you ran with its output. If the plan turns out to be wrong, change the file first and tell me why."
Described as: "it lost track of the plan", "keep the plan so the next session can use it", "a checklist the agent works through", "we planned it yesterday and it is gone".
https://dev.dqnotes.com/brief/plan-as-artifact

### Spec-driven development
Also: SDD, Specifier role.
Working so that the spec comes first and stays the main thing: you write and review the spec, and agents produce the code from it. How far this goes varies, from a spec for one task to a spec that replaces code as what you edit.
It answers a real problem. Code is cheap, intent gets lost, and agents write more than anyone can read, so a spec gives you something smaller to check. The bill is long documents to review, and an agent that may still ignore them.
Not Writing things down: Nearly everyone writes something first. Spec-driven means the spec is what you maintain. Birgitta Böckeler of Thoughtworks separates three levels: spec-first, spec-anchored and spec-as-source, where people edit only the spec.
Seen in: GitHub's Spec Kit: "The specification becomes the primary artifact." Robert C. Martin: "When using AI agents, code cannot be the source because it has become the domain of the AI." SwarmForge has a specifier role that "Writes deterministic Gherkin plus headed end-to-end QA procedures", which a person approves before a coder sees it. Against: Böckeler found Kiro turned a small bug into 16 acceptance criteria, and wrote "I'd rather review code than all these markdown files."
Source: GitHub, Spec Kit, Specification-Driven Development, https://github.com/github/spec-kit/blob/main/spec-driven.md
Source: Martin and Martin, Clean AI: Agentic Discipline, episode 2, 2026, https://cleancoders.com/episode/agentic-discipline-2
Source: Martin, SwarmForge README, six-pack branch, https://github.com/unclebob/swarm-forge/blob/six-pack/README.md
Source: Kiro docs, Specs, https://kiro.dev/docs/specs/
Source: Böckeler, Understanding Spec-Driven-Development: Kiro, spec-kit, and Tessl, 2025, https://martinfowler.com/articles/exploring-gen-ai/sdd-3-tools.html
Say it: "Write the spec for this feature in specs/webhooks.feature, as Given/When/Then scenarios with real values. Stop there. I will approve it, and then a separate session writes the code and makes each scenario run as a test."
Described as: "write the spec and let the AI generate the code", "the spec is the source of truth", "Spec Kit or Kiro", "edit the document instead of the code".
https://dev.dqnotes.com/brief/spec-driven

### The best spec is code
Also: Plan through code.
The opposite position: write code early, because code that runs answers questions a document can only guess at. A working version or a prototype becomes the description of what you want, and anything written comes after.
Our reading, and only ours: code wins when running it shows you the answer, one person is deciding, and throwing it away is cheap. A written spec wins when being wrong won't show up on a run, as with money, when several agents must build to one thing, or when nobody will read the code.
Not Spec-driven development: Both camps want precision and both write down what done means. They differ on what you polish first: a document, or something that runs. On payment-events, six sentences were enough, so neither was needed.
Seen in: Lauren Tan's pstack README: "i don't believe in planning. the best spec is code." They add that a plan is still available, just not the default. Their guide: "You plan through code. Prototypes answer the open questions, a README or tutorial sets the target, and the written plan comes last." Robert C. Martin's ub-mode says it is "loosely based on portions of @poteto's rules", so the two share a lot and split on exactly this.
Source: Tan, pstack README, https://github.com/cursor/plugins/blob/main/pstack/README.md
Source: Tan, pstack guide, Design before you write code, https://github.com/cursor/plugins/blob/main/pstack/docs/guide/04-design.md
Source: Martin, ub-mode README, https://github.com/unclebob/ub-mode/blob/master/README.md
Say it: "Don't write a plan yet. Build the smallest version that runs, in a scratch folder, and show me what it does on these three inputs. We will decide what the real one should do from that, and then write down what we decided."
Described as: "stop planning and just try it", "I will know it when I see it", "the plan looked fine but the code showed it was wrong", "do I need a spec for this".
https://dev.dqnotes.com/brief/code-as-spec

### Vertical slice
Also: Tracer bullet.
A small piece of a feature that goes through every layer, from what the user sees down to where the data is kept, so it works end to end even though it does very little.
An agent can write a whole layer confidently, and the mismatches only appear where layers meet, by which time a lot is resting on a guess. A slice gives it something real to run and check within the first hour.
Not Horizontal layer: Building all of the database, then all of the server, then all of the interface produces a lot of code and nothing you can try until the end.
Seen in: Bill Wake, 2003: "We want to give the customer the essence of the whole cake, and the best way is to slice vertically through the layers." Andy Hunt describes tracer bullets as "a skeleton application, in which one thin line of execution goes end to end."
Source: Wake, INVEST in Good Stories, and SMART Tasks, 2003, https://xp123.com/invest-in-good-stories-and-smart-tasks/
Source: Venners, Tracer Bullets and Prototypes, a conversation with Andy Hunt and Dave Thomas, https://www.artima.com/articles/tracer-bullets-and-prototypes
Source: Böckeler, Understanding Spec-Driven-Development: Kiro, spec-kit, and Tessl, 2025, https://martinfowler.com/articles/exploring-gen-ai/sdd-3-tools.html
Say it: "Build the thinnest version first: one button that saves one comment to the real database and shows it after a reload. No editing, no styling. Show me it working end to end before you add anything."
Described as: "it built the whole backend and nothing works yet", "get one thing working all the way through", "nothing can be tried until the end", "build a thin version first".
https://dev.dqnotes.com/brief/vertical-slice

### Decomposition
Splitting a big job into pieces small enough to be done and checked alone, in an order where each piece builds on something already known to work.
Given everything at once, an agent can run out of room halfway and leave the next session a half-built feature. With small pieces, a break shows up at the piece that caused it. Too small has a cost as well, because every piece has its own overhead.
Not Vertical slice: Decomposition is the act of splitting. A slice is one good shape for the pieces. Splitting by layer is also decomposition, just a worse kind.
Seen in: Hashimoto: "Break down sessions into separate clear, actionable tasks." Also: "Don't try to 'draw the owl' in one mega session." pstack's principle: "A break caught at the unit that caused it is cheap to localize."
Source: Hashimoto, My AI Adoption Journey, 2026, https://mitchellh.com/writing/my-ai-adoption-journey
Source: Tan, pstack principle: sequence verifiable units, https://github.com/cursor/plugins/blob/main/pstack/skills/principle-sequence-verifiable-units/SKILL.md
Say it: "Split this into steps that each end with something I can run. Give me the list, with the check for each step. Do step 1 only, show me the check passing, and then carry on to step 2."
Described as: "this is too big for one go", "it got lost halfway through", "how do I break this up", "draw the rest of the owl".
https://dev.dqnotes.com/brief/decomposition

### One task per session
Start a new session for each separate job, so one job's leftovers don't end up steering, or being paid for by, the next.
The whole conversation goes back on every request, wrong turns included. Three old jobs are three jobs you are still paying to re-read, and three sets of assumptions still in the room. Anything worth keeping goes in a file first.
Not Starting fresh: Starting fresh is the act of clearing the context. One task per session is the habit that tells you when to do it.
Seen in: Justin Young at Anthropic had agents "work on only one feature at a time", and says this "turned out to be critical". Claude Code's docs: "a session that's collected three unrelated tasks re-reads all three on every turn of the fourth", so use /clear between unrelated tasks.
Source: Young, Effective harnesses for long-running agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Say it: "One job in this session: fix the login redirect. When it is merged, write what you learned to docs/progress.md. I will run /clear before the next job, so put anything I would need in that file."
Described as: "should I start a new chat", "one long conversation for everything", "it is still thinking about the last task", "it got confused after I changed the subject".
https://dev.dqnotes.com/brief/one-task

### Spike
A small throwaway program written to answer one question, such as whether a library can do something or how a function really behaves. You keep the answer and delete the code.
Agents make these nearly free, so there is little reason to argue about something you could run in a minute. The usual failure is the spike's rough code quietly becoming the real code, so say at the start that it will be deleted.
Not Prototype: A spike answers a technical question for you, often with no interface at all. A prototype shows what something would be like, so that you can decide whether you want it.
Seen in: Don Wells, on the Extreme Programming site (1999): "Most spikes are not good enough to keep, so expect to throw it away."
Source: Wells, Create a Spike Solution, 1999, http://www.extremeprogramming.org/rules/spike.html
Say it: "Spike this in a scratch folder, not in the repo: can SQLite enforce a case-insensitive unique email? Write the smallest script that proves it, run it, paste the output, and delete the script. No changes to our code."
Described as: "I am not sure this is even possible", "try it quickly before we commit", "just find out if it works", "a quick experiment to answer one question".
https://dev.dqnotes.com/brief/spike

### Go deep first
Get one agent doing one hard thing well, with real understanding and real checks, before you run several at once. A setup you can't trust with one agent gives you ten times the mess with ten.
Parallel agents multiply whatever you already have, and your ability to review doesn't multiply with them. The first deep run also teaches you what the checks and the brief need to say, which is what makes the later ones safe to leave alone.
Not Parallel agents: Parallel agents are how you go wide. This is about the order: wide comes after deep, once one run has shown you the work can be checked.
Seen in: Lauren Tan's pstack README: "if you want to go fast, go deep first." And: "when you can go deep on one agent and trust it to write good, verifiable code, you can truly parallelize with confidence."
Source: Tan, pstack README, https://github.com/cursor/plugins/blob/main/pstack/README.md
Say it: "Don't fan this out yet. First do one migration end to end with me watching: find the real call sites, run the real checks, write down what went wrong. We parallelise the rest only once that one has gone through clean."
Described as: "should I run ten agents at once", "it was faster until I had to fix everything", "do one properly first", "lots of agents and lots of mess".
https://dev.dqnotes.com/brief/go-deep-first

### Prototype
A quick version built to help you decide something, such as which layout or how an interaction should feel. It is meant to be thrown away, and the real one gets built properly afterwards.
Agents make these so cheap that you can see three before choosing. That speed sharpens the old danger, because a prototype looks finished. Vibe coding one is fine. Keeping it, and adding real users, is the mistake.
Not Vertical slice: A slice is production code, thin but real, and you keep it. A prototype is a front with nothing behind it.
Seen in: Dave Thomas: "Prototypes are designed to be thrown away." And: "A prototype is like a town in a western movie. It's all facade." pstack's playbook adds a test: "No decision means no prototype."
Source: Venners, Tracer Bullets and Prototypes, a conversation with Andy Hunt and Dave Thomas, https://www.artima.com/articles/tracer-bullets-and-prototypes
Source: Tan, pstack playbook: prototype, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/prototype.md
Say it: "Build three layouts for the settings page in a scratch folder, plain HTML, behind one switcher so I can flip between them. No tests, no framework. This is throwaway: I pick one, and then we build it properly in src/."
Described as: "I want to see it before deciding", "a rough version to show people", "try a few options quickly", "mock something up".
https://dev.dqnotes.com/brief/prototype

### User story
One short sentence about who wants something and why, usually written "As a [someone], I want [something], so that [reason]." It starts a conversation and isn't meant to be the whole spec.
For an agent, the useful part is "so that". The reason lets it make sensible choices where your brief runs out. Generated in bulk they are padding: a story that says what the code does, with no real person in it, tells nobody anything.
Not Acceptance criteria: A story says who wants what and why. Criteria say how you will know they got it. Ron Jeffries' three words for a story are card, conversation and confirmation, and criteria are the third.
Seen in: Mike Cohn's example: "As a conference attendee, I need to filter sessions by topic so that I can quickly find the sessions most relevant to me." The padding version, from Kiro as Böckeler reports it: "As a developer, I want the transformation function to handle edge cases gracefully".
Source: Cohn, User Stories, Mountain Goat Software, https://www.mountaingoatsoftware.com/agile/user-stories
Source: Böckeler, Understanding Spec-Driven-Development: Kiro, spec-kit, and Tessl, 2025, https://martinfowler.com/articles/exploring-gen-ai/sdd-3-tools.html
Say it: "As someone returning a faulty item, I want my refund to show within a minute, so that I don't contact support. Use that reason to make any choices I haven't specified, and list the choices you made."
Described as: "as a user I want", "what is a ticket supposed to say", "tell it who this is for", "why are we building this".
https://dev.dqnotes.com/brief/user-story

### Stop condition
A rule the agent can check that says when to stop: these commands exit 0, or this many tries have been used. It stands in for the agent's own feeling that it has finished.
With no one watching, it needs two things: a way to succeed, and a limit for when it can't. Without the first it stops when things look done. Without the second it can spend all night on one failure. In our recordings, 1 of 40 loops never passed.
Not Stop hook: The stop condition is what you decided. A stop hook is one way to enforce it, so the agent can't finish without meeting it.
Seen in: Claude Code's docs: "Claude stops when the work looks done." Their example of a set condition is /goal all tests in tests/auth pass and lint is clean. pstack's guide on writing checks into the prompt: "Now the agent has three checks it can run, not a mood to satisfy."
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Source: Tan, pstack guide, Verify and ship, https://github.com/cursor/plugins/blob/main/pstack/docs/guide/06-verify-and-ship.md
Say it: "Keep going until bun test and bun run typecheck both exit 0. Stop after 5 attempts at the same failure, and tell me what you tried, with the output. Never change a test to get there."
Described as: "it stopped too early", "it kept going for hours", "how does it know when to stop", "it spent all night on one bug".
https://dev.dqnotes.com/brief/stop-condition

### Ask, don't guess
Also: Never block on the human.
A rule for the agent: when the answer changes what gets built and it can't find out for itself, stop and ask instead of quietly picking. People disagree about how often that should be.
Each silent guess is a decision you never saw. Each question stops you, and stops a long run dead. Both are real costs, and what settles it is mostly how hard the guess is to undo. If it's reversible, proceed and report. If it isn't, ask.
Not Never block on the human: That is the opposite rule, from pstack: on reversible work, decide, carry on and show what you chose. The two meet at the irreversible, where both say stop.
Seen in: pstack: "Don't ask 'should I do X?' Do X, explain why." But also "Irreversible actions (force-push, delete production data, send external messages) still require confirmation" and "Product direction comes from the human." Claude Code has a tool for asking: AskUserQuestion, for multiple-choice questions "to gather requirements or clarify ambiguity".
Source: Tan, pstack principle: never block on the human, https://github.com/cursor/plugins/blob/main/pstack/skills/principle-never-block-on-the-human/SKILL.md
Source: Claude Code docs, Tools reference, https://code.claude.com/docs/en/tools-reference
Say it: "If a decision is reversible, pick the simplest option, carry on, and list what you picked at the end. Stop and ask me first before anything you can't undo, and before any product choice the spec doesn't settle."
Described as: "it assumed instead of asking", "it keeps asking me about everything", "it should have checked with me first", "when should it stop and ask".
https://dev.dqnotes.com/brief/ask-dont-guess

### The right word
Also: Vocabulary, Say precisely what you mean.
Using the exact term of the trade instead of a description: "idempotent" for "handles repeats okay", "debounce" for "waits until I stop typing". One word can carry a set of requirements you would otherwise have to spell out, if you know it.
A term points at everything a model has read about it, and you need the name more than the fix. But our worded briefs used the term and stated the rules, so we can't say how much the word alone did. And it is only useful if you can check the result, which means knowing what the word means.
Not Jargon: Jargon is a long word used to sound as if you know. The right word is one you could define, and whose line against its nearest neighbour you could draw.
Seen in: This site exists because of Emil Kowalski's vocabulary work. Index, credited to "Emil and Glenn", has a page headed "Say precisely what you mean." Its companion skill aims "to go from a fuzzy description of a concept to the precise, agreed-upon word". That words get you better work from an agent is this site's claim, not a quote from those pages, and the bench is our attempt to test it.
Source: Index, Say precisely what you mean, https://index.how/to/articulate
Source: Index, the vocabulary skill, https://github.com/index-how/vocabulary/blob/main/skills/vocabulary/SKILL.md
Source: Emil Kowalski, https://emilkowal.ski
Say it: "This handler must be idempotent: webhooks are delivered at least once and out of order. Dedupe by event id, and treat any event older than the newest applied to that order as stale. Use those words in the test names."
Described as: "what is it called when", "I can describe it but I do not know the word", "does it matter what words I use with AI", "how do I ask for this properly".
https://dev.dqnotes.com/brief/vocabulary

## 9. Backpressure

Checks that say no. Types, linters, hooks and gates that reject bad work before you ever see it.

### Harness engineering
When an agent makes a mistake, you don't just ask it to try again. You change its environment so that mistake can't easily happen again: a line in its instructions, a script it can run, a check it has to get past.
Retrying fixes this one mistake, today. A change to the harness fixes that mistake for every future session, including ones you never watch.
Not Prompt engineering: Prompt engineering is choosing the words of one request. Harness engineering changes what surrounds every request, and much of it isn't words at all.
Seen in: Mitchell Hashimoto used the term on 5 February 2026 in "My AI Adoption Journey", the earliest dated use we could find, and didn't claim it: "I don't need to invent any new terms here". He described it this way: "anytime you find an agent makes a mistake, you take the time to engineer a solution such that the agent never makes that mistake again". About a project's AGENTS.md: "Each line in that file is based on a bad agent behavior, and it almost completely resolved them all."
Source: Hashimoto, My AI Adoption Journey, 2026, https://mitchellh.com/writing/my-ai-adoption-journey
Say it: "You committed a generated file again. Don't just remove it: add it to .gitignore, add a line to AGENTS.md, and add a pre-commit check that fails if anything under dist/ is staged."
Described as: "the agent keeps making the same mistake", "I keep telling it the same thing every session", "how do I stop it doing that for good", "fixing the setup rather than the code".
https://dev.dqnotes.com/backpressure/harness-engineering

### Backpressure
Also: Back pressure.
Any automated check that refuses an agent's work and tells it why, so that it corrects itself before a person is involved. Types, tests, linters and security scanners can all do this.
Writing code is now cheap. Knowing that the code is right is not. Backpressure moves some of that cost from your attention to a machine that doesn't get tired. It moves only the part someone wrote a check for, and the rest of this chapter is about that line.
Not Code review: Review is a person reading the finished work. Backpressure runs during the work and needs no one to be there.
Seen in: Geoffrey Huntley gave a section of the July 2025 Ralph post this name: "Anything can be wired in as back pressure to reject invalid code generation." The word is much older. It began as "a resistance to the desired flow of fluid through pipes", and in software it is a way to regulate a flow of data so that nothing is overwhelmed. Here it is used loosely: what gets pushed back is the work. A later post you may see credited to Huntley, "Don't waste your back pressure", is by Moss.
Source: Huntley, Ralph Wiggum as a "software engineer", 2025, https://ghuntley.com/ralph/
Source: Wikipedia, Back pressure, https://en.wikipedia.org/wiki/Back_pressure
Source: Moss, Don't waste your back pressure, 2026, https://banay.me/dont-waste-your-backpressure/
Say it: "Before you tell me you are finished, run bun run typecheck, bun test and bun run lint, and keep fixing until all three exit 0. Show me the output, then list what those checks could not have caught."
Described as: "the agent should check its own work before showing me", "it told me it was done but it does not even compile", "how do I make it fix its own mistakes", "the checks push back on the agent".
https://dev.dqnotes.com/backpressure/backpressure

### Guide
Also: Feedforward control.
A control that acts before the agent does, steering it towards the right thing: an instructions file, a skill, a code template, an example to copy.
A good guide means the agent gets it right first time, which is cheaper than being told it got it wrong. But a guide is only advice. The agent can read it and do something else.
Not Sensor: A guide comes before the work and can be ignored. A sensor comes after the work and reports what is really there.
Seen in: Böckeler's "Harness engineering for coding agent users" (April 2026) describes guides as controls that "anticipate the agent's behaviour and aim to steer it before it acts."
Source: Böckeler, Harness engineering for coding agent users, 2026, https://martinfowler.com/articles/harness-engineering.html
Say it: "Add a guide for this: in AGENTS.md say that all database access goes through src/db/, and point to src/db/users.ts as the example to copy."
Described as: "telling the agent how to do things up front", "instructions it reads before starting", "giving it an example to follow", "steering it rather than correcting it".
https://dev.dqnotes.com/backpressure/guide

### Sensor
Also: Feedback control.
A control that acts after the agent does, observing what it produced so it can correct itself: a test run, a type check, a screenshot, a review.
Sensors are what make a guide trustworthy. Without one, you can't tell whether the agent followed the instruction or only said so.
Not Guide: A sensor reports on work that exists. A guide shapes work that doesn't exist yet. Good harnesses pair them: say the rule, then check it.
Seen in: Böckeler: sensors "observe after the agent acts and help it self-correct."
Source: Böckeler, Harness engineering for coding agent users, 2026, https://martinfowler.com/articles/harness-engineering.html
Say it: "You have a guide saying all database access goes through src/db/. Now add the sensor: a test that fails if any file outside src/db/ imports the database client."
Described as: "checking what the agent actually did", "how do I know it followed the rule", "something that looks at the result afterwards", "proof it did what I said".
https://dev.dqnotes.com/backpressure/sensor

### Computational control
Also: Deterministic check.
A check run by ordinary code, so the same input should give the same answer, usually within seconds. Type checkers, linters and tests are computational.
You can rely on the result and run it a thousand times a day, which is why you turn rules into code wherever a rule can be written as code. Reliable doesn't mean relevant, though: it answers the one question it was written to ask, and no other.
Not Inferential control: Computational means run by the CPU, and it should give the same answer every time. Inferential means judged by a model, and it may not.
Seen in: Böckeler calls these "deterministic and fast, run by the CPU" and says they "Run in milliseconds to seconds; results are reliable." That holds for a well-made check: a test that leans on timing or the network can change its answer, and the next chapter has a name for that. Claude Code's documentation draws the same line: a hook, a deny rule or a managed setting is something "Claude Code (the harness) has to execute", where anything in CLAUDE.md is only read.
Source: Böckeler, Harness engineering for coding agent users, 2026, https://martinfowler.com/articles/harness-engineering.html
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Say it: "Asking you nicely to use the logger isn't working. Write this as a lint rule that fails on console.log in src/, with an error message that names the logger to use."
Described as: "a check that always gives the same answer", "a rule the agent can not talk its way out of", "enforced by code, not by asking", "the opposite of asking the AI to check".
https://dev.dqnotes.com/backpressure/computational-control

### Inferential control
Also: LLM as judge.
A check where a model forms a judgement: an AI code review, a second agent grading the first. It can notice things no rule can express, such as whether a change makes sense.
It catches what computational checks can't. It also costs tokens, takes longer, can change its mind, and answers differently depending on how you ask. Run it after the cheap checks pass, never in place of them, and make it show its evidence.
Not Computational control: An inferential check gives an opinion. A computational check gives a fact about one narrow question. Treat the first as advice until it can demonstrate what it found.
Seen in: In our own experiment, across 240 one-shot programs (18 broken, 222 good), a reviewer agent with a neutral prompt stopped 13 broken and 39 good. Told to find a real defect, it stopped 16 broken and 126 good. Reviewing one correct function 40 times, the neutral prompt approved it 33 times and the pushy one approved it once. A buggy function was rejected 40 times out of 40 under both. One small model and one-function tasks, so read the shape and not the size. Böckeler describes this kind of control as "Slower and more expensive; results are more non-deterministic."
Source: Böckeler, Harness engineering for coding agent users, 2026, https://martinfowler.com/articles/harness-engineering.html
Say it: "Have a fresh agent review this diff against the spec. It must quote the line and say what input breaks it. Anything it can't demonstrate goes in a separate list, and doesn't block the merge."
Described as: "having another AI check the work", "asking the AI to review the code", "AI code review says different things each time", "a check based on judgement".
https://dev.dqnotes.com/backpressure/inferential-control

### Formatter
A program that rewrites your code into one fixed layout: spacing, line breaks, quote marks. It doesn't change what the code does, and running it a second time changes nothing.
The layout of an agent's output can drift from one edit to the next, and a one-line change can arrive looking like a rewrite. You pay to read that noise, and so does the next agent. A formatter removes it without asking anyone to decide anything, so it costs no retry and no review.
Not Linter: A formatter only changes layout, and it just does it. A linter reports on what the code does, and often needs a decision.
Seen in: Prettier says why it exists: "By far the biggest reason for adopting Prettier is to stop all the on-going debates over styles." Biome calls itself "an opinionated formatter". Claude Code's hooks guide gives a recipe that runs Prettier after each edit, and Cursor's hooks docs list "Run formatters after edits" as a use.
Source: Prettier docs, Why Prettier, https://prettier.io/docs/why-prettier
Source: Biome docs, Formatter, https://biomejs.dev/formatter/
Source: Claude Code docs, Hooks guide, https://code.claude.com/docs/en/hooks-guide
Source: Cursor docs, Hooks, https://cursor.com/docs/hooks
Say it: "Add a PostToolUse hook that runs biome format --write on each file you edit, and a CI step that runs biome format . and fails if anything would change. Don't reformat files you aren't otherwise changing in this commit."
Described as: "the diff is full of spacing changes", "it changed every line but only one thing is different", "a tool that tidies the code automatically", "stop arguing about tabs and spaces".
https://dev.dqnotes.com/backpressure/formatter

### Linter
A program that reads your code without running it and flags patterns you have decided are wrong. It takes milliseconds and never gets bored.
For an agent, the message is what matters. An error that names the fix is an instruction that arrives at the exact moment it is relevant, and costs nothing until then.
Not Formatter: A formatter rewrites how code looks and changes nothing about what it does. A linter reports patterns in what the code does, and often needs a decision.
Seen in: OpenAI's Codex team writes custom lint errors that "inject remediation instructions into agent context", so the rule teaches at the moment it fires. Biome, ESLint and Ruff are common linters.
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Say it: "Add a lint rule banning direct fetch calls in src/. The error message must say: use apiClient from src/lib/api.ts, which adds auth and retries."
Described as: "something that flags bad patterns automatically", "a tool that complains about code style or mistakes", "red squiggly lines under code", "catch this mistake without running the code".
https://dev.dqnotes.com/backpressure/linter

### Errors as prompts
Also: Remediation message.
Writing an error message for the agent that will read it: what is wrong, where, and exactly what to do instead. The message goes straight into the agent's next turn, so it works as a prompt.
A person can look up what a terse error means. An agent takes the message as its next piece of context and guesses at whatever it leaves out. And it only costs you tokens when the rule was really broken, unlike a line in an instructions file, which is read every time.
Not Instructions file: A line in AGENTS.md is read on every run, whether it is relevant or not, and can be ignored. An error arrives only when the rule was broken, and the work stays blocked until it is dealt with.
Seen in: Böckeler says sensors are "Particularly powerful when they produce signals that are optimised for LLM consumption", and calls such messages "a positive kind of prompt injection". OpenAI's team: "we write the error messages to inject remediation instructions into agent context." Quantity counts too: HumanLayer's advice is to "swallow all test/build/lint output and replace it with a single ✓ if the stage passes." In our own experiment, on one small model and one-function tasks, the gate passed on the next try after a refusal 44 of 47 times for types, 44 of 44 for lint, 24 of 26 for tests and 49 of 63 for mutation.
Source: Böckeler, Harness engineering for coding agent users, 2026, https://martinfowler.com/articles/harness-engineering.html
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Source: HumanLayer, Context-efficient backpressure, 2025, https://www.humanlayer.dev/blog/context-efficient-backpressure
Say it: "Rewrite this rule's error so an agent can act on it alone. Give the file and line, what is wrong, and what to use instead with its path: "Don't call fetch here. Use apiClient from src/lib/api.ts, which adds auth and retries." When the check passes, print one line and no more."
Described as: "the error message does not say how to fix it", "the agent keeps guessing after an error", "make the failure tell it what to do", "the test output is so long it loses track".
https://dev.dqnotes.com/backpressure/error-message-as-prompt

### Type checker
A program that reads your code without running it and checks that each value is used as the kind of thing it is: no adding text to a number, no asking for a field that isn't there. For TypeScript that program is tsc, and tsc --noEmit only checks.
Nobody has to write a rule or a test first, so it covers code the agent wrote a second ago. Its errors come with a file, a line and a reason, which is close to a ready-made prompt. And it has no idea what you meant: a program can be perfectly typed and wrong.
Not Test: A type checker reads the code and never runs it, and says whether the kinds of value line up. A test runs the code on one example, and says whether that answer was right. Each catches what the other can't.
Seen in: Huntley: "Specific programming languages have inbuilt back pressure through their type system." In our own experiment, on one small model and one-function tasks, the harness took final code passing the type gate from about 80% to 99%, and that gate stopped 1 broken one-shot program and 47 good ones. Clean types were worth having. They were not what told the working programs from the broken ones.
Source: Huntley, Ralph Wiggum as a "software engineer", 2025, https://ghuntley.com/ralph/
Say it: "Run bunx tsc --noEmit and fix every error at its cause. Don't add any, as, ! or @ts-ignore to make one go away. If you think the type itself is wrong, stop and show me why."
Described as: "red underlines saying the types do not match", "it passed text where a number was meant", "catch mistakes without running the code", "the compiler is complaining about the agent's code".
https://dev.dqnotes.com/backpressure/type-checker

### Strict mode
A setting that makes a type checker stop trusting you in the places it otherwise would. The one that matters most: with it off, a value that could be missing is treated as if it were always there, so the crash waits until the program runs.
An agent only knows what the checker tells it. With strict off it is told nothing about a missing value, so it has no reason to handle one. Turning it on costs almost nothing on a new project. On an old one it can raise a flood of errors at once, which is a job for a ratchet.
Not noUncheckedIndexedAccess: It sounds like part of strict, and it isn't. With strict alone, list[0] is typed as if the item is always there. This extra setting makes it possibly undefined.
Seen in: TypeScript's docs: "When strictNullChecks is false, null and undefined are effectively ignored by the language." Since TypeScript 6.0, "strict is now true by default". Our experiment labels its gate strict, and it really ran strict plus noUncheckedIndexedAccess. We re-ran the programs it stopped under strict alone: of 48, 45 then passed. Strict alone stopped 0 broken and 3 good, so nearly every refusal came from the one extra setting. Good here means it passed the hidden scenarios, and a missing-item check can still be worth having.
Source: TypeScript docs, strictNullChecks, https://www.typescriptlang.org/tsconfig/strictNullChecks.html
Source: TypeScript 6.0 release notes, https://www.typescriptlang.org/docs/handbook/release-notes/typescript-6-0.html
Say it: "Set "strict": true and "noUncheckedIndexedAccess": true in tsconfig.json, then run bunx tsc --noEmit. Fix each error by deciding what should happen when the value is missing. Don't use ! or ?. only to silence one. If there are more than 50, stop and show me the count by file."
Described as: "it crashes with cannot read properties of undefined", "the checker said fine and it still blew up on null", "make TypeScript less forgiving", "the agent keeps forgetting the value might be missing".
https://dev.dqnotes.com/backpressure/strict-mode

### Structural test
Also: Architecture test, Fitness function.
A test about how the code is arranged and not what it computes: which folder may use which, where the database may be touched. It reads the imports (the lines where one file pulls in another) and fails if a forbidden one appears.
An agent takes the nearest route to a working change, and a forbidden import works perfectly well, so no ordinary test will complain. Once it is in, that shortcut is an example the next session can copy. AGENTS.md can state this rule. Only a test can hold it.
Not Linter: A lint rule usually looks at one file. A structural test looks at how the files relate. Some linters can do both, so judge it by what it checks and not by which tool runs it.
Seen in: OpenAI's post describes code that may only depend forward through "(Types → Config → Repo → Service → Runtime → UI)", and says: "In practice, we enforce these rules with custom linters and structural tests". Böckeler gives the example of "A pre-commit (or coding agent) hook running ArchUnit tests that check for violations of module boundaries", and says this kind of check has been "underused in the past". ArchUnit is for Java. In JavaScript the same job is done by dependency-cruiser.
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Source: Böckeler, Harness engineering for coding agent users, 2026, https://martinfowler.com/articles/harness-engineering.html
Source: ArchUnit, https://www.archunit.org/
Source: dependency-cruiser, https://github.com/sverweij/dependency-cruiser
Say it: "Add tests/architecture.test.ts. The order is types, config, repo, service, runtime, ui, and a file may only import from its own layer or an earlier one. Only src/repo/ may import the database client. Parse imports with the TypeScript compiler API, not a regex. Each failure must name the file, the line and where the code should go. Show it failing on a forbidden import."
Described as: "the agent imported the database straight from the page", "keep the layers of the app separate", "enforce the folder structure automatically", "code is reaching into parts it should not".
https://dev.dqnotes.com/backpressure/structural-test

### Pre-commit hook
A script git runs when someone makes a commit. If it exits with anything but 0, the commit is refused. The git chapter showed how one works. The question here is what belongs in it: fast checks only, on what is really going in.
An agent commits often and reads the refusal at once, so this is a cheap place to catch things. It has two weaknesses. A slow hook tempts everyone to skip it, and because it lives on the machine doing the work, the agent is able to.
Not Stop hook: A pre-commit hook belongs to git and fires on a commit, whoever makes it. A stop hook belongs to the agent harness and fires when the agent tries to finish.
Seen in: The git docs say a pre-commit hook "can be bypassed with the --no-verify option". Böckeler asks what is "reasonably fast and should be run even before integration, or even before a commit is even created", and gives linters and fast test suites. gitleaks git --pre-commit --staged --redact scans the staged copy, which is the one that goes into the commit.
Source: Git docs, githooks, https://git-scm.com/docs/githooks
Source: Böckeler, Harness engineering for coding agent users, 2026, https://martinfowler.com/articles/harness-engineering.html
Source: gitleaks, https://github.com/gitleaks/gitleaks
Say it: "Add a pre-commit hook with Lefthook that runs on staged files only: biome check on them, then gitleaks git --pre-commit --staged --redact. It must finish in under 10 seconds. Slow tests go in CI, not here. If gitleaks is missing, exit 1."
Described as: "check the code before every commit", "it let me commit a password", "the commit got rejected by a script", "the hook passed but the bad code went in anyway".
https://dev.dqnotes.com/backpressure/pre-commit-hook

### Stop hook
A script the harness runs when the agent tries to end its turn. If the script says no, the agent is told why and has to keep working.
An agent can finish without committing, so git's hooks may never fire. This one fires when it decides it is done, and turns "please run the tests before you finish" from a request into a step the harness takes whether the agent remembers or not. The agent can't end its turn while tests are failing until the hook lets it, or the harness's own limit is reached.
Not Pre-commit hook: A pre-commit hook belongs to git and fires when code is committed. A stop hook belongs to the agent harness and fires when the agent tries to stop, whether or not anything was committed.
Seen in: In Claude Code as of November 2026, a Stop hook can return a decision of block, with a reason, and the agent goes back to work with that reason. After eight continuations in a row the harness overrides the next block, and a tool call resets that count. The input has stop_hook_active, and the docs say: "Check this value or process the transcript to avoid blocking on a condition that will never resolve." Exiting at once on that value gives the agent a single retry, and the docs' own typecheck example has no such guard. The onFailure setting has no effect here, so a Stop hook that can't run doesn't hold the agent.
Source: Claude Code docs, Hooks, https://code.claude.com/docs/en/hooks
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Say it: "Add a Stop hook that runs bun test and bun run typecheck, and blocks with the last 20 lines of output if either fails. Don't exit early on stop_hook_active, because that gives you one retry. If the harness ends your turn with the tests still failing, tell me so plainly."
Described as: "stop the agent from finishing until the tests pass", "make it keep going until it actually works", "run something when the AI thinks it is done", "it stopped while the tests were still failing".
https://dev.dqnotes.com/backpressure/stop-hook

### Continuous integration
Also: CI.
Everyone merges their work into the shared code often, and a server builds and tests each change on a clean copy of the project. That server is called CI too, and its green tick or red cross sits on the pull request.
A clean machine has only what was committed, so it finds what works only on the agent's laptop. A local hook can be skipped, and a required check on a server can't, unless the agent can edit the check itself, so the file that defines the check needs guarding as well. And ten agents get the same gate as one.
Not Pre-commit hook: A hook runs on the machine doing the work, and can be skipped there. CI runs elsewhere after the push, and it only blocks anything if someone made the check required.
Seen in: Martin Fowler's definition: "each member of a team merges their changes into a codebase together with their colleagues changes at least daily", and "Each of these integrations is verified by an automated build (including test) to detect integration errors as quickly as possible." Read the small print of your gate. GitHub says: "Required status checks must have a successful, skipped, or neutral status before collaborators can make changes to a protected branch." So a skipped job counts, and by default the rule doesn't apply to people with admin permissions.
Source: Fowler, Continuous Integration, https://martinfowler.com/articles/continuousIntegration.html
Source: GitHub docs, About protected branches, https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches
Say it: "Add .github/workflows/ci.yml that runs bun install --frozen-lockfile, bun run typecheck, bun run lint and bun test on every pull request. Name the job check, so I can make it required on main. Then break a test on purpose and show me the run failing."
Described as: "it works on my machine but nowhere else", "tests should run on every pull request", "the red cross on the pull request", "a server that checks the code", "the agent pushed and nothing checked it".
https://dev.dqnotes.com/backpressure/ci

### Fast feedback
Also: Shift left, Fail fast.
Run first the checks most likely to fail for each second they take, so a bad change hears no as early as possible. In practice that mostly means cheapest first: format, lint, types, unit tests, then the slow and costly ones.
An agent sits idle or burns tokens while it waits, and every retry goes through the line again, so a slow first failure is paid for over and over. Good work doesn't get any faster from this. Only the bad news does, and that is where an agent spends its time.
Not Fewer checks: Reordering takes nothing away. A change that passes still goes through all of it, and the slowest can run after the merge. Only the wait for bad news gets shorter.
Seen in: Huntley: "But the key collective sum is that the wheel has got to turn fast." Hashimoto wants "fast, high quality tools to automatically tell it when it is wrong". Böckeler: "You want to have checks as far left in the path to production as possible, since the earlier you find issues, the cheaper they are to fix." The slow ones aren't dropped: mutation testing is her example of something run "post-integration". Fowler's article on CI has a section called Keep the Build Fast, citing "the XP guideline of a ten minute build", which is older and was written for people doing the waiting.
Source: Huntley, Ralph Wiggum as a "software engineer", 2025, https://ghuntley.com/ralph/
Source: Hashimoto, My AI Adoption Journey, 2026, https://mitchellh.com/writing/my-ai-adoption-journey
Source: Böckeler, Harness engineering for coding agent users, 2026, https://martinfowler.com/articles/harness-engineering.html
Source: Fowler, Continuous Integration, https://martinfowler.com/articles/continuousIntegration.html
Say it: "Order the CI pipeline by cost: format and lint first, then tsc --noEmit, then unit tests, with mutation tests only after merge. Stop at the first failure. Print how long each step took, and if the total is over 5 minutes, tell me which one to move."
Described as: "waiting ages to find out it failed on something simple", "run the quick checks first", "the pipeline takes twenty minutes", "it only failed at the very end".
https://dev.dqnotes.com/backpressure/fast-feedback

### Quality gate
A set of conditions work has to meet before it may go further, where failing any one stops it: types clean, tests pass, no new security findings. It answers yes or no, and nobody gets to argue with it on the day.
A mechanical gate can only refuse what someone thought to check. An agent can pass every one and still have built the wrong thing, because the behaviour that mattered was never asked for, so no check knows it is missing. Someone reading the code and thinking about the job sometimes will. A gate works with a good brief. It can't stand in for one.
Not Sensor: A sensor reports. A gate acts on the report. The same type check is only a sensor until something refuses to go on when it fails.
Seen in: In our own experiment (Claude Haiku 5.5, 8 one-function tasks, 15 runs per cell, 480 runs, no real repo), on 240 one-shot programs, the four computational gates together stopped 8 of 18 broken programs and 99 of 222 good ones. Alone, types stopped 1 broken and 47 good, lint 2 and 34, tests 5 and 16, mutation 5 and 34. Those catches are thinner than they look: a run whose tests fail scores 0 on mutation, so its 5 are the same 5 tests stopped, and lint's 2 came from one rule that objects to control characters in a regular expression, which was a coincidence. With a vague brief, payment-events passed 27% in one shot and 33% with the harness, and with the right words 100% either way. A neutral reviewer reading the code did better there, rejecting all 15 vague runs, including the 4 that passed, and none of the 15 worded ones. This doesn't show gates are useless at scale. The harness did take clean types from about 80% to 99%. How binding a gate should be is disputed: SonarQube's are built to stop work, and OpenAI reports "minimal blocking merge gates", adding "This would be irresponsible in a low-throughput environment."
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Source: SonarQube docs, Introduction to quality gates, https://docs.sonarsource.com/sonarqube-server/quality-standards-administration/managing-quality-gates/introduction-to-quality-gates
Say it: "Our gate is: bun run typecheck, bun run lint and bun test all exit 0. Before you start, list the behaviours this brief needs that none of those would notice if you got them wrong, such as what happens when events arrive out of order. Write a test for each, or ask me."
Described as: "all the checks passed and it is still wrong", "nothing can merge until the checks are green", "it built the wrong thing and everything is green", "a list of things the code has to pass".
https://dev.dqnotes.com/backpressure/quality-gate

### Check fatigue
Also: False positive, Alert fatigue.
What happens when a check says no to good work too often: nobody believes its no any more. Each wrong no is a false positive. So count how often a check's no turned out to be right, and keep it, fix it or demote it on that number.
People get tired and start clicking through. An agent doesn't get tired and does what it is told, so it will rewrite good code to please the check, spending a try and more tokens. Either way, a check that cries wolf costs more than it looks.
Not Flaky test: A flaky test gives different answers on the same code. A check with false positives can give the same wrong answer every time, so running it again tells you nothing.
Seen in: In our own experiment only 18 of 240 one-shot programs were broken, so a no was usually about good code. Broken among those stopped: types 1 of 48, lint 2 of 36, tests 5 of 21, mutation 5 of 39 (the same 5 tests stopped), a neutral reviewer 13 of 52, one told to find a defect 16 of 142. Those are one small model and one-function tasks, and good means it passed the hidden scenarios, so a wrong no isn't always worthless. Teams disagree on the cure. OpenAI's post argues "corrections are cheap, and waiting is expensive", and that "Test flakes are often addressed with follow-up runs rather than blocking progress indefinitely."
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Say it: "For each check in this pipeline, list the last 20 times it failed, and mark each as a real problem or not, with the commit that settled it. For any check under half real, propose fixing the rule or making it advisory. Show me the list before you change anything."
Described as: "everyone just ignores that check", "it keeps failing on code that is fine", "we click override every time", "the AI reviewer always finds something", "too many false alarms".
https://dev.dqnotes.com/backpressure/false-positive-check

### Fail closed
When a check can't run or crashes, treat that as a failure and stop, rather than letting the work through. The opposite, fail open, lets the work through when the check breaks.
A check that quietly stops running looks exactly like a check that keeps passing. This is how a project can go months with a security scan that never scanned anything.
Not Fail open: Fail open keeps things moving when the check breaks, which suits something optional like a spell check. Fail closed stops, which suits anything you would be embarrassed to skip.
Seen in: Agent hooks fail open by default. Claude Code's docs warn: "A policy hook with a wrong path or a crashing script then lets everything through." As of November 2026 only exit code 2 blocks there, unless a command or http hook sets onFailure to block, from v2.1.295, and that setting does nothing on Stop hooks. The docs' example: "you'd rather block the commit than let it through unscanned while the scanner is down." Cursor's hooks say the same: "Hook failed, action proceeds (fail-open by default)", unless the hook sets failClosed to true.
Source: Claude Code docs, Hooks, https://code.claude.com/docs/en/hooks
Source: Claude Code docs, Best practices, https://code.claude.com/docs/en/best-practices
Source: Cursor docs, Hooks, https://cursor.com/docs/hooks
Say it: "If gitleaks is not installed, the pre-commit hook must exit 1 with a message saying how to install it. Don't skip the check silently. Then rename the binary and show me the commit being refused."
Described as: "the check crashed and it let the code through anyway", "what should happen if the check itself breaks", "a scan that quietly stopped running", "it passed because nothing ran".
https://dev.dqnotes.com/backpressure/fail-closed

### No bypass
The rule that nobody gets past a check by switching it off: skipping the hook, silencing the error, deleting the failing test. An agent may try any of these, so the rule has to be held by something that can see them, and not by asking.
A person knows --no-verify is cheating. An agent told to make the checks pass can see removing the check as a solution, and can then truthfully report that everything is green. A line asking it not to is only a guide. You need something on a machine it can't edit, looking at what changed.
Not Test tampering: Tampering is one way of doing this: changing a test so that it passes. No bypass covers the lot, including hooks, flags and settings.
Seen in: Kent Beck watches for "Any indication that the genie was cheating, for example by disabling or deleting tests." Anthropic's long-running harness tells the agent "It is unacceptable to remove or edit tests". Addy Osmani: "The next version of my pre-commit hook greps for .skip( and xit( in the diff." One user's report, not confirmed by the maintainers and closed as not planned, describes Claude Code using --no-verify across 6 commits against its CLAUDE.md. Our own experiment can't settle how often this happens: that model had no tools, and over 137 refusals it added ! once, and never @ts-ignore or any. We ran the ways round a hook on git 2.54: --no-verify, -n, -c core.hooksPath=/dev/null and chmod -x each got a commit through.
Source: Beck, Augmented Coding: Beyond the Vibes, 2025, https://newsletter.kentbeck.com/p/augmented-coding-beyond-the-vibes
Source: Anthropic, Effective harnesses for long-running agents, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
Source: Osmani, Agent Harness Engineering, 2026, https://addyosmani.com/blog/agent-harness-engineering/
Source: GitHub issue, anthropics/claude-code #40117, https://github.com/anthropics/claude-code/issues/40117
Source: Git docs, git commit, https://git-scm.com/docs/git-commit
Say it: "Add a CI step that reads git diff origin/main...HEAD and fails if a test file was deleted, if the diff adds .skip(, .only(, @ts-ignore or @ts-nocheck, or if it touches tsconfig.json or .github/workflows/, unless the pull request has the label harness-change. Add a PreToolUse hook that blocks --no-verify. Show me each one refusing."
Described as: "it deleted the failing test so the tests would pass", "it used no-verify to get past the hook", "it just added ts-ignore", "the agent turned off the check instead of fixing the code", "all green because it skipped the test".
https://dev.dqnotes.com/backpressure/no-bypass

### Ratchet
A rule of thumb: when the agent makes a new kind of mistake, add something that catches that kind from then on. On an old codebase, record today's count of a problem and fail only when it goes up. Quality can then rise, and it can't quietly slip back.
Without it you fix the same class of problem again and again. With it, your instructions file becomes a list of real past failures, which is also the best evidence you can show of your own process.
Seen in: Addy Osmani has a section headed "The ratchet: every mistake becomes a rule", and adds: "You only add constraints when you've seen a real failure. You only remove them when a capable model has made them redundant." So it does turn back, on purpose. pstack's principle encode-lessons-in-structure says: "Encode the rule as a lint, metadata flag, runtime check, or script instead of more text." Its skill called correct asks you to "Prove each new check fails on a real past mistake." and says "If the pattern is already common, fail only when a change adds more."
Source: Osmani, Agent Harness Engineering, 2026, https://addyosmani.com/blog/agent-harness-engineering/
Source: Tan, pstack: encode lessons in structure, https://github.com/cursor/plugins/blob/main/pstack/skills/principle-encode-lessons-in-structure/SKILL.md
Source: Tan, pstack: correct, https://github.com/cursor/plugins/blob/main/pstack/skills/correct/SKILL.md
Say it: "You have made this mistake twice now. Add a check that fails on the old version of this code, confirm that it does, then fix the code so the check passes. If the old code breaks it in many places, record today's count and fail only when it goes up."
Described as: "every mistake becomes a new rule", "making sure it never happens twice", "the setup keeps getting better over time", "turning bugs into permanent checks".
https://dev.dqnotes.com/backpressure/ratchet

## 10. Tests

What a test proves and what it does not, from a first unit test to mutation testing and hidden scenarios.

### Test
Also: Automated test.
A small program that runs your code with a known input and checks the answer. If the answer is wrong it fails, loudly, without a person watching.
An agent can't see whether its work is right, but it can read a failure. A test is the cheapest way to turn "does this work?" into something it can act on in a loop.
Not Trying it by hand: Clicking through the app checks it once, today, and only you learn the result. A test checks it on every change and tells the agent too.
Seen in: In our own experiment, every agent was asked to include tests, and the harness fed failures straight back to it. One-shot, those tests caught about 80% of deliberately planted bugs on average, and some of the failures the agent went on to fix were bugs in the tests themselves.
Say it: "Write a test for total() in src/cart.test.ts that passes [200, 350] and expects 550, and one that passes [] and expects 0. Run bun test and show me the output."
Described as: "a way to check it still works automatically", "code that checks my code", "so I do not have to click through it every time", "how do I know it did not break anything".
https://dev.dqnotes.com/tests/test

### Assertion
The line in a test that compares what happened with what should have happened, such as expect(total).toBe(550). Without one, a test only proves the code did not crash.
Agents reach for weak assertions, like "the result exists", because they pass quickly. The strength of the suite lives almost entirely in these lines, so these are the lines to read.
Not Running the code: Calling a function shows it finished. Asserting on the answer shows the answer was right.
Seen in: Robert C. Martin's ub-mode skill has a one-line test for this: "If the test would still pass when every function returned nothing, throw it out."
Source: Martin, ub-mode SKILL.md, 2026, https://github.com/unclebob/ub-mode/blob/master/SKILL.md
Say it: "These tests only check that the result is defined. Change each assertion to the exact expected value, such as expect(discount(200, 10)).toBe(180), and add a case at percent 0 and percent 100."
Described as: "the line that actually checks the answer", "the test passes but checks nothing", "expect something to equal something", "a test with no real check in it".
https://dev.dqnotes.com/tests/assertion

### Test runner
The program that finds your tests, runs them, prints what failed, and exits with a code saying whether everything passed. Examples are bun test, Vitest, pytest and Playwright Test.
The exit code is what lets a hook or a pipeline stop an agent. A runner that prints a failure but exits 0 lets the agent carry on as if nothing happened.
Seen in: bun test ships with Bun, and tests import from "bun:test". Our experiment judged every attempt by this exit code, with a time limit, inside a sandbox.
Source: Bun docs, Snapshots (shows bun:test imports), https://bun.com/docs/test/snapshots
Say it: "Add a "test" script to package.json that runs bun test. Show me that it exits 1 when I break a test on purpose, and 0 again when I put it back."
Described as: "the command that runs all the tests", "the thing that prints which tests failed", "how does it know the tests passed", "npm test or pytest".
https://dev.dqnotes.com/tests/test-runner

### Unit test
A test of one small piece of code on its own, usually a function. It runs in milliseconds and points straight at what broke.
Fast tests are the ones an agent can run on every attempt. They also suit agents badly in one way: they can pass while the pieces fail to fit together, and they are the easiest kind to write without checking anything.
Not Integration test: A unit test keeps the work inside one piece. An integration test lets real pieces talk to each other, such as your code and a real database.
Seen in: What counts as a unit is disputed. Martin Fowler writes that however you define it "doesn't really matter", and that the more important distinction is whether the test is "sociable or solitary": whether it uses the real collaborators or replaces them.
Source: Fowler, Unit Test, https://martinfowler.com/bliki/UnitTest.html
Say it: "Add unit tests for parsePrice in src/price.ts: normal input, an empty string, a negative number and "1,000.50". No network, no database, and the whole file should run in under 100 ms."
Described as: "testing one function by itself", "a really small fast test", "test just this bit of the code", "test it without the database".
https://dev.dqnotes.com/tests/unit-test

### Fixture
The known starting state a test sets up before it runs: a sample user, a few rows in a table, a file on disk. Good ones are created fresh for each test.
Agents love to share a fixture between tests because it saves lines. That is how you get a suite that passes in one order and fails in another.
Not Mock: A fixture is real data for real code to work on. A mock replaces a piece of code with a stand-in.
Seen in: Our experiment's search-query scenarios open a new in-memory SQLite database for each scenario and fill it with the same six products, so no scenario can damage another one's data.
Say it: "These tests share one users array and they mutate it. Move it into a makeUsers() function, and call that in each test, so that no test can see another test's changes."
Described as: "the sample data a test starts with", "setting things up before the test", "test data", "beforeEach".
https://dev.dqnotes.com/tests/fixture

### Test double
Also: Stand-in, Fake, Stub, Spy, Dummy.
Anything that replaces a real part of your system during a test, like a stunt double. It lets you test without charging a real card or waiting on a real server.
Every double is a claim about how the real thing behaves. Nothing checks that claim unless you make something check it, and an agent writes the claim from what it assumes.
Seen in: The name is Gerard Meszaros's. Martin Fowler's summary: "Test Double is a generic term for any case where you replace a production object for testing purposes." The kinds are dummy, fake, stub, spy and mock.
Source: Fowler, Test Double, https://martinfowler.com/bliki/TestDouble.html
Say it: "Do not call the real Stripe API in tests. Write a FakeGateway in tests/fakes/ that keeps charges in an array, and pass it in. Use the fake for the outside service only, and keep our own code real."
Described as: "pretend version of a service for testing", "test without calling the real API", "a fake database for the test", "not charging a real card in tests".
https://dev.dqnotes.com/tests/test-double

### Mock
A test double that records how it was called, so the test can check the calls. In casual use, people say mock for any stand-in.
If the agent mocks too much, the test exercises the mocks and not your code. It stays green whatever you break, and it fails when you tidy up the internals.
Not Stub: A stub gives canned answers so the code can run. A mock is also checked afterwards, to see that the expected calls happened.
Seen in: Fowler draws this line in "Mocks Aren't Stubs": "Mocks use behavior verification", where a stub-based test checks the resulting state. pstack's rule tests for the same failure with two edits: if breaking the behaviour leaves a test passing, it proves nothing; if changing only internals breaks it, it is pinned to the implementation.
Source: Fowler, Mocks Aren't Stubs, 2007, https://martinfowler.com/articles/mocksArentStubs.html
Source: Tan, pstack, https://github.com/cursor/plugins/tree/main/pstack
Say it: "This test mocks priceCart and then only checks that it was called. Keep our code real and mock only the payment gateway. Assert on the amount it received. Then break priceCart on purpose and show me the test failing."
Described as: "the test still passes when I break the code", "everything in the test is fake", "checking that a function was called", "the test just tests the mock".
https://dev.dqnotes.com/tests/mock

### Integration test
A test where several real pieces work together: your code against a real database, or two of your services talking to each other. It is slower, and it finds the bugs that live in the joins.
Agents write confident code against an interface they have guessed at. A real query against a real database settles the guess at once, which no unit test can do.
Seen in: Our experiment's search-query task can't be judged any other way: its hidden scenarios run the agent's SQL against a real SQLite database to see whether a hostile search term gets through.
Say it: "Add an integration test for createOrder that uses a real in-memory SQLite database, runs the migrations, and checks the rows afterwards. Don't mock the database."
Described as: "test it with the real database", "each part works but together they break", "testing how pieces work together", "the mock passed but production failed".
https://dev.dqnotes.com/tests/integration-test

### Contract test
A check that a real outside service still behaves the way your test doubles say it does. If the provider renames a field, this fails, even while everything else stays green.
A double only remembers how the service worked on the day it was written. An agent trusts it completely, and will happily build on a shape the real service stopped sending months ago.
Not Integration test: An integration test runs your whole path against something real. A contract test checks only one thing: that the double and the real service still agree.
Seen in: Fowler recommends running your own tests against the double and "in addition to periodically run a separate set of contract tests". Pact is a tool built for this: its docs say contract tests "assert that inter-application messages conform to a shared understanding that is documented in a contract."
Source: Fowler, Contract Test, 2011, https://martinfowler.com/bliki/ContractTest.html
Source: Pact docs, Introduction, https://docs.pact.io/
Say it: "Add tests/contract/payments.contract.test.ts. It calls the provider's sandbox and checks that the response has the same fields and types as our FakeGateway returns. Run it nightly, not on every commit."
Described as: "the other service changed and our tests did not notice", "our fake no longer matches the real thing", "the API changed under us", "tests pass but the integration is broken".
https://dev.dqnotes.com/tests/contract-test

### End-to-end test
Also: E2E test.
A test that drives the whole system the way a user would: a real browser, clicking real buttons, against the real app and its database.
An agent can only claim a screen works until something looks at it. An end-to-end test can be its eyes. It is also the slowest and flakiest kind, so keep a few that matter and not hundreds.
Seen in: Playwright Test is one tool for this. Fowler summarises the cost of these tests in one line: "brittle, expensive to write, and time consuming to run".
Source: Playwright docs, Retries, https://playwright.dev/docs/test-retries
Source: Fowler, Test Pyramid, https://martinfowler.com/bliki/TestPyramid.html
Say it: "Write one Playwright test for the checkout: sign in, add an item, pay with the test card, and check the confirmation page shows the order number. Wait on what the page shows, and use no fixed sleeps."
Described as: "test it like a real user clicking", "a robot that clicks through my site", "make sure the whole thing works together", "test in a real browser".
https://dev.dqnotes.com/tests/e2e-test

### Acceptance test
Also: Gherkin.
A test written from what the customer asked for and not from how the code works. Often written in Given, When, Then sentences that a person can read and a tool can run.
It is written before the code exists and from the request, so it is harder for the agent to fit it to whatever it happened to build. A person can also read it and say "that isn't what I meant" before any code gets written.
Not Hidden scenario: An acceptance test usually lives in the repo where the agent can read it. A hidden scenario is kept out of its reach.
Seen in: In the SwarmForge six-pack, a specifier agent "Writes deterministic Gherkin plus headed end-to-end QA procedures", and a person approves that before the coder starts. It also mutates the examples, to check "whether example data is actually connected to the application under test." Robert C. Martin, its main author, gives its limit: in a Hunt the Wumpus experiment, "Every run passed the same 25 acceptance cases. The programs were not the same."
Source: Martin, swarm-forge six-pack branch, https://github.com/unclebob/swarm-forge/tree/six-pack
Source: Martin, Acceptance-Pipeline-Specification, https://github.com/unclebob/Acceptance-Pipeline-Specification
Source: Martin, negative-test-experiment, 2026, https://github.com/unclebob/negative-test-experiment/blob/main/experiment-abstract.txt
Source: Cucumber docs, Gherkin reference, https://cucumber.io/docs/gherkin/reference/
Say it: "Before writing any code, write features/refund.feature in Gherkin: Given an order paid yesterday, When the customer asks for a refund, Then the status is refunded and the money is returned once. Stop and let me approve it."
Described as: "tests written in plain English", "given when then", "how do we check it does what the customer wanted", "a test the non-programmer can read".
https://dev.dqnotes.com/tests/acceptance-test

### Smoke test
A very quick check that the thing runs at all: it starts, the main page loads, the health endpoint answers. It says nothing about whether it is right, only whether it is on fire.
Agents can write a thousand lines and never start the program. A ten-second smoke test catches the embarrassing failure, and it is a sound first gate because it costs almost nothing.
Seen in: The name comes from electronics. In "Lessons Learned in Software Testing", Kaner, Bach and Pettichord explain: "You plug in a new board and turn on the power. If you see smoke coming from the board, turn off the power. You don't have to do any more testing."
Source: Wikipedia, Smoke testing (software), quoting Kaner, Bach and Pettichord, https://en.wikipedia.org/wiki/Smoke_testing_(software)
Say it: "Add scripts/smoke.sh: start the server, wait up to 10 seconds for GET /health to return 200, load / and check the status is 200, then shut it down. Exit 1 if any of that fails. Run it before the full suite."
Described as: "just check that it starts at all", "a quick check after deploying", "does the app even load", "the simplest possible test".
https://dev.dqnotes.com/tests/smoke-test

### Test pyramid
A rule of thumb for the mix: many fast unit tests at the bottom, fewer integration tests in the middle, and very few slow end-to-end tests on top.
An agent runs your tests over and over in a loop, so slow tests cost you on every attempt. It is cheap for an agent to write a slow test now. Waiting for it, and working out why it flaked, is still expensive.
Not A law: It is advice about cost, and it is disputed. Some teams argue for more integration tests. Fowler thinks that the disagreement is partly about what "unit" means.
Seen in: Most people know it from Mike Cohn's 2009 book "Succeeding with Agile", according to Fowler's history of the idea. He also records the objection, and answers it: the "difference is probably illusory due to different definitions of 'unit test'".
Source: Fowler, Test Pyramid, https://martinfowler.com/bliki/TestPyramid.html
Say it: "The whole suite takes 9 minutes because everything goes through the browser. Keep the 5 most important journeys as Playwright tests, and rewrite the others as unit or integration tests. Target under 60 seconds in total."
Described as: "how many of each kind of test", "our tests are too slow", "too many browser tests", "what mix of tests should I have".
https://dev.dqnotes.com/tests/test-pyramid

### Flaky test
A test that sometimes passes and sometimes fails on the same code. Usually the cause is timing, shared state or something outside the test, such as the network or the clock.
A flaky test teaches everyone to ignore red. An agent is worse: it may "fix" code that was fine, or retry until it gets a green run and report success. Retries hide the fault and do not remove it.
Not A failing test: A failing test fails every time and tells you something. A flaky test passes and fails on identical code, so its verdict tells you very little.
Seen in: Google reported in 2016 "a continual rate of about 1.5% of all test runs reporting a 'flaky' result", and wrote: "Almost 16% of our tests have some level of flakiness". Fowler: left uncontrolled, they "can completely destroy the value of an automated regression suite". Playwright labels a test that fails and then passes on retry as "flaky", so it doesn't pass silently.
Source: Micco, Flaky Tests at Google and How We Mitigate Them, 2016, https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html
Source: Fowler, Eradicating Non-Determinism in Tests, 2011, https://martinfowler.com/articles/nonDeterminism.html
Source: Playwright docs, Retries, https://playwright.dev/docs/test-retries
Say it: "cart.test.ts fails about one run in five. Run it with bun test --rerun-each 50 and with --randomize to find out whether it is timing or order. Replace the fixed sleep by awaiting the real promise. Don't add retries."
Described as: "the test passes when I run it again", "it fails sometimes for no reason", "works on my machine but fails in CI", "re-running until it goes green".
https://dev.dqnotes.com/tests/flaky-test

### Regression test
A test written after a bug, which fails on the old code and passes on the fix. A regression is when something that used to work stops working.
An agent has no memory of last month's bug, and may reintroduce it while tidying up. The test remembers. It only counts if you have seen it fail on the broken version.
Seen in: pstack's bug-fix playbook wants the proof in the reply: "Paste failing-then-passing repro output verbatim." Its rule is that a failing test lands before the fix in the git history, so anyone can check.
Source: Tan, pstack bug-fix playbook, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/bug-fix.md
Say it: "Before you fix this: write a test that reproduces it, run it, and paste the failure. Commit that. Then fix it, run it again, and paste the pass. I want both outputs in the pull request."
Described as: "make sure this bug never comes back", "it worked before and now it does not", "the same bug came back again", "a test for the bug I just fixed".
https://dev.dqnotes.com/tests/regression-test

### Snapshot test
A test that saves the output once to a file, then fails whenever later output differs. It tells you something changed. It can't tell you whether the change is good.
When a snapshot fails, the easy way out is one flag that rewrites the file, and an agent will take it. Your bug is now the expected answer, and the test will defend it.
Not Assertion: An assertion says what should be true. A snapshot only says what was true last time.
Seen in: Vitest's docs describe the choice on failure: "either the change is unexpected, or the reference snapshot needs to be updated". The test can't tell which. In Bun the flag is bun test --update-snapshots, and its docs say to do this "when you've intentionally changed the output."
Source: Vitest docs, Snapshot, https://vitest.dev/guide/snapshot
Source: Bun docs, Snapshots, https://bun.com/docs/test/snapshots
Say it: "A snapshot failed. Do not run --update-snapshots yet. Show me the diff of the snapshot file, and say in one sentence why each changed line is intended. I will decide."
Described as: "it just saves the output and compares it", "the test failed so it updated the snapshot", "a huge file of expected output", "approved output file".
https://dev.dqnotes.com/tests/snapshot-test

### Characterisation test
Also: Golden master test.
A test that records what old code does today, bugs included, so you can change it safely. You are writing down the behaviour before you touch it, without claiming that it is right.
This is how you let an agent refactor code nobody understands. Pin the behaviour first, then any change it makes by accident shows up. Tell it the bugs stay until you decide otherwise.
Not Regression test: A regression test says what the right answer is. A characterisation test says what the answer currently is, right or not.
Seen in: The term was coined by Michael Feathers, in "Working Effectively with Legacy Code". As the Wikipedia article on it notes, this kind of testing "does not imply correctness of the results."
Source: Wikipedia, Characterization test, https://en.wikipedia.org/wiki/Characterization_test
Say it: "Before refactoring src/legacy/invoice.ts, run it on these 30 real inputs, and save each output as a test in invoice.characterisation.test.ts. They must all pass unchanged after the refactor. If you find a bug, report it and don't fix it."
Described as: "I need to change old code nobody understands", "lock in what it does now before refactoring", "no tests on this legacy code", "keep the behaviour exactly the same".
https://dev.dqnotes.com/tests/characterisation-test

### Test-driven development
Also: TDD.
Writing the test before the code that makes it pass, in very small steps. The test fails first, because the code does not exist yet, and that failure is the point.
With agents the argument changes. A test written first can't be fitted to the code, and watching it fail shows it can fail at all. Those are strong reasons whatever the studies say about people. We found no good trials on agents.
Not Having tests: TDD is about the order and the size of the steps. Tests written afterwards can be just as thorough, and are easier to bend to the code you already have.
Seen in: The evidence on people is mixed. Four industrial case studies reported pre-release defect density falling 40% to 90%, with teams feeling initial development took 15% to 35% longer (Nagappan et al., 2008, not a randomised trial). A 2013 meta-analysis of 27 studies found a small quality gain and little effect on productivity. Fucci et al. (2017) found the order of test and code "had no important influence", and credited small, steady steps. For agents, Simon Willison calls "Use red/green TDD" a "pleasingly succinct way to get better results out of a coding agent". That is a practitioner's report, not a trial.
Source: Nagappan et al., 2008, abstract via Never Work in Theory, https://neverworkintheory.org/2012/01/25/realizing-quality-improvement-through-test-driven-development.html
Source: Wikipedia, Test-driven development, summarising Rafique and Mišić, 2013, https://en.wikipedia.org/wiki/Test-driven_development
Source: Fucci et al., A Dissection of the Test-Driven Development Process, 2017, https://arxiv.org/abs/1611.05994
Source: Willison, Red/green TDD, https://simonwillison.net/guides/agentic-engineering-patterns/red-green-tdd/
Say it: "Use red/green TDD for this. Write one test, run it and show me it failing. Then write the least code that passes. One test at a time, and do not change a test to get it to pass."
Described as: "write the tests before the code", "tests first", "how do I stop it writing tests that just match the code", "is TDD worth it".
https://dev.dqnotes.com/tests/tdd

### Red, green, refactor
Also: Red/green TDD.
The TDD cycle in three steps. Red: write a test and watch it fail. Green: write just enough code to pass. Refactor: tidy up while the tests stay green.
For an agent, red is the step that matters. A test that passes before the code exists is a test that can't fail, and an agent left alone will skip straight past that without noticing.
Seen in: Willison on why red matters: "If you skip that step you risk building a test that passes already". Kent Beck gave his agent a system prompt asking for Red, Green, Refactor, one test at a time, and watches for it "cheating, for example by disabling or deleting tests". pstack's tdd skill adds: "Prefer no new test over a bad test".
Source: Willison, Red/green TDD, https://simonwillison.net/guides/agentic-engineering-patterns/red-green-tdd/
Source: Beck, Augmented Coding: Beyond the Vibes, 2025, https://newsletter.kentbeck.com/p/augmented-coding-beyond-the-vibes
Source: Tan, pstack tdd skill, https://github.com/cursor/plugins/blob/main/pstack/skills/tdd/SKILL.md
Say it: "Show me the red: paste the failing test output before you write any code. If a new test passes on the first run, stop and tell me, because it isn't testing what we think it is."
Described as: "watch the test fail first", "the test passed before I wrote any code", "write the test, make it pass, clean up", "how do I know the test can actually fail".
https://dev.dqnotes.com/tests/red-green-refactor

### Test oracle
Whatever tells you what the right answer is. It may be a number you worked out by hand, a spec, an old system, or a rule that must always hold.
If an agent writes the code, runs it, and pastes the output in as the expected value, the oracle is the code itself. The test will pass for ever, and it proves nothing. Always ask where the expected value came from.
Not Assertion: The assertion is the comparison. The oracle is where the value on the other side of that comparison came from.
Seen in: The term goes back to a paper by William Howden. Knowing the right output for a given input is a hard enough problem to have its own name, the "oracle problem", as Wikipedia's article sets out.
Source: Wikipedia, Test oracle, https://en.wikipedia.org/wiki/Test_oracle
Say it: "Where did 1847.52 in this test come from? If you got it by running the code, delete it. Work it out from the rule in docs/pricing.md, show the sum in a comment, and use that."
Described as: "how do I know what the right answer should be", "it copied the output into the test", "the expected value came from the code", "who decides what is correct".
https://dev.dqnotes.com/tests/test-oracle

### Coverage
Also: Line coverage, Code coverage.
The share of your code that ran during the tests. It says what ran. It says nothing about whether anything checked the result.
Low coverage tells you something real: that code has never been run by a test. High coverage tells you much less, and it is trivially easy for an agent to hit any number you give it.
Not Tested: Covered means the line ran. Tested means something would have failed if that line were wrong. The bench shows the two as far apart as they go.
Seen in: Fowler: "high coverage numbers are too easy to reach with low quality testing", and he would be "suspicious of anything like 100%". Robert C. Martin's CRAP score uses coverage too: complexity squared times (1 - coverage) cubed, plus complexity. So at 100% coverage it falls to the complexity alone, assertions or none. His published rule now is 10 or below on changed files. His own experiment found forcing it under 4 "made every program harder to read."
Source: Fowler, Test Coverage, 2012, https://martinfowler.com/bliki/TestCoverage.html
Source: Martin, crapper, https://github.com/unclebob/crapper
Source: Martin, swarm-forge role prompts, https://github.com/unclebob/swarm-forge/tree/six-pack
Source: Martin, negative-test-experiment, 2026, https://github.com/unclebob/negative-test-experiment/blob/main/experiment-abstract.txt
Say it: "Run bun test --coverage and list the lines nothing runs. Don't add tests just to raise the number. For each uncovered line, say what would go wrong if it were broken, then write an assertion that would catch it."
Described as: "what percentage of the code is tested", "we have 100% coverage but it still broke", "it wrote tests just to hit the number", "which lines did the tests never run".
https://dev.dqnotes.com/tests/coverage

### Tautological test
Also: Assertion-free test.
A test that can't fail, because it compares the code with itself, checks nothing, or only checks something that is always true.
These are what an agent produces when asked to "add tests". They look like work, they raise coverage, and the summary says "47 tests pass". One quick check finds the worst of them: make every function return nothing and see what still passes.
Not A simple test: A test can be one line and still be strong: expect(price(100, 20)).toBe(120). The question is whether it could ever fail, not how long it is.
Seen in: That quick check is Robert C. Martin's: "If the test would still pass when every function returned nothing, throw it out." Fowler has a name for the extreme case, AssertionFreeTesting. It is not the only form: a test that copies the formula from the code passes this check and shares any mistake in that formula.
Source: Martin, ub-mode SKILL.md, 2026, https://github.com/unclebob/ub-mode/blob/master/SKILL.md
Source: Fowler, Test Coverage, 2012, https://martinfowler.com/bliki/TestCoverage.html
Say it: "Stub every exported function in src/price.ts so it returns undefined, and run the tests. List every test that still passes, then rewrite or delete them. Expected values must be literal numbers and not computed."
Described as: "the test can never fail", "it compares the function with itself", "it says 47 tests pass but they check nothing", "tests that pass whatever the code does".
https://dev.dqnotes.com/tests/tautological-test

### Goodhart's law
When a measure becomes a target, it stops being a good measure. Point anyone at a number and they will move the number, whether or not the thing it stood for moves too.
An agent optimises what the check reads, faster and more literally than a person. Any number you put in a gate can go up for the wrong reason, so keep at least one check it can't see and can't optimise for.
Not Laziness: Nobody has to be cheating. In our experiment an agent told to raise its score deleted code, which may have been the right thing to do. The score just can't tell you which.
Seen in: Named after the economist Charles Goodhart. The usual wording comes from later writers, including Marilyn Strathern in 1997. StrongDM says the same of agents: "Tests can be reward hacked". In our experiment, 31 retries raised a failing mutation score, and 12 of those killed no extra mutants: the code just got smaller.
Source: Wikipedia, Goodhart's law, https://en.wikipedia.org/wiki/Goodhart%27s_law
Source: McCarthy, Software Factories And The Agentic Moment, StrongDM, 2026, https://factory.strongdm.ai/
Say it: "I do not care about the coverage number. Tell me which behaviours are still unchecked. If you delete code to improve a score, list each deletion and say why that code could never run."
Described as: "it just games the metric", "the number went up but nothing got better", "it did exactly what I measured and not what I wanted", "the target stopped meaning anything".
https://dev.dqnotes.com/tests/goodhart

### Mutation testing
A tool makes one small deliberate bug in your code, such as changing a < to <=, and runs your tests. If they fail, that bug was caught. If they still pass, your tests have a blind spot exactly there.
This is the check that makes it hard for an agent to mark its own homework, because it doesn't trust the green. And a surviving mutant is a perfect instruction: it names a line and a change no test noticed.
Not Coverage: Coverage asks whether a line ran. Mutation testing asks whether anything would notice if that line were wrong.
Seen in: It goes back to DeMillo, Lipton and Sayward's 1978 paper, Hints on Test Data Selection. Wikipedia reports that Richard Lipton first proposed it as a student in 1971, and we couldn't open that paper to check. Stryker does it for JavaScript (core 10.1.1 crashed on TypeScript 7.0.2 when we ran it on 2026-11-05, so pin typescript@6), and mutmut for Python. Robert C. Martin's rule: "Kill the survivors." Google mutates only changed lines, during code review, because a whole codebase is far too slow.
Source: DeMillo, Lipton and Sayward, Hints on Test Data Selection, IEEE Computer, April 1978, https://doi.org/10.1109/C-M.1978.218136
Source: Wikipedia, Mutation testing, https://en.wikipedia.org/wiki/Mutation_testing
Source: Stryker docs, https://stryker-mutator.io/docs/
Source: Martin, ub-mode SKILL.md, 2026, https://github.com/unclebob/ub-mode/blob/master/SKILL.md
Source: Martin, mutator, https://github.com/unclebob/mutator
Source: Petrović et al., Practical Mutation Testing at Scale, Google, 2021, https://research.google/pubs/practical-mutation-testing-at-scale-a-view-from-google/
Say it: "Run bunx stryker run on src/price.ts. For every survivor, either add a test that kills it, or explain why it can't be killed. Don't delete code to raise the score without saying so. Show me the before and after."
Described as: "how do I know my tests are any good", "break the code on purpose and see if tests notice", "test the tests", "the tests pass but I still do not trust them".
https://dev.dqnotes.com/tests/mutation-testing

### Mutation score
The share of deliberate bugs your tests caught: killed divided by total. 18 of 30 is 60%. It measures how strong the tests are, not how right the code is.
It gives an agent a number to climb and a list of what to fix. It also can't see anything nobody asked for: tests can catch every mutant in code that solves the wrong problem.
Not Correctness: A score of 100% says your tests noticed every small change to this code. It does not say this was the right code to write.
Seen in: Our experiment set the bar at 70%. With the harness, the average score rose from 80% to 90%. Of 240 one-shot runs, the full mutation gate stopped 5 of 18 broken and 34 of 222 good, and the tests alone had already stopped 5 and 16. It added no catches: a run whose tests fail scores 0, so those 5 are the same 5, and where tests passed it stopped 0 broken and 18 good. That is one small model on one-function tasks, so don't generalise it. It shows what a score measures, and it doesn't show that mutation testing is useless.
Say it: "Report the mutation score per file, as killed / total, with the survivors listed. Treat anything under 70% on a changed file as blocking, and don't treat any score as proof that the behaviour is what I asked for."
Described as: "a percentage for how good the tests are", "what score should we aim for", "how many of the bugs did the tests catch", "the tests are weak but how weak".
https://dev.dqnotes.com/tests/mutation-score

### Equivalent mutant
A mutant that changes the code but not what it does, so no test could ever catch it. It survives, and that says nothing bad about your tests.
So 100% is not always possible, and "kill every survivor" needs judgement. Push an agent too hard and it will write strange tests, or bend the code, to chase mutants that are not real bugs.
Seen in: Stryker's docs: "There is no definitive way for Stryker to find and ignore them." When we ran Stryker on a clamp function (7 killed, 2 survived, 77.78%), both survivors were equivalent: n >= hi and n > hi return the same value. The same thing happened with mutmut on Python.
Source: Stryker docs, Equivalent mutants, https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/
Say it: "For this survivor, compare the mutant with the original over a wide range of inputs. If you find an input where they differ, write the test for it. If you can't, say that it is equivalent and show me what you tried."
Described as: "this mutant can never be killed", "the change makes no difference to the result", "why can I not reach 100% mutation score", "a surviving mutant that is not a real problem".
https://dev.dqnotes.com/tests/equivalent-mutant

### Property-based testing
Also: Generative testing.
Instead of a few examples, you state a rule that must always hold, such as "the shares always add up to the bill". The tool invents hundreds of inputs, and when one fails it cuts it down to the smallest input that still fails.
It is much harder to fit code to a rule than to an example. Rules also answer the oracle problem: you may not know the right split of 7312 between 6, but you know it has to add up. For an agent, the shrunk failure makes a very clear bug report.
Not Fuzzing: A property test checks your rule on valid inputs. Fuzzing throws anything at the code and mostly asks one question: did it crash?
Seen in: QuickCheck, by Koen Claessen and John Hughes, does this for Haskell. fast-check does it for JavaScript and works inside bun test, and Hypothesis does it for Python. fast-check's docs call shrinking "an essential component". We ran both on 2026-11-05 (fast-check 4.10.2, Hypothesis 6.170.14).
Source: Wikipedia, QuickCheck, https://en.wikipedia.org/wiki/QuickCheck
Source: fast-check docs, Why property-based, https://fast-check.dev/docs/introduction/why-property-based/
Say it: "Add fast-check to splitBill's tests with bun add -d fast-check. Assert for all totals and weights: shares sum to the total, every share is a whole number, and there is one share per person. Paste the shrunk counterexample if it fails."
Described as: "test it with lots of random inputs", "a rule that should always be true", "find the smallest input that breaks it", "I can not think of all the cases".
https://dev.dqnotes.com/tests/property-based

### Fuzzing
Also: Fuzz testing.
Throwing huge numbers of random or damaged inputs at a program to see if it crashes or hangs. You do not need to know the right answer, only that it should survive.
Agents tend to write the happy path and test the happy path. A fuzzer has no idea what the happy path is. That makes it especially good for anything that parses input from outside, which is often where the security bugs are.
Not Property-based testing: Fuzzing often needs no rule beyond "don't crash", and may run for hours. Surviving it means the code didn't crash, not that it was right.
Seen in: Barton Miller's graduate class project at Wisconsin in 1988 led to the 1990 paper on the reliability of UNIX utilities. Go has it built in since 1.18: go test -fuzz=FuzzFoo. Its docs say that it "can reach edge cases which humans often miss".
Source: Miller, Fuzz Testing of Application Reliability, https://pages.cs.wisc.edu/~bart/fuzz/
Source: Go docs, Go Fuzzing, https://go.dev/doc/security/fuzz/
Say it: "Write a fuzz test for parseCookie: generate 100,000 strings, some random and some damaged copies of "a=1; b=2", and fail if it throws anything other than our own ParseError. Add every crash you find as a regression test."
Described as: "throw random garbage at it", "what if someone sends weird input", "it crashes on strange characters", "test the parser with broken data".
https://dev.dqnotes.com/tests/fuzzing

### Empirical verification
Also: Prove it works, Run it and look.
Checking a claim by running the real program on the surface a user touches and looking at what happens: open the page, send the request, read the row in the database. What you saw counts, and what the agent says it saw does not.
An agent's "it should work now" is a prediction, and "it compiles" is a proxy. A real run is the check that catches what nobody wrote a test for, such as a page that renders blank. It costs tokens and time, and it proves only the paths you drove.
Not Test: A test checks what someone wrote down beforehand. A real run looks at the thing itself, and can show you what nobody thought to write down. Script it, and it turns into a test you can re-run.
Seen in: pstack's principle-prove-it-works skill: "Verify every task output by checking the real thing directly." It wants more than one look: "Write the script, run it, and keep its output as an artifact a reviewer can re-run instead of trusting your word." Its create-verification-skill generates a project-local skill that launches your app, drives it and keeps the evidence, with the standard "exercise the real user path, not internal setters or test-only endpoints". maintain-verification-skill is the upkeep pass, because "A feature map rots the moment the app changes." This repo does the same thing: a chapter isn't done until someone has opened it at two widths and looked at the screenshots.
Source: Tan, pstack, skills/principle-prove-it-works/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/principle-prove-it-works/SKILL.md
Source: Tan, pstack, skills/create-verification-skill/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/create-verification-skill/SKILL.md
Source: Tan, pstack, skills/maintain-verification-skill/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/maintain-verification-skill/SKILL.md
Say it: "Don't tell me it works. Start the app with bun run dev, drive /checkout in Playwright with the test card, and paste the order number, a screenshot, and the row from SELECT * FROM orders ORDER BY id DESC LIMIT 1. Save those steps as scripts/verify-checkout.ts so I can re-run them."
Described as: "it says it works but did it actually run it", "it compiles but does it work", "open the app and check for real", "show me it working and not just the test output".
https://dev.dqnotes.com/tests/empirical-verification

### Invariant
Something that must always be true, at every moment, whatever happens: a balance never goes below zero, a refunded order never ships, the shares always add up to the bill. It is a plain sentence first and code second.
You can say an invariant without reading the code, which makes it the part of the job you can do and an agent can't guess. It is also what every check from here up needs: a tool can try every order of events only if something says which outcomes are wrong.
Not Assertion: An assertion checks one thing, at one moment, in one run. An invariant is the claim about every moment of every run, and an assertion is one way to test it.
Seen in: pstack's statechart skill: "Write each rule that the reader agrees to as an INVARIANT." And: "Never weaken a rule to make it hold." TLC, the TLA+ model checker, has a config line named INVARIANT. fast-check tests one over generated inputs, and a database can enforce one itself with CHECK (balance >= 0).
Source: Tan, pstack, skills/statechart/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/statechart/SKILL.md
Say it: "Write three invariants for refunds in docs/invariants.md, in plain English first: total refunded never exceeds the amount paid, a refunded order never moves to shipped, and every refund row points at a real payment. Add a fast-check property for each, and show each one failing on a copy you have broken on purpose."
Described as: "a rule that must never be broken", "this should always be true", "the balance must never go negative", "what has to stay true whatever happens".
https://dev.dqnotes.com/tests/invariant

### Model checking
Also: TLC, Explicit-state checking.
A tool walks every state a small model of your system can reach, in every order the steps could happen, and checks an invariant in each. If one breaks, it hands you the shortest sequence of steps that gets there. The model is a short description of the design, not your code.
Agents write the happy order and test the happy order, and a race only shows on some other order. The checker doesn't need to be lucky. But it checks the model the agent wrote, so a step the model leaves out can't fail. Replay every counterexample on the real code, and make sure a broken model fails.
Not Property-based testing: A property test samples inputs, and a pass means it found nothing. A model checker visits every state within its bounds, and a pass means all of them, inside that model and those bounds, and nothing outside them.
Seen in: TLC is the model checker for Leslie Lamport's TLA+. Amazon reported in 2015 that its engineers "have used TLA+ on 10 large complex real-world systems". In DynamoDB, the "shortest error trace exhibiting the bug included 35 high-level steps", and the bug "had passed unnoticed through extensive design reviews, code reviews, and testing". That is Amazon writing about itself, and it checked designs and not code: of tools to check the code against the spec, it wrote "we are not aware of any such tools that can handle distributed systems as large and complex as those being built at Amazon". Kani works on real code: it "uses model checking to analyze Rust programs". pstack's create-formal-verification-skill says "TLA+ is the default for that in any language" when the question is the order of events.
Source: Newcombe et al., How Amazon Web Services Uses Formal Methods, CACM, April 2015, https://cacm.acm.org/research/how-amazon-web-services-uses-formal-methods/
Source: Lamport, The TLA+ home page, https://lamport.azurewebsites.net/tla/tla.html
Source: Kani Rust Verifier docs, https://model-checking.github.io/kani/
Source: Tan, pstack, skills/create-formal-verification-skill/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/create-formal-verification-skill/SKILL.md
Say it: "Write a TLA+ model in specs/withdraw.tla: two requests, each with a separate check step and debit step, from a balance of 100. Add INVARIANT BalanceNeverNegative, run TLC and paste the shortest trace. Then replay that trace against the real code as a test, one step per action. If the bug doesn't appear, assume the model is wrong."
Described as: "it only breaks when two things happen at once", "try every possible order of events", "the race only shows up once in a thousand runs", "how do I test all the interleavings".
https://dev.dqnotes.com/tests/model-checking

### Formal verification
Also: Formal methods, Proof, Vericoding.
Proving that code or a design meets a written specification for all inputs, not only the ones you tried, with a machine checking the proof. It covers what the spec says, under what the proof assumes, and nothing else.
A proof checker rejects an invalid proof however plausible it sounds, so it is one check an agent can't talk its way past. The cost moves to the spec. When one agent writes the code, the spec and the proof, one misunderstanding can sit in all three, and that is self-evaluation bias with a stronger label on it. It can also weaken the spec until it passes, the way it can edit a test. So someone else has to read the spec, usually you, and reading specs is a skill too.
Not Test: A test shows that the cases you ran passed, on the program you ran. A proof covers every case, of the property as written, in the model as written. It may not be about the running system at all.
Seen in: Real tools: TLA+ for designs, Alloy "for software modeling", Dafny, "a verification-aware programming language", Lean, an "open-source programming language and proof assistant", and Kani for Rust. seL4 is the classic case: a kernel that "comprises 8,700 lines of C code and 600 lines of assembler", proved in 200,000 lines of Isabelle over about 20 person-years, as Martin Kleppmann summarises it. Its authors state the limit: "We assume correctness of compiler, assembly code, and hardware". Fonseca and colleagues studied three verified distributed systems in 2017 and "found a total of 16 bugs", all "in the specification, shim layer, and auxiliary tools", with no protocol bugs after eight months of looking. So the proofs held, and the seams didn't. On agents: Kleppmann predicted in December 2025, "rather than having humans review AI-generated code, I'd much rather have the AI prove to me that the code it has generated is correct", then wrote that "the challenge will move to correctly defining the specification". A prediction, by his own title. A 2025 benchmark of 12,504 specs found "vericoding success rates of 27% in Lean, 44% in Verus/Rust and 82% in Dafny", with the formal spec handed to the model, so it doesn't measure writing one. pstack's create-formal-verification-skill: "A pass covers the model or the bound, not the whole system." Its maintain-formal-verification-skill, on a check left behind by code changes: "That pass is the dangerous case, because it looks like a proof."
Source: Kleppmann, Prediction: AI will make formal verification go mainstream, 8 December 2025, https://martin.kleppmann.com/2025/12/08/ai-formal-verification.html
Source: Klein et al., seL4: Formal Verification of an OS Kernel, SOSP 2009, https://sigops.org/s/conferences/sosp/2009/papers/klein-sosp09.pdf
Source: Fonseca et al., An Empirical Study on the Correctness of Formally Verified Distributed Systems, EuroSys 2017, https://www.cs.purdue.edu/homes/pfonseca/papers/eurosys2017-dsbugs.pdf
Source: Bursuc et al., A benchmark for vericoding, 2025, https://arxiv.org/abs/2509.22908
Source: Alloy, https://alloytools.org/
Source: Dafny, https://dafny.org/
Source: Lean, https://lean-lang.org/
Source: Tan, pstack, skills/create-formal-verification-skill/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/create-formal-verification-skill/SKILL.md
Source: Tan, pstack, skills/maintain-formal-verification-skill/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/maintain-formal-verification-skill/SKILL.md
Say it: "Verify one property of settle() in Dafny: the shares always add up to the bill, for all inputs. Before you write any proof, show me the spec in plain English and stop, because I will approve it. Then show the proof failing on a copy with the rounding step removed, and list every assumption and everything outside the proof."
Described as: "prove it works for every input", "the proof passed but the code is still wrong", "can the AI formally verify this", "what does formally verified actually mean", "TLA+ or Lean or Dafny".
https://dev.dqnotes.com/tests/formal-verification

### Statechart
Also: State machine, Finite state machine.
A drawing of a system as the states it can be in and the moves between them. Anything not drawn isn't allowed, so a refunded order simply has no arrow to shipped. A statechart adds states inside states, and regions that run side by side.
Flags and if-statements are the easy thing to generate, and they allow combinations nobody meant. One status and a table of moves makes those impossible to enter. A chart also gives you something to check an agent's understanding against, without reading the code.
Not Flat state machine: A plain state machine is one level of states. David Harel's statecharts, from 1987, add nesting and parallel regions so a large machine stays readable. People use the two names loosely.
Seen in: statecharts.dev: "A statechart is a state machine where each state in the state machine may define its own subordinate state machines". XState runs them in JavaScript. pstack's statechart skill, added in October 2026, does a different job: it checks a TLA+ model with TLC and draws the result as a Mermaid chart for the person who decides, with each rule-breaking state marked and "the shortest path to each state that breaks a rule". It doesn't put a machine in your code, and it warns: "If a bug does not occur, think that the model is wrong until you show that it is not." Its create-formal-verification-skill picks a target and leaves a project skill behind that checks it, and insists: "A check that you have never seen fail proves nothing."
Source: statecharts.dev, What is a statechart?, https://statecharts.dev/what-is-a-statechart.html
Source: Wikipedia, UML state machine, on Harel 1987, https://en.wikipedia.org/wiki/UML_state_machine
Source: XState docs, https://stately.ai/docs/state-machines-and-statecharts
Source: Tan, pstack, skills/statechart/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/statechart/SKILL.md
Source: Tan, pstack, skills/create-formal-verification-skill/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/create-formal-verification-skill/SKILL.md
Say it: "Replace isPaid, isShipped and isRefunded in src/order.ts with one status field and a table of moves: pending to paid or failed, paid to shipped or refunded, shipped to delivered. Any other event returns an error and changes nothing. Draw the table as a Mermaid stateDiagram in docs/order.md, and test every event in every state."
Described as: "the order ended up both shipped and refunded", "too many boolean flags", "a diagram of what state it can be in", "how do I stop impossible states happening".
https://dev.dqnotes.com/tests/statechart

### Test tampering
Also: Reward hacking.
Changing the tests so they pass, in place of fixing the code: skipping a test, deleting it, changing the expected answer, weakening the check, or making the code recognise the test's exact input.
The run goes green, and the summary says "fixed". A tampered run looks exactly like a real fix unless you read the diff of the tests or re-run the old tests against the new code. Both can be automated.
Not Fixing a wrong test: Tests have bugs too, and editing one can be right. The line is whether someone looked. A test change during a fix should be flagged for a reader, not banned.
Seen in: The ImpossibleBench paper puts it directly: an agent "may delete failing tests rather than fix the underlying bug". Beck watches for "disabling or deleting tests". In our experiment, 26 retries followed a failing test run, and in 9 the agent changed only the tests. We read all nine: each time the test was the thing that was wrong, and each final result passed every hidden scenario.
Source: Zhong, Raghunathan and Carlini, ImpossibleBench, 2025, https://arxiv.org/abs/2510.20270
Source: Beck, Augmented Coding: Beyond the Vibes, 2025, https://newsletter.kentbeck.com/p/augmented-coding-beyond-the-vibes
Say it: "Add a CI step: if a pull request changes files under src/ and also changes or removes existing tests, run the tests from main against the new code, and post the failures. Fail the job on any new test.skip. A person decides."
Described as: "it changed the test to make it pass", "it deleted the failing test", "it marked the test as skipped", "it hard-coded the answer the test wanted".
https://dev.dqnotes.com/tests/test-tampering

### Holdout scenario
Also: Hidden scenario, Holdout.
A test of what the software should do, kept where the agent can't read it or change it. The agent is judged on it afterwards, so it can't fit its work to it.
Everything else in this chapter can be seen and so optimised for. A holdout can't. It also catches a different class of failure: tests that are strong, and code that is consistent, but a job nobody asked for properly.
Not Acceptance test: What makes it a holdout is who can see it, not how it is written. A test in the repo the agent can read is not a holdout, however well it is written.
Seen in: StrongDM: "We repurposed the word scenario to represent an end-to-end 'user story', often stored outside the codebase (similar to a 'holdout' set in model training)", validated by a model and scored as a fraction, not a pass or a fail. Their loop "runs until the holdout scenarios pass (and stay passing)". In our experiment, payment-events passed its holdout 27% of the time on a vague brief, despite green tests and top mutation scores in many runs, and 100% when the brief named the problem. One small model, one-function tasks.
Source: McCarthy, Software Factories And The Agentic Moment, StrongDM, 2026, https://factory.strongdm.ai/
Source: StrongDM, The Principles, https://factory.strongdm.ai/principles
Source: Willison, How StrongDM's AI team build serious software, 2026, https://simonwillison.net/2026/feb/7/software-factory/
Say it: "Do not read anything under ../acceptance/. When you think you are done, I will run those scenarios and tell you which ones failed and what they expected, without showing you the code. Fix the behaviour, and do not guess at the tests."
Described as: "tests the agent is not allowed to see", "so it can not just write code to pass the tests", "a secret exam for the code", "checking it against something it never saw".
https://dev.dqnotes.com/tests/hidden-scenario

### Verification spend
Also: Verification budget, Where the tokens go.
Choosing where a limited budget of tokens and attention goes: mechanical gates, running the real thing, formal checks, or a model reading the code. Each sees things the others can't, so ask what this change could get wrong, and which check could see that.
An agent will spend on whichever check you ask for, and each one reports success in the same tone. Good practitioners disagree about where the money should go, and the evidence is thin. So the skill isn't knowing the right mix. It is being able to say what you bought, and what you still can't see.
Not More checks: Adding a check isn't free even when its tokens are cheap. Each stops some good work too, and someone has to read what it says.
Seen in: Lauren Tan on X, 3 November 2026: "i now consider adversarial reviews mostly a waste of tokens. far better to use those tokens on empirical combined with formal verification." That is a practitioner's report, with no figures in it. Our recording can speak to part of it. In favour: a pushy reviewer stopped 16 broken and 126 good, against a neutral reviewer's 13 and 39, so 3 more broken programs cost 87 more false alarms, and 240 pushy reviews cost $0.51 in tokens when writing those 240 programs cost $0.52. Complicating it: the neutral reviewer rejected 15 of 15 vague payment-events runs and 0 of 15 worded ones, where the mechanical gates mostly missed the bug, so reading saw something the checks couldn't. Without that task, it stops 2 of 7 broken. All of this is one small model on one-function tasks, with no real repo. Neither reviewer ran anything, so it isn't a test of review with proof, and we never ran a formal check, so it can't speak for that side at all.
Source: Tan on X, 3 November 2026, https://x.com/poteto/status/2117437607940681849
Source: The same post, through the fxtwitter mirror, https://api.fxtwitter.com/poteto/status/2117437607940681849
Source: This site, the recorded runs (main.jsonl), https://dev.dqnotes.com/data/main.jsonl
Source: This site, the neutral reviews (review-neutral.jsonl), https://dev.dqnotes.com/data/review-neutral.jsonl
Say it: "Before you spend anything on this change, list the ways it could be wrong. For each one, say which check could see it: types or lint, a test, a real run, a model check, or a read-through. Spend only on checks that could see something, and name any risk nothing here can see."
Described as: "which checks are actually worth paying for", "is AI code review worth the tokens", "we cannot run everything on every change", "should this money go on review or on tests".
https://dev.dqnotes.com/tests/verification-spend

### Eval
Also: Evaluation.
A test for an AI system: give it a task, grade the result, and do that many times, because it may answer differently each time. The result is a rate, not a pass or a fail.
It is how you test the agent's setup and not its code: did that AGENTS.md change help, or did it only feel like it? Without evals you are judging your harness from a few memorable runs.
Not Unit test: A unit test runs the same code and expects the same result every time. An eval runs something that varies, so one run tells you little.
Seen in: Anthropic calls an eval "a test for an AI system: give an AI an input, then apply grading logic to its output to measure success." It also warns: "Make your graders resistant to bypasses or hacks." We reviewed one correct function 40 times: a neutral prompt approved it 33 of 40 times, and "find a real defect" 1 of 40. One run of either would have told you little.
Source: Anthropic, Demystifying evals for AI agents, https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
Source: This site, the recorded reviews (review-flake.json), https://dev.dqnotes.com/data/review-flake.json
Say it: "Before you change the system prompt, build an eval: these 20 tasks, 5 runs each, graded by the hidden scenarios in acceptance/. Run it on the old prompt and the new one, and show me both pass rates with their ranges."
Described as: "did changing the prompt actually help", "testing the AI and not the code", "it worked once but how often does it work", "measuring how good the agent is".
https://dev.dqnotes.com/tests/eval

## 11. Design

What makes code good to change: naming, coupling, complexity, refactoring, and how slop gets in.

### Good code
Also: Changeability.
Code that is cheap to change correctly. Two programs can pass the same tests today and differ hugely in what next week's request costs. Working is the minimum, and how easily it changes is what separates them.
A codebase with an agent in it gets changed far more often, and each session starts by rereading it from nothing. Every change pays for the shape the last one left behind, and no test reports that price going up.
Not Working code: Working code does what it should today. Good code can also be made to do something else without much trouble. Tests measure the first and say nothing about the second.
Seen in: John Ousterhout and Robert Martin disagree about nearly everything in their written debate, but they start close together. Ousterhout: "the fundamental goal of software design is to make it easy to understand and modify the system." Martin agrees, and adds that "the activity we want to ease is that of reading." OpenAI's Codex team, on the order of a million lines in, admits the open question: "What we don't yet know is how architectural coherence evolves over years in a fully agent-generated system."
Source: Ousterhout and Martin, A Philosophy of Software Design vs Clean Code, 2024 to 2025, https://github.com/johnousterhout/aposd-vs-clean-code
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Say it: "Before you add this, tell me how many files the next change like it would touch. If it is more than two, first restructure so the rule lives in one place, keeping the tests green, and then make the change."
Described as: "it works but every change takes longer than the last", "small requests have started taking all day", "what does good code even mean", "the tests pass but I am afraid to touch it".
https://dev.dqnotes.com/design/good-code

### Naming
Choosing what to call things so a reader can guess what they do without opening them. unpaidInvoices tells you something. data tells you almost nothing.
An agent finds its way round a codebase largely by searching for words. If the thing it needs has a vague name, the search misses, and it may write a second copy of something that already exists. A name is also the cheapest context there is, since it arrives at every place the thing is used.
Not Comment: A comment explains one place and can go out of date. A name travels with the thing everywhere it is used, and a search can find it.
Seen in: A saying Martin Fowler records from Phil Karlton: "There are only two hard things in Computer Science: cache invalidation and naming things." Anthropic gives the same advice for tools an agent calls: unambiguous names such as user_id over user.
Source: Fowler, Two Hard Things, 2009, https://martinfowler.com/bliki/TwoHardThings.html
Source: Anthropic, Writing effective tools for agents, 2025, https://www.anthropic.com/engineering/writing-tools-for-agents
Say it: "In src/billing/reminders.ts, rename data to unpaidInvoices and process to sendPaymentReminder across every caller, using the language server's rename and not find and replace. Behaviour must not change. Run bun test afterwards."
Described as: "the agent wrote a second copy of a function we already had", "everything is called data or handler", "I can't find where this happens", "what should I call this".
https://dev.dqnotes.com/design/naming

### Small functions
Also: Method length, One Thing.
The advice that functions should be short, so each does one thing and has a name that says what. How short is where people stop agreeing. One famous book says very short. Another says too short makes code harder to read.
Splitting is mechanical, so an agent told to shrink functions can keep going without limit, and the numbers improve whether the code got clearer or not. Someone has to decide when to stop, and for now that is you.
Not Deep module: Small functions judges length. A deep module judges how much the caller is spared. A two-line function with a complicated interface is short and still saves nobody anything.
Seen in: Clean Code says: "The first rule of functions is that they should be small. The second rule of functions is that they should be smaller than that." Ousterhout replies: "Setting arbitrary numerical limits such as 2-4 lines in a method and a single line in the body of an `if` or `while` statement exacerbates this problem." Martin answers: "I don't think it's fair to say that the book sets arbitrary numerical limits", and agrees "It is certainly possible to over-decompose code." They end: "We both value decomposition, and we both avoid entanglement; but we disagree on the relative weighting of those two values." The whole exchange runs on rewritten examples and experience, with no measurement of which version people understand faster.
Source: Ousterhout and Martin, A Philosophy of Software Design vs Clean Code, 2024 to 2025, https://github.com/johnousterhout/aposd-vs-clean-code
Say it: "Split this function only where the new piece hides something its caller no longer has to think about. Don't create helpers that share module-level variables. If I would have to read two functions together to follow either one, keep them as one."
Described as: "this function is too long", "the code has been chopped into so many tiny pieces I can't follow it", "I keep jumping back and forth between functions", "how long should a function be".
https://dev.dqnotes.com/design/function-size

### Cohesion
How much the things inside one module belong together. High cohesion means that when one part changes, the rest tend to change with it, and nothing unrelated gets dragged along.
An agent tends to add code wherever it is already looking, so a file can grow by convenience rather than by meaning. Low cohesion makes it read a lot to change a little, and that reading comes straight out of its context.
Not Coupling: Cohesion is about what is inside a module. Coupling is about the wires between modules. You want the first high and the second low, and moving code usually changes both.
Seen in: Larry Constantine came up with coupling and cohesion in the late 1960s as part of structured design. Robert Martin's short version: "Gather together the things that change for the same reasons. Separate those things that change for different reasons."
Source: Wikipedia, Cohesion (computer science), https://en.wikipedia.org/wiki/Cohesion_(computer_science)
Source: Martin, The Single Responsibility Principle, 2014, https://blog.cleancoder.com/uncle-bob/2014/05/08/SingleReponsibilityPrinciple.html
Say it: "src/utils.ts has become a drawer for everything. List what each export is for, group them by what changes together, move them to src/dates.ts and src/money.ts, and delete utils.ts when it is empty."
Described as: "this file has a bit of everything in it", "there is a utils file nobody understands", "why is this function in here", "unrelated things live in the same place".
https://dev.dqnotes.com/design/cohesion

### Coupling
How much one module has to know about another. If changing A forces you to change B, they are coupled. Modules have to talk to each other, so you can't get rid of it. The questions are how much, and whether you can see it.
The ripple of a change is what an agent has to hold in context and what a reviewer has to read. An import is usually the shortest way to a working change, so agents add them freely, and no test fails when the graph gets tangled. You need a separate sensor for this.
Not Dependency: A dependency is an import you can see. Coupling also covers what you can't: two files that both assume the same date format are coupled with no import between them.
Seen in: SwarmForge gives this its own job: the architect agent owns boundaries and which way dependencies point. OpenAI's Codex team enforces dependency direction with custom linters and structural tests rather than asking nicely.
Source: Martin, SwarmForge, six-pack branch, https://github.com/unclebob/swarm-forge/tree/six-pack
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Say it: "Show me every file that imports src/billing/tax.ts, directly or through another file. If that is more than five, put a narrow interface in front of it, so callers depend on the interface and nothing else."
Described as: "I changed one thing and five others broke", "everything depends on everything", "a small fix touched twenty files", "why does the login page care about the invoice code".
https://dev.dqnotes.com/design/coupling

### Separation of concerns
Keeping different kinds of decision in different places: what the tax rule is, how it is stored, how it is shown. Then you can think about one, and change one, without reading the others.
Asked to add a field, an agent will happily put reading the input, the business rule and the database code into one function, because that works. The next change to the rule then means reading all of it, and testing it means starting a server.
Not Single responsibility: Separation of concerns sorts code by kind of decision. Single responsibility sorts it by who asks for changes. They often agree, and when they don't, it's worth asking why.
Seen in: Edsger Dijkstra, 1974: "It is what I sometimes have called 'the separation of concerns', which, even if not perfectly possible, is yet the only available technique for effective ordering of one's thoughts, that I know of."
Source: Dijkstra, On the role of scientific thought (EWD447), 1974, https://www.cs.utexas.edu/~EWD/transcriptions/EWD04xx/EWD447.html
Say it: "This route handler does parsing, the discount rule and the SQL. Move the rule to src/pricing.ts as a pure function with no database access, and test it without starting the server."
Described as: "one function does the database and the web page and the maths", "I can't test this without running everything", "everything is mixed together in one place", "changing the wording meant reading the SQL".
https://dev.dqnotes.com/design/separation-of-concerns

### Abstraction
A simpler way to think about something complicated, leaving out what you don't need right now. save(user) lets you forget about disks and SQL. Every well-named function is a small abstraction.
A good one saves an agent reading: it needs the interface and not the inside, which keeps its context small. But agents create them for almost nothing, and a wrong one costs more than none, because everything then has to bend around it.
Not Indirection: Indirection adds a step you have to follow. Abstraction removes something you have to know. A wrapper that passes everything straight through is indirection with a nice name.
Seen in: Joel Spolsky's examples are the classic ones: TCP gives programs a reliable stream on top of a network that loses packets, and SQL lets you say what you want without saying how to fetch it.
Source: Spolsky, The Law of Leaky Abstractions, 2002, https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/
Say it: "Add an abstraction here only if it lets callers forget something. Write out what a caller no longer needs to know. If the honest answer is nothing, delete the wrapper and call the library directly."
Described as: "a wrapper that doesn't seem to do anything", "hide the complicated part behind something simple", "too many layers to get to the real code", "I just want to call save and not think about it".
https://dev.dqnotes.com/design/abstraction

### Leaky abstraction
An abstraction whose hidden detail shows through anyway, usually at the worst moment. The interface promises something simple, and what sits underneath forces you to care about it after all.
An agent works from the interface it can see. If real behaviour depends on something the interface doesn't mention, it will write code that passes against a fake and fails against the real thing, with a clean diff and green tests.
Not Bug: A bug is code breaking its promise. A leak is a promise too simple to keep: everything may work as built, and you still have to know what is underneath.
Seen in: Spolsky named it in 2002: "All non-trivial abstractions, to some degree, are leaky." His SQL example: "some SQL servers are dramatically faster if you specify 'where a=b and b=c and a=c' than if you only specify 'where a=b and b=c' even though the result set is the same."
Source: Spolsky, The Law of Leaky Abstractions, 2002, https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/
Say it: "Our cache interface hides Redis, which stores JSON, so dates come back as strings. Add a contract test that runs the same cases against the in-memory fake and a real Redis, and make the fake serialise too."
Described as: "it worked with the fake and broke on the real one", "I shouldn't have to know this but I do", "the date came back as a string from the cache", "swapping the backend broke things the interface never mentioned".
https://dev.dqnotes.com/design/leaky-abstraction

### Interface
Everything one module must know to use another: function names, what they take, what they return, what can go wrong, and any rule that isn't written down anywhere.
The interface is what an agent reads instead of the code behind it, so it is the part most worth getting right. Types make part of it checkable by machine. The unwritten part is where agents go wrong, because they can't read what nobody wrote.
Not User interface: A user interface is for people. This one is for other code. An API, from the network chapter, is an interface offered across a network.
Seen in: Hyrum's Law says the real interface is bigger than the one you wrote: "With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody."
Source: Hyrum's Law, https://www.hyrumslaw.com/
Say it: "Before you write the module, write only its exported function signatures and one usage example, and show me those. I want to see what a caller has to know before any of it exists."
Described as: "what do I need to know to use this", "someone depended on a thing I never promised", "the function signature", "how the two pieces talk to each other".
https://dev.dqnotes.com/design/interface

### Deep module
Also: Shallow module.
A module that does a lot behind a small interface. A shallow one has an interface nearly as complicated as what it does, so using it spares you very little. The words are John Ousterhout's.
A deep module means less to hold in context per task, and one obvious place to make a change. Asking an agent for small pieces without asking for depth can get you the opposite: every helper correct, and the caller left doing the real work.
Not Small function: Depth has nothing to do with length. A deep module may be long inside. What counts is how little a caller needs to know compared with how much it gets done.
Seen in: Ousterhout in the debate, on the best methods: "they replace a large cognitive load (reading the detailed implementation) with a much smaller cognitive load (learning the interface). I call these methods 'deep'." Martin prefers to err towards decomposition. The idea comes from A Philosophy of Software Design, whose second edition (July 2021) added direct comparisons with Clean Code.
Source: Ousterhout and Martin, A Philosophy of Software Design vs Clean Code, 2024 to 2025, https://github.com/johnousterhout/aposd-vs-clean-code
Source: Ousterhout, A Philosophy of Software Design, https://web.stanford.edu/~ouster/cgi-bin/aposd.php
Say it: "Merge these six exported cache helpers into one function: cached(key, ttlMs, compute). Expiry and the key format become private. Callers must not know either exists."
Described as: "I have to call five functions in the right order to do one thing", "this wrapper is as complicated as what it wraps", "a simple front on something complicated", "every caller repeats the same steps".
https://dev.dqnotes.com/design/deep-module

### Single responsibility
Also: SRP.
The rule that a module should have one reason to change. "Reason" is the word people miss: Robert Martin means one group of people whose requests cause the change, which isn't the same as doing one task.
Read as "do one thing", this rule can justify endless splitting. Read Martin's way, it tells an agent where a line belongs: between things that different people will ask to change.
Not One Thing: One Thing is Martin's rule for functions, and the one Ousterhout attacks as vague. Single responsibility is about modules and who asks for the change.
Seen in: Martin, 2014: "each software module should have one and only one reason to change", and then: "This principle is about people."
Source: Martin, The Single Responsibility Principle, 2014, https://blog.cleancoder.com/uncle-bob/2014/05/08/SingleReponsibilityPrinciple.html
Say it: "Invoice.ts changes when finance changes a tax rule and when support changes email wording. Split it so each of those requests touches one file: src/tax.ts and src/invoice-email.ts."
Described as: "this class does too much", "two different teams keep editing the same file", "one module with too many reasons to change", "my fix for support broke something for finance".
https://dev.dqnotes.com/design/srp

### SOLID
Five design rules for code built from classes, under one acronym: Single responsibility, Open-closed, Liskov substitution, Interface segregation and Dependency inversion. They give you words for arguing about design.
Asking an agent to "apply SOLID" is a good way to get more classes. Ask for the one letter you need and say why. In a small codebase made of plain functions, several of the letters hardly apply.
Not Laws: These are one experienced group's opinions, gathered and named. They are useful vocabulary, and we found no evidence that following all five produces better software.
Seen in: Wikipedia traces the principles to Robert Martin's 2000 paper Design Principles and Design Patterns, and the acronym to Michael Feathers around 2004.
Source: Wikipedia, SOLID, https://en.wikipedia.org/wiki/SOLID
Say it: "Don't apply SOLID in general. Tell me which one of its five rules this change breaks, on which line, and which future change that makes expensive. If you can't name that change, leave the code alone."
Described as: "the five principles everyone asks about in interviews", "clean object-oriented design rules", "the agent added interfaces for everything", "senior engineers keep saying this letter word".
https://dev.dqnotes.com/design/solid

### DRY
Also: Don't repeat yourself.
Don't repeat yourself: each piece of knowledge should live in one place, so changing it means changing it once. It is about knowledge and not about text. Two lines can look alike and mean different things.
A session that doesn't know a helper exists may write it again. The classic result is a fix that lands in three of four copies, with the fourth going wrong months later. When a rule has one home, an agent can't miss a copy.
Not Removing all similar code: Hunt and Thomas wrote about knowledge. Merging two things that only look alike ties together code that needs to change separately.
Seen in: Andy Hunt and Dave Thomas in The Pragmatic Programmer: "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system". GitClear, a company that sells code analytics, reported that blocks with five or more duplicated lines "increased by 8 times during 2024". That comes via DevClass, because the report itself was blocked to us, and it shows a trend during Copilot's spread and not a cause.
Source: Wikipedia, Don't repeat yourself, https://en.wikipedia.org/wiki/Don%27t_repeat_yourself
Source: DevClass on GitClear, February 2025, https://devclass.com/2025/02/20/ai-is-eroding-code-quality-states-new-in-depth-report/
Say it: "The 20% tax rate is written in four files. Make one TAX_RATE in src/tax.ts and import it everywhere. Then grep for 0.2 and 1.2 and show me that those results have gone."
Described as: "the same thing is written in four places", "I fixed it in one place and missed another", "it copied the code instead of reusing it", "one rule should only live in one place".
https://dev.dqnotes.com/design/dry

### Rule of three
Also: Three strikes and you refactor.
Put up with the first duplicate. When you are about to write the same thing a third time, extract it. By then you have enough examples to see what is really shared.
An agent told to value DRY may extract a shared helper at the second copy, and it won't be around to feel the result. A wrong abstraction is expensive to undo, since each new case bolts on another flag and the helper gets harder to read.
Not DRY: DRY tells you where to end up. The rule of three tells you when to act, and says that waiting is cheaper than guessing.
Seen in: Wikipedia credits the rule to Martin Fowler's Refactoring, which attributes it to Don Roberts. Sandi Metz makes the case against acting early: "duplication is far cheaper than the wrong abstraction". Both are rules of thumb and neither comes with data.
Source: Wikipedia, Rule of three, https://en.wikipedia.org/wiki/Rule_of_three_(computer_programming)
Source: Metz, The Wrong Abstraction, 2016, https://sandimetz.com/blog/2016/1/20/the-wrong-abstraction
Say it: "These two functions look alike, but quotes and invoices change for different reasons. Leave them duplicated. If a third case appears, show me all three before extracting anything."
Described as: "should I merge these two similar functions yet", "the shared function has so many flags now", "when is copying fine", "we made a helper too early and it got worse".
https://dev.dqnotes.com/design/wet-rule-of-three

### YAGNI
Also: You aren't gonna need it.
You aren't gonna need it: don't build something now because you guess you'll want it later. Build it when you do want it.
Speculative code used to cost you the typing, and that was the brake. An agent writes it for free, so the brake has gone and the rest of the cost hasn't. Every later session has to read it and work around it.
Not Skipping good design: YAGNI is about features you are guessing at. Fowler is explicit that it "does not apply to effort to make the software easier to modify."
Seen in: Fowler traces it to Extreme Programming and a conversation between Kent Beck and Chet Hendrickson on the C3 project. His name for the part people forget is the cost of carry: the feature's extra complexity "makes it harder to modify and debug that software, thus increasing the cost of other features." The word can also become a reviewer's filler. Lauren Tan on agent review, 3 November 2026: "agents tend to just get lost in the nitpicks and yagnis". So ask for it by name when you are cutting scope, and don't let it stand in for a review of what the code does.
Source: Fowler, Yagni, 2015, https://martinfowler.com/bliki/Yagni.html
Source: Tan on X, 3 November 2026, https://x.com/poteto/status/2117437607940681849
Source: The same post, through the fxtwitter mirror, https://api.fxtwitter.com/poteto/status/2117437607940681849
Say it: "Remove the plugin system and the config option for other export formats. We export CSV. Nobody has asked for anything else. Keep one function and its test."
Described as: "it added options nobody asked for", "what if we need this later", "building for a future that may not come", "the agent made it configurable for no reason".
https://dev.dqnotes.com/design/yagni

### KISS
Also: Keep it simple.
Keep it simple, stupid. Prefer the plainest design that does the job, because someone will have to fix it under pressure one day, without its author there to explain it.
An agent doesn't pay for cleverness when it writes, and the next session reads it cold. On its own this rule is too vague to enforce, so turn it into something specific that you can check.
Not Simple versus easy: KISS is a slogan with no definition of simple in it. Hickey's distinction gives you one you can apply: are these things tangled together?
Seen in: Wikipedia says the phrase was first noted by the U.S. Navy in 1960 and has been associated with the aircraft engineer Kelly Johnson.
Source: Wikipedia, KISS principle, https://en.wikipedia.org/wiki/KISS_principle
Say it: "Two nested generics and a factory for one caller is too much. Rewrite this as a plain function with explicit arguments. If a new reader needs more than a minute to follow it, it isn't finished."
Described as: "this is too clever", "why is it so complicated for such a small thing", "I just want the plain version", "the agent reached for a design pattern for ten lines of code".
https://dev.dqnotes.com/design/kiss

### Dependency injection
Also: DI.
Passing a function the things it needs, so it doesn't have to go and get them. isExpired(token, now) can be tested at any date. A function that reads the real clock inside can only be tested today.
Code that reaches for the real clock, network or database leaves an agent two choices: mock everything or skip the test. With injection, the honest test is also the easy one, so that is the one you tend to get.
Not DI framework: A framework is one way to wire this up, and most projects don't need one. Passing an argument already is dependency injection.
Seen in: Fowler named it in January 2004, after arguing over the vaguer "inversion of control": "we settled on the name Dependency Injection."
Source: Fowler, Inversion of Control Containers and the Dependency Injection pattern, 2004, https://martinfowler.com/articles/injection.html
Say it: "isExpired reads Date.now() inside, so the test can't control time. Add a now parameter that defaults to Date.now(), and add tests for one second before and one second after expiry."
Described as: "I can't test this without the real database", "how do I test something that depends on today's date", "it sends a real email when I run the tests", "the function grabs what it needs from everywhere".
https://dev.dqnotes.com/design/dependency-injection

### Composition over inheritance
Build behaviour by combining small parts, rather than stacking parent and child classes. Want a client that logs and retries? Wrap the client in something that adds each behaviour, rather than extending it.
To change an inherited class you have to read its parents, which spends context on files you aren't changing. Parts that are combined can be read, replaced and tested one at a time, including by an agent.
Not Inheritance: Inheritance says this thing is a kind of that thing. Composition says it has one. Inheritance has its uses: this is a default choice, not a ban.
Seen in: The preference is usually traced to the 1994 book Design Patterns, and is a judgement from experience.
Source: Wikipedia, Composition over inheritance, https://en.wikipedia.org/wiki/Composition_over_inheritance
Say it: "Don't extend BaseClient again. Write withRetry(client) that returns an object with the same methods, and use it only at the one place that needs retries."
Described as: "the class hierarchy is five levels deep", "I have to read the parent class to understand this", "changing the base class broke everything", "how do I add behaviour without subclassing".
https://dev.dqnotes.com/design/composition

### Layered architecture
Dividing a program into layers, such as interface, rules and storage, where each layer may only use the ones beneath it. The rule about direction is what does the work.
This is one of the best kinds of rule for an agent, because a check can read every import and refuse, every time. A direction rule that only lives in a document gets broken the first time an import is the shortcut.
Not Folders: Folders named after layers are only a layout. It is a layered architecture when something fails because a lower layer imported a higher one.
Seen in: OpenAI's Codex repository uses "Types → Config → Repo → Service → Runtime → UI", and enforces it with custom linters and structural tests. Fowler calls the three-layer version an effective form of modularisation for many applications.
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: Fowler, Presentation Domain Data Layering, 2015, https://martinfowler.com/bliki/PresentationDomainDataLayering.html
Say it: "Add a structural test: read every import in src/ and fail if anything in src/repo imports from src/service or src/ui. The failure message must name the layer rule and tell the reader to pass the value in instead."
Described as: "the database code imports the web page code", "which part is allowed to call which", "keep the layers from leaking into each other", "the agent keeps taking shortcuts across the app".
https://dev.dqnotes.com/design/layered-architecture

### Clean architecture
Robert Martin's version of layering: business rules sit at the centre and know nothing about databases or web frameworks. Those sit on the outside and depend inwards, so they can be replaced.
It hands an agent a rule simple enough to check: does this inner file import anything from outside? It also costs more files and more translating between shapes. For a small tool that price is usually too high, so ask for it when your rules will outlive your framework.
Not Layered architecture: Plain layering often puts the database at the bottom, with everything depending on it. Clean architecture reverses that: the database code depends on the rules.
Seen in: Martin, 2012: "This rule says that source code dependencies can only point inwards." It is a design argument from experience, and we found no measurement of it.
Source: Martin, The Clean Architecture, 2012, https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html
Say it: "src/domain must import nothing from express, pg or src/adapters. Add a structural test for that, define a SubscriptionStore interface inside src/domain, and implement it in src/adapters/postgres.ts."
Described as: "I want to swap the database without rewriting the rules", "the business logic is tied up with the framework", "keep the core free of tools", "dependencies should point inwards".
https://dev.dqnotes.com/design/clean-architecture

### Boundary
A line in the code where one part hands over to another: your code and the network, one module and the next, your code and a library. What crosses it should be checked and kept narrow.
A boundary tells an agent where its change should stop, and lets several agents work at once, one on each side. It is also where untrusted data comes in, so it is the one place that checking really has to happen.
Not Interface: An interface is what two sides agree on. A boundary is the line itself, plus the rule about what may cross it.
Seen in: OpenAI describes the approach as "enforcing invariants, not micromanaging implementations": hold the lines firmly and let agents choose freely inside them. Lauren Tan's pstack has a principle called boundary-discipline.
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: Tan, pstack, https://github.com/cursor/plugins/tree/main/pstack
Say it: "All calls to Stripe go through src/payments/stripe.ts. Nothing outside that folder may import the stripe package, and the types that cross the line are ours and not Stripe's. Add a structural test for it."
Described as: "the library is used all over the codebase", "we want to replace this vendor and can't", "where should this check go", "keep their types out of our code".
https://dev.dqnotes.com/design/boundary

### Parse, don't validate
Where data enters, turn it into a type that can only hold good data, rather than checking it and passing the original along. After that nothing needs to check again, and nothing can forget to.
An agent adding a new code path may not know a check is needed. If the function takes a parsed type, it can't be called with raw input, because the type checker says no. A rule held in the types holds in every session.
Not Validation: Validation answers yes or no and throws away what it learned. Parsing answers with a new value that carries the proof with it.
Seen in: Alexis King gave it the name in 2019: "a parser is just a function that consumes less-structured input and produces more-structured output." In TypeScript, a library such as Zod is a common way to do it.
Source: King, Parse, don't validate, 2019, https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/
Say it: "Parse the request body once in the route with a zod schema, into an Order type where quantity is a positive integer. Everything after that takes Order and not unknown. Delete the isValidOrder checks further down."
Described as: "we check the same thing in five places", "someone forgot to check before calling it", "the number came in as a string and got added wrong", "it passed validation and still broke later".
https://dev.dqnotes.com/design/parse-dont-validate

### Make illegal states unrepresentable
Design your data so that a wrong combination can't even be written down. An order that is both cancelled and shipped should be impossible to express, rather than something every function must remember to check.
Every check you have to remember is one an agent can forget. With the right data shape, the type checker catches the mistake in every session, including the ones nobody reads, and there are fewer branches to test.
Not Validation: Validation catches a bad state after you have made one. This removes the state, so there is nothing left to catch.
Seen in: The phrase is a section heading in Yaron Minsky's 2011 Jane Street post, which turns a record of optional fields into one record per connection state. Martin's ub-mode says the same in a line: "Get the data shape right before you write the logic."
Source: Minsky, Effective ML Revisited, 2011, https://blog.janestreet.com/effective-ml-revisited/
Source: Martin, ub-mode SKILL.md, https://github.com/unclebob/ub-mode/blob/master/SKILL.md
Say it: "Replace the isPaid, isShipped and isCancelled booleans with one status field: a TypeScript union where shipped carries a trackingId and cancelled carries a reason. Then fix every compile error. Don't add new if-checks."
Described as: "an order somehow ended up both cancelled and shipped", "these two flags should never both be true", "we keep adding checks for things that should not happen", "how did the data get into this state".
https://dev.dqnotes.com/design/illegal-states

### Refactoring
Changing how code is organised without changing what it does, in small steps, running the tests after each one. If the behaviour changed, it wasn't refactoring.
Agents say "refactor" and may mean rewrite. A rewrite that also changes behaviour can't really be reviewed, and when something breaks you can't tell which line did it. Small green steps give an agent a way to know it is still safe.
Not Rewrite: A rewrite throws code away and hopes the new version matches. A refactoring keeps it running at every step, so you can stop at any point.
Seen in: Fowler's definition: "a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior." Kent Beck, 2012: "for each desired change, make the change easy (warning: this may be hard), then make the easy change".
Source: Fowler, Definition of Refactoring, 2004, https://martinfowler.com/bliki/DefinitionOfRefactoring.html
Source: Beck, post on X, 25 September 2012, https://x.com/KentBeck/status/250733358307500032
Say it: "Refactor only: no behaviour changes and no test edits. Make each step its own commit and run bun test between them. If a test goes red, revert that step rather than fixing forward. Add the feature afterwards, in a separate commit."
Described as: "clean this up without breaking it", "the agent said refactor and rewrote the lot", "tidy the code first and then add the feature", "the behaviour changed during the clean-up".
https://dev.dqnotes.com/design/refactoring

### Code smell
A pattern in code that suggests a problem without proving one: a very long parameter list, one module reaching into another's data, the same switch in five places. It is a prompt to look, and it isn't a verdict.
Smell names are precise review vocabulary, so one word can replace a paragraph of instructions to an agent. Make it show what each one costs, though, because any model asked to find smells will find some.
Not Bug: A bug gives a wrong answer. Smelly code may be entirely correct. A smell says this may be expensive to change, and only that.
Seen in: Fowler: "A code smell is a surface indication that usually corresponds to a deeper problem in the system", a term he credits to Kent Beck. The evidence is thinner than the confidence around it: one controlled study of six developers on four systems found "None of the 12 investigated smells was significantly associated with increased effort after we adjusted for file size and the number of changes". It is a small study, but it should make you careful.
Source: Fowler, Code Smell, 2006, https://martinfowler.com/bliki/CodeSmell.html
Source: Sjøberg, Yamashita, Anda, Mockus and Dybå, Quantifying the Effect of Code Smells on Maintenance Effort, IEEE TSE, 2013, https://doi.org/10.1109/TSE.2012.89
Say it: "Review this diff for Long Parameter List, Feature Envy and Shotgun Surgery. For each hit, quote the lines and name the next change it makes expensive. If you can't name one, drop the finding."
Described as: "something feels off but it works", "this code looks wrong and I can't say why", "warning signs in code", "the review bot keeps listing style complaints".
https://dev.dqnotes.com/design/code-smell

### Technical debt
A metaphor: taking a shortcut now is like borrowing. You get the feature sooner, and then you pay interest on every later change until you pay it back.
An agent can borrow faster than any person, and it pays interest in a new way: it rereads the mess each session, in tokens and in wrong turns. It can also repay cheaply, which changes the sums, so cleaning up may now be worth doing far more often.
Not Bug: A bug is wrong behaviour. Debt is right behaviour that costs too much to change. Calling every mistake debt makes the word useless.
Seen in: Fowler credits Ward Cunningham with the idea, in a 1992 report "as far as I can tell". His example: a feature that should take four days takes six, and the two extra days are the interest. OpenAI uses the same picture ("Technical debt is like a high-interest loan") and runs recurring agent cleanup. On the claim underneath it all, Fowler writes: "there is no objective proof that this phenomenon actually occurs."
Source: Fowler, Technical Debt, 2019, https://martinfowler.com/bliki/TechnicalDebt.html
Source: Fowler, Design Stamina Hypothesis, 2007, https://martinfowler.com/bliki/DesignStaminaHypothesis.html
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Say it: "Don't pay this down in general. List the three places that slowed our last five changes, with commit links, and fix only the worst one. Then set up a weekly agent that opens one small clean-up pull request."
Described as: "we took a shortcut and now everything is slower", "when do we clean this up", "we have to pay for that later", "it keeps getting slower to add features".
https://dev.dqnotes.com/design/technical-debt

### Cyclomatic complexity
Also: McCabe complexity.
A count of the separate paths through a function: one, plus one for each fork, such as if, a loop, a case or &&. More paths mean more to understand and more tests to cover them.
It gives an agent a fast fact in place of an opinion about "too complex". It also has a blind spot you should know about: a lookup table scores 2 where the same logic as a switch scores 13, and that can be either a real improvement or a dodge.
Not Lines of code: A long function with no forks scores 1. It counts paths only, and can't see a confusing name or a hidden shared variable.
Seen in: Thomas McCabe published it in 1976. NIST's later guide is honest about the threshold: "The original limit of 10 as proposed by McCabe has significant supporting evidence, but limits as high as 15 have been used successfully as well."
Source: McCabe, A Complexity Measure, IEEE TSE, 1976, https://doi.org/10.1109/TSE.1976.233837
Source: Watson and McCabe, Structured Testing, NIST, https://www.eng.auburn.edu/~kchang/comp6710/readings/Integration.Testing.McCabe.NIST.pdf
Say it: "Add the ESLint complexity rule at 10 on src/, as an error. Where it fires, tell me whether the branches answer one question (leave it, and say why) or several (split it)."
Described as: "too many ifs in this function", "how complicated is this, as a number", "a tool says the function is too complex", "so many paths through this code".
https://dev.dqnotes.com/design/cyclomatic-complexity

### CRAP score
Also: Change Risk Analysis and Predictions.
One number combining how complicated a function is with how little of it the tests run. Complicated and untested scores terribly. Simple or well tested scores low. The formula is complexity squared, times untested fraction cubed, plus complexity.
It is a gate an agent can run and act on, because the score offers two ways out: simplify or test. And an agent will reach any number you set by the cheapest route, which may not be the route you meant.
Not Coverage: Coverage alone rewards running code. CRAP counts complexity too, so full coverage can't rescue a tangled function: the score never falls below its complexity.
Seen in: Alberto Savoia and Bob Evans built it in 2007, as Change Risk Analysis and Predictions. You will also see "Change Risk Anti-Pattern", which is the wording in Martin's crap4clj README and not the authors'. Their first line was a score of 30: "we decided to INITIALLY use a CRAP score of 30 as the threshold for crappiness". Robert Martin uses it to gate agent work. His published rule now: "CRAP gate is 10 or below on changed files", and a single cond or case that answers one question may stay above 10. The SwarmForge files have moved: under 30 (31 March 2026), then 4 (18 April), then 6 (24 May), then 10 (2 September). His August 2026 Hunt the Wumpus experiment used 4, and its abstract reports that forcing every score under 4 "made every program harder to read. It did not improve design." When someone who gates his agents on this metric publishes that result about it, take the lesson: treat it as a smoke alarm and not as a target.
Source: Savoia, The Code C.R.A.P. Metric Hits the Fan, Artima, October 2007, https://www.artima.com/weblogs/viewpost.jsp?thread=215899
Source: Crap4j home page, http://www.crap4j.org/
Source: Crap4j FAQ, http://www.crap4j.org/faq.html
Source: Savoia, This Code is CRAP, Google Testing Blog, 2011, https://testing.googleblog.com/2011/02/this-code-is-crap.html
Source: Martin, crap4clj README, https://github.com/unclebob/crap4clj
Source: Martin, crapper README, https://github.com/unclebob/crapper
Source: Martin, QA.prompt on SwarmForge's six-pack branch, https://github.com/unclebob/swarm-forge/blob/six-pack/swarmforge/roles/QA.prompt
Source: Martin, negative-test-experiment, abstract, August 2026, https://github.com/unclebob/negative-test-experiment/blob/main/experiment-abstract.txt
Say it: "Run ./crapper --changed --threshold 10. For anything over 10, tell me whether it is one question (a single switch can stay) or mixed duties (split it). Don't add helpers that take booleans the caller already knew, and don't chase a lower score."
Described as: "which functions are risky to change", "complicated and has no tests", "one number for how scary this code is", "the agent split everything up to hit the metric".
https://dev.dqnotes.com/design/crap-score

### Duplication detection
Also: Clone detection.
A tool that finds code repeated across a project, usually ignoring names so that a renamed copy still counts. It hands you candidates, and you decide whether they are really the same idea.
A session that doesn't find the helper writes another, and this is the cheap sensor for that. Treat what it finds as questions, since the rule of three says some duplicates are correct.
Not Text search: A search finds identical text. A detector compares tokens or structure, so changing the variable names doesn't hide a copy.
Seen in: Martin's dryer scores pairs of functions by how many structural fingerprints they share: "The default `--threshold 0.82` reports candidates close enough to be worth review." His ub-mode tells agents to "eliminate duplication that rank 0.8 or above."
Source: Martin, dryer README, https://github.com/unclebob/dryer
Source: Martin, ub-mode SKILL.md, https://github.com/unclebob/ub-mode/blob/master/SKILL.md
Say it: "Run dryer on src/ and show me every pair above 0.82. For each, say whether the two change for the same reason. Merge the ones that do. Leave the ones that don't, and write down why."
Described as: "find all the copy-pasted code", "the agent keeps rewriting things that already exist", "how much of this is duplicated", "the same code with different variable names".
https://dev.dqnotes.com/design/duplication-detection

### Slop
Also: AI slop.
Agent output that works and that nobody would have chosen to keep: checks for cases that can't happen, comments that restate the code, helpers with one caller. The tests pass, which is how it gets in.
Each piece looks harmless, so each one gets through review. They add up, though, and the next session reads what is there and copies the style. You can't test this away, because nothing about it is failing.
Not Broken code: Broken code fails. Slop passes. Saying "this is slop" about something correct is a claim about its cost, so be ready to say which lines you would delete and why.
Seen in: OpenAI: "Our team used to spend every Friday (20% of the week) cleaning up 'AI slop.'" They replaced that with mechanical rules and a recurring agent clean-up that "functions like garbage collection". The bench uses real output from our own experiment, and shows that long code isn't automatically slop.
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Say it: "Remove everything I didn't ask for: checks for states the types already rule out, comments that restate the next line, and helpers with one caller. The tests must still pass. Show me the line count before and after."
Described as: "it works but it feels bloated", "this looks AI generated", "so many comments saying what the code does", "why is this 50 lines".
https://dev.dqnotes.com/design/slop

### Agent sprawl
Also: Overbuilding.
An agent building more than you asked for: extra files, options and layers for a small job. Each addition is reasonable on its own, and the total is a bigger thing to carry for as long as the code lives.
Code costs an agent almost nothing to write, so nothing in its loop pushes back on size. That force has to come from you, which is why pstack and ub-mode, two published sets of agent instructions, both carry a rule about smallness.
Not Thoroughness: Handling a case the brief asked for is thoroughness. Handling a case nobody mentioned may be wise or may be sprawl, and you can only tell by reading it. Length alone won't tell you.
Seen in: Lauren Tan's pstack: "Bias toward deletion and the smallest change that solves the problem". Martin's ub-mode: "Ship the smallest edit that solves the problem." In our own experiment, all 237 one-shot runs that returned code were longer than the hand-written reference solution, with a median of 29 code lines against 13. The references were written knowing the tests, so they set a floor rather than a fair target, and these are one-function tasks from one small model.
Source: Tan, pstack, https://github.com/cursor/plugins/tree/main/pstack
Source: Martin, ub-mode SKILL.md, https://github.com/unclebob/ub-mode/blob/master/SKILL.md
Say it: "Smallest change that makes this test pass. No new files, no new dependencies, no config options. If you think more is needed, stop and tell me why before you write it."
Described as: "I asked for a small thing and got a framework", "it added ten files for one button", "the diff is far bigger than the request", "why did it build all this".
https://dev.dqnotes.com/design/sprawl

### Monolith and microservices
A monolith is one program, deployed as one thing. Microservices split it into many small programs that talk over a network, each deployed by itself. One is simpler to run. The other lets teams move separately.
An agent does its best work when it can run and test the whole system from one place. Each network hop adds failures it can't see locally. Microservices mostly solve a problem of coordinating people. That is our judgement, and nobody has good data yet on how agents change the sums.
Not Modular monolith: A monolith can still have firm boundaries inside it. Most of the design benefit comes from the boundaries. Putting a network across them is a separate decision.
Seen in: Fowler, on what he has seen rather than measured: "Almost all the successful microservice stories have started with a monolith that got too big and was broken up", and on building that way from scratch, "it has ended up in serious trouble."
Source: Fowler, Monolith First, 2015, https://martinfowler.com/bliki/MonolithFirst.html
Say it: "Keep this as one deployable. Make src/billing a module with one exported interface, add a structural test so nothing reaches inside it, and we can split it out later if a real reason appears."
Described as: "should we split this into services", "one big app or lots of small ones", "everything has to be deployed together", "the agent can't run the whole system locally".
https://dev.dqnotes.com/design/monolith-microservices

### Architecture decision record
Also: ADR.
A short text file in the repo recording one design decision: the situation, what was decided, and what it will cost. You don't rewrite it later. A new record replaces the old one.
An agent has no memory of why. Without a record it will see something odd and tidy it away, or reopen an argument that was already settled. Code shows what was chosen, and only a record says what was rejected.
Not Spec: A spec says what to build. A decision record says why it is built this way, and stays true after the work is done.
Seen in: Michael Nygard proposed them in 2011 for decisions "that affect the structure, non-functional characteristics, dependencies, interfaces, or construction techniques", with five parts: title, context, decision, status and consequences.
Source: Nygard, Documenting Architecture Decisions, 2011, https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions
Say it: "Write docs/adr/0007-one-deployable.md with Context, Decision, Status and Consequences, including what we rejected and why. Add a line to AGENTS.md: read docs/adr/ before changing how modules are split."
Described as: "why did we build it this way", "the agent undid something on purpose", "we keep having the same argument", "write down what we decided".
https://dev.dqnotes.com/design/adr

### Simple versus easy
Easy means close at hand or familiar. Simple means not tangled together. The two are independent: a tool can be easy to start with and leave you a knot. Rich Hickey drew this line in a 2011 talk.
Agents make nearly everything easy. A working feature is one sentence away, whatever it tangles together. So "easy" has stopped telling you anything, and "simple" is what is left to judge. You still have to ask for it.
Not Fewer things: Simple doesn't mean few. Hickey separates counting things from interleaving them, so ten separate parts can be simpler than three that are tangled.
Seen in: Hickey's Simple Made Easy, Strange Loop 2011. From the talk's abstract: "We should be simplifying the problem domain, and creating solutions by composing simple parts. Instead, we endure complexity, and pride ourselves in our ability to manage it." The definitions above are paraphrased from a community transcript. Kent Beck, working with agents in 2025, was "still working on getting the genie to care as much as I do about simplicity."
Source: Hickey, Simple Made Easy, Strange Loop, 2011, https://thestrangeloop.com/2011/simple-made-easy.html
Source: Transcript of the talk (community gist), https://gist.github.com/typesanitizer/e828f909faf22beeeedf208ba58b259b
Source: Beck, Augmented Coding: Beyond the Vibes, 2025, https://newsletter.kentbeck.com/p/augmented-coding-beyond-the-vibes
Say it: "This works, but caching and pricing are tangled: I can't change one without reading the other. Separate them so the price function knows nothing about the cache. The test is whether each can be changed without opening the other."
Described as: "it was easy to build and now it is a knot", "quick to start and painful to change", "what is the difference between simple and easy", "these two things should not know about each other".
https://dev.dqnotes.com/design/simple-made-easy

## 12. Review

Reading a diff for what matters, using a second agent well, and owning what you merge.

### Green doesn't mean right
Also: Verification gap.
A passing check only proves what it checked. When one agent writes the code and its tests, all of them can pass while the code does the wrong thing, because the tests came from the same misunderstanding.
Before agents, the person writing the tests had usually spent hours with the problem, so a green run carried that thinking with it. An agent produces code and tests in one breath, from one reading of your brief, in seconds. Green now tells you the two agree with each other.
Not Coverage: Coverage says a line ran during the tests. Green says the assertions held. Neither says the assertions were about the right thing.
Seen in: In our experiment, 13 of 18 broken one-shot solutions passed the tests their own author wrote. Mutation testing stopped no more: 5 of 18. One small model, one-function tasks. Built In's proposed audit interview asks the same thing directly: "This code passes tests but will fail in production. Tell me why."
Source: Ewing, The audit interview, Built In, 2026, https://builtin.com/articles/audit-interview-scorecard
Say it: "The tests are green and I don't trust that yet. List three inputs or orderings your tests don't cover. For each one you think is a real bug, write a failing test and show me the red output before you change any code."
Described as: "all the tests pass but it still does the wrong thing", "CI is green so why is it broken", "the agent wrote the tests and they prove nothing", "it passes but I do not trust it".
https://dev.dqnotes.com/review/verification-gap

### Code review
Someone other than the author reads a change before it joins the main branch and decides whether it should. It is a judgement about the change, made by a person or an agent, on top of whatever the automatic checks said.
Review used to teach a second person the code as a side effect. With an agent as author, that side effect matters more: the author forgets everything when the session ends, so the reviewer may be the only one who ever understands the change.
Not Backpressure: Backpressure is a machine rejecting work while it's being made. Review is a judgement on the finished work, and it asks things no machine can, such as whether this should exist at all.
Seen in: Google's guide sets the bar: "reviewers should favor approving a CL once it is in a state where it definitely improves the overall code health of the system being worked on, even if the CL isn't perfect." (CL is Google's word for a pull request.) It is disputed whether humans need to do this at all: StrongDM's software factory has the rule that code "must not be" reviewed by humans, and relies on hidden scenarios instead.
Source: Google, The standard of code review, https://google.github.io/eng-practices/review/reviewer/standard.html
Source: McCarthy, Software Factories And The Agentic Moment, StrongDM, 2026, https://factory.strongdm.ai/
Say it: "Before you open the pull request, run git diff main...HEAD, read the whole diff as if someone else wrote it, and tell me what you would ask them to change."
Described as: "someone checks my code before it goes in", "getting a second opinion on a change", "what happens to a pull request before merging", "who approves the code".
https://dev.dqnotes.com/review/code-review

### Reading a diff
A diff shows a change rather than a program: red lines went away, green lines arrived, and plain lines are context. You read it for what changed, and for what should have changed and did not.
Agents reformat, re-indent and move code as they go, and a whole-function re-indent can hide a removed check in plain sight. Learning a few ways to cut the noise pays for itself on the first agent-written pull request you read.
Not Reading code: Reading code asks how the program works. Reading a diff asks whether this change is the right one. You need both, and the diff comes first.
Seen in: git diff -w, in git's own words: "Ignore whitespace when comparing lines." The opposite habit has a famous description in Andrej Karpathy's post that named vibe coding: "I 'Accept All' always, I don't read the diffs anymore." The same post calls this "not too bad for throwaway weekend projects".
Source: git documentation, git diff, https://git-scm.com/docs/git-diff
Source: Karpathy, 2 February 2025, https://x.com/karpathy/status/1886192184808149383
Say it: "Show me git diff --stat main...HEAD first, then git diff -w main...HEAD for src/auth/ only. Tell me which files changed only mechanically, so I can skip them, and say how you know."
Described as: "the whole file shows as changed", "it re-indented everything and I cannot see what changed", "the red and green lines in a pull request", "how do I read what the agent changed".
https://dev.dqnotes.com/review/diff-reading

### What to look for, in order
A review has an order: is this the right change, is it designed sensibly, does it work, is it tested, can it be read. Style comes last, because a linter should be handling it.
Agents rarely get syntax wrong. They solve the wrong problem, handle a case nobody asked about and miss the one you meant, add more than you asked for, and weaken a test to get to green. So those go first, and you stop as soon as an early one fails.
Seen in: Google's list starts: "The most important thing to cover in a review is the overall design of the CL." It then goes through functionality, complexity, tests, naming, comments, style, consistency and documentation. The agent-specific items in our say line below are our suggestion, not Google's.
Source: Google, What to look for in a code review, https://google.github.io/eng-practices/review/reviewer/looking-for.html
Say it: "Review in this order and stop to tell me if a step fails. 1. Does it do what the brief asked? Quote the brief. 2. What did it change that wasn't asked for? 3. Did any test get weakened or deleted? 4. Which inputs aren't tested? Ignore formatting, because Biome handles that."
Described as: "what am I supposed to check in a review", "I do not know where to start with a big change", "I only ever comment on style", "a checklist for reviewing code".
https://dev.dqnotes.com/review/what-to-look-for

### Blast radius
How far a change can reach: everything that imports it, calls it or relies on how it behaves, directly or through something else. A one-line change to shared code can have a bigger blast radius than a thousand new lines nobody uses yet.
An agent sees the file it is editing far better than the callers it hasn't opened, and a reviewer reading the diff sees only the lines that changed. Nobody in that exchange is looking at the code that will really break, unless someone goes and looks.
Not Pull request size: Size counts lines changed. Blast radius counts what those lines can touch. A small pull request can be the dangerous one.
Seen in: pstack has a skill with this name, and it goes further than a search: "Listing the callers is not the job. The agent can grep those in a second. The job is the breakage grep won't show you." People also use the phrase for what an agent's permissions could damage, rather than what a diff could.
Source: Tan, pstack, blast-radius skill, https://github.com/cursor/plugins/blob/main/pstack/skills/blast-radius/SKILL.md
Say it: "Before you change formatMoney in src/lib/money.ts, run rg "formatMoney" and list every caller. Pick the riskiest one, write a test that pins how it behaves today, and run that test before and after your change."
Described as: "what else could this change break", "a tiny change took everything down", "how many things depend on this file", "I changed one line and something unrelated broke".
https://dev.dqnotes.com/review/blast-radius

### Self-evaluation bias
An agent marking its own work gives itself too high a mark. It checks what it meant to build rather than what it built, and from the inside those look the same.
"Are you sure?" is a weak question, because you are asking the same mind, with the same assumptions in front of it, to find what those assumptions hid. Separate the checker from the author and the check starts to mean something.
Not Sycophancy: Sycophancy is an agent agreeing with you. Self-evaluation bias is an agent agreeing with itself, and it happens when you say nothing.
Seen in: Anthropic, on long-running harnesses: "agents reliably skew positive when grading their own work." A stranger version turned up in our own data. In 3 of the 240 one-shot runs the agent returned an empty file, and the neutral reviewer was asked to review it. Twice it wrote the code itself, reviewed that, and approved.
Source: Rajasekaran, Harness design for long-running application development, Anthropic, 2026, https://www.anthropic.com/engineering/harness-design-long-running-apps
Say it: "Don't mark this as done yourself. Write what you changed and how to check it into HANDOFF.md. I will give that file and the diff to a new session, and that session decides."
Described as: "it says everything is perfect", "the AI always thinks its own code is fine", "I asked if it was sure and it said yes", "it graded its own homework".
https://dev.dqnotes.com/review/self-evaluation-bias

### A fresh agent
A new session with none of the conversation that produced the code, handed the diff and the spec and asked to judge it. It has no history to defend and no memory of why any of this seemed like a good idea.
This has no real equivalent before agents. You could never make a colleague forget a week of context on demand. Now a stranger who has read nothing is one command away and costs a few cents, so there is little excuse for letting an author mark its own work.
Not Subagent: A subagent is the mechanism. Fresh is the property you want. A subagent handed the author's summary of what it did is not fresh, because it has inherited the author's story.
Seen in: pstack's shipping playbook: "Only a verdict from an agent that did not write the code counts, because an author checks what it meant to build." Give it the code and the brief, and leave out the author's explanation.
Source: Tan, pstack, shipping playbook, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/shipping.md
Say it: "Open a new session and run: claude -p "Read docs/spec.md and the output of git diff main...HEAD. You did not write this. For each defect, quote the line and give the input that breaks it. If you find none, say so.""
Described as: "have another AI check it without the backstory", "start a new session just to review", "a second opinion from something that did not write it", "it keeps defending its own code".
https://dev.dqnotes.com/review/fresh-eyes

### The reviewer's prompt
The words you give an AI reviewer. They set how hard it pushes, and they can move the verdict more than the code does. So the prompt is a setting with a cost on each side, and you choose it knowingly.
A sharper prompt catches a little more and blocks far more. Good code that gets blocked costs real time, and it teaches a team to ignore the reviewer. Missed bugs cost something too. Neither prompt is correct. The mistake is not knowing which trade you made.
Seen in: We reviewed one correct function 40 times under each of two prompts. The neutral one approved 33 of 40. The other approved 1 of 40. It differs by more than one sentence: it says the code is another agent's, adds "You did not write it", and above all "Find a real defect if there is one, and do not approve out of politeness", so we can't say which words did it. A buggy function was rejected 40 of 40 under both. On 240 real one-shot solutions, neutral stopped 13 of 18 broken and 39 of 222 good. "Find a real defect" stopped 16 and 126. One small model, Claude Haiku 5.5, on one-function tasks with no real repo, so this does not show what your reviewer will do.
Source: This site, the recorded reviews (review-flake.json), https://dev.dqnotes.com/data/review-flake.json
Say it: "Review this diff. Report only defects you can demonstrate: quote the line, give the input, and say what it returns and what it should return. If you find none, say so and approve. Approving is a normal outcome here."
Described as: "the AI reviewer always finds something", "it never approves anything", "how do I word the prompt for a code review", "changing the prompt changed the verdict".
https://dev.dqnotes.com/review/reviewer-prompt

### Findings must show evidence
A review finding has to come with what proves it: the line, the input, what happens and what should happen. A finding without that is an opinion, which is worth having, and which should not stop a merge.
A model writes a false finding in exactly the same confident voice as a true one. Asking for a failing input changes the job from persuading you to showing you. It also separates two questions that often get mixed up: is this true, and does it matter.
Not Nit: A nit is a real but small point. An unevidenced finding might be huge or might be nothing. The trouble is that you can't tell yet.
Seen in: pstack's interrogate skill says "A finding survives only when a run proves it." Its autopilot playbook sets the bar for stopping a merge: "Block a PR only for a problem you can show, with a failing run at the head or a doc line the PR made wrong." We tested the idea on our own recording. In reviews that rejected passing split-bill code, we ran the 18 calls the reviewers wrote out, up to two per review. 15 really broke the code, all on inputs like weights of 1e308, and 3 did not. Separately, of the 13 broken runs the neutral reviewer stopped, one was rejected for a different reason from the real fault, and one because the file was empty. On 3 November 2026, the skill's author, Lauren Tan, wrote that they now consider adversarial reviews "mostly a waste of tokens". That is about whether to run such a round at all. It doesn't withdraw the rule that a finding needs proof, and it doesn't endorse it either, so these are two questions, and verification spend takes up the first.
Source: Tan, pstack, interrogate skill, https://github.com/cursor/plugins/blob/main/pstack/skills/interrogate/SKILL.md
Source: Tan, pstack, autopilot playbook, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/autopilot-full.md
Source: Tan on X, 3 November 2026, https://x.com/poteto/status/2117437607940681849
Source: The same post, through the fxtwitter mirror, https://api.fxtwitter.com/poteto/status/2117437607940681849
Say it: "Run every finding before you report it. For each, give me the command you ran and its output. Anything you could not run goes in a separate list called unverified, and it does not block this merge. Finding nothing is a fine result."
Described as: "the reviewer complains but cannot show the bug", "prove it", "how do I know this review comment is real", "it sounds right but is it actually a problem".
https://dev.dqnotes.com/review/evidence-finding

### Review in the abstract
Also: Abstract review.
Reading code with nothing to check it against: no run, no stated intent, no question to answer. The reviewer fills the gap with what can be said about any code, which means small style points and warnings about things you may not need.
A reviewer needs a narrower job than "review this". Give it one behaviour to check and a way to check it, and the easy comments stop being the answer. Leave it open and you can spend your tokens, and your afternoon, on comments that were never going to change what the code does.
Not Reading the code: Reading isn't the trouble. Our neutral reviewer could only read, and it rejected all 15 vague payment-events runs, 4 of which passed every hidden scenario. What it could do was compare the code with a stated job, which is the part this term is missing.
Seen in: Lauren Tan on X, 3 November 2026: "i have found very little value with code review in the abstract, agents tend to just get lost in the nitpicks and yagnis". That is a practitioner's report with no figures in it. pstack's interrogate skill tells its adversaries to leave out "style, naming, structure, and risks that need a future change to matter", and its guide says "no nitpicks unless it's an actual bug or regression." We haven't measured this ourselves, as our reviewers were all given the job and the code to compare.
Source: Tan on X, 3 November 2026, https://x.com/poteto/status/2117437607940681849
Source: The same post, through the fxtwitter mirror, https://api.fxtwitter.com/poteto/status/2117437607940681849
Source: Tan, pstack, interrogate skill, https://github.com/cursor/plugins/blob/main/pstack/skills/interrogate/SKILL.md
Source: Tan, pstack guide, Design before you write code, https://github.com/cursor/plugins/blob/main/pstack/docs/guide/04-design.md
Say it: "Review this diff against one question: does POST /refunds still refund at most what was paid, when the same request arrives twice? Run it and paste the output. Leave out style, naming and anything that only matters after a future change. If you have nothing on that question, say so and stop."
Described as: "the reviewer only talks about style", "it keeps telling me I do not need this", "a review full of nitpicks and no bugs", "the AI review comments are all noise", "how do I keep the reviewer on behaviour".
https://dev.dqnotes.com/review/abstract-review

### AI code reviewers
A product that reads your pull requests with a model and leaves comments on the lines it doubts. Most of them comment and leave the decision with a person.
They catch things no rule can express, and they cost tokens, take time, and give a different answer on a second run. Their vendors know this: the common default is that the bot's opinion is not an approval. Treat one as a very fast colleague who has never met your users.
Not Linter: A linter gives the same answer every time and can be a hard gate. An AI reviewer gives an opinion that varies, so make it advice unless you have measured it.
Seen in: As of November 2026, Claude Code's Code Review: "A review never approves or blocks the pull request". GitHub: "Copilot's reviews do not count toward required approvals" by default. Cursor's Bugbot publishes findings as a neutral check, so "Requiring the status alone does not block merges on findings". CodeRabbit's figure of about 1.7 times as many issues per AI pull request comes from a vendor measuring with its own reviewer, on 470 pull requests. pstack is blunter: "Green CI and an approving bot review are not verdicts, because neither runs the change as a user would."
Source: Claude Code docs, Code Review, read November 2026, https://code.claude.com/docs/en/code-review
Source: GitHub docs, Copilot code review, https://docs.github.com/en/copilot/concepts/agents/code-review
Source: Cursor docs, Bugbot, https://cursor.com/docs/bugbot
Source: CodeRabbit, State of AI vs Human Code Generation, 2025, https://coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report
Source: Tan, pstack, shipping playbook, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/shipping.md
Say it: "Add a REVIEW.md telling the reviewer to report only logic errors and security holes, each with the input that triggers it, and to ignore style because Biome handles that. Then run /code-review on this branch and show me what it says."
Described as: "a bot that comments on my pull requests", "should I let an AI approve my code", "automatic code review tools", "copilot left comments on my PR".
https://dev.dqnotes.com/review/ai-reviewer

### You own what you merge
Whoever merges a change answers for it, whoever or whatever typed it. That means being able to say what it does, what could go wrong, how you would notice, and how you would undo it.
An agent has nothing at stake. It won't be paged, can't be asked next month why it did this, and won't remember if you ask. This is what is left of the job when typing is cheap, and employers now say so in writing.
Not Authorship: Authorship is who typed it. Ownership is who answers for it. With agents those come apart completely, and only the second one still matters.
Seen in: Canva expects engineers to "take full ownership of any code they produce, whether they wrote it themselves or with AI assistance." pstack puts it to the lead agent: "You own every subagent's work. Review the diff and write your own summary, don't pass through what it said." How to own it is disputed. Robert Martin wrote on X that he has "very high confidence" in unread agent code that has "had to run the gauntlet of all of my constraints and tests". StartupFortune reports Grady Booch replying "Trust but verify", and we have only that press report for it.
Source: Newton, Yes, you can use AI in our interviews, Canva, 2025, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Source: Tan, pstack, poteto-mode skill, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/SKILL.md
Source: Martin on X, 23 July 2026 (read through the fxtwitter mirror of this status), https://api.fxtwitter.com/unclebobmartin/status/2080257779395154409
Source: StartupFortune, on Uncle Bob Martin not reading AI-generated code, 2026, https://startupfortune.com/uncle-bob-martin-says-he-no-longer-reads-ai-generated-code-and-the-developer-world-is-split/
Say it: "I am merging this under my name. Before I do, tell me what could go wrong in production, how we would notice, and the exact command that undoes it. If you can't answer one of those, say so and don't guess."
Described as: "the AI wrote it so whose fault is it", "who is responsible for AI generated bugs", "I got blamed for code the agent wrote", "can I say the agent did it".
https://dev.dqnotes.com/review/ownership

### Code owners
Also: CODEOWNERS.
A file in the repo that maps paths to the people or teams responsible for them, so a pull request touching those paths automatically asks the right reviewer. On GitHub it can also be made a requirement for merging.
An agent is happy to edit anything it can reach, and it has no idea that the payments folder is sensitive at your company. A code owners rule turns that knowledge into something the machine enforces, whoever or whatever opened the pull request.
Seen in: GitHub looks for it at .github/CODEOWNERS, the repo root or docs/. "Order is important", says its example file, and the last matching pattern takes the most precedence. So a catch-all line at the bottom silently replaces everything above it. Owners are only required if "Require review from Code Owners" is turned on for the branch.
Source: GitHub docs, About code owners, https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
Say it: "Add .github/CODEOWNERS with the catch-all first, because the last match wins, then /src/payments/ for @acme/payments. Then tell me the setting to turn on so that review is required and not only requested."
Described as: "the right person never gets asked to review", "make sure payments code goes past the payments team", "automatically assign reviewers by folder", "anyone can approve changes to the sensitive bit".
https://dev.dqnotes.com/review/code-owners

### Comprehension debt
The gap between how much code exists and how much of it anyone understands. Like money borrowed, it costs nothing today, and you pay later, in the time it takes to work out the code before you can safely change it.
Agents remove the friction that used to build understanding as a side effect. The code can be clean, tested and passing and still be something nobody on the team could change safely. No tool in your pipeline measures this.
Not Technical debt: Technical debt is in the code, and you can point at it. Comprehension debt is in the gap between code and people, and the code can look perfect.
Seen in: Jason Gorman, September 2025: "When teams produce code faster than they can understand it, it creates what I've been calling 'comprehension debt'." It is the earliest named use we found, and Ahmad's 2026 paper credits Gorman. Addy Osmani's essay gives the usual definition, "the growing gap between how much code exists in your system and how much of it any human being genuinely understands", and treats it as the same thing as cognitive debt. Evidence is thin: Ahmad studied 621 diaries from 207 students, with no measure of defects.
Source: Gorman, Comprehension Debt, Codemanship, September 2025, https://codemanship.wordpress.com/2025/09/30/comprehension-debt-the-ticking-time-bomb-of-llm-generated-code/
Source: Osmani, Comprehension Debt, March 2026, https://addyosmani.com/blog/comprehension-debt/
Source: Ahmad, Comprehension Debt in GenAI-Assisted Software Engineering Projects, 2026, https://arxiv.org/abs/2604.13277
Say it: "Before you add anything to src/billing/, write a one-page map of how it works today, from reading the code. Mark anything you couldn't explain. Those marked parts are where I need to catch up before we go further."
Described as: "nobody understands this code any more", "it works but I have no idea how", "the agent wrote it and now I am scared to touch it", "we are shipping code nobody has read".
https://dev.dqnotes.com/review/comprehension-debt

### Cognitive debt
A close cousin of comprehension debt, and many writers use the two as one. Where a difference is drawn, this one is about people rather than code: a team gradually losing its shared picture of how the system works and why.
One person can understand their corner and the team can still have lost the whole. That explains a team that can't make simple changes without breaking things, even with tidy code. It also shows why one person reading more diffs doesn't fix it.
Not Comprehension debt: Often no difference at all. Osmani treats them as one thing. Storey's version is a property of a team over time, and that paper points to comprehension debt as related work. Say which you mean.
Seen in: Who coined it isn't clear. Margaret-Anne Storey's February 2026 post says the term is "gaining traction recently", and does not claim it. An MIT Media Lab paper has it in its title in June 2025, about essay writing rather than code. Storey's later paper defines it: "a team-level, project-level property reflecting the erosion of shared understanding across a software system over time." The example in the February post is a student team that hit a wall by weeks 7 or 8. The idea is older than the term: Peter Naur argued in 1985 that a program's real theory lives in its programmers.
Source: Storey, How Generative and Agentic AI Shift Concern from Technical Debt to Cognitive Debt, February 2026, https://margaretstorey.com/blog/2026/02/09/cognitive-debt/
Source: Storey, From Technical Debt to Cognitive and Intent Debt, 2026, https://arxiv.org/abs/2603.22106
Source: Kosmyna and others, Your Brain on ChatGPT, 2025, https://arxiv.org/abs/2506.08872
Source: Naur, Programming as Theory Building, 1985, https://gist.github.com/dpritchett/fd7115b6f556e40103ef
Say it: "Write docs/decisions/0007-payment-retries.md explaining why we retry payments this way and what we rejected, from the commit history and our conversation. Then each of us reads it and I will quiz the team on it on Friday."
Described as: "the team has lost track of how this works", "nobody can explain why it was built like this", "everyone is afraid to change things", "we move fast but understand less each week".
https://dev.dqnotes.com/review/cognitive-debt

### Explain it back
Before accepting code, say what it does in your own words, including what it does on awkward inputs, then check yourself by running it. If reality disagrees with you, you have just found what you did not understand.
Reading code gives a strong feeling of understanding that often isn't real, and an agent's tidy explanation makes that feeling stronger. Predicting and then running is a test the feeling can fail. It is also the skill an interviewer is checking when they switch the agent off.
Not Explain it out loud: Talking to a rubber duck is how you find your own bug. Explaining back is how you find out whether you understood someone else's code, and the code gets the last word.
Seen in: In Anthropic's randomised trial, 52 mostly junior engineers learned a new library with or without an AI assistant. The AI group averaged 50% on the quiz afterwards and the others 67%. That is 17 percentage points, not 17%, and small and short-term, in a chat sidebar rather than an agent. The patterns that kept scores up involved asking questions rather than delegating.
Source: Anthropic, How AI assistance impacts the formation of coding skills, 2026, https://www.anthropic.com/research/AI-assistance-coding-skills
Say it: "Don't explain src/cart.ts to me yet. I am going to tell you what happens on an empty cart and when the payment call times out. Then write those two cases as tests, run them, and show me where I was wrong."
Described as: "I read it and it made sense but could I explain it", "check I actually understand this", "quiz me on this code", "could I defend this in an interview".
https://dev.dqnotes.com/review/explain-back

### The review bottleneck
The point where agent-written work piles up waiting for someone to read it. Writing code got cheaper and reading didn't, so the queue in front of reviewers is now what limits how fast a team ships.
Producing more pull requests stops helping once they wait longer, and the pressure to clear the queue is exactly what produces careless approvals. It is also why individual speed-ups can vanish at team level.
Not Slow reviewers: A bottleneck is a property of the system, not a failing of the people in it. Adding pressure on reviewers moves the cost into missed bugs.
Seen in: Chen and Stratton, a Harvard job-market paper not yet peer reviewed as far as we could tell: 720,542 workers at 655 firms, version of 1 November 2026. With agents, pull requests opened rose 35.7% and lines 44.0%, while time from submission to merge rose 56.1%, time in active review 58.5%, and comments per pull request 38.6%. Issues resolved rose 13.7%, which they call marginally insignificant. AI assistants, as opposed to agents, had no significant effect on review length. Earlier press figures came from an older version. OpenAI takes the other side for its own repo: "corrections are cheap, and waiting is expensive."
Source: Chen and Stratton, Artificial Intelligence in the Firm, version of 1 November 2026, https://fion.ac/jellyfish.pdf
Source: Lopopolo, Harness engineering, OpenAI, 2026, https://openai.com/index/harness-engineering/
Say it: "Stop opening new pull requests. Run gh pr list --author "@me", tell me which of mine are waiting and for how long, and which one would unblock the most others if it merged first."
Described as: "my pull requests sit there for days", "we write code faster but ship no faster", "everyone is waiting on reviews", "the agents produce more than we can read".
https://dev.dqnotes.com/review/review-bottleneck

### Small pull requests
Keeping each pull request to one reviewable idea, often a few hundred lines at most. Long changes get split so that each piece makes sense and passes its checks by itself.
An agent can produce 2,000 lines as easily as 20, and it doesn't feel the difference. The reviewer does. Long diffs get skimmed, and a skimmed diff is a diff that wasn't reviewed. Size is the limit you have to set, since nothing about the agent will.
Not Small commits: A commit is one step in the history. A pull request is one thing a person has to read and approve. You can have tidy commits in a pull request nobody will read properly.
Seen in: Google: "100 lines is usually a reasonable size for a CL, and 1000 lines is usually too large". SmartBear, which sells review tools, says from a study of a Cisco team that developers "should review no more than 200 to 400 lines of code (LOC) at a time". DORA lists working in small batches among the capabilities that amplify AI's effect. These are guidelines rather than laws, and the numbers disagree.
Source: Google, Small CLs, https://google.github.io/eng-practices/review/developer/small-cls.html
Source: SmartBear, Best practices for peer code review, https://smartbear.com/learn/code-review/best-practices-for-peer-code-review/
Source: DORA, State of AI-assisted Software Development, 2025, https://services.google.com/fh/files/misc/2025_state_of_ai_assisted_software_development.pdf
Say it: "This is 1,400 lines, which is too much to review well. Split it: first a refactor with no change in behaviour, then the schema, then the feature, each under 300 lines and each passing bun test by itself."
Described as: "the pull request is huge", "nobody wants to review this", "how big should a change be", "it is too much to read in one go".
https://dev.dqnotes.com/review/pr-size

### Stacked diffs
Also: Stacked pull requests.
A long change split into a chain of pull requests, each built on the one before it, so each can be reviewed alone and you don't have to wait for the first to merge before writing the next.
An agent works faster than any reviewer, so it will always want to carry on before the last piece has been read. A stack lets it, while keeping each piece small. The cost is that a fix low down means updating everything above, and that is the kind of chore you can hand to an agent.
Not Long-lived branch: A long-lived branch is one big thing reviewed at the end. A stack is many small things reviewed one at a time, in order.
Seen in: Google describes the move: "write one small CL, send it off for review, and then immediately start writing another CL" based on the first. Git's --update-refs does the chore on rebase: "Automatically force-update any branches that point to commits that are being rebased." stacking.dev lists Graphite, ghstack, git town, Sapling and spr as tools for it.
Source: Google, Small CLs, https://google.github.io/eng-practices/review/developer/small-cls.html
Source: git documentation, git rebase, https://git-scm.com/docs/git-rebase
Source: stacking.dev, https://www.stacking.dev/
Say it: "Split this into a stack: payments-1-schema off main, payments-2-api off that, payments-3-ui off that, with one pull request per branch, based on the branch below. After any fix lower down, go to payments-3-ui and run git rebase --update-refs payments-1-schema."
Described as: "one PR depends on another one", "I do not want to wait for review before starting the next part", "chain of small pull requests", "split a feature into reviewable pieces that build on each other".
https://dev.dqnotes.com/review/stacked-diffs

### Making work reviewable
Doing the work so a reviewer can check it cheaply: one idea per pull request, a description saying what changed and why, and pasted evidence that it works, with the output of what you ran. You spend your time to save theirs.
When reading is the bottleneck, the most useful thing an author can do is make reading easier. An agent will happily write a description that claims everything worked. A description that pastes the output has to be true, or at least it can be checked.
Not Pull request description: A description tells a reviewer what you say you did. Evidence shows them. A long description with no output pasted in is still only a claim.
Seen in: pstack's bug-fix playbook sets the reply format: "what was broken, root cause, fix, how you verified. Paste failing-then-passing repro output verbatim." It also wants every claim to carry "its evidence or its label in the same sentence. Measured, inferred, or guess."
Source: Tan, pstack, bug-fix playbook, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/bug-fix.md
Source: Tan, pstack, poteto-mode skill, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/SKILL.md
Say it: "Write the pull request description with these headings: Why, What changed, How I checked. Under How I checked, paste the failing test output from before the fix and the passing output from after. Label anything you didn't run as not run."
Described as: "what should go in a pull request description", "make this easy for someone to review", "the description says it works but how do they know", "help the reviewer".
https://dev.dqnotes.com/review/reviewable

### Nit
Also: Nitpick.
A review comment marked as small and optional: a matter of polish that the author can take or leave. The label tells them not to hold the change up for it.
To a person, "nit:" means "no rush". An agent told to address the review comments may treat every one as an order, including the ones you meant as a passing thought, so say which are optional. And if the same nit keeps coming up, make it a lint rule and stop saying it.
Seen in: Google: prefix it with "Nit: " "to let the author know that it's just a point of polish that they could choose to ignore." Conventional Comments: "Nitpicks are trivial preference-based requests. These should be non-blocking by nature."
Source: Google, The standard of code review, https://google.github.io/eng-practices/review/reviewer/standard.html
Source: Conventional Comments, https://conventionalcomments.org/
Say it: "Fix the two comments labelled blocking. Leave the ones labelled nit alone. Then add a Biome rule for the naming one, so that it doesn't come up again."
Described as: "a small comment on a pull request", "this is not important but", "what does nit mean", "the reviewer keeps picking on tiny things".
https://dev.dqnotes.com/review/nit

### Blocking and non-blocking
A blocking comment must be dealt with before the change can merge. A non-blocking one is advice. Saying which one you mean is half of what makes a review useful.
An AI reviewer can produce a dozen comments in seconds, written in the same serious tone. If all of them block, the queue stops; if none do, the real one gets lost. A good rule for agents: only what can be demonstrated blocks, and the rest goes in the notes.
Not Severity: Severity is how bad a problem would be. Blocking is whether it stops this merge. A serious problem that nobody can demonstrate may be a note, and a small one you can prove may block.
Seen in: Conventional Comments writes them as decorations: (blocking) "should prevent the subject under review from being accepted, until it is resolved", and (non-blocking) "should not prevent" it. In our recording, 166 of the 480 reviews use the word blocking or non-blocking without being asked, so the habit is already in the model.
Source: Conventional Comments, https://conventionalcomments.org/
Say it: "Label every finding blocking or non-blocking. Blocking means you ran it and can show me the failure. Everything else is non-blocking, however serious it sounds. Put the blocking ones first."
Described as: "does this comment have to be fixed before merging", "which review comments actually matter", "the reviewer left twelve comments", "must fix or nice to have".
https://dev.dqnotes.com/review/blocking-comment

### LGTM
Also: Looks good to me.
Short for "looks good to me": the usual way to approve a pull request. It is supposed to be a claim that you read it and would stand behind it.
It costs four letters and carries your name. When agents make the diffs long and frequent, it's very easy to type without the reading behind it. Interviewers now look for exactly this.
Not Approval: Approval is what the system records. LGTM is what you are telling your colleagues. A bot can do the first. Only you can honestly say the second.
Seen in: Elicit, on what it sees in interviews: "Rapidly skimming a big diff and saying 'looks good' when there's no realistic chance they've reviewed it" is "more common than we'd like."
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Say it: "I am not approving this on reading alone. Check out the branch, run bun test and the app, try the case from the brief where the cart is empty, and paste me what happened."
Described as: "what does LGTM mean", "how do I approve a pull request", "I just said it looked fine", "someone approved it in ten seconds".
https://dev.dqnotes.com/review/lgtm

### Rubber-stamping
Approving without really reviewing, because the last fifty were fine and there are ten more waiting. The approval still gets recorded, so from outside it looks exactly like a review.
The better the agent gets, the more dangerous this is. If it is right nineteen times in twenty, you learn to stop looking, and the twentieth goes through with your name on it. Human review is not a reliable backstop just because it exists.
Not LGTM: LGTM is only the words. Rubber-stamping is when nothing stands behind them.
Seen in: The research behind the idea is about people watching for rare signals, not code. Norman Mackworth's clock experiments in the 1940s simulated radar watch, and helped establish that detection falls off after about half an hour on task. Applying that to code review is an analogy: we found no study that measures it in reviewers, so treat the bench here as an illustration only.
Source: Wikipedia, Mackworth Clock (we read this and not the 1948 paper), https://en.wikipedia.org/wiki/Mackworth_Clock
Say it: "I approved your last five without finding anything, so I no longer trust my own attention here. For this one, tell me the single riskiest line and the input that would break it. I will read that first and run it myself."
Described as: "I approve everything without really reading it", "review has become a formality", "it is always fine so I stopped looking", "I clicked approve on autopilot".
https://dev.dqnotes.com/review/rubber-stamping

## 13. Security

Injection, secrets, access control, supply chains, and the new attacks that target the agent itself.

### Threat model
A short written answer to: what are we building, what can go wrong, what will we do about each thing, and did that work? It turns "be secure" into a list you can check.
An agent builds what the brief describes, and a brief usually describes honest users. Until someone writes down what a dishonest one would try, nothing in the work points at it.
Not Security review: A threat model comes first and asks what could go wrong with the design. A review comes after and asks what went wrong in this code.
Seen in: The Threat Modeling Manifesto boils it down to four questions: "What are we working on?", "What can go wrong?", "What are we going to do about it?" and "Did we do a good enough job?"
Source: Threat Modeling Manifesto, https://www.threatmodelingmanifesto.org/
Say it: "Before you build the upload endpoint, write docs/threats.md. List what an attacker could try against it, at least five things, with what stops each one. Then add a failing test for each one we say is covered."
Described as: "how do I work out what could be attacked", "I want the agent to think like an attacker before it builds", "what should I worry about in this design".
https://dev.dqnotes.com/security/threat-model

### Attack surface
Also: Hardening.
Every place someone outside can send your system something: forms, URLs, uploads, webhooks, headers, and any file or page your agent reads. The bigger it is, the more you have to defend.
Agents grow it quietly. A debug route left in, a new webhook, a dependency, an MCP server: each one works, each one passes its tests, and each one is another door. It rarely shows up as the point of the change, so read the diff for it.
Not Threat model: The attack surface is where someone can get in. The threat model is what they would do there, and what you will do about it.
Say it: "List every new way in that this branch adds: routes, webhooks, file uploads, environment variables read, dependencies. For each, say who can reach it and what checks it. Remove the debug route."
Described as: "how many ways in does my app have", "the agent added an endpoint I never asked for", "what can someone outside actually reach".
https://dev.dqnotes.com/security/attack-surface

### OWASP Top 10
A list of the ten most serious kinds of web application risk, kept by the non-profit OWASP and updated every few years. The trade uses it as shared vocabulary, and it is a good first checklist for a review.
It gives you words an agent knows well. "Check this against A01 and A05" gets a far more specific answer than "is this safe?"
Not CWE Top 25: OWASP's list groups risks into broad categories. MITRE's CWE Top 25 ranks specific kinds of weakness, such as cross-site scripting at number 1 for 2025.
Seen in: The 2025 edition runs: A01 Broken Access Control, A02 Security Misconfiguration, A03 Software Supply Chain Failures, A04 Cryptographic Failures, A05 Injection, A06 Insecure Design, A07 Authentication Failures, A08 Software or Data Integrity Failures, A09 Security Logging and Alerting Failures, A10 Mishandling of Exceptional Conditions.
Source: OWASP Top 10:2025, https://top10.owasp.org/2025
Source: MITRE, 2025 CWE Top 25, https://cwe.mitre.org/top25/archive/2025/2025_cwe_top25.html
Say it: "Review this branch against OWASP Top 10:2025. For each category, either name the file and line where it applies and what stops it, or write "not applicable" with a reason. Skip nothing."
Described as: "is there a standard list of what to check", "what are the most common web vulnerabilities", "what does A01 mean".
https://dev.dqnotes.com/security/owasp-top-ten

### Injection
Text from a user gets pasted into something another program reads as instructions: a database query, a shell command, a web page. The program can't tell which part you wrote, so the user's text can become commands.
Joining strings is the shortest way to write these, and it works perfectly on every honest input. So an agent can ship it with a full set of green tests.
Not Prompt injection: A database has a separate channel for values, so classic injection has a complete fix. A language model reads one stream of text, and no such channel exists.
Seen in: Injection is A05 in OWASP's 2025 list. OS command injection, its shell form, is number 9 in MITRE's 2025 CWE Top 25.
Source: OWASP Top 10:2025, https://top10.owasp.org/2025
Source: MITRE, 2025 CWE Top 25, https://cwe.mitre.org/top25/archive/2025/2025_cwe_top25.html
Say it: "Don't build this shell command by joining strings. Call execFile('convert', [inputPath, outputPath]) with an argument array, so a filename like a.png; rm -rf ~ stays one filename. Add a test with that exact name."
Described as: "user input ends up running as code", "someone typed a command into my form", "why is joining strings dangerous".
https://dev.dqnotes.com/security/injection

### SQL injection
Also: SQLi, parameterised query, prepared statement.
SQL is the language for asking a database for things, as in SELECT * FROM users WHERE name = 'alice'. Paste a user's text into that sentence and they can rewrite the question. A parameterised query sends the value separately, so it can only ever be a value.
One typed line can return every row or change data it should never touch. And the fix is complete, which is rare in security, so there's no reason to accept anything less.
Not Escaping quotes: Escaping tries to neutralise dangerous characters by hand, and it's easy to get wrong. A parameter never enters the sentence at all, so there's nothing to escape.
Seen in: OWASP: "Prepared statements treat bound values as data rather than SQL syntax" and "Do not use manual escaping as a substitute for parameter binding." One limit it notes: markers can't stand in for table or column names, so those need an allowlist. SQL injection is number 2 in the 2025 CWE Top 25.
Source: OWASP Cheat Sheet, SQL Injection Prevention, https://cheatsheetseries.owasp.org/cheatsheets/SQL_Injection_Prevention_Cheat_Sheet.html
Source: MITRE, 2025 CWE Top 25, https://cwe.mitre.org/top25/archive/2025/2025_cwe_top25.html
Say it: "Use a parameterised query here and add a test that passes '; DROP TABLE users;-- as the name. The sort column can't be a parameter, so check it against an allowlist of column names."
Described as: "someone logged in by typing a quote", "a search box returned every user", "how do I stop people messing with my database queries".
https://dev.dqnotes.com/security/sql-injection

### Cross-site scripting
Also: XSS.
An attacker gets their text into your page, and the visitor's browser runs it as code. That code runs as your site, with the visitor's logged-in session, so it can do whatever they can.
A comment box is enough. Everyone who views the comment runs the attacker's script, and passing tests won't show it, because a test comment like "hello" looks right either way.
Not CSRF: XSS runs attacker code inside your page. CSRF never gets into your page: it makes the browser send a request from somewhere else. OWASP warns that XSS can defeat all CSRF defences.
Seen in: Number 1 in the 2025 CWE Top 25. React escapes values by default, and dangerouslySetInnerHTML is where that stops. OWASP notes that React 19 blocks javascript: URLs in src and href, and recommends DOMPurify when you really must allow HTML.
Source: OWASP Cheat Sheet, Cross Site Scripting Prevention, https://cheatsheetseries.owasp.org/cheatsheets/Cross_Site_Scripting_Prevention_Cheat_Sheet.html
Source: MITRE, 2025 CWE Top 25, https://cwe.mitre.org/top25/archive/2025/2025_cwe_top25.html
Say it: "Render this comment as plain text with {comment}, not dangerouslySetInnerHTML. If we have to allow formatting, sanitise with DOMPurify and add a test that <img src=x onerror=alert(1)> ends up as text in the DOM."
Described as: "someone put a script in a comment", "an alert box popped up on my site", "user text is running in the browser".
https://dev.dqnotes.com/security/xss

### Output encoding
Also: escaping, contextual encoding.
Converting a value when you put it into a page, so the browser reads it as text. The right conversion depends on where it lands: inside a tag, in an attribute, or in a link are three different problems.
Escaping HTML does nothing for a link. javascript:alert(1) has no angle brackets to escape, and clicking it runs the code. This is exactly the kind of gap the code passes tests with.
Not Input validation: Validation decides whether to accept a value when it arrives. Encoding makes it safe at the moment it leaves, for the place it goes. You usually need both.
Seen in: OWASP: "There are many different output encoding methods because browsers parse HTML, JS, URLs, and CSS differently." In our experiment, recorded renderers that allowed only http, https and mailto still let through anything with no visible scheme, as a relative link. One control character in front of javascript: hides the scheme from that check, and a URL parser drops the character. The renderers also cut at the first closing bracket, so alert(1) arrives as alert(1 and doesn't run, which is why you test with a payload that has no brackets as well.
Source: OWASP Cheat Sheet, Cross Site Scripting Prevention, https://cheatsheetseries.owasp.org/cheatsheets/Cross_Site_Scripting_Prevention_Cheat_Sheet.html
Say it: "This value goes in an href, so escaping HTML isn't enough. Parse it with new URL, allow only https: and mailto:, and add a test with a control character before javascript: to prove a hand-written scheme check isn't what's protecting us."
Described as: "I escaped it and it still ran", "what is the right way to put user text in a link", "do I escape on the way in or the way out".
https://dev.dqnotes.com/security/output-encoding

### CSRF
Also: cross-site request forgery.
Another website makes your visitor's browser send a request to your site. The browser attaches their cookie automatically, so your server sees a logged-in user asking to do something they never asked for.
The attacker never sees your site or the response, and no password is stolen. A route that changes data and trusts the cookie alone can be fired from any page they happen to visit.
Not CORS: CORS decides whether another site's script may read a response. CSRF doesn't need to read anything. The request has already done its damage.
Seen in: OWASP: "SameSite is useful as a defense-in-depth control but it does not replace a proper CSRF defense in most deployments." And: "Do not use GET requests for state changing operations." It names the Sec-Fetch-Site header as the primary signal.
Source: OWASP Cheat Sheet, Cross-Site Request Forgery Prevention, https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
Say it: "Reject any request that changes state when Sec-Fetch-Site is cross-site, set SameSite=Lax on the session cookie, and move the delete action off GET. Add a test that sends the cross-site header and expects 403."
Described as: "another site made my user do something", "a link deleted something while logged in", "why does a form need a hidden token".
https://dev.dqnotes.com/security/csrf

### SSRF
Also: server-side request forgery.
A feature fetches a URL the user gave it, such as an image from a link. The attacker gives it an internal address, and your server, which sits inside your network, fetches what the attacker could never reach.
Cloud servers can often reach a metadata address that hands out credentials. "Fetch this URL" sounds harmless and takes one line, so it passes without a second look.
Not CSRF: CSRF tricks a visitor's browser into sending a request. SSRF tricks your server into sending one, from inside your network.
Seen in: Krebs reported that the 2019 Capital One breach, which exposed more than 100 million consumer applications, used SSRF against a misconfigured firewall to reach the metadata service. OWASP: "Match the host against an allowlist, and build the request yourself." And: "Domain allowlisting alone does not prevent DNS rebinding."
Source: Krebs on Security, What We Can Learn from the Capital One Hack, 2019, https://krebsonsecurity.com/2019/08/what-we-can-learn-from-the-capital-one-hack/
Source: OWASP Cheat Sheet, Server Side Request Forgery Prevention, https://cheatsheetseries.owasp.org/cheatsheets/Server_Side_Request_Forgery_Prevention_Cheat_Sheet.html
Say it: "This endpoint fetches a URL from the user. Match the host against an allowlist, turn off redirects, and add tests for http://169.254.169.254/, http://127.0.0.1/ and http://2130706433/, which is the same address written as a number."
Described as: "my server fetched something internal", "the image URL feature can reach my database", "can a user make my server call things".
https://dev.dqnotes.com/security/ssrf

### Path traversal
Also: directory traversal, dot dot slash.
A filename from a user includes ../, which means "the folder above", until it climbs out of the folder you meant to serve. Then your server hands over files you never meant to expose.
Removing ../ with a replace looks like a fix, and it isn't: ....// becomes ../ once the inner part is stripped. The check has to run on the final resolved path.
Not File permissions: Permissions are what the operating system allows your server to read. Traversal gets your server to read something it can, on someone else's behalf.
Seen in: Number 6 in the 2025 CWE Top 25.
Source: MITRE, 2025 CWE Top 25, https://cwe.mitre.org/top25/archive/2025/2025_cwe_top25.html
Say it: "Resolve the path with path.resolve after decoding it, and refuse it unless it starts with the uploads folder plus a separator. Add tests for ../../etc/passwd, ....//etc/passwd and ..%2f..%2fetc%2fpasswd. Better still, store files under random ids so a user's filename never touches the disk."
Described as: "someone downloaded a file outside the folder", "dot dot slash in a URL", "can a user read my config by changing the filename".
https://dev.dqnotes.com/security/path-traversal

### Open redirect
A link on your own site sends visitors wherever a parameter says, as in /login?next=... An attacker sends a link that starts with your trusted address and ends on theirs, ready for a convincing fake page.
The check looks easy and has many ways round it. //evil.example has no scheme but still leaves your site, and so does a backslash in some positions, because browsers read addresses differently from how a string comparison does.
Not SSRF: An open redirect sends the visitor's browser somewhere. SSRF sends your server somewhere.
Seen in: Credit where it's due: in our experiment's safe-redirect task, all 60 recorded final programs held against every attack on this bench, judged by a real URL parser. The name of that task states the danger, and one small model on one small function handled it. Two vague one-shot runs did fail a hidden scenario on upper-case hostnames.
Say it: "Parse next with new URL against our own origin, allow only same-origin or hosts on the allowlist, and fall back to /. Test //evil.example, /\evil.example, https://shop.example@evil.example and javascript:alert(1). Don't use startsWith."
Described as: "the login page sends people to another site", "phishing link that starts with my domain", "how do I check a return URL".
https://dev.dqnotes.com/security/open-redirect

### Input validation
Checking that what arrives is what you expect, in type, length and format, and refusing it if not. It must run on the server, because anyone can skip the browser and send the request directly.
It stops plenty of bugs and shrinks what an attacker can try. It does not make a query safe, and an agent asked to "sanitise inputs" may reach for a list of banned characters and call the job done.
Not Output encoding: Validation asks whether this value is acceptable at all. O'Brien is a valid name, and it still breaks a query that pastes it in. That part is a job for parameters and encoding.
Seen in: OWASP: "Do not try to recognize every malicious string." And: "Blocking apostrophes, for example, rejects legitimate names without making a database query safe."
Source: OWASP Cheat Sheet, Input Validation, https://cheatsheetseries.owasp.org/cheatsheets/Input_Validation_Cheat_Sheet.html
Say it: "Validate this request body on the server with a schema: name is 1 to 60 characters, age is an integer from 0 to 150, and unknown fields are rejected. Don't filter for bad words. Injection is handled by the parameterised query, not by this."
Described as: "the browser check got skipped", "should I block special characters", "how do I make sure the data is sensible".
https://dev.dqnotes.com/security/input-validation

### Memory safety
Also: Buffer overflow, Use after free, Memory-safe language.
Code is memory-safe when it can't read or write memory it doesn't own. C and C++ leave that to the programmer, so one wrong length can overwrite whatever sits next to it. Languages like Rust, Go, Java and Python check it for you, or won't let you write it.
It is the largest single kind of serious bug in big C and C++ systems, and ordinary tests rarely see it, because the program carries on. Writing JavaScript or Python covers most of it for you. It comes back the moment an agent writes a native module, or offers to port one.
Not Memory leak: A leak is memory you keep and never give back, so it costs you speed and in the end a crash. An unsafe write touches memory that was never yours, and an attacker can steer it.
Seen in: CISA, the NSA and others, December 2023: "Memory safety vulnerabilities are the most prevalent type of disclosed software vulnerability." and "About 70 percent of Microsoft common vulnerabilities and exposures (CVEs) are memory safety vulnerabilities (based on 2006-2018 CVEs)." Chromium: "around 70% of our serious security bugs are memory safety". Google on Android: "the percentage of memory safety vulnerabilities in Android dropped from 76% to 24% over 6 years as development shifted to memory safe languages", then in 2025 "memory safety vulnerabilities falling below 20% of total vulnerabilities for the first time". Rust is not a guarantee, since it lets you opt out: that same 2025 post reports a near miss, "a linear buffer overflow in CrabbyAVIF". These are companies reporting on their own code, and none of it is about agents writing the code. People moved those numbers by changing which language new code was written in.
Source: CISA, NSA and others, The Case for Memory Safe Roadmaps, 2023, https://www.ic3.gov/CSA/2023/231206.pdf
Source: Chromium, Memory safety, https://www.chromium.org/Home/chromium-security/memory-safety/
Source: Google, Eliminating memory safety vulnerabilities at the source, 2024, https://security.googleblog.com/2024/09/eliminating-memory-safety-vulnerabilities-Android.html
Source: Google, Rust in Android: move fast and fix things, 2025, https://security.googleblog.com/2025/11/rust-in-android-move-fast-fix-things.html
Source: Clang docs, AddressSanitizer, https://clang.llvm.org/docs/AddressSanitizer.html
Say it: "This C parser reads files from strangers. Build it with -fsanitize=address, fuzz it for ten minutes, and paste each crash with its stack. Don't fix anything yet. Then estimate what it would take to port only this parser to Rust, and list every place the port would still need unsafe."
Described as: "what is a buffer overflow", "it crashed with a segfault", "should we rewrite this in Rust", "what does memory-safe mean", "the agent wrote a native module".
https://dev.dqnotes.com/security/memory-safety

### Authentication
Also: authn, login.
Proving who someone is: a password, a code from their phone, a passkey. The result is usually a session, so the server recognises them on every later request.
Login looks simple and is full of traps, from resets to session handling to error messages that reveal which accounts exist. When an agent offers to write it from scratch, the safer answer is usually an established library.
Not Authorisation: Authentication asks who you are. Authorisation asks what you may do. A user can pass the first and should still fail the second.
Seen in: OWASP: "Multi-factor authentication (MFA) is by far the best defense against the majority of password-related attacks." NIST's current guidance wants a minimum of 15 characters for a password used alone, and says verifiers "SHALL NOT impose other composition rules".
Source: OWASP Cheat Sheet, Authentication, https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html
Source: NIST SP 800-63B-4, https://pages.nist.gov/800-63-4/sp800-63b.html
Say it: "Use the framework's session auth and don't write your own. Show the same error for a wrong password and an unknown email, require 15 characters with no composition rules, and rotate the session id at login."
Described as: "how does the site know who I am", "should I build my own login", "proving a user is really them".
https://dev.dqnotes.com/security/authentication

### Authorisation
Also: authz, authorization, permissions.
Deciding what a known user may do, on this thing, right now. The server has to decide on every request, because anything in the browser can be changed by the person using it.
It's the check that has no visible effect when missing. The honest user sees the same screen either way, so nothing in a normal demo gives it away.
Not Authentication: A valid login says nothing about whether this user may delete this record. "Logged in" is not the same as "allowed".
Seen in: OWASP: "Permission should be validated correctly on every request." And: "For security purposes an application should be configured to deny access by default."
Source: OWASP Cheat Sheet, Authorization, https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html
Say it: "Put the permission check in one middleware that denies by default, so each route has to say who may use it. A new route with no rule should return 403, and I want a test proving that."
Described as: "a normal user could do admin things", "who is allowed to do what", "being logged in is enough to do anything".
https://dev.dqnotes.com/security/authorisation

### Broken access control
The server fails to enforce what a user may do, so they can do it anyway: open an admin page, change a role, act on data that isn't theirs. It tops the OWASP list.
Hiding a button isn't a control, because the route behind it still answers. Agents often write the visible part and test the happy path, and static scanners struggle with this class, because no line of code is wrong. The bug is a line that isn't there.
Not Authentication failure: There the system is wrong about who you are. Here it knows exactly who you are and lets you do it anyway.
Seen in: A01 in OWASP Top 10:2025. It lists CORS misconfiguration and JWT tampering among the ways this happens.
Source: OWASP Top 10:2025, A01 Broken Access Control, https://top10.owasp.org/2025/A01_2025-Broken_Access_Control/
Say it: "Write a table test: every route against signed out, ordinary user and admin, with the expected status for each. Fail the build if a route is missing from the table, so the next one added can't be forgotten."
Described as: "I can see the admin page by typing the URL", "the button is hidden but the API still works", "a normal user changed their own role".
https://dev.dqnotes.com/security/broken-access-control

### IDOR
Also: insecure direct object reference, BOLA.
The kind of broken access control you can do with a keyboard. /invoices/1003 is yours, so you try /invoices/1004 and get a stranger's, because the server looks up the id and never asks whose it is.
A test that a user can read their own invoice passes with or without the check. You need a test with two users, and an agent doesn't write that unless asked.
Not Random ids: A UUID makes ids hard to guess, not safe to share. Links end up in logs, emails and histories. OWASP is clear that the check is still essential.
Seen in: OWASP gives the shape of the fix in one line: @current_user.projects.find(params[:id]), which searches only what the user owns. It adds that "even with complex identifiers, access control checks are essential".
Source: OWASP Cheat Sheet, Insecure Direct Object Reference Prevention, https://cheatsheetseries.owasp.org/cheatsheets/Insecure_Direct_Object_Reference_Prevention_Cheat_Sheet.html
Say it: "Look the invoice up through the signed-in user, with WHERE id = ? AND owner_id = ?, and return 404 if it isn't theirs. Add a test where user B requests user A's invoice and gets 404."
Described as: "changing a number in the URL showed someone else's data", "I can see other people's orders", "walking through ids one by one".
https://dev.dqnotes.com/security/idor

### Password hashing
Also: salt, Argon2id, bcrypt.
Storing a scrambled form of a password that can't be turned back. At login you scramble what they typed and compare. It has to be deliberately slow, and salted with a random value for each user.
One day your database may be stolen. Fast hashes like SHA-256 let an attacker try guesses at enormous speed, and without a salt, two people with the same password show up as the same value.
Not Encryption: Encryption can be reversed by anyone with the key, and then all the passwords are out. A password hash has no key and no way back. You can only guess and compare.
Seen in: OWASP: "Use Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism." Its figure for PBKDF2-HMAC-SHA256 is 600,000 iterations. The bench uses PBKDF2 because your browser has it built in, not because it's the better choice.
Source: OWASP Cheat Sheet, Password Storage, https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html
Say it: "Hash passwords with Argon2id, 19 MiB of memory, 2 iterations, parallelism 1. No SHA-256, no MD5. Add a test that two users with the same password get different stored values."
Described as: "how should I store passwords", "the database got leaked, what can they do", "why is login meant to be slow".
https://dev.dqnotes.com/security/password-hashing

### Encryption in transit and at rest
Also: TLS, AES-GCM, encryption at rest.
Scrambling data so only someone with the key can read it. In transit protects it on the wire, and that is what HTTPS does. At rest protects it on disk. Each covers one place, and neither stops a request your app treats as valid.
Saying "the data is encrypted" can be true and useless. A disk-encrypted database still hands plain text to a SQL injection. Ask who can still read it, not whether it is encrypted.
Not Password hashing: Encryption is meant to be undone by whoever holds the key. Use it for data you need back, and a password hash for passwords.
Seen in: OWASP: "The most commonly used authenticated modes are GCM and CCM, which should be used as a first preference." And: "Where possible, encryption keys should be stored in a separate location from encrypted data." On writing your own algorithm: "Don't do this."
Source: OWASP Cheat Sheet, Cryptographic Storage, https://cheatsheetseries.owasp.org/cheatsheets/Cryptographic_Storage_Cheat_Sheet.html
Say it: "Encrypt this token column with AES-256-GCM in the app before it reaches the database, with a fresh IV for every value and the key in our secret manager, not in the same database. Use the platform's crypto library, and add a test that a changed byte fails to decrypt."
Described as: "is the data safe if someone steals the disk", "what does encrypted at rest actually protect", "someone on the Wi-Fi can see the traffic".
https://dev.dqnotes.com/security/encryption

### JWT
Also: JSON Web Token, alg none.
A login token the server signs and the browser sends back. It has three parts: header, payload, signature. The payload is only encoded, so anyone can read it. The signature proves it hasn't been changed, if the server checks it properly.
The header says which algorithm was used, and the attacker writes the header. A verifier that believes it can be told no signature is needed. And a guessable key can be cracked offline, with nothing to notice it.
Not Session: With a session, the server keeps the record and the cookie is only a pointer, so you can end it instantly. A JWT carries its own claims, and it stays valid until it expires unless you build a way to revoke it.
Seen in: In 2015 Tim McLean wrote: "Unfortunately, some libraries treated tokens signed with the none algorithm as a valid token with a verified signature." RFC 8725 now says libraries "MUST enable the caller to specify a supported set of algorithms", and that memorable passwords "MUST NOT be directly used" as the key for HS256.
Source: McLean, Critical vulnerabilities in JSON Web Token libraries, Auth0, 2015, https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/
Source: RFC 8725, JSON Web Token Best Current Practices, 2020, https://www.rfc-editor.org/rfc/rfc8725.html
Say it: "Verify with a maintained library and pin algorithms to ['HS256'], with a key of 32 random bytes from the secret manager. Add tests that reject an edited payload and an alg: none token. Put nothing private in the payload, because it's readable."
Described as: "I decoded my login token and can read it", "what is the long string with two dots", "can a user change their role in the token".
https://dev.dqnotes.com/security/jwt

### Secret
Also: API key, credential, token.
A value that grants access: a password, an API key, a signing key. It belongs outside the code, in an environment variable or a secret manager, and it should only ever be readable by what needs it.
Agents want things to work, so a key pasted into a chat can end up in a config file, a test, a log line or a commit. Anything committed stays in the history, even after you delete the line.
Not Environment variable: That is one place to keep a secret, not a kind of secret. A .env file on disk is still a plain file, and an agent with file access can read it.
Seen in: GitHub's advice for a leaked secret starts: "then as a first step you need to revoke and/or rotate that secret." It warns that if you only rewrite your history and force push, "the commits with sensitive data may still be accessible elsewhere".
Source: GitHub docs, Removing sensitive data from a repository, https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository
Say it: "Read the key from process.env.STRIPE_KEY, add .env to .gitignore and commit a .env.example with the name and no value. Never print it, and if it's ever been committed, stop and tell me so that I can rotate it."
Described as: "I pasted my API key into the code", "I committed my password", "where should the key go".
https://dev.dqnotes.com/security/secret

### Secret scanning
Also: gitleaks, push protection.
A tool that searches your code and history for things that look like keys. It looks for known shapes, like an AWS key, and for strings that look too random to be ordinary text. That second measure is called entropy.
It's a cheap check that runs whether or not anybody remembers, and it catches the mistake before the push. It's also fallible: a password with no special shape and no keyword nearby goes straight past it.
Not Rotating a secret: Scanning tells you a secret is exposed. Only replacing it with a new one, and cancelling the old one, ends the exposure.
Seen in: This site runs gitleaks in its pre-commit hook. Gitleaks describes itself as "a tool for detecting secrets like passwords, API keys, and tokens in git repos, files", and rules can set a minimum entropy. GitHub's push protection is "designed to prevent hardcoded credentials, such as secrets or tokens, from ever being pushed to your repository."
Source: gitleaks, README, https://github.com/gitleaks/gitleaks
Source: GitHub docs, About push protection, https://docs.github.com/en/code-security/secret-scanning/introduction/about-push-protection
Say it: "Add gitleaks to the pre-commit hook and to CI, and run gitleaks git across the whole history once now. If it finds anything, list it and stop. Don't add an ignore rule without asking me, and treat any real hit as already leaked."
Described as: "how do I stop keys getting committed", "can a tool find passwords in my repo", "GitHub blocked my push because of a key".
https://dev.dqnotes.com/security/secret-scanning

### Least privilege
Giving each person, program and agent only the access its job needs, and no more. Then when one of them is fooled or compromised, the damage stops at what it could reach.
Broad access is the quickest way to make something work, so it's what tends to get set up, and it stays. This decides how bad a bad day gets, and it works even against attacks nobody has thought of yet.
Not Authorisation: Authorisation is the check at the door. Least privilege is deciding how few doors this key should open in the first place.
Seen in: OWASP calls it "assigning users only the minimum privileges necessary to complete their job". AWS blamed a July 2025 incident with its Amazon Q extension on an "inappropriately scoped GitHub token in their CodeBuild configuration", which let malicious code into a release. Krebs reported that the Capital One firewall had been "assigned too many permissions".
Source: OWASP Cheat Sheet, Authorization, https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html
Source: AWS security bulletin AWS-2025-015, https://aws.amazon.com/security/security-bulletins/AWS-2025-015/
Source: Krebs on Security, 2019, https://krebsonsecurity.com/2019/08/what-we-can-learn-from-the-capital-one-hack/
Say it: "Give this agent a read-only database user with access to the orders table only. The CI token gets contents: read, and only the release job gets a token that can publish. Show me each permission and what breaks if it's removed."
Described as: "why does this token have admin", "the agent has my whole AWS account", "how much access should this actually have".
https://dev.dqnotes.com/security/least-privilege

### CORS
Also: cross-origin resource sharing.
A browser rule: a script on one site can't read a response from another site, unless that site's server says it may. The server says so in a header, Access-Control-Allow-Origin. It only constrains browsers, so curl ignores it.
A CORS error is loud and blocks the feature, so the fastest way to silence it is to allow everything. That makes the error vanish and may open up logged-in data to any website. Look at what your agent did to make that error go away.
Not Authentication: CORS doesn't protect your server, and it can't stop a request from arriving. It only decides who gets to read the answer in a browser.
Seen in: MDN notes that the server "still must opt-in using Access-Control-Allow-Origin to share the response with the script", and that browsers block the wildcard for requests with credentials. So the dangerous version is a server that copies back whatever Origin it was sent. OWASP lists CORS misconfiguration under A01.
Source: MDN, Cross-Origin Resource Sharing, https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CORS
Source: OWASP Top 10:2025, A01 Broken Access Control, https://top10.owasp.org/2025/A01_2025-Broken_Access_Control/
Say it: "Allow only https://app.example.com. Don't echo back the Origin header, and don't use a wildcard. Add a test that a request from https://evil.example gets no Access-Control-Allow-Origin header in the response."
Described as: "the browser says blocked by CORS policy", "it works in curl but not in the browser", "can another website read my API".
https://dev.dqnotes.com/security/cors

### Content Security Policy
Also: CSP.
A header that tells the browser what this page may load and run: scripts only from here, no plug-ins, no framing. If an attacker's script gets into the page anyway, the browser can refuse to run it.
It's a second line of defence for when escaping fails somewhere. It's also easy to weaken: one 'unsafe-inline' added to quiet an error removes much of the protection, and an agent fixing a broken page may well do that.
Not Output encoding: Encoding stops the attack getting in. CSP limits what it can do if it does. MDN puts it plainly: "Setting a CSP is not an alternative to sanitizing input."
Seen in: This site's own policy allows 'unsafe-inline' and 'unsafe-eval' for scripts: the framework's bootstrap script differs on each page, and the benches run code in a blob: worker. Our headers file says the defence is that the site takes no input from other people and loads only one third-party script, Cloudflare's analytics beacon, allowed by its exact origin. That trade-off would be wrong for an app with user content. MDN: "Developers should avoid 'unsafe-inline', because it defeats much of the purpose of having a CSP."
Source: MDN, Content Security Policy, https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CSP
Say it: "Add a CSP with default-src 'self'; script-src 'self' 'nonce-…'; object-src 'none'; base-uri 'self'; frame-ancestors 'none'. Ship it as Content-Security-Policy-Report-Only first, then enforce. Don't add 'unsafe-inline' to fix an error. Tell me which script broke."
Described as: "the browser refused to run my script", "a header that says what the page can load", "how do I limit the damage from XSS".
https://dev.dqnotes.com/security/csp

### Security headers
Lines your server adds to each response that turn on browser protections: always use HTTPS, don't guess file types, don't let other sites frame this page, don't leak the full address when linking out. They take minutes to add.
No app needs to do without them, and none of them will appear unless someone asks, because the app works the same without them. That also makes them easy to check, with one curl.
Not Content Security Policy: CSP is one header, and the largest. The rest are smaller switches, each with one job.
Seen in: OWASP recommends X-Content-Type-Options: nosniff and Referrer-Policy: strict-origin-when-cross-origin, and says of the old X-XSS-Protection header: "Do not set this header or explicitly turn it off." This site sets its own in public/_headers, and the bench reads that policy.
Source: OWASP Cheat Sheet, HTTP Headers, https://cheatsheetseries.owasp.org/cheatsheets/HTTP_Headers_Cheat_Sheet.html
Say it: "Add Strict-Transport-Security, X-Content-Type-Options: nosniff, Referrer-Policy: strict-origin-when-cross-origin and frame-ancestors 'none'. Remove X-Powered-By. Add a test that fetches / and asserts each one, so that it can't quietly go."
Described as: "my security scan says headers are missing", "how do I stop my site being put in an iframe", "what should curl -I show".
https://dev.dqnotes.com/security/security-headers

### Rate limiting
Also: throttling, token bucket, 429.
Capping how many requests one person or address can make in a period, and answering 429 Too Many Requests after that. It makes guessing a password or a code slow enough to be pointless.
Without it, a 6-digit code falls to a script in a sitting. An endpoint that calls a paid model can also run up your bill overnight. It's invisible in testing, since nobody tests by sending a million requests.
Not Account lockout: Limiting slows guesses down. Locking the account after failed tries stops them, and it also lets a stranger lock out the real user on purpose. So you want a limit that recovers with time.
Seen in: NIST says to limit consecutive failed attempts on an account "to no more than 100". OWASP says to tie failed-login counters to the account, not the IP. OWASP also lists Unbounded Consumption as LLM10 for apps that call a model.
Source: NIST SP 800-63B-4, https://pages.nist.gov/800-63-4/sp800-63b.html
Source: OWASP Cheat Sheet, Authentication, https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html
Source: OWASP, Top 10 for LLM Applications 2025, https://genai.owasp.org/llm-top-10/
Say it: "Rate limit /login to 5 tries a minute per account and 100 per IP, in a store shared across servers. Return 429 with Retry-After. Add a test that the sixth try fails and that one account being attacked doesn't block a different account."
Described as: "someone is trying thousands of passwords", "too many requests error", "a bot ran up my bill".
https://dev.dqnotes.com/security/rate-limiting

### Supply chain attack
Also: dependency confusion, install script.
Attacking you through something you depend on. A package gets taken over or swapped, you install the update as usual, and its code runs with all the access your program or your laptop has.
A package can run code the moment you install it, with no import needed. A version range like ^1.4.0 accepts new releases on its own, and an agent adds dependencies with less hesitation than you would.
Not Vulnerability: A vulnerability is a mistake in a package that someone might exploit. A supply chain attack is malicious code put there deliberately.
Seen in: In March 2024, Andres Freund noticed slow SSH logins and traced them to a backdoor "present in the tarballs for 5.6.0 and 5.6.1" of xz, a compression library present on many Linux systems. Software supply chain failures became A03 in OWASP's 2025 list. npm's docs: "You should almost never have to explicitly set a preinstall or install script."
Source: Freund, oss-security, 29 March 2024, https://www.openwall.com/lists/oss-security/2024/03/29/4
Source: OWASP Top 10:2025, https://top10.owasp.org/2025
Source: npm docs, scripts, https://docs.npmjs.com/cli/v11/using-npm/scripts
Say it: "In CI, install with npm ci --ignore-scripts, so the lockfile decides the versions and its hashes are checked. Before you add any dependency, tell me its name, its owner, its age, and why ten lines of our own code wouldn't do."
Described as: "a package I use got hacked", "npm install ran something by itself", "can an update steal my keys".
https://dev.dqnotes.com/security/supply-chain

### Slopsquatting
A model invents a package name that sounds right. It fails to install, and that's a clue. Then somebody notices the invented name and registers it with their own code, and now the same mistake installs without a murmur.
The name only needs to be guessed once by an attacker and repeated by a model. An agent that installs what it thinks exists is how that turns into code running on your machine, with nobody having chosen it.
Not Typosquatting: Typosquatting bets on a person mistyping a real name. Slopsquatting bets on a model confidently inventing one.
Seen in: Socket credits the word to Seth Larson of the Python Software Foundation. We couldn't find the original post. A study of 576,000 code samples from 16 models found invented package names in at least 5.2% of samples from commercial models and 21.7% from open-source ones. Socket reports that, re-running prompts, 43% of the invented names came back every time. These are rates of invention. They don't measure how often anyone has been attacked this way.
Source: Socket, Slopsquatting, 2025, https://socket.dev/blog/slopsquatting-how-ai-hallucinations-are-fueling-a-new-class-of-supply-chain-attacks
Source: Spracklen et al., We Have a Package for You, arXiv 2406.10279, https://arxiv.org/abs/2406.10279
Say it: "Before you add a package, show me its registry page, first publish date and weekly downloads. If you can't find it, or it's less than 30 days old, stop and tell me. Don't install it to see whether it works."
Described as: "the agent imported a package that does not exist", "someone registered a package the AI made up", "npm install says 404 for a package the code needs".
https://dev.dqnotes.com/security/slopsquatting

### Dependency audit
Also: npm audit.
Checking the versions you have installed against a public list of known vulnerabilities. npm audit is one such command. It can only find what has already been reported.
It costs nothing and catches old, known problems. But it can be noisy, flagging code you never call, and it won't notice a brand-new malicious package at all. An agent told to make it clean may do something worse to get there.
Not Supply chain attack: An audit finds known mistakes in published versions. A fresh attack is unknown on the day it arrives, so an audit will pass it.
Seen in: npm warns that npm audit fix --force can "install modules outside your stated dependency range (including SemVer-major changes)", which is a real way to break an app while trying to tidy a report.
Source: npm docs, npm audit, https://docs.npmjs.com/cli/v11/commands/npm-audit
Say it: "Run npm audit --audit-level=high in CI. Don't use --force. For each high finding, tell me whether our code reaches the vulnerable function, what the upgrade changes, and which tests prove it still works."
Described as: "npm says I have 12 vulnerabilities", "should I run audit fix force", "how do I know my packages are safe".
https://dev.dqnotes.com/security/dependency-audit

### CVE
Also: CVSS, vulnerability id.
A public id for one known security flaw, such as CVE-2025-54136, so everyone can talk about the same bug. Its severity is usually given as a CVSS score from 0 to 10.
A score tells you how bad a flaw is in general. It doesn't know your app. A 9.8 in code you never call matters less than a 6.1 on your login page, so ask whether it's reachable before you ask about the number.
Not CVSS: A CVE is the name of the bug. CVSS is a formula for rating its severity. FIRST, which maintains it, says the score indicates severity "relative to other vulnerabilities".
Seen in: CVE-2025-54136 is the flaw Check Point named MCPoison, in Cursor, fixed in version 1.3. The CVE list is run by MITRE, with assigning organisations issuing the ids. CVSS 3.1 bands run: Low 0.1 to 3.9, Medium 4.0 to 6.9, High 7.0 to 8.9, Critical 9.0 to 10.
Source: Check Point Research, MCPoison, 2025, https://research.checkpoint.com/2025/cursor-vulnerability-mcpoison/
Source: Wikipedia, Common Vulnerabilities and Exposures, https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures
Source: FIRST, CVSS v3.1 specification, https://www.first.org/cvss/v3.1/specification-document
Say it: "This CVE is rated 9.8. Before you upgrade anything, show me whether our code reaches the vulnerable function, with the call path, and what an attacker would need. Then give me your own view of the priority."
Described as: "what does CVE-2025 something mean", "how serious is this vulnerability", "a scanner says critical, should I panic".
https://dev.dqnotes.com/security/cve

### SAST and DAST
Also: static analysis, dynamic scan, OWASP ZAP.
Two kinds of automated security check. SAST reads your code without running it. DAST runs your app and attacks it from the outside. Each finds things the other can't.
Both are ready-made sensors for an agent loop. Both miss things: OWASP says static tools have "High numbers of false positives" and struggle with authentication and access control. So a clean scan isn't proof, and in particular it says little about whose data a user can reach.
Not Security review: A scanner matches patterns or fires probes. A review reasons about what this code is for, which is what catches a check that was never written.
Seen in: OWASP ZAP is a free DAST tool. Its docs warn: "Active scanning is a real attack on those targets and can put the targets at risk." And: "You should only use ZAP to attack an application you have permission to test with an active attack."
Source: OWASP, Source Code Analysis Tools, https://community.owasp.org/Source_Code_Analysis_Tools
Source: ZAP, Getting started, https://www.zaproxy.org/getting-started/
Say it: "Add a static scan to CI that fails on new high findings only. Then run ZAP against the local dev server on localhost, and nowhere else. For each finding, show a request that demonstrates it, or mark it as a false positive and say why."
Described as: "a tool that scans my code for security bugs", "what is the difference between static and dynamic scanning", "can I test my app by attacking it".
https://dev.dqnotes.com/security/sast-dast

### Prompt injection
Also: indirect prompt injection.
Text gives a model orders it wasn't meant to take. A model reads your instructions and a web page as one stream of words, so a line in that page can say what to do next. Indirect means it was planted somewhere the agent would read.
Your agent reads issues, documents and web pages all day, and each could be written by anyone. You can't filter this out completely, so the job is deciding what the agent is allowed to do once it's fooled.
Not Jailbreaking: A jailbreak is a user talking a model out of its rules. Prompt injection is an attacker slipping orders into an app built on a model. Willison, who named it, calls them "a different issue".
Seen in: Simon Willison named it in September 2022, writing "The obvious parallel here is SQL injection", and later judged a parameterised fix "extremely difficult, if not impossible, to implement". OWASP: "it is unclear if there are fool-proof methods of prevention". EchoLeak (CVE-2025-32711) used a crafted email against Microsoft 365 Copilot with no click needed.
Source: Willison, Prompt injection attacks against GPT-3, 2022, https://simonwillison.net/2022/Sep/12/prompt-injection/
Source: Willison, The lethal trifecta for AI agents, 2025, https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
Source: OWASP, LLM01:2025 Prompt Injection, https://genai.owasp.org/llmrisk/llm01-prompt-injection/
Source: Reddy and Gujral, EchoLeak, arXiv 2509.10540, https://arxiv.org/abs/2509.10540
Say it: "This agent reads web pages, so treat everything it fetches as hostile. Run that step in a session with no credentials and no write tools, have it return only a summary in a fixed format, and require my approval before anything is sent."
Described as: "a web page told my agent what to do", "hidden text in a document changed what the AI did", "can someone hack the agent with an email".
https://dev.dqnotes.com/security/prompt-injection

### Lethal trifecta
Simon Willison's name for three powers that are dangerous together: access to your private data, exposure to untrusted content, and a way to send data out. With all three, a stranger's text can steal your data. Remove any one and that attack has no route.
It turns an unsolved problem into a design question you can answer today. You can't guarantee a model won't be fooled, but you can list its tools and see whether all three legs are there.
Not Least privilege: Least privilege trims each power on its own. The trifecta warns about a combination, in which each power might look reasonable alone.
Seen in: Willison, June 2025: "If your agent combines these three features, an attacker can easily trick it into accessing your private data and sending it to that attacker." Filters don't settle it, the post argues, because "in web application security 95% is very much a failing grade". Invariant Labs showed a planted public GitHub issue leading an agent to leak a private repo through a pull request.
Source: Willison, The lethal trifecta for AI agents, 2025, https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
Source: Invariant Labs, GitHub MCP vulnerability, 2025, https://invariantlabs.ai/blog/mcp-github-vulnerability
Say it: "This agent reads support tickets, which is untrusted, and it can read the customer table, which is private. So remove its web and email tools, restrict the network to our own API, and have its replies go to a draft a person sends."
Described as: "is my agent setup dangerous", "which combination of tools is risky", "how could my data get out through the agent".
https://dev.dqnotes.com/security/lethal-trifecta

### Tool poisoning
Also: rug pull, MCP poisoning.
An MCP tool's description is read by the model, and you usually see a short version. A malicious server hides instructions in it, such as read this file and pass it along. The tool can also change after you approved it, which is called a rug pull.
Installing an MCP server means letting its author write into your agent's instructions. Nothing has to be run for that to happen: the text only has to be read.
Not Prompt injection: Tool poisoning is a form of prompt injection. What sets it apart is the place: the tool list, which the model tends to treat as part of its own setup.
Seen in: Invariant Labs, April 2025: "malicious instructions are embedded within MCP tool descriptions that are invisible to users but visible to AI models." Its demo was an add tool that read ~/.ssh/id_rsa and passed it in an argument. Its advice: "Clients should pin the version of the MCP server and its tools to prevent unauthorized changes." Check Point's MCPoison showed Cursor trusting a server config by its key name, so its contents could change after approval.
Source: Invariant Labs, Tool Poisoning Attacks, 2025, https://invariantlabs.ai/blog/mcp-security-notification-tool-poisoning-attacks
Source: Check Point Research, MCPoison, 2025, https://research.checkpoint.com/2025/cursor-vulnerability-mcpoison/
Say it: "Before we add this MCP server, print the full description and parameters of every tool, and flag any text addressed to the model, any file path, and any unexplained parameter. Pin its version, and tell me if a description ever changes."
Described as: "can an MCP server be malicious", "the tool description has hidden instructions", "the tool changed after I approved it".
https://dev.dqnotes.com/security/tool-poisoning

### OWASP Top 10 for LLM apps
Also: OWASP LLM Top 10, excessive agency.
A separate OWASP list for apps built on language models. Prompt injection is first. It also covers leaks of private data, poisoned models, trusting model output too readily, and giving a model too much power.
Two entries matter most for agents. Improper Output Handling: model output is untrusted input, so paste it into a query or a page and you're back to injection. Excessive Agency: a model with more tools than it needs.
Not OWASP Top 10: That list covers web applications generally. This one covers what is new when a model is in the loop. An app with a model in it needs both.
Seen in: The 2025 list: LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM03 Supply Chain, LLM04 Data and Model Poisoning, LLM05 Improper Output Handling, LLM06 Excessive Agency, LLM07 System Prompt Leakage, LLM08 Vector and Embedding Weaknesses, LLM09 Misinformation, LLM10 Unbounded Consumption.
Source: OWASP, Top 10 for LLM Applications 2025, https://genai.owasp.org/llm-top-10/
Say it: "Go through the OWASP LLM Top 10 for this feature, starting with LLM01, LLM05 and LLM06. Show where model output reaches a query, a page or a shell, and list each tool the model has with the reason it needs it. Remove any without one."
Described as: "is there a security checklist for AI apps", "what can go wrong with a chatbot on my site", "should I trust what the model returns".
https://dev.dqnotes.com/security/owasp-llm

### Sandboxing an agent
Having the operating system fence in an agent's commands: which files they can touch and which other machines they can reach. It holds even if the agent is fooled, because it's enforced from outside the model.
Instructions are requests, and a sandbox is a wall. It only works as a pair, too. A filesystem rule without a network rule lets a key be read and sent. A network rule without a filesystem rule still leaves your machine open to change.
Not Permission mode: A permission mode is the agent's own software asking before it acts, and you can get tired of the prompts. A sandbox is enforced by the operating system, whatever gets approved.
Seen in: Claude Code's docs are specific about the limits: "The sandbox covers shell commands only. Claude's file tools, MCP servers, and hooks run outside it." By default reads cover "Most of the machine, including credential files such as ~/.ssh and ~/.aws/credentials". Codex offers --sandbox read-only, workspace-write and danger-full-access.
Source: Claude Code docs, Sandboxing, https://code.claude.com/docs/en/sandboxing
Source: OpenAI Codex docs, Sandboxing, https://learn.chatgpt.com/docs/agent-approvals-security
Say it: "Run this agent with the sandbox on, deny reads on ~/.ssh and ~/.aws, and allow network only to registry.npmjs.org. Then test it: have it try to cat a fake key and curl an address off the list, and show me both refusals."
Described as: "how do I stop the agent reading my keys", "can I run it so it cannot reach the internet", "what does the sandbox actually block".
https://dev.dqnotes.com/security/agent-sandboxing

### How often AI code is insecure
Also: Veracode GenAI report.
Studies ask models to write code for risky tasks and scan the result. Numbers near half failing get quoted widely. They come from particular tests with particular limits, so they're a warning that this happens, not a rate for your codebase.
If you quote the headline as a fact, a sceptical listener will take it apart. The defensible claim is smaller and more useful: models often write insecure code when nobody asks them for secure code, and passing tests won't tell you.
Not Your own measurement: A vendor benchmark tells you something about models on its prompts. What your own scan finds on your own repo tells you about your code.
Seen in: Veracode reported that 45% of samples failed in 2025, from over 100 models, and about 44% in 2026. Veracode sells scanners, we only read the blog posts, the code was generated in single shots on security-sensitive prompts and scored by its own static analysis, and no agent loop was involved. Our own recorded comment renderers: 7 of 29 vague-brief programs produced a javascript: link from one added attack and 0 of 29 with the right words, with the harness making no difference. That's one small model, one function and 15 runs per cell, and we added the attack afterwards.
Source: Veracode (a vendor), 2025 GenAI Code Security Report, blog post, https://www.veracode.com/blog/genai-code-security-report/
Source: Veracode (a vendor), 2026 GenAI Code Security Report, blog post, https://www.veracode.com/blog/2026-genai-code-security-report-ai-risk
Say it: "Render comments as Markdown. The threats are script tags, event handler attributes and javascript: links, including ones hidden by control characters. Allow only https: and mailto: links, escape everything else, and add a test for each threat."
Described as: "what percent of AI code has security bugs", "is AI-written code less secure", "someone quoted a scary statistic about Copilot".
https://dev.dqnotes.com/security/ai-code-vulnerability

### Security review
Reading a change only for how it could be abused, separately from whether it works. It asks: who can reach this, what can they send, and what's the worst they can do? A person or a fresh agent can do it.
The agent that wrote the code reads it as an author. A second pass with a narrow brief catches what a general review doesn't, above all a check that was never written, which no scanner can point at.
Not Code review: A code review asks whether this is correct, clear and worth merging. A security review assumes hostile input and looks only for that, so its findings need a demonstration, not an opinion.
Seen in: Claude Code has a built-in /security-review, which it describes as: "Analyze the changes on your current branch for security vulnerabilities." It reviews the diff, so it's scoped to what changed, not to the rest of your app. Use it as one more pass, not as a sign-off.
Source: Claude Code docs, Commands, https://code.claude.com/docs/en/commands
Say it: "Review this branch for security only, in a fresh session. For every new route, say who can call it, what they can send and whose data it touches. For each finding, give the file, the line, and a request that demonstrates it. Ignore style."
Described as: "how do I check the agent did not add a hole", "who reviews this for security", "I want a second look only for attacks".
https://dev.dqnotes.com/security/security-review

## 14. Data and failure

Databases, transactions, migrations, concurrency and retries: the failures that only show up in production.

### Database
A program whose job is to keep data safe and find it again quickly, while many other programs read and change it at once. Your app asks it questions over a connection. If your app crashes, or the database itself does, what it said it saved is still there.
Agents reach for the nearest thing that works: a JSON file, an array, a local SQLite file. All of those pass a demo with one user. The difference only appears with two copies of your server, two requests at once, or a crash. So choosing the store is your decision, not a detail to leave to the agent.
Not A file: A file gives you bytes. A database gives you rules about those bytes: who can change what at the same time, what happens in a crash, and what shape the data has to be.
Seen in: PostgreSQL's docs state the promise: "A transactional database guarantees that all the updates made by a transaction are logged in permanent storage (i.e., on disk) before the transaction is reported complete." A file you wrote to makes you no such promise.
Source: PostgreSQL docs, Transactions, https://www.postgresql.org/docs/current/tutorial-transactions.html
Say it: "Don't store orders in a JSON file or in memory. Use Postgres with the connection string from DATABASE_URL. Then prove it survives: create an order, kill the server with kill -9, restart it, and show me the row."
Described as: "where should the data actually live", "can I just use a JSON file", "two copies of the server see different data", "the data vanished when it crashed".
https://dev.dqnotes.com/data/database

### Table and row
A table is a named collection of records that all have the same fields, like a strict spreadsheet. A row is one record, such as one order. A column is one field, such as its total, and every value in it is the same kind of thing.
Agents copy the shape of the objects in their code straight into tables, often one big table with a text column full of JSON. It runs, and then it can't be searched, checked or joined. Describing the tables yourself is one of the cheapest ways to steer the whole design.
Seen in: PostgreSQL's docs: "Each table is a named collection of rows." And a trap worth knowing: "SQL does not guarantee the order of the rows within the table in any way". A list page with no ORDER BY can pass every test on a small table, then change order in production.
Source: PostgreSQL docs, Concepts, https://www.postgresql.org/docs/current/tutorial-concepts.html
Say it: "Use three tables: customers, orders and order_items, with one row per thing and no JSON columns for data we filter on. Every list query needs an ORDER BY, ending in the primary key so that the order is stable."
Described as: "it is like a spreadsheet but stricter", "the list comes back in a different order each time", "it put everything in one big JSON column", "how should the data be laid out".
https://dev.dqnotes.com/data/table-row

### Schema
Also: Constraints.
The declared shape of your data: which tables exist, which columns they have, what type each holds, and the rules each must obey, such as not empty, or never below zero. The database enforces those rules on every write, whoever is writing.
A rule in your code protects the one path that has it. An agent adding a new route, a script or a backfill won't know the rule is there. A constraint in the schema holds against all of them, including the agent itself. It is backpressure that lives in the data.
Not Validation in the app: Validation in your code gives a friendly message on one path. A constraint is the last word on every path. Have both: the first for the person, the second for the truth.
Seen in: PostgreSQL's docs: "If a user attempts to store data in a column that would violate a constraint, an error is raised." So a bad write arrives at your agent as an error message it can read, at the moment it happened, and not as a strange row found three months later.
Source: PostgreSQL docs, Constraints, https://www.postgresql.org/docs/current/ddl-constraints.html
Say it: "Add these to the schema, not just the code: total_cents integer NOT NULL CHECK (total_cents >= 0), status text NOT NULL CHECK (status IN ('pending', 'paid', 'refunded')). Then write a test that inserts a bad row with raw SQL and expects an error."
Described as: "bad data keeps getting into the table", "how do I stop a negative price being saved", "the rule is in the code but a script got round it", "the shape of the database".
https://dev.dqnotes.com/data/schema

### Primary key
The column that identifies one row and only one, for ever. It must be present and it must be unique. Usually it's a number or a random id with no meaning, because anything with meaning, like an email address, can change.
An agent asked to model users may reach for the email as the key, which works until someone changes their email and every link to them breaks. It may also pick counting numbers for public URLs, which tells anyone how many orders you have and lets them guess the next one. Say what you want.
Not Unique constraint: A table has one primary key, and it can't be empty. It can also have many unique constraints, on things like an email address that are unique but aren't the row's identity.
Seen in: PostgreSQL's docs say a primary key "requires that the values be both unique and not null", and that adding one "will automatically create a unique B-tree index". You will meet that tree properly in a moment.
Source: PostgreSQL docs, Constraints, https://www.postgresql.org/docs/current/ddl-constraints.html
Say it: "Give every table a primary key column called id, generated by the database, with no business meaning. Don't use email as a key. Use a random public id in URLs, so nobody can count our orders or guess the next one."
Described as: "what identifies a row", "two rows are exactly the same and I can not delete one", "the customer changed their email and everything broke", "should the id be a number".
https://dev.dqnotes.com/data/primary-key

### Foreign key
A column in one table that must match the primary key of a row in another. An order's customer_id has to point at a real customer. The database refuses anything that would leave it pointing at nothing, and you choose what happens when the row it points at is deleted.
Without one, nothing stops an order pointing at a customer who no longer exists, and an agent writing a clean-up script will happily create thousands of them. With one, choose the delete rule yourself. An agent adding CASCADE to fix an error has decided that deleting one customer may silently delete their whole history.
Seen in: PostgreSQL's docs call this keeping "the referential integrity" of your data. About the delete rules: "CASCADE specifies that when a referenced row is deleted, row(s) referencing it should be automatically deleted as well." The default refuses the delete, which is usually what you want.
Source: PostgreSQL docs, Constraints, https://www.postgresql.org/docs/current/ddl-constraints.html
Source: PostgreSQL docs, Foreign keys, https://www.postgresql.org/docs/current/tutorial-fk.html
Say it: "Make orders.customer_id a foreign key to customers.id with ON DELETE RESTRICT. Don't add CASCADE to make an error go away. If a customer must be deletable, ask me what should happen to their orders first."
Described as: "the order points at a customer who does not exist", "deleting one thing deleted lots of other things", "cannot delete because something still refers to it", "link two tables together".
https://dev.dqnotes.com/data/foreign-key

### Unique constraint
A rule that no two rows may have the same value in a column, or in a set of columns. The database checks it as each row goes in, so it holds even when two people try in the same millisecond. The loser gets an error.
Asked to stop duplicate signups, an agent writes the obvious thing: look for the email, and insert if it isn't there. That passes every test, because tests run one at a time. In production, two requests can both look before either inserts. The only check that can't be raced is the one inside the database.
Not Check then insert: SELECT and then INSERT in your code is two steps, with a gap between them. A unique constraint is one step, with no gap. Keep the SELECT if you want a nicer message, but it doesn't protect anything.
Seen in: PostgreSQL's docs: "Unique constraints ensure that the data contained in a column, or a group of columns, is unique among all the rows in the table." To turn the error into a decision, INSERT has ON CONFLICT, and "ON CONFLICT DO UPDATE guarantees an atomic INSERT or UPDATE outcome".
Source: PostgreSQL docs, Constraints, https://www.postgresql.org/docs/current/ddl-constraints.html
Source: PostgreSQL docs, INSERT, https://www.postgresql.org/docs/current/sql-insert.html
Say it: "A SELECT before the INSERT won't stop duplicates under concurrency. Add a unique constraint on users.email, catch the duplicate key error and return a 409. Add a test that fires 20 signups with the same email at once with Promise.all, and asserts one row."
Described as: "we got two accounts with the same email", "it checks first and still creates duplicates", "only happens when people double click", "make sure this can only exist once".
https://dev.dqnotes.com/data/unique-constraint

### SQL
The language for asking a relational database for things. You describe what you want, such as which rows, in what order and how many, and the database works out how to get it. SELECT reads. INSERT, UPDATE and DELETE change. It looks like English, and small mistakes in it can be enormous.
You don't need to write much SQL when an agent writes it for you. You do need to read what it wrote, especially the part after WHERE, because an UPDATE or a DELETE with the wrong condition, or with none, runs without complaint. Ask to see the statement before it runs against anything you care about.
Not A database: SQL is the language. Postgres, MySQL and SQLite are programs that speak it, each with its own dialect and its own behaviour under load. Code that works on one may not work on another.
Seen in: PostgreSQL's docs on DELETE FROM tablename: "Without a qualification, DELETE will remove all rows from the given table, leaving it empty. The system will not request confirmation before doing this!" The bench below is a small real SQL engine, so go ahead and try it. It is a copy, and Reset brings everything back.
Source: PostgreSQL docs, Deletions, https://www.postgresql.org/docs/current/tutorial-delete.html
Say it: "Show me the exact SQL before you run it. For any UPDATE or DELETE, first run a SELECT count(*) with the same WHERE, and tell me how many rows it would change. Wrap it in BEGIN, and don't COMMIT until I say so."
Described as: "what is the code talking to the database", "SELECT star from", "it updated every row instead of one", "how do I read what the query does".
https://dev.dqnotes.com/data/sql

### Query plan
Also: EXPLAIN.
The steps the database chose to answer your query: which table to read first, whether to read all of it or use a shortcut, and how to join. You write what you want, and a part of the database called the planner decides how. EXPLAIN shows you its decision.
An agent asked why a page is slow will guess, usually by rewriting code that isn't the problem. The plan isn't a guess. And it can change by itself as a table grows, so a query that was fine on your laptop's 100 rows can be a different query on production's 10 million.
Not Running the query: EXPLAIN only shows the plan. EXPLAIN ANALYZE really runs the query and shows what happened, which is more useful, and on an UPDATE or a DELETE it really changes the data.
Seen in: PostgreSQL's docs: "Keep in mind that because EXPLAIN ANALYZE actually runs the query, any side-effects will happen as usual". They suggest wrapping it in BEGIN and ROLLBACK. Costs are "measured in arbitrary units", so compare them with each other and not with a clock. The planner on this bench is ours, and far simpler than theirs.
Source: PostgreSQL docs, Using EXPLAIN, https://www.postgresql.org/docs/current/using-explain.html
Say it: "Don't guess why this is slow. Run EXPLAIN (ANALYZE, BUFFERS) on the query against a copy with production-sized data, and paste the plan. Tell me which node takes the time, and how many rows it read compared with how many it returned."
Described as: "why is this query slow", "how do I see what the database is doing", "it was fast until the table got big", "it ignored my index".
https://dev.dqnotes.com/data/query-plan

### Index
A separate sorted structure that lets the database find rows without reading the whole table, like the index at the back of a book. The usual kind is a B-tree, a wide shallow tree you can walk from top to bottom in a few steps, however many rows there are.
On your laptop with 100 rows, a missing index is invisible, so no test or demo will show it. Agents rarely add one unless asked. But they also over-add when asked to make things fast, and every index is more work on every write. Ask for the index a real query needs, and for the plan that proves it gets used.
Not A cache: A cache is a copy that can go stale, and you decide when to refresh it. An index is kept exactly up to date by the database in the same transaction as the write, which is what you pay for on every write.
Seen in: PostgreSQL's docs: without one, "the system would have to scan the entire test1 table, row by row". With one, it "might only have to walk a few levels deep into a search tree". The price: "This adds overhead to data manipulation operations." And building one is a production event of its own, as a later term covers.
Source: PostgreSQL docs, Indexes: introduction, https://www.postgresql.org/docs/current/indexes-intro.html
Say it: "The orders page filters on customer_id and sorts by created_at. Add an index on (customer_id, created_at) with CREATE INDEX CONCURRENTLY. Show me EXPLAIN ANALYZE before and after on a table with a million rows, and don't add any other indexes."
Described as: "it was fast in testing and is slow in production", "the search gets slower as we get more customers", "how does it find a row so quickly", "the database is at 100% on one query".
https://dev.dqnotes.com/data/index

### N+1 query
One query to fetch a list, then one more for each item in it. A page of 50 orders that looks up each order's customer separately makes 51 trips to the database. Each is fast, and the page is slow. One query can usually fetch them all.
This is the signature performance bug of agent-written code, because the code reads perfectly: loop over the orders, show each customer's name. It passes with 3 rows in the test database. Only a real list shows it, and only if something is counting queries.
Not A slow query: A slow query is one bad trip, and you fix it with an index. N+1 is many good trips, so no single query ever looks wrong, and a slow-query log may never mention it.
Seen in: The Rails guides: "The above code executes 1 (to find 10 books) + 10 (one per each book to load the author) = 11 queries in total." With includes, "The above code will execute just 2 queries". Prisma's docs: "The n+1 problem occurs when looping through query results and performing one additional query per result."
Source: Rails guides, Active Record query interface, https://guides.rubyonrails.org/active_record_querying.html
Source: Prisma docs, Query optimisation, https://www.prisma.io/docs/orm/prisma-client/queries/advanced/query-optimization-performance
Say it: "This list page has an N+1. Fetch the customers in one query with WHERE id IN (...) or a join. Add a test that loads the page with 50 orders, counts the SQL statements, and fails if there are more than 3."
Described as: "the page gets slower the more rows it shows", "hundreds of tiny queries in the log", "each query is fast but the page is slow", "it fetches them one at a time in a loop".
https://dev.dqnotes.com/data/n-plus-one

### ORM
Also: Object-relational mapper.
A library that lets you work with database rows as objects in your language, and writes the SQL for you. Prisma, Drizzle, Django and Active Record have one. It's convenient, and it hides how many queries a single innocent line of code will run.
An agent writes ORM code fluently, and the ORM makes the expensive thing look exactly like the cheap thing: order.customer.name might be a field you already have, or a trip to the database. You can't tell from the diff. So turn on query logging and read the SQL it really sends.
Not The query language: An ORM writes SQL. It doesn't replace knowing what SQL got written, and when the query matters you can usually drop down and write it yourself.
Seen in: Django's docs describe the fix for that hidden trip. select_related "works by creating an SQL join", so the related object arrives "in the same database query". Their example marks the difference in comments: one line "Hits the database", and the next "Doesn't hit the database". prefetch_related is the other route, using "one extra batch query".
Source: Django docs, QuerySet API, https://docs.djangoproject.com/en/stable/ref/models/querysets/
Say it: "Turn on SQL query logging for this test. Load the orders page and paste every statement it runs. Then use the ORM's eager loading so the customer arrives with the order, and show me the log again."
Described as: "the library that writes the SQL for me", "one line of code ran fifty queries", "I can not see what it is sending to the database", "should I write raw SQL".
https://dev.dqnotes.com/data/orm

### Transaction
A group of database changes that succeed or fail as one. You BEGIN, make the changes, then COMMIT. If anything fails before the COMMIT, or the power goes, none of them happened. Moving money is the classic case: the debit and the credit must both happen, or neither.
Agents write each statement correctly and often don't wrap them, because every test passes without it. Tests don't crash between two lines. Production does, on a deploy, an out-of-memory kill or a lost connection. Look at any function with two writes in it and ask what happens if it dies between them.
Not A try/catch: A catch block runs if your code gets an error. It doesn't run if the machine loses power. A transaction is undone by the database itself on the way back up.
Seen in: PostgreSQL's tutorial: "The essential point of a transaction is that it bundles multiple steps into a single, all-or-nothing operation." Its example is a bank transfer: "It would certainly not do for a system failure to result in Bob receiving $100.00 that was not debited from Alice."
Source: PostgreSQL docs, Transactions, https://www.postgresql.org/docs/current/tutorial-transactions.html
Say it: "Creating the order and reducing the stock must happen together, so wrap both in one database transaction. Add a test that throws an error between the two writes and asserts neither is in the table afterwards. Don't call any external API inside the transaction."
Described as: "it crashed halfway and now the data is wrong", "the money left one account and never arrived", "make these changes all or nothing", "half of it saved".
https://dev.dqnotes.com/data/transaction

### ACID
Four promises a transaction makes. Atomic: all of it or none. Consistent: your rules still hold afterwards. Isolated: others don't see it half-done. Durable: once it's committed it stays, through a crash. It's a word you'll hear in interviews, and each letter is something you can break.
Agents will pick a data store for you, and some of the popular ones give up one of these letters by default or by configuration. Knowing the four lets you ask a real question: what happens to this write if the machine dies a millisecond after you told the user it worked?
Not The consistency in CAP theorem talk: They share a letter and not a meaning. The C in ACID means your rules hold. The C in CAP means that every read sees the latest write. This chapter gets to CAP later.
Seen in: PostgreSQL's glossary defines all four, and durability as "The assurance that once a transaction has been committed, the changes remain even after a system failure or crash." Its log is how: "changes to data files (where tables and indexes reside) must be written only after those changes have been logged". Wikipedia credits the acronym to Reuter and Härder in 1983, building on Jim Gray, which we took from there and not from the paper.
Source: PostgreSQL docs, Glossary, https://www.postgresql.org/docs/current/glossary.html
Source: PostgreSQL docs, Write-ahead logging, https://www.postgresql.org/docs/current/wal-intro.html
Source: PostgreSQL docs, Asynchronous commit, https://www.postgresql.org/docs/current/wal-async-commit.html
Source: Wikipedia, ACID, https://en.wikipedia.org/wiki/ACID
Source: Kleppmann, Please stop calling databases CP or AP, 2015, https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-cp-or-ap.html
Say it: "Before you pick a store for payments, tell me for each ACID letter what it guarantees and under what settings. In particular, is a write on disk before the client hears yes? Quote the docs for the version we would use."
Described as: "what does ACID stand for", "is the data safe if the power goes", "it said saved and then it was gone", "what guarantees does this database give me".
https://dev.dqnotes.com/data/acid

### Isolation level
How much one transaction can see of others running at the same moment. Stronger levels hide more, and may refuse a transaction and make it retry. Weaker levels let strange things through. Dirty read, non-repeatable read and lost update are the names of those strange things.
Every isolation bug needs two transactions at once, so your tests will never hit one, and the agent can't reason its way to one unless you ask. And the default may not be what you assume. Say which level a piece of code relies on, and if you go stronger, say what retries.
Not Thread safety: A lock in your code protects one process. Your database is shared by every copy of your server, so a lock in one copy protects nothing against another.
Seen in: PostgreSQL: "Read Committed is the default isolation level in PostgreSQL." MySQL: "The default isolation level for InnoDB is REPEATABLE READ." So the same code can behave differently on each. Postgres also says its "Read Uncommitted mode behaves like Read Committed", so our bench, which does let you see a dirty read, follows the standard there and not Postgres. Stronger levels have a cost: "Applications using this level must be prepared to retry transactions due to serialization failures."
Source: PostgreSQL docs, Transaction isolation, https://www.postgresql.org/docs/current/transaction-iso.html
Source: MySQL docs, Transaction isolation levels, https://dev.mysql.com/doc/refman/8.4/en/innodb-transaction-isolation-levels.html
Say it: "This reads a balance and then writes it, so at Read Committed it can lose an update. Run it at REPEATABLE READ, catch SQLSTATE 40001 and retry the whole transaction up to 3 times. Add a test with two concurrent connections that shows the retry happening."
Described as: "two users at once see weird results", "the number changed in the middle of my report", "the database told it to retry the transaction", "it only happens under load".
https://dev.dqnotes.com/data/isolation

### Race condition
A bug where the result depends on who gets there first. Two people buy the last ticket: both read one left, both subtract one, both write zero. Two sales, one ticket. Each step was right, and the bug was the order they ran in.
An agent's code is usually tested by running one thing at a time, which is the one order that never goes wrong. So a race can sit in green code for months, and show up only when traffic grows. You have to ask for it by name, and for a test that really runs things at once.
Not A flaky test: A flaky test may be caused by a race, but in your test. A race in your product is a real bug that a test only sometimes catches, and re-running until it goes green hides it.
Seen in: Amazon's own account of its October 2025 outage in Northern Virginia: "The root cause of this issue was a latent race condition in the DynamoDB DNS management system". A delayed process applied an old plan over a newer one, and a clean-up then removed the plan in use, so "all IP addresses for the regional endpoint were immediately removed". Their word is latent: it had been there, waiting for an unlucky order.
Source: AWS, Summary of the Amazon DynamoDB service disruption, https://aws.amazon.com/message/101925/
Say it: "This reads the stock and then writes it, which is a race. Write a test that runs 50 purchases at once against a real Postgres, not a mock, with stock set to 10. It must end with exactly 10 sold and stock at 0. Show it failing on the current code first."
Described as: "it only goes wrong when two people do it at once", "we sold more than we had", "the count is lower than it should be", "I can not reproduce it on my machine", "two requests overwrote each other".
https://dev.dqnotes.com/data/race-condition

### Locks and atomic updates
Also: SELECT FOR UPDATE, Optimistic locking.
Three ways out of a race. An atomic update does the maths inside one statement: SET stock = stock - 1. A lock makes others wait: SELECT ... FOR UPDATE. A version check lets everyone try and makes the loser start again. Try them in that order.
Asked to fix a race, an agent often adds a lock inside the program, which works perfectly until you run two copies of the server. The fix has to live where the shared data lives. And the simplest fix is often just moving the arithmetic into the UPDATE.
Not A mutex in your code: A mutex lives in one process. Your second server has its own. A row lock lives in the database, and everyone has to go through that.
Seen in: PostgreSQL on FOR UPDATE: it "causes the rows retrieved by the SELECT statement to be locked as though for update." And reading stays open: "Row-level locks do not affect data querying; they block only writers and lockers to the same row." The other side of it: "it is a bad idea for applications to hold transactions open for long periods of time".
Source: PostgreSQL docs, Explicit locking, https://www.postgresql.org/docs/current/explicit-locking.html
Say it: "Replace the read and write with one statement: UPDATE products SET stock = stock - 1 WHERE id = $1 AND stock > 0, and check the row count. Zero rows means sold out. Don't add an in-process mutex, because we run more than one server."
Described as: "how do I stop them overwriting each other", "we added a lock and it still happens", "make the second one wait", "I need to read it and then change it safely".
https://dev.dqnotes.com/data/atomic-update

### Deadlock
Two transactions each hold something the other wants, so neither can move. One locked row A and wants B, and the other locked B and wants A. They would wait for ever, so the database notices the cycle, kills one of them, and that one has to try again.
Locks fix races and make deadlocks possible, and an agent fixing the first won't think about the second. They appear only under real concurrency, as an occasional error nobody can reproduce. The fix is boring: take locks in the same order everywhere, and make sure anything that can be killed can retry.
Not A slow query: A slow query is making progress. A deadlock is making none, and it ends only because the database kills someone. In the log, it's an error and not a long wait.
Seen in: PostgreSQL's docs: it "automatically detects deadlock situations and resolves them by aborting one of the transactions involved". Don't count on which: it is "difficult to predict and should not be relied upon". And the fix: "The best defense against deadlocks is generally to avoid them by being certain that all applications using a database acquire locks on multiple objects in a consistent order."
Source: PostgreSQL docs, Explicit locking, https://www.postgresql.org/docs/current/explicit-locking.html
Say it: "We're getting deadlock detected errors on transfers. Lock the two accounts in id order, lowest first, whichever way the money is going. Catch SQLSTATE 40P01 and retry the whole transaction up to 3 times with jitter. Add a test that runs opposite transfers concurrently."
Described as: "deadlock detected", "two things stuck waiting for each other", "it fails randomly and works when retried", "everything froze until something got killed".
https://dev.dqnotes.com/data/deadlock

### Migration
A script that changes the structure of a live database, such as adding a column, and is kept in your repo so that every copy can run it in order. The hard part is that, for a while, old code and new code are running against it at once.
An agent treats a migration as one edit next to another, and on a laptop it is. In production the deploy isn't instant, so the old code meets the new schema, or the new code meets the old one. A migration that looks like one tidy line can also lock a busy table while it runs. Neither shows up in tests.
Not A deploy: You can roll back a deploy by running the old code again. You can't always roll back a migration, because the column you dropped took its data with it.
Seen in: strong_migrations, a Rails gem built to refuse dangerous ones, says: "Renaming a column that's in use will cause errors in your application." PostgreSQL on building an index: writes are "blocked until the index build is finished", adding "In production environments this is often unacceptable."
Source: strong_migrations, https://github.com/ankane/strong_migrations
Source: PostgreSQL docs, Indexes: introduction, https://www.postgresql.org/docs/current/indexes-intro.html
Say it: "Before you write this migration, tell me what the old code does when this runs and what the new code does before it runs. Set lock_timeout to 2s. Use CREATE INDEX CONCURRENTLY. Don't rename or drop anything in this change."
Described as: "the site went down when we changed the database", "it worked locally and broke on deploy", "change the database without downtime", "the column does not exist", "the migration has been running for ten minutes".
https://dev.dqnotes.com/data/migration

### Expand and contract
Also: Parallel change.
Making a breaking change in safe steps. Add the new thing alongside the old, have the code write to both, copy the old data over, move reads across, stop using the old, and only then remove it. At each step the old and the new code both work.
Told to rename a column, an agent will rename the column. Told "use expand and contract, as separate pull requests", it will produce a sequence you can pause and reverse. It feels slow, and it's the difference between a dull week and an outage. Agents make the tedious version cheap, so there's little excuse left.
Seen in: Danilo Sato, on Martin Fowler's site: "Parallel change, also known as expand and contract, is a pattern to implement backward-incompatible changes". For a renamed column, strong_migrations lays out the same steps: new column, write to both, backfill, move reads, stop writing to the old one, drop it.
Source: Sato, Parallel change, 2014, https://martinfowler.com/bliki/ParallelChange.html
Source: strong_migrations, https://github.com/ankane/strong_migrations
Say it: "Rename users.name to full_name using expand and contract, as separate pull requests. One: add the column. Two: write to both. Three: backfill in batches of 1000. Four: read from the new one. Five: stop writing the old one. Six, at least a week later: drop it. Each must be safe to roll back alone."
Described as: "rename a column without downtime", "how do I change this without breaking the running site", "old and new code at the same time", "do it in stages so I can stop halfway".
https://dev.dqnotes.com/data/expand-contract

### Backup and restore
A backup is a copy of your data, kept somewhere a mistake can't reach it. A restore is getting it back. Until you have really done a restore, you don't know the backup works, and a replica doesn't count, because it copies your mistakes too.
An agent will tell you backups are configured, because a setting says so. That is not evidence that you can get the data back, how long it takes, or that the backup isn't kept in the same blast radius as the thing it protects. Ask it to restore one and count the rows.
Not A replica: A replica is a live copy for when the machine dies. If someone deletes a table, the replica deletes it too within a second. Only a backup can go back in time.
Seen in: GitLab, 31 January 2017: an engineer wiped a data directory on the primary by mistake, and "around 300 GB of data had already been removed". Its nightly dump had been failing, because "the backup procedure was using pg_dump 9.2, while our database is running PostgreSQL 9.6", and the alerts were emails that got rejected. A snapshot an engineer had taken about 6 hours earlier, for other work, saved them. Alexey Grigorev's 2026 first-hand account of a deletion ends with a nightly job that restores a backup and runs a check query against it. PostgreSQL says of its continuous archive: "You are advised to monitor the archiving process to ensure that it is working as you intend."
Source: GitLab, Postmortem of database outage of January 31, 2017, https://about.gitlab.com/blog/postmortem-of-database-outage-of-january-31/
Source: Grigorev, How I dropped our production database, 2026, https://aishippingblog.com/p/how-i-dropped-our-production-database
Source: PostgreSQL docs, Continuous archiving, https://www.postgresql.org/docs/current/continuous-archiving.html
Say it: "Restore last night's backup into a new database called restore_test. Tell me how long it took, compare row counts on the five biggest tables with production, then write that up as a scheduled job that alerts when it fails or when the newest backup is over 25 hours old."
Described as: "can we get the data back", "we have backups, I think", "how far back can we restore", "the replica deleted it too", "nobody has ever tried restoring".
https://dev.dqnotes.com/data/backup-restore

### Partial failure
When some of the work happened and some didn't, and you may not be able to tell which. On one machine, a thing either ran or it crashed. Across a network, you sent a request and heard nothing: it might have failed, or it might have worked and the reply got lost.
Agents write the happy path as a straight line: charge the card, save the order, send the email. Each line is correct, and a crash between any two leaves a state nobody designed. Ask what happens at every gap. Wrapping it all in a database transaction doesn't help, because a rollback can't uncharge a card.
Not An error: An error tells you it failed. The hard case is silence. No answer doesn't mean no, so a timeout can't tell you whether to try again.
Seen in: Jacob Gabrielson of Amazon counts eight places a single request and reply can fail, and says that when no reply comes "the result of the request is UNKNOWN. It may or may not have happened." One way through is the outbox, in Chris Richardson's words: "store the message in the database" with the order, and "A separate process then sends the messages to the message broker."
Source: Gabrielson, Challenges with distributed systems, Amazon Builders' Library, https://builder.aws.com/content/3F08f7GPFiZMCgXD8gny6OjxR0Z/challenges-with-distributed-systems
Source: Richardson, Transactional outbox, https://microservices.io/patterns/data/transactional-outbox.html
Say it: "For this checkout, list every point it could crash and what state each one leaves. Then rewrite it so the order and a row in an outbox table commit in one transaction, and a worker makes the charge with the order id as the idempotency key. Add a test that kills it at each point."
Described as: "they were charged but there is no order", "it half worked", "the request timed out but did it go through", "one service succeeded and the other did not".
https://dev.dqnotes.com/data/partial-failure

### Idempotency
Also: Idempotency key.
An operation is idempotent if doing it twice has the same effect as doing it once. A client makes that true of a payment by sending a unique key with the request. The server saves the first result against that key, and a repeat gets that result back and charges nothing.
Agents add retries readily, since retries look like robustness, and a retry on a charge is how a customer pays twice. It doesn't appear in tests, because the test network never loses a reply. Done properly, this is the whole sentence: in one transaction, insert the key under a unique constraint, and only the winner does the work.
Not Checking whether you already did it: SELECT and then act has a gap in it, and a retry can land in that gap. That is why the bench has a middle setting. The key has to go in under a unique constraint, so that one of the two attempts really loses.
Seen in: Stripe: it "saves the status code and body of the first request made with a given idempotency key" for API v1, and "Retries with the same key return the saved response, including 500 errors." Keys are "up to 255 characters long", and can be removed "after they're at least 24 hours old", so a key isn't for ever (docs read November 2026). The network on this bench is simulated, and the server code running against it is real.
Source: Stripe API docs, Idempotent requests, https://docs.stripe.com/api/idempotent_requests
Say it: "Use a transaction with a unique constraint and an idempotency key. Require an Idempotency-Key header on POST /charges. In one transaction, INSERT it into idempotency_keys with ON CONFLICT DO NOTHING, and if no row went in, return the saved response. Test it with 20 concurrent requests using one key: one charge, 20 identical replies."
Described as: "the customer got charged twice", "is it safe to retry this", "I pressed the button twice", "the same thing happened two times", "the retry created a duplicate".
https://dev.dqnotes.com/data/idempotency

### Retry and backoff
Also: Jitter, Retry storm.
Trying a failed call again, and waiting longer each time. Backoff is the growing wait. Jitter is making each wait random, so that clients which failed together don't all come back together. Without those, retries can keep a struggling server down.
Ask an agent to make something robust and you get a retry loop, often with a fixed delay, at every level. It looks responsible and passes tests. Under a real failure, every client turns one request into several at exactly the moment the server can't cope. Ask for a limit, backoff and jitter, and only on what is safe to repeat.
Not A retry at every layer: Retries multiply. If each of three layers tries three times, the bottom one gets 27 calls for each user action. Retry in one place.
Seen in: Marc Brooker on AWS's blog, 2015, on what a simulation showed about backoff alone: "the solution isn't to remove backoff. It's to add jitter." The Google SRE book: "Always use randomized exponential backoff when scheduling retries." Brooker also calls retries selfish, spending the server's time to raise your own chances. The same pile-up is in AWS's 2025 outage report, where a recovery system "had entered a state of congestive collapse".
Source: Brooker, Exponential backoff and jitter, AWS, 2015, https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
Source: Google SRE book, Addressing cascading failures, https://sre.google/sre-book/addressing-cascading-failures/
Source: Amazon Builders' Library, Timeouts, retries and backoff with jitter, https://builder.aws.com/content/3EumjoZascWd1oZiEgL8ORlv3qE/timeouts-retries-and-backoff-with-jitter
Source: AWS, Summary of the Amazon DynamoDB service disruption, https://aws.amazon.com/message/101925/
Say it: "Retry only on 429, 503 and network errors, at most 3 times, with full jitter: wait a random time from 0 up to min(10s, 200ms x 2^attempt). Honour Retry-After. Don't retry 4xx errors or a POST with no idempotency key. Retry here only, not in the layers above or below."
Described as: "everything came back at once and it fell over again", "how long should it wait before trying again", "the retries made the outage worse", "it keeps hammering the server", "thundering herd".
https://dev.dqnotes.com/data/retry-backoff

### Delivery guarantees
Also: At-least-once, At-most-once, Exactly-once.
What a message system promises. At most once: it may be lost, and it never repeats. At least once: it won't be lost, and it may arrive twice. Most systems you use give you at least once, so your code has to cope with seeing the same thing again.
Webhook handlers and queue workers are written by agents as if each message came once, in order, because that's how a test sends them. In production that assumption is simply false, and it's written in the docs of the service you're using, which the agent didn't read. This is the trap in our experiment.
Not Exactly once: When a product says exactly-once, read what it covers. A sender can't know whether a lost acknowledgement means the message arrived, so what you can really build is at-least-once delivery plus an idempotent handler, which gives the effect of once.
Seen in: Amazon SQS: "Standard queues ensure at-least-once message delivery, but due to the highly distributed architecture, more than one copy of a message might be delivered, and messages may occasionally arrive out of order." Stripe: "Webhook endpoints might occasionally receive the same event more than once." SQS FIFO queues do have a page on exactly-once processing, and what it promises is narrow: no duplicates from a resent message "within the 5-minute deduplication interval".
Source: Amazon SQS docs, Standard queues, https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/standard-queues.html
Source: Stripe docs, Webhooks, https://docs.stripe.com/webhooks
Source: Amazon SQS docs, Exactly-once processing, https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues-exactly-once-processing.html
Say it: "Stripe delivers webhooks at least once, so this handler must be idempotent. Record each event id under a unique constraint in the same transaction as the work, and skip ids already there. Add a test that delivers every event twice and asserts we ship once."
Described as: "the webhook came in twice", "we shipped the order two times", "the job ran twice", "can the message get lost", "how do I make sure it only happens once".
https://dev.dqnotes.com/data/at-least-once

### Out-of-order delivery
Messages can arrive in a different order from the one they were sent in, because a retry of an old one can land after a newer one. If your code treats the latest to arrive as the latest to happen, a late "paid" event can undo a refund.
This is the behaviour nobody asks for, so nobody gets it. In our experiment, an agent told to handle payment webhooks passed the hidden scenarios 27% of the time, and 33% with types, lint, tests and mutation testing all behind it. Told that events arrive at least once and out of order, it passed 100%. One small model and one-function tasks, so read the shape and not the size.
Not A timestamp check: Comparing times is a start, and you need to know your clock. Stripe's event time is in whole seconds, so two can tie. A version number is stronger, and so is treating the event as a nudge to fetch the current state.
Seen in: Stripe: "Stripe doesn't guarantee the delivery of events in the order that they're generated." And a warning against the obvious fix: "Don't use created to determine event order or whether you've already processed an event." It suggests tracking event ids, and using the API to fetch the object's real state. Stripe's docs here were read in November 2026.
Source: Stripe docs, Webhooks, https://docs.stripe.com/webhooks
Say it: "Webhooks arrive at least once and out of order. Dedupe by event id. Don't trust arrival order: on each event, fetch the current state of the payment from the provider's API, and only act on that. Ship exactly once per order, never after a refund. Test all six orderings of paid, refunded and a repeat of paid."
Described as: "a refund got undone", "the old update overwrote the new one", "events arrive in the wrong order", "it went back to the previous status by itself".
https://dev.dqnotes.com/data/out-of-order

### Timeouts and deadlines
Also: Deadline propagation.
A timeout limits one call. A deadline limits the whole job: the moment by which the answer stops being any use, handed down through every call that serves it, so nobody keeps working for a user who has gone.
The network chapter said to set a timeout. An agent then sets one per call, each a round number, and retries inside each. The sum can be far longer than anyone will wait, and the work carries on long after the page has given up. Timeouts are a chain, so they have to be set as one.
Not A timeout: A timeout is local, and restarts at every hop. A deadline is a single time shared by everything below it, so a call further down knows how little time is left.
Seen in: The Google SRE book: "Rather than inventing a deadline when sending RPCs to backends, servers should employ deadline propagation." On retries through the layers it warns that "a single user action may create 64 attempts", from 4 tries in each of 3 layers. Amazon's version has three tries in five layers, where "the load on the database will increase 243x, making it unlikely to ever recover". Amazon picks timeouts from data: an acceptable rate of false timeouts, then "the corresponding latency percentile on the downstream service".
Source: Google SRE book, Addressing cascading failures, https://sre.google/sre-book/addressing-cascading-failures/
Source: Amazon Builders' Library, Timeouts, retries and backoff with jitter, https://builder.aws.com/content/3EumjoZascWd1oZiEgL8ORlv3qE/timeouts-retries-and-backoff-with-jitter
Say it: "Set one deadline of 2 seconds at the edge, pass the remaining time down with each call, and stop work once it has passed. Each outbound call uses the smaller of its own limit and what remains. Retry at the top layer only. Log how much time each hop had left."
Described as: "the user gave up but the server kept working", "the timeouts add up to more than anyone will wait", "one slow thing makes everything slow", "it keeps working after the page has closed".
https://dev.dqnotes.com/data/timeout-deadline

### Circuit breaker
A switch around a call to something that might fail. Count the failures, and past a limit stop calling it for a while and fail at once. After that, let one trial call through. If it works, close again. Named after the thing in your fuse box.
A timeout means that every request waits the full time before failing, and when the thing you depend on is down, that's all your capacity waiting. A breaker turns slow failure into fast failure. Agents won't add one unasked, and when they do it deserves a test, since a breaker that's never been open is one you're only hoping works.
Not A retry: A retry says try again, it might work. A breaker says stop, it won't work, and stop adding load. They pull in opposite directions, so they need tuning together.
Seen in: Martin Fowler: "Michael Nygard popularized the Circuit Breaker pattern" in Release It. Half-open means "the circuit is ready to make a real call as trial to see if the problem is fixed." It's disputed: Amazon's Builders' Library says "circuit breakers introduce modal behavior into systems that can be difficult to test", and it prefers "limiting retries locally using a token bucket".
Source: Fowler, CircuitBreaker, 2014, https://martinfowler.com/bliki/CircuitBreaker.html
Source: Amazon Builders' Library, Timeouts, retries and backoff with jitter, https://builder.aws.com/content/3EumjoZascWd1oZiEgL8ORlv3qE/timeouts-retries-and-backoff-with-jitter
Say it: "Wrap calls to the recommendations service in a circuit breaker: open after 5 failures in 10 seconds, fail at once with the cached list for 30 seconds, then allow one trial call. Log each state change, and add a test that forces it open and checks the page still loads."
Described as: "stop calling it when it is obviously down", "every request waits the full timeout", "one broken service ties up everything", "fail fast".
https://dev.dqnotes.com/data/circuit-breaker

### Queue
A list of jobs waiting to be done, usually run by a separate service. One program adds a message, and a worker takes it later and deletes it when it's finished. The caller doesn't wait, and a spike of work waits in the queue and not on your server.
Told to send emails on signup, an agent writes the call inline, and now a slow mail server makes signup slow. Told to use a queue, it may not think about a worker dying halfway, a job that always fails, or a queue growing faster than it drains. Those are the questions to ask.
Not A background thread: A background thread dies with your process, and its work goes with it. A queue is a separate store, so the work survives a deploy and another worker can pick it up.
Seen in: Amazon SQS: a received message stays in the queue but hidden, and if you "don't delete it before the timeout expires, the message becomes visible again" for another worker. The default is 30 seconds, so a job that takes longer runs twice. A dead-letter queue is where messages go that keep failing, so they don't circle for ever. Stripe advises processing webhooks this way: "Configure your handler to process incoming events with an asynchronous queue."
Source: Amazon SQS docs, Visibility timeout, https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
Source: Amazon SQS docs, Dead-letter queues, https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html
Source: Stripe docs, Webhooks, https://docs.stripe.com/webhooks
Say it: "Move sending the receipt onto a queue. The worker must be idempotent, keyed on the order id. Set the visibility timeout to 5 times the slowest job, send messages to a dead-letter queue after 5 failures, and add an alert on queue depth and on anything in the dead-letter queue."
Described as: "do this in the background", "the page waits for the email to send", "a spike of work took the site down", "the same job keeps failing over and over", "the worker died halfway through".
https://dev.dqnotes.com/data/queue

### Graceful degradation
Also: Load shedding.
Doing less when part of the system is unwell, in place of falling over. Hide the recommendations, serve a slightly old list, turn off the search, and keep the page up. You decide in advance what the product can live without, and what it can't.
An agent's default is to wait for everything and then render, so one optional panel can take down your whole checkout. You have to name which parts are optional. And you have to name what must never be faked: a stale recommendation is fine, a stale price at the moment of paying is not.
Not Hiding the error: Degrading is a designed state you can see in your metrics, and one you chose. Catching an error and showing nothing is the same picture with nobody aware that it's broken.
Seen in: The Google SRE book: "Load shedding drops some proportion of load by dropping traffic as the server approaches overload conditions." Graceful degradation "takes the concept of load shedding one step further" by reducing the work each request needs, for example with a faster, less accurate ranking.
Source: Google SRE book, Addressing cascading failures, https://sre.google/sre-book/addressing-cascading-failures/
Say it: "The product page must render if reviews or recommendations are down: give each a 200 ms budget and a fallback, and log and count each fallback used. The price is not optional. If it's unavailable, turn off buying and say why. Test each with the dependency switched off."
Described as: "one small widget took the whole page down", "keep the site up when part of it is broken", "what can we turn off under load", "show something rather than an error".
https://dev.dqnotes.com/data/graceful-degradation

### Money and floats
Computers store most decimals in binary, which can't hold 0.1 exactly, so sums drift. A float is fine for measuring distance, and wrong for money. Store money as a whole number of the smallest unit, such as pence, or use a decimal type, and decide where rounding happens.
An agent will write price * 1.2 in a float if nothing stops it, and the test passes, because the test checks one amount that happens to round well. The bug appears as a total that's a penny out on one invoice in a thousand. Say integers, say which unit, and say who gets the odd penny.
Not A rounding error: A rounding error comes from rounding. This comes from the number never having been what it said. 0.1 was already wrong before anyone added anything, and every sum on the bench below is really being run by your browser.
Seen in: MDN: JavaScript's number is "a double-precision 64-bit binary format IEEE 754 value". PostgreSQL: "If you require exact storage and calculations (such as for monetary amounts), use the numeric type instead." Stripe takes integers in the currency's minor unit, so 1000 charges 10 USD, and 10 charges 10 JPY, which it lists as a zero-decimal currency. So never assume two decimal places.
Source: MDN, Number, https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number
Source: PostgreSQL docs, Numeric types, https://www.postgresql.org/docs/current/datatype-numeric.html
Source: Stripe docs, Supported currencies, https://docs.stripe.com/currencies
Say it: "Store money as integer minor units in a column called amount_minor, with the currency beside it. Never use a float for money, in code or in the database. Round once, at the invoice line, half-even, and test a 3-way split of 10.00 that adds up to exactly 10.00."
Described as: "the total is a penny out", "0.1 plus 0.2 is not 0.3", "the invoice does not add up", "how should I store prices", "a long string of nines in the total".
https://dev.dqnotes.com/data/floating-point-money

### Dates and time zones
Also: UTC, Daylight saving.
An instant is one point in time, everywhere. What a clock on the wall shows depends on where it is and on that place's current rules. Store instants in UTC and convert when you display. A time in the future, like a meeting, is different: store the wall-clock time and the place.
Tests run in one time zone, on a normal day. So an agent's date code passes, and then a birthday shows up a day early in New York, or a daily job runs an hour off for half the year. Two days a year, some wall-clock times exist twice or not at all. Ask what happens on those days.
Not An offset: +01:00 is an offset, true for one moment. Europe/London is a zone, a set of rules that says which offset applies when. Store an offset for a future meeting, and the clocks will change under it.
Seen in: MDN: a Date holds "milliseconds since the midnight at the beginning of January 1, 1970, UTC", and its basic methods work "in the local (i.e., host system) time zone". For a local time inside a clock change: "if the local time does not exist (there is a gap), we go forward by the gap duration." MDN now calls Date "a legacy feature" in favour of Temporal, as read in November 2026. PostgreSQL stores timestamp with time zone "internally as UTC". Its wiki: "timestamptz records a single moment in time."
Source: MDN, Date, https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date
Source: PostgreSQL docs, Date/time types, https://www.postgresql.org/docs/current/datatype-datetime.html
Source: PostgreSQL wiki, Don't do this, https://wiki.postgresql.org/wiki/Don%27t_Do_This
Source: MDN, Intl.DateTimeFormat, https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/DateTimeFormat/DateTimeFormat
Say it: "Store created_at as timestamptz. For the daily 09:00 reminder, store 09:00 and the IANA zone name, not an offset, and work out each next run from the calendar. Don't add 24 hours. Test it across the Europe/London clock changes of 29 March and 25 October 2026."
Described as: "the date is a day out for some people", "the job ran an hour early", "it broke when the clocks changed", "what time zone should I store", "the timestamp is wrong by a few hours".
https://dev.dqnotes.com/data/time-zones

### Replication
Also: Read replica, Replica lag.
Keeping copies of a database on other machines. One copy, the leader, takes the writes and sends them to the followers, which can serve reads. Usually it sends them after telling the client yes, so a follower is a little behind. That gap is called replica lag.
An agent asked to scale reads will send them to a replica, and it works in every test, where there's no lag. In production someone saves their profile, the page reloads from a replica, and the old value comes back. And if the leader dies, writes that had been confirmed but not yet copied are gone.
Not A backup: A replica protects against a machine dying. It doesn't protect you against yourself, because a bad DELETE reaches it within a second. You need both.
Seen in: PostgreSQL: "Streaming replication is asynchronous by default", so there is "a small delay between committing a transaction in the primary and the changes becoming visible in the standby", typically under a second if the standby keeps up. And: "If the primary server crashes then some transactions that were committed may not have been replicated to the standby server, causing data loss." Amazon RDS read replicas are also copied asynchronously.
Source: PostgreSQL docs, Log-shipping standby servers, https://www.postgresql.org/docs/current/warm-standby.html
Source: Amazon RDS docs, Read replicas, https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html
Say it: "Send reports to the read replica, and keep anything a user just changed on the primary. After a write, read that user's data from the primary for 5 seconds. Add a metric and an alert on replica lag, and show me what happens to the page if lag reaches 30 seconds."
Described as: "I saved it and the old value came back", "it shows the right thing after a refresh", "the copy of the database is behind", "we failed over and lost some data".
https://dev.dqnotes.com/data/replication

### Eventual consistency
A promise that all copies will agree if you stop changing things and wait. Until then, different readers can see different answers. It's a fair trade for speed and uptime, as long as the product was designed for it and it isn't a surprise.
An agent writes code that reads back what it just wrote, because that always works locally. Against an eventually consistent store, that code is wrong some of the time. And when two copies took different writes, something has to pick. The default is often last write wins, which means one person's change is silently thrown away.
Not Replica lag: Lag is one copy being behind one leader, and it resolves by itself. Eventual consistency also covers copies that both accepted writes, where somebody has to decide how to merge them.
Seen in: Werner Vogels of Amazon, 2008: it's a form of weak consistency where "eventually all accesses will return the last updated value". He names the case users notice, read-your-writes: "process A, after it has updated a data item, always accesses the updated value and will never see an older value." And the rule that gives you strong reads from copies: "If W+R > N, then the write set and the read set always overlap".
Source: Vogels, Eventually consistent, 2008, https://www.allthingsdistributed.com/2008/12/eventually_consistent.html
Say it: "This store is eventually consistent, so don't read back after writing to get the new value: return it from the write. When two copies conflict, don't use last write wins for the cart. Merge the two sets of items, and add a test with two conflicting writes."
Described as: "different people see different numbers", "it will sort itself out in a minute", "one person's change got overwritten", "the item came back after I removed it".
https://dev.dqnotes.com/data/eventual-consistency

### CAP theorem
When a network splits your copies of the data apart, you have to choose. Either refuse some requests, so nobody gets a wrong answer, or answer them all, and accept that some answers are out of date. You can't have both while the split lasts.
It's a staple of interviews, and an agent can recite it confidently, including the slogan that's wrong. What matters is the question it forces on you: for this one piece of data, during an outage, would you rather show an error or show something stale? A bank balance and a like count have different answers.
Not Pick two of three: The slogan suggests a free choice. You don't get to opt out of a network splitting. The real choice comes only during a split, and at other times you can have both, at some cost in speed.
Seen in: Eric Brewer proposed it in 2000, and Seth Gilbert and Nancy Lynch proved it in 2002, per Wikipedia, which puts the practical point this way: "Thus, if there is a network partition, one has to choose between consistency or availability." In 2012 Gilbert and Lynch placed it within a wider trade-off: "the impossibility of guaranteeing both safety and liveness in an unreliable distributed system". Don't over-use it. Martin Kleppmann: "The CAP theorem is too simplistic and too widely misunderstood to be of much use for characterizing systems."
Source: Wikipedia, CAP theorem, https://en.wikipedia.org/wiki/CAP_theorem
Source: Gilbert and Lynch, Perspectives on the CAP theorem, 2012, https://groups.csail.mit.edu/tds/papers/Gilbert/Brewer2.pdf
Source: Kleppmann, Please stop calling databases CP or AP, 2015, https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-cp-or-ap.html
Say it: "For each kind of data here, tell me what the user sees during a network partition: an error, or possibly stale data. Balances must refuse. The product catalogue may be stale. Don't just call the database CP or AP. Say what happens to a read and a write."
Described as: "the data centres lost contact with each other", "error or stale data, which is better", "pick two of three", "should it stop working or show old data".
https://dev.dqnotes.com/data/cap

### Personal data
Also: PII.
Anything that can be tied to a real person: a name, an email address, a location, an account number. It carries legal duties in much of the world. It gets everywhere, into logs, backups, test databases and, now, an agent's context, which may leave your machine.
Agents log whole request bodies when debugging, seed test databases by copying production, and paste rows into a prompt to reason about them. All of it is helpful, and each one moves personal data somewhere you didn't decide on. A scrubber only catches what has a shape, so name the fields that may leave, and keep the rest out.
Not Secret: A secret can be rotated when it leaks. A person can't change their medical history. A secret leak is about access, and a personal data leak is about someone who was never asked.
Seen in: GDPR article 4 defines personal data as "any information relating to an identified or identifiable natural person", and includes location data and online identifiers. Article 5 wants it "limited to what is necessary", and article 17 gives people erasure "without undue delay", which raises a real question about your backups. OWASP lists "Sensitive personal data" among things to keep out of logs. Stripe says of its keys: "Avoid using sensitive data (for example, email addresses or personal identifiers) as idempotency keys." This is orientation and not legal advice.
Source: GDPR, Article 4, https://gdpr-info.eu/art-4-gdpr/
Source: GDPR, Article 5, https://gdpr-info.eu/art-5-gdpr/
Source: GDPR, Article 17, https://gdpr-info.eu/art-17-gdpr/
Source: OWASP, Logging cheat sheet, https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html
Source: Stripe API docs, Idempotent requests, https://docs.stripe.com/api/idempotent_requests
Say it: "Don't log request bodies. Log only these fields: level, event, order_id, status and ms. Add a test that sends a request with a marker email address and fails if the marker appears in the log output. Use generated fixtures, and never copy production rows into a test or into this conversation."
Described as: "customer emails are in the logs", "can I test with a copy of production", "someone asked us to delete their data", "the agent pasted real customer rows", "what counts as personal data".
https://dev.dqnotes.com/data/pii

### Agents and production data
An agent with production credentials can delete data as fast as it can type. So keep credentials it can read away from production, give it a read-only role when it needs to look, and put destructive steps behind something it can't talk its way past. A sentence in its instructions doesn't count.
An agent stuck on a problem looks around for a way through, and a token in a file is a way through. It doesn't feel the difference between staging and production, and it may only find out which one it was in by reading the error afterwards. Instructions are guides, and it can read them and do the other thing.
Not A machine gone rogue: These reports usually read as a machine disobeying. The cause you can act on is that it had permission. An agent can't delete what its credentials can't reach.
Seen in: Three reports, of different strength. Strongest, because it is first-hand and the author takes the blame: Alexey Grigorev writes that on 26 February 2026, Claude Code ran terraform destroy from an old state file, and "The database, VPC, ECS cluster, load balancers, and the bastion host were gone." The post also says "I over-relied on the AI agent to run Terraform commands." AWS restored a snapshot 24 hours later. Second: Jeremy Crane of PocketOS said in April 2026 that a Cursor agent running Claude Opus 4.6, stuck in staging, found a token in an unrelated file and deleted a Railway volume in about nine seconds, taking production and its volume-level backups. We read that through The Register and Decrypt, and not Crane's own post. Railway's Jake Cooper confirms the call and blames a "legacy endpoint", and accounts of the recovery differ. Third, the most repeated and the least checked: in July 2025 Jason Lemkin reported that Replit's agent wiped data on "more than 1,200 executives and over 1,190 companies" during a code freeze. Those numbers and the agent's apologies come only from Lemkin's posts as Fortune reported them, and we found no independent check. Replit's CEO, as quoted there, did call it "Unacceptable and should never be possible." Replit's own blog that week says "Until now, Replit apps used a single database for both development and live customer data". It doesn't mention the incident, and we could not open the posts on X.
Source: Grigorev, How I dropped our production database, 2026, https://aishippingblog.com/p/how-i-dropped-our-production-database
Source: The Register, Cursor-Opus agent snuffs out PocketOS, via a reader copy, https://hn.jakechampion.name/reader/https://www.theregister.com/2026/04/27/cursoropus_agent_snuffs_out_pocketos/
Source: Decrypt, AI agent deletes startup database in 9 seconds, 2026, https://decrypt.co/365897/ai-agent-deletes-startup-database-9-seconds-founder-says
Source: Fortune, Replit wiped database, 2025, https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure/
Source: The Register, Replit's response, 2025, https://www.theregister.com/2025/07/22/replit_saastr_response/
Source: Replit blog, Separate development and production databases, 2025, https://replit.com/blog/introducing-a-safer-way-to-vibe-code-with-replit-databases
Say it: "You don't get production credentials. Remove any from .env and from every file you can read, and use the local database. If you need to look at production, I'll give you a read-only role. Propose anything destructive as a migration in a pull request. Never run terraform destroy, DROP or DELETE yourself."
Described as: "the agent deleted my database", "should the AI have access to production", "it ignored the code freeze", "how do I stop it wiping data", "it found a password in a file and used it".
https://dev.dqnotes.com/data/deleting-data-agent

## 15. Performance and cost

How to measure speed honestly, where time goes, and what tokens cost.

### Measure first
Before changing anything to make it faster, time it, find the slow part and write the number down. Then time it the same way after the change. Without both numbers, nobody can say the change helped.
An agent reads code and spots what looks slow, and what looks slow often isn't what is slow. It reports the improvement in the same confident voice either way. So make the number part of the work you ask for, because it is the one thing it can't produce by reasoning.
Not Reasoning about the code: Saying a loop is now O(n) is an argument about the code. It may be right. It still isn't a measurement of your program on your data, and the part it improved may not have been slow.
Seen in: Knuth, in 1974: "It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail." Peng and colleagues found that agent performance pull requests on GitHub "rely predominantly on static reasoning checks", where those by people more often report a benchmark. That is one preprint, with the limits given at the top of this chapter.
Source: Knuth, Structured Programming with go to Statements, Computing Surveys 6(4), December 1974, p. 268 (scan), https://pic.plover.com/knuth-GOTO.pdf
Source: The same paper at the ACM, https://doi.org/10.1145/356635.356640
Source: Peng et al., How Do Agents Perform Code Optimization?, preprint, December 2025, https://arxiv.org/abs/2512.21757
Say it: "Don't change anything yet. Run hyperfine --warmup 3 --runs 20 "bun run scripts/report.ts" and show me the mean and the spread. Make your change, run the same command, and put both results in the pull request. If the two ranges overlap, tell me it made no measurable difference."
Described as: "the agent says it should be faster now", "how do I know the fix actually helped", "it optimised something without checking", "prove that it is quicker", "it feels faster but I am not sure".
https://dev.dqnotes.com/performance/measure-first

### Latency and throughput
Latency is how long one thing takes. Throughput is how many things get done per second. They are separate numbers, and improving one can hurt the other: waiting to send items in a batch gets more done per second, and makes each item wait longer.
Tell an agent to make something faster and it has to guess which you meant. Batching can be a real win for a nightly import and a real slowdown for the person waiting on one request, and both get described as a speed-up. Name the number you want to move.
Not Bandwidth: Bandwidth is how much a connection could carry. Throughput is what really got done. A fast pipe with a slow server has high bandwidth and low throughput.
Seen in: Little's law ties the two together: "the long-term average number of customers (L) in a stationary system is equal to the long-term average effective arrival rate (λ) multiplied by the average time that a customer spends in the system". So 100 requests per second that each take 0.2 seconds means about 20 in progress at any moment, whatever the server is built from.
Source: Wikipedia, Little's law, https://en.wikipedia.org/wiki/Little%27s_law
Say it: "The problem here is p95 latency on GET /orders, not throughput, so don't add batching. Measure p95 at 50 requests per second with k6 before and after, and report both."
Described as: "how many requests can it handle", "each request is slow but the server is not busy", "it handles more but every one takes longer", "what does fast actually mean here".
https://dev.dqnotes.com/performance/latency-throughput

### Percentiles
Also: p50, p95, p99.
Sort every timing from fastest to slowest. The 95th percentile, or p95, is the value that 95% of them are at or under. p50 is the middle one. A percentile is a time someone really waited, which an average need not be.
An agent asked if a change helped will often quote an average, because an average is what a quick script prints. A few very slow requests drag it up to a value nobody experienced, and a real regression for 5% of people can barely move it. Ask for the percentiles.
Not Average: The mean adds everything up and shares it out evenly. If most requests take 100 ms and a few take 3 seconds, the mean can land on a time that no request took. p50 says what is typical, and p99 says how bad it gets.
Seen in: Google's SRE book: "Most metrics are better thought of as distributions rather than averages." It adds that a high percentile "shows you a plausible worst-case value", while the 50th "emphasizes the typical case". Core Web Vitals are assessed at the 75th percentile of page loads and not at the average.
Source: Google SRE book, Service Level Objectives, https://sre.google/sre-book/service-level-objectives/
Source: web.dev, Web Vitals, read 5 November 2026, https://web.dev/articles/vitals
Say it: "Don't give me the average. Report p50, p95 and p99 for this endpoint, the sample size and the time window, before and after your change."
Described as: "the average looks fine but people are complaining", "what does p99 mean", "it is usually fast but sometimes really slow", "the dashboard says it is fine", "one slow request in a hundred".
https://dev.dqnotes.com/performance/percentile

### Tail latency
Also: p99 latency, Hedged request.
The slowest few requests, far along the end of the distribution. One in a hundred sounds rare until a single page makes dozens of requests and waits for them all, at which point a lot of people meet that one.
Chances stack up: with ten requests per page, about one page view in ten hits a p99. An agent run does the same thing, because a session making 40 calls has a one in three chance of meeting a 1-in-100 slow one. So a slow call that looks rare per request is routine per session, and a retry or a timeout has to expect it.
Not Outlier: An outlier suggests a freak result you can throw away. The tail is a normal part of how the system behaves, and it comes back every day.
Seen in: Dean and Barroso, at Google in 2013: "If a user request must collect responses from 100 such servers in parallel, then 63% of user requests will take more than one second", where each server is slow only 1 time in 100. One fix they describe is a hedged request, a second copy sent if the first is slow. In one Google benchmark, "sending a hedging request after a 10ms delay reduces the 99.9th-percentile latency for retrieving all 1,000 values from 1,800ms to 74ms while sending just 2% more requests." That was their system at their scale. Your gain will be different.
Source: Dean and Barroso, The Tail at Scale, Communications of the ACM, 2013, https://www.barroso.org/publications/TheTailAtScale.pdf
Source: Google Research, The Tail at Scale, https://research.google/pubs/the-tail-at-scale/
Say it: "This page makes 12 API calls and waits for all of them. Log the duration of each, tell me which one sets the page's p99, and give every call a timeout. Don't retry a call that writes data unless it is idempotent."
Described as: "the page is slow for some people some of the time", "each service is fast but the page is slow", "one slow call holds up the whole page", "why is it slow when the averages are fine".
https://dev.dqnotes.com/performance/tail-latency

### Benchmark
A repeatable test that times a piece of code many times under the same conditions, so two versions can be compared. One timing isn't a benchmark, because the same code gives a different number on each run.
An agent asked to prove its change helped can write a benchmark that shows what it hopes for: one run, a tiny input, the old code cold and the new code warm. It need not mean any harm by that. So read the benchmark as well as the result, and ask how many runs it took and whether the ranges overlap.
Not Model benchmark: Chapter 5's SWE-bench scores models on a fixed set of tasks. This one times your code. Both can mislead in the same way: they tell you about the test, and you have to argue that the test resembles your case.
Seen in: hyperfine runs a command repeatedly and reports the spread: its README shows --runs to set the count and --warmup to run it first so the caches are warm. In a browser, MDN says performance.now() "is coarsened", to 100 microseconds unless the page is cross-origin isolated, so a function faster than that needs many repetitions before its timing means anything.
Source: hyperfine, https://github.com/sharkdp/hyperfine
Source: MDN, performance.now(), https://developer.mozilla.org/en-US/docs/Web/API/Performance/now
Say it: "Benchmark both versions with hyperfine --warmup 3 --runs 30, on the real 50 MB fixture and not a sample. Show me both means with their spreads. If the ranges overlap, say you can't tell them apart."
Described as: "it was faster the first time I ran it", "the numbers change every run", "how many times should I run it", "the benchmark says it is faster but it does not feel it".
https://dev.dqnotes.com/performance/benchmark

### Profiler
A tool that watches a program run and reports how much time each function took. Self time is what the function spent on its own. Total time also includes everything it called. The answer often surprises people.
This is the most useful thing you can hand an agent for speed work. Without a profile it optimises what looks suspicious. With one it starts from a list of where the time went, and you can check whether its change moved the top line.
Not Instrumentation: A sampling profiler checks what is running many times a second and adds little cost. Instrumentation times each call, so it is exact about counts and can slow small functions down. Our benches use it because the browser's own profiling API needs the page to be served with a special policy.
Seen in: Knuth wanted this in 1974: "all compilers written from now on should be designed to provide all programmers with feedback indicating what parts of their programs are costing the most". Today, Node's --cpu-prof "Starts the V8 CPU profiler on start up, and writes the CPU profile to disk before exit", and py-spy samples Python programs. Brendan Gregg gave up function tracing for one early visualisation because "the overhead of function tracing was too high, perturbing the target".
Source: Knuth, Structured Programming with go to Statements, Computing Surveys 6(4), December 1974, p. 268 (scan), https://pic.plover.com/knuth-GOTO.pdf
Source: Node.js docs, Command-line API, https://nodejs.org/api/cli.html
Source: py-spy, https://github.com/benfred/py-spy
Source: Gregg, Flame Graphs, https://www.brendangregg.com/flamegraphs.html
Source: MDN, JS Self-Profiling API, https://developer.mozilla.org/en-US/docs/Web/API/JS_Self-Profiling_API
Say it: "Run node --cpu-prof scripts/report.js, read the .cpuprofile, and list the top five functions by self time with their percentages. Don't edit anything until I have seen that list."
Described as: "where is all the time going", "which function is the slow one", "find out what is taking so long", "it is slow and I have no idea why".
https://dev.dqnotes.com/performance/profiler

### Flame graph
Also: Flame chart.
A picture of a profile. Each bar is a function, sitting on top of the one that called it, and the wider the bar, the more time it took. You look for the wide bars at the top: they are doing the work themselves, with nothing above them to blame.
A flame graph lets you check an agent's story at a glance. If it says the slowness is in the formatter and the widest bar is somewhere else, you can see that for yourself without reading a line of code, and without taking its word for it.
Not Flame chart: In a flame graph, left to right is alphabetical and identical stacks are merged, so you see the total. In a flame chart, left to right is time, so you see the order things happened. Chrome's Performance panel draws the second kind.
Seen in: Brendan Gregg invented them and released them in December 2011. His definition: "The x-axis shows the stack profile population, sorted alphabetically (it is not the passage of time), and the y-axis shows stack depth". On the other kind: "flame charts put the passage of time on the x-axis instead of the alphabet."
Source: Gregg, Flame Graphs, https://www.brendangregg.com/flamegraphs.html
Say it: "Run py-spy record -o profile.svg -- python report.py and tell me the three widest frames with nothing above them, with their percentages. Propose a change only for those, and rerun it afterwards to show that frame got narrower."
Described as: "the colourful graph of where time went", "what are the wide bars", "how do I read this profile", "show me which code is slow as a picture".
https://dev.dqnotes.com/performance/flame-graph

### Big O
Also: Time complexity, O(n).
A way of saying how a program's work grows as its input grows. With O(n), twice the data takes about twice as long. With O(n²), twice the data takes about four times as long. It says nothing about how long any one run takes.
An agent's tests run on ten rows, and at ten rows everything is instant. A loop inside a loop can pass every check and only hurt a year later, when the table has grown. It is also a claim agents love to make, so ask to see it measured at two or three sizes.
Not Being fast: Big O ignores constant factors. On small inputs, a plain nested loop can beat a clever structure, because the structure has costs of its own. The bench shows this on your device, at sizes where the lines cross.
Say it: "dedupe in src/report.ts has a loop inside a loop. Time it at 1,000, 10,000 and 100,000 rows and show me the three numbers. If ten times the rows takes about a hundred times as long, replace the inner search with a Set and rerun the same three sizes."
Described as: "it was fine until we had more data", "it works on my test data and dies in production", "a loop inside a loop", "what does O(n) mean", "it gets slower and slower as the table grows".
https://dev.dqnotes.com/performance/big-o

### Bottleneck
Also: Theory of constraints.
The one stage that limits how much the whole line can do. Speeding up any other stage changes nothing at the end, and just makes a bigger pile in front of the narrow part.
Agents make writing code far faster, so for many teams the bottleneck has moved to reading it. Generating twice as many pull requests against the same review capacity gets you a longer queue, and pressure to approve without reading. Find the narrow stage before you buy speed anywhere else.
Not The slowest function: A slow function only matters if work is waiting on it. A bottleneck is defined by what is holding up the whole line. The thing that takes longest may not be it, if it runs at night and nothing depends on it.
Seen in: The idea is usually credited to Eliyahu Goldratt's 1984 book The Goal, about a factory. Fred Hebert, writing about software queues: "Step 1. Identify the bottleneck. Step 2: ask the bottleneck for permission to pile more data in". The review chapter has what happened to reviews when agents arrived, measured across hundreds of firms.
Source: Wikipedia, Theory of constraints, https://en.wikipedia.org/wiki/Theory_of_constraints
Source: Hebert, Queues Don't Fix Overload, https://ferd.ca/queues-don-t-fix-overload.html
Say it: "Before you speed up the build, take the last 20 merged pull requests from gh pr list --state merged --json createdAt,mergedAt and tell me where each one waited longest: CI, review or merge. Work only on the longest wait."
Described as: "we write code faster but ship no faster", "it all piles up at one step", "we sped that up and nothing changed", "everything is waiting on one thing".
https://dev.dqnotes.com/performance/bottleneck

### Amdahl's law
If part of a job has to be done one step at a time, adding more workers can only speed up the rest. If 5% must run alone, you can never go more than 20 times faster, however many workers you add.
This is the sum behind running many agents at once. Ten agents writing in parallel still queue for one person to read, one merge to main and one deploy. That single-file part sets the limit, and past a point another agent mostly adds cost.
Not Gustafson's law: Amdahl fixes the size of the job and asks how much faster it gets. Gustafson's law is the other view: with more workers you often take on a bigger job, in the same time.
Seen in: Wikipedia states the general form: "the overall performance improvement gained by optimizing a single part of a system is limited by the fraction of time that the improved part is actually used". It is named after Gene Amdahl, who presented the argument at a conference in 1967. So making something that takes 10% of the time infinitely fast saves 10%, and no more.
Source: Wikipedia, Amdahl's law, https://en.wikipedia.org/wiki/Amdahl%27s_law
Say it: "dedupe takes 8% of this run according to the profile, so a perfect rewrite saves 8% at most. Is there a part bigger than that? Show me the sum before you start on this one."
Described as: "we added more servers and it barely helped", "how much faster can it get", "adding more agents stopped helping", "is it worth optimising this part".
https://dev.dqnotes.com/performance/amdahl

### Premature optimisation
Making code faster, and usually harder to read, before you know that it matters. The cost is real and paid at once in complexity. The gain is a guess until someone has measured that this code is where the time goes.
An agent can do this at no effort to itself, so the old brake has gone: a person got tired of writing a clever cache, and an agent doesn't. You then own the extra code. But the same line is often used to excuse not measuring at all, and it doesn't say that.
Not Optimisation: The word that carries the weight is premature. Fixing a function that a profile says takes 70% of the time isn't premature. Knuth's objection is to spending the effort before you have looked.
Seen in: Knuth, in the 1974 paper, in full: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified." The same page says "In established engineering disciplines a 12% improvement, easily obtained, is never considered marginal", and the 97% is his rough estimate and not a measurement. So the fuller quote is an argument for measuring, then acting.
Source: Knuth, Structured Programming with go to Statements, Computing Surveys 6(4), December 1974, p. 268 (scan), https://pic.plover.com/knuth-GOTO.pdf
Source: The same paper at the ACM, https://doi.org/10.1145/356635.356640
Say it: "Take out the cache you added to getUser. Nothing shows this code is slow, and it adds a stale-data risk. If the profile later puts it in the top five, we can bring it back with the numbers in the pull request."
Described as: "it added a cache nobody asked for", "why is this simple thing so complicated", "the code got clever for no reason", "someone quoted root of all evil at me", "should I bother making this faster".
https://dev.dqnotes.com/performance/premature-optimisation

### Caching
Also: LRU, Hit rate.
Keeping a copy of an answer somewhere quick, so you can skip the slow work next time. A hit is when the copy is there. A miss means doing the work anyway. A cache is small, so something has to decide what to throw out, often the least recently used.
A cache is an easy diff that makes a benchmark look great, so it is a favourite agent fix, and it swaps a speed problem for a correctness problem. Check the hit rate on real traffic first: a cache that rarely hits costs memory and complexity, and gives nothing back.
Not Prompt caching: This keeps a stored answer and hands it back. Prompt caching only skips reprocessing the input, and the model still writes a new answer. Same word, and the last part of this chapter works out what that one is worth.
Seen in: On our bench, a cache one slot smaller than a loop over its keys hits 0% of the time: a real LRU throws out each key just before it is asked for again. That is why you measure a hit rate on your own access pattern. A size that is excellent on one pattern can be worthless on another.
Say it: "Before adding a cache to getProduct, log 10,000 real lookups and work out the hit rate for a cache of 100, 1,000 and 10,000 entries. Show me those numbers and how stale the data could get. Only then write it."
Described as: "keep a copy so it does not have to do that again", "the cache is not helping", "what is a cache hit", "it only gets fast the second time".
https://dev.dqnotes.com/performance/caching

### Cache invalidation
Also: Stale read, TTL.
Getting rid of a stored copy when the real answer changes. Do it too late and people read old data, which is called a stale read. A time limit on each copy, or TTL, is the blunt way: it is simple, and wrong until it runs out.
Tests rarely catch this, because they read a value once and never check again after a write. So an agent's cache can pass everything and still show someone the old price for an hour. Whoever adds a cache has to say what happens on every write path, including the ones they didn't read.
Not Eviction: Eviction throws out a copy because there is no room, and the copy was still right. Invalidation throws out a copy because it is now wrong. A cache with plenty of room can still serve stale data for ever.
Seen in: Martin Fowler's page keeps the old joke: "There are only two hard things in Computer Science: cache invalidation and naming things." He credits Phil Karlton, and says he "couldn't find a satisfactory URL" for it, so treat the attribution as a well-known saying and not a checked citation. Deleting on write isn't a complete answer either, as the race on the bench shows.
Source: Fowler, Two Hard Things, 2009, https://martinfowler.com/bliki/TwoHardThings.html
Say it: "Add a test: read the price, update it, read it again, and assert the second read returns the new price. List every code path that writes to prices, and show me each one clearing the cache. Keep a TTL of 5 minutes as a backstop."
Described as: "I changed it but the old value still shows", "it is right after a few minutes", "clearing the cache fixes it", "users see old data", "it shows the old price".
https://dev.dqnotes.com/performance/cache-invalidation

### Queueing and utilisation
Also: M/M/1, Load shedding.
When work arrives at random, it sometimes bunches up and has to wait in line. Utilisation is how busy the server is. As it nears 100%, the line grows faster and faster: at 50% a request takes about twice its work time, at 90% ten times, at 99% a hundred times.
Running everything at 95% looks efficient, and the people waiting pay for it. The same sum applies to you: an agent that hands a reviewer more than they can read doesn't get faster reviews, only a longer line. And a word to keep straight: this is where chapter 9's word has its software meaning.
Not Backpressure in chapter 9: In a queueing system, backpressure means a full queue makes the sender slow down or stop. Chapter 9 uses the word loosely, for a check that pushes work back at an agent. Same picture of pushing back, and different things being pushed.
Seen in: For one server with random arrivals and service, the average time in the system is 1/(μ − λ), the service rate minus the arrival rate. As for the word: it began as "a resistance to the desired flow of fluid through pipes", and software borrowed it for this. Fred Hebert puts the choice at a full queue: "You'll have to pick between blocking on input (back-pressure), or dropping data on the floor (load-shedding)." Chapter 9's sense came later, and is the loose one.
Source: Wikipedia, M/M/1 queue, https://en.wikipedia.org/wiki/M/M/1_queue
Source: Wikipedia, Back pressure, https://en.wikipedia.org/wiki/Back_pressure
Source: Hebert, Queues Don't Fix Overload, https://ferd.ca/queues-don-t-fix-overload.html
Say it: "This worker runs at 90% busy and p99 is climbing. Don't just raise the queue size, since that only lengthens the wait. Put a limit of 1,000 on the queue, return a 429 with Retry-After when it is full, and graph queue depth next to p99."
Described as: "it was fine at 80% and fell over at 95%", "why does it get so slow when it is busy", "the queue keeps growing", "the server is not even at full capacity", "should we just add a bigger queue".
https://dev.dqnotes.com/performance/queueing

### Load test
Also: Stress test, Coordinated omission.
Sending a system a realistic amount of made-up traffic, on purpose, to see how it behaves before real people do. A stress test goes past the expected load, to find where it breaks.
An agent's checks run one request at a time, so everything it builds looks fast. Load is what finds the queue and the exhausted pool. The test can also lie: a tool that waits for each reply before sending the next quietly stops testing at exactly the moment things go wrong.
Not Benchmark: A benchmark times one piece of code in isolation. A load test exercises the whole system, under traffic, and looks at how it behaves together.
Seen in: k6 can make a load test a gate: http_req_duration: ['p(95)<200'] fails the run if p95 goes over 200 ms. Its docs warn that "no consensus even exists about the names of these test types". Gil Tene's wrk2 README describes the trap: a tool whose connections wait for replies "exhibits a strong Coordinated Omission effect, through which most of the high latency artifacts exhibited by the measured server will be ignored."
Source: Grafana k6 docs, Thresholds, https://grafana.com/docs/k6/latest/using-k6/thresholds/
Source: Grafana k6 docs, Load test types, https://grafana.com/docs/k6/latest/testing-guides/test-types/
Source: Tene, wrk2, https://github.com/giltene/wrk2
Say it: "Write a k6 script with a constant-arrival-rate of 100 requests per second for 5 minutes, and thresholds of p(95)<200 and rate<0.01 on errors, against staging. Don't use closed virtual users: the rate has to stay fixed when the server slows down."
Described as: "will it cope with a thousand users", "it works for me but falls over in production", "test it under heavy traffic", "the load test passed and production still went down".
https://dev.dqnotes.com/performance/load-test

### Concurrency and parallelism
Concurrency means juggling many tasks, swapping between them while each waits for something. Parallelism means truly doing several at the same instant, on separate cores, the independent workers inside a CPU. Waiting can be juggled. Crunching numbers needs more cores.
An agent will happily wrap heavy work in Promise.all and call it parallel. That speeds up work that waits, such as network calls, and does nothing for work that computes, because in JavaScript it all still takes turns on one thread. Ask which kind of work it is, then time it.
Not Async: Async code lets other things run while this one waits. It doesn't make anything run on a second core. For that you need a Worker, a worker thread or another process.
Seen in: Rob Pike, in the talk Concurrency is not Parallelism: "Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once."
Source: Pike, Concurrency is not Parallelism, 2012 (slides), https://go.dev/talks/2012/waza.slide
Source: Node.js docs, Worker threads, https://nodejs.org/api/worker_threads.html
Say it: "Promise.all won't speed up hashing these files, because that work is CPU and not waiting. Move the hashing to a pool of worker_threads with one worker per core, time it against the current version on the same 200 files, and show me both numbers."
Described as: "I made it async and it is no faster", "run these at the same time", "does it use all my cores", "Promise.all did not help".
https://dev.dqnotes.com/performance/concurrency-parallelism

### Event loop
Also: Long task, Blocking the main thread.
In a browser or Node, a single line of work, called a thread, takes jobs off a list and finishes each before starting the next. While a job runs, nothing else can: no clicks, no animation, no other requests. A job over 50 ms in a browser is called a long task.
An agent's code works on its small test data and passes everything, and then it sorts 200,000 rows inside a click handler and the page freezes. No test notices a freeze. In Node it is worse: one slow job stalls every other person's request on that server.
Not A slow network: When the page waits on the network, the loop is free and it stays responsive. When the loop is blocked, nothing is responsive, even with the data already there. A spinner that stops spinning is a clue, if it is drawn by script.
Seen in: MDN calls this run-to-completion: "Each job is processed completely before any other job is processed." web.dev: "Any task that takes longer than 50 milliseconds is a long task." One fix is to give way part-way through with scheduler.yield(), which web.dev listed for Chrome and Edge 129 and Firefox 142, and not Safari, as read on 5 November 2026. On our bench in Chromium it still left gaps of about 100 ms between frames, and a plain message task didn't, so measure this too. Node has a whole guide called Don't Block the Event Loop.
Source: MDN, JavaScript execution model, https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Execution_model
Source: web.dev, Optimize long tasks, read 5 November 2026, https://web.dev/articles/optimize-long-tasks
Source: Node.js, Don't Block the Event Loop, https://nodejs.org/en/learn/asynchronous-work/dont-block-the-event-loop
Say it: "This click handler sorts all the rows on the main thread. Move the sort to a Web Worker. Then record PerformanceObserver longtask entries before and after with 200,000 rows, and show me that none are over 50 ms."
Described as: "the whole page freezes for a second", "the button does nothing and then it all happens at once", "one slow request holds up everyone", "the spinner stops spinning", "it locks up while it works".
https://dev.dqnotes.com/performance/event-loop

### Connection pool
A fixed set of open database connections that requests borrow and hand back. Opening a new one each time is slow, so they are reused. If all are in use, a request waits. If code forgets to hand one back, the pool slowly runs out.
This one bites agents because everything works in development, where there is one user. A missing release only shows up under load. A pool per serverless instance can multiply past what the database allows. And the instinct to fix it with a bigger pool can make it worse.
Not A bigger pool is faster: Past a point, more connections means the database spends its time switching between them. A pool smaller than you expect can be quicker, so test around a starting number instead of raising it.
Seen in: PostgreSQL says of max_connections: "The default is typically 100 connections". HikariCP's pool sizing page offers "connections = ((core_count * 2) + effective_spindle_count)" as a starting point, and says you should test around it. That is one project's rule of thumb, and the bench's numbers are made up.
Source: PostgreSQL docs, Connections and Authentication, https://www.postgresql.org/docs/current/runtime-config-connection.html
Source: HikariCP wiki, About Pool Sizing, https://github.com/brettwooldridge/HikariCP/wiki/About-Pool-Sizing
Say it: "Use one shared pool from src/db/pool.ts with max: 10, and release in a finally block on every path. Then run k6 at 100 requests per second for 2 minutes against staging and show me pool wait time and any connection errors. Don't raise max to fix an error until we know why it happened."
Described as: "too many connections error", "it hangs after a while until I restart it", "requests are stuck waiting on the database", "it works with one user and dies with fifty".
https://dev.dqnotes.com/performance/connection-pool

### Memory leak
Memory the program no longer needs, and can't release because something still points at it: a list that only grows, a listener never removed. Each bit is small, so it passes every test and takes days to bring a server down.
An agent's session ends long before a leak shows. It adds a cache with no limit or a listener with no cleanup, the tests pass, and nothing fails until your server has been up for a week. Restarting hides it, so it can run for months. You have to go looking.
Not High memory use: Using a lot of memory isn't a leak if it levels off. A leak keeps climbing under steady load. So what you need is the shape over time, and one reading tells you very little.
Seen in: Chrome's docs: "Identify detached DOM trees (a common cause of memory leaks) with Heap Snapshots." The usual method is to take a snapshot, repeat the suspect action, take another and compare them. MDN explains why this is on you: the garbage collector goes by what is reachable, and deciding what is still needed "is undecidable".
Source: Chrome DevTools, Fix memory problems, https://developer.chrome.com/docs/devtools/memory-problems
Source: MDN, Memory management, https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Memory_management
Say it: "Run the server under k6 at a steady 20 requests per second for 30 minutes, and log process.memoryUsage().heapUsed every 10 seconds. If it keeps climbing, take two heap snapshots ten minutes apart with node --inspect and tell me which object count grew. Every cache here needs a size limit."
Described as: "it gets slower the longer it runs", "restarting fixes it for a while", "memory keeps going up", "it crashes after a few days", "the tab uses more and more memory".
https://dev.dqnotes.com/performance/memory-leak

### Garbage collection
Also: GC, GC pause.
The part of a runtime that finds memory nothing can reach any more and frees it, so you don't have to. It costs work, and some of that work can pause your program. Those pauses are one reason a fast function sometimes isn't.
Most of the time you can ignore it, and an agent should too. It matters when you see a tail with no obvious cause and code that creates a lot of short-lived objects in a hot loop. Check with a profile first: rewriting code to avoid allocation on a hunch makes it worse to read and may not help.
Not Memory leak: A collector frees what is unreachable. A leak is memory that is still reachable, so the collector correctly leaves it alone. More collecting can't fix a leak.
Seen in: V8's blog describes "The Generational Hypothesis" as "most objects die young", so collecting young objects is cheap: it only pays for those that survive. It says the Orinoco project took "a sequential, stop-the-world garbage collector" and made it "a mostly parallel and concurrent collector with incremental fallback". That post is from January 2019, and engines have changed since.
Source: V8 blog, Trash talk: the Orinoco garbage collector, 3 January 2019, https://v8.dev/blog/trash-talk
Say it: "Before you rewrite this loop to avoid allocating, run it with node --cpu-prof and tell me what share of the profile is (garbage collector). If it is under 5%, leave the code alone."
Described as: "random pauses for no reason", "most requests are fast and a few stall", "does JavaScript free memory by itself", "the profile says garbage collector".
https://dev.dqnotes.com/performance/gc

### Core Web Vitals
Also: LCP, INP, CLS.
Three scores for how a page feels. LCP: how soon the main content appears. INP: how soon the page responds when you tap. CLS: how much things jump around. Good means LCP within 2.5 seconds, INP within 200 ms and CLS at 0.1 or less.
An agent can't feel a page. It sees that the tests pass, and it may never load the page on a slow phone. These numbers put what a person feels into a form it can read and act on, and be held to. A fast laptop on good wifi will flatter all three, so look at what real visitors got.
Not Lab score: A Lighthouse run is one test on one machine, useful for finding causes. Field data is what your real visitors got. Core Web Vitals are judged at the 75th percentile of real page loads, on mobile and desktop separately.
Seen in: As read on 5 November 2026, web.dev's table gives good and poor as: LCP up to 2500 ms and over 4000 ms, INP up to 200 ms and over 500 ms, CLS up to 0.1 and over 0.25, all at the 75th percentile. A page passes only if it meets all three. INP took over from First Input Delay in 2024. CLS has a trap for testers: "Layout shifts that occur within 500 milliseconds of user input will have the hadRecentInput flag set, so they can be excluded from calculations." And "Composited animations using translate can't impact other elements, and so don't count toward CLS."
Source: web.dev, Web Vitals, read 5 November 2026, https://web.dev/articles/vitals
Source: web.dev, How the Core Web Vitals thresholds were defined, read 5 November 2026, https://web.dev/articles/defining-core-web-vitals-thresholds
Source: web.dev, Cumulative Layout Shift, read 5 November 2026, https://web.dev/articles/cls
Source: web.dev, Optimize Cumulative Layout Shift, read 5 November 2026, https://web.dev/articles/optimize-cls
Source: web.dev, Interaction to Next Paint, read 5 November 2026, https://web.dev/articles/inp
Say it: "Add the web-vitals library and send LCP, INP and CLS to our analytics on every page view. In the CI Lighthouse run, fail if CLS is over 0.1 on / or /pricing. Give every image width and height, and don't insert anything above existing content after load."
Described as: "the page jumps when the image loads", "I went to tap and the button moved", "Google says my site is slow", "it takes ages for anything to appear", "I tap and nothing happens for a moment".
https://dev.dqnotes.com/performance/core-web-vitals

### Bundle size
How much code a page makes the browser download. A browser then has to unpack it, read it and run it, so on a cheap phone a big bundle costs time long after the download has finished.
Adding a package is one line for an agent, and it won't notice it just added 200 KB for one date format. Nothing fails, because each addition looks harmless, and no one reviewing the diff sees the size at all. You need something to fail when it grows.
Not Page weight: Page weight counts everything: images, fonts, the lot. An image only has to be decoded, while JavaScript has to be parsed, compiled and run, so the same number of bytes costs more as code.
Seen in: The Web Almanac 2025 puts the median mobile home page at 632 KB of JavaScript, in its July 2025 data. It says 100 KB of JavaScript "is work", since the CPU "must parse, compile, and execute that code". Size Limit is a tool that fails CI if a bundle goes over a limit, and its README suggests adding 25% to the current size for that limit.
Source: HTTP Archive, Web Almanac 2025, Page Weight, https://almanac.httparchive.org/en/2025/page-weight
Source: Size Limit, https://github.com/ai/size-limit
Say it: "Before adding a dependency, tell me how much it adds to the production bundle after minifying and compressing. Run size-limit and show me the figure. If it is over 10 KB, look for a smaller package or a few lines of our own first."
Described as: "the site takes forever on my phone", "how big is our JavaScript", "it added a huge library for one function", "why is the download so big".
https://dev.dqnotes.com/performance/bundle-size

### Lazy loading
Not loading something until it is needed: an image as it nears the screen, a chart's code when someone opens the chart. The first view gets lighter, and anyone who never scrolls that far never pays for it.
It's an easy thing to ask an agent for, and a common way to do harm. Told to lazy-load the images, it may well do every one of them, including the big one at the top, and that makes the main content appear later, which is the number you were trying to improve.
Not Preloading: Lazy loading delays what might not be needed. Preloading fetches early what surely will be. A good page does both, to different things.
Seen in: web.dev: "Don't lazy-load images that are likely to be in-viewport when the page loads, especially LCP images." This site does it too: each bench on this page only mounts as you near it, and the code for all of them loads when the first one is needed. One bench below counts them for you.
Source: web.dev, Browser-level image lazy loading, read 5 November 2026, https://web.dev/articles/browser-level-image-lazy-loading
Say it: "Add loading="lazy" to images below the fold only. Keep the first image in the page eager with fetchpriority="high", and give every image width and height. Show me LCP from a Lighthouse run before and after."
Described as: "only load images as you scroll", "the page loads things nobody looks at", "the first image appears late", "load it when it is needed".
https://dev.dqnotes.com/performance/lazy-loading

### Image optimisation
Sending an image no bigger than it will be shown, in a format that compresses well, with its size stated in the page. Images are often the largest thing a page downloads, so this can be the cheapest big win going.
An agent asked to add a picture adds the file you gave it: a 4000-pixel photo, in a box 400 wide. Nothing complains, and on a fast connection nobody notices. So tell it the display size and the format as part of the job, because neither shows up in a test.
Not Compression alone: Turning the quality down helps some. Sending fewer pixels usually helps far more, since halving the width leaves a quarter of the pixels. Do that first, then pick the format.
Seen in: The Web Almanac 2025 gives the median mobile home page 911 KB of images, the largest type it lists. MDN says AVIF "offers much better compression than PNG or JPEG", and that you should include a fallback when using it. web.dev: "Always include width and height size attributes on your images and video elements", so the browser can hold the space and nothing jumps when the image arrives.
Source: HTTP Archive, Web Almanac 2025, Page Weight, https://almanac.httparchive.org/en/2025/page-weight
Source: MDN, Image file type and format guide, https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats/Image_types
Source: web.dev, Optimize Cumulative Layout Shift, read 5 November 2026, https://web.dev/articles/optimize-cls
Say it: "hero.png is 4000 pixels wide and shown at 800. Generate 800 and 1600 wide versions in AVIF and WebP, serve them with srcset and sizes in a picture element with a JPEG fallback, and set width and height. Show me the bytes before and after."
Described as: "the photos make the page really slow", "why is this image so big", "which image format should I use", "the image looks fine but takes ages".
https://dev.dqnotes.com/performance/image-optimisation

### Performance budget
A limit you set in advance on something like bundle size or p95 latency, with a check that fails when it is broken. Speed then stops being something you remember to care about, and becomes something that says no.
This is backpressure for speed. Speed tends to get worse a little at a time, each change too small to notice, and an agent adds them faster than anyone can watch. With a budget in CI the agent gets refused in its own loop, with a number, and has to deal with it.
Not Target: A target is what you hope for. A budget is what stops the merge. Setting one a bit above where you are today and ratcheting it down works better than an ideal number that fails from day one and gets ignored.
Seen in: web.dev: "A performance budget is a set of limits imposed on metrics that affect site performance." k6 turns a threshold into a failed run, and Size Limit does the same for bundles in CI. Both are tools an agent can run in its own loop and read the failure from.
Source: web.dev, Performance budgets 101, read 5 November 2026, https://web.dev/articles/performance-budgets-101
Source: Grafana k6 docs, Thresholds, https://grafana.com/docs/k6/latest/using-k6/thresholds/
Source: Size Limit, https://github.com/ai/size-limit
Say it: "Add a CI job that fails on any of these: main bundle over 200 KB compressed by size-limit, k6 p(95) over 300 ms on GET /orders, Lighthouse CLS over 0.1. The failure message must name the budget and the number. Show me each one failing once."
Described as: "it keeps getting slower bit by bit", "how do I stop the site getting slower", "fail the build if it gets too big", "nobody notices until it is too late".
https://dev.dqnotes.com/performance/performance-budget

### Cost per task
What it really costs to get one piece of work finished: every try, every check and the failures you paid for and threw away. The price per token is only a price list. This is the number you can compare.
Comparing models on price per token is like comparing cars by the cost of a litre of fuel. A cheap model that takes three tries can lose to a dear one that takes one, and a wrong result that nobody catches can cost more than all of them. Measure it on your own tasks.
Not Token cost: Chapter 5's token cost is one request's price. Cost per task is everything it took to get an accepted result, including retries and review.
Seen in: Our own experiment (Claude Haiku 5.5, 8 one-function tasks, 15 runs per cell, 480 runs, no real repo) spent $2.51 over 1,097 calls: $1.50 on 617 writing attempts and $1.00 on 480 reviews. Writing cost per run: vague brief, one shot $0.0020, with harness $0.0038. Right words, one shot $0.0024, with harness $0.0043. The harness nearly doubled the cost per run, at an average of 1.6 tries. It took the pass rate from 88% to 91% on the vague brief and 98% to 100% on the worded one. Those are tiny tasks on a very cheap model, so read the ratios and not the dollars.
Say it: "Run these 10 tasks from git log on each candidate with claude -p --output-format json. Record total_cost_usd, the number of tries, and whether our test suite passed. Give me a table of cost per passing task, not cost per run."
Described as: "what does one feature actually cost", "is the cheap model really cheaper", "it took three tries so what did that cost", "how much did that agent run cost me".
https://dev.dqnotes.com/performance/cost-per-task

### Spending tokens well
Deciding what is worth putting in front of the model. Every turn sends the whole conversation again, so a big log read early isn't paid for once. You pay for it again on each turn until the session ends.
The agent decides what to read, and it has no reason to care what that costs you. Long test output, whole files and old conversation all ride along on every later turn. Trimming these saves money, and leaves room for the code. Chapter 7 is about what that does to quality. This is the bill.
Not Using a cheaper model: A cheaper model pays less per token for the same pile. Spending well shrinks the pile. They multiply, and shrinking the pile tends to help the work as well.
Seen in: Claude Code's docs: "Token costs scale with context size: the more context Claude processes, the more tokens you use". Their advice includes to "Use /clear to start fresh when switching to unrelated work." Its /usage command shows the session's tokens, and in print mode --max-budget-usd stops a run once its estimated spend reaches a limit, which is a client-side estimate and not a bill. Read 5 November 2026.
Source: Claude Code docs, Manage costs effectively, read 5 November 2026, https://code.claude.com/docs/en/costs
Source: Claude Code docs, CLI reference, read 5 November 2026, https://code.claude.com/docs/en/cli-reference
Say it: "Don't read the whole test log. Run bun test 2>&1 | tail -30, and if a test failed, grep for its name. Use a subagent for the codebase search and bring back only the file paths. Then /usage, and tell me what used the most."
Described as: "why did that session cost so much", "it read the whole log file", "how do I use fewer tokens", "my usage ran out really fast", "long chats get expensive".
https://dev.dqnotes.com/performance/token-budget

### Model routing
Sending each job to a model that fits it, instead of one model for everything: a cheap one for searching and summaries, an expensive one for planning and hard bugs. The router can be a rule, a setting or a person.
The prices differ by up to a hundred times, so on tokens alone routing nearly always wins. The danger is a cheap model that fails without saying so. A failure you see costs one retry. A failure that passes for success costs whatever it breaks later. Route freely where a check can catch it.
Not Model tiers: Tiers are the price list from chapter 5. Routing is the decision made from it, one job at a time.
Seen in: Anthropic's pricing page says: "Choose Haiku for simple tasks, Sonnet for most production workloads, and Opus for the most complex reasoning." Claude Code has a setting, opusplan, that "uses opus during plan mode, then switches to sonnet for execution", and a subagent can set its own model. Read 5 November 2026. Our experiment used a single model, so we have no recording of our own that compares routes, and the bench's failure rates are for you to set.
Source: Anthropic docs, Pricing, read 5 November 2026, https://platform.claude.com/docs/en/about-claude/pricing
Source: Claude Code docs, Model configuration, read 5 November 2026, https://code.claude.com/docs/en/model-config
Say it: "Set model: haiku on the search subagent and leave planning on opus. Run our 10 sample tasks both ways, score them with bun test and the holdout scenarios, and show me cost per passing task for each. If the routed version misses anything the tests catch, keep that step on the bigger model."
Described as: "use the cheap model for simple stuff", "do I need the best model for everything", "plan with one and build with another", "how do I cut my bill without getting worse results".
https://dev.dqnotes.com/performance/model-routing

### Cache hit rate
The share of a request's input tokens that came from the prompt cache. Because an agent resends everything each turn, a high rate can cut the input bill several times over, and a low one can quietly cost more than having no cache at all.
One changing byte near the top of the prompt can send this to zero, and nothing fails. The agent works just as well, and you pay more for it. So this is a number to watch: read it from the usage fields, and treat a fall as you would treat a failing check.
Not Hit rate of a normal cache: A normal cache hit skips the work and gives back the answer. A prompt cache hit makes the input cheap, and you still pay in full for the output, so even at a 100% rate the bill doesn't go to zero.
Seen in: Read 5 November 2026. Writing to the cache costs 1.25 times the input price, and lasts 5 minutes, refreshed free on each hit. Reading costs 0.05 times the input price on Opus 5.5 and Sonnet 5.5, 0.025 times on Fable 5.1, and 0.1 times on Haiku 5.5. In the usage fields, input is the sum of input_tokens, cache_creation_input_tokens and cache_read_input_tokens. On Haiku 5.5 a prompt over 100,000 tokens pays higher rates for the whole request, "even when part of its prompt is a cache hit".
Source: Anthropic docs, Pricing, read 5 November 2026, https://platform.claude.com/docs/en/about-claude/pricing
Source: Anthropic docs, Prompt caching, read 5 November 2026, https://platform.claude.com/docs/en/build-with-claude/prompt-caching
Say it: "Log usage on every request and work out cache_read_input_tokens divided by the sum of input_tokens, cache_creation_input_tokens and cache_read_input_tokens. Fail the eval if the second turn onwards averages under 80%. Move the date and anything else that changes out of the system prompt."
Described as: "cache_read_input_tokens is zero", "the bill went up and nothing changed", "is my prompt cache working", "it got expensive after I came back from lunch".
https://dev.dqnotes.com/performance/cache-hit-rate

### Cost of an agent bill
Also: Unit economics.
Working out the whole bill from its parts: people, tasks per day, cost per task. Once you have those three, you can tell what would double it, what a cut would save, and whether the work was worth what it cost.
Agent spending grows quietly and spreads across people and machines. A loop running overnight costs the same whether anyone looks or not. If you know your cost per task, a surprising bill becomes a question you can answer. It also gives you a real answer when someone asks if this is worth it.
Not Cost per token: A token price is what the provider charges. The bill also depends on how you work: how long the sessions run, how much they read, and how many tries they take. Those usually vary more than the price does.
Seen in: Claude Code's docs, read 5 November 2026: "Across enterprise deployments, the average cost is around $13 per developer per active day and $150-250 per developer per month, with costs remaining below $30 per active day for 90% of users." That is Anthropic's own figure, for its own product, and "costs vary widely". It compares oddly with McCarthy's $1,000 a day in chapter 5, which is one company's position and not a measurement. Our $2.51 experiment shows how much the unit matters: 480 tiny runs on the cheapest tier.
Source: Claude Code docs, Manage costs effectively, read 5 November 2026, https://code.claude.com/docs/en/costs
Source: McCarthy, Software Factories And The Agentic Moment, 2026, https://factory.strongdm.ai/
Say it: "Pull last month's usage and split it by person and by repository. Divide by merged pull requests to get a cost per pull request, and list the five most expensive sessions with what each one was doing. Set --max-budget-usd 5 on every headless job in CI."
Described as: "how much will agents cost my team", "is it worth what we pay", "the bill doubled and I do not know why", "how much does a developer use per day".
https://dev.dqnotes.com/performance/unit-economics

## 16. Shipping and running

Getting code to users safely and knowing what it does there: deploys, flags, rollbacks, logs and incidents.

### Production
Also: Prod, Live.
Where your software runs for real, with real users, real data and real money. Every other copy of it exists to prepare for this one.
It is the one place where a mistake reaches people, and the only place that can tell you what was true. An agent should have no way in unless you gave it one on purpose.
Seen in: In July 2025 Fortune reported that an agent on Replit had deleted Jason Lemkin's data, and that it "told Lemkin that a retrieval, or rollback, function would not work". Lemkin recovered the data by hand. Replit's chief executive wrote: "Replit Agent in development deleted data from the production database. Unacceptable and should never be possible". The promised fix included "automatic separation between development and production databases". We are relying on Fortune's report, because we found no post-mortem from Replit itself, and accounts differ on details, including how far this was really production.
Source: Fortune, on the Replit database deletion, 2025, https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure
Say it: "Use the staging database URL, and read-only production logs if you need them. Production credentials don't go in .env on this machine. If a job has to touch production, give me the exact command and I'll run it."
Described as: "the real live site customers use", "where it actually runs for people", "the real database and not the test one".
https://dev.dqnotes.com/ship/production

### Environment
Also: Deployment environment.
One complete place your software can run, with its own address, its own data and its own settings. Development on your machine, staging and production are the usual three.
The more two environments differ, the less a pass in one tells you about the other. And an agent that can't tell which one it is in will sooner or later run the right command in the wrong place.
Not Environment variable: An environment is the whole place. An environment variable is one named setting handed to a program in that place.
Seen in: Twelve-factor says: "Keep development, staging, and production as similar as possible", and names three gaps to close: time, personnel and tools. OpenAI's harness engineering team took the idea down to one per task: "we made the app bootable per git worktree, so Codex could launch and drive one instance per change."
Source: The Twelve-Factor App, Dev/prod parity, https://12factor.net/dev-prod-parity
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Say it: "Add a dev:fresh script so you can boot your own copy of this app on a free port, with a throwaway SQLite database and seed data. Test against that. Print which environment you are in on the first log line."
Described as: "it works on my machine but not on the server", "the different copies of the app", "a separate place to try it out".
https://dev.dqnotes.com/ship/environment

### Staging
Also: Pre-production, Preprod.
An environment that copies production as closely as you can afford, where a release runs before real users get it.
It catches what shows up only when the pieces run together: a bad setting, a missing variable, a migration that won't run. It can't show what only real traffic and real data reveal, and a pass there can feel like more proof than it is.
Not Smoke test: Staging is a place. A smoke test is a quick check that a release basically works, and you can run one in staging or in production.
Seen in: People disagree about how much it is worth, and both sides have evidence. Google's report on its 12 June 2025 Cloud outage said of the failing code: "If this had been flag protected, the issue would have been caught in staging." Charity Majors argues from the other end: "Testing in production is a superpower. It's our inability to acknowledge it that's the trouble." These fit together. Rehearse what you can, then roll out so that whatever you missed stays small.
Source: Google Cloud, incident report for 12 June 2025, https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW
Source: Majors, I test in prod, Increment, 2019, https://increment.com/testing/i-test-in-production/
Say it: "Deploy to staging and run the smoke tests against its URL. Then tell me what staging can't show us: it has 200 rows and one user. List anything that needs real traffic to prove, and we'll watch for that in the canary."
Described as: "a copy of the site to try it on first", "test it somewhere that looks like the real thing", "it passed on the test server and broke on the real one".
https://dev.dqnotes.com/ship/staging

### Build
The step that turns source code into the thing that actually runs: compiling, bundling, and copying everything into one output.
It is the first time all of the code has to fit together. A build also pulls things in from outside, such as dependencies, so it is where a change nobody made can arrive.
Not Deployment: A build makes the thing. A deployment puts it somewhere it can run.
Seen in: Twelve-factor: "The build stage is a transform which converts a code repo into an executable bundle known as a build." The build for this site is one line in its package.json: `bun scripts/export.ts && vite build`. Two steps, and because of the &&, if the first fails the second never runs.
Source: The Twelve-Factor App, Build, release, run, https://12factor.net/build-release-run
Say it: "Run bun run build from a clean checkout, in CI and not on your machine, and show me the output. If it needs anything that isn't in the repo or the lockfile, tell me what, because that means it builds only here."
Described as: "turning the code into what actually runs", "the compile step", "it runs in dev but the production bundle fails".
https://dev.dqnotes.com/ship/build

### Semantic versioning
Also: Semver, Version range, Caret range.
A three-part version number, like 2.4.1, that makes a promise. The last number means a bug fix, the middle one a new feature that breaks nothing, and the first means something may break.
It lets tools take safe updates on their own. But it is a promise made by a person, and nothing enforces it, so a patch can still break you.
Not Pinning: A range like ^1.2.3 is a rule for which versions you will accept, so what you get can change. A pin names one version and stays there.
Seen in: The spec: "MAJOR version when you make incompatible API changes", and "Major version zero (0.y.z) is for initial development. Anything MAY change at any time." Hyrum's law explains why the promise leaks: "With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody." A small real trap: this site's package.json says "typescript": "5.9". That looks pinned, and it means any 5.9.x.
Source: Semantic Versioning 2.0.0, https://semver.org/
Source: Wright, Hyrum's Law, https://www.hyrumslaw.com/
Say it: "Bump zod from 4.1 to 4.6. It is a minor, so it should be safe, and you will check that and not assume it: run the full tests, read the changelog, and tell me anything we use whose behaviour changed."
Described as: "what do the three numbers in a version mean", "what does the little hat before the version do", "is it safe to update this package".
https://dev.dqnotes.com/ship/semver

### Pinning
Also: Version pinning, Frozen lockfile.
Fixing the exact version of something you depend on, by number or by hash, so that it changes only when you decide. A range is a rule. A pin is this one and no other.
Without a pin, a build can change or break on a day nobody touched the code. But a pin holds still in both directions: a security fix doesn't arrive until someone moves it. So pin, and have something move the pins for you, with tests in front.
Not Lockfile: Pinning is the decision. A lockfile is the usual way to carry it out, because it records every exact version, including the ones you never asked for.
Seen in: npm says of its lockfile: "It describes the exact tree that was generated, such that subsequent installs are able to generate identical trees". Bun, with the frozen flag: "bun installs the exact versions specified in the lockfile and does not update it." This site's CI file uses that flag, and pins its actions by full commit hash, with the version in a comment: `actions/checkout@3d3c42e… # v7.0.1`. GitHub's security guidance gives the reason: "a tag can be moved or deleted". It now ranks its newer workflow lockfiles above hash pins, where you can adopt them.
Source: npm docs, package-lock.json, https://docs.npmjs.com/cli/v10/configuring-npm/package-lock-json
Source: Bun docs, bun install, https://bun.com/docs/pm/cli/install
Source: GitHub docs, Secure use reference, https://docs.github.com/en/actions/reference/security/secure-use
Say it: "Use bun install --frozen-lockfile in CI, so that a mismatch fails the build. Pin each GitHub Action to a full commit SHA with the version in a comment. Add Dependabot to open weekly PRs that move them, so the pins don't go stale, and let CI judge each one."
Described as: "it broke and nobody changed anything", "make sure everyone gets the same version", "the build is different today and the code is the same".
https://dev.dqnotes.com/ship/pinning

### SBOM
Also: Software bill of materials.
A list of every component in your software, including what your dependencies depend on, with exact versions. An ingredients label, generated from what you really built.
When an advisory lands, the first question is "do we have this?", and the answer is often in a package nobody here chose. With a list it takes seconds. An agent can add a dependency in one line of a diff, so the list is worth more than anyone's memory.
Not Lockfile: A lockfile is for your package manager, to repeat an install. An SBOM is for people and scanners, in a standard format, and it can also cover what is inside a container image.
Seen in: NTIA: "A 'software bill of materials' (SBOM) is a nested inventory for software, a list of ingredients that make up software components." Syft is one tool that generates one, and it can write both common formats, CycloneDX and SPDX: `syft ./my-project -o cyclonedx-json`. The bench below reads this site's real lockfile, which held 302 packages from 22 direct dependencies when we wrote this.
Source: NTIA, Software Bill of Materials, https://www.ntia.gov/page/software-bill-materials
Source: Anchore, Syft, https://github.com/anchore/syft
Say it: "Add a CI step that generates a CycloneDX SBOM with syft from the build output, and saves it next to the release. Then answer from it, and not from package.json: do we ship any version of zod below 4?"
Described as: "a list of everything we depend on", "do we use the package in that security alert", "what is actually in this thing we shipped".
https://dev.dqnotes.com/ship/sbom

### Artefact
Also: Artifact, Build artifact, Release artifact.
The finished output of a build, stored with a version so that you can move exactly that thing between environments. A bundle, a zip, a container image.
If you build again for production, what you tested isn't what you ship. Build once and move the same artefact along, and a pass in staging says something real about production.
Not Source code: Source is what you edit. An artefact is what came out of one particular build, and you never edit it: you make a new one.
Seen in: Twelve-factor: "The twelve-factor app uses strict separation between the build, release, and run stages." And: "Releases are an append-only ledger and a release cannot be mutated once it is created."
Source: The Twelve-Factor App, Build, release, run, https://12factor.net/build-release-run
Say it: "Have CI build once on merge and label the output with the commit SHA. Staging and production both deploy that same artifact, with no second build in the deploy job. Show me that the hash matches on both sides."
Described as: "the thing that comes out of the build", "what we tested is not what we deployed", "build it once and move it along".
https://dev.dqnotes.com/ship/artifact

### Container
Also: Docker container, Container image.
A way to package a program with all the files it needs and run it as an isolated process, so that it behaves the same on your laptop and on a server.
It makes "works on my machine" a smaller problem, because the machine's contents travel with the program. It also gives an agent a place to run commands where a mistake is more likely to stay inside.
Not Virtual machine: A virtual machine runs a whole operating system of its own. Containers share the host's kernel, so they start faster and are less strongly separated.
Seen in: Docker's own docs: "Simply put, containers are isolated processes for each of your app's components." And on the limit: "If you run multiple containers, they all share the same kernel".
Source: Docker docs, What is a container?, https://docs.docker.com/get-started/docker-concepts/the-basics/what-is-a-container/
Say it: "Write a Dockerfile: pin the base image by digest, copy only what the app needs, and run as a non-root user. Build it in CI, and run it here with docker run before you tell me it works."
Described as: "package it up with everything it needs", "what is Docker for", "run it the same everywhere".
https://dev.dqnotes.com/ship/container

### Serverless
Also: Functions as a service, FaaS.
You hand a provider your code, and they run it on their machines when requests arrive. There are still servers. They just aren't yours to manage.
There is nothing of your own to patch or keep running. The price is limits you don't choose, code that can't count on anything staying in memory between requests, and a dependence on one provider.
Not Container: With a container you still choose where it runs and keep it running. With serverless the provider decides, and your code has to work with no machine of its own.
Seen in: Mike Roberts: "Fundamentally, FaaS is about running backend code without managing your own server systems or your own long-lived server applications." And the catch: "With any outsourcing strategy you are giving up control of some of your system to a third-party vendor." This site's wrangler.jsonc configures a Cloudflare Worker, which is this kind of thing.
Source: Roberts, Serverless Architectures, 2018, https://martinfowler.com/articles/serverless.html
Say it: "This runs on Cloudflare Workers, so don't keep state in a module-level variable and expect it on the next request, which may not reach the same instance. Put it in the database, and show me every place you read it."
Described as: "I do not want to manage a server", "just run my function when someone calls it", "where did my variable go between requests".
https://dev.dqnotes.com/ship/serverless

### Infrastructure as code
Also: IaC, Terraform.
Describing your servers, databases and networks in files kept in the repo, and letting a tool make the real world match them, in place of clicking around a console.
Changes become reviewable and repeatable. It also means one wrong command can change everything at once. Agents are good at writing these files. The danger is in letting them run the result.
Not Configuration: Config sets values for a running app. Infrastructure as code defines what exists for it to run on, which includes what would be deleted.
Seen in: In February 2026 Alexey Grigorev let an agent run Terraform, and "a terraform auto-approve command inadvertently wiped out all production infrastructure". Their account: "The database, VPC, ECS cluster, load balancers, and the bastion host were gone." And "the full recovery took about 24 hours". What they changed: "Agents no longer execute commands", and "Every destructive action is run by me." Terraform's normal flow has a stop built in: "On approval, Terraform performs the proposed operations". Auto-approve removes that stop.
Source: Grigorev, How I dropped our production database, 2026, https://aishippingblog.com/p/how-i-dropped-our-production-database
Source: HashiCorp, What is Terraform?, https://developer.hashicorp.com/terraform/intro
Say it: "Write the Terraform and run terraform plan -out=tfplan. Show me every line that says destroy or replace. Never run apply and never use auto-approve: I'll apply that saved plan myself. If you can't read the remote state, stop and say so."
Described as: "set up the servers from a file", "the agent wants to run terraform", "so nobody has to click around the cloud console".
https://dev.dqnotes.com/ship/infrastructure-as-code

### Config and secrets
Also: Environment config, Deploy secrets.
Whatever changes between deploys belongs outside the code: addresses, keys, passwords. The same artefact can then run anywhere, reading its settings from wherever it is running.
Hardcoded config means rebuilding to move, and a key in the repo is a key in every clone, every log and every agent's context. Deleting it later doesn't help, because the history remembers: rotate it.
Not Secret: A secret is the kind of config that grants access. All secrets are config, and not all config is secret: a retry count can live in the repo.
Seen in: Twelve-factor gives the test: "A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials." Cloudflare says the same about its own config file: "Do not use vars to store sensitive information in your Worker's Wrangler configuration file. Use secrets instead."
Source: The Twelve-Factor App, Config, https://12factor.net/config
Source: Cloudflare docs, Secrets, https://developers.cloudflare.com/workers/configuration/secrets/
Say it: "Move the API key and database URL out of the code and read them from the environment. Fail at boot with a clear message if one is missing. List the names in .env.example with no values. Then check the git history, and tell me if the old key was ever committed, because then it needs rotating."
Described as: "where do I put the API key", "different settings for test and live", "could we publish this repo tomorrow".
https://dev.dqnotes.com/ship/env-secrets-deploy

### Deployment
Also: Deploy.
Putting a new version where it can run. The aim is to make it automatic, small and boring, so that it can happen often.
A manual step is one that a tired person or an eager agent will skip. So deploying should be a script, the same every time, and what matters most is what stands in front of that script.
Not Release: A deployment puts code on servers. A release turns it on for users. With flags they can be days apart, and that gap makes both of them safer.
Seen in: This site's own files make a useful small example, including a gap that is still open. The deploy script in package.json is now `bun run check && bun run build && wrangler deploy && node scripts/check/live.mjs`, so deploying from that script waits for every check and then checks the live site. It wasn't always: the first deploy script skipped the check. But anyone can still type `wrangler deploy` and walk around it, and `.github/workflows/ci.yml` has no deploy job. A gate only works if the deploy has to go through it. GitHub offers a way: "Use required reviewers to require a specific person or team to approve workflow jobs that reference the environment."
Source: GitHub docs, Deployments and environments, https://docs.github.com/en/actions/reference/workflows-and-actions/deployments-and-environments
Source: Cloudflare docs, GitHub Actions, https://developers.cloudflare.com/workers/ci-cd/external-cicd/github-actions/
Say it: "Add a deploy job to ci.yml with needs: check, so that it can't start unless check passed. Give it a GitHub environment called production with a required reviewer, and put CLOUDFLARE_API_TOKEN in that environment's secrets. Then tell me every way someone could still deploy around it."
Described as: "how does the code get from my machine to the website", "it is merged but it is not live yet", "pushing the new version to the server".
https://dev.dqnotes.com/ship/deployment

### Continuous delivery
Also: CD.
Keeping your software in a state where it could go to production at any moment, so that releasing is a decision and not a project.
Small changes are easier to check, and easier to take back. This matters more with agents, because they produce changes faster, and saving those up into one big release piles up risk that nobody can review.
Not Continuous deployment: Delivery means every change could go out, and someone chooses when. Deployment means every change that passes does go out, with no person deciding.
Seen in: Martin Fowler: "Continuous Delivery is a software development discipline where you build software in such a way that the software can be released to production at any time." The test he gives is about priorities: "your team prioritizes keeping the software deployable over working on new features".
Source: Fowler, Continuous Delivery, 2013, https://martinfowler.com/bliki/ContinuousDelivery.html
Say it: "Split this into four PRs that can each go to production alone: the migration, the reader, the writer, and the UI behind a flag. Main has to be deployable after each one. If it can't be, tell me which and why."
Described as: "we should be able to ship at any time", "we only release once a month and it is terrifying", "what is the CD in CI/CD".
https://dev.dqnotes.com/ship/continuous-delivery

### Trunk-based development
Also: TBD.
Everyone merges small changes into one main branch, at least daily, and nobody keeps a branch going for long. Unfinished work stays hidden behind flags.
The longer a branch lives, the more it and main both change, and conflicts grow faster than the time does. Parallel agents sharpen this, because each one is a branch.
Not Feature branch: A branch that lives a few hours is fine here. What this rules out is one that lives for weeks.
Seen in: trunkbaseddevelopment.com says to "resist any pressure to create other long-lived development branches". DORA's guidance is more exact: "branches in trunk-based development typically last no more than a few hours", "Have three or fewer active branches in the application's code repository.", and "Merge branches to trunk at least once a day." Its analysis rests on DORA data from 2016 and 2017, before coding agents. Whether three branches still makes sense when each agent has its own is an open question.
Source: Trunk Based Development, https://trunkbaseddevelopment.com/
Source: DORA, Trunk-based development, https://dora.dev/capabilities/trunk-based-development/
Say it: "Rebase on main before you start, keep this branch under a day, and merge behind the new-search flag even though it isn't finished. If you can't split it that small, stop and tell me why before you write more."
Described as: "the branch has been open for weeks and now it will not merge", "everyone works on main", "merge day is a nightmare".
https://dev.dqnotes.com/ship/trunk-based

### Feature flag
Also: Feature toggle, Kill switch, Percentage rollout.
A condition in your code that turns something on or off, for some or all users, without a deploy. The code ships switched off, and you switch it on when you are ready.
If a release goes wrong, turning it off takes seconds and not a rollback. A percentage lets you try it on a few people first. The cost is real: each flag is a second version of your code, which has to be tested and later removed.
Not Canary: A flag splits users inside one deployed version, and can be flipped at once. A canary splits traffic between two deployed versions, so it also covers what nobody thought to put behind a flag.
Seen in: Google Cloud's report on 12 June 2025 said the failing code "did not have appropriate error handling nor was it feature flag protected", and promised: "We will enforce all changes to critical binaries to be feature flag protected and disabled by default." Pete Hodgson on the cost: "Savvy teams view their Feature Toggles as inventory which comes with a carrying cost", and he cites "Knight Capital Group's $460 million dollar mistake" as the warning about unmanaged flags. Stable percentages come from a hash. LaunchDarkly: "The hash is deterministic, which means a context keeps receiving the same variation as long as the rollout percentages and the context's key don't change."
Source: Google Cloud, incident report for 12 June 2025, https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW
Source: Hodgson, Feature Toggles, on Martin Fowler's site, 2017, https://martinfowler.com/articles/feature-toggles.html
Source: LaunchDarkly docs, Percentage rollouts, https://launchdarkly.com/docs/home/releases/percentage-rollouts
Say it: "Put the new checkout behind a flag called new-checkout, off by default, with a hash of the user id picking the percentage, so nobody flips between versions. Test it both on and off. Add a ticket, with an owner, to delete the flag within two weeks of reaching 100%."
Described as: "turn it off without deploying", "release it to some users and not others", "ship it dark".
https://dev.dqnotes.com/ship/feature-flag

### Canary
Also: Canary release, Canary deployment, Gradual rollout.
Sending a small share of real traffic to the new version, comparing it with the old one, and only then sending everyone. If it is bad, few people met it.
Some problems appear only under real traffic, and a canary limits how many people find them. It only helps if you can tell in time, which takes enough requests and a real comparison. A tiny canary on a quiet site tells you nothing.
Not A/B test: An A/B test asks which version people prefer. A canary asks whether the new version is broken. It is the same split with a different question.
Seen in: Danilo Sato, on Martin Fowler's site, explains it as "slowly rolling out the change to a small subset of users before rolling it out to the entire infrastructure". After its global failure of 19 July 2024, which it reverted 78 minutes after release, CrowdStrike promised to "Implement a staggered deployment strategy for Rapid Response Content in which updates are gradually deployed to larger portions of the sensor base, starting with a canary deployment." On Cloudflare Workers: "Gradual deployments let you incrementally deploy new versions of your Worker by splitting traffic across versions." Note its catch: without version affinity, "consecutive requests from the same user - including page reloads and asset fetches - can be handled by different versions".
Source: Sato, Canary Release, on Martin Fowler's site, 2014, https://martinfowler.com/bliki/CanaryRelease.html
Source: CrowdStrike, Preliminary Post Incident Review, 2024, https://www.crowdstrike.com/en-us/blog/falcon-content-update-preliminary-post-incident-report/
Source: Cloudflare docs, Gradual deployments, https://developers.cloudflare.com/workers/configuration/versions-and-deployments/gradual-deployments/
Say it: "Deploy with wrangler versions deploy at 10%, and compare its 5xx rate with the old version's for 15 minutes. At 1,000 requests a minute, 10% is enough to judge. If it is worse, go back to 0% and tell me. If not, go to 50%, then 100%. "No alerts fired" doesn't count as a pass."
Described as: "try it on a few users first", "send 5% of traffic to the new version", "find out it is broken before everyone does".
https://dev.dqnotes.com/ship/canary

### Blue-green deployment
Also: Blue-green.
Two identical production environments. One serves users while you deploy to the other and check it. Then you switch the traffic across, and the old one stays ready to switch back to.
Going back becomes flipping a switch, and nobody meets a half-finished deploy. You pay for two environments while you keep both, and there is usually one database, which both versions have to work with.
Not Canary: Blue-green moves everyone at once, relying on checks beforehand and a quick way back. A canary moves a few people first and learns from them.
Seen in: Martin Fowler: "The fundamental idea is to have two easily switchable environments to switch between." And the part people forget: "The trick is to separate the deployment of schema changes from application upgrades."
Source: Fowler, Blue Green Deployment, 2010, https://martinfowler.com/bliki/BlueGreenDeployment.html
Say it: "Deploy to the idle environment and run the smoke tests on its private URL. Only then switch the load balancer, and keep the old one running for an hour. Any database change has to work with both versions, so do that first, separately."
Described as: "two copies and flip between them", "switch all the traffic over at once", "keep the old version running in case".
https://dev.dqnotes.com/ship/blue-green

### Rollback
Also: Roll back, Roll forward, Fix forward.
Going back to the previous version after a bad release. Rolling forward is the other choice: shipping a new version with the fix. Rolling back is usually faster, so you stop the harm first and fix it calmly.
How quickly you can undo decides how much risk each change carries. The trap is data. Code goes back and data doesn't, so one migration can quietly remove your way back, and you find out on the worst possible day.
Not Git revert: A revert changes the repo, and still has to be deployed. A rollback changes what is running, and the repo may disagree until you revert as well.
Seen in: Google's SRE book names "Rolling back changes safely when problems arise" among its core practices for changes. Cloudflare's docs are plain about the limit: "Resources connected to your Worker will not be changed during a rollback." And: "Errors could occur if using code for a prior version if the structure of data has changed between the version in the active deployment and the version selected to rollback to." And test it yourself, because an agent's word isn't enough: in the Replit case above, the agent said rollback would not work, and Lemkin got the data back by hand anyway.
Source: Google, Site Reliability Engineering, Introduction, https://sre.google/sre-book/introduction/
Source: Cloudflare docs, Rollbacks, https://developers.cloudflare.com/workers/configuration/versions-and-deployments/rollbacks/
Source: Fortune, on the Replit database deletion, 2025, https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure
Say it: "Before we deploy, tell me the rollback: the exact command, how long it takes, and what happens to rows written by the new version. If this migration makes that impossible, split it with expand and contract, so that going back works at every step. Then rehearse it on staging."
Described as: "undo the release", "put the old version back", "can we go back after this migration".
https://dev.dqnotes.com/ship/rollback

### Observability
Also: o11y, Monitoring.
How well you can work out what a running system is doing from the outside, using what it emits, including for problems you didn't predict.
It decides whether you find out from your signals or from your users. It also changes what an agent can do: if it can read logs and traces, it can check its own work, and you aren't the one relaying symptoms.
Not Monitoring: Monitoring watches for things you chose in advance and tells you they happened. Observability is about asking new questions afterwards. People use the two loosely, and the line is blurry.
Seen in: OpenTelemetry: "Observability lets you understand a system from the outside by letting you ask questions about that system without knowing its inner workings." OpenAI gave its agents the same access: "Agents can query logs with LogQL and metrics with PromQL." In this site's wrangler.jsonc you can see the small version: an `observability` block with `enabled` set to true. Cloudflare's docs now say that "all newly created Workers will come with the observability setting enabled by default", and an older Worker may not have it. So look, and don't assume.
Source: OpenTelemetry, Observability primer, https://opentelemetry.io/docs/concepts/observability-primer/
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Source: Cloudflare docs, Workers Logs, https://developers.cloudflare.com/workers/observability/logs/workers-logs/
Say it: "Before you ship this, give me a way to see it in production: log one JSON line per request with route, status, duration and release, and tell me the exact query that would show the new code failing. Run that query on staging and show me its output."
Described as: "how would we know if this broke", "I can not tell what it is doing in production", "let the agent see the logs".
https://dev.dqnotes.com/ship/observability

### Logs, metrics and traces
Also: Telemetry, Spans, The three pillars.
Three kinds of signal. A log is a timestamped event. A metric is a number over time, such as requests a second. A trace follows one request through every call it makes.
They answer different questions: a metric says something is wrong, a trace says where, and a log says what happened there. Give an agent only one of them, and it will guess at the rest.
Not Golden signals: Logs, metrics and traces are kinds of data. Latency, traffic, errors and saturation are what to measure first, mostly using metrics.
Seen in: OpenTelemetry: "A log is a timestamped message emitted by services or other components." And "A span represents a single unit of work or operation." The SRE book on where to start: "The four golden signals of monitoring are latency, traffic, errors, and saturation." OpenAI's harness work exposed all three: "Logs, metrics, and traces are exposed to Codex via a local observability stack that's ephemeral for any given worktree."
Source: OpenTelemetry, Observability primer, https://opentelemetry.io/docs/concepts/observability-primer/
Source: Google, Site Reliability Engineering, Monitoring Distributed Systems, https://sre.google/sre-book/monitoring-distributed-systems/
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Say it: "Checkout p95 went from 300 ms to 900 ms on Tuesday. Pull one slow trace and one normal trace, and compare them span by span. Tell me which span grew, then find the log lines for that trace id. Don't guess from the code until you have the trace."
Described as: "the graph says it is slow but not why", "see where the time went in one request", "what is the difference between a log and a metric".
https://dev.dqnotes.com/ship/logs-metrics-traces

### Structured logging
Also: Structured logs, JSON logs, Canonical log line.
Writing each log entry as named fields, usually JSON, and not as a sentence. A query can then ask for exact values, without guessing at the wording.
In an incident the question is "which users, which release?". Against free text, every answer is a guess at how someone phrased it, and it is wrong without saying so. An agent reading the logs does far better with fields it can query than with prose it has to interpret.
Not Print debugging: Print lines are for you, now. Structured logs are for a query, later, by someone who didn't write them.
Seen in: Twelve-factor: "Treat logs as event streams". Stripe's canonical log lines keep the ordinary logging and add one summary per request: "requests also emit one long log line at the end that includes many of their key characteristics."
Source: The Twelve-Factor App, Logs, https://12factor.net/logs
Source: Leach, Canonical Log Lines, Stripe, 2019, https://stripe.com/blog/canonical-log-lines
Say it: "Log one JSON line at the end of each request, with route, status, duration_ms, user_id, order_id and release. No names, emails, card numbers or tokens. Then give me the query for which users got a 502 on release v2, and run it on staging."
Described as: "which users were affected", "grep the logs and hope", "I can not search these log lines".
https://dev.dqnotes.com/ship/structured-logging

### SLI and SLO
Also: Service level indicator, Service level objective, SLA.
An SLI is a measure of how your service is doing, like the share of requests that succeed in time. An SLO is the target you set for that measure, like 99.9% over 30 days.
Without an agreed number, "is it reliable enough?" is an argument about feelings, and perfect isn't on offer. With one, you know when to keep building and when to stop and fix things. Choose what counts as good with care, because that choice can hide a real problem.
Not SLA: An SLA is a contract with consequences, such as refunds. An SLO is your own target, and it has no consequence unless you agree one.
Seen in: The SRE book defines an SLI as "a carefully defined quantitative measure of some aspect of the level of service that is provided", and an SLO as "a target value or range of values for a service level that is measured by an SLI." It tells you how to spot the difference from an SLA: "if there is no explicit consequence, then you are almost certainly looking at an SLO." Two warnings: "You shouldn't use every metric you can track in your monitoring system as an SLI." And: "It's both unrealistic and undesirable to insist that SLOs will be met 100% of the time."
Source: Google, Site Reliability Engineering, Service Level Objectives, https://sre.google/sre-book/service-level-objectives/
Say it: "Propose one SLI and one SLO for checkout: the share of POST /checkout requests that return under 500 ms and aren't 5xx, measured at the load balancer, over 30 days. Compute the last 30 days from real logs and tell me where we are before you suggest a target."
Described as: "how reliable does this have to be", "what does 99.9% actually measure", "agree a target for uptime".
https://dev.dqnotes.com/ship/sli-slo

### Error budget
The failure your SLO allows. At 99.9% over 30 days, that is 43.2 minutes of total outage. While there is budget left you ship, and when it is gone you stop and work on reliability.
It turns an argument into arithmetic, and it applies directly to agents. If they let you ship faster than your budget can absorb, the budget says slow down, whatever the velocity chart says.
Not Uptime: Uptime is what happened. An error budget is what you decided in advance you could afford, and what you agreed to do once it is spent.
Seen in: The SRE book: "The error budget provides a clear, objective metric that determines how unreliable the service is allowed to be within a single quarter." It adds: "This metric removes the politics from negotiations between the SREs and the product developers". And the rule: "If SLO violations occur frequently enough to expend the error budget, releases are temporarily halted". On the target: "100% is probably never the right reliability target". The book works in quarters and the Workbook's alerting chapter works in 30 days, which is what the bench uses.
Source: Google, Site Reliability Engineering, Embracing Risk, https://sre.google/sre-book/embracing-risk/
Source: Google, The Site Reliability Workbook, Alerting on SLOs, https://sre.google/workbook/alerting-on-slos/
Say it: "We run at 99.9% over 30 days, so we have 43.2 minutes. Tuesday's incident was 18 minutes at 30% failing, about 5.4 budget minutes. Work out what is left from the real logs. If we have spent over half in the first week, hold the risky migration and work on the cause."
Described as: "when should we stop shipping and fix things", "how much downtime can we afford", "product wants features and ops wants stability".
https://dev.dqnotes.com/ship/error-budget

### Alert
Also: Page, Burn-rate alert, Alert fatigue.
A signal that tells a person to act now. A good one fires when users are being hurt and someone can do something about it. A burn-rate alert does this by asking how fast the error budget is going.
An alert that cries wolf gets ignored, including on the day it is right, and one set too high misses the slow leak. Asking an agent to "add monitoring" can produce alerts by the dozen, so ask for few, and ask what each would have caught.
Not Threshold alert: A threshold fires when a number crosses a line, however little that costs. A burn-rate alert fires on how fast you are spending budget, so it can ignore a blip and still catch a slow leak.
Seen in: The SRE book: "Every page should be actionable." And: "When pages occur too frequently, employees second-guess, skim, or even ignore incoming alerts". The Workbook: "Burn rate is how fast, relative to the SLO, the service consumes the error budget." Its recommended set pages at 14.4 times the budget rate over 1 hour, and at 6 times over 6 hours, and files a ticket at 1 times over 3 days. Each also has a shorter window: "A good guideline is to make the short window 1/12 the duration of the long window".
Source: Google, Site Reliability Engineering, Monitoring Distributed Systems, https://sre.google/sre-book/monitoring-distributed-systems/
Source: Google, The Site Reliability Workbook, Alerting on SLOs, https://sre.google/workbook/alerting-on-slos/
Say it: "Replace these 14 threshold alerts with a burn-rate alert on the checkout SLO: page at 14.4x over 1 hour and 5 minutes, page at 6x over 6 hours and 30 minutes, ticket at 1x over 3 days and 6 hours. Replay last month, and show me how many pages we would have had and what each one caught."
Described as: "the alerts go off all the time and we ignore them", "it was broken for days and nothing told us", "get woken up only when it matters".
https://dev.dqnotes.com/ship/alert

### On-call
Also: Pager, Rotation.
The rota that decides who answers when something breaks, at any hour. That person has to respond quickly, and needs the access and the knowledge to act.
An agent can write the code and it can't be accountable at 3am. Whoever is on call has to understand what shipped, and a codebase nobody has read is a bad thing to be woken up by.
Not Code ownership: Ownership says who decides about the code. On-call says who gets woken up. Healthy teams keep those close together.
Seen in: Google's numbers for its own teams: "no more than 25% can be spent on-call", which with two people on week-long shifts is why "the minimum number of engineers needed for on-call duty from a single-site team is eight". It counts "the maximum number of incidents per day is 2 per 12-hour on-call shift", from an average where handling one "takes 6 hours". Response times: "Typical values are 5 minutes for user-facing or otherwise highly time-critical services". Canva asks engineers to "take full ownership of any code they produce, whether they wrote it themselves or with AI assistance."
Source: Google, Site Reliability Engineering, Being On-Call, https://sre.google/sre-book/being-on-call/
Source: Canva Engineering, Yes, you can use AI in our interviews, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Say it: "Write the on-call note for this change: what it touches, how we would know it is broken, the exact command to turn it off, and what you aren't sure about. Assume the reader was just woken up, has never seen this code and can't ask you."
Described as: "who gets the phone call at 3am", "who looks after it once it is live", "taking turns to be responsible for the site".
https://dev.dqnotes.com/ship/on-call

### Incident
Also: Outage, Incident commander, Mitigation.
An unplanned event that is hurting users, or about to, and needs people to coordinate. The work has an order: stop the harm first, then understand why.
The instinct is to hunt for the root cause while users keep failing, and an agent's instinct is to start changing things. Someone has to hold the whole picture and decide what gets tried, because a change made during an incident needs the most care of all.
Not Bug: A bug is a fault in code, and it can sit quietly for years. An incident is harm happening now, and may have no bug in it: a full disk or an expired certificate will do.
Seen in: The SRE book: "It is better to declare an incident early and then find a simple fix and close out the incident than to have to spin up the incident management framework hours into a burgeoning problem." And: "The incident commander holds the high-level state about the incident." Cloudflare on 18 November 2025 shows how it goes for real: impact from 11:28 UTC, a first theory that was wrong ("we initially wrongly suspected the symptoms we were seeing were caused by a hyper-scale DDoS attack"), and recovery by "manually inserting a known good file into the feature file distribution queue". "Core traffic was largely flowing as normal by 14:30." "As of 17:06 all systems at Cloudflare were functioning as normal."
Source: Google, Site Reliability Engineering, Managing Incidents, https://sre.google/sre-book/managing-incidents/
Source: Prince, Cloudflare outage on November 18, 2025, https://blog.cloudflare.com/18-november-2025-outage/
Say it: "We have an incident and I am running it. You are read-only: pull the error rate by release for the last hour, and the last three deploys. Change nothing, even things you are sure about. Give me two options for stopping the harm, each with what it could make worse."
Described as: "the site is down, what do we do first", "everyone is in the channel and nobody is in charge", "stop the bleeding before finding out why".
https://dev.dqnotes.com/ship/incident

### Runbook
Also: Playbook.
Written steps for a known situation, which someone tired can follow: what you see, what to check, the exact commands, and when to call for help.
Thinking it through while calm beats thinking it through in an emergency. A runbook in the repo is also something an agent can read and follow, and that gives you a cheap way to find out whether it has gone stale.
Not Postmortem: A postmortem looks back at one incident. A runbook looks ahead to the next one, and a postmortem's follow-ups often become one.
Seen in: Google's SRE book says that "thinking through and recording the best practices ahead of time in a 'playbook' produces roughly a 3x improvement in MTTR as compared to the strategy of 'winging it.'" That is Google's own report of its own experience from around 2016, and the book gives no study behind it.
Source: Google, Site Reliability Engineering, Introduction, https://sre.google/sre-book/introduction/
Say it: "Write docs/runbooks/checkout-5xx.md: how to confirm it, how to tell a bad deploy from a gateway outage, the exact rollback command, and who to call. Then have a fresh session follow it on staging with no other context, and fix every step where it got stuck."
Described as: "written steps for when this breaks", "only one person knows how to fix it", "what do I do when this alert fires".
https://dev.dqnotes.com/ship/runbook

### Time to restore
Also: MTTR, Mean time to recover, Failed deployment recovery time.
How long it takes to get back to healthy after something breaks. DORA's current version, failed deployment recovery time, counts only failures caused by a change.
You can't stop every failure, and you can decide how fast you recover. It is why rollbacks and flags pay for themselves. Take care with the mean, because one bad day can dominate it, so report the median as well.
Not Time to detect: Detection is only the first part. A team can recover fast once it knows, and still lose an hour because nothing told it.
Seen in: DORA changed this in 2023: "The metric historically known as 'mean time to recover (MTTR)' or 'time to restore service' was renamed and redefined as failed deployment recovery time." The reason: "previous definitions did not distinguish between a failure initiated by a software change and a failure caused by external factors, such as a data center outage."
Source: DORA, A history of DORA's software delivery metrics, https://dora.dev/insights/dora-metrics-history/
Say it: "Compute our recovery time from this quarter's incident log, both the median and the worst, and leave out the two provider outages. Then split each incident into time to notice, time to decide and time to fix, and tell me which part is largest."
Described as: "how long were we down", "how quickly can we get back", "one awful outage ruined our average".
https://dev.dqnotes.com/ship/mttr

### Postmortem
Also: Blameless postmortem, Incident review.
A written record of an incident: what happened, what it cost, why, and what changes so that it doesn't happen again. Blameless means it asks how the system allowed this, and doesn't stop at who.
People who fear blame hide what they did, and then nobody learns. With agents there is a new version of this: "the agent did it" is blame with a new target. Who gave it that access, and which check let it through?
Not Root cause: A root cause is one part of a postmortem, and there is rarely only one. An agent's own account of why it did something isn't evidence of a cause. It is a plausible story.
Seen in: Google's definition: "A postmortem is a written record of an incident, its impact, the actions taken to mitigate or resolve it, the root cause(s), and the follow-up actions to prevent the incident from recurring." On blame, it must focus on "identifying the contributing causes of the incident without indicting any individual or team". And: "A blamelessly written postmortem assumes that everyone involved in an incident had good intentions and did the right thing with the information they had." The purpose: "Removing blame from a postmortem gives people the confidence to escalate issues without fear."
Source: Google, Site Reliability Engineering, Postmortem Culture, https://sre.google/sre-book/postmortem-culture/
Say it: "Draft the postmortem from the timeline, the logs and the diff. No names, and never "the agent decided". Say what the agent was able to do and what the checks allowed. Mark each cause as evidenced or guessed. Each follow-up gets an owner and a date, and at least one has to be a check that would have failed."
Described as: "writing up what went wrong", "no finger pointing", "make sure that can't happen again".
https://dev.dqnotes.com/ship/postmortem

### DORA metrics
Also: DORA, Four keys.
Five measures of software delivery from Google's DORA research: change lead time, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate. Speed and stability, side by side.
They stop you fooling yourself with lines written or pull requests merged. Speed with no stability number beside it hides what it cost, and that is the case you need to catch when agents raise the rate.
Not Productivity: These describe how one service's delivery is going. They don't measure a person, or say whether the software was worth building.
Seen in: DORA's guide: "Top performers do well across all five metrics, and low performers do poorly." And "DORA's research has repeatedly demonstrated that speed and stability are not tradeoffs." There were four until 2024, when "they introduced a fifth metric: deployment rework rate." The warnings are worth as much as the metrics: setting them as targets "increases the likelihood that teams will try to game the metrics", and "These metrics are meant to be applied at the application or service level." The guide gives no numeric bands for good or bad, so neither do we.
Source: Harvey, DORA software delivery metrics, the guide, https://dora.dev/guides/dora-metrics/
Source: DORA, A history of DORA's software delivery metrics, https://dora.dev/insights/dora-metrics-history/
Say it: "From the deployment history and incident log, compute all five DORA metrics for this service over the last 90 days. Give me the definition you used for each, including what counts as failed. Compare us with our own previous 90 days and not with other teams."
Described as: "how do we measure if we are getting better", "are we shipping faster and breaking more", "what numbers should a team watch".
https://dev.dqnotes.com/ship/dora

### Change fail rate
Also: Change failure rate, CFR.
The share of deployments that need fixing straight away, by a rollback or a hotfix. Five bad deploys out of fifty is 10%.
It is the number that shows whether going faster is costing you. It is also easy to fool: the definition of failed moves it a lot, and adding harmless deploys lowers it with the same number of bad ones.
Not Bug count: A bug found next month, and fixed with no rush, doesn't count here. This measures deployments that made someone act at once.
Seen in: DORA's definition: "The ratio of deployments that require immediate intervention following a deployment." That leaves room for judgement. Is turning a flag off an intervention? The bench shows how far your answer moves the number, which is why a figure with no definition next to it tells you little.
Source: Harvey, DORA software delivery metrics, the guide, https://dora.dev/guides/dora-metrics/
Say it: "Count change fail rate over the last 90 days. Failed means rolled back, hotfixed within a day, or a flag turned off because of harm. Leave out no-change deploys and say how many there were. Show me the list, so I can argue with the definition before I see the number."
Described as: "how often does a deploy break something", "our number looks great but it does not feel great", "what counts as a failed release".
https://dev.dqnotes.com/ship/change-failure-rate

### The amplifier
Also: AI as an amplifier.
DORA's description of what AI does to a software team: it makes strong teams stronger, and makes the problems of struggling ones worse. What decides which is what was already around the tool.
So buying the tool doesn't settle what you get. This chapter is the surrounding part: small changes, a way back, real signals and an agreed budget. Without them, a faster rate of change can reach your users as more failures.
Not Proof that AI causes incidents: DORA found that AI adoption goes with more instability, in a survey, as people report it. We found no primary study measuring incident rates caused by AI-written code. If someone quotes one, ask to see it.
Seen in: The 2025 report: "AI's primary role in software development is that of an amplifier. It magnifies the strengths of high-performing organizations and the dysfunctions of struggling ones." On the numbers: "AI adoption now improves software delivery throughput, a key shift from last year. However, it still increases delivery instability." Its reading: "This suggests that while teams are adapting for speed, their underlying systems have not yet evolved to safely manage AI-accelerated development." It surveyed nearly 5,000 professionals, self-reported and correlational. A figure you may see credited to it is wrong: 24% of its respondents reported high trust in AI (4% a great deal, 20% a lot). The 3% high-trust number is Stack Overflow's 2025 survey, with 3.1% saying they highly trust it, which is a different question put to different people.
Source: DORA, State of AI-assisted Software Development, 2025, https://services.google.com/fh/files/misc/2025_state_of_ai_assisted_software_development.pdf
Source: Stack Overflow, Developer Survey 2025, AI, https://survey.stackoverflow.co/2025/ai
Say it: "Before we double the agents on this repo, pull our change fail rate, recovery time and budget spend for the last 90 days. Tell me which of those gets worse at twice the deploys, and what would have to change first. Use numbers from our logs, and not from a report about other companies."
Described as: "will AI make our team better or worse", "we went faster and things started breaking", "what did the DORA report actually find".
https://dev.dqnotes.com/ship/amplifier

## 17. Many agents

Parallel work, roles, handoffs, loops, evals and factories: running a team that never sleeps and never remembers.

### Parallel agents
Several agents work at the same time on separate parts of a job, each in its own session. You get more done per hour only if the parts really are independent, and only if you can check all of what comes back.
The limit is rarely the agents. It is how cleanly the work splits and how much you can review. Starting a fourth agent takes seconds, and reading what it did does not.
Not Subagent: A subagent is a helper that one session sends off and waits for. Parallel agents are whole sessions running side by side, usually each with its own branch, and you are the one who brings them together.
Seen in: Claude Code's docs say "Agent teams add coordination overhead and use significantly more tokens than a single session" and advise "Start with 3-5 teammates for most workflows". Their guide adds: "If you have 15 independent tasks, 3 teammates is a good starting point." The disagreement is real. Cognition's 2025 post argued "running multiple agents in collaboration only results in fragile systems". Its April 2026 follow-up reports some setups now working, and adds: "Our original observations still hold today for parallel-writer swarms".
Source: Claude Code docs, Agent teams, https://code.claude.com/docs/en/agent-teams
Source: Yan, Don't build multi-agents, Cognition, 2025, https://cognition.ai/blog/dont-build-multi-agents
Source: Cognition, Multi-agents: what's actually working, 2026, https://cognition.ai/blog/multi-agents-working
Say it: "Split this into tasks that touch different files. List the files for each task, and tell me which ones can't be separated. Don't start any of them until I've picked."
Described as: "can I run several agents at once", "this is too slow with one agent", "three agents on three features", "how many agents should I run".
https://dev.dqnotes.com/many/parallel-agents

### Fan-out
Also: fan-in, map-reduce with agents.
One job is split into pieces, each goes to its own worker at the same moment, and their results are gathered into one answer, which is the fan-in. The whole job takes as long as the slowest worker, plus the gathering.
It suits reading far better than writing. Ten agents can look through ten modules and report back without touching each other. Ten agents editing the same app cannot.
Not Pipeline: A pipeline hands work from one stage to the next, in order. Fan-out does the pieces at the same time, and it only helps when no piece needs another's answer.
Seen in: Anthropic's "Building effective agents" (December 2024) names two forms of this: sectioning, "Breaking a task into independent subtasks run in parallel", and voting, "Running the same task multiple times to get diverse outputs". pstack's swarm skill is the same shape as an instruction: "Fan out N parallel workers, drain them, and return one report." It adds a line worth stealing for your own fan-in: "A gap does not count as a pass."
Source: Anthropic, Building effective agents, 2024, https://www.anthropic.com/engineering/building-effective-agents
Source: pstack, skills/swarm/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/swarm/SKILL.md
Say it: "Use subagents to read each package under src/ and report every place it calls the old auth client, with file and line. One subagent per package. If one fails to report, tell me which, and don't treat it as "nothing found"."
Described as: "look through the whole repo at once", "one agent per folder", "run the same thing on every package", "split it up and combine the answers".
https://dev.dqnotes.com/many/fan-out

### Orchestrator
Also: coordinator, orchestrator-workers.
An agent whose job is running other agents. It splits the work, briefs each worker, tracks what comes back and judges it, and writes no code itself. A mistake in its brief is repeated by every worker it sends.
You can't watch fifty agents, and neither can your context window. An orchestrator holds the plan and stays small, because the details stay in the workers.
Not Fan-out: Fan-out is a shape, fixed before you start. An orchestrator decides the shape as it goes, from what the workers report.
Seen in: Anthropic describes it as "a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results". pstack's orchestrate playbook is the most concrete public version we found. It opens "You run the program, never the code." It sets a threshold, "Work that one agent could finish inside the session's budget is not a program." It tells you to "Start with one unit." so "a flaw in the kickoff costs one agent instead of fifty". And it warns against trusting reports: "Read the diff or the artifact, not only the worker's summary."
Source: Anthropic, Building effective agents, 2024, https://www.anthropic.com/engineering/building-effective-agents
Source: pstack, skills/poteto-mode/playbooks/orchestrate.md, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/orchestrate.md
Say it: "You're coordinating, so don't edit code. Run one unit first and show me the result before you start the rest. For each worker, read its diff yourself. Don't pass on its summary as fact."
Described as: "one agent managing the others", "who keeps track of all these agents", "a boss agent", "the main agent keeps running out of context".
https://dev.dqnotes.com/many/orchestrator

### Worktree isolation
Each agent gets its own git worktree: a separate folder with its own branch, sharing the same repository. One agent's half-saved file can't break another's test run, and an edit can't quietly overwrite another. They only meet at merge time.
Agents on one folder do not see each other coming. One saves a file the other read a minute ago, and the first agent's work is gone with no error. Two worktrees turn that silent loss into a merge conflict you can see.
Not Sandbox: A sandbox limits what an agent may do to your machine. A worktree only separates files. Both agents still share your database, ports and API keys, so two can still fight over the same test database.
Seen in: Claude Code: claude --worktree feature-auth makes a folder at .claude/worktrees/feature-auth/ on a branch called worktree-feature-auth, and a subagent can ask for the same with isolation: worktree. The docs are precise about the limit: "They isolate file edits." Cursor has the same idea: "Use worktrees when you want to start several agents on the same repo without conflicts." That sentence undersells the merge, which is still yours to do.
Source: Claude Code docs, Worktrees, https://code.claude.com/docs/en/worktrees
Source: Cursor docs, Worktrees, https://cursor.com/docs/configuration/worktrees
Say it: "Do this in its own git worktree on a new branch, not in this folder. Run the tests there. Use a separate test database and port, and tell me which you chose."
Described as: "two agents on the same repo", "my changes keep disappearing", "tests fail in a file nobody touched", "agents overwriting each other".
https://dev.dqnotes.com/many/worktree-isolation

### Coordination cost
The time a team spends staying in step and not working: waiting, conflicts, re-reviews, explaining things again. It grows with the pairs of workers and not the workers. Past some point, adding one makes the whole team slower.
Agents make this worse in one way. They are confident, they do not talk to each other unless you build that, and each conflict costs another run and another review. A reviewer is a queue too, and that queue is you.
Not Amdahl's law: Amdahl's law caps the gain at the share of work that can't be split, but extra workers never make things worse. Coordination cost can send the line back down.
Seen in: Fred Brooks wrote in 1975: "Adding manpower to a late software project makes it later." Brooks himself called that an "outrageous oversimplification". Agents do not need training, but they do start out knowing nothing. In Anthropic's compiler project, when all 16 agents hit one bug: "Every agent would hit the same bug, fix that bug, and then overwrite each other's changes." Steve Yegge on Gas Town: "Some bugs get fixed 2 or 3 times, and someone has to pick the winner." Claude Code's docs: "Two teammates editing the same file leads to overwrites."
Source: Wikipedia, Brooks's law, https://en.wikipedia.org/wiki/Brooks%27s_law
Source: Carlini, Building a C compiler with a team of parallel Claudes, Anthropic, 2026, https://www.anthropic.com/engineering/building-c-compiler
Source: Yegge, Welcome to Gas Town, 2026 (archived copy), https://web.archive.org/web/2026/https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16dd04
Source: Claude Code docs, Agent teams, https://code.claude.com/docs/en/agent-teams
Say it: "Before you add another agent, show me how much time the current ones spent waiting or redoing work. If that's rising, we fix the split or the review first."
Described as: "more agents is making it slower", "they keep stepping on each other", "merge conflicts all day", "I spend all my time reviewing".
https://dev.dqnotes.com/many/coordination-cost

### Merge queue
Pull requests wait in line, and each is tested on top of main plus everything ahead of it before it merges. Two changes that pass alone and break together get caught before they reach main, not after.
Agents open pull requests faster than people do, so two green ones colliding stops being rare. One renames a function, the other adds a new call to it. Git sees no conflict, and main is red for everyone.
Not Required CI: A required check tests your branch as it stands, against the main you started from. A queue tests what main will actually be after the merge.
Seen in: On GitHub, "the changes in the pull request are grouped into a merge_group with the latest version of the base_branch as well as changes from pull requests ahead of it in the queue". Your workflow has to listen for merge_group as well as pull_request, or required checks never report. The old idea is Graydon Hoare's, describing the bors bot in 2014: "automatically maintain a repository of code that always passes all the tests". OpenAI's Codex team argues the other way, with "minimal blocking merge gates" and "corrections are cheap, and waiting is expensive", and admits "This would be irresponsible in a low-throughput environment."
Source: GitHub docs, Managing a merge queue, https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue
Source: Graydon Hoare, The Not Rocket Science Rule, 2014 (archived copy), https://web.archive.org/web/2024/https://graydon2.dreamwidth.org/1597.html
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Say it: "Add merge_group to the triggers of our required CI workflow, alongside pull_request. Then tell me the exact setting I need to turn on in branch protection, since you can't do that part."
Described as: "both pull requests were green and main broke", "main keeps going red", "it passed on the branch", "too many PRs merging at once".
https://dev.dqnotes.com/many/merge-queue

### Roles
Also: role specialisation, specialised agents.
Give each agent one job with its own prompt, tools and check: one writes specs, one writes code, one tidies, one reviews. Each is told what not to do, so that each check stays with someone who did not write the code.
An agent asked to do everything cuts its own corners, and marks its own work. Roles put the corner and the check in different hands. Every role is also another handoff, another prompt to maintain and another bill.
Not Persona: Telling a model it is a senior engineer changes the tone. A role changes what the agent may touch and what it must prove. Cognition calls the first kind "gimmicky techniques".
Seen in: SwarmForge's six-pack puts this in plain words. Its coder prompt says: "Do not run language mutation, CRAP, or DRY checks; the cleaner, architect, and hardender own those checks." Its QA prompt says: "Reproduce failures before changing code." On the other side, Cognition's 2026 follow-up reports working setups where "multiple agents contribute intelligence to a task while writes stay single-threaded", meaning several can think, and only one writes.
Source: six-pack, swarmforge/roles/coder.prompt, https://github.com/unclebob/swarm-forge/blob/six-pack/swarmforge/roles/coder.prompt
Source: six-pack, swarmforge/roles/QA.prompt, https://github.com/unclebob/swarm-forge/blob/six-pack/swarmforge/roles/QA.prompt
Source: Cognition, Multi-agents: what's actually working, 2026, https://cognition.ai/blog/multi-agents-working
Say it: "You're the reviewer on this, so don't edit anything. Run the code and report what fails, with output. Someone else will fix it. If everything passes, say so and stop."
Described as: "one agent to write and one to check", "a tester agent", "give each agent a job", "it keeps marking its own homework".
https://dev.dqnotes.com/many/role-specialisation

### Evaluator-optimiser
One side writes and the other judges, in a loop. The judge's notes go back to the writer until it passes or the tries run out. It works when the judge has clear criteria, and the result ends up as good as what the judge can see.
Agents grade their own work too kindly. Splitting the writer from the judge fixes more than a stricter prompt does, and an automatic loop saves you from carrying the notes yourself. A blind spot in the judge stays a blind spot, however many rounds you run.
Not Retrying: A retry asks the same thing again and hopes. Here each round carries a specific refusal into the next.
Seen in: Anthropic defines it: "one LLM call generates a response while another provides evaluation and feedback in a loop", and says it is "particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value". Its later harness post found "agents reliably skew positive when grading their own work", and warned that "the evaluator is still an LLM that is inclined to be generous towards LLM-generated outputs". In one comparison it ran, solo cost $9 over 20 minutes and the full harness $200 over 6 hours: "The harness was over 20x more expensive, but the difference in output quality was immediately apparent." That is one build, judged by its authors.
Source: Anthropic, Building effective agents, 2024, https://www.anthropic.com/engineering/building-effective-agents
Source: Anthropic, Harness design for long-running application development, 2026, https://www.anthropic.com/engineering/harness-design-long-running-apps
Say it: "Write this, run ./check.sh, and read what it says. Fix what it reports, and run it again. Stop after four tries, or when it passes, and show me the full output of the last run either way."
Described as: "have one agent check the other", "keep going until the tests pass", "feed the errors back", "write then review in a loop".
https://dev.dqnotes.com/many/evaluator-optimizer

### Adversarial review
A reviewer whose job is to break the work and not to approve it. It is fair only when each finding must be proved by a run. Without that rule, a reviewer told to find problems will find problems, whether or not they exist.
A pushy prompt finds more real bugs and invents more false ones, and every round costs tokens you could spend on a test or a real run. Asking for a failing run first makes a false alarm cheap to spot, because an opinion cannot produce one. It doesn't make the round free, and the author of the best-known rule for this now doubts it pays.
Not Fresh-eyes review: Fresh eyes means a reviewer with no memory of writing the code, and it may still be polite. Adversarial means its aim is to make the code fail.
Seen in: Two dated views, from the same person. pstack's interrogate skill, as we read it on 6 November 2026: "Adversaries try to break a change or a design. A finding survives only when a run proves it." And: "No findings is a valid result." Lauren Tan on X, 3 November 2026: "i now consider adversarial reviews mostly a waste of tokens. far better to use those tokens on empirical combined with formal verification." They add: "i have updated pstack to use the interrogate skill less, and eventually i think i may remove it altogether". The public repo doesn't show that yet: version 0.15.36 has no commit under pstack/ after 27 October, and poteto-mode still sends a contested design to it, "(multi-model adversarial) before shipping". The post gives no figures. Interrogate requires a run, and the post also complains of review "in the abstract". It names both, and we can't tell from it how Tan weighs one against the other. Our own recording is one part of the trade. Over 240 one-shot runs, 18 broken and 222 good, a neutral reviewer stopped 13 broken and 39 good, and a pushy one 16 broken and 126 good: 3 more broken programs for 87 more false alarms. The neutral figure leans on one task. It rejected all 15 vague payment-events runs and none of the 15 with the right words, and without that task it stopped 2 of 7. One small model, one-function tasks, no real repo, and those reviewers could only read, so none of this tests interrogate itself.
Source: pstack, skills/interrogate/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/interrogate/SKILL.md
Source: Tan on X, 3 November 2026, https://x.com/poteto/status/2117437607940681849
Source: The same post, through the fxtwitter mirror, https://api.fxtwitter.com/poteto/status/2117437607940681849
Source: pstack, skills/poteto-mode/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/SKILL.md
Say it: "This change retries payments, so I want one targeted attempt to break it. Name the property you are attacking, write a script that fails, and paste the output. Leave out anything you did not run, including style. No findings is a fine answer. Don't fix anything. If a property test or a real run would answer this for less, do that and tell me."
Described as: "try to break it", "a harsh reviewer", "the reviewer keeps inventing problems", "red team this change", "is adversarial review worth the tokens".
https://dev.dqnotes.com/many/adversarial-review

### Ralph loop
Also: Ralph Wiggum loop, Ralph.
The same prompt, fed to a fresh agent, over and over, in a shell loop. Each run does one thing, writes down what it did, and exits. Its memory is on disk. It is cheap to set up, and it has no way of stopping unless you build one.
A fresh session each time means no drift and no bloat, and a loop means it carries on while you are away. It only works with a plan on disk, a check that can say no, and a limit, or it will happily spend your money making things worse.
Not Agent loop: The agent loop runs inside one session. Ralph is a loop around whole sessions, each knowing nothing of the last. Claude Code's /goal is nearer the first: it keeps one session going until a condition holds.
Seen in: Geoffrey Huntley's post of 14 July 2025: "In its purest form, Ralph is a Bash loop." The whole thing is one line: while :; do cat PROMPT.md | claude-code ; done The rule: "you need to ask Ralph to do one thing per loop." The limits are also in that post: "There is no way this is possible without senior expertise guiding Ralph." and "There's no way in heck would I use Ralph in an existing code base". The widely repeated $50k contract for $297 is a tweet quoted there, and we found no way to check it. In January 2026, Huntley said of multi-agent work "At this stage, it's not needed." and "Ralph is monolithic."
Source: Huntley, Ralph Wiggum as a "software engineer", 2025, https://ghuntley.com/ralph/
Source: Huntley, everything is a ralph loop, 2026, https://ghuntley.com/loop/
Source: Claude Code docs, Goal, https://code.claude.com/docs/en/goal
Say it: "Write me a script that runs claude -p in a loop on PROMPT.md: one item from fix_plan.md per run, then commit. Stop when ./check.sh passes or after 20 runs. Pass --max-turns and --max-budget-usd each time."
Described as: "just run it in a loop", "let it work overnight", "keep restarting the agent", "the Ralph thing".
https://dev.dqnotes.com/many/ralph-loop

### Long-running agents
An agent working on one goal for hours or days, usually across many sessions that cannot remember each other. What lets it last is outside the model: a plan on disk, notes on progress, tests it can run, and a clean way to start again.
Nothing goes well for long on its own. Each session starts cold, and long ones drift or declare victory early. A run this long is decided by the harness you set up before it began.
Not Scheduled agent: A scheduled agent does a small job on a clock and finishes. A long-running agent has one big goal and keeps at it.
Seen in: Anthropic's harness post: "they must work in discrete sessions, and each new session begins with no memory of what came before", like "engineers working in shifts". OpenAI, on Codex: "We regularly see single Codex runs work on a single task for upwards of six hours (often while the humans are sleeping)." Anthropic's C compiler used "nearly 2,000 Claude Code sessions across two weeks", at "a total cost just under $20,000". These are vendors reporting their own work. The compiler's author adds: "it is easy to see tests pass and assume the job is done, when this is rarely the case".
Source: Young, Effective harnesses for long-running agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Source: Carlini, Building a C compiler with a team of parallel Claudes, Anthropic, 2026, https://www.anthropic.com/engineering/building-c-compiler
Say it: "This will take more than one session. Before you code anything, write the plan to PLAN.md and a feature list with a test for each. In each session, read those first and work on one item only."
Described as: "run for hours on its own", "the session ran out half way", "overnight agent", "it forgets where it was".
https://dev.dqnotes.com/many/long-running-agent

### Progress files
A file in the repo saying what has been done and what comes next, which each new session reads first and updates last. It is how a session with no memory finds its place. It is only as true as whatever checks it.
A fresh agent believes the file, so a wrong line sends every later session the wrong way. Agents also like finishing, so "done" gets written too soon. Tie each claim to a test that a script can run.
Not Handoff note: A handoff note is written once, for the next session on a task. A progress file is kept up to date for the whole job, and a script can check a good one.
Seen in: Anthropic's setup keeps a claude-progress.txt and a JSON feature list. It chose JSON because the model is "less likely to inappropriately change or overwrite JSON files compared to Markdown files". The failure it names: "Claude marks features as done prematurely." Its guard: "It is unacceptable to remove or edit tests because this could lead to missing or buggy functionality". Uncle Bob's ub-mode skill asks for the same kind of trail: "Keep a log of every action you take and the reason you took it."
Source: Young, Effective harnesses for long-running agents, Anthropic, 2025, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
Source: unclebob/ub-mode, SKILL.md, https://github.com/unclebob/ub-mode/blob/master/SKILL.md
Say it: "Keep features.json with an id, a step and passes for each feature. Only change passes, and only after its test really passes. Never remove or reword a feature. Read it at the start of each session."
Described as: "it says it finished but it didn't", "where did the last session get to", "keep a list of what is done", "it starts over every time".
https://dev.dqnotes.com/many/progress-file

### Handoff artefacts
A file one agent commits for the next to pick up: a spec, a diff, a report. Unlike a chat message it outlives the session, has a version, and can be checked. If you pass a commit, the next stage knows exactly what it received.
A summary passed along a chain changes at each step, like a game of telephone. A committed file doesn't. It also means you can open what each stage really handed over, not what it said it did.
Not Handoff note: A handoff note helps one person or agent carry on the same work later. An artefact is the output of a stage, and the input to a different one.
Seen in: SwarmForge's README: "Agents exchange committed work through durable handoffs". Its protocol uses "durable inbox files", and "Queue state is represented by file location". pstack's version: "Its kickoff carries everything it needs, including each upstream report it depends on, pasted in full." Cognition argues a summary alone is not enough: "Share context, and share full agent traces, not just individual messages", because "Actions carry implicit decisions, and conflicting decisions carry bad results".
Source: unclebob/swarm-forge, README.md on main, https://github.com/unclebob/swarm-forge/blob/main/README.md
Source: unclebob/swarm-forge, swarmforge/handoff-protocol.md on main, https://github.com/unclebob/swarm-forge/blob/main/swarmforge/handoff-protocol.md
Source: pstack, skills/poteto-mode/playbooks/orchestrate.md, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/orchestrate.md
Source: Yan, Don't build multi-agents, Cognition, 2025, https://cognition.ai/blog/dont-build-multi-agents
Say it: "Write your findings to reports/auth-audit.md and commit it, with the commit hash in your final message. The next agent gets that file, and it won't see this conversation."
Described as: "pass the result to the next agent", "what did the last agent actually do", "the second agent got it wrong", "agents passing work along".
https://dev.dqnotes.com/many/handoff-artifact

### Checkpoints
A saved state you can go back to when a run goes wrong. Claude Code makes one per prompt, covering files its editing tools changed. Git commits cover everything git tracks. Neither can undo a database change or a sent email.
An unattended run makes many changes before you look, so you need to know how far back you can go, and what that leaves behind. People find out that files changed by shell commands are not covered at the worst moment.
Not Commit: A commit is yours, permanent and shareable. A checkpoint is a quick undo for this session. Use commits for anything you might want next week.
Seen in: From the Claude Code docs, read 5 November 2026: "Every prompt you send that starts a turn creates a new checkpoint", reached with /rewind or Esc twice. "A file that only Bash commands changed isn't tracked and stays as it is." One useful exception: "Once Claude has edited a file with its file editing tools in your conversation, /rewind undoes changes that Bash commands make to that file afterward." And the docs set the limit themselves: "Checkpoints are designed for quick, session-level recovery."
Source: Claude Code docs, Checkpointing, https://code.claude.com/docs/en/checkpointing
Say it: "Commit before each step of this, with a message saying which step. Don't run migrations or anything that touches a database or external service without asking me, since git can't undo those."
Described as: "undo what the agent just did", "go back to before that", "it deleted something", "can I rewind this".
https://dev.dqnotes.com/many/checkpoint

### Scheduled agents
Also: routines, cron agents.
An agent started by a clock or an event and not by a person: triage overnight, check a deploy every five minutes. Nobody is watching when it runs, so it needs narrower permissions than you'd give yourself, and its own way to stop.
It is the cheapest way to get work done while you sleep, and also a good way to get surprised. It acts with whatever access you gave it, on a schedule you may have forgotten about.
Not Ralph loop: A Ralph loop goes round until the job is done. A scheduled agent does a small job, stops, and waits for the next tick.
Seen in: Claude Code has /loop 5m check the deploy for use inside an open session, and "Recurring tasks automatically expire 7 days after creation." The docs give the reason: "This bounds how long a forgotten loop can run." Cloud routines run with your identity: "Anything a scheduled task does through your connected GitHub identity or connectors appears as you." At the time of writing: "Scheduled tasks are in research preview."
Source: Claude Code docs, Scheduled tasks, https://code.claude.com/docs/en/scheduled-tasks
Source: Claude Code docs, Routines, https://code.claude.com/docs/en/routines
Say it: "Every morning, read the open issues labelled bug, try to reproduce the newest, and post what you found as a comment. Don't push code or close anything. If you find nothing new, say so and stop."
Described as: "run this every night", "check it every five minutes", "an agent on a timer", "do this while I am asleep".
https://dev.dqnotes.com/many/scheduled-agent

### Runaway loops
Also: cost runaway.
A loop that keeps spending because nothing told it to stop, or a growing session where each turn resends everything before it. Spend rises with every turn, and if nobody is watching, the first sign is the bill.
A bug in ordinary code wastes CPU. A bug in a loop wastes money and carries on doing it, since a confused agent often does more work, not less. Set the cap before you leave. A cap you add afterwards helps nobody.
Not Token budget: A token budget is how you plan what fits in a context. This is what happens when no plan or limit exists at all.
Seen in: Anthropic: "agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats". Claude Code's docs: agent teams use "approximately 7x more tokens than standard sessions when teammates run in plan mode". For limits, --max-turns is "No limit by default", and --max-budget-usd stops a print-mode run, though "Spend can pass the cap". Steve Yegge on Gas Town: "It is a cash guzzler." We looked for a single documented runaway bill with a figure we could check, and did not find one, so none appears here.
Source: Anthropic, How we built our multi-agent research system, 2025, https://www.anthropic.com/engineering/multi-agent-research-system
Source: Claude Code docs, Manage costs, https://code.claude.com/docs/en/costs
Source: Claude Code docs, CLI reference, https://code.claude.com/docs/en/cli-reference
Source: Yegge, Welcome to Gas Town, 2026 (archived copy), https://web.archive.org/web/2026/https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16dd04
Say it: "Run this headless with --max-turns 30 and --max-budget-usd 5. If either limit stops it, write what is left to do into PROGRESS.md so I can decide whether to carry on."
Described as: "how much will this cost", "the bill was huge", "it ran all night", "put a limit on it".
https://dev.dqnotes.com/many/cost-runaway

### SwarmForge
A tool for running a line of AI agents, one role each, in separate git worktrees and tmux sessions (tmux keeps several terminals running side by side). Work passes on as committed handoffs, and you approve it at a gate.
It is the clearest public example we found of using checks in place of reading the code: every role owns one gate, so trust comes from the line and not from anyone's eyes. Whether that is enough is exactly what people disagree about.
Not The Gauntlet: There is no product with that name. "Run the gauntlet" is a figure of speech, used in talk about his checks, which some third-party blogs then turned into a name. The tool is SwarmForge.
Seen in: The repo is unclebob/swarm-forge. Justin Martin made its first commit on 31 March 2026, and Robert C. Martin has made 716 of the 723 commits across its branches (git log, 5 November 2026). It has packs of two, four and six agents on branches of those names (two-pack, four-pack, six-pack). The six-pack README gives "specification, implementation, cleanup, architecture, mutation hardening, and final QA to six separate agents": specifier, approval, coder, cleaner, architect, hardender (his spelling in the files), QA. Handoffs out of the specifier are "held in Attention for operator approval". The specifier writes Gherkin, a plain-text format for test scenarios. His own report of one run, on 4 June 2026: "Outcome was a usable application with a solid architecture and overloaded with unit tests, acceptance tests, and QA tests." That was not measured. On not reading the code, his own post of 23 July 2026, which we read through a mirror because X blocks fetching: "My current strategy is to not read any of the code written by my agents." It ends: "In the end, I have very high confidence in the code they produce because they've had to run the gauntlet of all of my constraints and tests." StartupFortune misquotes that last part as all these constraints, so check before you repeat it. That report also quotes Grady Booch against: "As an experienced developer, I know the smell of what is good and what is not. And no agent has either the experience or the context to know those things." We have Booch from that one secondary report only. We found no outside measurement of SwarmForge.
Source: unclebob/swarm-forge, README.md on six-pack, https://github.com/unclebob/swarm-forge/blob/six-pack/README.md
Source: unclebob/swarm-forge, README.md on main, https://github.com/unclebob/swarm-forge/blob/main/README.md
Source: six-pack, swarmforge/roles/hardender.prompt, https://github.com/unclebob/swarm-forge/blob/six-pack/swarmforge/roles/hardender.prompt
Source: Martin, post about a six-pack run, X, 4 June 2026, https://x.com/unclebobmartin/status/2062557016086786435
Source: Martin, reply on X, 23 July 2026, https://x.com/unclebobmartin/status/2080257779395154409
Source: The same post, through the fxtwitter mirror, https://api.fxtwitter.com/unclebobmartin/status/2080257779395154409
Source: Barr, StartupFortune, 25 July 2026 (a secondary report), https://startupfortune.com/uncle-bob-martin-says-he-no-longer-reads-ai-generated-code-and-the-developer-world-is-split/
Source: Carrion, Letting go of the wheel (a secondary report), https://www.emiliocarrion.com/en/blog/letting-go-of-the-wheel
Say it: "Clone unclebob/swarm-forge, check out the two-pack branch, and read the role prompts and README with me before running it. Explain what each role can and can't do, and what approval I would be giving."
Described as: "Uncle Bob's agent setup", "six agents in a line", "the gauntlet", "code nobody reads, checked by gates".
https://dev.dqnotes.com/many/swarmforge

### pstack
A set of skills by Lauren Tan (poteto) for Cursor and Claude Code. It teaches going deep with one agent first, then running many with proof. It is instructions for the agent you already use, with no runtime of its own, and poteto-mode is the router.
It shows that a lot of this can live in text. Its central claim is an order of work: earn trust in one agent's output, and only then parallelise. It also has the most exact public rules we found for what counts as proof before a merge.
Not SwarmForge: SwarmForge is a program that starts fixed roles in terminals. pstack is prose the agent reads and follows, so nothing enforces it but the agent. Uncle Bob's own ub-mode says it is "loosely based on portions of" Lauren Tan's rules, reduced and paired with quality-gate tools.
Seen in: It lives in cursor/plugins/pstack: version 0.15.36 when we read it (latest commit 27 October 2026), with 34 skills, 24 principle skills and 23 playbooks. Lauren's README: "if you want to go fast, go deep first", and "when you can go deep on one agent and trust it to write good, verifiable code, you can truly parallelize with confidence." Also: "the goal is not to maximize loc, in fact it's the opposite". It reports no measured results. From shipping.md: "Only a verdict from an agent that did not write the code counts", "Land only the contiguous verified run rooted at the bottom.", and "Compare git patch-id at the verdict SHA with the current head before trusting an older verdict." Git's patch id is a hash of a change "with line numbers ignored". Community ports exist, and they aren't the original. On 3 November 2026, Lauren posted that they had "updated pstack to use the interrogate skill less". We read the repo three days later and could not see that yet, so we say both, with dates.
Source: cursor/plugins, pstack/README.md, https://github.com/cursor/plugins/blob/main/pstack/README.md
Source: pstack, skills/poteto-mode/playbooks/shipping.md, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/shipping.md
Source: pstack, skills/poteto-mode/playbooks/orchestrate.md, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/orchestrate.md
Source: pstack, skills/principle-separate-before-serializing-shared-state/SKILL.md, https://github.com/cursor/plugins/blob/main/pstack/skills/principle-separate-before-serializing-shared-state/SKILL.md
Source: git docs, git-patch-id, https://git-scm.com/docs/git-patch-id
Source: unclebob/ub-mode, README.md, https://github.com/unclebob/ub-mode/blob/master/README.md
Source: Tan on X, 3 November 2026, https://x.com/poteto/status/2117437607940681849
Source: The same post, through the fxtwitter mirror, https://api.fxtwitter.com/poteto/status/2117437607940681849
Say it: "Install pstack with /plugin marketplace add cursor/plugins, then /plugin install pstack@cursor-plugins. Read me the shipping playbook, then tell me what we would need to change to follow it here."
Described as: "poteto mode", "Lauren Tan's skills", "fearless parallelism", "prove it before merging".
https://dev.dqnotes.com/many/pstack

### Software factory
A setup where specs and scenarios go in, and agents write, test and ship code, with few or no people reading it. People build the line and its checks. The phrase names a way of working. It is not a product, and not a measured result.
If it holds, your job moves from writing code to writing checks that decide whether code is any good. If the checks are weak, you get a lot of confident software that nobody understands. Everything turns on that if.
Not CI pipeline: A pipeline checks what people send it. A factory also has agents producing what goes in, and tries to run without a person at the end.
Seen in: Checked at source: StrongDM's factory page (Justin McCarthy, 6 February 2026) states "Code must not be written by humans" and "Code must not be reviewed by humans", and it keeps scenarios "often stored outside the codebase", so agents can't game them. Its Attractor repo holds specs and no code. Reported, and not measured from outside: its results, and its line "If you haven't spent at least $1,000 on tokens today per human engineer, your software factory has room for improvement". Simon Willison's reply: "If these patterns really do add $20,000/month per engineer to your budget they're far less interesting to me." OpenAI's Codex team reports, in its own words: "0 lines of manually-written code" as a deliberate constraint ("We intentionally chose this constraint"), "on the order of a million lines of code across application logic, infrastructure, tooling, documentation, and internal developer utilities", "roughly 1,500 pull requests", "3.5 PRs per engineer per day", three engineers growing to seven, over five months. "We estimate that we built this in about 1/10th the time" is an estimate, with no control. And: "What we don't yet know is how architectural coherence evolves over years in a fully agent-generated system."
Source: McCarthy, Software Factories and the Agentic Moment, StrongDM, 2026, https://factory.strongdm.ai/
Source: strongdm/attractor, the repository, https://github.com/strongdm/attractor
Source: Willison, How StrongDM's AI team build serious software without even looking at the code, 2026, https://simonwillison.net/2026/feb/7/software-factory/
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Say it: "Look at what we would need before agents could ship this service unread: which behaviours have no scenario, which checks a change could fake, what we couldn't roll back. List gaps only, and don't change anything."
Described as: "a software factory", "nobody writes code any more", "agents shipping on their own", "lights-out development".
https://dev.dqnotes.com/many/software-factory

### Dark factory
A software factory where no person writes or reads the code, so the lights can stay off. All the trust has to come from evidence such as hidden scenarios and real runs. Few teams claim to do this, and we found no outside audit of any.
It is the far end of the range, and it is useful to think about even if you never go there, because it forces the question: if you stopped reading this code tomorrow, what would tell you it broke? For most teams, the honest answer is the customers.
Not Vibe coding: Vibe coding skips reading the code and skips the proof too. A dark factory claims to replace reading with engineered evidence. From outside, the two can look alike. The difference is entirely in those checks.
Seen in: Dan Shapiro's "The Five Levels" (23 January 2026) puts it at the top: "It's a black box that turns specs into software." and "It's dark, because it's a place where humans are neither needed nor welcome." The caution comes from inside the same camp. Anthropic's compiler author: "the thought of programmers deploying software they've never personally verified is a real concern". OpenAI says of its own result: "should not be assumed to generalize without similar investment". Uncle Bob's own experiment, forcing CRAP scores below 4, found it "made every program harder to read. It did not improve design." So passing a stricter gate does not always give you better code.
Source: Shapiro, The Five Levels, 2026, https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/
Source: Carlini, Building a C compiler with a team of parallel Claudes, Anthropic, 2026, https://www.anthropic.com/engineering/building-c-compiler
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Source: unclebob/negative-test-experiment, experiment-abstract.txt, https://github.com/unclebob/negative-test-experiment/blob/main/experiment-abstract.txt
Say it: "Suppose nobody reads the code in this repo from tomorrow. List each way it could break in production without any of our checks noticing, most likely first. Don't fix them yet."
Described as: "nobody reads the code at all", "level five", "no humans in the loop", "can we stop reviewing altogether".
https://dev.dqnotes.com/many/dark-factory

### Entropy
Agents copy what they find. Good and bad patterns alike spread with every run, so a repo nobody tidies gets less consistent, and faster than people alone would make it. No single change looks wrong. The drift shows up later.
More agents means more copying per day. The harm is that bad patterns become examples: the next agent finds them, sees that they pass, and copies them. A rule enforced by a machine is the only kind that works at that speed.
Not Sprawl: Sprawl is too much code. Entropy can happen with no extra code at all: the same job done five different ways.
Seen in: OpenAI's Codex team has a section called "Entropy and garbage collection": "Codex replicates patterns that already exist in the repository", and "Over time, this inevitably leads to drift." Their first fix was by hand: "Our team used to spend every Friday (20% of the week) cleaning up". They now write down "golden principles" and enforce them, on the view that "Technical debt is like a high-interest loan". That is one team's report on one repo.
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Say it: "Find every place we call fetch directly instead of apiClient. Add a lint rule that fails on it, naming apiClient in the error message. Fix the five oldest in this pull request, and list the rest."
Described as: "the code is getting messier", "it keeps copying the old pattern", "five ways of doing the same thing", "the agent learned a bad habit from the repo".
https://dev.dqnotes.com/many/entropy

### Garbage collection agents
Also: doc gardening, doc-gardening agent, cleanup agents.
Small background agents that each look for one kind of decay, such as stale docs or a broken convention, and open tiny pull requests to fix it. They pay down entropy in small amounts, so nobody has to hold a clean-up day.
Small fixes are cheap to review and easy to automate. A giant clean-up is expensive and gets put off. Stale docs matter more with agents, because an agent believes them.
Not Refactoring: A refactor is one decision someone made. Garbage collection is a background job, with narrow rules, that runs whether anyone remembers or not.
Seen in: OpenAI describes a recurring agent that "scans for stale or obsolete documentation that does not reflect the real code behavior and opens fix-up pull requests", and says of such cleanups: "Most of these can be reviewed in under a minute and automerged. This functions like garbage collection." That is their report. Automerging depends on checks you trust, and for many repos that is a later step.
Source: Lopopolo, Harness engineering: leveraging Codex in an agent-first world, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Say it: "Every Monday, compare docs/ with the code. For each statement that is no longer true, open one small pull request fixing only that. Don't reword anything that is still correct, and don't merge."
Described as: "the docs are out of date", "clean up in the background", "small automatic fixes", "an agent to keep the repo tidy".
https://dev.dqnotes.com/many/doc-gardening

### Eval-driven development
Write down what good looks like as a set of scored cases before you change anything, then change the prompt or harness and run them again. An eval is a test of behaviour across many runs. It tells you if the change helped, not how it felt.
With agents, a change can feel better and be worse, and any single run can mislead you. If you don't set the bar first, you will end up picking the bar that suits the result you got.
Not Unit test: A unit test is one input with one right answer. An eval runs many cases, many times, and gives a rate, because the thing being tested does not behave the same way twice.
Seen in: Anthropic: "We recommend practicing eval-driven development", which means you "build evals to define planned capabilities before agents can fulfill them, then iterate until the agent performs well". It also says "20-50 simple tasks drawn from real failures is a great start". OpenAI's Cookbook has a worked recipe, now archived. In our own 240 runs, a pushy review prompt looked better on bugs caught, 16 of 18 against 13, and far worse on false alarms, 126 of 222 against 39. You would see that only by counting both. One small model, one-function tasks.
Source: Anthropic, Demystifying evals for AI agents, 2026, https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
Source: OpenAI Cookbook, Eval Driven System Design, 2025 (archived recipe), https://cookbook.openai.com/examples/partners/eval_driven_system_design/receipt_inspection
Say it: "Before we change this prompt, build an eval: 20 past failures and 20 good cases. Decide and write down what score counts as better. Run it 3 times on the current prompt, then on the new one, and show me both."
Described as: "is the new prompt actually better", "measure the agent", "how would we know this helped", "it felt better".
https://dev.dqnotes.com/many/eval-driven

### When not to
Most work does not need many agents. If it fits in one session, or the parts share files, or you can't check the output, use one. If you could do it in the time it takes to write the brief, use none. Add more only once you have measured a gain.
Every idea in this chapter adds something to run, pay for and understand. Many agents can feel productive because a lot is happening. The test is whether more correct work reached users, per hour of your attention. Very often it did not.
Not Fear of agents: This isn't saying don't use them. Choosing one agent over five is a decision about the shape of a task. The right answer does change with the task.
Seen in: Anthropic: "finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all." On coding in particular: "most coding tasks involve fewer truly parallelizable tasks than research, and LLM agents are not yet great at coordinating and delegating to other agents in real time". Claude Code's docs on sequential tasks and same-file edits say "a single session" or subagents are better. Even pstack, which is built for parallel work, says "Work that one agent could finish inside the session's budget is not a program." And its feature playbook has the rule for splitting: "Disjoint files, services, or layers parallelize. Shared writes serialize."
Source: Anthropic, Building effective agents, 2024, https://www.anthropic.com/engineering/building-effective-agents
Source: Anthropic, How we built our multi-agent research system, 2025, https://www.anthropic.com/engineering/multi-agent-research-system
Source: Claude Code docs, Agent teams, https://code.claude.com/docs/en/agent-teams
Source: pstack, skills/poteto-mode/playbooks/orchestrate.md, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/orchestrate.md
Source: pstack, skills/poteto-mode/playbooks/feature.md, https://github.com/cursor/plugins/blob/main/pstack/skills/poteto-mode/playbooks/feature.md
Say it: "Before you plan anything: would one agent in one session be enough for this? Say yes unless you can name the parts that are truly independent, the files each one touches, and how I would check each result."
Described as: "do I need more than one agent for this", "is this overkill", "it would be quicker to do it myself", "everyone says to run ten agents".
https://dev.dqnotes.com/many/when-not-to

## 18. The proof

What the evidence really says, what to claim, and how to show it to someone who is paid to doubt you.

### How strong is this evidence
Also: Evidence hierarchy.
A rough ranking of how far a kind of study lets you believe its claim: randomised trials near the top, then comparisons over time, then surveys and benchmarks, then single stories. A ranking only makes sense for one question at a time.
An agent summarising research gives a survey and a trial the same confident sentence, so the ranking has to come from you. And the person across the table has heard all the loud numbers. You can't check every study, but you can find out what kind it is in a minute.
Not How useful it is to you: Higher up means a stronger claim about cause. It doesn't mean closer to your situation. METR's 2025 trial has the stronger design. Chen and Stratton's paper is the one with agents in real firms into 2026.
Seen in: There is no official ranking for this field, and the one on this bench is our judgement. The designs really do differ. METR randomised each of 246 real issues. Anthropic randomised 52 engineers. Chen and Stratton compare firms before and after they adopted tools at different times. Stack Overflow asks whoever answers. Veracode runs its own prompts through models. Fortune reports what happened at one company on one occasion.
Source: METR, Measuring the impact of early-2025 AI on experienced open-source developer productivity, 2025, https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
Source: Anthropic, How AI assistance impacts the formation of coding skills, January 2026, https://www.anthropic.com/research/AI-assistance-coding-skills
Source: Chen and Stratton, Artificial Intelligence in the Firm, current version 1 November 2026 (PDF), https://fion.ac/jellyfish.pdf
Source: Stack Overflow Developer Survey 2026, https://survey.stackoverflow.co/2026/
Source: Veracode, 2026 GenAI code security report (vendor), https://www.veracode.com/blog/2026-genai-code-security-report-ai-risk
Source: Fortune on the Replit incident, 23 July 2025, https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure
Say it: "You cited a study for that. Tell me what kind it is: randomised trial, comparison over time, survey, vendor benchmark or single story. Give me the sample size, the date of the tools it tested, and the sentence in the source. If you haven't opened the source, say so."
Described as: "is this study any good", "someone sent me a statistic about AI", "how much should I believe this number", "which research can I actually cite", "one study says one thing and another says the opposite".
https://dev.dqnotes.com/proof/evidence-hierarchy

### Randomised trial
Also: RCT, Randomised controlled trial.
A test where chance decides who or what gets the treatment, so the two groups differ only by luck and by the thing you are testing. That is what lets you say one thing caused another, and not just that they went together.
If you choose which tasks to give the agent, you will pick the ones you expect it to help with, and comparing times afterwards mostly measures your choice. It is why a before-and-after on your own work proves so little, and why a coin flip on your next ten tasks proves more.
Not Before and after: Before and after compares you now with you earlier, when the model, the project and you were all different. A randomised trial compares at the same time, with chance doing the sorting.
Seen in: METR let chance decide whether each issue allowed AI. Randomising doesn't make a trial immune. In its follow-up, METR reports that 30% to 50% of surveyed developers said they were choosing not to submit some tasks because they didn't want to do them without AI, so some of the tasks that mattered most never reached the coin. METR says this "implies we are systematically missing tasks which have high expected uplift from AI."
Source: METR, Measuring the impact of early-2025 AI on experienced open-source developer productivity, 2025, https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
Source: METR, We are changing our developer productivity experiment design, February 2026, https://metr.org/blog/2026-02-24-uplift-update/
Say it: "I want to know if using you on this kind of task is worth it. Make a table for my next 20 tasks. I write each task down first, then assign it with python3 -c "import random; print(random.choice(['agent', 'hand']))". Record minutes and whether it merged without rework. I don't get to swap after the flip."
Described as: "how would you prove it actually helped", "we compared before and after we got the tool", "people only use it on the easy tasks", "flip a coin to decide", "what does controlled trial mean".
https://dev.dqnotes.com/proof/rct

### Confidence interval
A range around a result showing how far it could be from the truth just from the luck of what got sampled. Fewer samples, wider range. It covers bad luck only: a biased study can have a narrow interval around the wrong answer.
An agent doesn't give the same answer twice, so one or two good runs tell you very little. A prompt that passed 15 times out of 15 could still fail one run in five. This is a daily tool when agents do the work, and not only something for reading papers.
Not A 95% chance the truth is inside: It's close, and not quite that. It means the method, repeated many times, catches the truth about 95 times in 100. A given range either contains it or doesn't.
Seen in: In our own experiment, the vague brief passed 105 of 120 one-shot runs: 88%, interval 80 to 92. One task, payment-events, passed 15 of 15 with the right words, and even that perfect score has an interval of 80 to 100. These are Wilson intervals, which this bench really computes. METR's 2025 interval is +2% to +39%, so its low end barely clears zero. DORA reports 89% credible intervals, a Bayesian relative, so don't compare widths across the two.
Source: METR, We are changing our developer productivity experiment design, February 2026, https://metr.org/blog/2026-02-24-uplift-update/
Source: DORA, State of AI-assisted Software Development, 2025 (PDF, v. 2025.2), https://services.google.com/fh/files/misc/2025_state_of_ai_assisted_software_development.pdf
Say it: "You ran this prompt 3 times and it passed 3 times, which proves very little. Run it 30 times, each in a fresh session, count how many pass tests/holdout.test.ts, and give me the Wilson 95% interval. Don't call it reliable unless the low end is above 90%."
Described as: "it worked three times so it must be fine", "how many runs is enough", "the plus or minus on a result", "is that difference real or just luck", "the sample was tiny".
https://dev.dqnotes.com/proof/confidence-interval

### Base rate
Also: Prevalence.
How common something is before you test for it. When the thing is rare, even a good detector's alarms are mostly wrong, because the many good cases set off more false alarms than the few bad ones set off true ones.
Your gates and your reviewer agents are detectors. If one change in thirteen is broken, most refusals land on good work, and an agent told to satisfy them rewrites good code. The same sum applies to you in an interview: when nearly everyone uses AI, saying you do tells them almost nothing.
Not Accuracy: "90% accurate" describes the detector. What you want to know is how often its alarm is right, and that also depends on how common the problem is.
Seen in: In our own experiment, 18 of 240 one-shot programs were broken, a base rate of 7.5%. A reviewer told to find a real defect stopped 16 of those and 126 good ones, so about 1 refusal in 9 was right. On one small model and one-function tasks, so read the shape. In DORA's 2025 survey, 90% of respondents use AI at work, and in Stack Overflow's 2025 survey 84% use or plan to use it.
Source: DORA, State of AI-assisted Software Development, 2025 (PDF, v. 2025.2), https://services.google.com/fh/files/misc/2025_state_of_ai_assisted_software_development.pdf
Source: Stack Overflow Developer Survey 2025, AI, https://survey.stackoverflow.co/2025/ai
Say it: "Before we make this reviewer a blocking check, measure it. Run it over our last 50 merged pull requests, count how many it would have stopped, and mark each stop as a real problem or not, with the evidence. Tell me how many of its refusals were right."
Described as: "the alarm goes off and it is nearly always nothing", "a 90% accurate tool that is usually wrong", "most of what it flags is fine", "why does the check cry wolf", "everyone says they use AI so what does it prove".
https://dev.dqnotes.com/proof/base-rate

### METR's studies
Studies by METR, a non-profit, timing experienced open-source developers on their own real tasks, with chance deciding whether AI was allowed. The 2025 one found them slower. The 2026 follow-up says it couldn't produce a reliable new number.
If you say you build with agents, this is the study they will ask about, and you can get it wrong in two directions. "AI makes developers slower" overstates a trial of 16 people on early-2025 tools. "METR now shows AI speeds you up" overstates a follow-up METR itself distrusts.
Not AI makes developers slower: That is the headline and not the finding. It measured 16 experienced developers, on projects they knew well, with tools from early 2025, and METR says so itself.
Seen in: The 2025 trial had 16 developers and 246 issues, mostly using Cursor Pro with Claude 3.5/3.7 Sonnet: "when developers use AI tools, they take 19% longer than without", interval +2% to +39%. METR adds: "We do not claim that our developers or repositories represent a majority or plurality of software development work." The follow-up had 57 developers. The 10 who returned came out at -18% (-38% to +9%), and the 47 new ones at -4% (-15% to +9%). Both intervals include zero. METR calls its estimate "likely a bad proxy" and its data "only very weak evidence", while saying it believes developers are probably sped up more now. That is a stated belief and not a measurement. In one place METR itself rounds the first result to 20%, so you may see both numbers.
Source: METR, Measuring the impact of early-2025 AI on experienced open-source developer productivity, 2025, https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
Source: METR, We are changing our developer productivity experiment design, February 2026, https://metr.org/blog/2026-02-24-uplift-update/
Say it: "Open metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ and metr.org/blog/2026-02-24-uplift-update/. Draft two sentences I can say in an interview about what they do and don't show, with the sample, the date of the tools and the intervals. Mark any number you didn't read on those pages."
Described as: "that study that said AI makes you slower", "the one where developers thought they were faster", "what about the METR paper", "didn't someone prove AI slows you down", "is that study still true".
https://dev.dqnotes.com/proof/metr

### Perception gap
The distance between how much a tool feels like it helps and how much it measurably does. It can be large, it can point the wrong way, and the person inside it can't see it from there.
An agent makes work feel easier. Waiting feels like resting, and a wall of new code feels like progress. So "it makes me faster" is no evidence, even when you say it in good faith, and a strong candidate knows that about their own impression.
Not Lying: Nobody here was lying. The developers in the trial reported what they believed, right after doing the work, and the clock disagreed.
Seen in: METR 2025: "developers expected AI to speed them up by 24%, and even after experiencing the slowdown, they still believed AI had sped them up by 20%." Set against 19% longer, that is 39 points by our subtraction, and METR later calls it 40. In METR's survey of 349 technical workers in early 2026, the median self-reported speed change was 3x and the value change 1.4 to 2x. METR warns that "survey results are not necessarily grounded in reality", and that it is difficult to say how far surveys overshoot. This is one trial of 16 people on older tools, so it doesn't tell you how big your own gap is. It does tell you not to rely on the feeling.
Source: METR, Measuring the impact of early-2025 AI on experienced open-source developer productivity, 2025, https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
Source: METR, Measuring the self-reported impact of early-2026 AI on technical worker productivity, May 2026, https://metr.org/blog/2026-05-11-ai-usage-survey/
Say it: "Don't tell me this was faster. Run gh pr list --state merged --limit 40 --json createdAt,mergedAt,additions,title and give me the median time from open to merge before and after the date I started using you for this work. Then say what that can't prove, since nothing was randomised."
Described as: "it feels so much faster", "I am sure it saves me hours", "it felt productive but nothing shipped", "how do I know I am not fooling myself", "everyone says it makes them faster".
https://dev.dqnotes.com/proof/perception-gap

### Skill atrophy
Losing a skill, or never building it, because something else does the work for you. With agents, the worry is that you can ship code you can't explain or debug, and that you don't notice until the agent gets stuck and so do you.
Directing an agent well depends on the skill it might erode: you can only catch what you are able to read. You don't feel this happening, since the work still gets done. It comes up in interviews that take the agent away, which some do on purpose.
Not Comprehension debt: Comprehension debt is about this codebase: code nobody here understands. Skill atrophy is about you: what you can no longer do on any codebase.
Seen in: Anthropic's randomised trial, January 2026: 52 mostly junior engineers learning an unfamiliar Python library. "The AI group averaged 50% on the quiz, compared to 67% in the hand-coding group", with Cohen's d=0.738 and p=0.01, and the largest gap on debugging questions. They finished about two minutes faster, which wasn't statistically significant. The gap is 17 percentage points. Anthropic's own page also says "17% lower", which is how that wording spread: in relative terms it is about 25%. It measured learning during one task, with the quiz soon after, so it isn't a skill fading over months, and we found no study that measures that. Anthropic calls the results preliminary, sells an AI assistant itself, and expects agentic products to be worse: "likely to be more pronounced than the results here."
Source: Anthropic, How AI assistance impacts the formation of coding skills, January 2026, https://www.anthropic.com/research/AI-assistance-coding-skills
Say it: "Explain the approach and ask me what I'd do before you write anything. After you write it, quiz me on three things: why this line exists, what happens if this input is null, and where I'd look first if it failed. Don't give me the answers until I've tried."
Described as: "I can not code without it any more", "am I getting worse at programming", "I shipped it but could not explain it", "I feel lost when the agent is down", "juniors are not learning anything".
https://dev.dqnotes.com/proof/skill-atrophy

### Trust surveys
Surveys asking developers how far they trust AI tools. The number that gets quoted depends mostly on how the question was asked and which answers someone decided to count as trust.
These numbers get used to argue that agent output is or isn't reliable. A survey can't show that. It tells you how people feel about the output, and it says nothing about how often the output is right. Only your own checks can tell you that.
Not A trend: A trend needs the same question twice. In 2025 Stack Overflow asked how much you trust the accuracy of the output. In 2026 it asked under what conditions you trust it. Those two numbers can't be joined with a line.
Seen in: Stack Overflow 2026: "more than 87% of respondents had some level of trust in AI's outputs, a huge increase from 31% last year." That 87% is everyone who didn't pick "I do not trust it for most work tasks" or "Not sure": n = 14,304. Only 6.6% picked trusting it for many tasks including important decisions. The 2025 page says 46% distrust and 33% trust, with 3.1% highly trusting, which doesn't match the 31% the 2026 page gives for last year. We can't say why. DORA 2025 asked another question again: 24% said a great deal or a lot, and 30% a little or not at all. Some coverage says DORA found 3%, which isn't in the report.
Source: Stack Overflow Developer Survey 2026, https://survey.stackoverflow.co/2026/
Source: Stack Overflow Developer Survey 2025, AI, https://survey.stackoverflow.co/2025/ai
Source: DORA, State of AI-assisted Software Development, 2025 (PDF, v. 2025.2), https://services.google.com/fh/files/misc/2025_state_of_ai_assisted_software_development.pdf
Say it: "Don't write that trust in AI tripled. Quote the exact wording of each year's question from survey.stackoverflow.co/2025/ai and survey.stackoverflow.co/2026/, list which answers each headline counted, and tell me whether the two can be compared at all."
Described as: "87% of developers trust AI", "trust in AI went way up", "nobody trusts AI code according to a survey", "two surveys say opposite things", "developers say they trust it".
https://dev.dqnotes.com/proof/stack-overflow-trust

### DORA on AI
Google Cloud's DORA research on how software teams deliver. Its 2025 report, from a survey of nearly 5,000 technology professionals, calls AI an amplifier: it makes strong teams stronger and struggling teams worse.
It changes the question from "does AI work?" to "what is it amplifying?". An agent pointed at a repo with no tests and vague briefs gives you more of both, faster. It is also a good answer to anyone asking what you would watch once agents write most of the code.
Not A measured effect: DORA is a survey, read through statistical models. Its own report says it compares groups and doesn't want to overclaim about cause, so say "associated with" when you quote it.
Seen in: "AI adoption now improves software delivery throughput, a key shift from last year. However, it still increases delivery instability." And it calls AI an amplifier, which "magnifies the strengths" of high-performing organisations "and the dysfunctions of struggling ones." Without the foundation, it says, gains are "lost to downstream chaos". 90% of respondents use AI at work and more than 80% believe it has raised their productivity. Its seven capabilities include a clear and communicated AI stance, strong version control practices and working in small batches. A footnote is worth knowing: "we don't want to give false assurances that we understand the underlying causal structure."
Source: DORA, State of AI-assisted Software Development, 2025 (PDF, v. 2025.2), https://services.google.com/fh/files/misc/2025_state_of_ai_assisted_software_development.pdf
Say it: "Read our last 30 deployments from CI history. Give me deployment frequency, lead time and change fail rate. Then look at git log and tell me which of DORA's AI capabilities we are weakest on, starting with small batches and version control practice, since those two you can check."
Described as: "AI made our good team better and our bad team worse", "it just speeds up whatever we were already doing", "what should we fix before adding agents", "we ship more and break more", "the amplifier thing".
https://dev.dqnotes.com/proof/dora-ai

### Output isn't delivery
More code isn't more software shipped. Output is what gets produced: lines, commits, pull requests. Delivery is what reaches people and does its job. Agents can raise the first a long way while the second moves much less.
An agent can always produce more of anything you count, so counting what it produces rewards the wrong thing. If you tell an interviewer you opened twice as many pull requests, the next question is where they all went.
Not Productivity: Productivity is a claim about value for effort. Output and delivery are two things you might count to estimate it, and they can disagree.
Seen in: Chen and Stratton, a Harvard job-market paper, not peer reviewed as far as we could tell: 720,542 workers at 655 firms, version of 1 November 2026. After agents arrived, pull requests opened rose 35.7% and merged 26.0%. Issues resolved rose 13.7%, marginally insignificant, and they rule out more than 30.7%. Time from submission to approval rose 56.1%. "Code review becomes the bottleneck." The data comes from Jellyfish, an analytics company, and the paper says its conclusions aren't Jellyfish's. An August version reported in the press had different figures, so cite the version. Faros AI, a vendor, says something similar on a page with no method: "Engineers are completing twice as many code changes, but company metrics remain flat."
Source: Chen and Stratton, Artificial Intelligence in the Firm, current version 1 November 2026 (PDF), https://fion.ac/jellyfish.pdf
Source: Faros AI, The AI productivity paradox (vendor), https://www.faros.ai/ai-productivity-paradox
Say it: "Don't report how many pull requests you opened. Using gh, give me for each of the last 8 weeks: pull requests merged, median hours from open to merge, and how many were reverted or followed by a fix within 7 days. Then say what this can't show about value to users."
Described as: "we write way more code and ship the same", "lots of pull requests but nothing is finished", "my numbers look great and nothing changed", "more commits does not mean more done", "the agent keeps producing and the queue keeps growing".
https://dev.dqnotes.com/proof/output-not-delivery

### Vendor studies
Research published by a company that sells something in the same area. It can be perfectly good work. You say who paid when you quote it, and you read what it measured with that in mind.
Agent tooling is full of these, and the label falls off as they get repeated. Many also score code from one shot with nothing around it, which isn't how you are working if you have a harness. So their numbers don't carry across, in either direction.
Not Wrong: Vendor doesn't mean false. Anthropic sells an assistant and published an unflattering result about assistants. Ask the same questions of everyone, and don't dismiss anyone on sight.
Seen in: Veracode sells security scanners and reports about 44% of one-shot tasks introduced a vulnerability in 2026, against 45% in 2025. Pass rates run from 87% on cryptographic algorithms and 83% on SQL injection down to 15% on cross-site scripting and 12% on log injection. It is generation on Veracode's prompts, scored by its analysis, with no review or tests in the loop, and the blog gives no method. CodeRabbit sells AI review: 470 pull requests, 10.83 issues per AI one against 6.45, and it says "we cannot guarantee all the PRs we labelled as human authored were actually authored only by humans." GitClear sells analytics: 211 million lines, with copy and paste up from 8.3% to 12.3%, and that doesn't separate AI lines from human ones. Include us: this course has a stake in harnesses, and our experiment found they moved passing scenarios from 88% to 91% on a vague brief. We published that anyway, and you should still check it.
Source: Veracode, 2026 GenAI code security report (vendor), https://www.veracode.com/blog/2026-genai-code-security-report-ai-risk
Source: Veracode, GenAI code security report, 2025 (vendor), https://www.veracode.com/blog/genai-code-security-report/
Source: CodeRabbit, State of AI vs human code generation, 2025 (vendor), https://coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report
Source: GitClear, AI Copilot code quality, 2025 (vendor, archived landing page), https://web.archive.org/web/2025/https://www.gitclear.com/ai_assistant_code_quality_2025_research
Say it: "For every number in this document, add who published it, whether they sell something related, and whether it came from one shot on their prompts or from a real workflow. Delete any number you can't trace to a page you opened."
Described as: "45 percent of AI code is insecure", "a company that sells this says it works", "who funded this report", "the report came from the people selling the fix", "that number is from a blog post by a tool company".
https://dev.dqnotes.com/proof/vendor-study

### What nobody has measured
The questions with no good study behind them yet. For agent-written code the biggest is this: does it cause more production incidents? We found no primary, peer-reviewed measurement, and saying so is a stronger answer than filling the gap.
An agent asked for a statistic will find something that sounds like one. Stories about agents deleting databases are real, and they are stories, so they can't give you a rate. A number with no source is the easiest thing in an interview to catch.
Not Evidence of no effect: No study doesn't mean no effect. It means nobody you can point to has measured it, so either way you're guessing, and you should say you're guessing.
Seen in: The closest primary evidence is DORA's instability finding, which comes from a survey and is about comparisons. Replit is the usual story: Fortune reports that its agent wiped data on "more than 1,200 executives" and said "This was a catastrophic failure on my part." That is one incident in one trial. Benchmarks from LinearB, Opsera and CircleCI reached us only through search summaries, so they aren't here. We also found no study of skills fading over months, and no employer saying which portfolio artefact persuades them. All of this was searched by 5 November 2026 and we may have missed something, so say you know of none, and not that there is none.
Source: DORA, State of AI-assisted Software Development, 2025 (PDF, v. 2025.2), https://services.google.com/fh/files/misc/2025_state_of_ai_assisted_software_development.pdf
Source: Fortune on the Replit incident, 23 July 2025, https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure
Say it: "Search for a primary study measuring production incident rates from AI-written code. List what you searched and what you opened. If you find none, say "I found none" and stop. Don't stretch a nearby figure to fit, and don't cite a page you haven't read."
Described as: "does AI code cause more outages", "is there a study on that", "I keep hearing about agents wiping databases", "nobody seems to have a number for this", "what do we actually not know".
https://dev.dqnotes.com/proof/missing-evidence

### Reading a productivity claim
Four questions to ask of any number about AI and productivity, in this order: who measured it, what was it compared with, what did they count, and who paid. A claim that survives usually comes out smaller and more useful.
You will meet these numbers in interviews, in pitch decks, and from your own agent, which will repeat them cheerfully. Being able to take one apart calmly, in front of someone, is itself evidence that you know what you are doing.
Not Debunking: The goal isn't to knock numbers down. Most of them are true of something. You are finding out what they are true of, and keeping that part.
Seen in: The bench runs the four questions on real claims, and on one of our own. Our experiment, with one small model and one-function tasks, is the 88%, 91%, 98% and 100% on this site. Most of that gap comes from a single task. Nobody should repeat those as a harness making code 12 points better, and the bench shows why.
Source: METR, Measuring the impact of early-2025 AI on experienced open-source developer productivity, 2025, https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
Source: Chen and Stratton, Artificial Intelligence in the Firm, current version 1 November 2026 (PDF), https://fion.ac/jellyfish.pdf
Source: DORA, State of AI-assisted Software Development, 2025 (PDF, v. 2025.2), https://services.google.com/fh/files/misc/2025_state_of_ai_assisted_software_development.pdf
Source: Faros AI, The AI productivity paradox (vendor), https://www.faros.ai/ai-productivity-paradox
Source: Stack Overflow Developer Survey 2026, https://survey.stackoverflow.co/2026/
Source: Veracode, 2026 GenAI code security report (vendor), https://www.veracode.com/blog/2026-genai-code-security-report-ai-risk
Source: CodeRabbit, State of AI vs human code generation, 2025 (vendor), https://coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report
Source: Anthropic, How AI assistance impacts the formation of coding skills, January 2026, https://www.anthropic.com/research/AI-assistance-coding-skills
Say it: "Someone claims agents made their team 40% more productive. Find the original source. Tell me who measured it, what the comparison was, what was counted and who paid. Where you can't find an answer, write "unknown". Don't infer one."
Described as: "our CEO read that AI makes you 40% faster", "a study proves it doubles output", "how do I push back on a big AI number", "where does this number come from", "that claim sounds too good".
https://dev.dqnotes.com/proof/productivity-claims

### Prediction or measurement
Also: Forecast.
A measurement says what happened, to what, compared with what. A prediction says what will happen. Both can be worth hearing, and they need different treatment: you check a measurement, and with a prediction you ask what would show it right or wrong, and when.
A forecast from someone who knows the field sounds just like a finding, and it gets repeated as one. Sort it sentence by sentence: what could be tested today, what only later, and what no result could touch. Then you know what to do this week, whichever way it turns out.
Not A productivity claim: A productivity claim offers a number, so you can ask what it counted. A forecast offers none, and that is no fault in it. The first question is just different.
Seen in: A live one. Guillermo Rauch on X, 5 November 2026: "AI will rewrite it all. Many times. It will make it fast, cost-efficient, and secure." Also: "It will prove, formally verify, load test, harden." and "In many ways, we are witnessing the true birth of software engineering." It offers no measurement and doesn't claim one. Rauch's bio reads "@vercel CEO", and Vercel sells v0 and describes its site as "Agentic Infrastructure", so that is worth stating, and it doesn't make the forecast wrong. What we have today: Veracode, which sells scanners, reports about 44% of one-shot tasks introduced a vulnerability, with no human baseline. METR measured 19% longer in 2025 and calls its 2026 follow-up data "only very weak evidence". On proofs, Kleppmann predicts much the same and writes "At present, a human with specialist expertise still has to guide the process". Some of this may turn out right. Nobody can know yet, including us.
Source: Rauch on X, 5 November 2026, https://x.com/rauchg/status/2118166970273292299
Source: The same post, through the fxtwitter mirror, https://api.fxtwitter.com/rauchg/status/2118166970273292299
Source: Vercel, home page, https://vercel.com/
Source: v0 docs, https://v0.app/docs
Source: Veracode, 2026 GenAI code security report (vendor), https://www.veracode.com/blog/2026-genai-code-security-report-ai-risk
Source: METR, Measuring the impact of early-2025 AI on experienced open-source developer productivity, 2025, https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
Source: METR, We are changing our developer productivity experiment design, February 2026, https://metr.org/blog/2026-02-24-uplift-update/
Source: Kleppmann, Prediction: AI will make formal verification go mainstream, 8 December 2025, https://martin.kleppmann.com/2025/12/08/ai-formal-verification.html
Say it: "Someone says AI will rewrite all software and make it secure. Take this one service and test the small version: have an agent rewrite one module, then compare latency, scanner findings and failing scenarios before and after. Give me the numbers and what you could not measure. Don't tell me what it means for the industry."
Described as: "AI will rewrite all software", "my CEO shared a post about the future of coding", "how do I answer a big prediction", "nobody will write code soon", "is this a fact or a forecast".
https://dev.dqnotes.com/proof/prediction-or-measurement

### The claim you can defend
A short paragraph you can say out loud in which every sentence points at something you can open: a file, a commit, a failing run. If a sentence has nothing behind it, you either drop it or go and do the work.
Saying you use agents is close to saying nothing, because nearly everyone does. What is rarer, and what employers describe wanting, is someone who can lead the agent and can show how they know the result is any good. The sentences only count if they are specific, and only if they are true of you.
Not Confidence: A claim isn't a tone. Modest words with nothing behind them fail in exactly the same way as bold ones, and in exactly the same two questions.
Seen in: Elicit, on what it rejects: AI slop, "generated code that hasn't been reviewed, shaped, or understood", and on what it wants: "candidates are distinguished by being able to lead them." Canva expects engineers to "take full ownership of any code they produce, whether they wrote it themselves or with AI assistance." Omnea wants people who "use AI as an accelerator rather than as a crutch." These are statements about what they watch for in interviews. The sentences on this bench are our suggestion, built to match. A tick on the bench proves nothing, because it only asks you.
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Source: Newton, Yes, you can use AI in our interviews, Canva, June 2025, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Source: Anthony, AI in our programming interviews, Omnea, 2026, https://www.omnea.co/blog/ai-in-our-programming-interviews
Say it: "Here is my claim paragraph. For each sentence, open this repo and find the file, commit or CI run that supports it, and give me the link. If you can't find one, mark the sentence unsupported and don't soften it. I'll remove it or go and do the work."
Described as: "how do I say I build with AI without sounding like a vibe coder", "what do I put on my CV about agents", "how do I prove I know what I am doing", "I want to claim this and back it up", "what do I actually say in the interview".
https://dev.dqnotes.com/proof/the-claim

### A repo worth showing
One public project small enough to read in ten minutes, where the process is visible as well as the result: the brief, the checks, the history, and what went wrong. This is our suggestion. We found no employer saying which artefact persuades them.
Anyone can now generate a plausible app in an afternoon, so an app shows little by itself. Our reasoning is that the process around it is harder to fake, and gives an interviewer something real to ask about. Size matters too: nobody reads your big project, and you can't defend every line of it.
Not Showcase app: A showcase asks them to admire the result. This asks them to check your working, and it can be a very small program.
Seen in: What this rests on is thin, and you should know that. Elicit describes a strong candidate writing a plan in a markdown file, recording a gap in the spec in QUESTIONS.md and having "a critic agent poke holes in their work", but that was in a live interview. Cursor ends with two days building real projects with the team, as reported second-hand, and doesn't ask for a portfolio. Anthropic says of its take-home that "scores correlate well with the caliber of candidates' past work", which is one remark about a test. Everything else here is us reasoning. Some will never open it, and the point of it may be mostly what building it does to your answers.
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Source: Business Insider on Cursor's hiring, via AOL, June 2025 (secondary), https://www.aol.com/inside-cursors-hiring-strategy-no-052015324.html
Source: Hume, Designing AI-resistant technical evaluations, Anthropic, January 2026, https://www.anthropic.com/engineering/AI-resistant-technical-evaluations
Say it: "Audit this repo as a sceptical senior engineer with ten minutes. In order: does the README say what this does and what isn't done, does one command run it, is there a brief, what does AGENTS.md say, what does CI block, and do the tests fail when the code is wrong? List what you can't find. Don't add anything yet."
Described as: "what should I put on my GitHub", "what project should I show", "do employers look at portfolios", "how do I show my process and not just the app", "my repo is one big commit".
https://dev.dqnotes.com/proof/portfolio-repo

### A commit trail
A project's history read as evidence: what changed, in what order, and why. A good trail shows small steps with reasons, checks arriving after mistakes, and things being undone. One giant commit shows nothing.
An agent will write a lovely message for a bad commit, so the wording proves little. The shape is harder to fake: sizes, order, a revert, a check that appears the day after the bug. Say plainly which commits an agent wrote. Nobody minds that, and being caught hiding it is the end of the conversation.
Not Green squares: An activity graph counts commits, and an agent can fill it in an afternoon. A trail is something you read, and what it shows is how decisions were made.
Seen in: This is our suggestion, not an employer's statement. The closest thing a hiring team has written is Omnea on the moment in an interview "when the pace suddenly accelerates", and why it wants people to "develop a shared understanding of what the code is doing". The bench reads a real git log in your browser. It reads words and numbers and can't judge the truth, so a good score there shows only that the trail is worth asking about.
Source: Anthony, AI in our programming interviews, Omnea, 2026, https://www.omnea.co/blog/ai-in-our-programming-interviews
Say it: "Keep this history honest. One change per commit, a message saying why, and a Co-Authored-By line when you wrote it. Don't squash failures out of the history. If a check caught you, say so in the fix commit. Then show me git log -30 --format='%h %s' --shortstat."
Described as: "my git history is a mess", "one huge commit called final", "can they tell I used AI from my commits", "should I clean up my history before showing it", "what does a good commit history look like".
https://dev.dqnotes.com/proof/commit-trail

### AGENTS.md as evidence
Your instructions file read as a record of what has gone wrong. A line naming a real command, file or past mistake shows someone was paying attention. A line like "write clean code" shows nothing at all.
This is one of the few places where how you direct an agent is written down and dated. Its history shows rules arriving after failures, and it gives a sceptic something specific to ask: what happened to make you write this line? You either have a story, or you copied it.
Not A prompt collection: A collection of clever prompts shows you read about prompts. A short file where each line has a reason shows you noticed something and did something about it.
Seen in: Hashimoto on a project's AGENTS.md: "Each line in that file is based on a bad agent behavior, and it almost completely resolved them all." The bench has real lines from this site's own instructions file, including the one that says when a mistake recurs, add a check, and that a line in the file is the last resort. That no employer has said they read these files is worth knowing. The question about what your instructions file says, and why, is our suggestion.
Source: Hashimoto, My AI Adoption Journey, 2026, https://mitchellh.com/writing/my-ai-adoption-journey
Say it: "Go through AGENTS.md line by line. For each one, use git log -p AGENTS.md to find the commit that added it and the mistake behind it, and tell me whether a check now holds it. Mark any line with no reason and no check for deletion. Don't add anything."
Described as: "what should my CLAUDE.md say", "my rules file is full of generic advice", "I copied someone else's instructions file", "can I show my prompts in an interview", "why is this rule even here".
https://dev.dqnotes.com/proof/agents-md-evidence

### The log of past failures
Also: Ratchet log, Failure log.
A running list of mistakes: what went wrong, what now prevents it, and proof that the check fails on the old code. It shows you noticed a failure, generalised it, and closed it off.
An agent can write you a convincing list of failures in a minute, so the entries are worthless by themselves. The proof is what counts. Run the check against the old code and it has to fail, and that is a thing someone can repeat in front of you.
Not Changelog: A changelog says what shipped. This says what went wrong, and what stops it happening again.
Seen in: pstack's skill called correct asks you to "Prove each new check fails on a real past mistake." Addy Osmani's heading: "The ratchet: every mistake becomes a rule". In our own experiment the harness could not make a vague brief catch what nobody had asked for, so a log like this records what you learned to ask for as well. This site's /receipts page publishes its own check results, failures included, as one example of the idea. That this persuades anyone is our suggestion.
Source: Tan, pstack: correct, https://github.com/cursor/plugins/blob/main/pstack/skills/correct/SKILL.md
Source: Osmani, Agent Harness Engineering, 2026, https://addyosmani.com/blog/agent-harness-engineering/
Say it: "Add an entry to docs/failures.md: the date, what went wrong, the commit with the old code, and the check that now catches it. Then check out that old commit, run the check, and paste the failing output into the entry. If it passes on the old code, the check is wrong, so tell me."
Described as: "how do I show what I learned from mistakes", "a list of everything the agent got wrong", "prove this check would have caught it", "keep a record of failures", "turn a bug into something I can show".
https://dev.dqnotes.com/proof/ratchet-log

### Decision records as evidence
A short dated note of a decision: what you chose, what you rejected, and why. As evidence, it shows a person made a call the agent couldn't make alone, and it lets someone ask you about it a year later.
An agent will happily produce options and has nothing riding on the outcome. Choosing between them is the job that stays yours. Be careful with these, though: an agent can write twenty backdated ones in a minute, and they fall apart the moment someone asks what you would do differently now.
Not Documentation: Documentation says how the thing works now. A decision record says why it isn't some other way, including what you turned down.
Seen in: Michael Nygard's format has five parts: title, context, decision, status and consequences. Elicit says what it is after in the room: "We need you to make a crisp decision and be able to explain why." Omnea lists "design decisions and trade-offs" among the things it watches. As portfolio evidence this is our suggestion, and the interviewing.io author's line about delegating core design decisions being a failure is one person's opinion.
Source: Nygard, Documenting Architecture Decisions, 2011, https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Source: Anthony, AI in our programming interviews, Omnea, 2026, https://www.omnea.co/blog/ai-in-our-programming-interviews
Source: Wahome, Meta's AI-assisted coding interview, interviewing.io (secondary, one author's view), https://interviewing.io/blog/how-to-use-ai-in-meta-s-ai-assisted-coding-interview-with-real-prompts-and-examples
Say it: "Write docs/adr/0004-queue-choice.md in Nygard's format: context, decision, status, consequences. List the two options I rejected and my reasons, in my words from this conversation. Date it today. If you don't know a reason, ask me, and don't invent one."
Described as: "why did we do it this way", "write down why I chose this", "which decisions did I make and which did the agent", "I want to show my thinking", "nobody remembers why it is like this".
https://dev.dqnotes.com/proof/adr-evidence

### Reviewing live in an interview
Reading code you didn't write in front of someone, saying what you are looking for and what you aren't getting to. Going by what hiring teams write, they are watching your order and your honesty at least as much as what you find.
When an agent writes the code, reading is most of what is left of your job, so this is a fair test of it. Time is always short. Saying "I read the payment handler and the migration, and I did not get to the tests" is a better answer than having looked at everything at speed.
Not LGTM: LGTM is a claim that you read it and would stand behind it. In this room you will be asked to back that claim up, one line at a time.
Seen in: Elicit: "Rapidly skimming a big diff and saying 'looks good' when there's no realistic chance they've reviewed it" is "more common than we'd like", and it calls this ability "an increasingly important engineering skill". Assembled: "One impressive candidate identified subtle bugs in an AI-generated algorithm that even our interviewers initially missed." Richard Ewing's proposed audit interview, which is one author's idea and no company's confirmed practice, asks: "This code passes tests but will fail in production." The ten-minute, 400-line version on the bench is made up.
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Source: Wang, How we learned to stop worrying and love the AI in coding interviews, Assembled, April 2025, https://www.assembled.com/blog/how-we-learned-to-stop-worrying-and-love-the-ai-in-coding-interviews
Source: Ewing, The audit interview, Built In, February 2026 (one author's proposal), https://builtin.com/articles/audit-interview-scorecard
Say it: "Generate a 400-line pull request on this repo with three seeded defects: one in auth, one off-by-one, and one test that passes for the wrong reason. Don't tell me where. Give me 10 minutes, then score what I found, what order I read it in, and what I said I hadn't read."
Described as: "they gave me a big diff in the interview", "how do I review code with someone watching", "I do not have time to read all of it", "where do I even start with this pull request", "I just said it looks fine".
https://dev.dqnotes.com/proof/live-review

### Debugging without the agent
Finding and fixing a bug yourself with the agent off: read the error, make a guess, test the guess, narrow it down. You don't have to be quick. You have to have a method, and be able to say what it is as you go.
Agents get stuck in loops, trying the same fix in new words. That is the moment a person is needed, and some interviews create it on purpose. If you have only ever watched an agent debug, you will feel this one, so practise it before someone is watching.
Not Algorithm puzzles: Puzzles test what you can recall. This tests what you do when something is wrong and nobody is helping, on code that looks like the job.
Seen in: Assembled lists candidates who "get stuck on bugs that AI couldn't solve out of the box" among its weak patterns. Canva's interviewers ask: "Can they identify and fix issues in AI-generated code?" Michael Truell of Cursor, as reported second-hand, says programming without AI is "a really great time-boxed test for skill and intelligence". Assembled also keeps "dedicated sessions focusing on core CS fundamentals without AI assistance". The largest gap in Anthropic's skill study was on debugging questions.
Source: Wang, How we learned to stop worrying and love the AI in coding interviews, Assembled, April 2025, https://www.assembled.com/blog/how-we-learned-to-stop-worrying-and-love-the-ai-in-coding-interviews
Source: Newton, Yes, you can use AI in our interviews, Canva, June 2025, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Source: Business Insider on Cursor's hiring, via AOL, June 2025 (secondary), https://www.aol.com/inside-cursors-hiring-strategy-no-052015324.html
Source: Anthropic, How AI assistance impacts the formation of coding skills, January 2026, https://www.anthropic.com/research/AI-assistance-coding-skills
Say it: "Stop. You have tried the same fix twice, and a third try won't help. Write down what we know: the exact error, the last commit where it worked, your top two hypotheses, and the one command that tells them apart. Don't change any code. I'll run it myself."
Described as: "the agent is stuck and I do not know what to do", "what if they make me code without AI", "it keeps trying the same fix", "I have never debugged this by hand", "my provider is down and I have a bug".
https://dev.dqnotes.com/proof/whiteboard-debug

### Explaining trade-offs
Saying what you chose, what you gave up for it, and when you would choose differently. If you can't name the cost of a decision, you probably didn't make it.
An agent will write a reasonable case for either side, so being able to recite one proves nothing. What it can't supply is the reason this was right for your project, and what would change your mind. Someone who only knows the upside has been told about the choice, and hasn't lived with it.
Not Best practice: A best practice comes with no cost attached. A trade-off is the same advice with the cost named, and the situation where it stops being good advice.
Seen in: Elicit on what it assesses: "We don't need you to make the same call we would." Ewing's proposed format ends by asking: "You can only deploy one change today. What is it?" OpenAI reports "minimal blocking merge gates" and adds "This would be irresponsible in a low-throughput environment." That is a good model of the form, because it comes with its own limit. The objections on this bench are ours, using this course's evidence.
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Source: Ewing, The audit interview, Built In, February 2026 (one author's proposal), https://builtin.com/articles/audit-interview-scorecard
Source: Lopopolo, Harness engineering, OpenAI, 2026, https://openai.com/index/harness-engineering/
Source: The same post, archived copy, https://web.archive.org/web/2026/https://openai.com/index/harness-engineering/
Say it: "Give me the strongest case against what I just decided: blocking merges on mutation score. Use what is in this repo and not general points. Then tell me what would have to be true for me to be wrong. Don't soften it, and don't finish by telling me I was right."
Described as: "why did you pick that", "what is the downside of your approach", "I only know the reasons for it", "how do I answer when they push back", "what would make you change your mind".
https://dev.dqnotes.com/proof/explain-tradeoffs

### Saying what you don't know
Saying plainly where your knowledge and your evidence stop. "I don't know, and here is how I would find out" is a complete answer. An absolute such as "always" or "fully tested" only needs one counter-example to sink you.
If you spend your days reading agent output, you can start to sound like it: fluent, sure of itself, and short of limits. A sceptic listens for that. Naming what your checks can't catch is also the fastest way to show you have really run them.
Not False modesty: Hedging everything is as empty as claiming everything. A limit is specific: it says exactly what you tested and exactly what you didn't.
Seen in: Elicit lists this among what it assesses: "When is something safe to assume? When do you need to stop and check?" This site has to do the same. Whenever it reports its own experiment it adds that this was one small model and one-function tasks, with no real repo. The rules in the bench's linter come from the list of things our research says not to say, and it only catches what we thought of, like any other gate.
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Say it: "Read this paragraph of mine for overclaims. Mark every absolute, every number with no source, and everything I state as general that I only saw once. Rewrite it with each limit attached, in under 80 words. Don't add anything I haven't told you."
Described as: "what if they ask something I do not know", "I do not want to sound like I am bluffing", "am I overclaiming", "how do I admit I have not measured it", "it sounds too confident".
https://dev.dqnotes.com/proof/honest-limits

### Employer AI policies
What a company says about using AI at work and in its hiring. They differ, often within one company: the same process may forbid AI in one round and expect it in the next. So you ask each time, and never assume.
If you work with agents all day, you can break a rule from pure habit. It matters on the job as well: what you may send to a model is a real boundary, and an agent will read whatever you point it at, including things that should never leave the building.
Not One rule for the whole process: Elicit, Assembled and Anthropic each keep some rounds free of AI, and allow it in others. Being good in one room and caught out in the next is an easy way to fail.
Seen in: Anthropic: application answers and live interviews are "without Claude unless we indicate otherwise". Canva replaced a fundamentals interview with an AI-assisted one. Eightfold runs a 60-minute round that "judges problem-solving and thought process rather than memory-based skills", with any tool. Read the source quality as well. Cursor's no-AI first screens are reported second-hand. Meta's line, "it's not 'cheating,' it's how the job is evolving", comes through CoderPad, a vendor to Meta, and another author says Meta's own materials say AI use has "no bearing on the outcome". Google's pilot of "human-led, AI-assisted" interviews is second-hand. Shopify's "Reflexive AI usage is now a baseline expectation at Shopify" is about working there, as reported. We found no primary statement from Stripe or Vercel. All of it can change, so this is as read by November 2026.
Source: Anthropic, Guidance on candidates' AI usage, read November 2026, https://www.anthropic.com/candidate-ai-guidance
Source: Newton, Yes, you can use AI in our interviews, Canva, June 2025, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Source: Nambi, Rethinking engineering interviews, Eightfold, October 2025, https://eightfold.ai/engineering-blog/rethinking-engineering-interviews-in-the-age-of-ai-from-code-memory-to-ai-mastery/
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Source: Wang, How we learned to stop worrying and love the AI in coding interviews, Assembled, April 2025, https://www.assembled.com/blog/how-we-learned-to-stop-worrying-and-love-the-ai-in-coding-interviews
Source: Business Insider on Cursor's hiring, via AOL, June 2025 (secondary), https://www.aol.com/inside-cursors-hiring-strategy-no-052015324.html
Source: CoderPad on Meta's AI-enabled interviews, October 2025 (a vendor to Meta, secondary), https://coderpad.io/blog/hiring-developers/ai-in-the-interview-is-not-cheating-it-is-the-job-according-to-meta/
Source: Wahome, Meta's AI-assisted coding interview, interviewing.io (secondary, one author's view), https://interviewing.io/blog/how-to-use-ai-in-meta-s-ai-assisted-coding-interview-with-real-prompts-and-examples
Source: Kashmir Reader, reporting Business Insider on Google, 17 May 2026 (secondary), https://kashmirreader.com/2026/05/17/were-evolving-our-interview-processes-to-recruit-best-talent-in-ai-era-googles-vp/
Source: Digital Commerce 360 on the Shopify memo, 8 April 2025 (secondary), https://www.digitalcommerce360.com/2025/04/08/internal-memo-shopify-ceo-declares-ai-non-optional/
Say it: "Draft a two-line message to the recruiter asking which rounds allow AI tools, which tools, whether they want to see my screen, and how they would like AI use disclosed on the take-home. Nothing else. I'll send it myself."
Described as: "am I allowed to use AI in the interview", "is it cheating to use Claude on the take-home", "what is their policy on AI", "can I paste work code into it", "do they want me to use it or not".
https://dev.dqnotes.com/proof/ai-policy

### Take-homes in the agent era
Work you do in your own time for a hiring process. Agents can now finish many of them, so companies are banning AI on them, redesigning them, or moving to scoped builds and live follow-up. Expect to defend every line.
If an agent can pass a test, passing it shows little, so the result matters less and the conversation after it matters more. Read the rules first. Then keep your plan, your questions and your changes, because those show the part the agent didn't do.
Not A chance to show the agent off: The artefact they get is the same whoever made it. What separates candidates is what you decided, what you deferred, and whether you can explain it.
Seen in: Anthropic's Tristan Hume says of its performance-engineering take-home, which allows AI: "each new Claude model has forced us to redesign the test." Claude Opus 4 "outperformed most human applicants", and Opus 4.5 "matched even those". He adds: "Judgment about how to invest in tooling is part of the signal." Assembled found a candidate using ChatGPT could "solve our standard algorithm challenge in under three minutes, leaving us with 40 minutes of awkward small talk", and moved to scoped problems. Anthropic's guidance for other assessments lists "having Claude write your code when we haven't explicitly allowed it" as not allowed. Elicit's strong candidate "didn't end up finishing every requirement".
Source: Hume, Designing AI-resistant technical evaluations, Anthropic, January 2026, https://www.anthropic.com/engineering/AI-resistant-technical-evaluations
Source: Wang, How we learned to stop worrying and love the AI in coding interviews, Assembled, April 2025, https://www.assembled.com/blog/how-we-learned-to-stop-worrying-and-love-the-ai-in-coding-interviews
Source: Anthropic, Guidance on candidates' AI usage, read November 2026, https://www.anthropic.com/candidate-ai-guidance
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Say it: "This is a take-home, and AI is allowed. Before any code, write PLAN.md and QUESTIONS.md: what I do first, what I defer, every gap in the spec, and what I am assuming. Then stop, and I'll decide. Keep a log of what you wrote and what I changed, so I can disclose it."
Described as: "can the agent just do the take-home", "they sent me a coding test to do at home", "should I tell them I used AI", "I did not finish all of it", "everyone can pass this test with AI now".
https://dev.dqnotes.com/proof/take-home

### Red flags interviewers look for
What interviewers report seeing when someone's claim to work with agents is hollow: no plan, permission checks off, a skimmed diff, stuck the moment the agent is stuck. A follow-up question or two is usually all it takes to find out.
These all look fine from the inside, which is what makes them dangerous. The code runs and the demo works. The flags only show when someone asks you to go one level down, and with agents it is possible to reach a finished product without ever having been one level down.
Not Using AI: None of these is about using AI. Elicit's own engineers spend most of their day steering agents. The flag is not being able to say what the agent did.
Seen in: Elicit: "Plan-first thinking is rarer than we expected", and "candidates who yolo claude --dangerously-skip-permissions and blithely accept every suggestion without reviewing the diff". Of those: "the lack of care, indifference to detail, and hubris make it easy to reject these candidates." Canva says struggling candidates "lacked the judgment to guide AI effectively or identify when its suggestions were suboptimal." Assembled saw candidates "get stuck on bugs that AI couldn't solve out of the box". Omnea looks at testing beyond "the binary", and says of AI that "it's all too keen to jump to a solution". The transcripts on the bench are made up.
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Source: Newton, Yes, you can use AI in our interviews, Canva, June 2025, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Source: Wang, How we learned to stop worrying and love the AI in coding interviews, Assembled, April 2025, https://www.assembled.com/blog/how-we-learned-to-stop-worrying-and-love-the-ai-in-coding-interviews
Source: Anthony, AI in our programming interviews, Omnea, 2026, https://www.omnea.co/blog/ai-in-our-programming-interviews
Say it: "Interview me as a sceptical senior engineer. Pick one claim from my README. Ask one question at a time and follow up twice on whatever I answer, asking to see the file. Don't help and don't reassure me. At the end, quote the exact moment my answer stopped being specific."
Described as: "how would they catch me out", "what makes someone look like a vibe coder", "what do interviewers hate", "I sounded fine until they asked a follow-up", "how can they tell I did not really understand it".
https://dev.dqnotes.com/proof/red-flags

### The hard questions
The 57 questions a sceptical engineer could ask about how you build with agents, the same ones as the interview room. Only a few are asked verbatim by a source, and most are our own reasoning. The bench tells you which is which.
You find out what you really know by being asked, and an agent can ask you all day without getting bored. Do these out loud with the agent off. If an answer needs the agent, that is your next piece of work.
Not Questions employers ask: Nobody has published their question list. Six of these are a source's own words put to the candidate, 19 rest on a published criterion, and 32 are our own reasoning, for reasons we give.
Seen in: The six asked verbatim: on Richard Ewing's audit interview page in Built In, the interviewer says "This code passes tests but will fail in production. Tell me why.", "You can only deploy one change today. What is it?" and "What is the primary constraint here?". The other three are Elicit's, written to candidates, such as "When is something safe to assume? When do you need to stop and check?". Ewing is one author's proposal and not a company's practice, and he sells a tool built on it. Some criterion questions are in Ewing's words too, but they are what his high-scoring candidate says, such as "What happens to this function if the database latency spikes?". The other criteria come from Canva, Elicit, Omnea, Assembled and Cursor, with our wording. The brutal questions, including the one about fooling yourself, are ours.
Source: Ewing, The audit interview, Built In, February 2026 (one author's proposal), https://builtin.com/articles/audit-interview-scorecard
Source: Newton, Yes, you can use AI in our interviews, Canva, June 2025, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Source: Anthony, AI in our programming interviews, Omnea, 2026, https://www.omnea.co/blog/ai-in-our-programming-interviews
Source: Wang, How we learned to stop worrying and love the AI in coding interviews, Assembled, April 2025, https://www.assembled.com/blog/how-we-learned-to-stop-worrying-and-love-the-ai-in-coding-interviews
Source: Business Insider on Cursor's hiring, via AOL, June 2025 (secondary), https://www.aol.com/inside-cursors-hiring-strategy-no-052015324.html
Say it: "Ask me this with no hints: "You have ten minutes and a 400-line agent-generated diff. How do you review it?" When I answer, ask me to show it on a real pull request in this repo. Then mark which parts of my answer I could back up and which I couldn't."
Described as: "what will they ask me", "give me practice interview questions about AI", "how do I prepare for this interview", "could I pass this with the AI off", "test me".
https://dev.dqnotes.com/proof/hard-questions

### What to do in your first week
Learn the rules on AI and data. Read before you generate. Find out what blocks a merge. Ship something small and reviewed. Ask what failed last. These are the same habits as this course, on code you don't own yet.
An agent lets a new person change a lot before they understand anything, which is the fastest way to lose a team's trust. Your claim was that you know what is going on. In week one you don't yet, so say so, and make your changes small enough that someone can check them.
Not Setting up your agent: Pointing your own harness at their repo can wait. Their checks, their rules and their history come first, and they may already have reasons you can't see.
Seen in: This list is our suggestion, and no employer we read sets it out. DORA lists a clear and communicated AI stance as a capability, so if your new team has no stated rule, that is worth asking about. Canva's line applies from your first commit: engineers "take full ownership of any code they produce, whether they wrote it themselves or with AI assistance."
Source: DORA, State of AI-assisted Software Development, 2025 (PDF, v. 2025.2), https://services.google.com/fh/files/misc/2025_state_of_ai_assisted_software_development.pdf
Source: Newton, Yes, you can use AI in our interviews, Canva, June 2025, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Say it: "Don't change anything. Read this repo and tell me how to run it, where the checks are, what blocks a merge, and which files change most (git log --format= --name-only | sort | uniq -c | sort -rn | head -20). Mark anything you guessed, and I'll ask the team about it."
Described as: "I just started a new job", "how do I use agents on a codebase I do not know", "what should I do first at a new company", "I do not want to break anything in week one", "am I allowed to use my agent here".
https://dev.dqnotes.com/proof/day-one

### What you can now say
Only what you can point at. If you did the work, it comes out short: my agents write most of the code, here is how I brief them, here is what stops bad work, here is what still gets through, and here is how I find out.
Eighteen chapters gave you the words, and that is all reading can give. Each sentence below comes true when you have done the thing it describes, and not before. The agents will keep changing, and these numbers will date. Being able to show how you know is the part that should last.
Seen in: What to point at, if you have it. 'I brief it': a brief with a definition of done, and the question it made you answer. 'Bad work gets stopped': a CI run refusing a real change, and a table of the ways round it you tried. 'I know what still gets through': a surviving mutant, or a bug your tests missed, and what you did next. 'I learn from it': a failure log whose checks fail on the old code. 'I can read it': a file you can walk through with the agent off. 'I don't know how much faster I am': your own small trial, with its wide interval. Everything in this chapter was read by 5 November 2026. Check it again before you quote it, and if you can't point at something, leave it out.
Source: METR, We are changing our developer productivity experiment design, February 2026, https://metr.org/blog/2026-02-24-uplift-update/
Source: Brady, Engineering interviews in the era of agents, Elicit, 2026, https://elicit.com/blog/engineering-interviews-in-the-era-of-agents
Source: Newton, Yes, you can use AI in our interviews, Canva, June 2025, https://www.canva.dev/blog/engineering/yes-you-can-use-ai-in-our-interviews/
Say it: "Here is my claim, and here is my repo. Ask me the two hardest follow-up questions the repo can't answer for me. Then I'm turning you off and answering them out loud."
Described as: "so what can I actually say now", "I finished the course, what now", "am I ready to say I build with agents", "how do I sum all this up", "does reading this count".
https://dev.dqnotes.com/proof/what-you-can-say