The proof

A rough ranking of how far a kind of study lets you believe its claim: randomised trials near the top, then comparisons over time, then surveys and benchmarks, then single stories. A ranking only makes sense for one question at a time.

An agent summarising research gives a survey and a trial the same confident sentence, so the ranking has to come from you. And the person across the table has heard all the loud numbers. You can't check every study, but you can find out what kind it is in a minute.

A test where chance decides who or what gets the treatment, so the two groups differ only by luck and by the thing you are testing. That is what lets you say one thing caused another, and not just that they went together.

If you choose which tasks to give the agent, you will pick the ones you expect it to help with, and comparing times afterwards mostly measures your choice. It is why a before-and-after on your own work proves so little, and why a coin flip on your next ten tasks proves more.

A range around a result showing how far it could be from the truth just from the luck of what got sampled. Fewer samples, wider range. It covers bad luck only: a biased study can have a narrow interval around the wrong answer.

An agent doesn't give the same answer twice, so one or two good runs tell you very little. A prompt that passed 15 times out of 15 could still fail one run in five. This is a daily tool when agents do the work, and not only something for reading papers.

How common something is before you test for it. When the thing is rare, even a good detector's alarms are mostly wrong, because the many good cases set off more false alarms than the few bad ones set off true ones.

Your gates and your reviewer agents are detectors. If one change in thirteen is broken, most refusals land on good work, and an agent told to satisfy them rewrites good code. The same sum applies to you in an interview: when nearly everyone uses AI, saying you do tells them almost nothing.

Studies by METR, a non-profit, timing experienced open-source developers on their own real tasks, with chance deciding whether AI was allowed. The 2025 one found them slower. The 2026 follow-up says it couldn't produce a reliable new number.

If you say you build with agents, this is the study they will ask about, and you can get it wrong in two directions. "AI makes developers slower" overstates a trial of 16 people on early-2025 tools. "METR now shows AI speeds you up" overstates a follow-up METR itself distrusts.

The distance between how much a tool feels like it helps and how much it measurably does. It can be large, it can point the wrong way, and the person inside it can't see it from there.

An agent makes work feel easier. Waiting feels like resting, and a wall of new code feels like progress. So "it makes me faster" is no evidence, even when you say it in good faith, and a strong candidate knows that about their own impression.

Losing a skill, or never building it, because something else does the work for you. With agents, the worry is that you can ship code you can't explain or debug, and that you don't notice until the agent gets stuck and so do you.

Directing an agent well depends on the skill it might erode: you can only catch what you are able to read. You don't feel this happening, since the work still gets done. It comes up in interviews that take the agent away, which some do on purpose.

Surveys asking developers how far they trust AI tools. The number that gets quoted depends mostly on how the question was asked and which answers someone decided to count as trust.

These numbers get used to argue that agent output is or isn't reliable. A survey can't show that. It tells you how people feel about the output, and it says nothing about how often the output is right. Only your own checks can tell you that.

Google Cloud's DORA research on how software teams deliver. Its 2025 report, from a survey of nearly 5,000 technology professionals, calls AI an amplifier: it makes strong teams stronger and struggling teams worse.

It changes the question from "does AI work?" to "what is it amplifying?". An agent pointed at a repo with no tests and vague briefs gives you more of both, faster. It is also a good answer to anyone asking what you would watch once agents write most of the code.

More code isn't more software shipped. Output is what gets produced: lines, commits, pull requests. Delivery is what reaches people and does its job. Agents can raise the first a long way while the second moves much less.

An agent can always produce more of anything you count, so counting what it produces rewards the wrong thing. If you tell an interviewer you opened twice as many pull requests, the next question is where they all went.

Research published by a company that sells something in the same area. It can be perfectly good work. You say who paid when you quote it, and you read what it measured with that in mind.

Agent tooling is full of these, and the label falls off as they get repeated. Many also score code from one shot with nothing around it, which isn't how you are working if you have a harness. So their numbers don't carry across, in either direction.

The questions with no good study behind them yet. For agent-written code the biggest is this: does it cause more production incidents? We found no primary, peer-reviewed measurement, and saying so is a stronger answer than filling the gap.

An agent asked for a statistic will find something that sounds like one. Stories about agents deleting databases are real, and they are stories, so they can't give you a rate. A number with no source is the easiest thing in an interview to catch.

Four questions to ask of any number about AI and productivity, in this order: who measured it, what was it compared with, what did they count, and who paid. A claim that survives usually comes out smaller and more useful.

You will meet these numbers in interviews, in pitch decks, and from your own agent, which will repeat them cheerfully. Being able to take one apart calmly, in front of someone, is itself evidence that you know what you are doing.

A measurement says what happened, to what, compared with what. A prediction says what will happen. Both can be worth hearing, and they need different treatment: you check a measurement, and with a prediction you ask what would show it right or wrong, and when.

A forecast from someone who knows the field sounds just like a finding, and it gets repeated as one. Sort it sentence by sentence: what could be tested today, what only later, and what no result could touch. Then you know what to do this week, whichever way it turns out.

A short paragraph you can say out loud in which every sentence points at something you can open: a file, a commit, a failing run. If a sentence has nothing behind it, you either drop it or go and do the work.

Saying you use agents is close to saying nothing, because nearly everyone does. What is rarer, and what employers describe wanting, is someone who can lead the agent and can show how they know the result is any good. The sentences only count if they are specific, and only if they are true of you.

One public project small enough to read in ten minutes, where the process is visible as well as the result: the brief, the checks, the history, and what went wrong. This is our suggestion. We found no employer saying which artefact persuades them.

Anyone can now generate a plausible app in an afternoon, so an app shows little by itself. Our reasoning is that the process around it is harder to fake, and gives an interviewer something real to ask about. Size matters too: nobody reads your big project, and you can't defend every line of it.

A project's history read as evidence: what changed, in what order, and why. A good trail shows small steps with reasons, checks arriving after mistakes, and things being undone. One giant commit shows nothing.

An agent will write a lovely message for a bad commit, so the wording proves little. The shape is harder to fake: sizes, order, a revert, a check that appears the day after the bug. Say plainly which commits an agent wrote. Nobody minds that, and being caught hiding it is the end of the conversation.

Your instructions file read as a record of what has gone wrong. A line naming a real command, file or past mistake shows someone was paying attention. A line like "write clean code" shows nothing at all.

This is one of the few places where how you direct an agent is written down and dated. Its history shows rules arriving after failures, and it gives a sceptic something specific to ask: what happened to make you write this line? You either have a story, or you copied it.

A running list of mistakes: what went wrong, what now prevents it, and proof that the check fails on the old code. It shows you noticed a failure, generalised it, and closed it off.

An agent can write you a convincing list of failures in a minute, so the entries are worthless by themselves. The proof is what counts. Run the check against the old code and it has to fail, and that is a thing someone can repeat in front of you.

A short dated note of a decision: what you chose, what you rejected, and why. As evidence, it shows a person made a call the agent couldn't make alone, and it lets someone ask you about it a year later.

An agent will happily produce options and has nothing riding on the outcome. Choosing between them is the job that stays yours. Be careful with these, though: an agent can write twenty backdated ones in a minute, and they fall apart the moment someone asks what you would do differently now.

Reading code you didn't write in front of someone, saying what you are looking for and what you aren't getting to. Going by what hiring teams write, they are watching your order and your honesty at least as much as what you find.

When an agent writes the code, reading is most of what is left of your job, so this is a fair test of it. Time is always short. Saying "I read the payment handler and the migration, and I did not get to the tests" is a better answer than having looked at everything at speed.

Finding and fixing a bug yourself with the agent off: read the error, make a guess, test the guess, narrow it down. You don't have to be quick. You have to have a method, and be able to say what it is as you go.

Agents get stuck in loops, trying the same fix in new words. That is the moment a person is needed, and some interviews create it on purpose. If you have only ever watched an agent debug, you will feel this one, so practise it before someone is watching.

Saying what you chose, what you gave up for it, and when you would choose differently. If you can't name the cost of a decision, you probably didn't make it.

An agent will write a reasonable case for either side, so being able to recite one proves nothing. What it can't supply is the reason this was right for your project, and what would change your mind. Someone who only knows the upside has been told about the choice, and hasn't lived with it.

Saying plainly where your knowledge and your evidence stop. "I don't know, and here is how I would find out" is a complete answer. An absolute such as "always" or "fully tested" only needs one counter-example to sink you.

If you spend your days reading agent output, you can start to sound like it: fluent, sure of itself, and short of limits. A sceptic listens for that. Naming what your checks can't catch is also the fastest way to show you have really run them.

What a company says about using AI at work and in its hiring. They differ, often within one company: the same process may forbid AI in one round and expect it in the next. So you ask each time, and never assume.

If you work with agents all day, you can break a rule from pure habit. It matters on the job as well: what you may send to a model is a real boundary, and an agent will read whatever you point it at, including things that should never leave the building.

Work you do in your own time for a hiring process. Agents can now finish many of them, so companies are banning AI on them, redesigning them, or moving to scoped builds and live follow-up. Expect to defend every line.

If an agent can pass a test, passing it shows little, so the result matters less and the conversation after it matters more. Read the rules first. Then keep your plan, your questions and your changes, because those show the part the agent didn't do.

What interviewers report seeing when someone's claim to work with agents is hollow: no plan, permission checks off, a skimmed diff, stuck the moment the agent is stuck. A follow-up question or two is usually all it takes to find out.

These all look fine from the inside, which is what makes them dangerous. The code runs and the demo works. The flags only show when someone asks you to go one level down, and with agents it is possible to reach a finished product without ever having been one level down.

The 57 questions a sceptical engineer could ask about how you build with agents, the same ones as the interview room. Only a few are asked verbatim by a source, and most are our own reasoning. The bench tells you which is which.

You find out what you really know by being asked, and an agent can ask you all day without getting bored. Do these out loud with the agent off. If an answer needs the agent, that is your next piece of work.

Learn the rules on AI and data. Read before you generate. Find out what blocks a merge. Ship something small and reviewed. Ask what failed last. These are the same habits as this course, on code you don't own yet.

An agent lets a new person change a lot before they understand anything, which is the fastest way to lose a team's trust. Your claim was that you know what is going on. In week one you don't yet, so say so, and make your changes small enough that someone can check them.

Only what you can point at. If you did the work, it comes out short: my agents write most of the code, here is how I brief them, here is what stops bad work, here is what still gets through, and here is how I find out.

Eighteen chapters gave you the words, and that is all reading can give. Each sentence below comes true when you have done the thing it describes, and not before. The agents will keep changing, and these numbers will date. Being able to show how you know is the part that should last.