Security

A short written answer to: what are we building, what can go wrong, what will we do about each thing, and did that work? It turns "be secure" into a list you can check.

An agent builds what the brief describes, and a brief usually describes honest users. Until someone writes down what a dishonest one would try, nothing in the work points at it.

Every place someone outside can send your system something: forms, URLs, uploads, webhooks, headers, and any file or page your agent reads. The bigger it is, the more you have to defend.

Agents grow it quietly. A debug route left in, a new webhook, a dependency, an MCP server: each one works, each one passes its tests, and each one is another door. It rarely shows up as the point of the change, so read the diff for it.

A list of the ten most serious kinds of web application risk, kept by the non-profit OWASP and updated every few years. The trade uses it as shared vocabulary, and it is a good first checklist for a review.

It gives you words an agent knows well. "Check this against A01 and A05" gets a far more specific answer than "is this safe?"

Text from a user gets pasted into something another program reads as instructions: a database query, a shell command, a web page. The program can't tell which part you wrote, so the user's text can become commands.

Joining strings is the shortest way to write these, and it works perfectly on every honest input. So an agent can ship it with a full set of green tests.

SQL is the language for asking a database for things, as in SELECT * FROM users WHERE name = 'alice'. Paste a user's text into that sentence and they can rewrite the question. A parameterised query sends the value separately, so it can only ever be a value.

One typed line can return every row or change data it should never touch. And the fix is complete, which is rare in security, so there's no reason to accept anything less.

An attacker gets their text into your page, and the visitor's browser runs it as code. That code runs as your site, with the visitor's logged-in session, so it can do whatever they can.

A comment box is enough. Everyone who views the comment runs the attacker's script, and passing tests won't show it, because a test comment like "hello" looks right either way.

Converting a value when you put it into a page, so the browser reads it as text. The right conversion depends on where it lands: inside a tag, in an attribute, or in a link are three different problems.

Escaping HTML does nothing for a link. javascript:alert(1) has no angle brackets to escape, and clicking it runs the code. This is exactly the kind of gap the code passes tests with.

Another website makes your visitor's browser send a request to your site. The browser attaches their cookie automatically, so your server sees a logged-in user asking to do something they never asked for.

The attacker never sees your site or the response, and no password is stolen. A route that changes data and trusts the cookie alone can be fired from any page they happen to visit.

A feature fetches a URL the user gave it, such as an image from a link. The attacker gives it an internal address, and your server, which sits inside your network, fetches what the attacker could never reach.

Cloud servers can often reach a metadata address that hands out credentials. "Fetch this URL" sounds harmless and takes one line, so it passes without a second look.

A filename from a user includes ../, which means "the folder above", until it climbs out of the folder you meant to serve. Then your server hands over files you never meant to expose.

Removing ../ with a replace looks like a fix, and it isn't: ....// becomes ../ once the inner part is stripped. The check has to run on the final resolved path.

A link on your own site sends visitors wherever a parameter says, as in /login?next=... An attacker sends a link that starts with your trusted address and ends on theirs, ready for a convincing fake page.

The check looks easy and has many ways round it. //evil.example has no scheme but still leaves your site, and so does a backslash in some positions, because browsers read addresses differently from how a string comparison does.

Checking that what arrives is what you expect, in type, length and format, and refusing it if not. It must run on the server, because anyone can skip the browser and send the request directly.

It stops plenty of bugs and shrinks what an attacker can try. It does not make a query safe, and an agent asked to "sanitise inputs" may reach for a list of banned characters and call the job done.

Code is memory-safe when it can't read or write memory it doesn't own. C and C++ leave that to the programmer, so one wrong length can overwrite whatever sits next to it. Languages like Rust, Go, Java and Python check it for you, or won't let you write it.

It is the largest single kind of serious bug in big C and C++ systems, and ordinary tests rarely see it, because the program carries on. Writing JavaScript or Python covers most of it for you. It comes back the moment an agent writes a native module, or offers to port one.

Proving who someone is: a password, a code from their phone, a passkey. The result is usually a session, so the server recognises them on every later request.

Login looks simple and is full of traps, from resets to session handling to error messages that reveal which accounts exist. When an agent offers to write it from scratch, the safer answer is usually an established library.

Deciding what a known user may do, on this thing, right now. The server has to decide on every request, because anything in the browser can be changed by the person using it.

It's the check that has no visible effect when missing. The honest user sees the same screen either way, so nothing in a normal demo gives it away.

The server fails to enforce what a user may do, so they can do it anyway: open an admin page, change a role, act on data that isn't theirs. It tops the OWASP list.

Hiding a button isn't a control, because the route behind it still answers. Agents often write the visible part and test the happy path, and static scanners struggle with this class, because no line of code is wrong. The bug is a line that isn't there.

The kind of broken access control you can do with a keyboard. /invoices/1003 is yours, so you try /invoices/1004 and get a stranger's, because the server looks up the id and never asks whose it is.

A test that a user can read their own invoice passes with or without the check. You need a test with two users, and an agent doesn't write that unless asked.

Storing a scrambled form of a password that can't be turned back. At login you scramble what they typed and compare. It has to be deliberately slow, and salted with a random value for each user.

One day your database may be stolen. Fast hashes like SHA-256 let an attacker try guesses at enormous speed, and without a salt, two people with the same password show up as the same value.

Scrambling data so only someone with the key can read it. In transit protects it on the wire, and that is what HTTPS does. At rest protects it on disk. Each covers one place, and neither stops a request your app treats as valid.

Saying "the data is encrypted" can be true and useless. A disk-encrypted database still hands plain text to a SQL injection. Ask who can still read it, not whether it is encrypted.

A login token the server signs and the browser sends back. It has three parts: header, payload, signature. The payload is only encoded, so anyone can read it. The signature proves it hasn't been changed, if the server checks it properly.

The header says which algorithm was used, and the attacker writes the header. A verifier that believes it can be told no signature is needed. And a guessable key can be cracked offline, with nothing to notice it.

A value that grants access: a password, an API key, a signing key. It belongs outside the code, in an environment variable or a secret manager, and it should only ever be readable by what needs it.

Agents want things to work, so a key pasted into a chat can end up in a config file, a test, a log line or a commit. Anything committed stays in the history, even after you delete the line.

A tool that searches your code and history for things that look like keys. It looks for known shapes, like an AWS key, and for strings that look too random to be ordinary text. That second measure is called entropy.

It's a cheap check that runs whether or not anybody remembers, and it catches the mistake before the push. It's also fallible: a password with no special shape and no keyword nearby goes straight past it.

Giving each person, program and agent only the access its job needs, and no more. Then when one of them is fooled or compromised, the damage stops at what it could reach.

Broad access is the quickest way to make something work, so it's what tends to get set up, and it stays. This decides how bad a bad day gets, and it works even against attacks nobody has thought of yet.

A browser rule: a script on one site can't read a response from another site, unless that site's server says it may. The server says so in a header, Access-Control-Allow-Origin. It only constrains browsers, so curl ignores it.

A CORS error is loud and blocks the feature, so the fastest way to silence it is to allow everything. That makes the error vanish and may open up logged-in data to any website. Look at what your agent did to make that error go away.

A header that tells the browser what this page may load and run: scripts only from here, no plug-ins, no framing. If an attacker's script gets into the page anyway, the browser can refuse to run it.

It's a second line of defence for when escaping fails somewhere. It's also easy to weaken: one 'unsafe-inline' added to quiet an error removes much of the protection, and an agent fixing a broken page may well do that.

Lines your server adds to each response that turn on browser protections: always use HTTPS, don't guess file types, don't let other sites frame this page, don't leak the full address when linking out. They take minutes to add.

No app needs to do without them, and none of them will appear unless someone asks, because the app works the same without them. That also makes them easy to check, with one curl.

Capping how many requests one person or address can make in a period, and answering 429 Too Many Requests after that. It makes guessing a password or a code slow enough to be pointless.

Without it, a 6-digit code falls to a script in a sitting. An endpoint that calls a paid model can also run up your bill overnight. It's invisible in testing, since nobody tests by sending a million requests.

Attacking you through something you depend on. A package gets taken over or swapped, you install the update as usual, and its code runs with all the access your program or your laptop has.

A package can run code the moment you install it, with no import needed. A version range like ^1.4.0 accepts new releases on its own, and an agent adds dependencies with less hesitation than you would.

A model invents a package name that sounds right. It fails to install, and that's a clue. Then somebody notices the invented name and registers it with their own code, and now the same mistake installs without a murmur.

The name only needs to be guessed once by an attacker and repeated by a model. An agent that installs what it thinks exists is how that turns into code running on your machine, with nobody having chosen it.

Checking the versions you have installed against a public list of known vulnerabilities. npm audit is one such command. It can only find what has already been reported.

It costs nothing and catches old, known problems. But it can be noisy, flagging code you never call, and it won't notice a brand-new malicious package at all. An agent told to make it clean may do something worse to get there.

A public id for one known security flaw, such as CVE-2025-54136, so everyone can talk about the same bug. Its severity is usually given as a CVSS score from 0 to 10.

A score tells you how bad a flaw is in general. It doesn't know your app. A 9.8 in code you never call matters less than a 6.1 on your login page, so ask whether it's reachable before you ask about the number.

Two kinds of automated security check. SAST reads your code without running it. DAST runs your app and attacks it from the outside. Each finds things the other can't.

Both are ready-made sensors for an agent loop. Both miss things: OWASP says static tools have "High numbers of false positives" and struggle with authentication and access control. So a clean scan isn't proof, and in particular it says little about whose data a user can reach.

Text gives a model orders it wasn't meant to take. A model reads your instructions and a web page as one stream of words, so a line in that page can say what to do next. Indirect means it was planted somewhere the agent would read.

Your agent reads issues, documents and web pages all day, and each could be written by anyone. You can't filter this out completely, so the job is deciding what the agent is allowed to do once it's fooled.

Simon Willison's name for three powers that are dangerous together: access to your private data, exposure to untrusted content, and a way to send data out. With all three, a stranger's text can steal your data. Remove any one and that attack has no route.

It turns an unsolved problem into a design question you can answer today. You can't guarantee a model won't be fooled, but you can list its tools and see whether all three legs are there.

An MCP tool's description is read by the model, and you usually see a short version. A malicious server hides instructions in it, such as read this file and pass it along. The tool can also change after you approved it, which is called a rug pull.

Installing an MCP server means letting its author write into your agent's instructions. Nothing has to be run for that to happen: the text only has to be read.

A separate OWASP list for apps built on language models. Prompt injection is first. It also covers leaks of private data, poisoned models, trusting model output too readily, and giving a model too much power.

Two entries matter most for agents. Improper Output Handling: model output is untrusted input, so paste it into a query or a page and you're back to injection. Excessive Agency: a model with more tools than it needs.

Having the operating system fence in an agent's commands: which files they can touch and which other machines they can reach. It holds even if the agent is fooled, because it's enforced from outside the model.

Instructions are requests, and a sandbox is a wall. It only works as a pair, too. A filesystem rule without a network rule lets a key be read and sent. A network rule without a filesystem rule still leaves your machine open to change.

Studies ask models to write code for risky tasks and scan the result. Numbers near half failing get quoted widely. They come from particular tests with particular limits, so they're a warning that this happens, not a rate for your codebase.

If you quote the headline as a fact, a sceptical listener will take it apart. The defensible claim is smaller and more useful: models often write insecure code when nobody asks them for secure code, and passing tests won't tell you.

Reading a change only for how it could be abused, separately from whether it works. It asks: who can reach this, what can they send, and what's the worst they can do? A person or a fresh agent can do it.

The agent that wrote the code reads it as an author. A second pass with a narrow brief catches what a general review doesn't, above all a check that was never written, which no scanner can point at.