Chapter 10: Tests

Everything so far relies on somebody choosing the inputs, and an agent chooses the inputs it already handled. So stop choosing them.

Property-based testing

Also called Generative testing

Instead of a few examples, you state a rule that must always hold, such as "the shares always add up to the bill". The tool invents hundreds of inputs, and when one fails it cuts it down to the smallest input that still fails.

It is much harder to fit code to a rule than to an example. Rules also answer the oracle problem: you may not know the right split of 7312 between 6, but you know it has to add up. For an agent, the shrunk failure makes a very clear bug report.

Say it to your agent

“Add fast-check to splitBill's tests with bun add -d fast-check. Assert for all totals and weights: shares sum to the total, every share is a whole number, and there is one share per person. Paste the shrunk counterexample if it fails.”

The workshop

Name a word, or describe what you mean in everyday language.