Now for the first way a test lies: it changes its mind.
Flaky test
A test that sometimes passes and sometimes fails on the same code. Usually the cause is timing, shared state or something outside the test, such as the network or the clock.
A flaky test teaches everyone to ignore red. An agent is worse: it may "fix" code that was fine, or retry until it gets a green run and report success. Retries hide the fault and do not remove it.
Say it to your agent
“cart.test.ts fails about one run in five. Run it with bun test --rerun-each 50 and with --randomize to find out whether it is timing or order. Replace the fixed sleep by awaiting the real promise. Don't add retries.”