Caching

Keeping a copy of an answer somewhere quick, so you can skip the slow work next time. A hit is when the copy is there. A miss means doing the work anyway. A cache is small, so something has to decide what to throw out, often the least recently used.

A cache is an easy diff that makes a benchmark look great, so it is a favourite agent fix, and it swaps a speed problem for a correctness problem. Check the hit rate on real traffic first: a cache that rarely hits costs memory and complexity, and gives nothing back.