The next surprise is that a server doesn't get slow when it is full. It gets slow well before that.
Queueing and utilisation
Also called M/M/1, Load shedding
When work arrives at random, it sometimes bunches up and has to wait in line. Utilisation is how busy the server is. As it nears 100%, the line grows faster and faster: at 50% a request takes about twice its work time, at 90% ten times, at 99% a hundred times.
Running everything at 95% looks efficient, and the people waiting pay for it. The same sum applies to you: an agent that hands a reviewer more than they can read doesn't get faster reviews, only a longer line. And a word to keep straight: this is where chapter 9's word has its software meaning.
“This worker runs at 90% busy and p99 is climbing. Don't just raise the queue size, since that only lengthens the wait. Put a limit of 1,000 on the queue, return a 429 with Retry-After when it is full, and graph queue depth next to p99.”