Tokeniser
Also called Tokenizer, Byte pair encoding, BPE
The program that cuts text into tokens before the model sees it and joins tokens back into text afterwards. Its vocabulary is fixed in advance, so the same text can count differently on different models.
It explains costs that look strange: deep indentation, long names in code, encoded data such as base64, and languages that were rare in its training text all use tokens differently from the English you estimated with.
Say it to your agent
“Don't estimate by dividing characters by 4. Call the count_tokens endpoint for this model with the real system prompt, tools and messages, and tell me input_tokens.”