AI agents cost so much because they resend the entire conversation as input on every turn. Turn 10 doesn’t just pay for turn 10 — it pays to reprocess turns 1 through 9 all over again. That makes total token cost grow with roughly the square of the turn count, so a 20-turn loop can cost several times more than 20 separate single calls. The AI agent conversation cost calculator shows the curve for your own settings.
The models are stateless
An LLM has no memory between calls. To continue a conversation, the client resends the whole transcript — system prompt, every previous user message, every previous model reply, every tool call and result — as the input for the next turn. The model re-reads all of it to produce one more message.
So the input for each turn keeps growing. Early turns are cheap. Later turns carry the weight of everything before them. This is why “it’s only 20 turns” is misleading: it’s not 20 small calls, it’s one call, then a slightly bigger one, then a bigger one, all the way up.
Why the growth is quadratic
Say every turn adds a fixed chunk of new tokens — a user message plus a model reply. On turn n, the input is roughly n chunks of accumulated history.
Add that up across all turns: 1 + 2 + 3 + … + n. That sum is about n²/2. Doubling the number of turns roughly quadruples the total tokens processed, not doubles them. That is the whole reason agents feel disproportionately expensive as conversations run long.
Worked example: history growth
Assume each turn adds 500 new tokens (user + model), on top of a 1,000-token system prompt that rides along every turn.
| Turn | History resent (input tokens) |
|---|---|
| 1 | 1,000 + 500 = 1,500 |
| 5 | 1,000 + 2,500 = 3,500 |
| 10 | 1,000 + 5,000 = 6,000 |
| 20 | 1,000 + 10,000 = 11,000 |
Now total the input across all 20 turns: it comes to roughly 125,000 input tokens, even though the conversation itself only ever contains 11,000 tokens at its longest.
Compare that to 20 independent single calls of the same 1,500-token size: 20 × 1,500 = 30,000 input tokens. The agent loop processes about 4× more input than the naive “20 calls” estimate — purely from re-reading history. Longer conversations widen the gap further.
Tool calls make it worse
Agents don’t just chat — they call tools, and each call adds two things to the transcript: the call itself and the returned result. Tool results are often the largest items in the whole conversation: a search result set, a page of JSON, a file dump, a stack trace. Once that lands in the history, it gets resent on every remaining turn.
A single 3,000-token tool result on turn 3 of a 20-turn loop gets reprocessed 17 more times. Verbose tools are one of the biggest hidden cost drivers in real agents.
How to cap the cost
You can’t remove history entirely, but you can stop it compounding:
- Trim or summarise old turns. Keep the recent messages verbatim and replace older ones with a short summary. This is the single biggest lever — it caps the history size instead of letting it grow every turn.
- Cache the system prompt. The fixed instructions ride along every turn, so they’re the perfect thing to cache. See how prompt caching and batching cut API costs — caching can knock most of the cost off that repeated prefix.
- Keep tool outputs compact. Return only the fields the model needs. Truncate long results, or store the full blob externally and pass back a short reference.
- Cap the turn count. Set a hard ceiling and force a summarise-or-stop when the loop hits it. Because cost is quadratic, the last few turns are the most expensive ones to allow.
One more compounding factor: if the agent uses a reasoning model, each turn also generates hidden thinking tokens on top of the growing history — see why reasoning models cost more.
Before you ship an agent, model the curve. Put your turn count, per-turn tokens, and system prompt size into the AI agent conversation cost calculator, and cross-check the per-model rate with the LLM cost calculator. Seeing the quadratic line is usually enough to convince anyone to add history trimming on day one.
