How to Reduce LLM Token Costs in Long Conversations: What Caching Saves, and Where It Stops
A long conversation costs far more than its number of turns suggests, because every request carries the system prompt and the entire history again, so total input grows with the square of the number of turns.
Maximem Synap Blog13 min read
