Understanding Input/Output tokens vs Latency Tradeoff

Every few months, openai or anthropic (or \) makes headlines for a new jaw dropping decrease in token costs. While these are impressive feats, cost is only ...