Decode-Latency Feedback Prefill: A Model-Free Controller and Its Generalization Limits

Concurrent autoregressive inference creates a fundamental interference problem: prefilling a newly arrived long prompt can delay tokens for requests that are already decoding.