On the Origin and Evolution of Training-Inference Mismatch in LLM Reinforcement Learning
On-policy reinforcement learning for large language models assumes that training responses are sampled from the current policy. In practice, an inference engine samples the responses and a separate trainer computes the gradients, and the \textbf{training-inference mismatch} between the two makes…
By Yu Li, Yufa Zhou, Tian LanarXiv cs.LG
Read the full story
Originally published by arXiv cs.LG on Nov 2, 2026.
Comments
Sign in to join the discussion.