TraceSpec: Understanding and Improving Speculative Decoding for LLM Agents
LLM agents execute tasks through repeated model calls over evolving histories of reasoning, actions, and tool feedback. Aggregate speculative-decoding metrics merge these distinct decisions, providing limited guidance on where draft supervision is needed.
arXiv - cs.CL (NLP)2 min read