Understanding Target Probability Reuse in Speculative Decoding
Speculative decoding uses a target model to verify a tree of draft tokens in parallel. We study whether the resulting next-token distributions can help select the next tree.
By Jungseob Lee, Giyoon Jang, Chailin Nam, Jiwoo Kum, Sihyeon Lee, Kyojun Choo, Jihun Song, Eunseo Song, Chanjun ParkarXiv - cs.CL (NLP)
Read the full story
Originally published by arXiv - cs.CL (NLP) on Oct 22, 2026.
Comments
Sign in to join the discussion.