Distributed LLM Inference via Communication-Efficient Speculative Decoding
Distributed speculative decoding has emerged as a promising approach for device--server collaborative large language model (LLM) inference, but its decoding speedup is often limited by the vocabulary-scale communication overhead.
By Joohyun Shin, Byeonghyeon Park, Keumseo Ryum, Joonhyuk KangarXiv cs.LG
Read the full story
Originally published by arXiv cs.LG on Oct 28, 2026.
Comments
Sign in to join the discussion.