SpecRelay2 min read
SpecRelay: Amortize or Avoid the Round Trip in Edge-Cloud Speculative Decoding
Edge-cloud speculative decoding moves autoregressive drafting to an edge device while a larger cloud model verifies draft tokens in parallel, so that edge drafting and cloud verification overlap under a synchronous draft-verify schedule.
By Jai Jain, Chihyo Ahn, Yong Liu, Charles Chen, Duc N. M Hoang, Ajay Kumar Jaiswal, Fatih Ilhan, Han-Byul Kim, Lauren Hannah, Richa Dixit, Minsik Cho, Yingyan Celine Lin, Hyesoon KimarXiv - cs.AI
Read the full story
Originally published by arXiv - cs.AI on Nov 17, 2026.
Comments
Sign in to join the discussion.