HybridSpec: Throughput-Aware Speculative Decoding for Hybrid GDN-MoE Models
In speculative decoding, parallel drafters such as DFlash make drafting cheap, so target verification dominates each round. A central choice is the verification length, the number of tokens that the target processes in each round.
arXiv - cs.AI2 min read