FAD-KV: Adaptive Dictionary KV Caching for Efficient Reasoning Decoding
Key-value (KV) caching is a core mechanism in large language models, where intermediate key and value representations are stored to accelerate autoregressive decoding.
arXiv - cs.CL (NLP)2 min read