ASSET: Adaptive Speech Segmentation via Reinforcement Learning
Speech large language models (LLMs) are often built using pretrained acoustic encoders as an audio interface to an LLM decoder. However, LLMs scale poorly with input sequence length, while acoustic encoders tend to produce substantially longer sequences than typical textual inputs.
arXiv - cs.CL (NLP)2 min read