Local Prototype Reconstruction for Text-Compatible Speech-to-LLM Bridge Pretraining
Speech-to-LLM systems often connect a frozen speech encoder to a frozen large language model (LLM) through a small trainable bridge. The bridge is usually treated as plumbing, but it in fact defines the geometry of the speech-to-LLM interface, and the pretraining objective decides whether that…
arXiv - cs.CL (NLP)2 min read