How to Use Parallel Data for Multilingual Pretraining
High-quality human-translated parallel corpora can improve cross-lingual transfer, but are typically too small for large-scale pretraining. Machine-translated web corpora help overcome this limitation, making parallel data available at pretraining scale.
By Zhihong Huang, Yao Lu, Derek F. Wong, Hamid Alinejad-Rokny, Min YangarXiv - cs.CL (NLP)
Read the full story
Originally published by arXiv - cs.CL (NLP) on Oct 22, 2026.
Comments
Sign in to join the discussion.