Bi-Phase Training: Learning Efficiently From Scratch
Pre-training foundational models is computationally expensive as models scale. In this work, we introduce Bi-Phase Training (BPT), a parameter-efficient pre-training method that preserves the high-rank learning dynamics observed in full-parameter optimization with substantially reduced trainable…
By Abel Gurung, Joseph CampbellarXiv cs.LG
Read the full story
Originally published by arXiv cs.LG on Oct 30, 2026.
Comments
Sign in to join the discussion.