Optimal Multitask Representations Under a Rate Constraint: An Information Bottleneck Analysis
Which pretraining objective yields the best representation for downstream tasks under an information budget? We formalize this problem as a multitask information bottleneck, using a mutual information constraint to model incomplete information retention.
By Nicholas Stiles, Gregory W. WornellarXiv cs.LG
Read the full story
Originally published by arXiv cs.LG on Oct 23, 2026.
Comments
Sign in to join the discussion.