Learning Style, Forgetting Semantics: A Case Study of SFT and RFT on Classification Tasks
Why does supervised fine-tuning (SFT) lead to more forgetting than reinforcement fine-tuning (RFT), even when all teacher demonstrations are semantically correct?