SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model | #ai #2025 #genai #google

Paper: https://arxiv.org/pdf/2501.17161 This research paper compares supervised fine-tuning (SFT) and reinforcement learning (RL) for post-training foundation models. Using novel and existing tasks involving arithmetic and spatial reasoning, the study finds that RL promotes better generalization to unseen data, unlike SFT which tends to memorize training data. Further analysis reveals RL enhances visual recognition capabilities in multimodal models, while SFT aids in stabilizing RL training by improving output formatting. The paper also explores the impact of increased inference-time computation on generalization. #ai, #artificialintelligence, #arxiv, #research, #paper, #publication, #llm, #genai, #generativeai, #largevisualmodels, #largelanguagemodels, #largemultimodalmodels, #nlp, #text, #machinelearning, #ml, #nvidia, #openai, #anthropic, #microsoft, #google, #technology, #cuttingedge, #meta, #llama, #chatgpt, #gpt, #elonmusk, #samaltman, #deployment, #engineering, #scholar, #science, #apple, #samsung, #turing, #aiethics, #innovation, #futuretech, #deeplearning, #datascience, #computervision, #autonomoussystems, #robotics, .css-j9qmi7{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;font-weight:700;margin-bottom:1rem;margin-top:2.8rem;width:100%;-webkit-box-pack:start;-ms-flex-pack:start;-webkit-justify-content:start;justify-content:start;padding-left:5rem;}@media only screen and (max-width: 599px){.css-j9qmi7{padding-left:0;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;}}.css-j9qmi7 svg{fill:#27292D;}.css-j9qmi7 .eagfbvw0{-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;color:#27292D;}