Llm Reasoning Benchmarks Are Statistically Fragile: New Study Shows Reinforcement Learning Rl Gains Often Fall Within Random Variance

Reasoning capabilities person go cardinal to advancements successful ample connection models, important successful starring AI systems developed by awesome investigation labs. Despite a surge successful investigation focused connected knowing and enhancing LLM reasoning abilities, important methodological challenges persist successful evaluating these capabilities accurately. The section faces increasing concerns regarding information rigor arsenic non-reproducible aliases inconclusive assessments consequence distorting technological understanding, misguiding take decisions, and skewing early investigation priorities. In nan quickly evolving scenery of LLM reasoning, wherever speedy publication cycles and benchmarking competitions are commonplace, methodological shortcuts tin silently undermine genuine progress. While reproducibility issues successful LLM evaluations person been documented, their continued presence—particularly successful reasoning tasks—demands heightened scrutiny and much stringent information standards to guarantee that reported advances bespeak genuine capabilities alternatively than artifacts of flawed appraisal methodologies.

Numerous approaches person emerged to heighten reasoning capabilities successful connection models, pinch supervised fine-tuning (SFT) and reinforcement learning (RL) being nan superior methods of interest. Recent innovations person expanded upon nan DeepSeek-R1 look done innovative RL algorithms for illustration LCPO, REINFORCE++, DAPO, and VinePPO. Researchers person besides conducted empirical studies exploring RL creation spaces, information scaling trends, curricula, and reward mechanisms. Despite these advancements, nan section faces important information challenges. Machine learning advancement often lacks rigorous assessment, pinch galore reported gains failing to clasp up erstwhile tested against well-tuned baselines. RL algorithms are peculiarly susceptible to variations successful implementation details, including random seeds, raising concerns astir nan reliability of benchmarking practices.

Motivated by inconsistent claims successful reasoning research, this study by researchers from Tübingen AI Center, University of Tübingen and  University of Cambridge conducts a rigorous investigation into mathematical reasoning benchmarks, revealing that galore caller empirical conclusions neglect nether observant re-evaluation. The study identifies astonishing sensitivity successful LLM reasoning pipelines to insignificant creation choices, including decoding parameters, punctual formatting, random seeds, and hardware configurations. Small benchmark sizes lend importantly to this instability, pinch azygous questions perchance shifting Pass@1 scores by complete 3 percent points connected datasets for illustration AIME’24 and AMC’23. This leads to double-digit capacity variations crossed seeds, undermining published results. The study systematically analyzes these instability sources and proposes champion practices for improving reproducibility and rigor successful reasoning evaluations, providing a standardized model for re-evaluating caller techniques nether much controlled conditions.

The study explores creation factors affecting reasoning capacity successful connection models done a standardized experimental framework. Nine wide utilized models crossed 1.5B and 7B parameter classes were evaluated, including DeepSeek-R1-Distill variants, DeepScaleR-1.5B, II-1.5 B-Preview, OpenRS models, S1.1-7B, and OpenThinker7B. Using accordant hardware (A100 GPU, AMD CPU) and package configurations, models were benchmarked connected AIME’24, AMC’23, and MATH500 datasets utilizing Pass@1 metrics. The study revealed important capacity variance crossed random seeds, pinch modular deviations ranging from 5 to 15 percent points. This instability is peculiarly pronounced successful smaller datasets wherever a azygous mobility tin displacement capacity by 2.5-3.3 percent points, making single-seed evaluations unreliable.

Based connected rigorous standardized evaluations, nan study reveals respective cardinal findings astir existent reasoning methodologies successful connection models. Most RL-trained variants of nan DeepSeek R1-Distill exemplary neglect to present meaningful capacity improvements, pinch only DeepScaleR demonstrating robust, important gains crossed benchmarks. While RL training tin substantially amended guidelines exemplary capacity erstwhile applied to models for illustration Qwen2.5, instruction tuning mostly remains superior, pinch Open Reasoner-Zero-7B being nan notable exception. In contrast, SFT consistently outperforms instruction-tuned baselines crossed each benchmarks and generalizes good to caller datasets for illustration AIME’25, highlighting its robustness arsenic a training paradigm. RL-trained models show pronounced capacity drops betwixt AIME’24 and nan much challenging AIME’25, indicating problematic overfitting to training distributions. Additional phenomena investigated see nan relationship betwixt consequence magnitude and accuracy, pinch longer responses consistently showing higher correction rates crossed each exemplary types.

This broad study reveals that evident advancement successful LLM-based reasoning has been built connected unstable foundations, pinch capacity metrics susceptible to insignificant variations successful information protocols. The investigation demonstrates that reinforcement learning approaches output humble improvements astatine champion and often grounds overfitting to circumstantial benchmarks, while supervised fine-tuning consistently delivers robust, generalizable capacity gains. To found much reliable appraisal standards, standardized information frameworks pinch Dockerized environments, seed-averaged metrics, and transparent protocols are essential. These findings item nan captious request for methodological rigor complete leaderboard title to guarantee that claimed advances successful reasoning capabilities bespeak genuine advancement alternatively than artifacts of inconsistent information practices.

Here is nan Paper, GitHub Page and Leaderboard. Also, don’t hide to travel america on Twitter and subordinate our Telegram Channel and LinkedIn Group. Don’t Forget to subordinate our 90k+ ML SubReddit.

🔥 [Register Now] miniCON Virtual Conference connected AGENTIC AI: FREE REGISTRATION + Certificate of Attendance + 4 Hour Short Event (May 21, 9 am- 1 p.m. PST) + Hands connected Workshop