{"id":1384,"date":"2025-10-06T18:15:00","date_gmt":"2025-10-06T18:15:00","guid":{"rendered":"https:\/\/scipapermill.com\/index.php\/2025\/10\/06\/llm_math-rl_optimized-breakthroughs-a-digest-of-recent-advancements-in-mathematical-reasoning-for-large-language-models\/"},"modified":"2025-12-28T22:00:48","modified_gmt":"2025-12-28T22:00:48","slug":"llm_math-rl_optimized-breakthroughs-a-digest-of-recent-advancements-in-mathematical-reasoning-for-large-language-models","status":"publish","type":"post","link":"https:\/\/scipapermill.com\/index.php\/2025\/10\/06\/llm_math-rl_optimized-breakthroughs-a-digest-of-recent-advancements-in-mathematical-reasoning-for-large-language-models\/","title":{"rendered":"$$LLM_{Math} + RL_{Optimized} = Breakthroughs$$: A Digest of Recent Advancements in Mathematical Reasoning for Large Language Models"},"content":{"rendered":"

Latest 50 papers on mathematical reasoning: Oct. 6, 2025<\/h3>\n

L<\/em>L<\/em>M<\/em>M<\/em>a<\/em>t<\/em>h<\/em><\/sub>\u2005+\u2005R<\/em>L<\/em>O<\/em>p<\/em>t<\/em>i<\/em>m<\/em>i<\/em>z<\/em>e<\/em>d<\/em><\/sub>\u2004=\u2004B<\/em>r<\/em>e<\/em>a<\/em>k<\/em>t<\/em>h<\/em>r<\/em>o<\/em>u<\/em>g<\/em>h<\/em>s<\/em><\/span>: A Digest of Recent Advancements in Mathematical Reasoning for Large Language Models<\/h2>\n

Mathematical reasoning has long been a formidable frontier for artificial intelligence, demanding not just factual recall but also complex logical inference, multi-step problem-solving, and robust generalization. Large Language Models (LLMs), despite their impressive capabilities, often struggle with the precise and systematic nature of mathematics. This challenge has fueled a surge in innovative research, particularly at the intersection of reinforcement learning (RL) and advanced model architectures. This post dives into a collection of recent papers that are pushing the boundaries of what LLMs can achieve in mathematical reasoning, from enhancing training stability and efficiency to developing novel evaluation benchmarks.<\/p>\n

The Big Idea(s) & Core Innovations:<\/h3>\n

The overarching theme uniting this research is the quest for more robust, efficient, and interpretable mathematical reasoning in LLMs, often by leveraging advanced RL techniques and novel architectural designs. A significant challenge in applying RL to reasoning tasks, as highlighted by Phuc Minh Nguyen et al.\u00a0from VinUniversity<\/strong> in their paper, \u201cThe Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models<\/a>\u201d, is the paradoxical shrinkage of the reasoning boundary due to negative interference and \u2018winner-take-all\u2019 phenomena. Their proposed SELF algorithm<\/strong> offers a solution by curating data to focus on low-likelihood problems, mitigating coverage shrinkage.<\/p>\n

Building on this, several papers introduce frameworks to enhance exploration and guidance. Xiaoyang Yuan et al.\u00a0from Tongji University<\/strong> introduce AMPO<\/strong> in \u201cMore Than One Teacher: Adaptive Multi-Guidance Policy Optimization for Diverse Exploration<\/a>\u201d, which uses multiple teacher models and an adaptive \u2018guidance-on-demand\u2019 mechanism to boost reasoning diversity and performance, particularly in out-of-distribution tasks. This idea of guided exploration resonates with \u201cEAPO: Enhancing Policy Optimization with On-Demand Expert Assistance<\/a>\u201d by Siyao Song et al.\u00a0from ByteDance BandAI<\/strong>, where expert consultation is a learnable action, allowing models to internalize expertise over time.<\/p>\n

Another critical innovation focuses on structured reasoning and planning. Zhihao Dou et al.\u00a0from Case Western Reserve University<\/strong> in \u201cPlan Then Action: High-Level Planning Guidance Reinforcement Learning for LLM Reasoning<\/a>\u201d propose PTA-GRPO<\/strong>, a two-stage framework that integrates high-level planning with fine-grained Chain-of-Thought (CoT) reasoning. This planning-first approach is echoed in Shihao Qi et al.\u2019s<\/strong> \u201cPlan before Solving: Problem-Aware Strategy Routing for Mathematical Reasoning with LLMs<\/a>\u201d from Xi\u2019an Jiaotong University<\/strong>, which introduces PRISM<\/strong> for dynamically routing to optimal strategies based on problem characteristics. Similarly, Yingqian Cui et al.\u00a0from Michigan State University<\/strong>, with Amazon<\/strong> and Pennsylvania State University<\/strong>, introduce DREAM<\/strong> in \u201cAdaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search<\/a>\u201d, separating reasoning into planning and execution with dynamic budget allocation for enhanced efficiency and accuracy.<\/p>\n

The challenge of instability and entropy collapse in RL is addressed by several works. Tao Ren et al.\u00a0from Peking University<\/strong> present RiskPO<\/strong> in \u201cRiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Training<\/a>\u201d, a risk-sensitive RL framework that mitigates entropy collapse by amplifying gradient signals on challenging instances. This is complemented by Yuhua Jiang et al.\u00a0from Tsinghua University<\/strong> in \u201cRisk-Sensitive RL for Alleviating Exploration Dilemmas in Large Language Models<\/a>\u201d, introducing RS-GRPO<\/strong> to improve Pass@k performance through dynamic re-weighting of optimization, emphasizing hard prompts. Further, Zhenpeng Su et al.\u00a0from Kuaishou Technology<\/strong>\u2019s CE-GPPO<\/strong> in \u201cCE-GPPO: Controlling Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learning<\/a>\u201d offers fine-grained control over policy entropy by managing gradients from clipped tokens, balancing exploration and exploitation.<\/p>\n

Efficiency in training and inference is another crucial focus. Ziniu Li et al.\u00a0from ByteDance Seed<\/strong> introduce \u201cKnapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation<\/a>\u201d, which dynamically allocates exploration budgets to tasks based on their learning potential, significantly improving gradient effectiveness. Dongqi Zheng from Purdue University<\/strong> presents ARS<\/strong> in \u201cARS: Adaptive Reasoning Suppression for Efficient Large Reasoning Language Models<\/a>\u201d, a training-free method that suppresses redundant reasoning steps, achieving significant reductions in token usage and latency without sacrificing accuracy. For multimodal tasks, Jiwan Chung et al.\u00a0from Yonsei University and Seoul National University<\/strong> introduce v1<\/strong> in \u201cv1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning<\/a>\u201d, a lightweight extension that enables MLLMs to dynamically reference visual information using a point-and-copy mechanism, enhancing grounded reasoning.<\/p>\n

Finally, the very definition and evaluation of mathematical reasoning are being refined. Jiayi Kuang et al.\u00a0from Sun Yat-sen University<\/strong> introduce \u201cAtomic Thinking of LLMs: Decoupling and Exploring Mathematical Reasoning Abilities<\/a>\u201d, a framework that breaks down complex abilities into atomic components, revealing strengths in algebra but weaknesses in geometry. This granular approach to evaluation is supported by new benchmarks like SKYLENAGE<\/strong> from Hu Wei et al.\u00a0at Alibaba Group<\/strong> in \u201cSKYLENAGE Technical Report: Mathematical Reasoning and Contest-Innovation Benchmarks for Multi-Level Math Evaluation<\/a>\u201d, and EEFSUVA<\/strong> by Nicole N. Khatibi et al.<\/strong> in \u201cEEFSUVA: A New Mathematical Olympiad Benchmark<\/a>\u201d, which focuses on challenging problems from Eastern European competitions to counter data contamination.<\/p>\n

Under the Hood: Models, Datasets, & Benchmarks:<\/h3>\n

Recent advancements are underpinned by innovative models, specialized datasets, and rigorous benchmarks that push the boundaries of LLM capabilities. Here\u2019s a closer look:<\/p>\n