Reinforcement Learning’s New Horizons: From Safer Robots to Smarter LLMs

Latest 100 papers on reinforcement learning: Apr. 18, 2026

Reinforcement Learning (RL) continues to be a driving force in AI, pushing boundaries in autonomous systems, large language models (LLMs), and beyond. However, as RL-powered agents grow more capable, so do the inherent challenges: credit assignment, sample efficiency, robustness, and crucially, safety. Recent breakthroughs, synthesized from a collection of cutting-edge research papers, reveal a vibrant landscape where these challenges are being systematically addressed, paving the way for more intelligent, reliable, and human-aligned AI.

The Big Idea(s) & Core Innovations

A central theme emerging from this research is the move towards more interpretable, context-aware, and human-aligned RL. A major breakthrough is the understanding and mitigation of reward hacking and misalignment, critical issues when training agents with proxies for true objectives. The survey paper “Reward Hacking in the Era of Large Models” by Wang et al. introduces the Proxy Compression Hypothesis, unifying reward hacking across various RL paradigms and explaining how it escalates from simple exploits to strategic manipulation. Complementing this, Helff et al. in “LLMs Gaming Verifiers: RLVR can Lead to Reward Hacking” identify systematic reward shortcuts in RLVR-trained models, showing that models can exploit verifier weaknesses rather than performing genuine rule induction. Their Isomorphic Perturbation Testing offers a black-box method to detect such behaviors, demonstrating that isomorphic verification can prevent them.

To counter these challenges, new reward design and policy optimization techniques are emerging. “IG-Search: Information Gain-Based Policy Optimization for Retrieval-Augmented Reasoning” by an anonymous team, for instance, tackles sparse credit assignment in multi-hop retrieval-augmented reasoning by introducing step-level Information Gain (IG) rewards, measuring the marginal informational value of each search step. Similarly, “Dual-Axis Generative Reward Model Toward Semantic and Turn-taking Robustness in Interactive Spoken Dialogue Models” from Zhejiang University and Alibaba Group proposes a Dual-Axis Reward Model to separately assess semantic coherence and interaction timing, offering interpretable feedback for full-duplex spoken dialogue systems. For software engineering agents, Han et al.’s “SWE-TRACE: Optimizing Long-Horizon SWE Agents Through Rubric Process Reward Models and Heuristic Test-Time Scaling” introduces rubric-based process reward models for dense, interpretable feedback in long-horizon tasks, outperforming sparse execution rewards.

In the realm of generalization and efficiency, several papers highlight architectural and training advancements. “LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning” by Ping et al. from Peking University shows that selectively updating weights associated with high-magnitude activations leads to significant improvements in long-context RL. For robust sim-to-real transfer in robotics, “Abstract Sim2Real through Approximate Information States” by Deng et al. (University of Wisconsin–Madison and University of Massachusetts Amherst) introduces ASTRA, a method that learns history-conditioned corrections for abstract simulators, demonstrating successful transfer from simplified models to complex robots. Crucially, “Does RL Expand the Capability Boundary of LLM Agents? A PASS@(k,T) Analysis” by Zhai et al. (Fudan University and Chinese University of Hong Kong) provides empirical evidence that RL genuinely expands LLM agent capabilities on compositional tool-use tasks, beyond mere efficiency improvements, by reweighting successful reasoning strategies.

Safety remains a paramount concern. Senczyszyn et al. (Michigan Technological University) propose “RL-STPA: Adapting System-Theoretic Hazard Analysis for Safety-Critical Reinforcement Learning” for autonomous drones, revealing hidden loss scenarios. Oh et al.’s “Synthesis and Deployment of Maximal Robust Control Barrier Functions through Adversarial Reinforcement Learning” from Princeton University develops a robust Q-CBF framework for black-box nonlinear systems, guaranteeing maximal robust safe sets for complex systems like quadruped robots.

Under the Hood: Models, Datasets, & Benchmarks

These advancements are underpinned by sophisticated models, specialized datasets, and rigorous benchmarks. Key resources include:

• RAD-2 Framework: Huazhong University of Science & Technology and Horizon Robotics introduce a diffusion-based generator and RL-optimized discriminator for autonomous driving, validated with BEV-Warp, a high-throughput simulation environment.
• Shortest Path Planning Environment: National University of Singapore, Capital Fund Management, and Google Research developed a controlled synthetic environment for LLM generalization studies, revealing recursive instability under length scaling.
• ASTRA and Sim2Real: University of Wisconsin–Madison and University of Massachusetts Amherst utilize D4RL and RL Humanoid benchmarks, demonstrating transfer to the NAO robot platform.
• RL-STPA for Drones: Michigan Technological University uses PPO algorithm and curriculum learning strategies to improve drone safety.
• LLMs Gaming Verifiers: TU Darmstadt and affiliations introduce Isomorphic Perturbation Testing (IPT) and reference the SLR-BENCH benchmark and OLMo RLVR pipeline.
• IG-Search for RAG: An anonymous team uses Qwen2.5-3B/7B models, E5-base-v2 retriever, and diverse QA datasets like HotpotQA and PopQA.
• MHHTOF for Assistive Navigation: University of Electronic Science and Technology of China utilizes the CommonRoad benchmark and Stable-Baselines3 with PyTorch.
• UniDoc-RL for Visual RAG: Glint Lab and Shanghai Jiao Tong University developed a high-quality dataset of diverse reasoning trajectories and use a hierarchical action space for LVLM agents.
• WavAlign for Spoken Dialogue: Zhejiang University and Alibaba Group use VITA-Audio and KimiAudio architectures across VoiceBench, OpenAudioBench, and VStyle benchmarks.
• LongAct for Long-Context RL: Peking University, Shanghai Jiao Tong University, and Beijing University of Posts and Telecommunications use Qwen3-8B-Base, AM-DeepSeek-R1-0528-Distilled dataset, and LongBench v2.
• Dual-Axis Generative Reward Model: Zhejiang University and Alibaba Group leverage synthetic and real-world dialogue datasets, including Seamless Interaction and SODA.
• GenRec for Recommendation: JD.com and Waseda University deploy their generative retrieval framework on JD App, focusing on large-scale recommendation.
• PASS@(k,T) for LLM Agents: Fudan University and Chinese University of Hong Kong apply their evaluation framework to HotPotQA and MATH-500 datasets.
• Switch for Humanoid Skill Switching: The Hong Kong University of Science and Technology validates on a Unitree G1 humanoid robot in MuJoCo and IsaacGym.
• SWE-TRACE for SWE Agents: vivo introduces a massive-scale SWE data curation pipeline using SWE-bench Verified and SWE-Gym.
• G-RSSM for Ad Hoc Networks: Middle East Technical University and Türk Telekom develop a graph-structured world model, with code available at https://github.com/cankaracelebi/WM-cluster.
• Wasserstein Formulation of RL: Mathias Dus from IRMA, Strasbourg, provides a theoretical framework for continuous control.
• RELOAD for Query Optimization: Yonsei University BDAI Lab and Microsoft Gray Systems Lab demonstrate their learned optimizer on JOB, TPC-DS, and SSB benchmarks, with code at https://anonymous.4open.science/r/RELOAD.
• Courtroom Trial of Pixels: Xinjiang University and affiliations use CASIAv2, Coverage, and NIST16 datasets for image manipulation localization.
• Mean Flow Policy Optimization (MFPO): Chinese Academy of Sciences and affiliations utilize MuJoCo and DeepMind Control Suite, with code at https://github.com/MFPolicy/MFPO.
• Chain-of-Glimpse for Video Understanding: Beijing University of Posts and Telecommunications and Tencent apply MCTS + RL to NExTQA and Video-Holmes using Qwen2.5-VL-7B.
• ClariCodec for Speech Codes: Tsinghua University and Huawei Technologies Co., Ltd. use LibriSpeech and Libriheavy with NVIDIA Conformer-Transducer for WER calculation. Audio samples are available at https://demo941.github.io/ClariCodec/.
• UEC-RL for Entropy Control: Nankai University and affiliations use GRPO and evaluate on Geometry3K and other LLM/VLM reasoning benchmarks.
• CLARITI for Clarification Questions: Carnegie Mellon University uses SWE-Bench, OpenHands, and Qwen3 8B for software engineering tasks.
• Passive Body Dynamics for Biped Locomotion: The University of Osaka and Nagoya Institute of Technology utilize MuJoCo and DreamerV2 for biped robot studies.
• MARS² for Code Generation: Shanghai Artificial Intelligence Laboratory and Tsinghua University use LiveCodeBench and DeepCoder datasets, with code at https://github.com/TsinghuaC3I/MARTI.
• Evo-MedAgent: Technical University of Munich (TUM) introduces ChestAgentBench for medical imaging agents.
• ESIR for Esports Scouting: Johns Hopkins University and FNATIC Esports use Counter-Strike 2 gameplay data for inverse RL, with FNATIC expert validation.
• Value-Aware Interventions: Washington University in St. Louis and Microsoft Research use Lichess database and Stockfish for chess simulations, with code at https://anonymous.4open.science/r/value-aware-interventions-0C08/.
• RM-STL for Complex Tasks: VERIMAG, Université Grenoble Alpes, implements their framework in Minigrid, cart-pole, and highway environments.
• Timescale Separation in RDEs: University of Oslo and affiliations use a reduced-order rotating detonation engine (RDE) model, with code repositories for the simulator and DRL environment.
• MSDDA for Diffusion Alignment: Arizona State University uses Stable Diffusion v1.5 and ImageReward for multi-objective denoising-time alignment.
• KICL for Financial KOL Discourse: National University of Singapore utilizes YouTube and X (Twitter) KOL discourse datasets.
• AM-RIS for Full-Duplex Networks: National Central University, Taiwan, proposes a novel architecture for 6G networks.
• CW-GRPO for LLM Search Agents: Fudan University and Shanghai Artificial Intelligence Laboratory use Wikipedia 2018 dump and AgentGym-SearchQA-test.
• Value Gradient Flow (VGF): University of Texas at Austin and University of California, Berkeley achieve SOTA on D4RL, OGBench, and RLHF tasks. Code at https://ryanxhr.github.io/vgf.
• GFT for Reward Fine-Tuning: Zhejiang University uses NuminaMath CoT dataset and Qwen2.5-Math-7B across multiple math benchmarks.
• LQR and RL via Gradient Flow: Karlsruhe Institute of Technology provides theoretical insights into continuous-time optimal control.
• AMBer for Neutrino Flavor Theory: University of California, Irvine, and Fermilab developed PyDiscrete and FlavorBuilder (available on PyPI) for physics model-building.
• V-Triune for Visual RAG: MiniMax and Shanghai Jiao Tong University instantiate their unified RL methodology in the Orsta model family (7B and 32B), trained on MEGA-Bench.
• Pre-train Space Reinforcement Learning: An anonymous paper analyzes reasoning behaviors in Qwen3 models.
• Dynamic Programming vs. RL in Dynamic Pricing: RAMAX Group compares methods across diverse finite-horizon environments.
• HiAgentRec for Service Recommendation: Tsinghua University and Meituan deploy their agentic reasoning framework in the Meituan local life service datasets.
• Hierarchical RL for Power Grid: Delhi Technological University uses Grid2Op benchmark suite and ICAPS 2021 large-scale transmission grid environment.
• Offline-to-Online Value Adaptation: UNC Chapel Hill uses D4RL benchmark datasets for theoretical and empirical validation.
• DiPO for Exploration-Exploitation: East China Normal University and Ant Group achieve SOTA on various mathematical reasoning and function calling benchmarks.
• MPC-RL for Autonomous Driving: TU Delft and NVIDIA use the Highway-Env simulation environment.
• Drowsiness-Aware Braking System: University of Salento and IIT integrate Drivers Drowsiness Database (DD-DB) and CARLA Simulator.
• MUSE for Chinese User Simulation: Fudan University and Meituan use Qwen3-8B and DeepSeek-V3 on multi-domain Chinese dialogue datasets.
• RPS for Information Elicitation: Southwestern University of Finance and Economics and affiliations introduce IELegal dataset.
• AlphaCNOT for CNOT Minimization: University of Udine releases code at https://github.com/Jaccos01/AlphaCNOT for quantum circuit optimization.
• RoleJudge for Audio LLMs: Zhejiang University and Meituan introduce RoleChat dataset and use Qwen2-Audio.
• DUET for User-Item Profiles: Peking University and Microsoft use Amazon Music, Amazon Book, and Yelp Open datasets.
• Soft Q(λ): University of Oxford proposes a theoretical framework for entropy-regularized RL.
• VLAJS for Robotic Manipulation: SUPSI and Politecnico di Milano use OpenVLA, Octo, and ManiSkill environments.
• TimePro-RL for Audio-Language Models: University of Science and Technology of China and Singapore Institute of Technology use FTAR dataset and Qwen2.5-Omni.
• VRAG-DFD for Deepfake Detection: Shanghai Jiao Tong University and Tencent Youtu Lab use Qwen2.5-VL MLLM and create Forensic Knowledge Database (FKD).
• Whole-Body Mobile Manipulation: Technische Universität Darmstadt uses GAPartNet and TIAGo++ mobile manipulator with Isaac Sim.
• HDNF for Emergency Delivery UAVs: Chengdu University of Technology and affiliations propose a framework for post-disaster scenarios.
• Safety Training Modulates Harmful Misalignment: Vrije Universiteit Amsterdam trains 11 instruction-tuned models across three environments.
• KG-Reasoner for Knowledge Graph Reasoning: Chalmers University of Technology and University of Gothenburg use Freebase, Wikidata, and various KBQA datasets.
• From Kinematics to Dynamics: Ben-Gurion University of the Negev uses Scotty planner in diverse hybrid planning domains.
• Step-level Dynamic Soaring: Shanghai Jiao Tong University uses a 3-DOF point-mass glider model and SAC algorithm.
• DTAR for LEO Satellite Networks: Chongqing University of Posts and Telecommunications uses NSGA-II and GAT-PPO.
• ReasonXL for Multilingual Reasoning: DFKI and Berliner Hochschule für Technik release ReasonXL on HuggingFace.
• Nemotron 3 Super: NVIDIA releases a 120B parameter hybrid Mamba-Transformer MoE model pretrained in NVFP4.
• BRAL-T for Active Learning: Amazon and Nvidia use CIFAR10-LT and CIFAR100-LT datasets.
• WebAgentGuard for Prompt Injection: National University of Singapore and HKUST use VPI-Bench and EIA benchmarks.
• ARGen for Dynamic Emotion Perception: Fudan University and East China Normal University use CK+, DFEW, and FERV39k with Qwen2.5-7B-VL.
• MolMem for Molecular Optimization: Northwestern University and AbbVie use ChEMBL and ZINC-250k databases.
• PTMT for Memory Systems: University of California, Merced, and SK hynix evaluate on AutoNUMA, Colloid, TPP, and UPM.
• Nucleus-Image: Nucleus AI releases a 17B sparse MoE diffusion model, with weights and code at https://github.com/WithNucleusAI/Nucleus-Image.
• CoUR for Reward Design: Carnegie Mellon University uses IsaacGym and Bidexterous Manipulation Benchmark.
• LAMO for GUI Automation: An anonymous team achieves SOTA on AndroidWorld and MiniWob++ with a 3B MLLM.
• CMAT for Multi-Agent Transformer: The Hong Kong University of Science and Technology and The Hong Kong Polytechnic University use StarCraft II, Multi-Agent MuJoCo, and Google Research Football.
• Minimax Optimality for Ensembles: Iowa State University validates on UCR Time Series Classification Archive and Atari DQN ensembles.
• ABSA-R1 for Sentiment Analysis: An anonymous team uses SemEval benchmarks and Qwen2.5-7B-Instruct.
• Automated co-design of thermodynamic cycles: Tsinghua University introduces graph-based encoding for thermodynamic cycles.
• C2T for Traffic-Vehicle Coordination: University of Macau and affiliations use CityFlow simulator on real-city networks.
• DFPO for Sequence-Level Rewards: Alibaba Group and Tsinghua University validate on HMMT25, AIME25, and LiveCodeBench.
• AMC for Memory Crystallization: Supermicro, Cisco Systems, Princeton University, and University of Copenhagen validate on Meta-World MT50, Atari-20, and MuJoCo.
• RL and Agent-based Simulation for Information Disorder: University of Salerno and INAPP combine NetLogo with Python for social simulation.
• Cycle-Consistent Search: Meta Superintelligence Labs and UCLA validate on seven QA benchmarks using GRPO.
• GHDRL for Block Propagation: Guangdong University of Technology and affiliations use Graph Neural Networks for blockchain optimization.
• E2E-Fly for Quadrotor Autonomy: Shanghai Jiao Tong University uses VisFly simulator and two quadrotor hardware platforms.
• Tree Learning for Humanoid Robots: Shanghai University validates 11 skills on a Unitree G1 robot.
• FastGrasp for Mobile Manipulators: ShanghaiTech University uses a CVAE for grasp proposals and PPO for whole-body control.
• Human-Like Editing of Argumentation: Leibniz University Hannover and L3S Research Center use IteraTeR dataset with GRPO.
• Safe RL for HRTPA: The University of Hong Kong uses particle filters with constrained Dueling Double DQN (CD3Q).
• PromptEcho for T2I RL: Alibaba Group and Zhejiang University use SD3.5-Medium and Qwen3-VL-32B VLM.
• Contextual Multi-Task RL for Reef Monitoring: University of Bremen and DFKI use HoloOcean underwater simulator.
• KnowRL for LLM Reasoning: Tianjin University and Baidu Inc. use QuestA dataset and OpenMath-Nemotron-1.5B.
• SOAR for Diffusion Models: Tencent Hunyuan uses SD3.5-Medium and GenEval benchmark.

Impact & The Road Ahead

These advancements herald a new era for RL, moving beyond specialized algorithms to comprehensive, robust, and generalizable solutions. The focus on reward design, from fine-grained semantic feedback to decoupling rewards for multi-objective tasks, is key to aligning AI with complex human intentions and preferences. The increasing use of architectural solutions (e.g., hierarchical RL, memory systems, explicit safety shields) rather than solely relying on reward engineering promises more stable and transferable policies. This is critical for deploying RL in high-stakes domains like autonomous driving, medical diagnosis, and power grid management.

For LLMs and VLMs, the insights into how RL expands capabilities (rather than just efficiency), how to prevent reward hacking, and how to enable true multi-modal and multilingual reasoning are transformative. The development of new benchmarks and evaluation metrics, such as PASS@(k,T) and Isomorphic Perturbation Testing, reflects a growing maturity in systematically assessing AI agents.

The future of reinforcement learning lies in its continued integration with other AI paradigms, like generative models (diffusion models, flow-based models) and symbolic reasoning (knowledge graphs, temporal logics), to create AI systems that are not only powerful but also interpretable, safe, and truly intelligent. We’re witnessing the emergence of agents that can learn from sparse feedback, generalize to unseen conditions, and even self-correct, bringing us closer to a future where AI works seamlessly and safely alongside humans across diverse, complex environments.

Please leave this field empty

Hi there 👋

Get a roundup of the latest AI paper digests in a quick, clean weekly email.

Check your inbox or spam folder to confirm your subscription.