Sample Efficiency Unleashed: Breakthroughs in LLMs, Robotics, and Beyond

Latest 50 papers on sample efficiency: Dec. 21, 2025

The quest for sample efficiency—the ability of AI models to learn robustly from minimal data—remains a cornerstone challenge across machine learning. In an era where data acquisition can be costly, time-consuming, or even unsafe, breakthroughs in this area are paramount for unlocking the full potential of AI in real-world applications. This digest dives into a fascinating collection of recent research, showcasing innovative techniques that are pushing the boundaries of sample efficiency, from enhancing large language model (LLM) reasoning to enabling more agile robot control and accelerating scientific discovery.

The Big Idea(s) & Core Innovations

Many of the recent advancements converge on ingenious ways to either make better use of existing data or to generate more effective synthetic data. For instance, in the realm of LLMs, the paper Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning from Johns Hopkins University introduces GAR, an adversarial reinforcement learning framework that significantly boosts mathematical reasoning. GAR employs adversarial training between a reasoner and a discriminator, enhancing reward calibration and dramatically improving sample efficiency by providing structured, step-level feedback. Similarly, Reflective Preference Optimization (RPO): Enhancing On-Policy Alignment via Hint-Guided Reflection by Zihui Zhao and Zechang Li (Tsinghua University, Alibaba Group) augments Direct Preference Optimization (DPO) with hint-guided self-reflection, amplifying preference signals and achieving state-of-the-art hallucination mitigation with fewer training steps.

Further demonstrating the power of self-supervised and self-distillation techniques, Purbesh Mitra and Sennur Ulukus from the University of Maryland introduce Semantic Soft Bootstrapping: Long Context Reasoning in LLMs without Reinforcement Learning. SSB, an RL-free self-distillation method, enables LLMs to improve long-context reasoning by leveraging their own reasoning as both teacher and student, bypassing the complexities of RL and achieving impressive accuracy gains on benchmarks like MATH500 and AIME2024. This avoids common pitfalls like reward hacking, a crucial insight into more stable LLM training.

In reinforcement learning, a dominant theme is the clever integration of model-based approaches and exploration strategies. The University of Exeter team (Ashish Sundar et al.) in Enter the Void – Planning to Seek Entropy When Reward is Scarce proposes an anticipatory exploration strategy using world models and entropy-driven state discovery, outperforming traditional curiosity-driven methods, especially in sparse-reward environments. Complementing this, Double Horizon Model-Based Policy Optimization by Akihiro Kubo et al. (Advanced Telecommunications Research Institute, Kyoto University) introduces DHMBPO, which uses two distinct rollout horizons to balance distribution shift, model bias, and gradient instability, leading to superior sample efficiency in continuous control. For tasks with history-dependent rewards, the work from Sapienza University of Rome (Alessandro Trapasso et al.) in Model-Based Reinforcement Learning in Discrete-Action Non-Markovian Reward Decision Processes factorizes Markovian transition learning from non-Markovian reward handling, offering PAC guarantees and reduced sample complexity, even extending to continuous states with SimHash.

Robotics research also sees significant strides in sample efficiency, often by exploiting inherent symmetries or leveraging large datasets. mimic robotics, Microsoft Zurich, and ETH Zurich (Liam Achenbach et al.) introduce mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs, a video-action model enabling generalizable robot control with minimal fine-tuning by using large-scale robotics data. Meanwhile, Peking University and BeingBeyond (Chuan Mao et al.) in Universal Dexterous Functional Grasping via Demonstration-Editing Reinforcement Learning present DemoFunGrasp, a framework for dexterous functional grasping that leverages demonstration-editing and multi-task learning for improved sample efficiency and robust sim-to-real transfer. HK PolyU and LimX Dynamics (Mingqi Yuan et al.) propose PvP in PvP: Data-Efficient Humanoid Robot Learning with Proprioceptive-Privileged Contrastive Representations, a contrastive learning framework that uses proprioceptive and privileged state representations to enhance policy learning for humanoid robots, minimizing the need for hand-crafted data augmentations.

Furthermore, the integration of quantum computing promises a future of extreme sample efficiency. The paper Quantum Bayesian Optimization for Quality Improvement in Fuselage Assembly by Jiayu Liu et al. (Rensselaer Polytechnic Institute, University at Albany) proposes a Quantum Bayesian Optimization (QBO) framework, demonstrating superior sample efficiency for tasks like fuselage assembly by leveraging quantum algorithms to reduce the number of required samples compared to classical methods.

Under the Hood: Models, Datasets, & Benchmarks

The innovations highlighted are powered by a combination of novel models, smart use of existing benchmarks, and the introduction of new resources:

• Generative Adversarial Reasoner (GAR): A novel framework using adversarial training for improved reward calibration and mathematical reasoning, tested on AIME24 and LiveMathBench-Hard.
• DHMBPO: A model-based RL algorithm combining ‘distribution rollouts’ and ‘training rollouts’ for continuous control, outperforming existing methods like MACURA.
• EUBRL: A Bayesian RL algorithm leveraging epistemic uncertainty for exploration, proven nearly minimax-optimal in infinite-horizon discounted MDPs.
• mimic-video: A video-action model framework for generalizable robot control, trained on large-scale robotics datasets and available with code at https://github.com/mimic-robotics/mimic-video.
• DemoFunGrasp: A universal RL framework for dexterous functional grasping, incorporating vision-language models and demonstration-editing for sim-to-real transfer (https://beingbeyond.github.io/DemoFunGrasp/).
• PvP (Proprioceptive-Privileged contrastive learning): A contrastive learning framework for humanoid robots, evaluated with the new SRL4Humanoid open-source framework (https://github.com/LimX-Dynamics/SRL4Humanoid).
• TreeGRPO: A tree-structured RL framework for fine-tuning visual generative models, reinterpreting denoising as a search tree for efficient exploration (https://treegrpo.github.io/).
• MPDiffuser: A model-based diffusion framework for offline decision-making and predictive control, achieving consistent gains on D4RL and DSRL benchmarks (https://anonymous.4open.science/status/MPD-Submission-126B).
• SQDF (Soft Q-based Diffusion Finetuning): A KL-regularized RL method for diffusion alignment, using consistency models and off-policy replay buffers for text-to-image and black-box optimization tasks (https://github.com/Shin-woocheol/SQDF).
• QR-MAX/BUCKET-QR-MAX: Model-based algorithms for discrete-action Non-Markovian Reward Decision Processes (NMRDPs) with PAC guarantees, extended to continuous spaces via SimHash (https://github.com/Alee08/).
• KAN-Dreamer: Benchmarking Kolmogorov-Arnold Networks (KANs) as function approximators within the DreamerV3 framework, demonstrating FastKAN’s parity with MLPs in sample efficiency (https://github.com/Blealtan/efficient-kan).
• PINS-CAD: A physics-informed self-supervised learning framework pre-training graph neural networks on 200,000 synthetic coronary artery digital twins to predict cardiovascular events (https://arxiv.org/pdf/2512.03055).
• R-AutoEval+: An autoevaluation framework with finite-sample reliability guarantees for model evaluation, dynamically tuning reliance on synthetic data for efficiency (https://github.com/kclip/R_AutoEval_plus).
• SuRe (Surprise-prioritised Replay): A continual learning method for LLMs, utilizing surprise-based selection and dual-learner LoRA adapters to mitigate catastrophic forgetting in the Large Number of Tasks (LNT) setting (https://arxiv.org/pdf/2511.22367).

Impact & The Road Ahead

These advancements herald a new era of AI systems that are not only more powerful but also significantly more resource-efficient. The implications are far-reaching. In robotics, improved sample efficiency means faster deployment, less need for dangerous or costly real-world data collection, and more adaptable machines capable of handling diverse tasks—from dexterous manipulation to robust locomotion in complex terrains. For large language models, these innovations promise more reliable, less ‘hallucinating’ AI, capable of sophisticated reasoning with far less training data, making them more accessible and safer for critical applications like medical diagnosis or scientific discovery. The integration of physics-informed models (like PINS-CAD) and quantum computing (like QBO) highlights a future where domain knowledge and advanced computational paradigms synergize with machine learning to solve previously intractable problems in fields from medicine to manufacturing.

The road ahead involves further pushing these boundaries, exploring how these techniques can be combined for even greater efficiency and robustness. Open questions remain around scaling these methods to even larger, more complex real-world scenarios, understanding their theoretical limits, and ensuring that efficiency gains don’t compromise ethical considerations or model interpretability. What’s clear is that the pursuit of sample efficiency is not just an optimization problem; it’s a driving force towards more intelligent, sustainable, and impactful AI.