Sample Efficiency Unleashed: Breakthroughs in AI/ML from RL to Foundation Models

Latest 50 papers on sample efficiency: Dec. 27, 2025

The quest for sample efficiency – getting more intelligence from less data – is a holy grail in AI/ML, driving innovation across reinforcement learning (RL), computer vision, and natural language processing. As models grow larger and tasks more complex, the ability to learn effectively with fewer examples becomes paramount. This digest dives into recent research that’s pushing the boundaries of sample efficiency, unveiling novel architectures, algorithms, and training paradigms.

The Big Idea(s) & Core Innovations

Many recent papers converge on a core theme: smarter data utilization and architectural design to overcome the challenges of data scarcity and computational cost. In the realm of Reinforcement Learning, a significant trend is the development of algorithms that learn more effectively from limited or suboptimal data. For instance, QR-MAX by Alessandro Trapasso, Luca Iocchi, and Fabio Patrizi from Sapienza University of Rome introduces a novel model-based algorithm for Non-Markovian Reward Decision Processes (NMRDPs). Their key insight is that decoupling Markovian environment dynamics from non-Markovian reward handling vastly improves sample efficiency and theoretical guarantees, even extending to continuous state spaces with their SimHash-based BUCKET-QR-MAX variant. This factorization strategy dramatically reduces sample complexity by reusing learned environment transitions across different automaton states.

Similarly, EUBRL: Epistemic Uncertainty Directed Bayesian Reinforcement Learning from Jianfei Ma and Wee Sun Lee (National University of Singapore) achieves nearly minimax-optimal regret and sample complexity by integrating epistemic uncertainty into the agent’s objective, guiding principled exploration in sparse-reward, long-horizon, and stochastic MDPs.

Addressing the critical issue of distribution shift in offline RL, Yuanhao Chen et al. from Harbin Institute of Technology Shenzhen in their paper Sample-Efficient Policy Constraint Offline Deep Reinforcement Learning based on Sample Filtering propose a simple yet effective sample filtering method to improve performance by focusing on high-quality transitions. Complementing this, MOORL: A Framework for Integrating Offline-Online Reinforcement Learning by Gaurav Chaudhary et al. from the Indian Institute of Technology Kanpur, leverages meta-learning to seamlessly combine offline and online data, enhancing exploration and sample efficiency in complex environments without introducing new hyperparameters.

For Deep Reinforcement Learning specifically, Averaging n-step Returns Reduces Variance in Reinforcement Learning by Brett Daley et al. (University of Alberta) demonstrates that compound returns (like λ-returns) strictly lower variance, leading to faster and more stable learning in methods like DQN and PPO. Pushing the boundaries of model-based RL, Double Horizon Model-Based Policy Optimization from Akihiro Kubo et al. (Advanced Telecommunications Research Institute, Kyoto University) introduces DHMBPO, using two distinct rollout horizons to elegantly balance distribution shift, model bias, and gradient instability for superior sample efficiency in continuous control.

In the realm of Foundation Models, especially for Vision and Language, advancements focus on efficient training and robust adaptation. The AMoE: Agglomerative Mixture-of-Experts Vision Foundation Model by Sofian Chaybouti et al. (Technology Innovation Institute, Abu Dhabi) leverages multi-teacher distillation with innovations like Asymmetric Relation-Knowledge Distillation (ARKD) and token-balanced batching to improve sample efficiency and representation quality. For Large Language Models, Fine-Tuned In-Context Learners for Efficient Adaptation from Jörg Bornschein et al. (Google DeepMind) proposes a unified approach combining fine-tuning with in-context learning, achieving superior performance in data-scarce scenarios, particularly with a novel prequential evaluation protocol for hyperparameter tuning.

Diffusion Models also see significant sample efficiency gains. Control Variate Score Matching for Diffusion Models by Khaled Kahouli et al. (Google DeepMind) introduces Control Variate Score Identity (CVSI) to reduce variance in score estimation, enhancing both training and inference efficiency. Building on this, Diffusion Fine-Tuning via Reparameterized Policy Gradient of the Soft Q-Function from Hyeongyu Kang et al. (KAIST) introduces SQDF, an RL framework for fine-tuning diffusion models that avoids reward over-optimization while preserving sample diversity.

Beyond these, several papers explore novel architectural and learning paradigms. Discovering Lie Groups with Flow Matching by Jung Yeon Park et al. (Northeastern University) introduces LieFlow, a method that learns symmetries directly from data using flow matching on Lie groups, offering a more flexible and interpretable approach. Meanwhile, Memory-Amortized Inference: A Topological Unification of Search, Closure, and Structure by Author One and Author Two (University of Example) proposes MAI to reduce memory overhead while maintaining accuracy across tasks by unifying search, closure, and structural operations through topological methods.

Under the Hood: Models, Datasets, & Benchmarks

These innovations rely on cutting-edge models, carefully curated datasets, and robust benchmarks:

• Reinforcement Learning Algorithms: QR-MAX and BUCKET-QR-MAX (https://github.com/Alee08/), EUBRL, DHMBPO (https://github.com/4kubo/erl_lib), MOORL (https://github.com/gauravch/MOORL), RFL-TV, ECIM, Symphony (https://arxiv.org/pdf/2512.10477), and the adaptations of PPO and DQN using compound returns (https://github.com/brett-daley/pilar) are central to the progress in this domain.
• Foundation Models & Datasets: AMoE from the Technology Innovation Institute introduces OpenLVD200M, a 200M-image dataset, alongside their novel Mixture-of-Experts (MoE) architecture. LLM research often uses benchmarks like MATH500 and AIME2024 for reasoning tasks.
• Diffusion Models & Tools: CVSI and SQDF work on general diffusion models, improving their underlying mechanics. TreeGRPO specifically fine-tunes visual generative models, showcasing a tree-structured RL framework. MPDiffuser for offline decision-making leverages D4RL and DSRL benchmarks.
• Robotics Platforms: HydroGym (https://doi.org/10.5281/zenodo.13350586) provides a solver-independent RL platform with 42 validated environments for fluid dynamics. For humanoid robots, PvP: Data-Efficient Humanoid Robot Learning with Proprioceptive-Privileged Contrastive Representations from LimX Dynamics introduces SRL4Humanoid (https://github.com/LimX-Dynamics/SRL4Humanoid), an open-source framework for evaluating state representation learning. The mimic-video project (https://github.com/mimic-robotics/mimic-video) from mimic robotics aims to build generalizable robot control beyond VLAs using large-scale robotics data.
• Machine Learning Tools: The adoption of Kolmogorov-Arnold Networks (KANs) in KAN-Dreamer (https://github.com/Blealtan/efficient-kan) explores new function approximators for world models. Bayesian Optimization benefits from the methods presented in Informing Acquisition Functions via Foundation Models for Molecular Discovery (https://github.com/qichen876/LLMAT), utilizing LLMs and chemistry foundation models.

Impact & The Road Ahead

The implications of these advancements are profound. Improved sample efficiency means faster development cycles, reduced computational costs, and the ability to tackle problems in data-scarce domains like scientific discovery (e.g., molecular discovery with LLMAT) or real-world robotics where data collection is expensive and risky (e.g., humanoid control with PvP, fluid dynamics with HydroGym). The push towards more robust and generalizable models, whether through explicit compositional frameworks like ECO-Net for multi-part object representations or the integration of context-aware agentic systems in EV power optimization, points towards more reliable and adaptable AI systems.

For LLMs, innovations like Reflective Preference Optimization (RPO) and Generative Adversarial Reasoner (GAR) are making them not only more powerful but also safer and more robust against issues like hallucination. The ability to integrate LLMs into RL for wireless networks, as explored in Tutorial on Large Language Model-Enhanced Reinforcement Learning for Wireless Networks, signals a future where AI-driven network management is more intelligent and adaptive.

The emphasis on theoretical guarantees (e.g., in EUBRL, QR-MAX, and Dataset Distillation’s Utility Boundary) is crucial for building trust and understanding in complex AI systems. The exploration of symmetries (LieFlow, Symmetry-Aware Steering) and the integration of physics-informed methods (PINS-CAD for coronary artery digital twins) highlight a move towards grounding AI in fundamental scientific principles, promising not just better performance but also deeper insights. The future is bright with these innovations, as we move closer to truly intelligent and sample-efficient AI that can learn from minimal experience and generalize across vast, unseen domains.