Sample Efficiency: Unlocking Smarter, Faster AI with Less Data

Latest 50 papers on sample efficiency: Sep. 29, 2025

The quest for intelligent machines often hinges on one critical factor: data. Traditional AI/ML models typically demand vast quantities of labeled data and computational resources, a bottleneck hindering progress in many real-world applications. This challenge has fueled intense research into sample efficiency, the ability of models to learn effectively from limited data. Recent breakthroughs, as showcased in a collection of cutting-edge papers, are revolutionizing how we approach this problem, promising a future of smarter, more adaptable AI with significantly reduced overhead.

The Big Idea(s) & Core Innovations

The overarching theme in recent research is a multi-pronged attack on data scarcity, ranging from novel architectural designs to sophisticated learning paradigms. A prominent trend involves leveraging the power of large pre-trained models and guiding agents with rich contextual information. For instance, in “Teaching RL Agents to Act Better: VLM as Action Advisor for Online Reinforcement Learning”, researchers from OpenAI propose integrating Vision-Language Models (VLMs) as ‘action advisors’ in online reinforcement learning. This allows RL agents to incorporate human-like reasoning, enhancing decision-making and interpretability, and offering a scalable solution for complex environments.

Extending this idea to autonomous agents, “Exploration with Foundation Models: Capabilities, Limitations, and Hybrid Approaches” by Remo Sasso, Michelangelo Conserva, Dominik Jeurissen, and Paulo Rauber from Queen Mary University of London explores how foundation models can guide exploration in RL, especially in early stages. While powerful for high-level reasoning, the authors highlight a “knowing-doing gap” in low-level control, suggesting hybrid approaches are key. This aligns with findings in “Foundation Models as World Models: A Foundational Study in Text-Based GridWorlds” by the same Queen Mary University of London team, which demonstrates that foundation models can act as effective world models, drastically improving sample efficiency in text-based environments.

Another significant thrust focuses on enhancing policy learning and generalization. “Normalizing Flows are Capable Visuomotor Policy Learning Models” shows how normalizing flows can model complex latent spaces for efficient and accurate visuomotor control in robotics. Similarly, in “DINOv3-Diffusion Policy: Self-Supervised Large Visual Model for Visuomotor Diffusion Policy Learning”, Kaiyu Zhang et al. from MIT introduce a self-supervised large visual model combined with diffusion policies, significantly improving robotic control with minimal supervision. “LLM-Guided Task- and Affordance-Level Exploration in Reinforcement Learning” further illustrates the power of LLMs in guiding robotic exploration, leading to improved sample efficiency and impressive zero-shot sim-to-real transfer capabilities.

Beyond perception, architectural and algorithmic innovations play a crucial role. Qiyu Chen and Guozhang Chen from Peking University, in “Aligning Inductive Bias for Data-Efficient Generalization in State Space Models”, introduce Task-Dependent Initialization (TDI), which aligns the inductive bias of state space models with task-specific spectral characteristics, drastically improving generalization in low-data regimes. For multi-task settings, “Leveraging Temporally Extended Behavior Sharing for Multi-task Reinforcement Learning” by Author One and Author Two from University of Example and Example Tech Inc. proposes temporally extended behavior sharing to boost sample efficiency and performance across tasks.

Under the Hood: Models, Datasets, & Benchmarks

These advancements are often underpinned by novel architectures, strategic use of existing models, and robust evaluation benchmarks. Here’s a glimpse into the key resources enabling this progress:

• Foundation Models as Advisors/World Models: Papers like “Teaching RL Agents to Act Better: VLM as Action Advisor for Online Reinforcement Learning” and “Exploration with Foundation Models: Capabilities, Limitations, and Hybrid Approaches” heavily leverage large Vision-Language Models (VLMs) and Large Language Models (LLMs) to guide or simulate environment dynamics. These models, pre-trained on vast datasets, provide high-level understanding and reasoning that new RL agents can exploit.
• Diffusion Models for Control & Generation: “PIRF: Physics-Informed Reward Fine-Tuning for Diffusion Models” by Mingze Yuan et al. from Harvard University and Massachusetts General Hospital frames physics-informed generation as reward optimization for diffusion models, demonstrating state-of-the-art physical enforcement on PDE benchmarks. Similarly, “DINOv3-Diffusion Policy: Self-Supervised Large Visual Model for Visuomotor Diffusion Policy Learning” uses self-supervised DINOv3 visual models with diffusion policies for robotic control. The paper “Autoguided Online Data Curation for Diffusion Model Training” by Valeria Pais et al. from University of Glasgow and Dotphoton shows how autoguidance improves sample quality and diversity in diffusion model training.
• Hybrid Architectures & Learning Frameworks: “D3Grasp: Diverse and Deformable Dexterous Grasping for General Objects” introduces a multimodal framework with a tactile-based perception representation and an asymmetric actor-critic network for robust robotic grasping. “Hierarchical Reinforcement Learning with Low-Level MPC for Multi-Agent Control” by Xiao Wang et al. from Tsinghua University and others combines Hierarchical RL with Model Predictive Control (MPC) for safe multi-agent coordination. For mobile GUI agents, Yifan Xu et al. from Tsinghua University and Z.AI introduce MobileRL with the Difficulty-Adaptive GRPO (ADAGRPO) algorithm in “MobileRL: Online Agentic Reinforcement Learning for Mobile GUI Agents”, achieving state-of-the-art on AndroidWorld and AndroidLab benchmarks. The code for MobileRL is available here.
• Sample-Efficient Tools: “ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution” by Jeremy Berman et al. from Sakana AI, Google Research, and Meta-AI introduces an evolutionary framework with adaptive parent sampling and code novelty rejection-sampling, offering an open-source implementation at https://github.com/SakanaAI/ShinkaEvolve. “Statistical Inference Leveraging Synthetic Data with Distribution-Free Guarantees” by Meshi Bashari et al. from Technion IIT and University of Pennsylvania introduces GESPI for integrating synthetic data with distribution-free guarantees, with code at https://github.com/Meshiba/gespi.git.
• Koopman Operators & Real-time Updates: “Sample-Efficient Online Control Policy Learning with Real-Time Recursive Model Updates” by Zixin Zhang et al. from Stanford University, MIT, and UC Berkeley introduces RKL, a recursive Koopman learning framework for real-time online model updates, with code available here.
• Multimodal Integration: The paper “Sample-efficient Integration of New Modalities into Large Language Models” by Osman Batur Ince et al. from University of Edinburgh and others details a method for low-resource modality integration using hypernetworks, with code at github.com/ospanbatyr/sample-efficient-multimodality.

Impact & The Road Ahead

These innovations collectively pave the way for a new era of AI systems that are not only powerful but also practical and accessible. By drastically reducing data requirements and computational costs, they democratize AI development, allowing smaller teams and resource-constrained environments to deploy sophisticated models. Imagine robots that learn complex manipulation tasks with minimal human demonstration, autonomous vehicles trained faster and safer in diverse simulated environments, or scientific generative models producing accurate simulations from sparse data.

The implications are profound, from accelerating scientific discovery with physics-informed models like PIRF to making dexterous robotics more robust and adaptable with frameworks like D3Grasp. The ability of LLMs to guide exploration (as seen in LLM-Guided Task- and Affordance-Level Exploration) and even generate expert demonstrations (LEED, by Frans A Oliehoek et al. from Springer and OpenStreetMap contributors, in “LEED: A Highly Efficient and Scalable LLM-Empowered Expert Demonstrations Framework for Multi-Agent Reinforcement Learning”) promises to transform multi-agent learning.

The road ahead involves further refining these hybrid approaches, bridging the “knowing-doing gap” in foundation models, and exploring new theoretical underpinnings for generalization, as illuminated by Takeshi Koshizuka and Issei Sato from The University of Tokyo in “Understanding Generalization in Physics Informed Models through Affine Variety Dimensions”. The journey towards truly data-efficient, general-purpose AI is long, but these recent breakthroughs mark exciting milestones, promising a future where intelligent systems learn more from less, becoming ubiquitous and impactful across every domain.