Sample Efficiency: Accelerating AI/ML Learning from Robots to LLMs

Latest 22 papers on sample efficiency: Oct. 17, 2026

The quest for greater sample efficiency in AI/ML is a critical frontier, promising to unlock faster training, reduce computational costs, and enable real-world applications in data-scarce domains. From training sophisticated robots in dynamic environments to teaching large language models complex reasoning, researchers are pushing the boundaries of how much intelligence can be gleaned from limited interactions. Recent breakthroughs highlight innovative strategies that are fundamentally reshaping how we approach learning, making AI systems more adaptable and practical than ever before. This post dives into a collection of cutting-edge research, synthesizing their core ideas and practical implications.

The Big Idea(s) & Core Innovations

The central theme across these papers is the ingenious use of prior knowledge, structured learning, and novel reward mechanisms to dramatically reduce the amount of data needed for effective training. Instead of brute-force data collection, the focus is on smarter learning paradigms.

For instance, in the realm of generative models, Noise-Space Reward Alignment for Few-Step Generative Models by Shin et al. from KAIST, Stanford University, and Carnegie Mellon University introduces Noise Policy Optimization (NPO). Their key insight is to keep a pretrained generator frozen and instead learn a flow policy over the input noise space that generates reward-tilted output distributions. This preserves pretrained knowledge, avoids reward over-optimization, and achieves a superior quality-reward trade-off, with inference costs reduced by 16 to 549 times.

In robotic learning, Hierarchical Residual Policy Learning (HiRe-MoMa) by Gong et al. from Northeastern University, University of California, Riverside, and Georgia Institute of Technology, presented in Hierarchical Residual Policy Learning for Real-World Mobile Manipulation with Sparse Human Guidance, addresses the challenge of real-world fine-tuning. They propose a hierarchical residual architecture with unidirectional conditioning for mobile manipulators, using sparse human interventions (under 20% of data) to guide exploration. This significantly boosts success rates (from 39.2% to 75%) with just 25-40 minutes of online interaction. Similarly, for sparse-reward robotic manipulation, Task-Space Imitation Guidance for Efficient Reinforcement Learning (TIGER) by Asayesh et al. from Sanctuary AI, converts imitation policy action chunks into dense task-space progress rewards, improving early sample efficiency by 2-5x and reducing safety violations.

For large language models (LLMs), tackling sparse rewards in multi-step arithmetic reasoning is the focus of Bellman-Consistent Trajectory RL for LLM Arithmetic Reasoning by Li from Stanford University. This work introduces Trajectory Bellman Residual Minimization (TBRM) to transform sparse terminal rewards into dense token-level supervision, combined with RLOO variance reduction and hindsight goal relabeling. This innovative approach achieves 84.0% accuracy on the Countdown task, significantly outperforming baselines and demonstrating that a single LLM can serve as Q, V, and policy functions. Complementing this, Co-Evolving Acting and Forecasting for Language Agents (COPE) by Yu et al. from the University of Illinois Urbana-Champaign and Meta AI, adds multi-step action forecasting as an auxiliary objective, forming an implicit temporal ensemble that improves sample efficiency by up to 2.0x on benchmarks like ScienceWorld without additional models or inference overhead.

In the realm of deep learning fundamentals, HAND: A Biologically-Inspired Activation Function that Improves Generalisation and Sample Efficiency in Image Classification by Spratling and Schütt from the University of Luxembourg, introduces a novel activation function that incorporates biologically-inspired competition between neurons. This leads to an astounding 8x reduction in training epochs for ImageNet, while also boosting robustness and performance on imbalanced datasets.

For complex systems, integrating gradient information is proving highly beneficial. Exploiting Gradients in Bayesian Inference of Expensive Simulators by Soldát and Šmídl from the Czech Technical University in Prague, shows that incorporating reverse-mode (adjoint) differentiation gradients into Gaussian process surrogates dramatically accelerates Bayesian inference, especially in high-dimensional parameter spaces. Similarly, E2DRL: End-to-End Differentiable Reinforcement Learning for 3D Obstacle Avoidance in Quadruped Robots by Fan et al. from the National University of Singapore, achieves 4.2x higher sample efficiency in quadruped robot obstacle avoidance by combining a soft differentiable obstacle-interaction formulation with teacher-student distillation.

Other notable innovations include: * MA-JEPA from Kaplowitz et al. (University of Oxford) in MA-JEPA: Joint-Embedding World Models for Multi-Agent Reinforcement Learning uses joint-embedding prediction for multi-agent world models, bypassing observation reconstruction for efficient control. * BOLD in Example-driven Parametrisations for Bayesian Shape Optimisation by Diaz-Aylwin et al. (Lancaster University) learns shape parameterizations from examples using PCA on displacement fields, improving sample efficiency for aerodynamic and RF cavity designs. * ROUTEFLOW in Navigating Route Latent Space for Synthesizable Molecular Design by Li et al. (Emory University and University of Oxford) reformulates synthesizable molecular design as a search over a continuous route latent space, achieving best average sample efficiency in molecular optimization. * Ask the Expert in Ask the Expert: LLM-Guided Reinforcement Learning for Autonomous Cyber Defense by Martinez et al. (Fordham University) uses LLMs for training-time reward shaping in cyber defense, then discards them for deployment, yielding efficient and secure policies. * CATune from Lan et al. (Temple University) in CATune: Structural Constraint-Aware Bayesian Optimization for DBMS Configuration Tuning structurally encodes DBMS knob dependencies into Bayesian optimization, improving tuning speed by up to 12.5x. * VAE-BO in A Latent Space Optimization Approach for Symbolic Discovery of Dynamical Models by Liu and Mitrai (The University of Texas at Austin) combines Variational Autoencoders with Bayesian Optimization to discover symbolic dynamical models with superior sample efficiency. * The theoretical work in Symmetry-Aware Feature Learning: A Polynomial Separation for Multi-Index Models by Waber et al. (EPFL) formally proves the significant sample complexity advantages (r-fold polynomial separation) of symmetry-aware learning (via weight sharing or data augmentation) over symmetry-agnostic methods.

Under the Hood: Models, Datasets, & Benchmarks

These advancements are often powered by, or contribute to, new and improved models, datasets, and evaluation benchmarks:

• Generative Models: NPO utilizes models like SD-Turbo, SANA-Sprint, and ELF, and is evaluated using LAION aesthetic predictor, DeBERTa-v3-large, and ImageReward. Code for NPO is available at https://shin-woocheol.github.io/npo/.
• Robotics: HiRe-MoMa is validated on TIAGo++ tasks with Diffusion Policy and ACT architectures, and E2DRL leverages the NVIDIA Warp differentiable simulator and is deployed on a Unitree Go2 quadruped robot. TIGER shows improvements for robotic manipulation with RLPD and IBRL backends. The HiRe-MoMa project page is at https://hire-moma.github.io/.
• LLMs & Reasoning: Bellman-Consistent RL uses the Qwen/Qwen2.5-0.5B base model and datasets like Asap7772/cog_behav_all_strategies. COPE is evaluated across ALFWorld, ScienceWorld, WebShop, τ2-bench, and AppWorld benchmarks using Qwen2.5-7B/14B-Instruct and Qwen3-8B. COPE’s code is at https://github.com/ulab-uiuc/CoPE.
• Molecular Design: ROUTEFLOW utilizes the TDC PMO benchmark, PaRoutes dataset, and Enamine US catalog. PMO-Dock (PMO-Dock: Benchmarking Docking, Specificity, and Generalization in Molecular Optimization) by Simonyan et al. (YerevaNN, Yerevan State University) provides a new benchmark with 25 docking-based tasks, including novel specificity binding tasks, and evaluates methods like Saturn, GenMol, Genetic GFN, and ChemLlama.
• Optimization & Evaluation: Towards Easier-to-Tune Training Algorithms: New Results from AlgoPerf by Kasimbeg et al. (Google, AI For Tomorrow) presents the AlgoPerf v1 benchmark, evaluating 13 self-tuning algorithms (like Schedule-Free AdamW v2 and Muon) across 9 workloads. SteerScope (Does Steering Break Your Model? A Multi-Dimensional Evaluation Suite for LLM Steering Methods) by Yang et al. (Peking University, Columbia University) introduces a comprehensive evaluation suite with 15 metrics for LLM activation steering methods, benchmarking 23 different approaches. Its code is at https://github.com/yht0511/steerscope.
• Vision & Control: HAND activation function is tested on ImageNet, CIFAR, SVHN, and their long-tailed/corrupted variants. VideoAWR (VideoAWR: Reinforcing Video Flow Models with Decoupled-Timestep Advantage-Weighted Rectification) by Fu et al. (Beihang University) uses Wan2.1-1.3B/14B video models and the HPSv3 reward model, evaluated with VBench and VideoAlign. Code for HAND is at https://codeberg.org/mwspratling/HAND and for VideoAWR at https://github.com/VideoAWR/VideoAWR.
• RL Fundamentals: Can Jev be Your Q or Policy in Reinforcement Learning? by Ma et al. (Shanxi University, Nanjing University) explores the use of the Jev decision model in RL across MiniGrid environments and Atari games.

Impact & The Road Ahead

These advancements herald a future where AI systems are not only more powerful but also more practical and sustainable. The ability to learn effectively from fewer samples means:

• Faster Development Cycles: Researchers and developers can iterate on models and policies much more quickly.
• Reduced Computational Footprint: Less data and faster convergence translate directly to lower energy consumption and environmental impact, a crucial step toward sustainable AI.
• Enhanced Real-World Deployment: Robots can adapt to new environments with minimal human intervention, LLMs can be fine-tuned for specific tasks without massive proprietary datasets, and molecular design can be accelerated for drug discovery.
• Improved Robustness and Generalization: Biologically-inspired architectures and careful attention to training stability lead to models that perform better under diverse, real-world conditions.

The road ahead involves further integrating these diverse insights. Combining sophisticated reward shaping with gradient-enhanced surrogates, leveraging the power of foundation models for guidance, and developing even more biologically plausible architectures could lead to truly remarkable breakthroughs. The ongoing work on benchmarks like AlgoPerf and SteerScope will continue to guide the field, ensuring that new algorithms are not just theoretically sound but also practically effective and easier to deploy. The excitement is palpable as we move towards an era of more intelligent, efficient, and accessible AI.

Please leave this field empty

Hi there 👋

Get a roundup of the latest AI paper digests in a quick, clean weekly email.

Check your inbox or spam folder to confirm your subscription.