Sample Efficiency Unleashed: Breakthroughs in Learning from Less

Latest 26 papers on sample efficiency: Jun. 6, 2026

The quest for sample efficiency – enabling AI systems to learn robustly from minimal data – stands as one of the most pressing challenges in modern machine learning. Whether it’s reducing costly human annotations, accelerating robot learning, or training large language models (LLMs) more economically, finding ways to maximize the information extracted from each data point is paramount. This digest dives into recent research that tackles this challenge head-on, showcasing ingenious methods from adaptive data sampling to physics-informed learning and novel gradient estimation techniques.

The Big Ideas & Core Innovations

At the heart of these advancements is a collective push to imbue AI systems with smarter learning mechanisms, moving beyond brute-force data consumption. One prominent theme is the strategic allocation of learning resources. For instance, in “Cross-Epoch Adaptive Rollout Optimization for RL Post-Training”, Yiming Zong, Yige Wang, and Jiashuo Jiang from the Hong Kong University of Science and Technology introduce CERO. This framework optimizes rollout budget distribution in LLM reinforcement learning by identifying informative prompts – those with success probabilities near 0.5 – which yield the most learning signal due to high outcome variance. By using a Beta posterior to estimate prompt informativeness and a Fenchel-dual reformulation, CERO achieves an O(√K) regret guarantee and significant performance gains over baselines like GRPO.

Similarly, in safe reinforcement learning for robotics, the problem of over-conservatism can hinder sample efficiency. “COP-Q: Safety-First Reinforcement Learning for Robot Control via Cholesky-Ordered Projection” by Guopeng Li et al. from Delft University of Technology and Southeast University addresses this by incorporating inter-objective covariance (between reward and safety) into vector-valued Q-value estimation. Their Cholesky factorization approach prioritizes safety while adaptively reducing excessive conservatism on the reward objective, leading to improved sample efficiency without sacrificing safety guarantees.

Another innovative avenue explores leveraging auxiliary data and implicit signals. The “X4Val: Learning Neural Surrogates for Variance-Reduced Policy Evaluation” framework by Rachel Luo et al. from Nvidia Research and Harvard University demonstrates how to use heterogeneous, non-paired data sources (like simulation and historical logs) to learn neural surrogates. These surrogates act as control variates, reducing the variance of real-world metric estimation by up to 38.4% without needing paired samples – a critical gain for costly robotic validation. Meanwhile, in “Preference-Calibrated Human-in-the-Loop Reinforcement Learning for Robotic Manipulation”, Zeyi Liu et al. from Central South University and Nanyang Technological University tackle credit misassignment in human-in-the-loop (HIL-RL). Their PACT framework uses a self-supervised progress model to identify suboptimal segments in human-assisted trajectories and leverages intervention-induced preference signals to calibrate critic and actor updates, achieving 1.3x faster convergence and significant success rate improvements.

The challenge of generalization and transfer with fewer samples is also being rethought. In “Generalizable Multi-Task Learning for Wireless Networks Using Prompt Decision Transformers”, Fatih Temiz et al. from the University of Ottawa propose a Prompt Decision Transformer (PromptDT) based framework for multi-cell selection in wireless networks. By using task-specific trajectory prompts, they enable few-shot adaptation to unseen network configurations, achieving up to 49% QoE improvement without retraining. Further enhancing transferability, “ConTraIRL: Factorized Contrastive Abstractions for Transferable IRL” by Yikang Gui et al. from the University of Georgia learns decoupled latent representations of dynamics and goal factors using a dual-encoder architecture with contrastive objectives. This allows compositional reward transfer to unseen dynamics-goal pairings with minimal target supervision.

For LLMs, optimizing reasoning processes themselves is key. “Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planning” by Dong Liu et al. from UCLA and Columbia University frames chain-of-thoughts (CoT) optimization as sequential decision-making over a learned latent semantic space. Their latent world model simulates the effect of CoT edits, reducing reasoning chain tuning cost by over 70%. Complementing this, in “Transformers Provably Learn to Internalize Chain-of-Thought”, Yixiao Huang et al. from UC Berkeley provide theoretical proof that multi-layer transformers can internalize complex reasoning, maintaining explicit CoT’s sample efficiency gains with a novel Log-ICoT curriculum that reduces training stages from linear to logarithmic.

Finally, re-evaluating foundational assumptions like backpropagation opens new doors. “Is Backpropagation Optimal? When Synthetic Gradients Improve Sample Efficiency” by Yibo Jacky Zhang et al. from Stanford University introduces a unified vectorized feedback framework, proving that synthetic gradients can achieve lower mean squared error than backpropagation under specific conditions, leading to arbitrary sample efficiency gains in scenarios with gradient uncertainty or sparse connectivity. This echoes findings in “BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning” by Shijin Gong et al., which demonstrates that batchwise information sharing can reduce value estimation MSE by 69% with just one rollout per prompt, effectively replacing expensive multi-rollout sampling.

Under the Hood: Models, Datasets, & Benchmarks

These innovations rely on, and in turn advance, a rich ecosystem of models, datasets, and benchmarks:

• LLM Training & Reasoning: The DAPO-Math-17K dataset, MATH dataset, and various Qwen models (Qwen3-4B, Qwen3-4B-Instruct, Qwen2.5-Math-7B) are critical for LLM post-training and reasoning tasks, as seen in CERO and BASIS. Benchmarks like GSM8K, MATH, PIQA, HellaSwag, StrategyQA, and LogiQA are used to test CoT optimization.
• Robotics & Control: Brax (robot locomotion), Safety-Gymnasium (safe RL), DeepMind Control Suite, ManiSkill, and real-world Franka Panda robots are used for evaluating policy evaluation, safe RL, and generative policies. REASSEMBLE and LIBERO-Long datasets drive primitive-aware VLA training.
• Multimodal Learning: Flickr8k dataset, CLIP ViT-B/32 features, and ResNet18 features are used to study multimodal fusion strategies. OpenVLA and π0.5 are key architectures for Vision-Language-Action models.
• Time Series & Anomaly Detection: SMD (Server Machine Dataset) is a primary benchmark for efficient anomaly detection. FinStressTS (https://github.com/jiazeee/FinStressTS) provides a new parametric synthetic benchmark for financial time-series forecasting, enabling mechanism-aware evaluation. The ERA5 climate reanalysis dataset is used for probabilistic function modeling with Neural Processes.
• General RL & Optimization: Multi-Agent Particle Environments (MPE), Overcooked-AI, and Melting Pot are used for multi-agent reinforcement learning. POPGym benchmarks contextual bandits and partially observable RL tasks.
• Interpretability & Evaluation: shapiq package (https://github.com/automl/shapiq), TabPFN, and XGBoost are utilized for Shapley value estimation. AutoEval (https://github.com/PierreBoyeau/autoeval) leverages prediction-powered inference (PPI) for model evaluation across ImageNet, protein fitness prediction, and LLM ranking.
• Core Methodologies: Tools like JAX for automatic differentiation, BoTorch and GPyTorch for Gaussian processes, and general purpose RL platforms like OmniSafe and Stable-Baselines3 underpin many of these developments.

Impact & The Road Ahead

These breakthroughs promise a future where AI systems are not just powerful, but also economical and sustainable to develop. The ability to learn from less data directly translates to reduced computational costs, lower carbon footprints, and broader accessibility for researchers and practitioners without vast data resources. For instance, the findings in “Reward Learning from Best-of-N Preference Data” by Rattana Pukdee et al. offer actionable design principles for efficiently collecting preference data, crucial for LLM alignment.

The impact on robotics is particularly profound. Physics-informed methods like “L-Learning: A Lyapunov-Based Approach Leveraging Lagrangian Mechanics for Efficient and Stable Robot Tracking” by Quan Quan and Hao Li achieve 10-50x better sample efficiency than traditional RL while guaranteeing stability, paving the way for safer and more robust real-world robot deployment. Similarly, primitive-aware VLA training in “Primitive Subspaces Mediate Few-Shot Transfer in VLAs” offers a 3x sample efficiency advantage, significantly reducing the cost of deploying new manipulation tasks in industrial settings.

Looking forward, the insights into gradient uncertainty (from synthetic gradients) and ratio-variance regularization (“Ratio-Variance Regularized Policy Optimization”) are poised to make RL algorithms more robust and efficient. The development of principled synthetic benchmarks like FinStressTS will be vital for diagnostically understanding model failures and guiding informed model selection in high-stakes domains. Furthermore, the push towards interpretable communication protocols in multi-agent systems via methods like SCALE-COMM (https://arxiv.org/pdf/2605.27532) means more reliable and transparent coordination. These collective efforts are not just about training models faster or with less data; they are about building more intelligent, adaptive, and ultimately, more useful AI systems for complex real-world challenges.

Please leave this field empty

Hi there 👋

Get a roundup of the latest AI paper digests in a quick, clean weekly email.

Check your inbox or spam folder to confirm your subscription.