Sample Efficiency Revolution: LLMs, Robots, and Bayes Conquer Complex Tasks with Less Data
Latest 9 papers on sample efficiency: Sep. 13, 2026
The quest for greater sample efficiency is a persistent drumbeat in AI/ML, particularly as models grow larger and real-world interactions remain costly. Whether training sophisticated robot policies, optimizing complex molecular structures, or guiding large language model (LLM) agents, the ability to learn effectively from limited data is paramount. Recent breakthroughs are tackling this challenge head-on, leveraging diverse strategies from sophisticated world models to novel Bayesian optimization techniques.
The Big Ideas & Core Innovations
At the heart of these advancements is a shared drive to make learning smarter, not just bigger. One prominent theme is the integration of world models and predictive reasoning to reduce reliance on real-world interactions. Researchers from Joy Future Academy, JD and Jilin University introduce HaWMPO: Hallucination-Aware World Model-based Policy Optimization for Generalist Robot Policy, a system designed to combat prediction hallucinations in world models during long-horizon Visual-Language-Action (VLA) policy training. Their key insight is that by quantifying the reliability of world model predictions using multiple metrics (like DINO similarity, depth, and optical-flow consistency) and penalizing unreliable action chunks, they can train robot policies effectively with synthetic rollouts, achieving an 80.0% success rate on real-world manipulation tasks. Building on this, Stanford University’s GPU-Accelerated Astrodynamics World Models for Spacecraft Rendezvous and Proximity Operations applies similar principles to the incredibly sensitive domain of spacecraft control. Their Out-of-this-World-Model (OWM), a transformer-based world model, learns joint dynamics and observation models, leading to a remarkable 84% relative gain in docking success over RL baselines and dramatically better generalization to unseen docking ports. They highlight that world models, unlike RL policies, can generalize universally by adapting only the goal pose. Furthermore, Institute of Automation, Chinese Academy of Sciences presents WMLLM: Self-Evolving Optimization Agents via Predict-Then-Act World Modeling. This work endows LLM agents with a ‘predict-then-act’ mechanism, where the LLM forecasts outcomes before execution. The resulting prediction error becomes a powerful self-supervised signal, allowing the agent to continuously refine its implicit world model and achieve state-of-the-art performance in black-box optimization tasks like molecular design, often more efficiently than increasing model size alone.
Another significant thrust focuses on enhancing learning stability and efficiency through intelligent data utilization and architectural innovations. The team from University of Science and Technology of China and State Key Laboratory of Precision and Intelligent Chemistry introduces VLA-Precision: Asymmetric Co-Bootstrapping for Efficient Real-World Online RL of Vision-Language-Action Models. Their Asymmetric Co-Bootstrapping (ACoB) algorithm tackles policy drift and computational inefficiency in real-world online RL for large VLA models. By coupling rapid intervention-guided behavioral learning with progressive value calibration, they achieve an impressive 98.3% mean success rate in high-precision chemistry tasks with a 10.9x speedup in throughput. For LLM agents specifically, Mila – Québec AI Institute and Université de Montréal propose CoSkill: Collaborative Skill Induction and Learning for LLM Agents. CoSkill improves LLM agents in multi-step interactive environments by combining hierarchical skill induction with reinforcement learning. Their Meta-Skill Agent dynamically refines a two-level skill library during training, learning from both successful and failed trajectories to create a self-improving system.
Finally, Bayesian methods are making a resurgence, providing principled ways to quantify uncertainty and guide exploration efficiently. From Nara Institute of Science and Technology and Kyoto University, Constraint-Aware Discrete Black-Box Optimization Using Tensor Decomposition introduces CA-TD, which integrates symbolic feasibility constraints directly into tensor decomposition-based surrogate models for discrete black-box optimization. This approach, using T-norm-based differentiable penalty terms, significantly improves sample efficiency by steering optimization away from infeasible regions from the outset. Further pushing the boundaries of Bayesian methods, University of Southern California’s Subspace Inference Enables Efficient Active Reward Learning from Preferences presents PreferenceEKF. This method frames active reward learning as a sequential Bayesian filtering problem within a low-dimensional parameter subspace of neural network reward models. By enabling scalable uncertainty quantification, PreferenceEKF makes information-theoretic acquisition functions practical for large models, leading to competitive offline RL policy performance with significantly fewer preference queries. And AI Sweden and Chalmers University of Technology deliver No-Regret Bayesian Optimization with Finite-Library Input-Warped Kernels, offering the first high-probability sublinear cumulative regret guarantee for Bayesian Optimization that learns nonlinear input geometry from its own history, enhancing sample efficiency in challenging hyperparameter optimization problems.
Under the Hood: Models, Datasets, & Benchmarks
These innovations rely on a blend of new and established tools and techniques:
- HaWMPO utilizes the LIBERO benchmark, OpenVLA-OFT, Wan2.2-TI2V-5B world model, DINOv3 encoder, and Depth-Anything-v3, validated on a real-world G1 robot.
- VLA-Precision introduces the ACoB algorithm and ACoB-Stream architecture, demonstrating performance on nine high-precision chemistry tasks across four robot embodiments. Code and resources are available at vla-precision.github.io.
- WMLLM leverages Large Language Models as self-evolving agents, incorporating population-based search and Group Relative Policy Optimization (GRPO) for molecular design and other black-box optimization tasks.
- CA-TD proposes a constraint-aware surrogate modeling approach for discrete black-box optimization using tensor decomposition, with code available at github.com/k-onoue/c-bbo.
- Subspace Inference Enables Efficient Active Reward Learning from Preferences introduces PreferenceEKF for uncertainty quantification in neural networks and is validated on D4RL and V-D4RL benchmarks. Code is available at github.com/yutaizhou/bnn_pref.
- GPU-Accelerated Astrodynamics World Models contributes AstroJAX (a JAX-based astrodynamics framework) and the Out-of-this-World-Model (OWM), a transformer-based world model. Resources and code can be found at github.com/sisl/outofthisworldmodel and github.com/duncaneddy/astrojax.
- CoSkill implements a hierarchical skill library and a Meta-Skill Agent with GiGPO for LLM agents, tested on ALFWorld and WebShop environments.
- No-Regret Bayesian Optimization proposes FLIWBO based on input-warped kernels for GP-UCB and demonstrates its efficacy on 20-dimensional multi-agent system design and Fashion-MNIST HPO. Code is at github.com/edvin-ketabati/bogp-paper-experiments.
Impact & The Road Ahead
These advancements represent a significant leap towards more capable, autonomous, and data-efficient AI systems. The ability to effectively train robot policies with fewer real-world interactions, refine LLM agents through self-reflection, and optimize complex designs with limited evaluations will unlock new frontiers in robotics, scientific discovery, and human-computer interaction. The insights from world modeling, especially concerning hallucination detection and generalization, suggest a future where AI systems can simulate and plan more reliably. The integration of symbolic reasoning with learning, as seen in constraint-aware optimization, points to hybrid AI architectures that combine the strengths of both paradigms. Furthermore, the push for more scalable and accurate uncertainty quantification through subspace inference will be crucial for trustworthy AI, especially in domains requiring human oversight and active learning. We are moving towards a future where AI not only learns faster but also understands its own limitations, making it a more reliable and powerful partner in tackling the world’s most challenging problems.
Share this content:
Discover more from SciPapermill
Subscribe to get the latest posts sent to your email.
Post Comment