Few-Shot Learning: Navigating the Future of Data-Efficient AI

Latest 50 papers on few-shot learning: Sep. 8, 2025

Few-Shot Learning: Navigating the Future of Data-Efficient AI

Few-shot learning (FSL) stands at the forefront of AI innovation, promising to unlock robust model performance even with minimal labeled data. This capability is not just a convenience; it’s a necessity for real-world applications where data annotation is costly, time-consuming, or inherently scarce. From medical diagnostics and industrial quality control to natural language understanding and robotics, FSL is bridging the gap between data-hungry deep learning and practical deployment. Recent breakthroughs, as highlighted by a collection of cutting-edge research, are pushing the boundaries of what’s possible, tackling challenges like domain generalization, model interpretability, and real-time adaptation.

The Big Idea(s) & Core Innovations

The overarching theme in recent FSL research is to extract maximum utility from limited data by enhancing models’ ability to generalize, adapt, and reason. Several papers demonstrate novel ways to achieve this:

• Adaptive & Parameter-Efficient Tuning: Researchers are developing methods that adapt models with minimal changes. For instance, in “Singular Value Few-shot Adaptation of Vision-Language Models”, Taha Koleilat and colleagues from Concordia University introduce CLIP-SVD, which adapts VLMs using only 0.04% of total parameters by leveraging Singular Value Decomposition. Similarly, the Attn-Adapter framework from Phuoc-Nguyen Bui et al. at Sungkyunkwan University, presented in “Attn-Adapter: Attention Is All You Need for Online Few-shot Learner of Vision-Language Model”, dynamically refines CLIP features through dual attention mechanisms for better cross-category generalization. Another notable approach is LIMO from Ghassen Baklouti and his team, featured in “Language-Aware Information Maximization for Transductive Few-Shot CLIP”, which uses information-theoretic concepts and parameter-efficient fine-tuning (PEFT) to significantly improve transductive FSL performance.

• Intelligent Data & Feature Selection: A recurring challenge is identifying which limited samples are most informative. Parush Gera and Tempestt Neal of the University of South Florida introduce MLSD in “MLSD: A Novel Few-Shot Learning Approach to Enhance Cross-Target and Cross-Domain Stance Detection”, which uses metric learning with triplet loss to select contextually relevant samples, improving cross-domain stance detection. For visual tasks, Javier Ródenas and his collaborators from Universitat de Barcelona propose SPFF in “Stochastic-based Patch Filtering for Few-Shot Learning” and SAFF in “Slot Attention-based Feature Filtering for Few-Shot Learning”. These methods filter irrelevant features by focusing on class-specific patches or discriminative information using stochastic and slot attention mechanisms, respectively.

• Leveraging Foundational Models & Multi-Modality: Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly serving as powerful foundations for FSL. The M3F framework and its accompanying M3FD dataset, presented in “A Foundational Multi-Modal Model for Few-Shot Learning” by Pengtao Dang et al. from Oregon Health & Science University, showcase how LMMMs can generalize across diverse modalities like vision, tables, and time-course data. Similarly, Glo-VLMs, explored by Zhenhao Guo and his team at NYU in “Glo-VLMs: Leveraging Vision-Language Models for Fine-Grained Diseased Glomerulus Classification”, adapt VLMs for fine-grained medical image classification with minimal labeled data. In NLP, Jakub Šmíd and Pavel Přibáň from the University of West Bohemia demonstrate in “Prompt-Based Approach for Czech Sentiment Analysis” that prompt-based methods significantly outperform traditional fine-tuning for few-shot sentiment analysis in low-resource languages.

• Robotics & Embodied AI: FSL is crucial for equipping robots with human-like adaptability. “Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Models” by Hongyin Zhang et al. introduces ARFM, an adaptive offline reinforcement learning method that enhances VLA flow models for robotic tasks by balancing bias-variance trade-offs. The G0 Dual-System VLA Model and the Galaxea Open-World Dataset from the Galaxea Team in “Galaxea Open-World Dataset and G0 Dual-System VLA Model” provide a robust framework and high-quality data for mobile manipulation. Furthermore, “In-Context Iterative Policy Improvement for Dynamic Manipulation” by B. Ichter et al. reveals how pre-trained LLMs can iteratively improve robotic policies in-context without fine-tuning, accelerating adaptation in dynamic systems.

Under the Hood: Models, Datasets, & Benchmarks

The advancements in few-shot learning are heavily reliant on innovative models, diverse datasets, and rigorous benchmarks:

• Core Models:
  • ARFM (Adaptive Offline RL Flow Models): A method for fine-tuning Vision-Language-Action (VLA) models, demonstrating state-of-the-art in generalization and few-shot learning for robotics. (Code: https://github.com/huggingface/lerobot)
  • Attn-Adapter: A lightweight online few-shot learner that uses dual attention to refine CLIP embeddings, enhancing cross-category and cross-dataset generalization.
  • CLIP-SVD: A parameter-efficient adaptation technique for Vision-Language Models using singular value decomposition, adaptable to natural and biomedical domains. (Code: https://github.com/HealthX-Lab/CLIP-SVD)
  • MLSD (Metric Learning for Stance Detection): Improves cross-domain stance detection by selecting informative samples via metric learning and triplet loss. (Code: https://github.com/parushgera/mlsd-few-shot)
  • MetaMiDA: A meta-learning framework that leverages mirror descent to model learnable loss geometries for scalable and convergent adaptation. (Paper: https://arxiv.org/pdf/2509.02418)
  • TransMatch: A transfer-learning framework combining semi-supervised and few-shot learning for defect detection in additive manufacturing, achieving high accuracy with limited labeled data. (Code: https://github.com/transmatch-framework/)
  • QAgent: An LLM-based multi-agent system automating OpenQASM programming for quantum circuits, significantly improving code generation correctness. (Code: https://github.com/fuzhenxiao/QCoder)
  • FlowletFormer: A BERT-based pre-training model for network traffic classification, capturing packet structure, flow behavior, and protocol semantics for improved accuracy and few-shot performance. (Paper: https://arxiv.org/pdf/2508.19924)
  • WEBEYETRACK: A browser-based eye-tracking framework with on-device few-shot personalization and a lightweight CNN model (BlazeGaze) for real-time inference. (Code: https://github.com/RedForestAI/WebEyeTrack)
  • JVLGS (Joint Vision-Language Gas Leak Segmentation): Integrates visual and textual modalities for improved gas leak segmentation, outperforming existing methods in supervised and few-shot settings. (Code: https://github.com/GeekEagle/JVLGS)
  • CoFi (Coarse-to-Fine): A fast few-shot segmentation pipeline for glomerular basement membrane, combining lightweight models with SAM-based automated prompt generation. (Code: https://github.com/ddrrnn123/CoFi)
  • MedSpaformer: A transformer for medical time series classification using multi-granularity token sparsification, offering state-of-the-art performance and zero-shot transferability. (Paper: https://arxiv.org/pdf/2503.15578)
  • LathAdapter: A framework for fine-grained VLM fine-tuning that leverages hyperbolic space for improved adaptation and generalization to unseen classes. (Code: https://github.com/zhaoym55/HyperbolicAdapter.git)
  • SemPT (Semantic Prompt Tuning): Enhances VLM transferability by leveraging shared attribute-level knowledge for improved cross-category generalization. (Paper: https://arxiv.org/pdf/2508.10645)
  • MIST (Multiple Stochastic Prompt Tuning): Improves few-shot adaptation of CLIP under extreme domain and semantic shifts by modeling prompts as Gaussian distributions. (Paper: https://arxiv.org/pdf/2506.03926)
  • CC-Time: A time series forecasting framework leveraging pre-trained language models and cross-model fusion for enhanced accuracy in few-shot settings. (Paper: https://arxiv.org/pdf/2508.12235)
  • MSEF (Multi-layer Steerable Embedding Fusion): Integrates time series patterns into LLMs across multiple layers for enhanced few-shot forecasting. (Code: https://github.com/One1sAll/MSEF)
  • GLiClass: A generalist lightweight model for sequence classification with strong zero-shot and few-shot capabilities. (Code: https://github.com/Knowledgator/GLiClass)
  • GOOD (Generalized Few-shot OOD Detection): A framework for out-of-distribution detection using a General Knowledge Model and Generality-Specificity Balance theory. (Code: https://arxiv.org/pdf/2508.05732)
  • DExNet: A few-shot learning framework for leaf disease classification that combines domain-adapted CNNs and Bi-LSTM classifiers for high accuracy with limited data. (Code: https://github.com/rizqiamaliatuss/PotatoLeafDiseaseClassification)
  • DGS-MAML: A meta-learning algorithm combining gradient matching and sharpness-aware minimization to improve generalization in few-shot learning scenarios. (Code: https://github.com/sungyubkim/GBML/tree/master)
• Key Datasets & Benchmarks:
  • Galaxea Open-World Dataset: A large-scale, high-quality, real-world dataset for robot behavior collection in mobile manipulation. (Dataset: https://opengalaxea.github.io/G0/)
  • M3FD (Multi-Modal Model Few-shot Dataset): Over 10,000 samples covering vision, tables, and time-course data for scientific few-shot learning.
  • U-DIADS-TL and DIVA-HisDB: Datasets used for text line segmentation in historical documents, where few-shot methods show significant gains with less data.
  • Food-101, VireoFood-172, UECFood-256: Benchmarks for food image classification where SPFF demonstrates superior performance.
  • Reddit Datasets: Utilized in “Advancing Minority Stress Detection with Transformers” for evaluating transformer-based models in social media analysis.
  • GazeCapture: A benchmark for eye-tracking, used to validate WEBEYETRACK’s SOTA performance.

Impact & The Road Ahead

The collective efforts in few-shot learning research are ushering in a new era of AI, one where models are not just powerful but also practical, adaptable, and interpretable. The ability to learn effectively from sparse data is transformative for industries like healthcare, robotics, manufacturing, and cybersecurity, where large labeled datasets are a luxury. For instance, the progress in medical imaging, exemplified by CoFi for GBM segmentation and Glo-VLMs for diseased glomerulus classification, promises faster and more accurate diagnoses with minimal expert annotations.

Looking ahead, the fusion of LLMs with specialized FSL techniques, as seen in projects like QAgent for quantum programming and MSEF for time series forecasting, will continue to expand the scope and impact of AI. The theoretical underpinnings, such as “Curvature Learning for Generalization of Hyperbolic Neural Networks” and “Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning”, will lead to more robust and generalizable models. Furthermore, the development of robust benchmarks like MCPTox (presented in “MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers”) will be crucial for ensuring the security and reliability of these advanced AI systems.

As we continue to refine these techniques, the dream of truly autonomous, adaptable, and data-efficient AI—capable of learning and operating in the complex, ever-changing real world—moves ever closer to reality. The journey of few-shot learning is not just about making AI better; it’s about making AI more accessible and impactful for everyone.