Few-Shot Learning: Navigating Complexity with Less Data

Latest 50 papers on few-shot learning: Sep. 1, 2025

Few-Shot Learning: Navigating Complexity with Less Data

Imagine teaching a complex skill to an AI model with just a handful of examples. This isn’t science fiction; it’s the audacious goal of Few-Shot Learning (FSL), a rapidly evolving field at the forefront of AI/ML research. In a world where data annotation is costly and time-consuming, FSL promises to unlock new capabilities, allowing models to generalize from sparse data and adapt swiftly to novel tasks. This digest dives into recent breakthroughs, showcasing how researchers are tackling FSL challenges across diverse domains, from quantum computing to robotic manipulation and medical diagnostics.

The Big Idea(s) & Core Innovations

The central theme across these papers is the quest for models that can learn effectively from minimal data, often by leveraging prior knowledge or cleverly structured architectures. One major thrust involves enhancing the generalization capabilities of models. For instance, Xiaomeng Fan, Yuwei Wu, Zhi Gao, et al. (Beijing Institute of Technology, Monash University), in their paper “Curvature Learning for Generalization of Hyperbolic Neural Networks”, demonstrate that smoothing the loss landscape of hyperbolic neural networks leads to better generalization, even with few shots. This theoretical insight is complemented by practical applications in computer vision, where Pinxuan Li, Bing Cao, Changqing Zhang, and Qinghua Hu (Tianjin University) introduce GOOD (“Generalized Few-Shot Out-of-Distribution Detection”), a framework that leverages a General Knowledge Model (GKM) to improve out-of-distribution detection in few-shot settings by balancing generality and specificity.

Another significant innovation focuses on making large models more adaptable and efficient for FSL. Minh-Tan PHAM et al. (Université Bretagne Sud, CentraleSupélec, INRIA), in “Contributions to Label-Efficient Learning in Computer Vision and Remote Sensing”, explore various label-efficient methods, including multi-task partially supervised learning (MTPSL) and self-supervised contrastive learning, to make vision models perform better with limited annotations. Similarly, Zhenhao Guo et al. (New York University, Cornell Tech, Vanderbilt University)’s “Glo-VLMs: Leveraging Vision-Language Models for Fine-Grained Diseased Glomerulus Classification” showcases how fine-tuning large pretrained Vision-Language Models (VLMs) with parameter-efficient techniques can achieve high accuracy in medical image classification with only a few examples per class. This echoes the broader trend of leveraging pre-trained models, as seen in “Synthesizing DSLs for Few-Shot Learning” by Paul Krogmeier and P. Madhusudan (University of Illinois), which proposes automatically synthesizing domain-specific languages (DSLs) to define hypothesis classes for symbolic learning algorithms, making them more effective in few-shot scenarios.

The challenge of data scarcity is also met with novel data augmentation and feature filtering strategies. Javier Ródenas, Eduardo Aguilar, and Petia Radeva (Universitat de Barcelona) propose two methods: SPFF (“Stochastic-based Patch Filtering for Few-Shot Learning”) which focuses on class-specific patches for food image classification, and SAFF (“Slot Attention-based Feature Filtering for Few-Shot Learning”) which uses slot attention to filter irrelevant features in both support and query images, significantly improving classification performance. For critical applications like driver distraction detection, X. Han et al. (The Twelfth International Conference on Learning Representations)’s PQ-DAF (“Pose-driven Quality-controlled Data Augmentation for Data-scarce Driver Distraction Detection”) uses pose information to generate high-quality synthetic data, addressing the common challenge of limited real-world datasets.

Robotics and quantum computing are also seeing significant FSL advancements. B. Ichter et al. (MIT CSAIL, Google Research, Stanford University), in “In-Context Iterative Policy Improvement for Dynamic Manipulation”, show that pre-trained Large Language Models (LLMs) can iteratively improve policies for dynamic robotic manipulation tasks with a small dataset, without fine-tuning. For quantum computing, “QAgent: An LLM-based Multi-Agent System for Autonomous OpenQASM programming” by Zhenxiao Fu, Fan Chen, and Lei Jiang (Indiana University Bloomington) introduces a multi-agent system that automates OpenQASM programming with few-shot learning and retrieval-augmented generation, improving quantum circuit code generation accuracy by up to 71.6%.

Under the Hood: Models, Datasets, & Benchmarks

The research in few-shot learning heavily relies on innovative architectures, bespoke datasets, and rigorous benchmarks to validate improvements:

• QAgent: A hybrid multi-agent system for OpenQASM programming, validated on QCircuitNet dataset. Code available at https://github.com/fuzhenxiao/QCoder.
• FlowletFormer: A BERT-based pre-training model for traffic classification, leveraging novel pretraining tasks and evaluated on multiple network traffic datasets. Supplementary material available with the paper (https://arxiv.org/pdf/2508.19924).
• WEBEYETRACK: An open-source, browser-friendly framework for few-shot gaze estimation, utilizing BlazeGaze (a lightweight CNN model based on BlazeBlocks), and achieving SOTA on GazeCapture. Code available at https://github.com/RedForestAI/WebEyeTrack.
• JVLGS: A vision-language framework for gas leak segmentation, outperforming existing methods in few-shot settings. Code available at https://github.com/GeekEagle/JVLGS.
• Few-Shot Connectivity-Aware Text Line Segmentation: Uses a lightweight UNet++ architecture and a connectivity-aware loss function, achieving SOTA on U-DIADS-TL and strong generalization on DIVA-HisDB. Code: https://github.com/RafaelSterzinger/acpr_few_shot_hist.
• Curvature Learning for Hyperbolic Neural Networks: Proposes a sharpness-aware curvature learning method, validated across tasks like classification, long-tailed data, noisy data, and few-shot learning scenarios (https://arxiv.org/pdf/2508.17232).
• Synthesizing DSLs for Few-Shot Learning: Explores DSL synthesis using SyGuS format (syntax-guided synthesis) and SemGuS framework (https://arxiv.org/pdf/2508.16063).
• MSEF (Multi-layer Steerable Embedding Fusion): Integrates time series into LLMs for enhanced forecasting, achieving SOTA on multiple benchmarks including ElectricityLoadDiagrams20112014. Code at https://github.com/One1sAll/MSEF.
• Glo-VLMs: Adapts vision-language models for glomerular classification with limited supervision (8 shots per class) using parameter-efficient adaptation. Paper: https://arxiv.org/pdf/2508.15960.
• Bridging Generalization and Personalization in HAR: Uses a hybrid approach with on-device few-shot learning, optimized for the GAP9 microcontroller, tested on RecGym dataset. Code: https://github.com/kangpx/40khz-ultrasonicDHGR-onlineSemi.
• In-Context Iterative Policy Improvement: Demonstrates LLMs can perform policy improvements for dynamic robotic tasks without fine-tuning. Code related to Bayesian Optimization is available at https://github.com/bayesian-optimization/BayesianOptimization.
• MCPTox: A benchmark for Tool Poisoning attacks on real-world MCP servers, with 1312 malicious test cases, revealing vulnerabilities in LLM-integrated MCP ecosystems (https://arxiv.org/pdf/2508.14925).
• CC-Time: A tri-modal framework for time series forecasting, leveraging pre-trained language models and cross-model fusion, achieving SOTA on nine real-world datasets (https://arxiv.org/pdf/2508.12235).
• MedSpaformer: A transformer-based framework for medical time series classification, demonstrating SOTA on seven medical datasets in supervised and few-shot scenarios (https://arxiv.org/pdf/2503.15578).
• CoFi: A few-shot segmentation pipeline for glomerular basement membrane, using lightweight models and automated prompt generation via SAM. Code: https://github.com/ddrrnn123/CoFi.
• Meta-learning Structure-Preserving Dynamics: Introduces modulation-based meta-learning for dynamical systems, evaluated on energy-conserving and dissipative systems (https://arxiv.org/pdf/2508.11205).
• LatHAdapter: Leverages hyperbolic space for fine-tuning Vision-Language Models (VLMs) with latent hierarchical adapters. Code: https://github.com/zhaoym55/HyperbolicAdapter.git.
• SemPT (Semantic Prompt Tuning): Enhances VLMs’ transferability using shared attribute-level knowledge, demonstrating SOTA on 15 benchmark datasets (https://arxiv.org/pdf/2508.10645).
• SPFF: Uses stochastic patch filtering for food image classification, outperforming existing methods on Food-101, VireoFood-172, and UECFood-256 (https://arxiv.org/pdf/2508.10066).
• SAFF: Employs slot attention for feature filtering in few-shot learning, evaluated on CIFAR-FS and miniImageNet (https://arxiv.org/pdf/2508.09699).
• DGS-MAML: A meta-learning algorithm combining gradient matching with sharpness-aware minimization, validated on benchmark datasets. Code: https://github.com/sungyubkim/GBML/tree/master.
• MIST (Multiple Stochastic Prompt Tuning): Improves few-shot adaptation of CLIP under extreme domain shifts using Gaussian-based prompts (https://arxiv.org/pdf/2506.03926).
• GLiClass: A generalist lightweight model for sequence classification with a label-conditioned encoder transformer, providing excellent few-shot learning capabilities. Code: https://github.com/Knowledgator/GLiClass.
• GOOD: A framework for generalized few-shot OOD detection, utilizing a General Knowledge Model (https://arxiv.org/pdf/2508.05732).
• M3FD & M3F: A multi-modal few-shot dataset with 10K+ samples and a novel framework built on LMMMs for scientific domains. Code: https://github.com/ptdang1001/M3F.
• GraphProp: The first GFM achieving structural and node feature generalization across domains using graph invariants (https://arxiv.org/pdf/2508.04594).
• ProtoN: A prototype node graph neural network for multi-impression ear recognition in unconstrained environments (https://arxiv.org/pdf/2508.04381).
• T3Time: A tri-modal framework for multivariate time series forecasting, achieving SOTA with significant improvements. Code: https://github.com/monaf-chowdhury/T3Time/.
• MultiADS: A zero-shot learning approach for multi-type anomaly detection and segmentation, leveraging defect-specific knowledge from VLMs, and outperforming existing methods on five benchmark datasets. Code: https://github.com/boschresearch/MultiADS.
• Causal CLIP Adapter (CCA): A framework for enhanced few-shot learning using causal disentanglement and cross-modal alignment, outperforming SOTA on 11 benchmark datasets. Code: https://github.com/tianjiao-j/CCA.
• PointKAN: A novel architecture leveraging Kolmogorov-Arnold Networks (KAN) for point cloud analysis, achieving significant reductions in parameters and computational complexity. Code: https://github.com/Shiyan-cps/PointKAN-pytorch.
• MicroMix: A mixed-precision quantization algorithm for LLMs using MX data formats, outperforming TensorRT baselines by at least 20%. Code: https://github.com/lwy2020/MicroMix.git.
• InstructTime: A novel instruction-based time series editor using natural language, enabling high-quality edits with controllable strength. Code: https://github.com.
• MOFS: A multi-operator few-shot operator learning framework for generalization across PDE families, integrating frequency-aware self-supervision and semantic text conditioning (https://arxiv.org/pdf/2508.01211).
• UoMo: A universal foundation model for mobile traffic forecasting, combining diffusion models and transformers, demonstrating up to 27.85% improvement in short-term prediction accuracy. Code: https://github.com/tsinghua-fib-lab/UoMo.

Impact & The Road Ahead

The implications of these advancements are profound. Few-shot learning is transforming fields from medical diagnostics, where limited patient data often impedes AI development (e.g., Glo-VLMs, MedSpaformer, CoFi), to robotics, enabling quicker adaptation to new environments and tasks (e.g., In-Context Iterative Policy Improvement, Embodied Long Horizon Manipulation). The ability to learn from minimal examples democratizes AI, making powerful models accessible even in resource-constrained settings, such as on-device learning for wearables (Bridging Generalization and Personalization in HAR) or low-resource languages (Prompt-Based Approach for Czech Sentiment Analysis).

Looking ahead, the research points towards increasingly sophisticated hybrid models that combine the strengths of various AI paradigms. The synergy between large language models and other modalities (vision, time series, graph structures) is particularly promising, hinting at a future where AI systems possess a more holistic understanding of the world. Continued exploration into theoretical underpinnings, such as curvature learning in hyperbolic networks, and the development of robust benchmarks will be crucial. Addressing security vulnerabilities in LLM agents, as highlighted by MCPTox, will also be paramount as these models are deployed in critical systems. The ultimate vision is an AI that learns like a human: rapidly, flexibly, and with an inherent understanding of its environment, even when faced with novel situations and scarce data. The journey to truly generalizable, few-shot intelligence is exhilarating, and these papers mark significant strides on that path.