Few-Shot Learning: Navigating Data Scarcity with Smarter AI

Latest 50 papers on few-shot learning: Nov. 16, 2025

In the rapidly evolving world of AI and Machine Learning, the ability to learn effectively from limited data—known as few-shot learning (FSL)—is paramount. Traditional deep learning often demands vast datasets, which are frequently unavailable in critical domains like medical diagnosis, robotics, or specialized language tasks. Recent breakthroughs, however, are pushing the boundaries of what’s possible, enabling models to generalize from just a handful of examples, and even adapt to entirely new domains. This post dives into a collection of cutting-edge research, revealing how diverse strategies are making FSL more robust, efficient, and applicable across various real-world scenarios.

The Big Idea(s) & Core Innovations

The central challenge addressed by these papers is how to equip AI models with the intelligence to learn and adapt quickly, even when labeled data is scarce or when facing entirely new, unseen scenarios. A major theme is leveraging pre-trained knowledge and adaptive mechanisms. For instance, researchers at Google Research, USA, in their paper GEMMA-SQL: A Novel Text-to-SQL Model Based on Large Language Models, demonstrate how lightweight, open-source LLMs can achieve competitive text-to-SQL performance with fewer resources by employing self-consistency strategies and schema-aware prompting. This highlights the power of leveraging existing model capabilities with smart prompting.

Another significant innovation focuses on mitigating biases and enhancing feature generalization. The paper FreqGRL: Suppressing Low-Frequency Bias and Mining High-Frequency Knowledge for Cross-Domain Few-Shot Learning from National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an, introduces a frequency-space perspective to cross-domain few-shot learning (CD-FSL). Their FreqGRL framework actively suppresses low-frequency domain-specific biases while emphasizing high-frequency, generalizable features, leading to state-of-the-art performance in transferring knowledge across diverse domains. Similarly, Wuhan University’s Adv-SSL: Adversarial Self-Supervised Representation Learning with Theoretical Guarantees tackles bias in self-supervised learning through min-max optimization, providing theoretical guarantees that large unlabeled datasets can enhance downstream few-shot classification by improving representation clustering.

Several papers explore novel architectural and algorithmic approaches for FSL. Neural Variational Dropout Processes by Seoul National University introduces NVDPs, a Bayesian meta-learning framework using task-specific dropout rates to model conditional posteriors. This approach effectively combats under-fitting and posterior collapsing, delivering strong results in few-shot regression, image inpainting, and classification. For graph-based tasks, Jilin University’s Graph Few-Shot Learning via Adaptive Spectrum Experts and Cross-Set Distribution Calibration (GRACE) proposes adaptive spectrum experts and cross-set distribution calibration to handle local structural variations and distributional shifts, significantly improving generalization in graph FSL. Moreover, ADaMoRE: Adaptive Graph Mixture of Residual Experts from Tianjin University tackles unsupervised learning on diverse graphs by using a heterogeneous Mixture-of-Experts (MoE) architecture, achieving robust and adaptive training for few-shot node classification.

In the realm of multimodal learning, Columbia University’s Hierarchical Material Recognition from Local Appearance introduces a hierarchical taxonomy and graph attention networks for material recognition, enabling rapid adaptation to new materials with few-shot capabilities. SYNTRANS: Synergistic Knowledge Transfer of Large Multimodal Models for Few-Shot Learning by The Hong Kong Polytechnic University presents a framework to transfer visual and semantic knowledge from large multimodal models (like CLIP) to enhance few-shot learning by bridging visual and semantic spaces through dynamic fusion. Similarly, the University of Technology, Shanghai’s Packet Inspection Transformer uses self-supervised transformers on packet data for unseen malware detection with minimal samples, showcasing FSL’s utility in cybersecurity.

Finally, the integration of human-in-the-loop and reasoning mechanisms is proving crucial. The University of Zurich’s work, Multi Language Models for On-the-Fly Syntax Highlighting, uses token normalization to enable multi-language syntax highlighting with as few as 10 training samples. For complex reasoning, EPFL’s GRAD: Generative Retrieval-Aligned Demonstration Sampler for Efficient Few-Shot Reasoning dynamically generates input-specific demonstrations under strict token budgets, outperforming traditional Retrieval-Augmented Generation (RAG) methods for few-shot reasoning. The groundbreaking C²P: Featuring Large Language Models with Causal Reasoning from University of California, Irvine equips LLMs with causal reasoning capabilities, significantly improving accuracy and demonstrating strong few-shot gains, even with as few as ten examples.

Under the Hood: Models, Datasets, & Benchmarks

Innovations in few-shot learning are often propelled by new models, datasets, and benchmarks that stress-test capabilities and provide standardized evaluation. Here’s a look at some key resources:

• GEMMA-SQL (https://arxiv.org/pdf/2511.04710): A lightweight, open-source text-to-SQL model built on the Gemma 2B architecture, evaluated extensively on the SPIDER benchmark.
• QCircuitBench (https://arxiv.org/pdf/2410.07961): The first large-scale dataset for benchmarking AI’s capability in designing quantum algorithms, spanning 3 task suites, 25 algorithms, and 120,290 data points. Code available at https://github.com/EstelYang/QCircuitBench.
• HealthQuote.NL: A corpus of 130 Dutch metaphors related to living with cancer, extracted using LLMs and human-in-the-loop methods. Code available at https://github.com/aaronlifenghan/.
• Reasoning-Guided Claim Normalization (https://arxiv.org/pdf/2511.05078): This approach leverages 5W1H decomposition for robust cross-lingual transfer in claim normalization across 20 languages. Code: https://github.com/TIFIN/reasoning-guided-claim-normalization.
• CLP-SNN (https://arxiv.org/pdf/2511.01553): A spiking neural network architecture for real-time continual learning on the Intel Loihi 2 neuromorphic chip, achieving transformative efficiency gains. Simulated SNN code on the Lava framework: https://github.com/LAAS-CNRS/lava.
• ATTBHFA-Net (https://arxiv.org/pdf/2510.18326): A novel few-shot classification framework combining spatial-channel attention with Bhattacharyya-Hellinger distances for robust prototype formation, evaluated on four FSL benchmarks and two disaster-specific datasets. Code: https://github.com/GreedYLearner1146/ABHFA-Net.
• DETree (https://arxiv.org/pdf/2510.17489): A hierarchical tree-structured representation learning framework for detecting human-AI collaborative texts, featuring RealBench, a comprehensive benchmark dataset. Code: https://github.com/heyongxin233/DETree.
• ClapperText (https://arxiv.org/pdf/2510.15557): The first benchmark dataset for OCR in historical film footage, providing over 9,813 annotated video frames for text recognition in low-resource archival documents. Code: https://github.com/linty5/ClapperText.
• ProtoTopic (https://arxiv.org/pdf/2510.13542): A framework using prototypical networks for few-shot medical topic modeling, demonstrated on real-world medical datasets. Code: https://github.com/ProtoTopic-Team/ProtoTopic.
• Persian NLP Benchmark (https://arxiv.org/pdf/2510.12807): A comprehensive benchmark of open-source LLMs for Persian, evaluated on datasets like ParsiNLU and ArmanEmo across tasks like sentiment analysis and NER. Code: https://github.com/Mofid-AI/persian-nlp-benchmark.
• SYNTRANS (https://arxiv.org/pdf/2510.11115): Uses CLIP model and Large Language Models for synergistic knowledge transfer in few-shot learning. Code: https://github.com/SMU-CAIS/SYNTRANS.
• PKD (https://arxiv.org/pdf/2510.10116): A preference-driven knowledge distillation framework synergizing LLMs and GNNs for few-shot node classification on text-attributed graphs (TAGs). Code: https://github.com/GEEX-Weixing/PKD.
• LeFCert (https://arxiv.org/pdf/2510.08659): The first provably robust few-shot classifier for Language-Empowered Foundation Models, providing certified robustness against poisoning attacks.
• Remote Sensing FSL Benchmark (https://arxiv.org/pdf/2510.07135): A reproducible benchmarking framework for evaluating remote sensing vision-language models (RSVLMs) in few-shot settings. Code: https://github.com/elkhouryk/fewshot.
• GRAD (https://arxiv.org/pdf/2510.01165): A generative retrieval-aligned demonstration sampler for efficient few-shot reasoning, leveraging an RL-trained generative model. Code: https://github.com/charafkamel/GRAD-demonstration-sampler.
• Symmetria (https://arxiv.org/pdf/2510.23414): A novel synthetic, formula-driven dataset for learning symmetries in 3D point clouds, enabling self-supervised pre-training and symmetry detection. Data and code: http://deeplearning.ge.imati.cnr.it/symmetria.

Impact & The Road Ahead

These advancements in few-shot learning promise to reshape AI development and deployment across numerous domains. In healthcare, systems like Enhancing Early Alzheimer Disease Detection through Big Data and Ensemble Few-Shot Learning and TinyViT-Batten: Few-Shot Vision Transformer for Early Batten-Disease Detection on Pediatric MRI offer hope for early and accurate diagnosis of rare diseases with limited patient data. For smart cities and transportation, Leveraging Twitter Data for Sentiment Analysis of Transit User Feedback and Exploring Dissatisfaction in Bus Route Reduction through LLM-Calibrated Agent-Based Modeling enable real-time, targeted improvements in public services based on user feedback.

The robust application of LLMs in fields like software engineering (Large Language Models for Fault Localization) and operations research (Large Language Model enabled Mathematical Modeling) demonstrates their growing versatility beyond traditional NLP. Furthermore, the focus on security and robustness, exemplified by Adaptive and Robust Data Poisoning Detection and Sanitization in Wearable IoT Systems using Large Language Models and LeFCert, addresses critical concerns for deploying AI in sensitive environments.

Looking forward, the integration of causal reasoning, frequency-space analysis, and multimodal knowledge transfer suggests a future where AI models are not only more data-efficient but also more intelligent, adaptable, and trustworthy. The emphasis on open-source models and comprehensive benchmarks signals a collaborative effort to accelerate progress, making advanced AI accessible to a broader research community. Few-shot learning is not just a technique; it’s a paradigm shift, unlocking the full potential of AI in a data-scarce world and paving the way for truly intelligent, adaptive systems.