Few-Shot Learning: Navigating the Data Desert with AI’s Latest Breakthroughs

Latest 50 papers on few-shot learning: Dec. 13, 2025

In the rapidly evolving landscape of AI and Machine Learning, few-shot learning (FSL) stands as a monumental challenge and an area of intense innovation. Imagine training a powerful AI model with only a handful of examples – a task that traditional deep learning, ravenous for data, finds nearly impossible. Yet, the real world often offers scarcity, especially in specialized domains like medical imaging, remote sensing, or emerging product categories. Recent research showcases incredible strides in making AI models learn effectively from minimal data, pushing the boundaries of what’s possible.

The Big Idea(s) & Core Innovations:

This wave of innovation is defined by clever strategies to overcome data scarcity, from leveraging pre-trained models to infusing domain-specific knowledge and novel architectural designs. A prominent theme is the ingenious use of Large Language Models (LLMs), not just for text, but for diverse tasks. For instance, in “LLM Meeting Decision Trees on Tabular Data” by Hangting Ye et al. from Jilin University and CSIRO’s Data61, DeLTa integrates LLMs with decision trees to enhance tabular data prediction, notably by generating rules and avoiding cumbersome data serialization. This highlights LLMs’ potential as powerful rule generators rather than mere text processors. Similarly, “In-Context and Few-Shots Learning for Forecasting Time Series Data based on Large Language Models” by Saroj Gopali et al. from Texas Tech University and Johns Hopkins University demonstrates that LLMs like TimesFM excel in time series forecasting, outperforming traditional methods with minimal training.

Another critical advancement addresses biases and generalization. Zhenyu Zhang et al. from Huazhong University of Science and Technology and Peking University, in their paper “Decoupling Template Bias in CLIP: Harnessing Empty Prompts for Enhanced Few-Shot Learning”, introduce ‘empty prompts’ to mitigate template bias in CLIP, significantly boosting few-shot classification. This underscores the subtle yet profound impact of prompt engineering. The challenge of domain shift in FSL is tackled head-on by “Free Lunch to Meet the Gap: Intermediate Domain Reconstruction for Cross-Domain Few-Shot Learning” by Tong Zhang et al., which uses Intermediate Domain Proxies to bridge source and target domains, enabling rapid adaptation without extensive labeled data. Furthermore, Siqi Hu et al. from National Key Laboratory of Human-Machine Hybrid Augmented Intelligence introduce FreqGRL in “FreqGRL: Suppressing Low-Frequency Bias and Mining High-Frequency Knowledge for Cross-Domain Few-Shot Learning”, a framework that uses frequency-space analysis to combat low-frequency bias, enhancing generalization.

Medical and specialized imaging tasks also see significant breakthroughs. “PathCo-LatticE: Pathology-Constrained Lattice-Of Experts Framework for Fully-supervised Few-Shot Cardiac MRI Segmentation” by Author A et al. from Institution X, leverages pathology constraints and a lattice-of-experts architecture to dramatically improve cardiac MRI segmentation in low-data scenarios. “Balanced Few-Shot Episodic Learning for Accurate Retinal Disease Diagnosis” by Jasmaine Khale and Ravi Prakash Srivastava from Northeastern University employs balanced episodic sampling and CLAHE augmentation to enhance diagnosis of rare retinal diseases, tackling class imbalance.

Even in the realm of AI security, few-shot learning is proving vital. “SELF: A Robust Singular Value and Eigenvalue Approach for LLM Fingerprinting” by Hanxiu Zhang and Yue Zheng from The Chinese University of Hong Kong, Shenzhen, proposes a weight-based fingerprinting method for LLMs, resistant to false claim attacks, leveraging singular values and eigenvalues. This is critical for IP protection in the age of generative AI. Addressing model safety during fine-tuning, Thong Bach et al. from Deakin University introduce a “Curvature-Aware Safety Restoration In LLMs Fine-Tuning” method, using influence functions to restore safety alignment without sacrificing performance, preserving few-shot generalization.

Under the Hood: Models, Datasets, & Benchmarks:

These innovations are often built upon or necessitate novel models, datasets, and benchmarks:

• SWIFT: A method from Texas A&M University for Semi-Supervised Few-Shot Learning that enhances auto-annotation tasks by addressing ‘flat’ softmax distributions in VLMs. (Code: https://github.com/OSU)
• DeLTa: Integrates LLMs and decision trees for tabular data prediction, showing state-of-the-art results on diverse tabular benchmarks. (Code: https://github.com/HangtingYe/DeLTa)
• Domain-RAG: A training-free framework from Fudan University for Cross-Domain Few-Shot Object Detection, creating domain-consistent synthetic data. (Code: https://github.com/LiYu0524/Domain-RAG)
• MedIMeta: A comprehensive multi-domain, multi-task meta-dataset from University of Tübingen for medical imaging, supporting CD-FSL across 19 datasets and 54 tasks. (Code: https://github.com/StefanoWoerner/medimeta-pytorch)
• Logos Dataset: The largest Russian Sign Language dataset from SberAI, crucial for cross-language transfer learning in Sign Language Recognition. (Code: https://paperswithcode.com/)
• QCircuitBench: The first large-scale dataset from Peking University for benchmarking AI’s capability in quantum algorithm design, with 120,290 data points. (Code: https://github.com/EstelYang/QCircuitBench)
• TS-HINT: A time series foundation model from SUTD, Singapore, that integrates LLM reasoning with attention hints for semiconductor manufacturing prediction, demonstrating data efficiency. (Paper: https://arxiv.org/pdf/2512.05419)
• Strada-LLM: A graph LLM for spatio-temporal traffic prediction from Vrije Universiteit Brussel, explicitly modeling temporal and spatial patterns using a graph structure. (Paper: https://arxiv.org/pdf/2410.20856)
• ABounD: A framework from Nanjing University for few-shot multi-class anomaly detection, combining Dynamic Concept Fusion (DCF) and Adversarial Boundary Forging (ABF) on datasets like MVTec-AD. (Code: https://github.com/ABounD)
• VarCon: From University of Illinois, Urbana-Champaign, a variational supervised contrastive learning framework achieving SOTA on ImageNet and CIFAR with fewer epochs, showing robustness in few-shot settings. (Code: https://github.com/ziwenwang28/VarContrast)
• PHSD dataset and Human0 model: Introduced in “In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Data” by Xiongyi Cai et al. from UC San Diego, these resources facilitate large-scale pre-training for egocentric robot manipulation.

Impact & The Road Ahead:

The cumulative impact of these advancements is profound. We’re seeing AI systems becoming more adaptable, efficient, and robust, particularly in data-scarce environments. This empowers critical applications in medical diagnosis, environmental monitoring (e.g., bark beetle detection with “Detection of Bark Beetle Attacks using Hyperspectral PRISMA Data and Few-Shot Learning”), and industrial quality control. The intelligent integration of LLMs with other modalities, coupled with innovative architectures, is leading to models that not only understand context but also reason causally, as seen in “C²P: Featuring Large Language Models with Causal Reasoning” by Abdolmahdi Bagheri et al. from University of California, Irvine.

Looking ahead, the emphasis will likely shift towards greater transparency and human-AI collaboration. Projects like “Complementary Learning Approach for Text Classification using Large Language Models” by Navid Asgari and Benjamin M. Cole from Fordham University and “Dutch Metaphor Extraction from Cancer Patients’ Interviews and Forum Data using LLMs and Human in the Loop” by Lifeng Han et al. highlight the growing importance of human expertise in refining LLM outputs, especially in sensitive domains like healthcare. The development of robust defense mechanisms against prompt injection, such as “Semantics as a Shield: Label Disguise Defense (LDD) against Prompt Injection in LLM Sentiment Classification” by Ayub, A. et al. from University of California, Berkeley, is crucial for deploying these powerful models safely. As we continue to refine few-shot learning techniques, AI is poised to tackle increasingly complex, real-world problems with unprecedented agility and minimal data overhead, truly democratizing advanced AI capabilities.