Few-Shot Learning: Unlocking AI’s Potential in Data-Scarce Worlds

Latest 40 papers on few-shot learning: Aug. 11, 2025

Few-shot learning (FSL) is rapidly becoming one of the most exciting frontiers in AI/ML, tackling the pervasive challenge of building robust models with minimal labeled data. Imagine an AI that can learn a new concept from just a handful of examples, much like humans do. This capability is critical for deploying AI in specialized, data-scarce domains like healthcare, scientific discovery, and industrial automation. Recent breakthroughs, as highlighted by a collection of innovative research, are pushing the boundaries of what’s possible, moving us closer to truly adaptable and efficient AI systems.

The Big Idea(s) & Core Innovations

At the heart of these advancements is the drive to imbue models with superior generalization and adaptability. One major theme revolves around leveraging foundational models and cross-modal insights. Researchers from Oregon Health & Science University, in their paper “A Foundational Multi-Modal Model for Few-Shot Learning”, propose M3F, a framework built on Large Multi-Modal Models (LMMMs) for superior generalization across diverse scientific data, proving a single LMMM on varied tasks is highly effective. Complementing this, the paper “Causal Disentanglement and Cross-Modal Alignment for Enhanced Few-Shot Learning” by authors from the Australian Institute for Machine Learning, University of Adelaide, introduces Causal CLIP Adapter (CCA) to boost FSL by disentangling features and enhancing cross-modal alignment, proving robustness to distribution shifts.

Another significant innovation focuses on optimizing existing architectures for few-shot scenarios. Harbin Institute of Technology researchers, in “Shallow Deep Learning Can Still Excel in Fine-Grained Few-Shot Learning”, challenge the notion that deeper networks are always better, presenting LCN-4, a shallow network outperforming deeper models in fine-grained FSL by meticulously handling positional information. Similarly, their work “Color as the Impetus: Transforming Few-Shot Learner” introduces ColorSense Learner and Distiller, which mimic human color perception to significantly improve FSL performance and transferability. This emphasis on biological inspiration and shallow, efficient models offers a compelling alternative to ever-larger architectures.

Several papers explore domain-specific adaptations and novel data paradigms. For instance, the University of Central Florida and University of Surrey team, in “Doodle Your Keypoints: Sketch-Based Few-Shot Keypoint Detection”, pioneers few-shot keypoint detection using sketches, enabling source-free learning and tackling data scarcity in a highly creative way. In the realm of graph data, “GraphProp: Training the Graph Foundation Models using Graph Properties” from The Chinese University of Hong Kong, Shenzhen, introduces GraphProp, which leverages graph structural properties for superior generalization across domains, especially where node features are scarce. This highlights a shift towards more robust, structure-aware learning.

Finally, the versatility of LLMs in FSL is a recurring theme. “Large Language Models as Attribution Regularizers for Efficient Model Training” by the University of Belgrade team proposes LAAT, which uses LLMs to regularize training and enhance generalization, particularly for biased or skewed datasets. Furthermore, the paper “Beyond Class Tokens: LLM-guided Dominant Property Mining for Few-shot Classification” demonstrates how LLMs can move beyond simple class tokens to identify nuanced dominant properties, improving classification with limited examples.

Under the Hood: Models, Datasets, & Benchmarks

These advancements are often coupled with innovative architectural designs, new datasets, and rigorous benchmarks that push the field forward:

• M3F & M3FD: From “A Foundational Multi-Modal Model for Few-Shot Learning”, M3F is a novel framework built on LMMMs, supported by M3FD, a multi-modal few-shot dataset with over 10K+ samples, including vision, tables, and time-course data. Code: https://github.com/ptdang1001/M3F
• LCN-4: Introduced in “Shallow Deep Learning Can Still Excel in Fine-Grained Few-Shot Learning”, LCN-4 is a shallow, location-aware constellation network specifically designed for fine-grained few-shot learning, challenging traditional deep models. Code: https://github.com/ChaofeiQI/LCN-4
• ColorSense Learner & Distiller: From “Color as the Impetus: Transforming Few-Shot Learner”, these bio-inspired frameworks leverage human color perception for enhanced FSL. Code: https://github.com/ChaofeiQI/CoSeLearner
• Causal CLIP Adapter (CCA): Featured in “Causal Disentanglement and Cross-Modal Alignment for Enhanced Few-Shot Learning”, CCA enhances CLIP models for FSL using Independent Component Analysis (ICA). Code: https://github.com/tianjiao-j/CCA
• GraphProp: This method, presented in “GraphProp: Training the Graph Foundation Models using Graph Properties”, trains graph foundation models using cross-domain structural information and graph invariants, utilizing unlabeled and synthetic graphs for scalability.
• MultiADS: “MultiADS: Defect-aware Supervision for Multi-type Anomaly Detection and Segmentation in Zero-Shot Learning” introduces MultiADS for zero-shot multi-type anomaly detection and segmentation at the pixel level, leveraging pre-trained vision-language models and a Knowledge Base for Anomalies (KBA). Code: https://github.com/boschresearch/MultiADS
• T3Time: From “T3Time: Tri-Modal Time Series Forecasting via Adaptive Multi-Head Alignment and Residual Fusion”, T3Time is a tri-modal framework integrating temporal, spectral, and prompt-based representations for time series forecasting. Code: https://github.com/monaf-chowdhury/T3Time/
• MOFS: “Multi-Operator Few-Shot Learning for Generalization Across PDE Families” introduces MOFS, a multi-modal framework for few-shot operator learning across Partial Differential Equation (PDE) families.
• UoMo: “UoMo: A Foundation Model for Mobile Traffic Forecasting with Diffusion Model” introduces UoMo, a universal foundation model for mobile traffic forecasting combining diffusion models and transformers. Code: https://github.com/tsinghua-fib-lab/UoMo
• ProtoN: “ProtoN: Prototype Node Graph Neural Network for Unconstrained Multi-Impression Ear Recognition” introduces ProtoN, a prototype node-based graph neural network for multi-impression ear recognition.
• PointKAN & PointKAN-elite: “KAN or MLP? Point Cloud Shows the Way Forward” proposes PointKAN, a Kolmogorov-Arnold Network (KAN) based architecture for point cloud analysis, with an elite version significantly reducing parameters. Code: https://github.com/Shiyan-cps/PointKAN-pytorch
• MicroMix: From “MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Models”, MicroMix is a mixed-precision quantization algorithm leveraging MX data formats for LLMs. Code: https://github.com/lwy2020/MicroMix.git
• THREAD: “THREAD: Thinking Deeper with Recursive Spawning” presents THREAD, a framework for LLMs that dynamically spawns threads for task decomposition, enhancing few-shot learning. Code: https://github.com/philipmit/thread
• DAPT: “Decouple before Align: Visual Disentanglement Enhances Prompt Tuning” introduces an architecture-free framework for prompt tuning that decouples visual information to achieve symmetric image-text alignment. Code: https://github.com/Ferenas/DAPT
• H-RDT: “H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulation” proposes a novel approach that uses human manipulation data to enhance robotic policy learning through a diffusion transformer architecture. Code: Project page for code and pretrained models.
• MP1: “MP1: MeanFlow Tames Policy Learning in 1-step for Robotic Manipulation” introduces MP1, a MeanFlow-based framework for one-step policy learning in robotics, with a lightweight Dispersive Loss for few-shot generalization. Code: https://github.com/LogSSim/MP1.git
• FedVLM: “FedVLM: Scalable Personalized Vision-Language Models through Federated Learning” presents FedVLM, a federated learning framework for VLMs using personalized LoRA (pLoRA) for decentralized adaptation in non-iid settings.
• GLAD: “GLAD: Generalizable Tuning for Vision-Language Models” introduces GLAD, a parameter-efficient fine-tuning framework that improves VLM generalization in few-shot learning using LoRA with gradient-based regularization.
• CPLSR: “Cross-Domain Few-Shot Learning with Coalescent Projections and Latent Space Reservation” introduces CPLSR, combining Coalescent Projections (CP) and a pseudo-class generation method for Cross-Domain Few-Shot Learning (CD-FSL). Code: https://github.com/Naeem-Paeedeh/CPLSR
• LoopNet & LoopDB: “LoopNet: A Multitasking Few-Shot Learning Approach for Loop Closure in Large Scale SLAM” introduces LoopNet for loop closure detection in SLAM, along with a new benchmarking dataset called LoopDB. Code: https://github.com/RovisLab/LoopNet
• CodeMixEval: “Evaluating Code-Mixing in LLMs Across 18 Languages” introduces CodeMixEval, a comprehensive framework for evaluating LLM performance on code-mixed data, highlighting underperformance and proposing synthesis methods.
• FMC Dataset: “FMC: Formalization of Natural Language Mathematical Competition Problems” curates a high-quality dataset of Olympiad-level mathematical problems (natural language-Lean pairs) for automated theorem provers. Code: https://github.com/JadeXie1205/FMC

Impact & The Road Ahead

The collective impact of these research efforts is profound. We’re seeing a fundamental shift in how AI learns, moving from data-hungry paradigms to more efficient, human-like adaptability. This has immediate implications for real-world applications: from faster and more accurate anomaly detection in industrial settings (MultiADS) to robust mobile traffic forecasting (UoMo), and even more intuitive time series editing via natural language (InstructTime). The ability to perform few-shot learning across diverse data types, including multimodal (M3F), graph (GraphProp), and even sketches (Doodle Your Keypoints), signifies a significant leap towards more versatile AI.

Looking ahead, several exciting directions emerge. The ongoing exploration of LLMs for tasks beyond natural language processing, such as attribution regularization (LAAT) or guiding few-shot classification (Beyond Class Tokens), suggests a future where these powerful models serve as meta-learners. Addressing vulnerabilities in LLMs to novel input formats like vertical text (as highlighted in “Vulnerability of LLMs to Vertically Aligned Text Manipulations”) and improving their consistency in complex domains like Cyber Threat Intelligence (as revealed in “Large Language Models are Unreliable for Cyber Threat Intelligence”) will be crucial. Furthermore, the push for parameter-efficient fine-tuning (GLAD) and leveraging human manipulation priors for robotics (H-RDT, MP1) points to a future of more practical, deployable, and generalizable AI systems. The future of AI is not just about scale, but about smart, efficient, and adaptable learning, and few-shot learning is leading the charge.