Few-Shot Learning: Navigating the New Frontier of Data-Efficient AI

Latest 50 papers on few-shot learning: Oct. 20, 2025

Few-shot learning (FSL) stands as a pivotal challenge and a boundless opportunity in the realm of AI/ML. Imagine training sophisticated models with just a handful of examples – a feat that traditional deep learning often struggles with due to its insatiable hunger for data. This is precisely the promise of few-shot learning, and recent research is pushing the boundaries of what’s possible, tackling everything from medical diagnostics to robust AI systems. Let’s dive into some exciting breakthroughs that are shaping the future of data-efficient AI.

The Big Idea(s) & Core Innovations

At the heart of these advancements is the quest for models that can generalize from minimal examples, often by leveraging vast pre-trained knowledge or by intelligently structuring the learning process. A recurring theme is the synergistic combination of modalities and intelligent knowledge transfer. For instance, in remote sensing, Haotian Liu and colleagues from Ultralytics, Google AI, and other institutions, in their paper “Efficient Few-Shot Learning in Remote Sensing: Fusing Vision and Vision-Language Models”, propose fusing vision and vision-language models to enhance object detection in satellite imagery with minimal labeled data, proving significantly more efficient than traditional methods. This efficiency is mirrored in medical applications where data is inherently scarce. “Expert-Guided Explainable Few-Shot Learning for Medical Image Diagnosis” by Uddin, M. et al. integrates radiologist annotations into few-shot learning for improved accuracy and interpretability, especially through an explanation loss aligning Grad-CAM heatmaps with expert insights.

Bridging modalities is also key for language models. Wenhao Li and colleagues from Shandong University introduce VT-FSL in “VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning”, a framework generating cross-modal prompts using LLMs to achieve state-of-the-art performance across ten benchmarks. Similarly, Xing Wei and Chunchun Chen from Tongji University, in their “Preference-driven Knowledge Distillation for Few-shot Node Classification” paper, introduce PKD, which synergizes LLMs and GNNs for few-shot node classification on text-attributed graphs. This framework tailors knowledge transfer by selecting suitable GNNs based on node topology, outperforming methods with more labels.

Robustness and generalization are critical. Yuni Lai and co-authors from The Hong Kong Polytechnic University deliver LeFCert in “Provably Robust Adaptation for Language-Empowered Foundation Models”, a novel framework providing provable robustness guarantees for few-shot classifiers against poisoning attacks. For graph-based learning, Yonghao Liu et al. from Jilin University introduce GRACE in “Graph Few-Shot Learning via Adaptive Spectrum Experts and Cross-Set Distribution Calibration”, adapting to local structural variations and mitigating distributional shifts with node-specific filtering. This theme extends to practical industry applications, as seen in Kristian Løvland et al.’s work from Norwegian University of Science and Technology, “Multi-task and few-shot learning in virtual flow metering”, where a probabilistic, hierarchical model enables high-performance virtual flow metering with limited data from new wells.

A surprising discovery by Ji Zhang and co-authors from Southwest Jiaotong University, detailed in “From Channel Bias to Feature Redundancy: Uncovering the ‘Less is More’ Principle in Few-Shot Learning”, reveals that in few-shot scenarios, most features from pre-trained models are actually harmful due to channel bias and redundancy. They propose AFIA (Augmented Feature Importance Adjustment) to effectively reduce this redundancy, highlighting that ‘less is more’ for optimal performance. This echoes the finding by Ozan Irsoy et al. from Bloomberg in “Improving Instruct Models for Free: A Study on Partial Adaptation” that reducing instruction-tuning strength can improve few-shot in-context learning, challenging the notion that more fine-tuning is always better.

Under the Hood: Models, Datasets, & Benchmarks

The innovations highlighted above are often powered by novel architectures, specially curated datasets, and robust benchmarking frameworks. Here’s a snapshot:

• Cattle-CLIP: Huimin Liu et al. from University of Bristol introduced Cattle-CLIP, a multimodal deep learning framework for cattle behavior recognition, along with the CattleBehaviours6 dataset for six types of indoor behaviors, available via “Cattle-CLIP: A Multimodal Framework for Cattle Behaviour Recognition”.
• ProtoTopic: John Doe and Jane Smith from University of Health Sciences and National Institute of Medical Research developed ProtoTopic, leveraging prototypical networks for few-shot topic modeling in medical texts. Code is open-source at https://github.com/ProtoTopic-Team/ProtoTopic.
• Persian LLM Benchmarking: Mahdi Cherakhloo et al. from Sharif University of Technology benchmarked open-source LLMs for Persian in “Benchmarking Open-Source Large Language Models for Persian in Zero-Shot and Few-Shot Learning”, utilizing datasets like ParsiNLU and ArmanEmo. Code available at https://github.com/Mofid-AI/persian-nlp-benchmark.
• Remote Sensing Benchmarking: Youssef Elkhoury et al. from King Abdulaziz University created a reproducible framework for evaluating Remote Sensing Vision-Language Models (RSVLMs) in few-shot settings. The codebase is publicly available at https://github.com/elkhouryk/fewshot as per “Few-Shot Adaptation Benchmark for Remote Sensing Vision-Language Models”.
• MOMEMTO: Samuel Yoon et al. from Pohang University of Science and Technology introduced MOMEMTO, a patch-based memory gate model for time series anomaly detection, showcasing superior few-shot performance, detailed in “MOMEMTO: Patch-based Memory Gate Model in Time Series Foundation Model”.
• MetaChest: Berenice Montalvo-Lezama and Gibran Fuentes-Pineda from Universidad Nacional Autónoma de México presented MetaChest in “MetaChest: Generalized few-shot learning of patologies from chest X-rays”, a large-scale chest X-ray dataset (479,215 images) for few-shot pathology classification, with code at https://github.com/bereml/meta-cxr.
• RRDataset: Chunxiao Li et al. from Beijing Normal University introduced the RRDataset in “Bridging the Gap Between Ideal and Real-world Evaluation: Benchmarking AI-Generated Image Detection in Challenging Scenarios”, a comprehensive benchmark for AI-generated image detection under real-world conditions, accessible at https://zenodo.org/records/14963880.
• ICRL: Tianle Zhang et al. from Nanyang Technological University proposed In-Context Representation Learning (ICRL) for training-free integration of non-text modalities. Code for this work is available at https://github.com/ztlmememe/LLMxFM_ICRL from “Can LLMs Reason Over Non-Text Modalities in a Training-Free Manner? A Case Study with In-Context Representation Learning”.
• SYNTRANS: Hao Tang et al. from The Hong Kong Polytechnic University introduced SYNTRANS, a synergistic knowledge transfer framework for few-shot learning using CLIP and other large language models, with code at https://github.com/SMU-CAIS/SYNTRANS, as presented in “Connecting Giants: Synergistic Knowledge Transfer of Large Multimodal Models for Few-Shot Learning”.
• DiSTER: Elena Senger et al. from LMU Munich proposed DiSTER for Automatic Term Extraction (ATE) using distant supervision and LLMs, introducing the SynTerm dataset, available at https://huggingface.co/datasets/ElenaSenger/SynTerm as per “Crossing Domains without Labels: Distant Supervision for Term Extraction”.
• GRAD: Oussama Gabouj et al. from EPFL developed GRAD, a dynamic demonstration generator for few-shot reasoning, with code available at https://github.com/charafkamel/GRAD-demonstration-sampler, discussed in “GRAD: Generative Retrieval-Aligned Demonstration Sampler for Efficient Few-Shot Reasoning”.
• Sub-CP: Shaoyi Zheng et al. from New York University introduced Sub-CP, a block-aware context selection framework using submodular optimization for efficient in-context learning, detailed in “Submodular Context Partitioning and Compression for In-Context Learning-short paper”.
• ANROT-HELANet: Gao Yu Lee et al. from Nanyang Technological University introduced ANROT-HELANet for robust few-shot classification using Hellinger distance, with code at https://github.com/GreedYLearner1146/ANROT-HELANet/tree/main, from “ANROT-HELANet: Adverserially and Naturally Robust Attention-Based Aggregation Network via The Hellinger Distance for Few-Shot Classification”.

Impact & The Road Ahead

These advancements herald a new era for AI where high-performance models are no longer exclusively tied to massive, meticulously labeled datasets. The implications are profound, democratizing access to powerful AI tools for domains traditionally starved of data, such as rare disease diagnosis (“TinyViT-Batten: Few-Shot Vision Transformer with Explainable Attention for Early Batten-Disease Detection on Pediatric MRI”), specialized industrial applications (“An Advanced Convolutional Neural Network for Bearing Fault Diagnosis under Limited Data”), and personalized education (“Personalized Auto-Grading and Feedback System for Constructive Geometry Tasks Using Large Language Models on an Online Math Platform”).

The road ahead involves further enhancing robustness against adversarial attacks, refining cross-modal knowledge transfer, and developing more intelligent context-aware learning mechanisms. The ICDAR 2025 FEST competition (“ICDAR 2025 Competition on FEw-Shot Text line segmentation of ancient handwritten documents (FEST)”) exemplifies the community’s commitment to pushing these boundaries in challenging, low-resource settings. As Large Language Models become increasingly powerful, understanding their internal mechanisms, as explored in “Mechanism of Task-oriented Information Removal in In-context Learning” and “Understanding In-context Learning of Addition via Activation Subspaces”, will be crucial for building more reliable and interpretable few-shot systems.

From enabling humanoid robots to learn new manipulation tasks from human play videos (“MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videos”) to transforming transit feedback analysis through Few-Shot learning and VADER (“Leveraging Twitter Data for Sentiment Analysis of Transit User Feedback: An NLP Framework”), these breakthroughs underscore the versatility and transformative potential of few-shot learning. The future of AI is increasingly data-efficient, and these innovations are paving the way for more accessible, robust, and intelligent systems across every domain imaginable.