Segment Anything Model: Pioneering the Next Generation of Vision AI

Latest 50 papers on segment anything model: Dec. 21, 2025

The Segment Anything Model (SAM) has rapidly emerged as a transformative force in computer vision, offering unprecedented capabilities in object segmentation. Its ability to “segment anything” with simple prompts has ignited a wave of innovation, pushing the boundaries of what’s possible in diverse applications, from medical imaging to remote sensing and beyond. Recent research has focused on enhancing SAM’s efficiency, adaptability, and conceptual understanding, addressing its limitations and expanding its utility across specialized and real-world scenarios.### The Big Idea(s) & Core Innovationsits core, the latest advancements in the SAM ecosystem revolve around deepening its understanding of context and efficiency in deployment. The paradigm shift from SAM2’s prompt-based segmentation to SAM3’s concept-driven, vision-language fusion is a monumental leap. As explored by Ranjan Sapkota et al. from Cornell University in their paper, “The SAM2-to-SAM3 Gap in the Segment Anything Model Family: Why Prompt-Based Expertise Fails in Concept-Driven Image Segmentation”, SAM3 fundamentally changes the optimization objectives by integrating multimodal alignment and semantic grounding. This allows SAM3 (and its enhanced variant, SAM3-I, proposed by Jingjing Li et al. from University of Alberta in “SAM3-I: Segment Anything with Instructions”) to interpret complex natural language instructions, grounding intended instances directly without sacrificing concept-driven capabilities.is another crucial theme. Papers like “SAMCL: Empowering SAM to Continually Learn from Dynamic Domains with Extreme Storage Efficiency” by Zeqing Wang et al. from Xidian University introduce novel continual learning methods, significantly reducing catastrophic forgetting and storage costs for dynamic domain adaptation. For medical applications, Xiaoqing Qiu and Zhenghao Li from The Hong Kong University of Science and Technology (HKUST) present “UniUltra: Interactive Parameter-Efficient SAM2 for Universal Ultrasound Segmentation”, drastically reducing SAM2’s parameter count by over 94% for practical clinical deployment. Similarly, Nicola Farronato et al. from IBM Research Zurich in “Q-SAM2: Accurate Quantization for Segment Anything Model 2” demonstrate high accuracy for SAM2 even at ultra-low bit-widths, enabling deployment on resource-constrained devices.general enhancements, SAM is being specialized. Zhiguo Lu et al. from Zhejiang University introduce “3DTeethSAM: Taming SAM2 for 3D Teeth Segmentation”, achieving state-of-the-art 3D teeth segmentation by adapting SAM2 with lightweight modules. For remote sensing, Kaiyu Li et al. from Xi’an Jiaotong University in “SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images” showcase SAM3’s potential for training-free open-vocabulary semantic segmentation using mask fusion and presence score filtering. Moreover, S. S. Tary et al. in “A Unified Framework with Multimodal Fine-tuning for Remote Sensing Semantic Segmentation” propose a unified framework that improves accuracy by integrating multiple modalities with techniques like Adapter and LoRA.SAM is also being weaponized against image forgery. Qi Song et al. from Hong Kong Baptist University in “Creating Blank Canvas Against AI-enabled Image Forgery” introduce a “blank canvas” approach using adversarial perturbations and frequency-aware optimization to make tampered content more detectable.### Under the Hood: Models, Datasets, & Benchmarksinnovations are powered by new models, datasets, and refined evaluation strategies:SAM3 & SAM3-I: Meta AI’s “SAM 3: Segment Anything with Concepts” and the instruction-aware “SAM3-I: Segment Anything with Instructions” represent the vanguard of concept-driven, instruction-following segmentation, built upon multimodal large language models (MLLMs).EfficientSAM3: Chengxi Simon Zeng et al. from University of Bristol introduce “EfficientSAM3: Progressive Hierarchical Distillation for Video Concept Segmentation from SAM1, 2, and 3”, a family of distilled models for on-device video concept segmentation, with a code repository likely at https://github.com/ultralytics/.Medical Adapters & Datasets:MedSAM3, by Anglin Liu et al. from The Hong Kong University of Science and Technology (Guangzhou), detailed in “MedSAM3: Delving into Segment Anything with Medical Concepts”, uses semantic guidance and an agentic framework for diverse medical imaging modalities (Code).UltraSam (Paper, Code) from Amélie Meyer et al. leverages US-43d, the largest public ultrasound segmentation dataset, for a robust ultrasound foundation model.Z. Gong and X. Chen from University of Nottingham introduce SSL-MedSAM2 in “SSL-MedSAM2: A Semi-supervised Medical Image Segmentation Framework Powered by Few-shot Learning of SAM2” for semi-supervised medical image segmentation using few-shot learning (Code).GBT-SAM (Paper, Code) by Cecilia Diana-Albelda et al. from Universidad Autónoma de Madrid focuses on parameter-efficient brain tumor segmentation in mp-MRI.Jialun Pei et al.’s “Synergistic Bleeding Region and Point Detection in Laparoscopic Surgical Videos” introduces BlooDet and the SurgBlood dataset for real-time bleeding detection in laparoscopic surgery.SAMora (Paper, Code) by Shuhang Chen et al. from Zhejiang University enhances SAM via hierarchical self-supervised pre-training for medical images.Remote Sensing Tools:ELE-SAM (Paper, Code) by Hang Chen et al. from Wuhan University for power transmission corridor hazard segmentation, featuring the ELE-40K dataset.ReSAM (Paper) by M.Naseer Subhani uses a self-prompting framework for point-supervised remote sensing image segmentation.Futian Wang et al. from Anhui University present a framework for “SAM Guided Semantic and Motion Changed Region Mining for Remote Sensing Change Captioning” using SAM to identify changes and generate descriptions (Code).Quantization and Efficiency:SAQ-SAM (Paper, Code) by Jing Zhang et al. from Chinese Academy of Sciences offers semantically-aligned post-training quantization for SAM.X. Xiong and WZH0120 introduce SAM3-UNet in “SAM3-UNet: Simplified Adaptation of Segment Anything Model 3” for efficient fine-tuning.Other Noteworthy Innovations: SD-MVS (Paper) by Zhenlong Yuan et al. from Institute of Computing Technology, Chinese Academy of Sciences introduces SAM for semantic instance distinction in 3D reconstruction. Zihao Ding et al. from Rutgers University present “A Distributed Framework for Privacy-Enhanced Vision Transformers on the Edge”, leveraging SAM for privacy-preserving distributed vision tasks. Shweta Mahajan et al. from Qualcomm AI Research in “Attention Guided Alignment in Efficient Vision-Language Models” utilize SAM’s spatial knowledge to reduce object hallucination in efficient vision-language models.### Impact & The Road Aheadcollective intelligence embodied in these SAM-centric advancements is transformative. We are moving towards a future where segmentation is not merely pixel-level delineation but a deeply contextual, instruction-driven, and highly efficient process. The ability of models to understand concepts (SAM3), follow instructions (SAM3-I), and continually adapt with minimal resources (SAMCL, UniUltra) unlocks unprecedented possibilities in real-world deployments. From enhancing robotic surgery through better 3D perception (“More than Segmentation: Benchmarking SAM 3 for Segmentation, 3D Perception, and Reconstruction in Robotic Surgery” by W. Dong et al.) and real-time object tracking in sports (“Team-Aware Football Player Tracking with SAM: An Appearance-Based Approach to Occlusion Recovery” by Chamath Ranasinghe), to revolutionizing medical diagnostics and securing digital imagery, SAM’s impact is broad and profound. The focus on parameter efficiency and quantization will further democratize access to these powerful models, enabling their use on edge devices and in resource-constrained environments. However, challenges remain, particularly in SAM’s struggle with tree-like and low-contrast objects, as highlighted by Yixin Zhang et al. from Duke University in “Quantifying the Limits of Segmentation Foundation Models: Modeling Challenges in Segmenting Tree-Like and Low-Contrast Objects”, indicating a need for further architectural innovation. Nevertheless, the rapid evolution of the Segment Anything Model family paints a vivid picture of a future where AI vision is more intelligent, adaptable, and integrated into our daily lives than ever before.