Foundation Models: Reshaping Reality – From Virtual Humans to Medical Diagnostics and Autonomous Systems

Latest 100 papers on foundation models: Apr. 4, 2026

The world of AI/ML is abuzz with the transformative power of foundation models, which are rapidly reshaping how we interact with digital content, analyze complex data, and build autonomous systems. These large, pre-trained models are proving to be remarkably versatile, pushing the boundaries of what’s possible in diverse fields, often with surprising efficiency. Recent research delves into cutting-edge applications, from generating ultra-realistic digital humans and robust video content to enhancing medical imaging and guiding autonomous vehicles, all while tackling crucial challenges like data scarcity, computational cost, and ethical considerations.

The Big Idea(s) & Core Innovations

The overarching theme across recent breakthroughs is the ingenious adaptation and enhancement of these powerful foundation models to address previously intractable problems. A key innovation lies in resolving the tension between generalization and fidelity. For instance, researchers at Codec Avatars Lab, Meta, in their paper, “Large-scale Codec Avatars: The Unreasonable Effectiveness of Large-scale Avatar Pretraining”, introduce a novel pre/post-training paradigm. By pre-training on a million ‘in-the-wild’ videos and then post-training on high-quality studio data, they achieve photorealistic, fully animatable avatars that generalize robustly across diverse demographics, even demonstrating emergent capabilities like handling loose garments without explicit supervision.

Another significant thrust is making foundation models ‘smarter’ and more adaptable for specific, complex domains, often without extensive retraining. The “Bridging Large-Model Reasoning and Real-Time Control via Agentic Fast-Slow Planning” framework, from a team including E. Li and M. Tomizuka, tackles autonomous driving by dynamically balancing a ‘slow’ large model for high-level reasoning with a ‘fast’ controller for real-time execution, leading to up to a 45% reduction in lateral deviation. This reflects a broader trend seen in “Stop Wandering: Efficient Vision-Language Navigation via Metacognitive Reasoning” by Xueying Li et al. from Central South University, where a training-free agent, MetaNav, uses an LLM for metacognitive reasoning to self-diagnose and correct inefficient exploration, reducing VLM queries by over 20%.

In the realm of medical AI, models are being refined for greater precision and interpretability. “CheXOne: A Reasoning-Enabled Vision-Language Foundation Model for Chest X-ray Interpretation” by Yabin Zhang et al. from Stanford University, demonstrates that a VLM can not only diagnose but also generate clinically grounded reasoning traces, matching or exceeding resident-level reports in 55% of cases. Similarly, “Curia-2: Scaling Self-Supervised Learning for Radiology Foundation Models”, supported by EuroHPC Joint Undertaking, presents a refined pre-training recipe that enables vision-only models to compete with vision-language models for complex findings detection, establishing new state-of-the-art performance.

Efficiency and robustness are also critical. “EventHub: Data Factory for Generalizable Event-Based Stereo Networks without Active Sensors” by Luca Bartolomei et al., introduces a framework to train event-based stereo networks using synthetic data from RGB images, completely removing the need for costly active sensors like LiDAR and improving generalization by up to 50%. On the generative side, “From Understanding to Erasing: Towards Complete and Stable Video Object Removal” by D. Liu et al. from WeChatCV, tackles the persistent problem of shadows and reflections in video object removal by integrating external knowledge distillation from vision foundation models, making erasures truly complete and spatio-temporally consistent.

Finally, the research highlights a strong push towards parameter-efficient adaptation and training-free approaches. “AdaLoRA-QAT: Adaptive Low-Rank and Quantization-Aware Segmentation” by Prantik Deb et al., combines low-rank adaptation with quantization-aware training to compress foundation models for Chest X-ray segmentation by 2.24x while maintaining high accuracy, crucial for edge deployment. Similarly, “INSID3: Training-Free In-Context Segmentation with DINOv3” by Claudia Cuttano et al., astonishingly achieves state-of-the-art in-context segmentation using only a frozen DINOv3 backbone, demonstrating that powerful self-supervised features can lead to sophisticated capabilities without any task-specific training or auxiliary models.

Under the Hood: Models, Datasets, & Benchmarks

Recent advancements are underpinned by a combination of novel architectures, creative data generation strategies, and robust benchmarks:

• Large-Scale Codec Avatars (LCA): Leverages implicit 3D Gaussians for scalable architecture and is pre-trained on one million in-the-wild videos, followed by post-training on curated studio data.
• EventHub: Utilizes neural rendering data generation and cross-modal distillation from existing RGB foundation models to train event stereo networks without LiDAR. Resources and code are available at https://bartn8.github.io/eventhub.
• MetaNav: A training-free agent using LLMs for reflective correction and spatial memory, evaluated on GOAT-Bench, HM3D-OVON, and A-EQA benchmarks.
• Modular Energy Steering: Repurposes off-the-shelf vision-language foundation models like CLIP as semantic energy estimators for inference-time safety control in text-to-image generation. Paper available at https://arxiv.org/pdf/2604.02265.
• Large-Scale Codec Avatars: Pre-training on one million in-the-wild videos and fine-tuning on high-quality studio data demonstrates a novel pre/post-training paradigm. More info at https://junxuan-li.github.io/lca.
• Prior2DSM: A training-free framework for height completion using DINOv3 and monocular depth estimators with Low-Rank Adaptation (LoRA), achieving 46% RMSE reduction. Paper available at https://arxiv.org/pdf/2604.02009.
• Curia-2: A refined pre-training recipe for ViT-B to ViT-L radiology foundation models, achieving new SOTA in vision-focused tasks. Paper available at https://arxiv.org/pdf/2604.01987.
• GeoAI Agency Primitives: Proposes a conceptual framework for GIS with nine agency primitives and a new benchmarking framework focusing on human productivity. Paper at https://arxiv.org/pdf/2604.01869.
• DPMO-RPS: Leverages Segment Anything Model (SAM) with Nearest Neighbor Exclusive Circle constraints and Reinforced Point Selection (RPS) for crowd instance segmentation. Evaluated on ShanghaiTech, UCF-QNRF, JHU-Crowd++, and NWPU-Crowd datasets. Paper at https://arxiv.org/pdf/2604.01742.
• From Understanding to Erasing: Integrates external knowledge distillation from vision foundation models and an internal framewise context cross-attention mechanism. Code available at https://github.com/WeChatCV/UnderEraser.
• Agentic Fast-Slow Planning (AFSP): Integrates large foundation models with real-time control for autonomous driving, showing improved lateral deviation and completion time. Code: https://github.com/cjychenjiayi/icra2026_AFSP.
• Automatic Image-Level Morphological Trait Annotation: Combines Sparse Autoencoders (SAEs) as part-detectors with Multimodal Large Language Models (MLLMs) to create BIOSCAN-TRAITS dataset (80K annotations across 19K insect images). Code available at https://github.com/OSU-NLP-Group/sae-trait-annotation.
• ProdCodeBench: A benchmark curated from real-world production sessions for evaluating AI coding agents in industrial monorepos. Paper at https://arxiv.org/pdf/2604.01527.
• AffordTissue: A multimodal framework predicting dense affordance heatmaps using language prompts and video sequences with image diffusion techniques. Paper at https://arxiv.org/pdf/2604.01371.
• TEDDY: A family of transformer-based foundation models trained on 116 million single-cell RNA sequencing cells for zero-shot disease classification, using CELLXGENE data. Paper at https://arxiv.org/pdf/2503.03485.
• AdaLoRA-QAT: A two-stage framework combining adaptive low-rank adaptation with quantization-aware training for Chest X-ray segmentation. Code: https://prantik-pdeb.github.io/adaloraqat.github.io/.
• TRACE: A training-free partial audio deepfake detection framework using embedding trajectory analysis of frozen speech foundation models like WavLM-Large. Paper at https://arxiv.org/pdf/2604.01083.
• ONE-SHOT: A parameter-efficient framework for compositional human-environment video synthesis using spatial-decoupled motion injection and hybrid context integration. Code and more at https://martayang.github.io/.
• CL-VISTA: A novel benchmark for Continual Learning in Video Large Language Models (Video-LLMs), exposing catastrophic forgetting. Dataset and code: https://huggingface.co/datasets/MLLM-CL/CL-VISTA and https://github.com/Ghy0501/MCITlib.
• TF-SSD: A training-free framework for Co-salient Object Detection that synergizes SAM and DINO. Code: https://github.com/hzz-yy/TF-SSD.
• CheXOne: A reasoning-enabled vision-language model for chest X-ray interpretation, trained on CheXinstruct-v2 and CheXReason datasets (14.7 million samples). Code: https://github.com/YBZh/CheXOne.
• Mine-JEPA: An in-domain self-supervised learning pipeline for side-scan sonar mine classification, outperforming DINOv3 with only 1,170 unlabeled images using SIGReg regularization. Paper at https://arxiv.org/pdf/2604.00383.
• Collaborative AI Agents and Critics: A federated multi-agent system leveraging classical ML (XG Boosting) and Generative AI (Llama3.2, Mistral) for network telemetry fault detection. Paper at https://arxiv.org/pdf/2604.00319.
• EASe: An unsupervised semantic segmentation framework using attention-guided upsampling (SAUCE) and a training-free aggregator (CAFE) to overcome coarse-resolution limitations. Code: https://ease-project.github.io/.
• UCell: A small-scale recursive vision transformer (10-30M parameters) for single-cell segmentation, outperforming larger FMs without natural image pretraining. Code: https://github.com/jiyuuchc/ucell.
• Terminal Agents Suffice: Demonstrates terminal-based agents interacting directly with APIs outperform complex GUI agents in enterprise automation. Paper at https://arxiv.org/pdf/2604.00073.
• Scaling Video Pretraining for Surgical Foundation Models: Introduces SurgRec-MAE and SurgRec-JEPA trained on a 214 million surgical video frame corpus. Paper at https://arxiv.org/pdf/2603.29966.
• ShapPFN: A novel tabular foundation model integrating Shapley value regression directly for real-time explanations, achieving 1000x speedup over KernelSHAP. Code: https://github.com/kunumi/ShapPFN.
• ScoringBench: A benchmark for tabular foundation models using proper scoring rules for distributional regression. Live leaderboard at https://scoringbench.bolt.host/, code at https://github.com/jonaslandsgesell/ScoringBench.
• Task Scarcity and Label Leakage: Proposes K-Space architecture with gradient projection method to mitigate label leakage in relational transfer learning. Paper at https://arxiv.org/pdf/2603.29914.
• CADReasoner: Iteratively refines parametric CAD models by self-editing CadQuery programs based on geometric discrepancies. Code: GitHub repository for CADReasoner and Hugging Face model page.
• M-MiniGPT4: A multilingual Vision Large Language Model aligned via translated data and parallel text corpora across 11 languages. Paper at https://arxiv.org/pdf/2603.29467.
• EarthEmbeddingExplorer: A web application for cross-modal retrieval of global satellite images, integrating FarSLIP, SigLIP, DINOv2, and SatCLIP. Access at https://modelscope.ai/studios/Major-TOM/EarthEmbeddingExplorer.
• TriDerm: Multimodal framework for chronic wound assessment, adapting foundation models using expert ordinal triplet judgments and LLM simulations. Paper at https://arxiv.org/pdf/2603.29376.
• StereoVGGT: A training-free Visual Geometry Transformer for stereo vision leveraging frozen VGGT weights and an entropy-based optimization strategy. Paper at https://arxiv.org/pdf/2603.29368.
• AEC-Bench: A multimodal benchmark for agentic systems in Architecture, Engineering, and Construction, evaluating visual grounding and cross-document coordination. Code: https://github.com/nomic-ai/aec-bench.
• Segmentation of Gray Matters and White Matters from Brain MRI data: Modifies MedSAM for multi-class segmentation of brain tissues. Paper at https://arxiv.org/pdf/2603.29171.
• Drop the Hierarchy and Roles: Self-organizing LLM agents outperform designed structures in multi-agent systems. Paper at https://arxiv.org/pdf/2603.28990.
• A Computational Framework for Cross-Domain Mission Design: Uses Llama-3.3-70B for onboard cognitive decision support in distributed autonomous systems. Paper at https://arxiv.org/pdf/2603.28926.
• Fisheye3R: Adapts unified 3D feed-forward foundation models to fisheye lenses using trainable calibration tokens and masked attention. Paper at https://arxiv.org/pdf/2603.28896.
• OneComp: An open-source package for automating generative AI model compression, dynamically selecting quantization strategies. Code: https://github.com/FujitsuResearch/OneCompression.
• Generalizable Foundation Models for Calorimetry: Uses Mixture-of-Experts (MoE) and Parameter Efficient Fine-Tuning (PEFT) with LoRA for particle physics simulations. Code: https://github.com/wmdataphys/FM4CAL.
• VeoPlace: Leverages pre-trained Vision-Language Models (VLMs) for chip floorplanning via evolutionary optimization, achieving significant wirelength reductions. Paper at https://arxiv.org/pdf/2603.28733.
• EdgeDiT: Hardware-aware diffusion transformers optimized for mobile NPUs (Qualcomm Hexagon, Apple ANE) for efficient on-device image generation. Paper at https://arxiv.org/pdf/2603.28405.
• PReD: The first foundation model unifying electromagnetic (EM) perception, recognition, and decision-making within a multimodal LLM framework, trained on PReD-1.3M dataset. Paper at https://arxiv.org/pdf/2603.28183.
• RecycleLoRA: A dual-adapter design using Rank-Revealing QR decomposition for domain generalized semantic segmentation. Code: https://github.com/chanseul01/RecycleLoRA.git.
• Can Unsupervised Segmentation Reduce Annotation Costs: Investigates using SAM and SAM 2 for pseudo-label generation in video semantic segmentation. Paper at https://arxiv.org/pdf/2603.27697.
• CrossHGL: A text-free foundation model for cross-domain heterogeneous graph learning, relying solely on structural information. Paper at https://arxiv.org/abs/2603.27685.
• OpenDPR: A training-free vision-centric diffusion-guided prototype retrieval framework for open-vocabulary change detection in remote sensing. Code: https://github.com/guoqi2002/OpenDPR.
• SPROUT: A pixel-space diffusion transformer (UDiT) foundation model for agricultural vision, trained on 2.6 million diverse agricultural images. Code: https://github.com/UTokyo-FieldPhenomics-Lab/SPROUT.
• Transferring Physical Priors into Remote Sensing Segmentation via Large Language Models: Uses LLMs to extract physical constraints into a Knowledge Graph, refining frozen foundation models with PriorSeg. Paper at https://arxiv.org/pdf/2603.27504.
• Project Imaging-X: A survey of 1000+ open-access medical imaging datasets for foundation model development, introducing a Metadata-Driven Fusion Paradigm (MDFP). Code: https://github.com/uni-medical/Project-Imaging-X.
• Active In-Context Learning for Tabular Foundation Models (AICL): Combines in-context learning and active learning for efficient training on tabular data. Paper at https://arxiv.org/pdf/2603.27385.
• EpochX: A decentralized marketplace infrastructure for human-AI agent collaboration with a credits-based economy. Code: https://github.com/EpochX.
• From Foundation ECG Models to NISQ Learners: Distills ECGFounder into compact classical and quantum-ready student models (VQC) using knowledge distillation. Paper at https://arxiv.org/pdf/2603.27269.
• PRUE: A U-Net-based segmentation model with targeted data augmentations and composite loss functions for agricultural field boundaries. Code: https://github.com/fieldsoftheworld/ftw-prue.
• ChartNet: A million-scale multimodal dataset for robust chart understanding, generated via a code-guided synthesis pipeline. Dataset: https://huggingface.co/datasets/ibm-granite/ChartNet.
• MOOZY: A patient-first foundation model for computational pathology, learning whole-slide image representations with explicit inter-slide dependency modeling. Paper at https://arxiv.org/pdf/2603.27048.
• ROSClaw: An open-source framework for agentic robot control and interaction using ROS 2 and LLMs. Code for OpenClaw: https://github.com/openclaw/openclaw.
• AVAPrintDB: A multi-generator photorealistic talking-head public database and benchmark for avatar fingerprinting, using DINOv2 and CLIP. Code: https://github.com/BiDAlab/AVAPrintDB.
• VAN-AD: Integrates Visual Masked Autoencoders (ViT-based) with Normalizing Flows for time series anomaly detection. Code: https://github.com/PenyChen/VAN-AD.
• Survey on Remote Sensing Scene Classification: Highlights generative AI techniques (GANs, Diffusion models) for synthetic data generation and addressing annotation costs. Paper at https://arxiv.org/pdf/2603.26751.
• FEMBA on the Edge: A bidirectional Mamba-based EEG foundation model with physiologically-aware pre-training and QAT for ultra-low-power microcontrollers. Code: https://github.com/pulp-bio/BioFoundation.
• Lingshu-Cell: A masked discrete diffusion framework for single-cell RNA sequencing data to simulate realistic cellular states and predict perturbations. Paper at https://arxiv.org/pdf/2603.25240.
• OpenAVS: A training-free framework for open-vocabulary audio-visual segmentation using foundational models. Paper at https://arxiv.org/abs/2505.01448.
• GeoSR: A framework integrating geometric cues into VLMs for enhanced spatial reasoning, with Geometry-Unleashing Masking and Geometry-Guided Fusion. Code: https://suhzhang.github.io/GeoSR/.
• Benchmarking Tabular Foundation Models for Conditional Density Estimation: Evaluates TabPFN and TabICL on 39 real-world datasets. Paper at https://arxiv.org/pdf/2603.26611.
• VGGRPO: A latent-space reinforcement learning framework for world-consistent video generation using a Latent Geometry Model (LGM). Paper at https://arxiv.org/pdf/2603.26599.
• Generation Is Compression: Introduces Generative Video Codec (GVC), repurposing pretrained video generative models as zero-shot compression engines via Stochastic Rectified Flow. Paper at https://arxiv.org/pdf/2603.26571.
• LAMAE: A multi-lead masked autoencoder foundation model for ECG time series using latent attention to model cross-lead dependencies. Paper at https://arxiv.org/pdf/2603.26475.
• From Human Cognition to Neural Activations: Investigates spatial reasoning in LLMs, revealing ‘mechanistic degeneracy’ and fragmented internal representations. Paper at https://arxiv.org/pdf/2603.26323.
• A Human-Inspired Decoupled Architecture for Efficient Audio Representation Learning: Introduces HEAR, an architecture with decoupled pathways achieving high efficiency with 85M–94M parameters. Code: https://github.com/HarunoriKawano/HEAR.
• QUITO: A billion-scale, single-provenance time series corpus from Alipay for time series forecasting, introducing QUITOBENCH. Paper at https://arxiv.org/pdf/2603.26017.
• Adapting Segment Anything Model 3 for Concept-Driven Lesion Segmentation: Systematically evaluates SAM3 for medical lesion segmentation using concept-level prompts and prior knowledge. Code: https://github.com/apple1986/lesion-sam3.
• Geo²: A unified framework leveraging Geometric Foundation Models (GFMs) for Cross-View Geo-Localization and bidirectional Cross-View Image Synthesis. Code: https://fobow.github.io/geo2.github.io/.
• ArtHOI: An optimization-based framework for 4D hand-articulated-object interaction reconstruction using foundation model priors and MLLM-guided alignment. Code: https://arthoi-reconstruction.github.io.
• MuRF: A novel inference-time strategy that leverages multi-scale image processing to enhance Vision Foundation Models (VFMs). Code: https://github.com/orgs/MuRF-VFM.
• PointINS: A self-supervised framework for point clouds enhancing instance-aware representation learning through geometry-aware methods. Paper at https://arxiv.org/pdf/2603.25165.
• AirSplat: Improves feed-forward 3D Gaussian Splatting by addressing pose-geometry discrepancies and multi-view inconsistencies using Self-Consistent Pose Alignment (SCPA) and Rating-based Opacity Matching (ROM). Code: https://kaist-viclab.github.io/airsplat-site.
• π, But Make It Fly: Introduces AirVLA, fine-tuning the π0 vision-language-action model for aerial manipulation using physics-guidance. Code: https://airvla.github.io.
• SABER: A stealthy agentic black-box attack framework for Vision-Language-Action models. Paper at https://arxiv.org/pdf/2603.24935.
• CORA: A 3D vision foundation model for coronary CT angiography (CCTA) analysis and MACE risk assessment using pathology synthesis. Paper at [https://arxiv.org/pdf/2603.24847](https://arxiv.org/pdf/2603.24847].

Impact & The Road Ahead

The impact of these advancements is profound and far-reaching. We’re seeing AI evolve from task-specific tools to generalist intelligent agents capable of complex reasoning and real-world interaction. The breakthroughs in 3D avatar generation pave the way for hyper-realistic virtual experiences in gaming, entertainment, and virtual collaboration, blurring the lines between digital and physical identities. In robotics and autonomous systems, the fusion of high-level reasoning with real-time control promises safer and more efficient autonomous vehicles and intelligent agents in diverse environments, from factory floors to deep space. The emphasis on training-free and parameter-efficient methods is democratizing access to powerful AI, enabling deployment on resource-constrained edge devices, and making sophisticated tools accessible to smaller teams and lower-resource languages.

For medical AI, the ability to generate explainable diagnoses and segment lesions with unprecedented accuracy and efficiency means earlier detection, more personalized treatment, and reduced diagnostic burdens for clinicians. The focus on patient-first modeling and multimodal data fusion in pathology and genomics is creating a holistic view of human biology, accelerating drug discovery and disease understanding.

However, the road ahead is not without its challenges. AI security remains a critical concern, with new vulnerabilities emerging as models become more capable and integrated into sensitive systems, as highlighted by “AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspective”. Regulatory compliance is another burgeoning area, with findings like those in “Machine Learning in the Wild: Early Evidence of Non-Compliant ML-Automation in Open-Source Software” showing a significant gap between model capabilities and ethical deployment practices. Researchers are also grappling with fundamental questions of interpretability and alignment, as seen in “Concept frustration: Aligning human concepts and machine representations” which explores how to bridge the gap between human and machine reasoning.

Looking forward, the trend toward neuro-symbolic AI (e.g., in origami folding, “Learn2Fold: Structured Origami Generation with World Model Planning”) suggests a future where LLMs handle high-level planning while physics-aware world models ensure physical feasibility. The need for high-quality, domain-specific benchmarks is paramount, as demonstrated by papers like ScoringBench and CL-VISTA, pushing beyond simple accuracy to evaluate robustness, fairness, and utility in real-world scenarios. We are moving towards an exciting future where AI not only performs tasks but understands, reasons, and interacts with the world in a profoundly more integrated and trustworthy manner.

Please leave this field empty

Hi there 👋

Get a roundup of the latest AI paper digests in a quick, clean weekly email.

Check your inbox or spam folder to confirm your subscription.