Unpacking the Future: Foundation Models Tackle Real-World Complexity from Mars to Medicine

Latest 100 papers on foundation models: May. 30, 2026

Foundation models are revolutionizing AI/ML, pushing boundaries across diverse domains. However, applying these powerful models to complex, real-world scenarios introduces unique challenges, from ensuring reliability and interpretability to handling heterogeneous data and real-time demands. Recent research highlights exciting breakthroughs, demonstrating how these models are being adapted, refined, and audited to tackle these intricacies, paving the way for more robust and impactful AI.

The Big Idea(s) & Core Innovations

At the heart of these advancements is a common thread: adapting and specializing foundation models for nuanced, real-world tasks. One significant challenge lies in understanding and controlling what these models learn. For instance, “LLMSurgeon: Diagnosing Data Mixture of Large Language Models” from VILA Lab, MBZUAI and UCL proposes a novel ‘Data Mixture Surgery’ framework to audit the training data of LLMs. Instead of instance-level membership inference, they reframe it as an inverse problem, recovering domain-level pretraining distributions with impressive fidelity (over 94% accuracy). This enables crucial safety auditing, like detecting toxic content injection, without accessing raw training data, addressing a major transparency concern.

Another critical area is reliable and efficient deployment in specialized domains. In medical AI, “EVL-ECG: Efficient ECG Interpretation With Multi-Aspect Heterogeneous Knowledge Distillation” by researchers from VinUni-Illinois Smart Health Center and Hanoi University of Science and Technology introduces a distillation framework to compress large vision-language models for ECG interpretation into efficient models for clinical edge deployment. Their work, featuring Multi-Head Cross-Attention Alignment and Optimal Transport-based Visual Feature Matching, achieves up to 2.4% AUC improvement while reducing model size, proving that sophisticated cardiac reasoning can be deployed on resource-constrained devices. Similarly, “PulmoFoundation: A Clinically Validated Foundation Model for Comprehensive Lung Pathology Interpretation” from Hong Kong University of Science and Technology and Southern Medical University demonstrates the power of subspecialty-specific foundation models, outperforming pan-cancer models in lung pathology. Their model, built on continual pretraining, showed a remarkable improvement in pathologist accuracy (from 83.8% to 91.7%) in a randomized controlled trial, highlighting AI’s role in assisting, not replacing, human experts.

Beyond medicine, geometric and spatial understanding for robotics and autonomous systems sees major strides. “FOUND-IT: Foundation-model-first Task-driven 3D Scene Graphs with Granularity on Demand” by MIT presents a real-time framework to build hierarchical, task-driven 3D scene graphs using uncalibrated monocular cameras and geometric foundation models. This innovation allows robots to dynamically adjust object and region granularity based on queries, improving performance on tasks like ASHiTA SG3D by 79%. For long-range depth estimation in autonomous driving, “Sparse-LiDAR Prompting of Monocular Geometry Foundations: An Empirical Study Toward Long-Range Driving Depth” introduces SLIM by Benewake (Beijing) Co., Ltd., which adapts MoGe-2 for sparse LiDAR input, achieving a 39-51% error reduction at 100-150m ranges. This is critical for safety in high-speed navigation. “Every9D-21M: Large-Scale Real-World 9D Canonicalization of Everyday Objects” from the University of Freiburg and CISPA Helmholtz Center provides a massive dataset with 9D pose annotations for 21.8 million images, generated with minimal manual effort, improving pose estimation and generalization for 3D vision foundation models.

Generalization beyond training data is another persistent challenge. “Learning to Extrapolate to New Tasks: A Relational Approach to Task Extrapolation” by University of Michigan, Ann Arbor proposes the Relational Task Extrapolator (RTE) to address the limitation of modern systems in generalizing to unseen tasks. By decomposing novel tasks into known anchor tasks and transformations, RTE reframes difficult out-of-support problems into more tractable ones, achieving substantial improvements across various extrapolation regimes. In a striking example of domain adaptation, “VesselSim: learning 3D blood vessel segmentation without expert annotations” from Concordia University demonstrates that training a 3D U-Net solely on 16,500 synthetic angiographic volumes, combined with test-time adaptation, achieves competitive performance with models trained on real, large-scale clinical data, completely eliminating the need for manual annotations. This highlights the potential of synthetic data for data-scarce medical imaging.

Under the Hood: Models, Datasets, & Benchmarks

Recent research heavily relies on and contributes to a rich ecosystem of models, datasets, and benchmarks:

• LLMSurgeon: Introduces LLMScan benchmark for open-source LLMs (LLaMA-1, OLMo, Amber, Pythia, GPT-Neo, StarCoder) and uses SlimPajama-627B-DC reference corpus and The Pile/The Stack datasets. Code: LLMSurgeon.
• minWM: An open-source framework for interactive video world models, supporting conversion of Wan2.1-T2V-1.3B and HY1.5-TI2V-8B into camera-controllable AR models. Code: minWM.
• CalArena: A large-scale benchmark (~2000 experiments) for post-hoc calibration methods across tabular and computer vision tasks. Unified implementations in probmetrics package. Code: CalArena.
• LDCM: Leverages monocular depth foundation models for Poisson-based depth initialization and a point map regression head for 3D coordinates. Code: Not explicitly provided.
• Geometry Matters: Incorporates priors from 3D foundation models like SAM3D for reconstruction and PartField for 3D feature fields to enhance semantic correspondence. Code: 3D-SC.
• KAIROSAGENT: Agentic framework for time series forecasting, fusing LLM-based semantic reasoning with TSFM-based numerical prediction. Curates T-STAR corpus for training time series agents. Resources: KAIROSAGENT project page.
• GenBloom: First genetically aligned slide-level blood model integrating single white blood cell images with cytogenetic data. Uses AML-Hehr dataset and an in-house peripheral blood smear dataset. Code: GenBloom.
• EVL-ECG: Distills PULSE-7B (teacher) into Qwen3-VL-2B-Instruct (student) for ECG interpretation. Uses PTB-XL, MIMIC-IV-ECG, CODE-15% datasets. Code: PyTorch implementation with specified models.
• HoliTok: Continuous holistic speech tokenizer for unified generation-understanding. Leverages datasets like LibriSpeech, AISHELL-1/2, GigaSpeech, MLS, Common Voice, and WavLM for distillation. Code: HoliTok.
• CLUBench: Comprehensive clustering benchmark evaluating 24 algorithms on 131 datasets across tabular, text, and image. Evaluates pretrained embeddings from ResNet, CLIP, BERT, Llama3, and OpenAI models. Code: CLUBench.
• Text2BFM: Uses pretrained Behavioral Foundation Models (BFMs) like MetaMotivo as executable motion priors for text-to-motion generation. Code: Not explicitly provided.
• MIRAGE: Brain encoding framework combining multimodal foundation model (Qwen3-Omni-30B) with adaptive layer aggregation. Achieves SOTA on the Algonauts 2025 challenge. Code: MIRAGE.
• BuilDyn: Open-source Python package for excitation-driven data generation for building thermal dynamics. Built on BuilDa FMU-based simulation framework. Code: BuilDyn.
• When Do Graph Foundation Models Transfer?: Theoretical framework for GFM transfer using graphons. Provides code for graph merging augmentation. Code: GraphFM.
• Benchmarking Positional Encoding Strategies for Transformer-Based EEG Foundation Models: Evaluates positional encoding in CBraMod transformer backbone on Healthy Brain Network EEG, PhysioNet Motor Imagery, FACED datasets. Code: Not explicitly provided.
• Why Specialist Models Still Matter: Introduces HetMedAgent, a heterogeneous medical multi-agent framework orchestrating generalist LLMs, domain-specific specialist models, and clinicians. Resources: HetMedAgent paper.
• SLAD: Shared LoRA Adapters for Task Specific Distillation uses LoRA adapters with foundation models like DINOv2 for efficient knowledge transfer. Code: Not explicitly provided.
• Mind-Omni: Unifies seven brain, image, and text tasks in a single discrete diffusion architecture. Uses a novel Brain Tokenizer and Muddit backbone. Code: Mind-Omni.
• LoRA-Key: User-centric LoRA watermarking for text-to-image diffusion models. Works with Stable Diffusion v1.4, SDXL, PixArt-α. Code: Not explicitly provided.
• ViTA: Adapts Vision Foundation Models (SAM2) for traversability estimation. Uses DepthAnything3 for geometric distillation. Code: Not explicitly provided.
• GiPL: Cross-Domain Few-Shot Object Detection (CD-FSOD) framework using iterative pseudo-labeling and generative data augmentation with Qwen-image-2.0-pro. Resources: RUOD, CARPK, CarDD datasets. Code: Mentioned as available at CDiscover.
• FedSmoothLoRA: Federated LoRA fine-tuning framework for LLaMA-3.2-1B (and ViT-Small) on CIFAR-100, MetaMathQA, Code-Feedback, Aya datasets. Code: FedSmoothLoRA.
• Chain-of-Prompts: Training-free cell instance segmentation leveraging SAM’s frozen image encoder. Evaluated on CoNIC, CoNSeP, GlaS benchmarks. Resources: Chain-of-Prompts project page.
• POSTTIME: Post-training recipe for multimodal time-series forecasting. Uses TimesFM-2.5 as TSFM prior and Gemma-3-4B as LLM revisor. Evaluated on TimesX benchmark. Code: To be released.
• EarthShift: First comprehensive benchmark for robustness to real-world distribution shifts in Earth observation. Evaluates 8 geospatial FMs and 5 generic vision models. Resources: EarthShift website. Code: EarthShift GitHub.
• Do Physics Foundation Models Learn Generalizable Physics?: Bias-aware benchmark for physics foundation models (DPOT, GPhyT, MORPH, MPP, Poseidon) across 8 PDE families. Code: PhyFM-Bias-Bench.
• When and How Human Curation Backfires: Studies human curation in multi-model self-consuming loops using Gaussian models, CIFAR-10, Qwen2.5-0.5B. Code: curationBackfire.
• ChildVox: Comprehensive benchmark for audio, speech, and large audio-language models on child-centered acoustic signals. Integrates 20+ sub-tasks across 17 datasets. Code: Checkpoints planned under RAIL.
• GAP3D: Generative alignment of VLM (BLIP3) latents to patch-level DINOv2 embeddings for 3D generation with TRELLIS 3D generative model. Code: GAP3D.
• BrainSimSiam: Self-supervised framework for fMRI representations using Siamese architecture with GNN/CNN encoders. Uses Human Connectome Project (HCP) S1200. Code: Not explicitly provided.
• Neural Scaling Laws for Jet Generation: Investigates scaling laws using OmniJet-α on Aspen Open Jets dataset (CMS Open Data). Code: OmniJet-α referenced from Birk et al. 2024.
• Towards a Foundation Model for the Martian Atmosphere: Design study for a Mars Atmospheric Foundation Model (MAFM). Analyzes OpenMARS reanalysis, five orbital retrieval suites, and two GCMs (MGCM and MarsWRF). Code: Not explicitly provided.
• OmniVerifier-M1: Multimodal visual verifier using symbolic outputs. Evaluated on ViVerBench, WISE, T2I-CoreBench. Uses Qwen3-VL-8B. Code: OmniVerifier.
• Applications of temporal graph learning for predicting the dynamics of biological systems: Uses temporal graph learning on pseudotime-resolved gene regulatory networks, comparing against scGPT, scFoundation. Code: tgl-grn-1CCD.
• A Multi-dimensional Framework for Evaluating Generalization in EEG Foundation Models: Evaluates LaBraM, CSBrain, CBraMod on 6 EEG datasets (Physionet MI, BCI Competition IV-2A, Kaggle ERN, TUEV, MDD MAL Depression, Sleep EDF). Code: GitHub repository mentioned.
• High Performance, Low Reliability: Uncertainty Benchmarking for Tabular Foundation Models: Benchmarks 4 TFMs on 112 datasets from TALENT benchmark. Code: high-performance-low-reliability.
• DriveWAM: Adapts pretrained video diffusion transformer (e.g., Wan2.2-TI2V-5B) into an autoregressive video-action policy for autonomous driving. Evaluated on NAVSIM, PhysicalAI-Autonomous-Vehicles. Resources: DriveWAM project page.
• GS-FUSE: Multimodal financial forecasting framework. Combines LLMs (LLaMA, Phi-3) and TSFMs (MOMENT, Kronos). Uses CAMEF, FNSPID datasets. Code: GitHub repository mentioned.
• Revisiting Metafeatures to Explain Model Differences on Tabular Data: Evaluates meta-feature approaches on TabArena benchmark. Uses PyMFE library. Code: PyMFE library.
• Every9D-21M: Large-scale real-world 9D canonicalization dataset using uCO3D as source. Benchmarks include ImageNet3D, PASCAL3D+, HANDAL. Code: Every9D.
• FLORO: Multimodal geospatial foundation model. Pretraining on Sentinel-1, Sentinel-2, SkySAT imagery, elevation data, UAV products. Evaluated on PANGAEA benchmark. Code: Contact authors.
• Do Clinical Models Change Treatment Decisions?: Introduces ClinPivot benchmark built from PrimeKG-style biomedical knowledge graphs. Evaluates Qwen-3-8B and others. Resources: ClinPivot paper.
• Continual Learning in Modern Hopfield Networks with an Application to Diffusion Models: Uses Stable Diffusion v1.5 and pixel-space DDPM on split CIFAR-10. Resources: Paper URL.
• SIGMA: Parameter-efficient fine-tuning for Vision Foundation Models (DINOv2, SigLIP2, SAM) on dense prediction tasks. Evaluated on MS-COCO 2017, ADE20K, NYUv2. Code: Not explicitly provided.
• VERA: Leverages pretrained video generative models (e.g., Wan video model family, LVP initialization) as robot planners. Evaluated on Panda arm and Allegro hand. Resources: VERA project page.
• Benchmarking Ultrasound Foundation Models for Fetal Plane Classification: Benchmarks USFM, MOFO, UltraSAM, FetalCLIP against CNN/ViT baselines (ResNet50, EfficientNetV2, DINOv3). Uses Spanish and African fetal ultrasound datasets. Code: Not explicitly provided.
• AndroidDaily: Benchmark for mobile GUI agents with 350 tasks across 94 closed-source Android apps. Evaluates 12 state-of-the-art GUI agents (e.g., Gemini 3 Flash). Resources: AndroidDaily paper.
• Laguna M.1/XS.2 Technical Report: Introduces LAGUNA M.1 (225.8B params) and LAGUNA XS.2 (33.4B params) MoE foundation models for agentic coding. Uses AutoMixer framework. Resources: Laguna XS.2 on HuggingFace. Code: Upstreamed to vLLM.
• Uni-LaViRA: Zero-shot, training-free agentic architecture for embodied navigation. Unifies VLN-CE, ObjectNav, EQA, Aerial-VLN. Deployed on wheeled, quadruped, humanoid, UAV robots. Resources: Uni-LaViRA project page.
• Semantic-Aware Interpretable Multimodal Music Auto-Tagging: Uses EM-BANDED algorithm with audio and lyric features. Evaluated on MTG-Jamendo, Music4All. Code: SAMAT.
• The Point, the Vision and the Text: Introduces ScanReQA, a bias-controlled 3D spatial reasoning benchmark. Evaluates text, 2D, and 3D LLMs across point cloud, vision, and text modalities. Code: ScanReQA.code.
• SpatialBench: Comprehensive cross-paradigm benchmark for 3D spatial foundation models. Introduces DA-Next-5M dataset and DA-Next model. Code: SpatialBench.
• PlayClass: Automated play-behaviour classification in poultry leveraging SAM 3 tracking and V-JEPA 2.1 embeddings. Code: PlayClassCV4Animals.
• Falcon-X: Time series foundation model with Unified Prototype Diff-Attention. Achieves SOTA on GIFT-Eval and fev-bench. Code: Falcon-TST.
• LUCoS: Latent Unsupervised Context Selection for Tabular Foundation Models. Uses TabPFN-2.5 and TabClustPFN PIN Encoder on OpenML-CC18 benchmark. Code: S-LUCoS.
• FoundObj: Self-supervised 3D object segmentation using DINOv2 and TRELLIS as reward models. Evaluated on ScanNet, S3DIS, ScanNet200. Code: FoundObj.
• Boosting Knowledge Graph Foundation Models via Enhanced Negative Sampling: KMAS method enhances ULTRA, TRIX, MOTIF, SEMMA KGFMs. Resources: KGFMs-8C8B. Code: KGFMs-8C8B.
• DinoComplete: 3D Shape Completion with Distilled Semantic Priors (DINOv3) and State Space Models. Uses ShapeNet, ScanNet datasets. Code: Depth renderer from yinyunie/depth_renderer.
• The 2nd EReL@MIR Workshop: Focuses on efficient representation learning for multimodal information retrieval. Discusses Qwen, LLaVA, CLIP. Workshop website: EReL@MIR.
• EEG-FM-Audit: Evaluation pipeline for EEG Foundation Models. Evaluates four EEG-FMs against five baselines on TUAB, TUEV, BCI Competition IV-2b. Code: Mentioned as publicly available.
• On the Generalization Capabilities, Design Choices and Limitations of Keypoint Imitation Learning: Evaluates KIL using RADIOv2.5-B, DIFT, DINOv3-B for keypoint matching. Resources: KIL project website.
• JetViT: Efficient High-Resolution Vision Transformer. Converts full-attention ViTs to hybrid architectures, optimizing DINOv3 and DepthAnythingV2. Code: To be released.
• MSCGC-KAN: Enhances CBraMod foundation models for EEG emotion recognition. Uses FACED, SEED-VII datasets. Code: Not explicitly provided.
• Few-shot Cross-country Generalization of Tabular Machine Learning and Foundation Models for Childhood Anemia Prediction: Compares TabPFN with traditional ML methods on DHS data from 16 countries. Code: Python implementation mentioned.
• Auditing and Fixing Economic Validity in Tabular Foundation Models for Discrete Choice: Audits TabPFN v2, Mitra on Swissmetro, LPMC datasets. Code: Not explicitly provided.
• R^3: 3D Reconstruction via Relative Regression. Uses DA3 backbone. Resources: R3 project page. Code: Not explicitly provided.
• Aperiodic and Low-Frequency Spectral Bias in Reconstruction based EEG Foundation Models: Investigates spectral bias in EEG foundation models (LaBraM, CBraMod, CSBrain) on BCIC-IV 2A, Physionet-MI, Kaggle-ERN, Sleep-EDF. Code: spectralbiasaperiodic.
• Unified Panoramic Geometry Estimation via Multi-View Foundation Models: Introduces PaGeR, adapting DA3, DINOv2 to panorama domain. Introduces ZüriPano, PanoInfinigen datasets. Code: PanoInfinigen open-source tool mentioned.
• Evi-Steer: Evidential steering for BiomedCLIP on 15 biomedical datasets. Code: Evi-Steer.
• Sentinel: Embodied Cooperative Spatial Reasoning and Planning. Uses foundation models with classical navigation algorithms on Virtual Community dataset. Code: Sentinel.
• TSFMAudit: Data Contamination Auditing in Forecasting Time Series Foundation Models. Evaluates 6 TSFMs on 187 datasets (GIFT-Eval, TIME, Monash). Code: TSFMAudit.
• Lifting Data-Tracing Machine Unlearning to Knowledge-Tracing for Foundation Models: Case study on CLIP for fine-grained concept unlearning. Code: Knowledge-Tracing-MU-Page.
• From Model Scaling to System Scaling: Introduces CheetahClaws as an open-source reference harness for agentic AI. Code: CheetahClaws.
• Rethinking Weak Supervision in Anomaly Detection: WSADBench evaluates 36 algorithms across 61 datasets. Includes TabPFN, LimiX. Code: WSADBench.
• CITYREP: Unified benchmark for urban representations. Evaluates 11 models (AETHER, AlphaEarth, TESSERA) across 8 cities and 8 tasks. Code: CityRep.
• A Multimodal 3D Foundation Model for Light Sheet Fluorescence Microscopy: Pretrained on a curated dataset of 3D LSM volumes. Code: lsm_fm_public_repo.
• Small Models, Strong Priors: Introduces WaveLiT, a neural PDE solver. Competes with foundation models 100-1000× larger on TheWell benchmarks. Code: JAX/Flax implementation mentioned.
• EchoPilot: Training-free ultrasound video segmentation using SAM2, MedSAM2, VLM, VFM priors. Introduces fetal placenta ultrasound VOS dataset. Resources: EchoPilot project page.
• The Quantization Benefits of Residual-Free Transformers: Explores residual-free transformers on FineWeb-Edu dataset. Code: Not explicitly provided.
• SAM3-Assisted Training of Lightweight YOLO Models for Precision Pig Farming: Uses SAM 3 as auto-annotator for YOLOv8 on PigLife dataset. Code: Ultralytics YOLOv8 framework.
• Benchmarking Pathology Foundation Models for Spatial Domain Understanding: SpaPath-Bench evaluates 19 encoders (e.g., H-Optimus-1, MUSK) using paired WSI-ST datasets. Resources: SpaPath-benchboard.
• Towards Anatomically Plausible Human Image Generation: ASAP framework for anatomical alignment in human image generation using FLUX.1-dev, SDXL. Introduces HAP Dataset, HAF-Bench. Resources: ASAP framework.
• When Agents Control Robots: ZTPM Zero Trust Policy Model for agentic cyber-physical systems. Evaluated on Cobot-Claw with Gemma 4, Claude Sonnet 4.6 LLMs. Code: PydanticAI, ChromaDB, vLLM, Logfire, LlamaIndex Core, BAAI/bge-small-en-v1.5.
• Back to Parsimonious Latents: TC-WM framework projects DINOv2, DINOv3, Cosmos tokenizer embeddings into task-centric latents. Evaluated on RoboMimic, D4RL. Resources: TC-WM project page.
• RepSAM: CKA-guided parameter-efficient fine-tuning for SAM to robotic vision. Evaluated on OCID, ClearGrasp, GraspNet, WISDOM, YCB-Video, LINEMOD. Code: Not explicitly provided.
• MTLLFM: Multimodal-Temporal Laughter Localization. Uses HuBERT, MAE encoders. Introduces UR-FUNNY-Temporal, SMILE-Temporal datasets. Code: MTLLFM.
• ViroBench: First comprehensive benchmark for Nucleotide Foundation Models on viral genomics tasks. Evaluates 66 NFMs. Code: ViroBench.

Impact & The Road Ahead

These papers collectively paint a picture of an AI/ML landscape grappling with the real-world implications of foundation models. The impact is profound: from making medical diagnostics safer and more accessible (PulmoFoundation, EVL-ECG, GenBloom, Evi-Steer) to enabling more intelligent and robust autonomous systems (DriveWAM, FOUND-IT, Sentinel, SLIM, R^3, VERA). The focus on auditing and transparency (LLMSurgeon, TSFMAudit, Knowledge-Tracing MU) is crucial for building trust and ensuring ethical deployment. Furthermore, innovations in efficiency and generalization (minWM, SLAD, SIGMA, JetViT, WaveLiT) are making powerful models more practical for diverse applications and resource-constrained environments.

The road ahead involves continued efforts in several directions. First, developing architectures with stronger inductive biases tailored for specific data types (WaveLiT for PDEs, MSCGC-KAN for EEG, Falcon-X for heterogeneous time series) will allow smaller models to achieve performance competitive with much larger, more generic counterparts. Second, principled benchmarking and evaluation (CalArena, CLUBench, EarthShift, PhyFM-Bias-Bench, ChildVox, SpatialBench, WSADBench, CITYREP, ViroBench, EEG-FM-Audit, AndroidDaily, ClinPivot) that account for real-world complexities like distribution shifts, ethical considerations, and efficiency will be paramount. Third, hybrid approaches combining foundation models with classical methods or specialized modules (CoSaR, KAIROSAGENT, GS-FUSE, FoundObj) demonstrate a powerful synergy, where the strengths of each compensate for the weaknesses of the other.

Finally, the emerging concept of ‘system scaling’ (From Model Scaling to System Scaling) for agentic AI highlights that future progress isn’t just about bigger models, but about building intelligent, trustworthy, and dynamically adaptive systems around them. As we continue to push these boundaries, foundation models will increasingly become indispensable tools, not just for research, but for solving some of humanity’s most pressing challenges, even beyond Earth’s atmosphere, as envisioned by the Mars Atmospheric Foundation Model. The journey from abstract research to tangible, reliable impact has only just begun, and the pace of innovation is accelerating rapidly.

Please leave this field empty

Hi there 👋

Get a roundup of the latest AI paper digests in a quick, clean weekly email.

Check your inbox or spam folder to confirm your subscription.