Foundation Models Unleashed: From Curing Cancer to Coding Creativity and Beyond!

Latest 50 papers on foundation models: Oct. 20, 2025

Foundation models are revolutionizing AI/ML, tackling everything from complex biological challenges to advanced computing paradigms. These powerful, pre-trained models are not just pushing the boundaries of what’s possible; they’re redefining efficiency, generalization, and practical applicability across diverse domains. Recent breakthroughs, as showcased in a remarkable collection of papers, highlight the incredible velocity of innovation in this field.

The Big Idea(s) & Core Innovations

One of the most exciting overarching themes is the drive towards generalist intelligence and cross-modality integration, making AI systems more versatile and robust. Take, for instance, the work by Xinrui Huang et al. from Shanghai Jiao Tong University, who introduced DentVFM, the first vision foundation model tailored for dentistry. This model exhibits impressive generalist intelligence, outperforming existing models in accuracy and efficiency across multiple clinical tasks by generating task-agnostic visual representations.

Bridging the gap between diverse data types is also paramount. Dominik J. Mühlematter et al. from ETH Zürich unveiled UrbanFusion, a Geo-Foundation Model that intelligently unifies street view imagery, remote sensing, and other spatial modalities into robust representations. This integration significantly enhances urban analytics, proving superior in predicting phenomena like housing prices and public health indicators.

Security in the age of foundation models is another critical area. Amel Abdelraheem et al. from EPFL and Cyber-Defence Campus introduce TBAR in their paper, “Backdoor Unlearning by Linear Task Decomposition,” a novel method to remove backdoors from vision-language foundation models without compromising general capabilities. Similarly, Xiaoyu Xue et al. from HKPolyU reveal vulnerabilities in LM-powered graph foundation models with DTGBA, a stealthy dual-trigger backdoor attack, emphasizing the need for robust defenses.

In the realm of scientific computing, Hyunsu Kim et al. from KAIST and NYU propose Axial Neural Networks (XNN) for “Dimension-Free Foundation Models.” This architecture adeptly handles varying tensor dimensions in physics data, offering superior generalization for partial differential equations. Complementing this, Jeffrey Lai et al. from UT Austin introduce Panda, a pre-trained forecast model that achieves remarkable zero-shot forecasting for chaotic systems, even generalizing from low-dimensional ODEs to high-dimensional PDEs.

For natural language generation, Yunwen Li et al. from CUHK-Shenzhen and M-A-P present COIG-Writer, a high-quality dataset for Chinese creative writing. It uniquely captures the reasoning processes behind creative text generation, highlighting how creative capabilities are culturally bound and require process-level learning.

Medical AI sees significant strides with models like PRISM from Usama Sajjad et al. at The Ohio State University. This morphology-aware prognostic model for colorectal cancer predicts five-year survival from histopathological images with superior accuracy. Furthermore, Qinfan Xiao et al. from Shanghai Artificial Intelligence Laboratory and Tsinghua University introduce BrainOmni, a brain foundation model that unifies EEG and MEG signals, tackling device heterogeneity and modality-specific challenges through joint pretraining.

Under the Hood: Models, Datasets, & Benchmarks

The innovations discussed are often enabled by novel architectures, sophisticated training regimes, and specialized datasets. Here’s a glimpse:

• DentVFM (Model): The first vision foundation model for dentistry, leveraging a comprehensive benchmark, DentBench, covering eight dental subspecialties. Code available at: https://github.com/dentvfm/DentVFM
• UrbanFusion (Model): A Geo-Foundation Model utilizing Stochastic Multimodal Fusion (SMF) for contrastive learning, evaluated across 41 tasks in 56 cities. Code available at: https://github.com/DominikM198/UrbanFusion
• TBAR (Method): Backdoor unlearning for CLIP-based models through linear task decomposition in weight space.
• DTGBA (Attack Framework): Dual-trigger backdoor attacks targeting prompt tuning in LM-empowered graph foundation models. Code available at: https://github.com/xiaoyuxue/DTGBA
• XNN (Axial Neural Networks) (Model): A dimension-agnostic architecture for physics data, demonstrating superior generalization through multidimensional pretraining. Code available at: https://github.com/kim-hyunsu/XNN
• Panda (Model): A pretrained forecast model for chaotic dynamics, capable of zero-shot forecasting across differential equations, using synthetic data generated by evolutionary recombination. Code available at: https://huggingface.co/GilpinLab/panda
• COIG-Writer (Dataset): 1,665 expert-curated Chinese creative writing triplets spanning 51 genres with reverse-engineered prompts and reasoning chains. Resources: https://COIG-Writer.github.io/
• PRISM (Model): A morphology-aware AI model for five-year survival prediction in colorectal cancer using H&E Whole Slide Images. Code available at: https://github.com/AI4Path-Lab/PRISM
• BrainOmni (Model): Unifies EEG and MEG signals via joint pretraining, featuring a novel Sensor Encoder and BrainTokenizer. Code available at: https://github.com/sigureling/BrainOmni
• COFFEE (Model): A context-selective state space model with feedback mechanisms outperforming S6 on long-sequence tasks with fewer parameters. Paper: “Context-Selective State Space Models: Feedback is All You Need”
• KRLM (Model): Knowledge Reasoning Language Model unifying LLMs and KGs to mitigate knowledge distortion in inductive reasoning. Code available at: https://anonymous.4open.science/r/KRLM-EA36
• SerialBEHRT (Model): A domain-aligned foundation model for serialized EHR data, improving clinical representation and antibiotic susceptibility prediction. Paper: “Serialized EHR make for good text representations”
• G2L Framework (Method): Achieves giga-scale pathology FM performance with only 1K slides using knowledge distillation. Paper: “G2L:From Giga-Scale to Cancer-Specific Large-Scale Pathology Foundation Models via Knowledge Distillation”
• Spatial Forcing (SF) (Method): Enables vision-language-action models to develop spatial understanding without explicit 3D inputs. Paper: “Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Model”
• InfiniHuman (Framework/Dataset/Models): Generates realistic and controllable 3D human avatars at scale with precise control over attributes. Resources: https://yuxuan-xue.com/infini-human

Impact & The Road Ahead

The impact of these advancements resonates across research and industry. Healthcare is poised for a revolution, with models like DentVFM and PRISM offering more accurate diagnostics and prognostics, while BrainOmni pushes the frontier of brain signal analysis. The focus on privacy and security, as seen in the “An Investigation of Memorization Risk in Healthcare Foundation Models” paper by [Sana Tonekaboni et al. from MIT and others], and the personalized federated fine-tuning method FedOPAL by Adam Tupper and Christian Gagné, is crucial for building trust in AI systems in sensitive domains.

The push for efficiency and scalability, exemplified by State-Space Models like Hydra in “State-Space Models for Tabular Prior-Data Fitted Networks” by [Felix Koch et al.] and the lightweight COFFEE model, suggests a future where powerful AI isn’t confined to massive data centers. Furthermore, the systematic review of unsupervised deep generative models for anomaly detection in neuroimaging by Bercea et al. points to continued growth in medical image analysis without requiring extensive annotations.

Challenges remain, particularly in the fair benchmarking of Time Series Foundation Models, as highlighted by Marcel Meyer et al. from Paderborn University, who call for new methodologies to prevent information leakage. The findings in “The Role of Computing Resources in Publishing Foundation Model Research” by [Yuexing Hao et al. from MIT and others] also underscore the need for equitable access to computational power to foster inclusive innovation.

Ultimately, these papers paint a vibrant picture of a future where foundation models are not just intelligent but also adaptable, secure, and profoundly impactful. From understanding complex biological processes to facilitating creative writing and robust robotics, the journey of foundation models is just beginning, promising even more transformative breakthroughs on the horizon.