{"id":1857,"date":"2025-11-16T10:12:16","date_gmt":"2025-11-16T10:12:16","guid":{"rendered":"https:\/\/scipapermill.com\/index.php\/2025\/11\/16\/catastrophic-forgetting-no-more-recent-breakthroughs-in-continual-learning\/"},"modified":"2025-12-28T21:23:25","modified_gmt":"2025-12-28T21:23:25","slug":"catastrophic-forgetting-no-more-recent-breakthroughs-in-continual-learning","status":"publish","type":"post","link":"https:\/\/scipapermill.com\/index.php\/2025\/11\/16\/catastrophic-forgetting-no-more-recent-breakthroughs-in-continual-learning\/","title":{"rendered":"Catastrophic Forgetting No More: Recent Breakthroughs in Continual Learning"},"content":{"rendered":"

Latest 50 papers on catastrophic forgetting: Nov. 16, 2025<\/h3>\n

The dream of intelligent systems that continuously learn and adapt without forgetting old knowledge has long been a holy grail in AI. However, the phenomenon of catastrophic forgetting<\/strong> \u2014 where neural networks rapidly lose previously acquired skills upon learning new ones \u2014 has remained a formidable hurdle. This challenge is particularly acute in real-world applications, from self-driving cars to medical AI, where models must continuously evolve. Fortunately, recent research is pushing the boundaries, offering exciting new solutions to this pervasive problem.<\/p>\n

The Big Idea(s) & Core Innovations<\/h3>\n

One major theme emerging from recent papers is the development of parameter-efficient and memory-conscious strategies<\/strong> for knowledge retention. For instance, the paper \u201cCOLA: Continual Learning via Autoencoder Retrieval of Adapters<\/a>\u201d by Jaya Krishna Mandivarapu from Microsoft introduces COLA, a groundbreaking framework for Large Language Models (LLMs) that uses autoencoders to efficiently retrieve task-specific adapters. This innovative approach eliminates the need for data replay or large task-specific parameters, significantly reducing memory and parameter usage while outperforming existing methods. Similarly, \u201cMixtures of SubExperts for Large Language Continual Learning<\/a>\u201d by Haeyong Kang from Deep.AI proposes MoSEs, a novel framework leveraging sparsely-gated Mixture of SubExperts. MoSEs achieves minimal catastrophic forgetting without<\/em> explicit regularization or replay, by adaptively selecting task-specific sub-experts, marking a significant leap in LLM scalability and efficiency.<\/p>\n

Another innovative direction is the use of graph-based and subspace-based memory mechanisms<\/strong>. \u201cGraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation<\/a>\u201d by Zihao Guo et al.\u00a0from Beihang University addresses catastrophic forgetting in graph-domain incremental learning by disentangling knowledge across domains. Their method, utilizing parameter-efficient fine-tuning and deviation-free knowledge preservation, ensures stable performance in multi-domain scenarios. Complementing this, Quan Cheng et al.\u00a0from Nanjing University, in their paper \u201cContinuous Subspace Optimization for Continual Learning<\/a>\u201d, introduce CoSO, a framework that fine-tunes pre-trained models within multiple orthogonal subspaces. This dynamically adjusts to new tasks while preserving prior knowledge, especially effective in long-task-sequence scenarios.<\/p>\n

Multi-modal challenges are also seeing inventive solutions. \u201cConSurv: Multimodal Continual Learning for Survival Analysis<\/a>\u201d by Dianzhi Yu et al.\u00a0from The Chinese University of Hong Kong, pioneers the first multimodal continual learning (MMCL) method for survival analysis in cancer patients. Their ConSurv integrates a Multi-staged Mixture of Experts (MS-MoE) and Feature Constrained Replay (FCR) to overcome catastrophic forgetting and complex inter-modal interactions between genomic data and whole slide images. Additionally, \u201cMulti-Modal Continual Learning via Cross-Modality Adapters and Representation Alignment with Knowledge Preservation<\/a>\u201d by Evelyn Chee from the National University of Singapore, presents a PTM-based framework using cross-modality adapters and a novel representation alignment loss to preserve knowledge effectively.<\/p>\n

Even in niche applications, the battle against forgetting is being won. In medical imaging, \u201cPrivacy-Aware Continual Self-Supervised Learning on Multi-Window Chest Computed Tomography for Domain-Shift Robustness<\/a>\u201d by Ren Tasai et al.\u00a0from Hokkaido University, introduces a latent replay-based CSSL framework that ensures data privacy and mitigates catastrophic forgetting in chest CT scans. Similarly, \u201cPANDA – Patch And Distribution-Aware Augmentation for Long-Tailed Exemplar-Free Continual Learning<\/a>\u201d from Purdue University, tackles long-tailed imbalances in exemplar-free continual learning using CLIP-based patch transfer and adaptive balancing, leading to improved accuracy and reduced forgetting without storing past data.<\/p>\n

Under the Hood: Models, Datasets, & Benchmarks<\/h3>\n

Recent research heavily relies on specialized models, benchmarks, and data-centric approaches to tackle catastrophic forgetting. Here are some of the standout resources:<\/p>\n