Continual Learning: Navigating Dynamic AI with Resilience and Adaptability

Latest 50 papers on continual learning: Dec. 21, 2025

The world of AI/ML is constantly evolving, demanding models that can learn and adapt to new information without forgetting what they’ve already learned. This challenge, known as catastrophic forgetting, is at the heart of continual learning (CL). Recent breakthroughs are pushing the boundaries of what’s possible, enabling AI systems to become more resilient, efficient, and intelligent in dynamic environments.

The Big Idea(s) & Core Innovations

At its core, continual learning aims to imbue AI with a ‘lifelong’ ability to acquire new knowledge while retaining old. This collection of papers showcases diverse strategies to achieve this, from innovative memory management to novel architectural designs.

A recurring theme is the intelligent use of memory and experience. For instance, ODEDM in “Dynamic Dual Buffer with Divide-and-Conquer Strategy for Online Continual Learning” by Congren Dai and colleagues from Imperial College London introduces dynamic memory buffers and a Divide-and-Conquer (DAC) strategy, drastically reducing computational overhead while preserving semantic information. This is echoed in “MemVerse: Multimodal Memory for Lifelong Learning Agents” by Junming Liu and team from Shanghai Artificial Intelligence Laboratory, which proposes a hierarchical retrieval-based long-term memory system with periodic distillation to a lightweight parametric model, enabling lifelong multimodal reasoning. PRIMED, presented in “Forging a Dynamic Memory: Retrieval-Guided Continual Learning for Generalist Medical Foundation Models” by Zizhi Chen et al. from Fudan University, also leverages retrieval-augmented generation to dynamically adjust reference data and knowledge ratios, crucial for medical foundation models. Similarly, “Neuroscience-Inspired Memory Replay for Continual Learning” by Goutham Nalagatla and Shreyas Grandhe delves into biologically-plausible generative replay using predictive coding, achieving up to 15.3% better task retention.

Another significant thrust is architectural innovation and parameter-efficient adaptation. “MoB: Mixture of Bidders” by Dev Vyas of Georgia State University offers a game-theoretic approach to Mixture of Experts (MoE) by using Vickrey-Clarke-Groves (VCG) auctions for stateless routing, effectively making it immune to catastrophic forgetting. For resource-constrained scenarios, “Efficient Continual Learning in Neural Machine Translation: A Low-Rank Adaptation Approach” by Salvador Carrión and Francisco Casacuberta from Universitat Politècnica de València introduces Low-Rank Adaptation (LoRA) for NMT, achieving full-parameter performance with significantly fewer parameters. This is complemented by “PS-LoRA: Resolving Conflicts in Lifelong Learning via Aligning Updates in Subspaces” by Yueer Zhou et al. from Zhejiang University, which mitigates forgetting by aligning parameter updates within subspaces. “SAMCL: Empowering SAM to Continually Learn from Dynamic Domains with Extreme Storage Efficiency” by Zeqing Wang and colleagues from Xidian University introduces an AugModule and Module Selector for the Segment Anything Model (SAM), achieving high accuracy with ultra-low storage costs. In the realm of graph neural networks, “Condensation-Concatenation Framework for Dynamic Graph Continual Learning” by Tingxu Yan and Ye Yuan from Southwest University uses graph condensation and feature concatenation to handle dynamic graph structures.

The challenge of catastrophic forgetting extends beyond core model updates to critical real-world applications. For instance, “Unforgotten Safety: Preserving Safety Alignment of Large Language Models with Continual Learning” by Lama Alssum et al. from King Abdullah University of Science and Technology demonstrates that CL methods like DER effectively preserve LLM safety during fine-tuning. Similarly, “Bridging the Reality Gap: Efficient Adaptation of ASR systems for Challenging Low-Resource Domains” from Darshil Chauhan and co-authors at BITS Pilani leverages LoRA and multi-domain experience replay for privacy-preserving ASR adaptation on edge devices, addressing the “reality gap” in clinical settings.

Under the Hood: Models, Datasets, & Benchmarks

These advancements are built upon and contribute to a rich ecosystem of models, datasets, and benchmarks:

• Architectural Innovations:
  • MoB (Mixture of Bidders): A game-theoretic framework using VCG auctions for stateless routing in Mixture of Experts (MoE) models.
  • PPSEBM: An energy-based model with progressive parameter selection for dynamic adaptability. (https://github.com/your-repo/ppsebm)
  • SSD (Selective Subnetwork Distillation): A framework enhancing subnetwork connectivity in sparse neural networks for cross-task knowledge transfer. (https://arxiv.org/pdf/2512.15267)
  • REAL: A dual-stream pretraining and feature fusion buffer approach for Exemplar-Free Class-Incremental Learning (EFCIL).
  • ODEDM: Dynamic Dual Buffer with Divide-and-Conquer strategy for Online Continual Learning. (https://arxiv.org/pdf/2505.18101)
  • TAME (Task-Aware Multi-Expert): A lifelong deep learning algorithm using task similarity and attention for expert selection. (https://github.com/jianyuwang/TAME)
  • MIRA: Memory-Integrated Reconfigurable Adapters, a unified framework for DG, CIL, and DIL using Hopfield networks. (https://snimm.github.io/mira_web/)
  • MoDE (Modality-Decoupled Experts): A lightweight architecture addressing intra- and inter-modal forgetting in Unified Multimodal Generative Models (UMGMs). (https://github.com/Christina200/MoDE-official.git)
  • CIP-Net: An exemplar-free, self-explainable continual learning model using prototype-based reasoning. (https://github.com/KRLGroup/CIP-Net)
  • SAIDO: Scene-Aware and Importance-Guided Dynamic Optimization framework for AI-generated image detection. (https://arxiv.org/pdf/2512.00539)
• Resource Optimization & Efficiency:
  • LoRA (Low-Rank Adaptation): Parameter-efficient fine-tuning for NMT and ASR in Bridging the Reality Gap and Efficient Continual Learning in Neural Machine Translation. (https://arxiv.org/pdf/2512.09910, https://arxiv.org/pdf/2512.16401)
  • SketchOGD: A memory-efficient approach for continual learning. (https://arxiv.org/pdf/2305.16424)
  • LEE-CL (Link-Aware Energy-Frugal Continual Learning): Designed for fault detection in IoT networks, prioritizing energy efficiency.
• Key Datasets & Benchmarks:
  • Gram Vaani dataset: Utilized in Bridging the Reality Gap for low-resource ASR adaptation.
  • CIFAR-100, ImageNet-100, ImageNet-1k: Standard benchmarks for class-incremental learning, used in REAL.
  • MGTIL: A new comprehensive benchmark for evaluating medical generalist continual learning.
  • DriveLM dataset: For VQA in autonomous driving, used in VLM-Assisted Continual learning for Visual Question Answering in Self-Driving. (https://arxiv.org/pdf/2502.00843)
  • CFEE and RAF-DB: For facial expression recognition, used in Feature Aggregation for Efficient Continual Learning of Complex Facial Expressions.
  • SWE-Bench-Verified, SWE-Bench-Pro, PyTorch-Bench: Benchmarks for AI software engineers, used in Confucius Code Agent. (https://github.com/facebook/confucius)
  • ShanghaiTech and Charlotte Anomaly: Multi-scene datasets for video anomaly detection, used in CADE.

Impact & The Road Ahead

The implications of this research are profound, paving the way for truly intelligent and adaptable AI systems. From enhancing medical foundation models for real-time diagnosis (PRIMED) to enabling privacy-preserving ASR on edge devices (Bridging the Reality Gap), continual learning is making AI more robust and trustworthy. We see its impact across industrial IoT for real-time quality control (Continual Learning at the Edge), unsupervised visual anomaly detection in manufacturing (On-Device Continual Learning for Unsupervised Visual Anomaly Detection), and even fault detection in 6G networks (Multi-Generator Continual Learning for Robust Delay Prediction).

Beyond technical performance, continual unlearning (as explored in “Distill, Forget, Repeat: A Framework for Continual Unlearning in Text-to-Image Diffusion Models”) is becoming critical for responsible AI, allowing models to adapt to sequential deletion requests while preserving overall quality. The theoretical underpinnings are also being strengthened, with “Asymptotic analysis of shallow and deep forgetting in replay with Neural Collapse” extending Neural Collapse theory and “Knowledge Adaptation as Posterior Correction” unifying diverse adaptation techniques under a single Bayesian framework.

Perhaps most interestingly, the very definition of AI consciousness is being re-evaluated through the lens of continual learning, as seen in “A Disproof of Large Language Model Consciousness: The Necessity of Continual Learning for Consciousness”. As Erik Hoel argues, current LLMs, with their static structure, lack the essential capacity for continual learning that defines consciousness, pushing us to consider what true, lifelong intelligence entails.

The future of AI lies in its ability to continuously learn, adapt, and evolve. These papers highlight a vibrant research landscape, pushing the boundaries of what’s possible, and setting the stage for a new generation of intelligent systems that can truly grow and thrive in our dynamic world.