Continual Learning: Navigating Forgetting, Building Resilient Agents, and Exploring Biological Inspiration

Latest 21 papers on continual learning: Sep. 19, 2026

The dream of AI that learns and adapts continuously, much like humans do, remains a central pursuit in machine learning. However, this dream is constantly challenged by “catastrophic forgetting” – the tendency for neural networks to overwrite old knowledge when learning new information. Recent research efforts are tackling this fundamental stability-plasticity dilemma head-on, exploring innovative solutions from biologically inspired architectures to robust agentic frameworks and novel evaluation paradigms. This digest explores the latest breakthroughs, offering a glimpse into the future of adaptive and resilient AI.

The Big Idea(s) & Core Innovations:

One of the most profound ideas emerging from recent work is the re-evaluation of how AI learns. The position paper, “Position Paper: Neurotransmitters as a Missing Dimension in Artificial Neural Networks” by Yupei Li and colleagues from Imperial College London and Technical University of Munich, proposes a revolutionary concept: using neurotransmitter-inspired mechanisms as a third axis of learning in ANNs. Beyond neural activity and synaptic plasticity, these state-dependent, context-sensitive modulators could control whether, where, and to what extent learning occurs, potentially offering a more biologically plausible solution to catastrophic forgetting and training instability. This idea suggests moving beyond uniform, loss-centric optimization to a more regulated, context-sensitive learning paradigm.

Building on the need for robust adaptation, frameworks for long-horizon autonomous agents are making significant strides. Salesforce AI Research, in their paper “An Architecture for Long-Horizon Agents: Levels, Ticks and Cascaded Intelligence”, introduces a hierarchical architecture that enables language-model agents to operate for days or weeks without forgetting. Their key insight is that continual operation without forgetting belongs to the harness, not the model itself, addressing critical bottlenecks like state management and delegation through bounded file summaries and a “review-and-escalate” cascaded intelligence mechanism. This shifts the burden of memory persistence from the model’s weights to the system’s architecture.

Addressing the practicalities of continual learning, especially in resource-constrained or dynamic environments, several papers offer ingenious solutions. For instance, CLARE (Continual Learning Approach with pRE-trained models), introduced in “Realistic Continual Learning Approach using Pre-trained Models” by Nadia Nasri et al. from the University of Alcalá, demonstrates that combining frozen pre-trained visual encoders with a Dynamic Neural Adaptation Network (Dyn-NAN) is highly effective in a new “RealCL” paradigm where class distributions are random and unstructured. This suggests that pre-trained models, when dynamically adapted, can achieve near-zero forgetting in realistic scenarios, challenging assumptions about traditional CL benchmarks. Similarly, “CLARE: Scalable Class-Incremental Continual Learning via a Sparsity-Based Framework” from Yunxiang Fu and The University of Hong Kong leverages sparsity-driven adapter-based learning to scale class-incremental tasks to over 100 sequences, identifying task-critical sparse parameter masks for efficient and effective learning without inference-time routing. This represents a significant step towards scalable and practical continual learning.

In robotics, “Continual Learning for Traversability Prediction with Uncertainty-Aware Adaptation” by Hojin Lee et al. from UNIST proposes a framework for robot navigation that uses a generative experience recall model and uncertainty-aware adaptation to prevent catastrophic forgetting without storing raw data. Their finding that generative models can effectively replace memory buffers for replay, while uncertainty filtering enhances reliability, is a game-changer for deployable continual learning in autonomous systems. Further advancing robot autonomy, Jiangnan University’s REVOLVE framework in “REVOLVE: An Automated Closed-Loop Framework for Evolving Robot Manipulation with Minimal Human Intervention” showcases a closed-loop system where robots continually improve manipulation policies by converting deployment failures into effective training data, drastically reducing human effort.

Even in the complex realm of federated learning, “Sylvas: Synergistic Learning Value based Device Scheduling in Federated Continual Learning” by Yuxuan Sun et al. from Beijing Jiaotong University introduces an intelligent device scheduling framework. Sylvas quantifies device learning value by combining distributional dynamics and pseudo-label reliability, enabling timely model adaptation in dynamic edge environments by scheduling high-value devices under resource constraints.

Under the Hood: Models, Datasets, & Benchmarks:

These advancements are powered by and contribute to a rich ecosystem of models, datasets, and benchmarks:

• Neuromodulation Frameworks: “Position Paper: Neurotransmitters as a Missing Dimension in Artificial Neural Networks” proposes a two-network design (base + modulator network) for neuromodulated plasticity, with three-factor updates (eligibility trace × global/regional modulator).
• Long-Horizon Agent Architectures: The Salesforce AI Research paper on long-horizon agents introduces a hierarchical architecture with levels, ticks, protocols, and tiers, emphasizing bounded state files for persistence across context resets. It was validated through a ten-day autonomous campaign reproducing a reinforcement learning result.
• Diffusion-Based Replay for Semantic Segmentation: DR.WILSS, from Leon Arthur Marx et al. at the University of Padova, utilizes language-guided diffusion models (e.g., Stable Diffusion 2.1) with LoRA fine-tuning to generate high-quality replay data for weakly supervised continual semantic segmentation. Code available at https://github.com/LTTM/DRWILSS.
• Multimodal Continual Leaf Counting: MCLC-NET, by Ruchi Bhatt et al. from the Indian Institute of Technology Ropar, introduces the MMLC dataset (RGB, depth, thermal images for leaf counting) and employs a modality attention and cross-attention fusion strategy with uncertainty-diversity scoring for buffer management.
• Runtime-Adaptive Vision Transformers: WARD, developed by Mahdi Taheri et al. across Humboldt University of Berlin and Tallinn University of Technology, features channel-wise subnetwork partitioning and reliability-aware continual learning for Vision Transformers on edge devices. Validated on FPGA implementations using EuroSAT and ImageNet-21K pretrained weights.
• Scientific AI Agents with Self-Improvement: ScienceBuddy, from phai-labs, offers an interactive scientific research workspace with 224 tools across 22 modules. It uses a recursive-in-recursive self-improvement paradigm and rubric-supervised reinforcement learning (GRPO). Code is publicly available at https://github.com/Gen-Verse/ScienceBuddy.
• Uncertainty-Aware Traversability Prediction: The framework by Hojin Lee et al. (UNIST) uses a Conditional Variational Autoencoder (CVAE) for generative experience recall and an ensemble uncertainty modeling approach combining aleatoric and epistemic uncertainty. Real-world validation was done on a Clearpath Jackal robot with DINOv2 visual backbone. Code available at https://github.com/HMCL-UNIST/Continual-Traversability-Learning.
• Sparsity-Based Class-Incremental Learning: The CLARE framework from Yunxiang Fu et al. (The University of Hong Kong) employs L1-regularized objectives and sparse fine-tuning on pretrained Vision Transformers (ViT-B/16). Evaluated on OmniBenchmark-1k, ImageNet-R/A, CIFAR-100, and ObjectNet.
• AI-Enabled Space Robot Operations (AI-SRO): The survey from Zeyuan Huang et al. (Beijing University of Posts and Telecommunications) highlights critical resources like Astrobee dataset, SPEED/SPEED+ for pose estimation, and Space Robotics Bench (Isaac Sim-based).
• Federated Continual Learning Device Scheduling: Sylvas, by Yuxuan Sun et al. (Beijing Jiaotong University), uses ResNet-18 and CNNs on CIFAR-100 and SVHN datasets for evaluation.
• LLM Agent Test-Time Strategies: “When Agents Slow Down: Understanding LLM Agents’ Test-Time Strategies via Elo-per-token Analysis” introduces Elo-per-token analysis and evaluates on FrontierCS, ALE-Bench, MLS-Bench, FlashInfer-Bench, and AtCoder Heuristic Contests. Code is at https://github.com/agent-tts/Agent-TTS-Code.
• Automated Robot Manipulation Framework: REVOLVE by Hanyu Liu et al. (Jiangnan University) integrates a VLA-Adapter architecture and utilizes datasets like DROID and Open X-Embodiment.
• Homeostatic Continual Learning: Yue Jin (Nokia Bell Labs France) proposes a conceptual framework for identifying new relevant features based on outlier detection in intent outcomes.
• Continual DQN Expansion for Train Scheduling: “Mitigating the Stability-Plasticity Dilemma in Adaptive Train Scheduling with Curriculum-Driven Continual DQN Expansion” by Achref Jaziri et al. (Goethe University) uses Elastic Weight Consolidation (EWC) and Padé activation units within a Continual DQN Expansion (CDE) algorithm, tested on the Flatland Simulator.
• Realistic Continual Learning: CLARE (by Nadia Nasri et al.) combines a frozen visual encoder (e.g., CLIP) with a Dynamic Neural Adaptation Network (Dyn-NAN), evaluated on RealCL benchmarks from CIFAR-10, CIFAR-100, TinyImageNet, and CUB200. Code: https://github.com/gramuah/clare.
• Continual Learning for Gravity Fields: “Continuous Learning of Gravity Field Irregularities Around Small Bodies via Neural Hamiltonian ODEs” by Giacomo Acciarini and Dario Izzo (ESA Advanced Concepts Team) embeds a feed-forward neural network into the system Hamiltonian to form a Neural Hamiltonian ODE.
• Domain-Incremental Replay Speech Detection: Michael Neri (Tampere University) proposes a task-specific beamformer (TSB) architecture evaluated on the ReMASC corpus. Code: https://github.com/michaelneri/replay-speech-continual.
• Auditing Update Admission for Embodied Agents: “When Validation Stops Learning: Auditing Update Admission for Continual Embodied Agents” from Qinzhen Ma and Ruihai Wu (Rice University, UC Berkeley) uses paired binomial constructions to verify policy updates.
• Catastrophic Forgetting in Skill Retrieval for LLM Agents: Syed Shariyar Murtaza et al. (Manulife) conducted a large-scale study using LoRA fine-tuning and embedding regularization with Qwen3-Embedding and bi-encoder/cross-encoder retrievers. Code: https://github.com/manulife-ai/emnlp2026/.

Impact & The Road Ahead:

These breakthroughs collectively paint a promising picture for the future of continual learning. The shift towards biologically inspired mechanisms, as suggested by the neurotransmitter-focused paper, could unlock fundamentally new learning paradigms, moving AI closer to human-like adaptability. The development of robust agent architectures like those for long-horizon agents and scientific research agents signifies a move towards truly autonomous, self-improving AI capable of extended, complex tasks with minimal human oversight.

For real-world deployment, the advancements in generative replay, sparsity-driven learning, and multimodal continual learning address critical issues in data efficiency, privacy, and adaptability across diverse domains from robotics to edge AI and speech detection. The push for “Realistic Continual Learning” benchmarks will ensure that our evaluation metrics align with real-world complexities, accelerating the development of truly robust systems.

However, challenges remain. The insights from “When Agents Slow Down” highlight a persistent gap between human and agent scaling in complex tasks, suggesting that current LLM agents still struggle with sustained superlinear improvement over time. The warning from “When Synthetic Data Hurts” about catastrophic forgetting when fine-tuning with synthetic data for skill retrieval underscores the need for careful mitigation strategies. And as “When Validation Stops Learning” points out, overly conservative validation gates can inadvertently block crucial learning opportunities, necessitating more nuanced update admission protocols for embodied agents.

The path forward involves a synergistic approach: drawing inspiration from biology, building increasingly resilient and self-improving agent architectures, developing more realistic evaluation benchmarks, and carefully balancing stability and plasticity in diverse applications. Continual learning isn’t just about preventing forgetting; it’s about enabling a future where AI systems evolve, adapt, and learn throughout their operational lifetime, empowering them to tackle unforeseen challenges in dynamic, real-world environments from Earth to space.

Please leave this field empty

Hi there 👋

Get a roundup of the latest AI paper digests in a quick, clean weekly email.

Check your inbox or spam folder to confirm your subscription.