CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay
2402.01348
0
0
Abstract
This paper introduces a novel perspective to significantly mitigate catastrophic forgetting in continuous learning (CL), which emphasizes models' capacity to preserve existing knowledge and assimilate new information. Current replay-based methods treat every task and data sample equally and thus can not fully exploit the potential of the replay buffer. In response, we propose COgnitive REplay (CORE), which draws inspiration from human cognitive review processes. CORE includes two key strategies: Adaptive Quantity Allocation and Quality-Focused Data Selection. The former adaptively modulates the replay buffer allocation for each task based on its forgetting rate, while the latter guarantees the inclusion of representative data that best encapsulates the characteristics of each task within the buffer. Our approach achieves an average accuracy of 37.95% on split-CIFAR10, surpassing the best baseline method by 6.52%. Additionally, it significantly enhances the accuracy of the poorest-performing task by 6.30% compared to the top baseline. Code is available at https://github.com/sterzhang/CORE.
Create account to get full access
Overview
- This paper proposes a novel approach called CORE (Cognitive Replay) to mitigate catastrophic forgetting in continual learning.
- Catastrophic forgetting is a major challenge in continual learning, where a model forgets previously learned information when trained on new tasks.
- CORE aims to address this issue by leveraging insights from human cognitive processes, such as the role of episodic and semantic memory in lifelong learning.
Plain English Explanation
Continual learning is the ability for AI systems to learn new tasks or information over time, without forgetting what they've learned before. This is a challenging problem, as AI models often suffer from "catastrophic forgetting," where learning new things causes them to forget old things.
The CORE (Cognitive Replay) approach proposed in this paper takes inspiration from how the human brain learns and remembers. In humans, we have different types of memory - episodic memory stores specific events and experiences, while semantic memory stores general knowledge and facts. CORE tries to mimic this by storing important past experiences in an episodic memory, and then using that episodic memory to help the model learn new tasks without forgetting the old ones.
By incorporating these cognitive principles, CORE aims to be more effective at continual learning than traditional approaches. The key idea is to selectively replay relevant past experiences to update the model's knowledge, rather than simply fine-tuning it on new tasks. This helps the model retain what it has learned while also acquiring new capabilities.
Technical Explanation
The paper first provides background on forgetting in human lifelong learning, explaining the roles of episodic and semantic memory. It then introduces the CORE framework, which consists of three main components:
- Episodic Memory: This stores a compact representation of important past experiences, similar to how human episodic memory works.
- Semantic Memory: This stores general knowledge and skills learned over time, analogous to human semantic memory.
- Cognitive Replay: This mechanism selectively replays relevant experiences from the episodic memory to update the semantic memory, allowing the model to learn new tasks without forgetting old ones.
The authors evaluate CORE on several continual learning benchmarks and show that it outperforms state-of-the-art baselines in terms of retaining performance on previous tasks while also learning new ones effectively. They attribute this to CORE's ability to selectively replay relevant past experiences to consolidate knowledge.
Critical Analysis
The paper provides a well-designed and thorough evaluation of the CORE approach, using a variety of continual learning tasks and metrics. The authors also acknowledge some limitations, such as the need to carefully tune the size and composition of the episodic memory.
One potential area for further research could be to explore more adaptive or automated mechanisms for managing the episodic memory, rather than relying on manual tuning. Additionally, the authors could investigate how CORE's performance scales with the complexity of the tasks and the amount of data involved.
Overall, the CORE framework represents an interesting and promising approach to mitigating catastrophic forgetting in continual learning, by drawing inspiration from human cognitive processes. The insights and techniques presented in this paper could have important implications for the development of more robust and versatile AI systems.
Conclusion
The CORE (Cognitive Replay) framework proposed in this paper offers a novel approach to addressing the challenge of catastrophic forgetting in continual learning. By incorporating principles of human episodic and semantic memory, CORE demonstrates improved performance on a range of continual learning benchmarks compared to existing methods.
The key contribution of this work is the integration of cognitive insights into the design of continual learning systems, which appears to be a fruitful direction for further research. As AI systems become more advanced and are required to learn and adapt over time, approaches like CORE may play an important role in enabling them to acquire new knowledge and skills without losing their hard-earned expertise.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Adaptive Memory Replay for Continual Learning
James Seale Smith, Lazar Valkov, Shaunak Halbe, Vyshnavi Gutta, Rogerio Feris, Zsolt Kira, Leonid Karlinsky
0
0
Foundation Models (FMs) have become the hallmark of modern AI, however, these models are trained on massive data, leading to financially expensive training. Updating FMs as new data becomes available is important, however, can lead to `catastrophic forgetting', where models underperform on tasks related to data sub-populations observed too long ago. This continual learning (CL) phenomenon has been extensively studied, but primarily in a setting where only a small amount of past data can be stored. We advocate for the paradigm where memory is abundant, allowing us to keep all previous data, but computational resources are limited. In this setting, traditional replay-based CL approaches are outperformed by a simple baseline which replays past data selected uniformly at random, indicating that this setting necessitates a new approach. We address this by introducing a framework of adaptive memory replay for continual learning, where sampling of past data is phrased as a multi-armed bandit problem. We utilize Bolzmann sampling to derive a method which dynamically selects past data for training conditioned on the current task, assuming full data access and emphasizing training efficiency. Through extensive evaluations on both vision and language pre-training tasks, we demonstrate the effectiveness of our approach, which maintains high performance while reducing forgetting by up to 10% at no training efficiency cost.
4/22/2024
Controlling Forgetting with Test-Time Data in Continual Learning
Vaibhav Singh, Rahaf Aljundi, Eugene Belilovsky
0
0
Foundational vision-language models have shown impressive performance on various downstream tasks. Yet, there is still a pressing need to update these models later as new tasks or domains become available. Ongoing Continual Learning (CL) research provides techniques to overcome catastrophic forgetting of previous information when new knowledge is acquired. To date, CL techniques focus only on the supervised training sessions. This results in significant forgetting yielding inferior performance to even the prior model zero shot performance. In this work, we argue that test-time data hold great information that can be leveraged in a self supervised manner to refresh the model's memory of previous learned tasks and hence greatly reduce forgetting at no extra labelling cost. We study how unsupervised data can be employed online to improve models' performance on prior tasks upon encountering representative samples. We propose a simple yet effective student-teacher model with gradient based sparse parameters updates and show significant performance improvements and reduction in forgetting, which could alleviate the role of an offline episodic memory/experience replay buffer.
6/21/2024
✨
Brain-Inspired Continual Learning-Robust Feature Distillation and Re-Consolidation for Class Incremental Learning
Hikmat Khan, Nidhal Carla Bouaynaya, Ghulam Rasool
0
0
Artificial intelligence (AI) and neuroscience share a rich history, with advancements in neuroscience shaping the development of AI systems capable of human-like knowledge retention. Leveraging insights from neuroscience and existing research in adversarial and continual learning, we introduce a novel framework comprising two core concepts: feature distillation and re-consolidation. Our framework, named Robust Rehearsal, addresses the challenge of catastrophic forgetting inherent in continual learning (CL) systems by distilling and rehearsing robust features. Inspired by the mammalian brain's memory consolidation process, Robust Rehearsal aims to emulate the rehearsal of distilled experiences during learning tasks. Additionally, it mimics memory re-consolidation, where new experiences influence the integration of past experiences to mitigate forgetting. Extensive experiments conducted on CIFAR10, CIFAR100, and real-world helicopter attitude datasets showcase the superior performance of CL models trained with Robust Rehearsal compared to baseline methods. Furthermore, examining different optimization training objectives-joint, continual, and adversarial learning-we highlight the crucial role of feature learning in model performance. This underscores the significance of rehearsing CL-robust samples in mitigating catastrophic forgetting. In conclusion, aligning CL approaches with neuroscience insights offers promising solutions to the challenge of catastrophic forgetting, paving the way for more robust and human-like AI systems.
4/24/2024
Watch Your Step: Optimal Retrieval for Continual Learning at Scale
Truman Hickok, Dhireesha Kudithipudi
0
0
In continual learning, a model learns incrementally over time while minimizing interference between old and new tasks. One of the most widely used approaches in continual learning is referred to as replay. Replay methods support interleaved learning by storing past experiences in a replay buffer. Although there are methods for selectively constructing the buffer and reprocessing its contents, there is limited exploration of the problem of selectively retrieving samples from the buffer. Current solutions have been tested in limited settings and, more importantly, in isolation. Existing work has also not explored the impact of duplicate replays on performance. In this work, we propose a framework for evaluating selective retrieval strategies, categorized by simple, independent class- and sample-selective primitives. We evaluated several combinations of existing strategies for selective retrieval and present their performances. Furthermore, we propose a set of strategies to prevent duplicate replays and explore whether new samples with low loss values can be learned without replay. In an effort to match our problem setting to a realistic continual learning pipeline, we restrict our experiments to a setting involving a large, pre-trained, open vocabulary object detection model, which is fully fine-tuned on a sequence of 15 datasets.
5/13/2024