面向大模型时代的持续学习方法论演变

Machine learning methods, especially deep learning, have achieved remarkable progress across various fields. However, most approaches are limited to static scenarios and struggle to continually learn new knowledge in dynamic, open-world scenarios while retaining previously acquired knowledge, unlike humans. To address this challenge, continual learning (CL) has attracted increasing attention. Existing CL methods can be broadly categorized into two types: Traditional CL methods based on non-pretrained models, and CL methods based on pretrained models that have emerged with the advent of large models. This paper aims to provide a detailed review of the research progress in these two categories of methods, mainly comparing the similarities and differences between non-pretrained and pretrained model approaches from four perspectives: Data level, model level, loss/optimization level and theoretical level. We focus on analyzing the technical changes from methods employing non-pretrained models to those employing pretrained models, and analyze the underlying reasons for these differences. Finally, we summarize and envision the future trends in continual learning development.