HCD-SCL Net: Hierarchical Category Decoupling and Structural Continual Learning for Object Detection in Remote Sensing Images

Deep learning has been used for object detection in remote sensing images successfully. However, how to detect objects precisely and efficiently with proper architecture complexity is an important issue when there are limited computing resources. Furthermore, current object detection neural networks are usually static and cannot perform differentiated training on samples of varying difficulty. In this article, we propose a method combining hierarchical category decoupling (HCD) and structural continual learning (SCL), named HCD and SCL network (HCD-SCL Net), for object detection in remote sensing images, which can generate the optimal architecture adaptively according to the difficulty level of different categories, overcome overfitting and catastrophic forgetting, and thus the detection performance is improved. First, the hierarchical category decoupling method is proposed to select categories of samples for object detection according to the difficulties of samples and class probability. Then, the SCL method is proposed to generate a proper architecture with a different number of feature extraction layers for different object classes. Thereby, the current stage of the detection model can learn the difficult categories of objects without forgetting the learned detection capabilities for easy categories in previous stages. By doing so, the dynamic tradeoff between the complexity of different classes of object detection and the computational cost is reduced. The experimental results on the remote sensing object detection datasets RSOD, DIOR, and NWPU VHR-10 show that the proposed method had excellent performance compared with state-of-the-art deep learning object detection methods.