ARDVC: Eliminating Semantic Redundancy via Action Retrieval in Dense Video Captioning

Dense video captioning aims to identify and generate descriptions for all events in untrimmed videos. Recently, several methods have utilized ground-truth captions to construct the retrieval corpus, enhancing video features with semantic cues to improve the quality of the generated captions. However, employing full sentences as retrieval targets may lead to retrieval results that are semantically redundant and inconsistent with the video content. Such redundant information may interfere with the visual modality during cross-modal alignment. To address this issue, we propose the Action Retrieval Dense Video Captioning (ARDVC) approach. Instead of retrieving full sentences, we extract the action-relevant descriptions from the input sentences and construct an Action-Centric Text Corpus for retrieval. Additionally, to mitigate the impact of redundant visual information, we propose the Action-Event Queries Cross Decoder, which refines frame-level features by integrating action-discriminative cues, thereby guiding the model to focus on action-relevant visual content. The integration of two components enables the model to better attend to discriminative action content, thereby generating more accurate video captions. Experiments on the ActivityNet Captions and YouCook2 datasets show that ARDVC achieves state-of-the-art performance on dense video captioning. Further qualitative analysis demonstrates that our approach efficiently eliminates semantic redundancy in both visual and textual modalities.