Making the Implicit Explicit: Depression Detection in Web across Posted Texts and Images

The utilization of web social media for depression detection has been proven effective in recent years since the multimedia signal on web can reflect users' emotions, feelings, and personality traits in advance. However, most earlier studies simply used users' submitted words or user profiles to predict depression risk. The implicit information accessible in users' posted images, which can be effective in depression detection, still remains unexplored. In this paper, an implicit and explicit multi-modal feature fusion (IEMFF) model is proposed for depression detection. We successfully make the implicit information inherent in users' posted images explicit and further incorporate such explicit features with the textual features directly extracted from user-posted texts. A multi-modal feature fusion approach is applied for depression detection. Extensive experiments have been conducted on public Twitter datasets. Experimental results show that our approach has achieved state-of-the-art performance for depression detection.