Exploiting local semantic concepts for flooding-related social image classification

In this paper, we present an approach to identification of the images that depict passable and non-passable roads, from a collection of flood-related tweet images. Our key insight is that the local information from domain-specific concepts ('boat', 'person' and 'car') can be exploited to help determine whether an image depicts a location that is passable. We use concept detection as the basis for features that encode local information. We use conventional features, i.e., presence of concepts and visual features extracted from the concept region, but also a novel light-weight feature, i.e., the aspect ratio of the bounding box. Experimental results show that integrating local semantic information yields slightly better performance than only using image-level CNN representation. Text features are not competitive. Copyright held by the owner/author(s).