A deep penetration network for sentence classification

Sentence classification is an important task in natural language processing. The task makes use of deep networks to enclose a mass of features with different granularities in a sentence. However, the classification usually suffer from severe performance degradation when stacking a large number of networks. The main reason is that, in a deep architecture, the silent feature representations are easily weakened and mixed with noisy information, which is not effective in learning contextual features and constructing semantic dependencies in a sentence. In this paper, a deep penetration network (DPN) is designed to improve deep architectures’ ability to preserve the favourable semantic features. The DPN enables salient features to penetrate through a deeper architecture and to construct long semantic dependencies between them. This approach is evaluated on seven public datasets. Our experiments show that the DPN exhibits a stable performance with deeper architectures. It improves the performance on three types of sentence classification tasks, outperforming the existing state-of-the-art models.