Hierarchical Graph-Language Models for Sequential Sentence Classification

Given a sequence of sentences, sequential sentence classification (SSC) assigns a category to each sentence, which can facilitate document understanding tasks. Recent advances in neural language models improve SSC performance by enabling the learning of contextualized representation of sentences that incorporates their semantic content information and the sequential information between sentences. However, these works overlook the high-order structural dependencies between sentences in a constructed graph of sentences and words. In this work, we propose a complex model that integrates graph neural network models with language models for capturing both contextual content information and structural information in a sequence of sentences. We introduce a graph of sentences and words, and graph neural networks are used to model dependencies among them. Extensive experiments on several datasets show the effectiveness of our proposed method for sequential sentence classification.