Exploring Language Diversity to Improve Neural Text Generation

Text Generation aims to utilize contextual details to generate linguistically appropriate language. Research has demonstrated that integrating various linguistic features can significantly enhance the quality of text generation tasks. In light of this, this paper proposes an innovative approach Diversity Text Generation (DiversityGen)-and makes advancements in three aspects. Firstly, data augmentation techniques are employed to transform the original data, thereby enhancing the latent features of the text. Secondly, in the conversion of the model’s distributed vector output into text, a combination of Top-K and Beam Search decoding methods (Top-k-bs-m) is utilized. This extends the search space through random sampling during Beam Search decoding, thereby improving decoding performance and generating diversified text. Lastly, the concept of Over Generation (OGen) is introduced, wherein the results are filtered using three probability maximization-based methods to optimize output diversity. Experimental results demonstrate the effectiveness of this approach in question generation and text summarization tasks, surpassing current benchmarks.