Natural language paragraph generation

Abstract

In recent decades, with the continuous development of deep learning in natural language processing (NLP), natural language generation (NLG) has received increasing attention from researchers. Until now, most of the investigations about NLG mainly focus on sentence generation, which consists of the generation of isolated sentences instead of the generation of coherent texts. The purpose of this study is to select the most advanced and highly competitive sentence generator. Then paragraphwide context will be considered and the chosen generator will be modified into a paragraph generator. Right now, this study has chosen the winning system in the Surface Realization Shared Task of 2020 - IMSurReal system, which is also the most advanced one in the current time. In order to realize the study objectives, updates have been made on the IMSurReal system, which finally was converted it into a paragraph generation system. As for coherence, various measures have been applied to guarantee the generation of more coherent paragraphs. Such measures include converting the format of data, adding an attention mechanism and a multi-model fusion, etc. This thesis present the background of the whole system and the complete process of its modifications. Finally, we show that our modified system manages to get higher BLEU scores that the original system under the same experimental conditions, which validates our approach.