Hybrid Neural Knowledge Graph-to-Text and Text-to-Text Generation

In this document, we propose a novel Natural Language Generation (NLG) task, namely the hybrid knowledge graph-to-text and text-to-text generation. It aims at enriching the text obtained from a knowledge graph (KG) encoded in the format of the Resource Description Framework (RDF) with relevant information extrapolated from a complementary textual context. This task is particularly useful when dealing with small-sized ontologies on topics for which richer textual resources are available. In order to solve this task, we present a neural system based on a three-step pipeline: pure KG-to-text generation, content selection from the context, and, finally, the combination of the KG’s verbalization and the additional information into a fluent and cohesive textual output. Each step is based on the Transformer architecture, with the first and the third steps employing a suitably fine-tuned T5 model, and the second based on BERT. The KG-to-text generation model is fine-tuned on the WebNLG corpus; the others are trained on two custom datasets derived from the latter. The generated texts are then evaluated through the syntactic log-odds ratio, a referenceless model-dependent metric for fluency evaluation, and a questionnaire-based human evaluation on four dimensions, namely coherence, grammaticality, faithfulness, and informativeness. The generated texts overall reach good levels of grammatical correctness and informativeness, but there is room for improvement with regard to textual coherence and faithfulness.