Integrating Knowledge Graphs With Logic Tensor Networks

Logic Tensor Networks are a neuro-symbolic framework that combines logic (first-order fuzzy logic) with deep learning (neural networks). Knowledge graphs are stores of knowledge structured as a graph (with nodes, edges and weights). In this thesis, the aim is to inject prior knowledge from knowledge graphs in neural networks through the use of Logic Tensor Networks to improve the performances of scene graph generation (that is, the task of producing a structured description of images in the form of triples <subject, relationship, object>). In particular, this is made by aligning the Visual Genome dataset (that contains images with dense triplet annotations) with the ConceptNet knowledge graph and leveraging ConceptNet relationships and Numberbatch embeddings (that is a way of representing ConceptNet through vectors) to automatically generate first-order logic statements, which can work as logic constraints to improve the neural networks scene graph generation through Logic Tensor Networks.