Hierarchical Dense Knowledge Retrieval for Knowledge-enhanced Conversational Agents

In this work will be presented a solution developed to optimize in terms of computational time, the process of knowledge retrieval adopted to extract relevant unstructured documents from a knowledge base given an input query. The experiment relies on the setup delineated in the first track of the ninth Dialog System Technology Challenge (DSTC 9). The goal of this task is to design a frictionless knowledge-enhanced conversational agent able to deal with out-of-scope requests that cannot be addressed simply with the call of an API service but that require access to external knowledge. The knowledge base consists of unstructured textual documents collected from the Frequently Asked Question (FAQ) pages of several entities belonging to five domains. From an analysis of the results of the DSTC9, the knowledge retrieval step resulted as a crucial phase in the knowledge-enhancing process. Indeed, the results in this sub-task showed the highest correlation with human judgment and the model that performed the best in this sub-task resulted also as the winner in the final evaluation. However, the best performing models in the retrieval step are based on the Passage Re-Ranking strategy. This strategy requires a point-wise evaluation of all the knowledge documents, causing the time complexity of the system to scale linearly with the dimension of the knowledge base becoming soon unapplicable to real-case scenarios. The method developed is based on a Hierarchical Dense Knowledge Retrieval system that exploits the hierarchical structure of the documents present in the knowledge base in order to perform computationally efficient knowledge retrieval.