Generative enterprise search with extensible knowledge base using AI

In the digital era, companies are inundated with vast amounts of data, making accurate and efficient access to information a relevant concern. This master thesis delves into the integration of generative artificial intelligence (AI) in the realm of enterprise search in collaboration with Iriscube Reply, proposing a pipeline that synergies semantic embeddings and similarity search with generative AI capabilities. At the core of this approach is an information retriever that leverages semantic embeddings to understand the hidden relationships and meanings within the enterprise data. By doing so, it can effectively identify and return a set of documents most pertinent to a user's query. Once these relevant documents are retrieved, they are fed into a generative AI system. Unlike traditional search mechanisms that return verbatim excerpts from documents, the generative AI system crafts coherent and contextually relevant answers to the user's questions. This enhances user experience by providing direct and clear answers and reduces the cognitive overload on the user, eliminating the need to sift through multiple documents for information. During the component selection phase which forms the proposed pipeline, evaluations were conducted on the required resources to deliver a viable and scalable POC. Furthermore, a series of experiments were conducted to evaluate the effectiveness of the chosen component. In conclusion, this thesis underscores the potential of integrating generative AI into enterprise search systems. By combining the advantages of semantic embeddings for information retrieval and generative AI for answer generation, enterprises may lower the barriers to understanding numerous internal processes and offer a more efficient, streamlined, and user-friendly search experience. As enterprise data volumes steadily increase, such advancements will prove essential to enhance decision-making and boost productivity.