AI Multi-Agents Systems in support of Evidence-Based Medicine

This thesis addresses the challenges healthcare professionals face in implementing Evidence-Based Medicine (EBM) in clinical practice. Despite recognizing EBM's importance, clinicians struggle with time constraints, limited methodological knowledge, and language barriers when evaluating scientific literature. The research presents an integrated artificial intelligence solution addressing each stage of the EBM process. The system includes a Risk of Bias assessment tool that embeds research papers in a vectorial database and employs Retrieval-Augmented Generation (RAG) with Large Language Models to evaluate study quality. This component achieved a weighted mean accuracy comparable to GPT models' performance. For medical data extraction, the thesis developed a RAG-enhanced language model that outperformed GPT models across multiple study designs. Additionally, a novel Multi-Agents System was implemented to evaluate clinical applicability of research to real-world cases, achieving very-high accuracy across multiple clinical scenarios. The final component introduces a sophisticated Copilot bibliographic search tool utilizing a cooperative system of few-shot prompted agents. This proposed solution assists clinicians in formulating optimized literature queries by identifying research intent, recommending appropriate study designs, and structuring queries according to renowned frameworks (PICO, PECO, PIRO, PO). The Multi-Agent System interacts with users to retrieve missing information, then, it expands queries with relevant medical terminology, and provides for each term some synonyms to enhance search results. After retrieving documents through PubMed's API, the Copilot evaluates documents' abstracts using multiple weighted factors including topic relevance, publication recency, study methodology, and journal ranking position. The system, then, synthesizes information from the highest-ranked papers and generates evidence-based clinical recommendations citing the top-papers' passages. Performance evaluation demonstrated that the Copilot's literature search capabilities is consistent with established competitors, while substantially outperforming GPT models in adherence to medical guidelines when addressing specific clinical questions. Proposed enhancements for future improvements include incorporating multi-modal data, automating bias assessment before incorporating research findings into clinical recommendations and suggesting alternative research papers based on the applicability evaluation.