Hybrid Deep Learning Framework for Summarizing Radiology Reports Using Domain-Specific NLP Techniques

The increasing volume of radiology reports presents a critical need for automated, accurate summarization tools to support clinical efficiency and diagnostic clarity. This thesis proposes a hybrid deep learning framework for summarizing radiology reports, specifically tailored for the chest X-ray domain using the Indiana University Chest X-ray Collection. The objective is to generate fluent, accurate, and clinically meaningful impression-style summaries that align with radiologists’ diagnostic language, thereby supporting efficient clinical decision-making and documentation. The pipeline is composed of three sequential stages. First, an extractive summarization step selects key sentences from the findings section using BERT-based sentence embeddings and cosine similarity ranking. This ensures that structurally important and content-rich sentences are chosen. Second, a medical term filtering module based on the SciSpaCy model extracts only domain-relevant named entities (e.g., anatomical structures, diagnoses, conditions) from the extractive output. This filters out irrelevant or low-clinical-value information. Third, a BART transformer model, pre-trained and then fine-tuned on the filtered data, performs abstractive summarization to produce coherent, concise impressions resembling human-written radiology conclusions. The system was trained and evaluated on processed subsets of the Indiana dataset with dedicated train, validation, and test splits. Evaluation was carried out using both ROUGE metrics (ROUGE-1, ROUGE-2, ROUGE-L, ROUGE-Lsum) for lexical overlap and BERTScore for semantic equivalence. The fine-tuned BART model achieved a ROUGE-L score of 57.84 and a BERTScore F1 of 0.9293 on the test set, indicating a high degree of fluency and factual consistency. The extractive module, when evaluated independently, achieved a BERTScore F1 of 0.8553, confirming that the inclusion of abstraction signifi- cantly enhances the quality and clinical alignment of the generated summaries. Side-by-side qualitative comparisons of extractive, abstractive, and reference summaries further demonstrate the model’s ability to paraphrase accurately, avoid hallucinations, and preserve medically relevant content. Generated outputs correctly captured diagnostic find- ings such as pleural effusions, cardiomegaly, and pulmonary abnormalities, and adhered closely to expert-written impressions in both tone and terminology. This work contributes to the field of medical NLP by demonstrating the effectiveness of a domain-aware hybrid summarization system that balances semantic fidelity with language generation. The modular architecture allows for extensibility to other datasets and clinical specialties. Potential future improvements include multimodal fusion with imaging data, and reinforcement learning with factuality-based re-wards.