Machine Learning–based multimodal classification of Primary Progressive Aphasia (PPA) variants from automatically transcribed speech and diffusion MRI microstructure

Primary Progressive Aphasia (PPA) is a neurodegenerative syndrome characterized by the gradual decline of language abilities, while other cognitive domains remain relatively preserved in the early stages. Growing awareness and better diagnostic tools have led to an increasing number of identified cases. Accurate diagnosis and differentiation among the three main PPA variants, nonfluent/agrammatic (nfvPPA), semantic (svPPA) and logopenic (lvPPA), are crucial for clinical management, prognosis and the development of targeted interventions. Diagnosis typically relies on clinical and neuropsychological assessments, supported by neuroimaging and, when possible, pathological confirmation. In this context, artificial intelligence (AI) has emerged as a promising tool to assist clinicians in diagnostic decision-making. This study developed and validated a multimodal machine learning model for subtype classification in PPA. Linguistic features from automatic transcriptions of a neuropsychological test were combined with white matter microstructural features from diffusion tensor imaging (DTI) and used as joint inputs to a supervised classifier, with the aim to automatically distinguish nfvPPA, svPPA and lvPPA. Data were collected at IRCCS San Raffaele Hospital and included 91 healthy controls and 94 PPA patients (38 nfvPPA, 36 svPPA and 20 lvPPA). Of these, 80 healthy controls and 85 patients underwent DWI, and 38 controls and 81 patients completed the Picnic picture description task from the Western Aphasia Battery, designed to evaluate connected speech production. Audio and imaging data were preprocessed to ensure normalization and quality consistency. Speech samples were automatically transcribed with Microsoft Azure and processed to obtain linguistic features. Regarding Magnetic Resonance Imaging (MRI), DTI data were analyzed to obtain microstructural white matter indices for different brain regions. A multimodal classifier was trained and cross-validated to distinguish healthy individuals from PPA patients and to discriminate among PPA variants. Lastly, a prototype web application was developed to integrate audio preprocessing, automatic transcription and a supervised revision step, enabling clinicians to correct transcription errors before feature extraction and thus ensuring that the linguistic data accurately reflect the original speech.