Enhancing Personalized Prognostic Assessment of Myelodysplastic Syndromes through a Multimodal and Explainable Deep Data Fusion Approach (MAGAERA)

Background. Recent advancements in genome characterization have transformed the study of myelodysplastic syndromes (MDS). Accordingly, there has been a shift from traditional classification and prognostication methods, which relied mainly on morphological and clinical data, to next-generation systems that incorporate genomic features. However, genetic abnormalities account for only part of the overall risk related to survival, disease progression, and individual response to hypomethylating agents (HMA), indicating that a significant portion of these risks is still tied to clinical and non-mutational factors. Increasing evidence suggests that transcriptomics, immune dysfunctions, and high-dimensional tumor morphology data extracted by Artificial Intelligence (AI) may play a crucial role in predicting clinical outcomes in human cancers, thereby improving the implementation of personalized medicine programs.

Aim. In this scenario, we developed MEGAERA, an innovative, deep learning-based framework for multimodal analysis of hematological malignancies. MEGAERA integrates clinical, multi-omics, and histopathological data, using specific strategies to ensure full clinical explainability and interpretability of predictions. This study was conducted by the GenoMed4All and Synthema EU consortia, with MDS included as use case, to improve personalized predictions of patient outcomes.

Methods. The study population consisted of 605 MDS patients from Humanitas Research Hospital. For these patients, multi-modal data were available, including clinical characteristics, cytogenetics, somatic mutation screening on 31 genes, bulk RNA-seq of CD34⁺ bone marrow (BM) cells, and deep flow cytometry evaluation of T lymphocytes, natural killer and myeloid cells. Whole slide images (WSI) of BM biopsies stained with Hematoxylin and Eosin (H&E) and May-Grunwald Giemsa (MGG) were also retrieved. Comprehensive information on treatments and clinical outcomes was collected.

MEGAERA deep-learning workflow involves two processing layers. The first one exploits a custom fine-tuned implementation of Prov-GigaPath for WSI segmentation and features extraction. The second layer uses Self-Normalizing Networks with two hidden layers for handling clinical and molecular profiles. All the processed variables are combined and fed into a fusion model that correlates them to clinical outcomes. Explainability was implemented using attention-based maps for WSI and Shapley Additive Explanations Approach for feature importance rankings. The predictive ability of the approach was assessed using Harrell’s concordance index (CI).

Results. We used Prov-GigaPath on BM WSI for extracting morphological features. Prov-GigaPath outperformed current state-of-the-art methods including ResNet50 and DinoBloom, achieving 10% improvement of CI.

We evaluated MAGAERA’s predictive performance on MDS population, analyzing the multimodal integration alongside unimodal contributions, with Overall Survival (OS) as primary endpoint. Sequential integration of data modalities into the model showed an increasing CI for OS: starting with 0.56 CI considering clinical information alone, then rising to 0.81 CI by including cytogenetic, genomic, transcriptomic and immunologic signatures, and finally reaching 0.85 CI with morphological features integration. Our fusion model significantly enhanced the performance of conventional IPSS-R (0.68) and IPSS-M (0.76) scores. Similar improvements were observed in predicting leukemic evolution risk (0.83) and the individual probability of response to HMA treatment (0.84).

An extensive validation of the model’s performance was performed using multimodal synthetic data (PMID: 37390377), reaching comparable CI. To facilitate the clinical implementation of this framework, we are exploring innovative AI-based dimensionality reduction and inference approaches, to enable model’s knowledge transfer to patients lacking information collected outside diagnostic routine tests.

Conclusion. The MEGAERA multimodal fusion model demonstrated improved clinical outcome prediction in MDS patients. Our approach leverages full interpretability to elucidate features contribution to risk prediction. This framework is expected to significantly enhance clinical decision-making in MDS by supporting the implementation of personalized medicine programs.