Interpretable Artificial Intelligence (AI) Differentiates Prefibrotic Primary Myelofibrosis (prePMF) from Essential Thrombocythemia (ET): A Multi-Center Study of a New Clinical Decision Support Tool

Introduction

Overlapping clinical, molecular, and histopathological characteristics pose challenges in differentiating prePMF from ET. The median overall survival, however, significantly differs between prePMF and ET (11.9 vs 22.2 years, Jeryczynski, 2017). The difference in survival highlights the need to distinguish between these two myeloproliferative neoplasms (MPNs) to select disease-specific therapeutic options. This area of unmet need often requires expert assessment at high-volume academic institutions to render a definitive diagnosis. Our aim in this study is to develop and validate a biologically-motivated AI algorithm to rapidly, accurately, and inexpensively diagnose prePMF and ET directly from diagnostic bone marrow (BM) biopsy digital whole-slide images (WSI).

Methods

Patients with a clinical/histopathological diagnosis of prePMF or ET as determined by the International Consensus Classification of Myeloid Neoplasms were identified at the University of Florence, Italy (Florence) between 06/2007 and 05/2023 and Moffitt Cancer Center, Tampa, FL (Moffitt) between 01/2013 and 01/2022. Diagnostic H&E-stained BM biopsy slides were digitized using Aperio AT2 slide scanners (Leica Biosystems, Deer Park, IL) at each institution. The training cohort comprised of 200 (100 prePMF / 100 ET) patients from Florence, and the external test cohort entailed 26 (6 prePMF / 20 ET) patients from Moffitt.

In total, the resultant model was trained on 32,226 patient-derived WSI. Our chosen pretrained neural network, RetCCL, was previously trained on 32,000 diagnostic WSIs to potentially represent a histologically-informed model (Wang, 2023). BM WSI were tessellated into representative image tiles extracted at 10x magnification (302 microns per image dimensions) for model training. Finally, a prediction upon each patient’s WSI was calculated by attention-based multiple instance learning, which is a method that automatically assigns a numeric weight to an image portion representing its relative importance to the classification task.

Model performance was assessed utilizing the area under the receiver operator curve (AUC). The cutoff threshold for diagnosis classification was determined by maximizing Youden’s Index. For qualitative assessment, attention scores were plotted as a heatmap across the BM WSI and reviewed for morphological features by an expert hematopathologist.

Custom scripts were written using our open-source AI framework, Slideflow (Dolezal, 2021). Model development was performed on the Minerva High Performance Computer at Mount Sinai Hospital. Evaluation time upon a single WSI was estimated using a consumer-grade computer with an NVIDIA RTX 3080 graphics processing unit.

Results

Within the training cohort, 5-fold cross validation resulted in a mean AUC of 0.90 and standard deviation of 0.04. A final locked model re-trained on the entire training cohort resulted in an AUC of 0.90 upon evaluation of the test cohort (Figure 1). We optimized the classification threshold to balance sensitivity and specificity; the final diagnostic classification accuracy on the test cohort was 92.3% with a sensitivity and specificity for prePMF diagnosis of 66.6% and 100%, respectively. Upon review of the slides with highest prediction value per class, attention heatmaps highlighted the model’s reliance on areas of cellular marrow without reliance on image artifacts or background (Figure 2). Using affordable consumer-grade hardware, evaluation upon a previously unseen WSI was completed in approximately 6.1 seconds (4.9 for preprocessing and 1.2 for evaluation).

Conclusion

We developed a novel AI model with high accuracy for distinguishing between prePMF and ET in distinct clinical cohorts. To our knowledge, this study represents the largest image-based AI study within MPNs with external validation. Our proposed model may assist clinicians in appropriately identifying patient cohorts who would benefit from disease-specific therapies or enrollment onto clinical trials. We imagine that a potential high-speed, low-cost algorithm may reliably distinguish prePMF from ET patients with high specificity which can be democratized to the MPN clinical community in routine practice and drive clinical trial accrual for biologically rational novel therapeutics.