Can Machine Learning Supplant the Plasmic Score in Improving the Early Diagnosis of Thrombotic Thrombocytopenic Purpura?

Background:

Rapid clinical evaluation of immune thrombotic thrombocytopenic purpura (iTTP) is crucial due to its high mortality and the need for early intervention. Enhancing the accuracy of initial predictive models is important, as it could help improve early recognition and treatment of iTTP, and avoid unnecessary interventions in non-TTP patients. The PLASMIC score is a widely used clinical tool for predicting ADAMTS13 deficiency in suspected iTTP. However, subsequent studies have shown variability in its predictive performance across different populations. Machine learning, with its ability to analyze many continuous variables as well as the complex interactions between them, offers a promising approach to enhance predictive accuracy. We utilized XGBoost (extreme gradient boosting), a machine learning approach, to develop a novel tool that improves upon the PLASMIC score’s ability to rapidly predict iTTP.

Objectives:

To develop a novel machine learning tool to predict iTTP diagnosis that improves upon the PLASMIC score.

Methods:

We identified all ADAMTS13 tests performed between 2012-2022 using data from four US institutions: University of Utah (Salt Lake City, UT), Case Western/University Hospitals (Cleveland, OH), Rochester Regional Health (Rochester, NY), and University of Illinois (Peoria, IL). iTTP was defined as ADAMTS13 activity level < 10%, and controls were defined as ADAMTS13 activity level > 20%. We included all presenting variables from the PLASMIC score and added hemoglobin, lactate dehydrogenase (LDH), age, sex, and the presence of any neurologic deficit as potential predictors. Instead of the composite variable of hemolysis in the PLASMIC score, we used its individual components of haptoglobin, reticulocyte percentage, and indirect bilirubin as separate variables. We developed a new predictive model using XGBoost implemented in the R package xgboost. Variable importance (VI) was reported for each predictor. The model's performance was assessed using repeated 10-fold cross-validated area under the receiver operating characteristic curve (AUC). Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were reported at two thresholds for high and intermediate risk of TTP, respectively.

Results

We included 106 iTTP patients and 402 controls from four institutions. The machine learning model, named TTP-14, predicts the risk, or probability of iTTP diagnosis based on fourteen variables. The predictors, in order of highest VI were: Platelet count, LDH, creatinine, INR, reticulocyte percentage, active cancer, indirect bilirubin, MCV, total bilirubin, haptoglobin, neurologic deficit, hemoglobin, age, and history of transplant.

TTP-14 achieved a cross-validated AUC of 0.91 (95% CI: 0.90-0.92), improving upon the PLASMIC score’s AUC of 0.85 (95% CI: 0.81-0.89) in our dataset. We chose two risk cut-offs to define three risk categories:

• High risk (TTP-14 predicted risk ≥ 50%)
• Intermediate risk (TTP-14 predicted risk > 10% and < 50%)
• Low risk (TTP-14 predicted risk ≤ 10%)

For the high risk group (15.4% of patients), TTP-14 predicted iTTP diagnosis with high specificity and PPV:

• Sensitivity: 0.64
• Specificity: 0.97
• PPV: 0.83
• NPV: 0.91
• Overall accuracy: 0.90

For the low risk group (39.2% of patients), TTP-14 effectively ruled out iTTP with high NPV and sensitivity:

• Sensitivity: 0.90
• Specificity: 0.74
• PPV: 0.48
• NPV: 0.97
• Overall accuracy: 0.77

Conclusion:

• Our predictive model, TTP-14, implemented machine learning technique to accurately predict the probability of iTTP diagnosis in suspected patients at initial presentation.
• With a cross-validated AUC of 0.91, our machine learning model offered enhanced predictive performance compared to the PLASMIC score in our dataset (AUC 0.85).
• TTP-14 identified low-risk patients with an NPV of 0.97 and could confidently rule out iTTP, and high-risk patients were identified with PPV of 83% and high specificity of 97%.
• External validation is warranted to confirm these findings and integrate TTP-14 into clinical practice.
• If externally validated, TTP-14 can help improve early recognition and treatment of iTTP and reduce unnecessary interventions for non-TTP patients.