Predicting Caregiver Burden in Multiple Myeloma: Insights from the Carmma Study Using Machine Learning Models

Introduction

Multiple Myeloma (MM) has a significant impact on the quality of life (QoL) of patients and is associated with psychological, physical, social and financial stress on caregivers. Our previous work showed that older age, poor fitness, advanced disease stage and a longer course of treatment are associated with a higher burden of caregiving. We aim to design a method for early prediction of the need for a caregiver (NFC) and burden of caregiving (BoC).

Methods

CarMMa (“Characterization of patients with Multiple Myeloma treated in Portuguese Hospitals and of their caregivers”) is a multicentric, national, cross-sectional study that gathered data on MM patients, their disease, and their caregivers. The QASCI questionnaire was used to assess the BoC in seven dimensions - Emotional Burden, Personal Life Implications (PLI), Financial Overload (FO), Reactions to Demands, Mechanisms of Efficacy and Control (MEC), Familial Support (FSu, and Satisfaction with the Role - which were graded on a 5-point Linkert scale and normalized. Each dimension’s average was used to obtain the total BoC. QoL was assessed using the EORTC Core QoL (QLQ-C30 3rd version) and QoL Multiple Myeloma module (QLQ-MY20) questionnaires.

Data was divided into training (60%) and testing (40%) sets using stratified splitting to maintain the outcome prevalence in both sets. For the task of predicting the NFC, the training set was used to optimize and compare different classifiers using 10-fold cross-validation. A 3-fold cross-validation was used to develop models to predict different dimensions of burden due to the smaller cohort size in these tasks. The Matthews Correlation Coefficient (MCC) of CatBoost, Extra Trees, Extreme Gradient Boosting (EGB), Light Gradient Boosting Machine (LGBM), Random Forest, and Logistic Regression (LR) models were used to select the best model, which was then used to assess the performance on the testing set. Model performance on the testing set was assessed using ROC AUC, accuracy, sensitivity, and positive predictive value (PPV). This procedure was performed using PyCaret (version 3.3.1), where one-hot encoding was used for discrete low-cardinality features, and skrub (version 0.2.0) was used to encode high-cardinality features. The impact of the features on each model was determined using SHAP analysis.

Results

From July 2022 until May 2024, we enrolled 448 MM patients from 11 Portuguese centers. The median age was 68 (39-89) years; 54.8% were male, and 63.6% were newly diagnosed. The patients’ performance status (PS) was 0-1 in 77.3% and International Staging System (ISS) 2-3 in 70.9%. Patients with high-risk cytogenetics and extramedullary disease were 37.6% and 17.4%, respectively. The median time from diagnosis to data collection was 35.5 months.

A significant caregiver was identified by 41.3% of the patients, being 74.1% female, with a median age of 61.5 years. Our best predictive model for the NFC was obtained using the CatBoost Classifier and displayed an accuracy of 78%, sensitivity of 73%, PPV of 76%, and a ROC AUC of 0.84. The patient’s features with the highest impact in this model were age, PS, and ISS.

The median score for total BoC was low (21.8%) but four specific dimensions displayed significant rates of high or extreme burden: PLI (in 19.9% of the caregivers), MEC (12.6%), FSu (12.3%) and FO (11.4%).

For total burden, the Extra Trees model outperformed other algorithms yielding an accuracy of 71%, sensitivity of 60%, PPV of 71%, and ROC AUC of 0.67. Regarding PLI, FO, and MEC the best models were the CatBoost, EGB, and LGBM, which attained accuracies of 64%, 69%, 63%, sensitivities of 72%, 75%, 71%, PPVs of 64%, 70%, 67%, and ROC AUCs of 0.65, 0.71, 0.56, respectively. For the total BoC model, the feature with the highest impact on the models was ISS, while for the PLI, FO, and MEC models it was the gender, frailty, and QoL features of depression and pain.

Discussion and Conclusions:

Using machine learning (ML) to analyze the national data from the CarMMa study we were able to generate a predictive model of the need for a caregiver in MM patients. Using clinical and demographic parameters as input variables, ML also provides relevant insights into the factors predicting burden of caregiving in personal and familial dimensions, such as the patient’s age and gender, fitness score and disease stage. This sets a new standard for a potentially valuable tool to guide social support planning in real-world setting.