Patients’ Perception of Usability and Utility of a Digital Health Tool for Electronic Patient-Reported Outcomes Monitoring in Real-Life Hematology Practice: Evidence from the Gimema-Alliance Platform

Background

Digital health tools are increasingly being used in oncology practice to better monitor patients’ health status. They may include electronic-patient-reported outcome (ePRO) monitoring systems, with automated alerts triggered to the physician depending on specific conditions (e.g., when patients report clinically relevant problems). Although implementation of these tools in real-life practice may offer valuable benefits, it is important to assess their usability and utility from the users’ standpoint.

Objective

The aim of this study was to evaluate the patients’ perception of usability and utility of a digital health tool for ePRO monitoring of patients with hematologic malignancies in real-life practice.

Methods

In December 2020, the GIMEMA Group developed a digital health platform for adult patients with hematologic malignancies (GIMEMA-ALLIANCE platform) with the goal of facilitating patient-centered care. The platform was open to enrollment until December 2022 and involved 26 hospitals. After providing written informed consent, patients received a personal password to access the secure patient portal and complete ePRO questionnaires assessing health-related quality of life (HRQoL) and symptoms (EORTC QLQ-C30 and selected items from the EORTC Item Library). Real-time graphical presentation of PRO results is displayed for both patients and physicians. The platform allows hematologists to receive real-time alerts in the presence of clinically important problems and symptoms. For the purpose of this study, a dedicated section in the patient portal was developed to evaluate the usability and utility of the platform. In this section, patients had the possibility to complete the System Usability Scale (SUS). The SUS is a 10-item widely used questionnaire to evaluate users’ perceived system satisfaction. Its score ranges from 0 to 100 and a score ≥70 is considered a threshold for an acceptable usability. Analyses were performed overall and by age group category, based on the median age of the patient population. Patients also completed an ad-hoc survey consisting of 6 items covering aspects on the utility of the platform, for example in favoring shared decision-making or improving the communication with the treating hematologist. Only patients who have completed the ePRO questionnaires at least twice, i.e. those who had the possibility to sufficiently test its functionalities, were considered eligible for completing the survey on usability and utility of the platform.

Results

Out of the 362 eligible patients, a total of 161 (44%) completed the survey. No difference in age and sex was found between patients who completed or not the survey. The median age of patients who completed the survey was 59 years (IQR: 51 - 67) and 53 (34%) were women. The most prevalent diagnosis was multiple myeloma (n=40, 25%). At the time of survey completion, 69% of the patients were receiving a treatment for their disease. The mean SUS score of the overall population was 80.8 (SD 15.5) and the majority of patients (n=131, 81%) gave a rating ≥70 (the prespecified threshold for the acceptable usability). The mean SUS score for the younger and the older groups was 80.5 (SD 14.9) and 81.4 (SD 16), respectively. Eighty-eight percent of patients agreed or strongly agreed that the platform was easy to use, 83% felt very confident in using the platform, and 72% found the various functionalities offered by the platform well integrated (Figure 1). Positive feedbacks on the utility of the platform were also collected. For example, 71% of patients considered the ePRO questionnaires useful for their health conditions and 63% would recommend its use to other patients. However, amongst the patients who had visits at the clinic (n=127), only 39% reported that their doctor discussed ePRO results with them, and this may explain the lower agreement for some items (Figure 2). For example, 38% of the patients strongly agreed/agreed that the platform helped them to improve the communication with their doctor, while 44% neither agreed or disagreed and 18% strongly disagreed/disagreed.

Conclusion

This study showed a good usability and utility of the GIMEMA-ALLIANCE platform from the patients’ perspective, and this was true for younger and older patients. Future areas of improvement should include actions to facilitate physicians in discussing ePRO results during the clinical encounter with their patients.