Development of a Real-Time Dashboard Characterizing Acute Leukemia Clinical Trial Enrollment Diversity at the Practice, Investigator, and Individual Clinician Level

Introduction

Adult acute leukemia clinical trials in the United States under-enroll persons from historically marginalized groups. Practices, investigators, and individual clinicians have identified the lack of multidimensional, real-time enrollment data as a barrier to remediating participation inequities. We sought to overcome this by developing a visual data interface (“dashboard”) for assessing enrollment diversity across an institution in real-time.

Methods

Real-time dashboard data sources were the electronic health record (EPIC) and the trial registration system (OnCore); Tableau software was used for data visualization. The primary objective was to create an integrated data source that accurately assigned individuals as patients of the correct clinical group, primary oncologist, and research protocol(s). The primary endpoint was an accurate assignment algorithm, defined as an area under the receiver operator curve (AUROC) of >=0.90, as compared to manual review. Trial enrollment was assigned via OnCore and was itself the reference standard, so algorithm classification was considered accurate if individuals were assigned correctly to the clinical group and primary oncologist. The AUROC of a basic assignment algorithm was 0.78, requiring a sample size of 528 to show improvement to 0.90 or above. Interim testing was performed to assess potential improvements in logic accuracy. Exploratory objectives were to assess associations between exposures (e.g., age, race, ethnicity, sex) and outcomes (treatment trial and biobank enrollment) among a one-year cohort of patients followed by the clinical group, as determined by the final algorithm. Due to small numbers in some race and ethnicity categories, a People of Color aggregate variable was generated and included those listed as Hispanic Black, Other, Unknown, White, and Non-Hispanic Asian, Black, and Other. Associations were determined using parametric or non-parametric bivariate tests, as appropriate; multiplicity was not considered.

Results

Interim testing was used to revise diagnostic criteria assignment and inpatient encounter algorithms. This improved classification AUROCs to 0.80 and then 0.85, after which the physician assignment algorithm was further revised and finalized. Final testing of 528 patients seen on randomly chosen dates resulted in an AUROC of 0.98 for the assignment algorithm against manual classification. Real-time data visualizations were then developed to display multidimensional enrollment diversity data grouped by patient panels for individual clinicians (Figure 1) and by trial for primary investigators, with additional views showing trends over time and enrollment process metrics (i.e., who is consented, eligible, on study). From 6/22/2022-6/21/2023 there were 432 unique patients followed by the clinical group, of which 124 (28.7%) participated in a treatment trial and 287 (66.4%) in a biobanking study. Patient characteristics and bivariate comparisons of participation by study type are shown in Table 1. Characteristics associated with treatment trial participation included lower Social Deprivation Index score (20 vs 27, p=0.05), residing outside Massachusetts (36.2% vs 25.5%, p=0.03), and having ALL (33.8% vs 25.8%, p=0.02). Characteristics associated with biobank study participation included lower Social Deprivation Index score (21 vs 25, p=0.01) and having ALL (33.8% vs 25.8%, p=0.02). While Persons of Color participated in treatment trials (22.4% vs 29.7%) and biobanks (62.1% vs 66.7%) numerically less than NHW, but differences were not significant (p>0.05).

Conclusions

An enrollment diversity dashboard can accurately provide real-time data on acute leukemia clinical research participation at a large referral center. In this cohort, enrollment rates were relatively high and associated with socioeconomic status, place of residence, and disease type but not race-ethnicity. The dashboard is now being revised to incorporate trial eligibility metrics and will be piloted as performance feedback for practices, investigators, and individual clinicians.