Impact of Publicly Reported Center Specific Analysis on Patient Selection Practices for Hematopoietic Stem Cell Transplantation

Introduction

Public reporting of outcomes drives healthcare improvement in many fields, including hematopoietic cell transplantation (HCT), although with possible unintended consequences. The Center for International Blood & Marrow Transplant Research (CIBMTR) publishes an annual Center-Specific Survival Analysis (CSA), which compares HCT centers’ observed 1-year overall survival (OS) with their statistically modeled expected 1-year OS, with 95% confidence intervals (CI). Centers with OS within their 95% CI receive a 0 score, indicating as expected outcomes. Those with OS below / above their 95% CI receive a –1 / +1 score, indicating below / above expected OS, respectively. After a –1 report, centers may change their patient selection criteria, causing unintentional systematic exclusion of patient populations who could benefit from HCT. We analyzed how the CSA report influences patient selection practices among centers receiving a –1 score.

Methods

Centers receiving a –1 report between 2012 and 2016 that had ‘as expected’ survival in the preceding 2 years were classified as newly below expected OS centers (NBCs). The year of their -1 report was used as the index year. Centers with ‘as expected’ OS in the 3 years before and after each index year were identified as control centers, reflecting expected evolution of patient selection in the HCT field. The patient population variables analyzed are shown in Table 1. The difference in patient population characteristics in the 3 years before vs the 3 years after the index years defined the change in patient selection behavior at the NBCs and the controls. A multivariate model adjusting for baseline patient population characteristics and center size was used to compare the change in patient population from before and after the index year in the NBCs and that of the controls. The difference in differences (ΔinΔ) were calculated as ΔNBC – ΔControl. A significance threshold of p<0.01 was used to account for multiple testing.

Results

After adjusting for overall trends, center size, and baseline patient population characteristics, no differences in patient selection behavior meeting the pre-specified threshold for statistical significance were identified when comparing the NBCs (n=24 centers) with the controls (n=195 centers). In the 3 years following the index years, 4,150 and 25,013 patients were transplanted at NBCs and controls, respectively. The proportion of patients receiving reduced intensity or non-myeloablative conditioning regimens decreased 4.1% in NBCs and increased 5.0% in controls, for a net difference of -9.1% (95% CI: –16.7% to –1.6%, p=0.02). All other patient characteristic proportions changed in the same direction at NBCs and controls, albeit to varying degrees (Table 1). In some high-risk characteristics, (e.g. HCT-CI > 3), a greater increase was seen at NBCs vs controls (ΔinΔ 2.1% 95% CI: –2.8% to 6.9%, p=0.40). In others, (e.g. age 60+ years), a greater increase was seen at controls vs NBCs (ΔinΔ –2.6% 95% CI: –5.9% to 0.7%, p=0.12). To capture a holistic measure of centers’ patient population risk, the predicted 1-year survival was compared in the NBCs vs controls using the logistic regression model generated for each year’s CSA. The predicted OS increased by 3.08% and 3.30% in the NBCs and controls respectively, with no statistically significant difference (–0.23%, 95% CI: –1.4% to 0.9%, p=0.70, Figure 1). The observed overall survival (adjusted for predicted 1 year OS) also increased in both BECs and controls by 0.9% and 4.5% respectively, without statistically significant difference (–3.6%, 95% CI: –6.7% to –0.7%, p=0.02).

Discussion

For centers receiving a -1 report, no statistically significant changes were seen in patient population characteristics in the following 3 years when compared to centers with OS was as expected. There was variability in the changes in high-risk patient characteristics at the NBCs relative to the controls, and in some cases more patients with high-risk characteristics were selected at these centers compared to those at controls. The change in predicted overall survival at 1 year, a summary indicator of survival risk, was similar between BECs and control centers. Although the number of patients in centers with below expected OS was small, these findings suggest the public reporting of outcomes in HCT in the US does not unintentionally affect access to HCT for high-risk patients at NBCs.