Research and Evaluation of the Diagnostic Precision of an Artificial Intelligence Algorithm Using Ultrasound Images for Early Diagnosis of Arthropathy in People with Hemophilia

Background: Frequent joint bleeding in people with hemophilia can cause synovitis and progressive joint damage, leading to reduced quality of life over the long term. Despite widespread use of prophylactic treatment, frequent joint bleeding is still prevalent. Early diagnosis of symptomatic/asymptomatic hemarthrosis and synovitis is required through regular assessment to ensure appropriate therapeutic intervention. Ultrasonography is a convenient diagnostic tool, but accurate diagnosis of hemarthrosis and synovitis requires specialist skills and experience, and so its implementation in daily clinical practice is not currently widespread. By constructing an artificial intelligence (AI) algorithm to support diagnosis of hemarthrosis and synovitis using ultrasonography, increased uptake of joint ultrasonography is expected. This study researches and evaluates an AI algorithm for detecting the presence or absence of hemarthrosis and synovitis using ultrasound images obtained in people with hemophilia.

Methods: This study evaluated the diagnostic precision of AI model-based diagnoses in people with congenital hemophilia A or B. Ankle, elbow, and knee joint ultrasound images obtained at study sites between January 1, 2010 and March 31, 2022 were used to train and test six AI models for each joint to estimate the presence or absence of hemarthrosis and synovitis. The ultrasound images were stratified by presence or absence of hemarthrosis and synovitis, as determined by an Executive Committee consisting of a physician and two orthopedists who are experts in hemophilia care. Images were then randomly divided, with 70% used to train the AI models and 30% used to test their diagnostic precision. The AI algorithm used various image pre-processing methods and the convolutional neural network-based AI model. The primary endpoint was the area under the receiver operating characteristic curve (AUC) for the diagnostic precision of the AI model to diagnose hemarthrosis and synovitis. The other endpoints were examined using parameters such as the rate of accuracy, precision, recall, and specificity. In addition, to evaluate whether the lesion area was detected correctly or not by visualization techniques, we displayed the lesion area with region of interest and expressed the discrimination basis using a heat map (Figure 1).

Results: A total of 5649 images were collected from five study sites. Following assessment by the Study Executive Committee, 3435 images were used in this analysis. The AUC for hemarthrosis detection for the ankle, elbow, and knee joints was 0.87, 0.91, and 0.91, respectively, while for synovitis detection it was 0.90, 0.97, and 0.91, respectively (Table 1). When evaluated in subgroups according to age, AUCs for hemarthrosis detection for at the ankle, elbow, and knee joints, respectively, were: 0.85, 0.94, and 0.98 at 10–19 years old; 0.82, 0.91, and 0.84 at 20–29 years old; 0.91, 0.90, and 0.89 at 30–39 years old; 0.90, 0.93, and 0.93 at 40–49 years old; and 0.91, 0.93, and 0.95 at 50–59 years old. AUCs for synovitis detection for the ankle, elbow, and knee joints, respectively, for different age groups were 0.95, 0.92, and 0.94 at 10–19 years old; 0.89, 0.97, and 0.95 at 20–29 years old; 0.94, 1.00, and 0.91 at 30–39 years old; 0.84, 0.96, and 0.91 at 40–49 years old; and 0.90, 0.98, and 0.91 at 50–59 years old. The analysis of the hemarthrosis and synovitis algorithms by joint site indicated high accuracy, precision, recall, and specificity (Table 1).

Conclusions: Although the AI model produced high AUC values for detecting hemarthrosis and synovitis, further refinement is needed before it can be used in diagnosis. Additional research should determine why the AI model and physician results differed. This may be due to image brightness or AI training data quality/quantity. As joint shape changes greatly with growth, and the AI model uses the bone surface as a landmark, pediatric (<10 years old) images were excluded; the model has low discrimination accuracy if a bone surface cannot be detected. In future, features associated with joint growth should be incorporated into the algorithm to allow more accurate pediatric diagnosis. The future ability of an AI model to diagnose hemarthrosis and synovitis in clinical practice would support appropriate therapeutic intervention. This would be expected to help with the achievement of a healthy and active life for people with hemophilia.