-Author name in bold denotes the presenting author
-Asterisk * with author name denotes a Non-ASH member
Clinically Relevant Abstract denotes an abstract that is clinically relevant.

PhD Trainee denotes that this is a recommended PHD Trainee Session.

Ticketed Session denotes that this is a ticketed session.

4818 The Augmented Hematologist: Human-AI Feedback Loops to Assist Differential Cell Count during the Analysis of Bone Marrow Aspirates

Program: Oral and Poster Abstracts
Session: 803. Emerging Tools, Techniques and Artificial Intelligence in Hematology: Poster III
Hematology Disease Topics & Pathways:
artificial intelligence (AI), Technology and Procedures, imaging, machine learning
Monday, December 12, 2022, 6:00 PM-8:00 PM

David Bermejo-Peláez, PhD1*, Sandra Rueda Charro, MD2*, Adriana Mousa2*, Elisa Alamo2*, Maria Garcia Roa3*, Roberto Trelles-Martínez3*, Marta Hidalgo Soto4*, Alejandro Bobes-Fernández5*, Roberto Garcia-Vicente6,7*, María Luz Morales6,7*, Alba Rodríguez García6,8*, Alejandra Ortiz-Ruiz6,7*, Alberto Blanco-Sánchez, MD9*, Lin Lin2,10*, Elena Dacal2*, Rosa Ayala6,11*, Miguel Luengo-Oroz2*, María Linares6,9,12* and Joaquin Martinez Lopez6,7,9*

1Spotlab, Madrid, AL, Spain
2Spotlab, Madrid, Spain
3Department of Hematology, Hospital Universitario Fundación Alcorcón, Alcorcón, Spain
4Vall Hebron Institute of Oncology (VHIO), Barcelona, Spain
5Department of Hematology, Hospital Clínico San Carlos, Madrid, Spain
6Department of Translational Hematology, Research Institute Hospital 12 de Octubre (imas12), Madrid, Spain
7Hematological Malignancies Clinical Research Unit H120-CNIO, CIBERONC, Madrid, Spain
8Hematological Malignancies Clinical Research Unit H12O-CNIO, Madrid, Spain
9Department of Hematology, Hospital Universitario 12 de Octubre, Madrid, Spain
10Biomedical Image Technologies Laboratory, ETSI Telecomunicación, Universidad Politécnica de Madrid, Madrid, Spain
11Haematological Malignancies Clinical Research Unit, Hospital 12 de Octubre Universidad Complutense, CNIO, CIBERONC, Madrid, Spain
12Department of Biochemistry and Molecular Biology, Pharmacy School, Universidad Complutense de Madrid, Madrid, Spain

Introduction
Every day, thousands of hematologists worldwide analyze bone marrow aspirate (BMA) samples supported by an optical microscope and their own clinical expertise. A crucial part of the BMA analysis, the differential cell count (DCC), is still a time-consuming task that wastes professionals’ energy and time, and its results are subject to interobserver variability (Fuentes-Arderiu 2009). Significant advances have been made to date to assist professionals during the DCC process and to increase their efficiency by leveraging artificial intelligence. However, these tools have not been widely implemented in real clinical environments due to the complexity and high price of the devices that allow the digitization of BMA samples (Chandradevan 2020, Jin 2020).
We aimed to develop and evaluate a simple and integrated digital system (see Figure) to cover the entire process, from sample digitization to DCC facilitated by human-artificial intelligence (AI) interaction, by using a 3D printed microscopy adapter arm, a smartphone, a mobile application, and a web-based telemedicine platform.

Methods
First, a hematologist or a laboratory technician digitizes 20 microscopy fields (x100 objective) using the 3D-printed device that allows coupling a mobile phone and aligning its camera with a conventional optical microscope’s ocular lens, and a mobile app customized specifically for fast, standardized, and easy digitization of BMA microscopy images (Dacal 2021).
Once the images have been uploaded to the cloud, hematologists can visualize the samples by scrolling and zooming the images and label the different types of cells. This annotations were used to train the AI algorithm. Full traceability of the system allows multiple users to label the same image and also review others' analyses. Additionally, and once the AI algorithm has been developed, as the image samples arrive in the cloud system, an it automatically analyzes the images by detecting nucleated cells and classifying them into the 6 cell lineages (Myeloid, Erythroid, Monocytic, Lymphoid, Blasts, and Plasma cells). This process runs in a few seconds after image upload, so when hematologists review the image, the virtual AI annotations on the image sample are available and ready to be edited, confirmed, or deleted to obtain a satisfactory final DCC.
To evaluate the system’s impact on the DCC process, 16 BMA different from those used to train the algorithm were digitized using the 3D-printed adapter, and 4 hematologists remotely analyzed and classified at least 500 cells per BMA sample using the web platform. Each BMA sample was analyzed by 1 hematologist in a blind fashion (without AI support) and by 3 hematologists with the AI algorithm support. The selection of hematologists who were assisted by the AI and those who performed the blinded analysis was done on a rotational basis. Time to perform DCC (500 cells) and interobserver variability by cell lineage were also measured in both groups.
This system was supported by a two-stage deep learning algorithm that was created to detect and classify hematopoietic stem cells in 6 lineages from BMA images. A total of 22446 cells from 100 BMA samples were used to train the algorithm. AI algorithm performance was calculated by comparing its lineage prediction and the consensus label (majority vote) among 4 experts for each cell (N=4401 cells).

Results
The average performance of the AI algorithm for classifying cells in BMA samples was 92.52% (95%CI 90.92-93.62) (98.25% for myeloid, 92.43% for erythroid, 82.60% for blasts, 58% for plasma cells, 86.82% for lymphoid and 67.63% for monocytic class).
The interobserver agreement significantly increased when BMA analyses were performed with AI support, increasing from 92.27% (95%CI 90.92-93.62) when hematologists analyzed the samples without AI assistance to 94.11% (95%CI 93.28-94.93) when they were supported by AI. The time to perform a DCC (on 500 cells) decreased when AI was used, from 26.4 minutes in the blinded group to 21.8 minutes for the AI-supported experts.

Conclusions
Thanks to mobile technologies, any laboratory in the world can easily digitize and analyze BMA samples. AI can support hematologists when analyzing BMA samples increasing efficiency and reducing interobserver variability. Systems designed to augment hematologists' quantitative and analytics capabilities require interacting with AI in an intuitive and user-friendly way.

Disclosures: Bermejo-Peláez: Spotlab: Current Employment. Rueda Charro: Spotlab: Current Employment. Mousa: Spotlab: Current Employment. Alamo: Spotlab: Current Employment. Lin: Spotlab: Current Employment. Dacal: Spotlab: Current Employment. Luengo-Oroz: Spotlab: Current Employment, Current equity holder in private company. Linares: Spotlab: Consultancy.

*signifies non-member of ASH