ASH-G Bot: An Artificial Intelligent Chatbot Trained on the American Society of Hematology (ASH) Guidelines

Background:

General-purpose language models like ChatGPT, while versatile and proficient across a broad spectrum of topics, often fall short in delivering the specialized knowledge and precision necessary for specific fields such as oncology and hematology. The American Society of Hematology (ASH) guidelines serve as an invaluable resource for hematology professionals, offering comprehensive insights and directives. However, the sheer volume and complexity of these documents can make it challenging for practitioners to quickly extract pertinent information.

To bridge this critical gap, we introduce ASH-Gbot, an advanced AI-driven chatbot meticulously trained on the ASH guidelines. ASH-Gbot is specifically tailored to deliver precise, guideline-adherent advice, empowering hematologists with immediate access to expert recommendations and enhancing clinical decision-making.

Methods:

To develop ASH-Gbot, we harnessed the advanced language understanding capabilities of GPT-4 as the foundation. This customization involved a rigorous training regimen utilizing the complete text of the ASH guidelines. Our training process was meticulously crafted to ensure that ASHGbot not only assimilates the detailed information within these guidelines but also comprehends the context and intricacies of hematological care. We systematically extracted and organized the text from the ASH guidelines, ensuring comprehensive coverage of all relevant topics and recommendations. Using this extensive dataset, we trained ASH-Gbot to understand and apply the guidelines effectively, including fine-tuning the model to recognize the nuances and specific requirements of hematology practice. We created disease-specific versions of ASH-Gbot, each fine-tuned with guideline subsets pertinent to particular hematological conditions, ensuring highly relevant and precise advice. Additionally, we augmented the training with specific prompts to foster interactions that are kind, sincere, and respectful, particularly for patient-oriented responses. The model was designed to discern whether the user is a healthcare professional or a patient and adjust the complexity of its responses accordingly, with an emphasis on delivering information in a kind, responsible, and empathetic manner for patients.

Results:

ASH-Gbot's performance was rigorously evaluated for accuracy and reasoning against real-world clinical scenarios. The responses generated by the model were meticulously reviewed by experienced hematologists and researchers to confirm adherence to ASH guidelines and clinical relevance. This comprehensive evaluation process involved comparing ASH-Gbot's outputs with expert human responses, revealing that the model achieved a level of accuracy comparable to that of seasoned hematologists. The information extracted by ASH-Gbot was consistently correct and validated in accordance with ASH guidelines, demonstrating a high level of precision and contextual understanding. Furthermore, ASH-Gbot exhibited a remarkable ability to tailor its communication style based on the user's background, providing simplified explanations for patients and more detailed, technical responses for healthcare professionals. This dual capability underscores ASHGbot's potential as a valuable tool for enhancing clinical decision-making and improving patient education within the hematology community. Link to ASH-Gbot.

Conclusions:

We developed ASH-Gbot using GPT-4 and trained it extensively on the ASH guidelines, ensuring precise, disease-specific advice for hematological conditions. The model's accuracy was validated through rigorous evaluation, showing performance comparable to experienced hematologists. ASH-Gbot significantly enhances clinical decision-making and patient education by providing accurate, contextually relevant information tailored to users' backgrounds. This integration of AI and hematology resources marks a critical advancement in healthcare, promising improved standards of care and streamlined workflows in hematology.