AI Decision Support for Gastrointestinal Bleeding

Overseen By Sunny Chung, MD

Age: Any Age

Sex: Any

Trial Phase: Academic

Sponsor: Yale University

No Placebo Group All trial participants will receive the active study treatment (no placebo)

Approved in 2 Jurisdictions This treatment is already approved in other countries

Compensation: Varies

Number of Visits: 2

What You Need to Know Before You Apply

What is the purpose of this trial?

This trial aims to evaluate how effectively the Artificial Intelligent Clinical Decision Support System (AI-CDSS) assists doctors in managing upper gastrointestinal bleeding, a condition involving bleeding in the upper digestive system. Researchers seek to determine whether integrating a natural language chatbot enhances the tool's trustworthiness and usability. Participants will use either the machine tool alone or the tool with the chatbot in practice scenarios. Doctors training in internal or emergency medicine at the study institution are well-suited for this trial. As an unphased trial, this study provides a unique opportunity to contribute to innovative technology that could improve medical decision-making.

Will I have to stop taking my current medications?

The trial information does not specify whether participants need to stop taking their current medications.

What prior data suggests that this AI decision support system is safe for use in gastrointestinal bleeding cases?

Research has shown that using AI in healthcare is generally safe. One study on similar AI tools for stomach bleeding found that these systems can accurately predict risks and follow medical guidelines to provide reliable information. This suggests that the AI system in this trial is likely safe to use.

Although this trial focuses on the AI's usability and trustworthiness, past experiences with AI in healthcare have shown little risk to participants. AI doesn't involve taking medicine or undergoing physical treatments, which usually means fewer side effects. This trial uses AI to assist doctors in decision-making, focusing on enhancing doctors' work rather than directly affecting the body.

In summary, joining this study is expected to be safe based on previous research and the functioning of decision support systems. ^{1 2 3 4 5}

Why are researchers excited about this trial?

Researchers are excited about using Artificial Intelligent Clinical Decision Support Systems for gastrointestinal bleeding because these systems offer a new way to assess and manage patient risk. Unlike conventional treatments that rely heavily on physician judgment and standard procedures, this AI-driven approach uses a Large Language Model (LLM)-powered chatbot combined with a machine learning dashboard. This technology provides risk assessments and offers rationale based on interpretability metrics, allowing for direct interaction with the system using natural language. This could potentially enhance decision-making precision and personalize patient care, leading to better outcomes.

What evidence suggests that this AI decision support system is effective for gastrointestinal bleeding?

Research has shown that machine learning tools can outperform current methods in predicting the risk of gastrointestinal bleeding, aiding doctors in decision-making. In this trial, participants will use two different AI decision support systems. One group will use GutGPT, an AI system that offers risk predictions and answers based on clinical guidelines, paired with a chatbot interface. Early tests suggest this setup is easier to use and more trustworthy, as it clarifies the reasoning behind risk scores, helping doctors understand and trust the system. The other group will access a machine learning dashboard only, which explains the input factors contributing to the risk score. Overall, AI decision support systems show promise in managing upper gastrointestinal bleeding. ^{2 3 4 6 7}

Who Is on the Research Team?

Dennis Shung, MD

Principal Investigator

Yale School of Medicine Section of Digestive Diseases

Are You a Good Fit for This Trial?

This trial is for Internal Medicine and Emergency Medicine residency trainees at the study institution. It's designed to evaluate how a machine learning algorithm, with or without a large language model interface, helps in managing upper gastrointestinal bleeding.

Inclusion Criteria

Internal Medicine residency trainees at study institution

Emergency Medicine residency trainees at study institution

Exclusion Criteria

Not applicable.

Timeline for a Trial Participant

Screening

Participants are screened for eligibility to participate in the trial

1-2 weeks

Educational Module

Participants receive a baseline educational module about artificial intelligence, machine learning, and clinical decision support

1 day

1 visit (in-person)

Simulation Scenarios

Participants engage in simulation scenarios using either a machine learning algorithm alone or with a large language model interface

60 minutes

1 visit (in-person)

Follow-up

Participants are monitored for changes in attitudes towards machine learning algorithms in clinical care

Immediately after scenarios

What Are the Treatments Tested in This Trial?

Interventions

• Artificial Intelligent Clinical Decision Support System

Trial Overview

The study tests the impact of a large language model (LLM) interface on the use of a clinical decision support system for upper gastrointestinal bleeding. Participants will be randomly assigned to use either just the algorithm or the algorithm plus LLM in simulated scenarios.

How Is the Trial Designed?

Group I: Large Language Model-based Interaction Experimental Treatment 1 Intervention

LLM-powered chatbot with the machine learning dashboard to provide the risk assessment and provide rationale based on interpretability metrics provided by the dashboard in which study participants can directly interact with using natural language. Participants will be provided the Generative Pre-trained Transformer (GPT) chatbot powered machine learning model dashboard.

Group II: Machine Learning Dashboard Active Control 1 Intervention

Machine learning algorithm output with an interactive dashboard that can be used to explain, or interpret the input factors that contribute most towards the generated risk score. Participants will have access to the machine learning dashboard only.

Who Is Running the Clinical Trial?

Yale University

Lead Sponsor

Trials

1,963

Recruited

3,046,000+

National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK)

Collaborator

Trials

2,513

Recruited

4,366,000+

Published Research Related to This Trial

A machine learning algorithm, particularly the random forest model, significantly outperformed traditional scoring systems like the Glasgow-Blatchford score in predicting mortality in patients with upper gastrointestinal bleeding, achieving an area under the curve of 0.917 compared to 0.710.

The voting classifier model was most effective in predicting hypotension and rebleeding within 7 days, indicating that machine learning can enhance early risk assessment for patients with initially stable non-variceal UGIB in emergency settings.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Prediction of Adverse Events in Stable Non-Variceal Gastrointestinal Bleeding Using Machine Learning. Seo, DW., Yi, H., Park, B., et al.[2020]

The implementation of a clinical decision support system significantly increased adherence to gastrointestinal prophylaxis guidelines, with compliance rising from 84.0% to 94.5% in prescriptions after the intervention, based on a study of over 4,300 prescriptions.

The introduction of this system also led to a substantial reduction in irrelevant drug safety alerts, decreasing by 78.2%, which likely contributed to the improved compliance with the guidelines.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Improving the effectiveness of drug safety alerts to increase adherence to the guideline for gastrointestinal prophylaxis. Lilih, S., Pereboom, M., van der Hoeven, RT., et al.[2019]

Machine learning algorithms have the potential to significantly improve risk assessment and decision-making for gastrointestinal bleeding, outperforming traditional clinical risk scores, although they have yet to be validated in prospective clinical trials.

By utilizing electronic health records, these algorithms can automate the identification of patients with acute gastrointestinal bleeding, enabling real-time risk predictions and better triage to appropriate levels of care, ultimately enhancing patient management and outcomes.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Advancing care for acute gastrointestinal bleeding using artificial intelligence. Shung, DL.[2021]

A machine learning model called gradient boosting (GB) was developed to predict outcomes in patients with acute lower gastrointestinal bleeding (ALGIB), using non-endoscopic data from 300 patients across two hospitals.

The GB model demonstrated superior accuracy in predicting outcomes like recurrent bleeding and severe bleeding compared to traditional logistic regression models, achieving over 88% accuracy in internal validation, which could enhance patient triage and resource allocation in emergency care.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Prediction of Outcome in Acute Lower Gastrointestinal Bleeding Using Gradient Boosting. Ayaru, L., Ypsilantis, PP., Nanapragasam, A., et al.[2023]

A machine learning model developed from data of 1958 patients with upper gastrointestinal bleeding (UGIB) demonstrated superior performance in predicting the risk of hospital intervention or death within 30 days, achieving an area under the curve (AUC) of 0.91 compared to traditional scoring systems like the Glasgow-Blatchford score (AUC 0.88).

The machine learning model not only provided higher specificity at 100% sensitivity (26% vs. 12% for GBS) but also has the potential to better identify low-risk patients, allowing for safer outpatient management after emergency department visits.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding. Shung, DL., Au, B., Taylor, RA., et al.[2021]

A study involving 728 digestive tract hemorrhage samples identified 343 cases of fatal gastrointestinal bleeding recurrence, using machine learning models like XGBoost, which proved to be the most effective for prediction.

The top 12 key indicators for predicting recurrence included hemoglobin, calcium, and systolic blood pressure, achieving a high F1.5 score of 0.893, indicating strong potential for early warning in clinical settings.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

[Prediction and feature selection for fatal gastrointestinal bleeding recurrence in hospital via machine learning]. Wei, Z., Li, J., Li, X., et al.[2019]

A machine learning model developed for predicting mortality in ICU patients with acute gastrointestinal bleeding outperformed the traditional APACHE IVa risk score, achieving an area under the curve (AUC) of 0.85 compared to 0.80 for APACHE IVa.

The ML model demonstrated a higher specificity for identifying low-risk patients, with 27% specificity at 100% sensitivity, indicating it can more accurately classify patients at lower risk of mortality.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Explainable Machine Learning Model for Predicting GI Bleed Mortality in the Intensive Care Unit. Deshmukh, F., Merchant, SS.[2021]

Explainable artificial intelligence (AI) has shown high accuracy (ranging from 0.70 to 0.98) in the early diagnosis of gastrointestinal diseases, based on a review of eight studies published since 2020.

Key predictors for gastrointestinal bleeding were identified, including mean arterial pressure and bicarbonate levels, highlighting the potential of AI to assist in clinical decision-making through non-invasive methods.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Explainable Artificial Intelligence in the Early Diagnosis of Gastrointestinal Disease. Lee, KS., Kim, ES.[2022]

Citations

1.

nature.com

nature.com/articles/s41746-025-01896-5

Usability and adoption in a randomized trial of GutGPT ...

Assessing the usability of gutgpt: a simulation study of an AI clinical decision support system for gastrointestinal bleeding risk. Preprint ...

2.

arxiv.org

arxiv.org/abs/2312.10072

[2312.10072] Assessing the Usability of GutGPT

GutGPT provides risk predictions from a validated machine learning model and evidence-based answers by querying extracted clinical guidelines.

3.

clinicaltrial.be

clinicaltrial.be/en/details/265040?per_page=100&only_recruiting=0&only_eligible=0

Artificial Intelligent Clinical Decision Support System S...

The study will evaluate the effect of a large language model-based interaction with the machine learning algorithm with interpretability ...

4.

pmc.ncbi.nlm.nih.gov

pmc.ncbi.nlm.nih.gov/articles/PMC10882311/

Machine learning in the assessment and management of ...

Integration of machine learning has the potential to transform the management of acute gastrointestinal bleeding, but a transparent and collaborative approach ...

5.

withpower.com

withpower.com/trial/phase-gastrointestinal-hemorrhage-4-2023-0cd2b

AI Decision Support for Gastrointestinal Bleeding

Research shows that machine learning algorithms can outperform existing clinical risk scores for gastrointestinal bleeding, helping doctors make better ...

6.

clinicaltrials.gov

clinicaltrials.gov/study/NCT05816473

Study Details | NCT05816473 | Artificial Intelligent ...

The purpose of this research study is to measure the effect on of a large language model interface on the usability, attitudes, and provider trust when using a ...

7.

pmc.ncbi.nlm.nih.gov

pmc.ncbi.nlm.nih.gov/articles/PMC11874509/

Advancing care for acute gastrointestinal bleeding using ...

Machine learning algorithms can be used to identify patients with acute gastrointestinal bleeding using data extracted from the electronic health record.