AI Decision Support for Gastrointestinal Bleeding

Overseen By Sunny Chung, MD

Age: Any Age

Sex: Any

Trial Phase: Academic

Sponsor: Yale University

No Placebo Group All trial participants will receive the active study treatment (no placebo)

Approved in 2 Jurisdictions This treatment is already approved in other countries

Compensation: Varies

Number of Visits: 2

106 Participants Needed

Trial Summary

Will I have to stop taking my current medications?

The trial information does not specify whether participants need to stop taking their current medications.

What data supports the effectiveness of the treatment AI-CDSS, GutGPT, Machine Learning Algorithm-Based Clinical Decision Support System for gastrointestinal bleeding?

Research shows that machine learning algorithms can outperform existing clinical risk scores for gastrointestinal bleeding, helping doctors make better decisions about patient care. These algorithms can identify patients at risk and guide treatment, potentially improving outcomes. ^{1 2 3 4 5}

Is the AI Decision Support for Gastrointestinal Bleeding safe for humans?

The research does not provide specific safety data for the AI Decision Support system itself, but it suggests that machine learning models can improve risk prediction for gastrointestinal bleeding, potentially enhancing patient care without indicating any direct safety concerns. ^{1 3 4 6 7}

How does the AI decision support treatment for gastrointestinal bleeding differ from other treatments?

The AI decision support treatment for gastrointestinal bleeding is unique because it uses machine learning algorithms to predict patient outcomes and guide clinical decisions, unlike traditional treatments that rely on standard clinical assessments. This approach can help identify patients at different risk levels and optimize their care by using data from electronic health records to provide real-time decision support. ^{1 3 4 5 8}

What is the purpose of this trial?

The purpose of this research study is to measure the effect on of a large language model interface on the usability, attitudes, and provider trust when using a machine learning algorithm-based clinical decision support system in the setting of bleeding from the upper gastrointestinal tract (upper GIB). Specifically, the investigators are looking to assess the optimal implementation of such machine learning algorithms in simulation scenarios to best engender trust and improve usability. Participants will be randomized to either machine learning algorithm alone or algorithm with a large language model interface and exposed to simulation cases of upper GIB.

Research Team

Dennis Shung, MD

Principal Investigator

Yale School of Medicine Section of Digestive Diseases

Eligibility Criteria

This trial is for Internal Medicine and Emergency Medicine residency trainees at the study institution. It's designed to evaluate how a machine learning algorithm, with or without a large language model interface, helps in managing upper gastrointestinal bleeding.

Inclusion Criteria

Internal Medicine residency trainees at study institution

Emergency Medicine residency trainees at study institution

Exclusion Criteria

Not applicable.

Timeline

Screening

Participants are screened for eligibility to participate in the trial

1-2 weeks

Educational Module

Participants receive a baseline educational module about artificial intelligence, machine learning, and clinical decision support

1 day

1 visit (in-person)

Simulation Scenarios

Participants engage in simulation scenarios using either a machine learning algorithm alone or with a large language model interface

60 minutes

1 visit (in-person)

Follow-up

Participants are monitored for changes in attitudes towards machine learning algorithms in clinical care

Immediately after scenarios

Treatment Details

Interventions

• Artificial Intelligent Clinical Decision Support System

Trial Overview The study tests the impact of a large language model (LLM) interface on the use of a clinical decision support system for upper gastrointestinal bleeding. Participants will be randomly assigned to use either just the algorithm or the algorithm plus LLM in simulated scenarios.

Participant Groups

Group I: Large Language Model-based Interaction Experimental Treatment 1 Intervention

LLM-powered chatbot with the machine learning dashboard to provide the risk assessment and provide rationale based on interpretability metrics provided by the dashboard in which study participants can directly interact with using natural language. Participants will be provided the Generative Pre-trained Transformer (GPT) chatbot powered machine learning model dashboard.

Group II: Machine Learning Dashboard Active Control 1 Intervention

Machine learning algorithm output with an interactive dashboard that can be used to explain, or interpret the input factors that contribute most towards the generated risk score. Participants will have access to the machine learning dashboard only.

Who Is Running the Clinical Trial?

Yale University

Lead Sponsor

Trials

1,963

Recruited

3,046,000+

National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK)

Collaborator

Trials

2,513

Recruited

4,366,000+

Findings from Research

A machine learning model developed for predicting mortality in ICU patients with acute gastrointestinal bleeding outperformed the traditional APACHE IVa risk score, achieving an area under the curve (AUC) of 0.85 compared to 0.80 for APACHE IVa.

The ML model demonstrated a higher specificity for identifying low-risk patients, with 27% specificity at 100% sensitivity, indicating it can more accurately classify patients at lower risk of mortality.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Explainable Machine Learning Model for Predicting GI Bleed Mortality in the Intensive Care Unit. Deshmukh, F., Merchant, SS.[2021]

A machine learning model developed from data of 1958 patients with upper gastrointestinal bleeding (UGIB) demonstrated superior performance in predicting the risk of hospital intervention or death within 30 days, achieving an area under the curve (AUC) of 0.91 compared to traditional scoring systems like the Glasgow-Blatchford score (AUC 0.88).

The machine learning model not only provided higher specificity at 100% sensitivity (26% vs. 12% for GBS) but also has the potential to better identify low-risk patients, allowing for safer outpatient management after emergency department visits.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding. Shung, DL., Au, B., Taylor, RA., et al.[2021]

A machine learning model called gradient boosting (GB) was developed to predict outcomes in patients with acute lower gastrointestinal bleeding (ALGIB), using non-endoscopic data from 300 patients across two hospitals.

The GB model demonstrated superior accuracy in predicting outcomes like recurrent bleeding and severe bleeding compared to traditional logistic regression models, achieving over 88% accuracy in internal validation, which could enhance patient triage and resource allocation in emergency care.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Prediction of Outcome in Acute Lower Gastrointestinal Bleeding Using Gradient Boosting. Ayaru, L., Ypsilantis, PP., Nanapragasam, A., et al.[2023]

Machine learning algorithms have the potential to significantly improve risk assessment and decision-making for gastrointestinal bleeding, outperforming traditional clinical risk scores, although they have yet to be validated in prospective clinical trials.

By utilizing electronic health records, these algorithms can automate the identification of patients with acute gastrointestinal bleeding, enabling real-time risk predictions and better triage to appropriate levels of care, ultimately enhancing patient management and outcomes.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Advancing care for acute gastrointestinal bleeding using artificial intelligence. Shung, DL.[2021]

A study involving 728 digestive tract hemorrhage samples identified 343 cases of fatal gastrointestinal bleeding recurrence, using machine learning models like XGBoost, which proved to be the most effective for prediction.

The top 12 key indicators for predicting recurrence included hemoglobin, calcium, and systolic blood pressure, achieving a high F1.5 score of 0.893, indicating strong potential for early warning in clinical settings.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

[Prediction and feature selection for fatal gastrointestinal bleeding recurrence in hospital via machine learning]. Wei, Z., Li, J., Li, X., et al.[2019]

A machine learning algorithm, particularly the random forest model, significantly outperformed traditional scoring systems like the Glasgow-Blatchford score in predicting mortality in patients with upper gastrointestinal bleeding, achieving an area under the curve of 0.917 compared to 0.710.

The voting classifier model was most effective in predicting hypotension and rebleeding within 7 days, indicating that machine learning can enhance early risk assessment for patients with initially stable non-variceal UGIB in emergency settings.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Prediction of Adverse Events in Stable Non-Variceal Gastrointestinal Bleeding Using Machine Learning. Seo, DW., Yi, H., Park, B., et al.[2020]

The implementation of a clinical decision support system significantly increased adherence to gastrointestinal prophylaxis guidelines, with compliance rising from 84.0% to 94.5% in prescriptions after the intervention, based on a study of over 4,300 prescriptions.

The introduction of this system also led to a substantial reduction in irrelevant drug safety alerts, decreasing by 78.2%, which likely contributed to the improved compliance with the guidelines.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Improving the effectiveness of drug safety alerts to increase adherence to the guideline for gastrointestinal prophylaxis. Lilih, S., Pereboom, M., van der Hoeven, RT., et al.[2019]

Explainable artificial intelligence (AI) has shown high accuracy (ranging from 0.70 to 0.98) in the early diagnosis of gastrointestinal diseases, based on a review of eight studies published since 2020.

Key predictors for gastrointestinal bleeding were identified, including mean arterial pressure and bicarbonate levels, highlighting the potential of AI to assist in clinical decision-making through non-invasive methods.

NCBI (Pubmed)

pubmed.ncbi.nlm.nih.gov

Explainable Artificial Intelligence in the Early Diagnosis of Gastrointestinal Disease. Lee, KS., Kim, ES.[2022]

References

1. United States pubmed.ncbi.nlm.nih.gov

Explainable Machine Learning Model for Predicting GI Bleed Mortality in the Intensive Care Unit. [2021]

2. United States pubmed.ncbi.nlm.nih.gov

Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding. [2021]

3. United States pubmed.ncbi.nlm.nih.gov

Prediction of Outcome in Acute Lower Gastrointestinal Bleeding Using Gradient Boosting. [2023]

4. Australia pubmed.ncbi.nlm.nih.gov

Advancing care for acute gastrointestinal bleeding using artificial intelligence. [2021]

5. China pubmed.ncbi.nlm.nih.gov

[Prediction and feature selection for fatal gastrointestinal bleeding recurrence in hospital via machine learning]. [2019]

6. Switzerland pubmed.ncbi.nlm.nih.gov

Prediction of Adverse Events in Stable Non-Variceal Gastrointestinal Bleeding Using Machine Learning. [2020]

7. Ireland pubmed.ncbi.nlm.nih.gov

Improving the effectiveness of drug safety alerts to increase adherence to the guideline for gastrointestinal prophylaxis. [2019]

8. Switzerland pubmed.ncbi.nlm.nih.gov

Explainable Artificial Intelligence in the Early Diagnosis of Gastrointestinal Disease. [2022]