Completed

Automation Bias in Physician Diagnostic Reasoning with AI

What is being tested

ChatGPT-4o Recommendations with Hallucinations

Other

Who is being recruted

Disease

+ Pathologic Processes

+ Pathological Conditions, Signs and Symptoms

+4 Eligibility Criteria

See all eligibility criteria

How is the trial designed

Diagnostic Study

Interventional

Study Start: June 2025

See protocol details

Summary

Principal SponsorLahore University of Management Sciences

Last updated: January 28, 2026

Sourced from a government-validated database.Claim as a partner

Study start date: June 20, 2025

Actual date on which the first participant was enrolled.

Diagnostic errors are a major issue in healthcare, leading to patient harm that could otherwise be avoided. This study examines how doctors' reliance on AI tools, specifically large language models like ChatGPT-4o, might introduce errors into medical decision-making. Automation bias is a concern as doctors may overtrust AI-generated recommendations, especially in fast-paced clinical settings. The study seeks to understand how this bias affects physicians' diagnostic reasoning and aims to identify situations where AI might actually increase the risk of errors rather than reduce them. In this study, doctors are randomly divided into two groups. One group receives AI-generated medical advice with deliberate errors, while the other group gets advice without these errors. Participants review and diagnose six clinical scenarios to see how they respond to the AI's suggestions. Their decisions are then assessed by independent reviewers to identify any tendency to accept incorrect AI information. Before participating, doctors undergo training to understand AI capabilities and how to critically evaluate AI outputs. The study's results will help determine the extent of automation bias and how it can impact real-world medical decisions.

Official TitleTrust or Verify? Automation Bias in Physician-LLM Diagnostic Reasoning

NCT06963957

Principal SponsorLahore University of Management Sciences

Last updated: January 28, 2026

Sourced from a government-validated database.Claim as a partner

Protocol

This section provides details of the study plan, including how the study is designed and what the study is measuring.

Design Details

44 patients to be enrolled

Total number of participants that the clinical trial aims to recruit.

Diagnostic Study

Diagnostic studies focus on improving how we detect or confirm a disease. They test new tools or techniques that could provide faster or more accurate diagnoses.

Eligibility

Researchers look for people who fit a certain description, called eligibility criteria: person's general health condition or prior treatments.

Conditions

Criteria

Any sex

Biological sex of participants that are eligible to enroll.

Healthy volunteers allowed

If individuals who are healthy and do not have the condition being studied can participate.

Conditions

Pathology

DiseasePathologic ProcessesPathological Conditions, Signs and Symptoms

Criteria

3 inclusion criteria required to participate

Completed Bachelor of Medicine, Bachelor of Surgery (MBBS) Exam. The equivalent degree of MBBS in US and Canada is called Doctor of Medicine (MD).

Full or Provisionally Registered Medical Practitioners with the Pakistan Medical and Dental Council (PMDC).

Participants must have completed a structured training program on the use of ChatGPT (or a comparable large language model), totaling at least 10 hours of instruction. The program must include hands-on practice related to LLM's aspects, specifically prompt engineering and content evaluation.

1 exclusion criteria prevent from participating

Any other Registered Medical Practitioners (Full or Provisional) with PMDC (e.g., Professionals with Bachelor of Dental Surgery or BDS).

Study Plan

Find out more about all the medication administered in this study, their detailed description and what they involve.

Treatment Groups

Study Objectives

One single intervention group is designated in this study

This study does not include a placebo group

Treatment Groups

Group I

Active Comparator

Participants will evaluate six clinical vignettes. During the trial, they will have access to clinical recommendations from a specific, commercially available LLM (ChatGPT-4o) in addition to conventional diagnostic resources. LLM recommendations for three vignettes will contain deliberately flawed diagnostic information and for three vignettes it will contain accurate recommendations). The cases will be presented in random order.

Study Objectives

Primary Objectives

Secondary Objectives

Study Centers

These are the hospitals, clinics, or research facilities where the trial is being conducted. You can find the location closest to you and its status.

This study has 1 location

Suspended

Lahore University of Management Sciences

Lahore, PakistanOpen Lahore University of Management Sciences in Google Maps

CompletedOne Study Center