LLM-Based Tool Shows High Accuracy in Flagging Contraindications to Stroke Thrombolysis

A large language model (LLM)-driven screening tool developed by Cleveland Clinic demonstrates high sensitivity and near-perfect negative predictive value for identifying thrombolysis contraindications in telestroke evaluations. So finds a retrospective study reported by Cleveland Clinic researchers in a late-breaking presentation at the International Stroke Conference 2026.

“Our tool, which is integrated with the electronic health record (EHR), promises to be a scalable and rapid safety net for hyperacute stroke workflows,” says the study’s first and corresponding author, Bing Yu Chen, MDCM, a stroke neurologist with Cleveland Clinic’s Cerebrovascular Center. “It is designed to complement neurologist oversight rather than replace it.”

He continues: “We believe this tool can be integrated into practice to be automatically triggered when acute stroke pages are activated, safely accelerating the review process for contraindications while keeping clinicians firmly in control.”

A better way to review for contraindications?

The thrombolytic therapy tenecteplase carries more than 20 contraindications for use in the setting of acute ischemic stroke. Manually reviewing a patient’s EHR documentation for these contraindications is typically time-consuming and can be prone to error. “This is true even for experienced stroke clinicians, particularly if patients present at inopportune times like the middle of the night,” says Dr. Chen.

To address this challenge, he and Fares Antaki, MD, a vitreoretinal surgery fellow at Cleveland Clinic with deep artificial intelligence (AI) expertise, designed an AI tool called NeuroGlimpse that was integrated with the Epic EHR used throughout Cleveland Clinic. The tool’s design was accomplished using OpenAI’s o3-medium LLM deployed within a HIPAA-compliant Microsoft Azure environment. It uses custom prompts to scan patient charts in real time and quickly generate a list of potential contraindications to tenecteplase with source references.

The aim was to reduce the time and clinician burden involved in screening for contraindications while enhancing screening accuracy by supplementing clinician review with AI review, Dr. Chen explains. “NeuroGlimpse is a clinician-driven innovation that was designed by stroke neurologists for stroke neurologists,” he says, citing the senior mentorship of Ken Uchino, MD, Director of Research and Education in Cleveland Clinic’s Cerebrovascular Center, along with the support of Cleveland Clinic’s AI task force and funding from the health system’s Catalyst Grants program.

Design of the retrospective study

The new study is the team’s most comprehensive assessment of NeuroGlimpse to date. Dr. Chen and colleagues analyzed all consecutive adult telestroke patients evaluated across 16 Cleveland Clinic sites in Ohio during March 2025.

For each patient, the researchers retrospectively input the tool’s published prompt and notes into the NeuroGlimpse model. The 30 most recent clinician-authored notes from the patient’s EHR were retrieved as inputs, regardless of elapsed time. Even patients with no prior notes were included, to ensure testing under real-world conditions. The model then generated a structured contraindication list, with source attribution, for each patient.

To evaluate the outputs, two board-certified vascular neurologists, both masked to the NeuroGlimpse outputs, independently reviewed the pre-evaluation EHR inputs to identify contraindications according to national guidelines. Any discrepancies between the two reviewers were resolved by a consensus determination that served as “ground truth” for assessing the model’s outputs, with consensus adjudication as needed.

Primary endpoints included sensitivity, specificity, positive and negative predictive values, and accuracy.

Results: Strong performance with low latency

Among the 388 telestroke patients evaluated (median age of 69; 56.7% female), most (78.4%) had no contraindications documented in the EHR; 18.6% of patients had one contraindication and 3.1% had two contraindications. Only 4.6% of patients had no prior EHR notes to draw on.

Across 26 predefined contraindication categories, NeuroGlimpse produced 93 true positives, 91 false positives, 5 false negatives and 9,899 true negatives. Among the false positives, 83 (91.2%) were deemed clinically irrelevant, with 8 reflecting true contraindications that were missed by the ground-truth determination. No confabulations were produced.

These outputs translated to the following:

• Sensitivity of 94.9% (95% CI, 88.5-98.3)
• Specificity of 99.1% (95% CI, 98.9-99.3)
• Positive predictive value of 50.5% (95% CI, 43.1-58.0)
• Negative predictive value of 100% (95% CI, 99.9-100.0)
• Accuracy of 99.0% (95% CI, 98.8-99.2)

Notably, median model latency was only 14.3 seconds, and median cost per use was $0.07. “This supports feasibility for real-time clinical integration,” Dr. Chen notes.

A closer look at model performance

Further analyses showed that NeuroGlimpse performance did not differ by patient sex or race/ethnicity.

In contrast, sensitivity and positive predictive value did show significant variance across contraindication categories. The most problematic category was platelet count below 100,000/µL, which produced 28 false positives because the model flagged any mention of thrombocytopenia (notes only; platelet values were not accessible). “Layering structured lab-based logic on top of the LLM output could attenuate this source of false positives,” Dr. Chen observes.

The researchers note that the model’s modest positive predictive value (50.5%) is acceptable in stroke workflows where clinicians already manually verify contraindications. This is bolstered by the study’s finding that source attribution — i.e., identification of the exact note and author underlying each flagged contraindication — was perfect across all the model’s outputs, which allows clinicians to verify the model’s findings quickly.

“This tool’s value lies in rapidly sourcing potentially relevant items with great accuracy, reducing the chance that key historical details, such as prior surgeries or bleeds, remain buried during time-sensitive decision-making,” Dr. Chen explains.

From prototype to product

NeuroGlimpse has moved beyond the proof-of-concept phase and is now a fully functional application integrated into the secure cloud environment. The tool features a simple user interface where a clinician enters a Medical Record Number (MRN) and receives a structured safety report in approximately 30 seconds, further supporting its readiness for real-world clinical use.

Image content: This image is available to view online.

View image online (https://assets.clevelandclinic.org/transform/32f35dbd-decf-4abd-8a5c-01db8ba7cda4/two-screens-showing-ai-big-data-search)

Screenshots of the NeuroGlimpse app in action.

Randomized testing is next

Dr. Chen notes that no other health system or vendor currently offers an AI tool to screen in real time for contraindications to thrombolysis in stroke care. “NeuroGlimpse represents an important step forward with potential to reduce decision-making time while improving safety and easing clinician burnout,” he says.

He and his coinvestigators note the limitations of their study, including its retrospective, single-network design and its exclusive dependence on clinician-authored notes, with no medication lists, radiology reports or lab results among the inputs. Accordingly, they call for external validation across diverse EHR systems and prospective testing in live telestroke workflows.

Dr. Chen says a randomized clinical trial is being planned to compare stroke teams using NeuroGlimpse with those not using it. Endpoints will include impact on metrics such as door-to-needle time, accuracy of contraindication flagging and impact on patient outcomes such as hemorrhagic complications.

“While this tool is not a replacement for clinician judgment, its high sensitivity, perfect source attribution and near-instantaneous output position it as a promising adjunct for improving safety and efficiency in an easily scalable way,” Dr. Chen concludes.