AI System Combines Images With Written Impressions to Accurately Read Cardiac MRI

A new artificial intelligence (AI) framework named CMR-CLIP (Cardiovascular Magnetic Resonance-Contrastive Language Image Pretraining) combines visual data from CMR videos with text reports to learn to rapidly perform image analysis and diagnostics of CMR scans.

Tested in real-world clinical evaluations, the vision-language model achieved high accuracy in diagnosing multiple heart conditions. The tool, developed by researchers from Carnegie Mellon University in collaboration with Cleveland Clinic’s Cardiovascular Innovation Research Center, was described in a recent paper published in Nature Communications, which also reported its performance on real-world clinical tasks.

“CMR-CLIP not only performed well on diseases it had trained on, but also demonstrated ‘zero shot’ capabilities, meaning that it could recognize other conditions as well,” says David Chen, PhD, of Cleveland Clinic’s Cardiovascular Innovation Research Center. “The system has the potential to support clinicians through automated screening and interpretation support, particularly in settings where expert readers are limited.” He served as co-principal investigator on the project along with Ding Zhao, PhD, of Carnegie Mellon University’s Department of Mechanical Engineering.

CMR – an important but complex test

CMR is considered the gold standard for evaluating cardiac anatomy, physiology and microstructure. But interpretation of these information-rich scans is complicated and time-intensive, typically taking a reader 40 minutes or more to assess a single patient. Interpretation also requires a high level of expertise available only at major medical centers.

Currently CMR imaging is read and reported by either radiologists or cardiologists with up to two years of advanced training. However, significant variability exists in how imaging findings are translated into clinically actionable information. Differences in emphasis between detailed imaging characterization and clinical integration may affect the clarity and consistency of reported findings for referring physicians and subsequent clinical decision-making.

“There is currently a great deal of variability in CMR reporting,” says cardiologist Deborah Kwon, MD, Director of Cardiac MRI at Cleveland Clinic and a study co-author who served as its clinical lead. “CMR-CLIP could be immensely important in automating and standardizing reports, providing greater clarity to clinicians for determining next steps for patient care.”

CMR-CLIP development and performance

The framework was trained on left ventricular diseases, among the most common conditions evaluated with CMR. Unlike typical AI programs that use manual labels to train their systems, CMR-CLIP aligned image sequences with their corresponding natural language clinical summaries, providing direct learning from real-world interpretations of scans.

Drawing from more than 13,000 CMR studies performed at Cleveland Clinic between 2008 and 2022, the training dataset consisted of more than a million images and hundreds of thousands of motion sequences. The text component focused on the impression section of reports, summarizing key findings, differential diagnosis and recommendations for the plan of care.

The model was subsequently tested on two datasets separate from the one on which it was developed: one collected at the University Hospital of Dijon, France, and the other from Cleveland Clinic Florida care sites. The CMRs were conducted on different scanners from the training data and interpreted by readers from those institutions.

Using expert readers as the standard, CMR-CLIP achieved the following levels of accuracy:

• 88.5% for nonischemic cardiomyopathy
• 88.0% for ischemic cardiomyopathy
• 96.2% for cardiac amyloidosis
• 98.6% for hypertrophic cardiomyopathy

When tested against two other available AI CLIP systems, CMR-CLIP outperformed them in identifying common pathologies such as myocardial fibrosis and left ventricular hypertrophy by 32% or more. In addition, CMR-CLIP using a single instance of data (one-shot) achieved similar results as the other programs using 32 instances of data (32-shot).

“Developing AI tools for CMR is especially challenging because fewer studies are available compared with other cardiac imaging modalities, and there are not many qualified readers,” Dr. Chen says. “We were pleased to see not only CMR-CLIP’s high level of performance but also that it requires less data to achieve the same level of classification performance compared with other systems.”

A training and reference library

In addition to increasing efficiency of diagnostics and standardizing reporting, CMR-CLIP could serve as an important training and educational tool for residents, fellows and referring cardiologists.

Dr. Kwon notes that specialized training often involves exposure to many cases of common diagnoses and fewer rare ones. “Having a large, easily searched library of images with impressions and diagnoses can help close the gaps of uneven experience,” she says.

In addition, a reporting radiologist or cardiologist who is confronted with a rare diagnosis can refer to images of similar diagnoses in the library of past reported cases to increase confidence that a diagnosis is correct.

Moving beyond development

CMR-CLIP was developed in a research environment and is not ready for clinical implementation yet, according to the researchers.

Trained on a 15-year period of data ending in 2023, it will require continuous learning capabilities to keep it relevant, as well as a more varied mix of diagnoses and patient populations to widen its scope and strengthen performance.

Further testing also needs to be done on additional datasets from different locations to ensure generalizability. In addition, how the framework fits into hospital workflow will need to be evaluated.

“Because CMR interpretation is so highly specialized and time-intensive, reader assistant tools like CMR-CLIP are critical,” Dr. Chen concludes. “They have the potential to help expand patient access to this powerful diagnostic technology beyond major medical centers to settings where expert readers are limited.”

The CMR-CLIP codebase is publicly available at github.com/Makiya11/CMRCLIP.