Present and Future Innovations in AI and Cardiac MRI

Self-driving Scanning

Despite the rapid growth of clinical cardiac MRI over the past 2 decades, its availability remains largely limited to major academic centers. One of the challenges in making this imaging available in smaller health care systems is complicated image prescription. There have been efforts to develop automated or semiautomated image prescription in cardiac MRI (13). However, these approaches need to be more clinically robust. For instance, DL-based prescriptions eliminate manual contouring and do not require the collection of extra images beyond what is already included in the clinical protocol (14). Therefore, a DL framework can automate the entire scanning procedure, with automated section prescription and parameter optimization, shifting the role of the MRI technologists to one of quality control (14). A DL model can calculate imaging parameters automatically to automate the scan further (15) or annotate anatomic landmarks (16). AI-assisted scanning could also improve throughput, yielding shorter scan times, higher consistency, and enhanced image quality (17). Nevertheless, there will always be a need for active supervision and correction strategists, as AI-assisted scanning will be prone to errors. With advances in AI, growth in cardiac MRI, and the economic pressure for improved throughput, scanning will undoubtedly be nearly fully automated in the future. However, how technologists will adapt to such automation for routine clinical scanning remains to be determined.

Image Reconstruction

Cardiac MRI protocols are long, with a 45–60-minute examination time requiring repeated breath holds. Therefore, reducing scan time and breath holding is desirable. AI-based image reconstruction has been proposed to improve imaging efficiency (18). It can be generally classified into four strategies: (a) image reconstruction, (b) image enhancement, (c) denoising, and (d) quantitative myocardial tissue characterization.

Image reconstruction.—In MRI, k-space data are the raw data obtained from the MRI signal, with “fully sampled” data adhering to the Nyquist theorem for complete image information. “Undersampling” occurs when the data points collected are fewer than what is prescribed with this theorem. Undersampling reduces the signal-to-noise ratio (SNR) and results in “aliasing artifacts,” where the final images display distortions. Using AI-accelerated cardiac MRI techniques can remove such artifacts. These techniques are broadly divided into k-space-agnostic and -aware approaches (Fig 4). In k-space-agnostic models, an image-to-image network, such as a U-Net CNN (19–23), is trained to convert images reconstructed from undersampled k-space data to match images created from fully sampled k-space. Only reconstructed images are used during training for k-space-agnostic models, but in some contexts generating realistic input-output image pairs could require k-space data and access to reconstruction pipelines. K-space-aware models (24–27) include prior knowledge of the acquisition process and k-space consistency terms. For example, traditional regularization functions, which prevent overfitting or enforce prior knowledge, are learned using CNNs. In variational networks (28), iterative steps of a reconstruction algorithm are unrolled and replaced with CNNs, although unrolling is not always used (29). K-space-aware approaches offer some guarantees about the reconstructed images because of the mathematical formulation and the enforced k-space data consistencies. However, a direct head-to-head comparison of different AI-accelerated cardiac MRI techniques is challenging, and it is unlikely that one method will always be superior to the other. Many factors should be considered, including the training set size, k-space sampling trajectory (Fig 5), availability of a fully sampled data set, practical deployment, and reconstruction latency.

Most published AI literature in rapid cardiac MRI focuses on cine imaging using k-space-agnostic (19,22,23) or k-space-aware approaches (26,27,30). U-Net models (Fig 6A) trained using relatively modest training sets with typical sizes of 250–500 healthy subjects and patients have been widely used (19,23). Images reconstructed with compressed sensing have been used as a reference standard when only undersampled data are available (27). Although this approach can reduce reconstruction time, it limits data quality to that of current algorithms. The k-space-aware models have been trained on smaller cohorts, typically fewer than 100 healthy subjects and patients (26,27). The motion-compensation step may also be recast as a DL approach and combined with unrolled reconstruction networks (31). Numerous recent studies have applied DL to accelerate cardiac MRI. These studies include U-Net models that have also been proposed for rapid MRI angiography (32). AI-accelerated coronary MRI has shown good quality and good diagnostic performance (33), while AI-accelerated two-dimensional or four-dimensional phase-contrast MRI using U-Net (Fig 6B) or unrolled networks (24,25,34) is also available. Despite the need to accelerate myocardial perfusion and late gadolinium enhancement (LGE) image acquisition, few studies have focused on AI acceleration of these sequences (20,21,35,36).

Image enhancement.—In image resolution enhancement or restoration (also called superresolution techniques), AI models are trained to “recover” the spatial resolution of images acquired with a lower resolution (37,38). Acquiring images with lower resolution reduces scan time since fewer data are acquired. In that case, the acquired k-space data become truncated or cropped relative to higher-resolution images. Synthetic lower-resolution images for training can be generated by retrospectively cropping the raw k-space data before applying the inverse fast Fourier transform. In generative adversarial network–based models for resolution enhancement, the discriminator training can be modified. Specifically, the discriminator estimates the probability that an image based on complete k-space data appears “more realistic” than the resolution-enhanced image produced by the generator. This generative adversarial network approach aims to achieve more than just training the discriminator to differentiate between the two images. Instead, it aims to produce generated images that mimic those from the target data set and possess sharper details and edges, bringing them closer in quality to true high-resolution images. In addition to the adversarial loss function, there is a perceptual loss function for resolution enhancement that compares the features extracted from true high-resolution and resolution-enhanced images. This enables the production of resolution-enhanced images that are perceptually similar to those acquired with higher resolutions. Resolution enhancement models can reduce scan time in various sequences, including cine (37,39,40), coronary (41), and cardiac diffusion (42).

The advantages of resolution enhancement techniques include ease of deployment, flexibility for different sequences, and there being no need for sequence modifications. For instance, a recently proposed inline resolution enhancement technique pretrained with cine data (40) (Fig 7A) can be readily used without any modifications to reconstruct real-time tagging images (Fig 7B) or to reduce the scan time of LGE imaging (Fig 8). However, there are concerns that such models can hallucinate and create realistic-looking images that are not true. A certain threshold of k-space data and spatial resolution is likely essential to guarantee the capture of crucial anatomic details in low-resolution images, where the function of the AI model is primarily to refine the sharpness of existing structures. Thus, rigorous evaluation is necessary before the clinical adoption of these models.

Image denoising.—Image denoising in MRI can improve the SNR and is frequently accomplished using image filtering during image reconstruction. However, traditional denoising methods could also result in image blurring. In recent years, there have been substantial improvements in natural image denoising using AI (43), which could be adopted in MRI (44,45). AI denoising could achieve higher denoising capability with reduced image blurring (46). In supervised learning image denoising, a pair of noisy and noise-free images are used to train a DL model, such as a CNN (47) or a generative adversarial network (45). A trained model can generate the denoised image from acquired images. However, in vivo acquisition of noise-free MRI scans is not feasible; therefore, simulated noise is often added to generate synthetic paired images for training. The denoising technique has several applications in cardiac MRI. It can enhance image SNR when images have intrinsically low SNR, such as when they are acquired with a low magnetic field or in low-SNR sequences (eg, LGE). It can also be combined with imaging acceleration to recover the SNR intrinsically lost because of the imaging acceleration. Image denoising could also improve the precision of myocardial tissue characterization techniques. Finally, AI-based denoising algorithms can also reduce the number of signal averages. Signal averaging is a technique commonly used in cardiac diffusion imaging that improves the SNR by running multiple acquisitions (48).

Despite the progress, there are many practical challenges in AI-based denoising of cardiac MRI scans. The current pipeline for data collection using multicoil arrays and nonlinear image reconstruction impacts the noise characteristics of images. Removing spatially varying noise is more complicated than denoising natural images. Furthermore, each cardiac MRI examination generates thousands of images; therefore, the speed of image denoising will be an important factor in the clinical feasibility of AI-based denoising in cardiac MRI. Further developments and rigorous evaluation are still needed to demonstrate the clinical utility of AI-based denoising of cardiac MRI scans.

Quantitative myocardial tissue characterization.—In myocardial tissue characterization sequences (eg, T1 mapping), images with different weighting across multiple heartbeats are collected to estimate the tissue parameters (eg, native T1) using pixel-wise numerical fitting. Therefore, reducing the scan time of each image does not reduce the total scan time, which is governed by the number of collected images and the recovery time needed between images. AI-based models can reduce scan time by estimating quantitative values (eg, T1) from fewer images collected with different contrasts (49,50). A model can then be trained to estimate the tissue parameters using a simple neural network to replace numerical fitting based on the Bloch equation (49). The use of CNNs can exploit spatial information in images to improve the precision of tissue parameters (51). Further refinements, including training using simulated data or scanner information, have improved the robustness of such models (52,53). Cardiac MR fingerprinting–based myocardial tissue characterization, in which images acquired with different weighting are used to estimate different tissue properties simultaneously, can also benefit from using AI to speed up dictionary generation, matching, and image reconstruction (54).

Despite considerable growth in research in AI-based accelerated cardiac MRI techniques, many challenges remain. Irrespective of which architecture or imaging acceleration strategy, the quality of the training data and how they are synthesized will have an immense impact on the performance of imaging acceleration. Researchers must rely on carefully synthesized training data from original single-coil raw k-space data, mimicking rapid imaging. In addition, loss functions and metrics used to quantitatively evaluate the performance of image reconstruction methods, such as mean square error or structural similarity index, do not always reflect diagnostic image quality. AI reconstruction methods that do not consider these issues will provide overly optimistic results that will have little practical value. Therefore, there is always a need for rigorous subjective assessment by clinical imaging readers to evaluate the performance of AI-based cardiac MRI reconstruction algorithms tested in rigorous prospectively collected imaging experiments.

Image Analysis

AI can reduce the analysis burden on imagers at multiple stages (Fig 9). For instance, image segmentation remains one of the most time-consuming and least desired tasks in cardiac MRI. Research has steadily increased in automated AI-based cardiac MRI segmentation tools (55), which has resulted in improved performance compared with traditional non-DL approaches (56). U-Net has been a widely used DL architecture for cardiac MRI segmentation (Fig 10). A model based on U-Net achieved the highest performance in a competition of automated segmentation of short-axis cine images (57). In a similar competition, all proposed models were based on U-Net architectures (58). U-Net models also achieved the highest performance in a recent challenge in segmentation of the right ventricle across multiple anatomic views (59). Indeed, the majority of the models in the challenge were based on a self-configuring method that selected the optimal U-Net configuration based on the specific biomedical image segmentation task (60).

The Sørensen-Dice coefficient is the most used metric in validating medical segmentations (61). However, the accuracy of relevant clinical parameters, such as structural and functional parameters measured on cine images, should also be considered. For instance, automated AI cine segmentation software can provide functional and anatomic quantitative parameters with good correlation with parameters acquired using manual methods. However, in many cases, there can be substantial errors, and the ability of automated measurements to classify functions for treatment decision-making is limited (62). Therefore, the value of AI segmentation algorithms for diagnosis or prognostication should also be considered and arguably is more clinically relevant than segmentation metrics. Indeed, while AI-based segmentation has been relatively successful, there is often a need for manual adjustments and quality control (63), and such human adjustments can improve AI model segmentation performance and robustness (64).

Nearly all vendors provide AI-based solutions for calculating functional and volumetric parameters from cine images. A fully automated AI model outperformed human readers in speed and precision in measuring left ventricular structure and function (65). Automating image analysis for cardiac MRI sequences beyond cine has been more challenging due to lower data availability and lower contrast-to-noise ratio and SNR. Recent studies have investigated automating the processing of four-dimensional flow data (66–68), a very time-consuming process hindering the clinical application of four-dimensional flow MRI. Automated inline AI segmentation of myocardial perfusion images for pixel-wise quantification has been shown to provide measures of myocardial blood flow comparable to manually obtained measures (69). Automated analysis of T1 mapping also provides T1 measures with good correlation with manual analysis (70). Ensemble learning techniques for on-the-fly quality control of T1 mapping can minimize the need for manual adjustments (71). Further, pretrained AI models for T1 mapping along with transfer learning techniques can be leveraged for T2 quantification (72). LGE imaging is a cornerstone of cardiac MRI examinations and is part of most clinical cardiac MRI protocols. However, LGE scar quantification remains challenging, and evaluation remains mostly subjective, limited to assessing presence and location. Automating LGE quantification using DL techniques such as two-dimensional or three-dimensional U-Net is promising; however, such models are currently not commercially available (73–76).

There are many remaining challenges in robust cardiac MRI segmentation, particularly access to large training data sets from diverse patient populations, imaging equipment, and sequences. Integration of automated AI analysis algorithms into commercial analysis software remains challenging, and inline implementation could be limited to academic research centers. In addition, adjustments and quality control of inline segmentation tools are challenging; there is also no standardized format for contours in MRI, resulting in the incompatibility of contours among different analysis systems. These practical but important issues hinder the potential use of AI-based tools in cardiac MRI analysis. Due to the increasing use of cardiac MRI for clinical purposes, automated analysis will be the most desired and impactful application of AI in cardiac MRI in the immediate future.

Radiomic Signatures

With radiomic signatures (78), one extracts and analyzes many quantitative predefined image features (eg, myocardial texture) with high throughput. The hypothesis is that mining many quantitative features yields useful features with diagnostic or prognostic value not commonly obtained with conventional image analysis. A comprehensive review of radiomic analysis methods is available (79). Texture analysis of nonenhanced cine MRI enables the diagnosis of subacute and chronic myocardial infarction with high accuracy (80). Radiomic analysis of native T1 images can discriminate between hypertrophic and hypertensive cardiomyopathies (81). Radiomic features of LGE images distinguish between myocardial infarction and myocarditis (82). Radiomic features of LGE and cine images, relevant clinical information, and conventional MRI parameters can help predict major adverse cardiac events in patients with ST-segment elevation myocardial infarction (83). Texture analysis of myocardial T1 and T2 maps delivers quantitative imaging parameters for diagnosing acute or chronic heart failure–like myocarditis (84). Radiomic analysis of extracellular volume could help discriminate reversible from irreversible myocardial injury (85). Radiomic analysis of noncontrast MRI scans can also provide hints about the presence of scarring (80,86–88). Radiomic analysis of LGE images also provides prognostic value beyond traditional scar burden assessment (89).

While there is growing interest in applying radiomics in cardiac MRI, the field is in its infancy. There are many challenges regarding appropriate methodology, reproducibility, sensitivity, and generalizability that need addressing before using radiomic analysis in a clinical setting. Radiomic features are calculated using predefined mathematical operations, making the extraction step reproducible. However, the extracted features will vary based on the input images. For example, shape features depend highly on the region of interest where the radiomic features are calculated; therefore, if there is variability in the drawing of the region of interest, this will impact radiomic features. Similarly, imaging filters applied to original images impact the radiomic features. Radiomic features extracted from cardiac MRI scans acquired with different contrast weightings (eg, T1 or T2 weighting) have different sensitivity and reproducibility (90). Furthermore, sequence parameters and processing impact radiomic feature values (91,92).

However, these issues are not fundamentally different from any quantification in cardiac MRI. For example, it is well established that myocardial T1 imaging is sensitive to field strength and sequence. Similarly, radiomic features extracted from different sequences, with different field strengths, or on equipment from different vendors will differ. These limitations should be considered when interpreting radiomic studies. Additional efforts are warranted to develop standardized approaches to implementing radiomic analysis and promoting accurate reporting of methodology and more insightful interpretation.

DL Feature Extraction

Unlike radiomics-based analysis, DL models automatically extract features from input images such that the prediction of an outcome is successfully achieved. DL models for extracting deep imaging features are mainly based on CNNs. The hypothesis is that deep features, extracted after applying a large number of convolutional layers, have incremental diagnostic or prognostic value beyond established clinical and imaging parameters. DL models applied to medical imaging data have been widely used in other diseases and imaging modalities (93). However, fewer studies have explored their potential in cardiac MRI. A DL analysis of cardiac cine motion can predict survival in patients with pulmonary arterial hypertension (94). A DL analysis of LGE images from patients with ischemic heart disease can also predict survival (95). In both studies (94,95), the input and output images of a U-Net CNN were set to be identical. In such approaches, the imaging features are obtained from the halfway “bottleneck” features between the encoder and the decoder, rather than from the output of the decoder. Transfer learning has also been used for diagnosis (96). In one study, a semisupervised model was trained on the auxiliary task of classifying image type, and transfer learning was then applied to fine-tune the model for diagnosing cardiac amyloidosis (96).

Comparing Radiomic and DL Models and Their Real-World Potential

While both radiomic and DL models can extract features from images, to our knowledge a direct head-to-head comparison of these two methods in cardiac MRI has not been conducted. The jury is still out on whether there has been a rigorous demonstration of the potential of radiomic or DL models to extract novel imaging features beyond current metrics. In addition, it is necessary to develop approaches to understand what features contribute to model performance. While there are existing techniques for explaining model behavior (97), such as saliency maps, their practical utility and trustworthiness have recently been questioned (98). One could argue that cardiologists sometimes use medications for which the underlying mechanism of action is not completely understood and that researchers should treat AI models similarly. But AI researchers need to acknowledge that the efficacy of these medications has been investigated in rigorous prospective clinical trials. AI models should likewise be tested in rigorous multicenter prospective clinical trials to demonstrate accuracy and efficacy and thereby gain trust.

Statistical Modeling versus ML to Describe Data

After extracting all relevant clinical and imaging markers from patient data, the next step is to use these rich data sets to answer the relevant clinical question using statistical modeling or ML, each with its strengths and weaknesses. There are ongoing misconceptions and controversies in distinguishing statistical modeling from ML. In statistical modeling, the aim is to derive a mathematical model to describe the data. Statistical models are interpretable and can be readily used to investigate the effect of imaging or clinical variables and their uncertainty on outcomes of interest. With statistical modeling, estimating the effects of predictors on the outcome requires that the number of patients or observations be larger than the number of predictors. In ML, the machine learns the association from the presented data. The goal in ML is to learn the prediction by studying the data pattern, not necessarily to learn the effect of variables on the outcome; ML does not provide information about causality. An ML model typically results in a “black box” system and requires a much larger data set for the training process (99), which focuses on optimizing objective criteria rather than being based on statistical assumptions. ML models can handle high-dimensional data, as applies to cardiac disease involving multimodality images, clinical risk markers, serum biomarkers, genetic data, and omics data. However, one should not always assume that ML models outperform statistical modeling in cardiac care and should consider their strengths and limitations with caution. When dealing with numeric data such as cardiac MRI measurements or clinical variables, there are many advantages to using conventional statistical modeling, including interpretability and generalizability. In recent years, ongoing efforts have focused on improving the interpretability of ML models to enhance their clinical adoption (97,100). These “explainable” models could provide confidence in ML modeling but will not completely overcome its limitations (100).