Domain Shift Analyzer for Multi-Center MRI Datasets

Version	Summary	Created by	Modification	Content Size	Created at	Operation
1		Rafsanjany Kushol	--	2050	2023-10-21 02:12:24	\|
2	layout	Camila Xu	Meta information modification	2050	2023-10-23 03:39:12	\|

This entry is adapted from the peer-reviewed paper 10.3390/diagnostics13182947

Multi-center magnetic resonance imaging (MRI) datasets, incorporating data from multiple imaging centers or institutions, offer a unique opportunity to leverage diverse patient demographics, equipment, imaging platforms, and protocols.

MRI domain shift t-SNE UMAP quality control texture analysis

1. Introduction

The development of magnetic resonance imaging (MRI), which offers non-invasive examination of the human brain’s structure and composition, has revolutionized medical imaging. High-resolution images of brain anatomy are obtained during MRI scans, allowing medical professionals and researchers to examine different neuroanatomical aspects like cerebral structures, white matter, and grey matter. MRI datasets play a vital role in advancing medical research, not only in aiding in the understanding, diagnosing, and treating of numerous neurological disorders but also in training deep learning models. Radiologists and neurologists use MRI scans to identify abnormalities, such as tumours, lesions, atrophy, or other anatomical changes, that may be signs of disorders like Alzheimer’s, multiple sclerosis, epilepsy, and brain tumours ^[1]^[2]^[3]. In recent years, the availability of large-scale multi-center datasets has significantly advanced medical imaging research, which opens the avenues for developing powerful machine learning (ML) algorithms and data-driven methodologies.

Multi-center MRI datasets, incorporating data from multiple imaging centers or institutions, offer a unique opportunity to leverage diverse patient demographics, equipment, imaging platforms, and protocols. These datasets are valuable resources that can improve the generalizability and representativeness of research, producing more robust and reliable findings. Additionally, multi-site databases make it easier to investigate rare disorders, assess novel imaging techniques, and establish clinical benchmarks ^[4]. However, despite their potential advantages, multi-center MRI datasets introduce significant challenges due to a phenomenon known as “domain shift" ^[5]. Domain shift is the term used to describe the variances in data distributions across different centers resulting from variations in hardware, acquisition protocols, patient demographics, and environmental factors. These distributional shifts can severely affect the performance and generalizability of ML strategies and analysis techniques trained in one center and applied to another.

In a multi-center MRI dataset, domain shift primarily arises due to the heterogeneity among MRI scanners and imaging protocols across different centers. Some examples of domain shift parameters include imaging protocol (flip angle, acquisition orientation, slice thickness, and resolution) and scanner (manufacturer, model, magnetic field strength, and number of channels per coil). As a result, the appearance, contrast, intensity distributions, spatial resolutions, and noise levels of MR images differ qualitatively and quantitatively from site to site and study to study.

The problem of domain shift creates several challenges in analyzing and interpreting multi-center MRI datasets. Firstly, it impacts the performance and reliability of ML analysis pipelines as models trained in one center may fail to generalize effectively to the data from other centers ^[6]. This issue can hinder the adoption of automated tools for diagnosis, treatment planning, and disease monitoring as their efficacy relies on their capability to handle data from diverse sources. Secondly, domain shift can introduce biases and confounds in research studies that utilize multi-site MRI datasets. In clinical trials or population studies involving data from multiple centers, the variations originating from domain shift might distort statistical analysis, leading to erroneous conclusions and misleading findings. Thirdly, the inherent variability in scanner hardware and software across centers can introduce technical discrepancies, further complicating the comparison and fusion of data. These issues pose significant challenges for researchers and clinicians seeking to extract reliable and reproducible insights from multi-center MRI datasets.

Addressing the challenges of domain shift in multi-site MRI datasets requires advanced techniques and methodologies. Domain adaptation (DA) ^[7]^[8] and harmonization ^[9] methods aim to bridge the gap among different domains by aligning and normalizing the data from different centers. These approaches involve transforming the data distribution or features to minimize domain-specific variations, enabling more bias-free and reliable analysis across centers. Before developing DA or harmonization algorithms, it is essential to comprehensively understand the nature of domain shift in existing or target datasets. To foster reproducibility and knowledge sharing, the Python source code of the DSMRI has been made publicly available at https://github.com/rkushol/DSMRI. The applications and benefits of analyzing and dealing with domain shift in multi-center MRI datasets are numerous. Here are some crucial ones:

Improved generalizability: Domain shift analysis facilitates the development of ML models that can generalize across multiple centers. By identifying and mitigating the variations caused by domain shift, the methods become more robust and applicable to data from different imaging centers.

Reliable and reproducible research: It helps overcome biases and confounds triggered by the variations across different sites. By accounting for the domain-specific effects, research studies utilizing multi-center MRI datasets can yield more reliable and reproducible results.

Cross-center comparison and validation: It enables meaningful comparisons and validation of imaging biomarkers, algorithms, and protocols across various centers. Thus, researchers and clinicians can assess the performance and consistency of imaging techniques and analysis methods in diverse settings.

Enhanced collaborative research: Multi-center collaborations have become prevalent in medical imaging research. Analyzing domain shift encourages data sharing and collaboration among different centers by enabling a harmonized data analysis from various sources. It promotes data integration, pooling, and joint analysis, thereby facilitating large-scale studies and advancing scientific knowledge in the field.

Adaptation to new centers and populations: As new imaging centers are established or new patient cohorts are included in studies, domain shift analysis can guide the adaptation of existing models to these new domain configurations. This reduces the time and effort required to deploy analysis tools in new settings, allowing faster translation of research findings into clinical practice.

Quality control (QC) and outlier detection: Analyzing domain shift can serve as a QC measure for MRI datasets. It allows for identifying centers or specific scans that exhibit significant variations compared to others. Such insights can help in data validation as well as detect potential sources of errors or outliers.

2. Domain Shift in Multi-Center MRI Datasets

Prior studies have widely acknowledged and examined the presence of domain shift in multi-center MRI datasets. Researchers have consistently reported variations and challenges originating from domain shift, highlighting the need for robust analysis techniques.

A study by Dadar et al. ^[10] examined the impact of scanner manufacturers on a brain MRI dataset collected from multiple imaging centers. They reported significant differences in grey and white matter volume estimation among scanner manufacturers. These variations affected the reliability of automated brain segmentation algorithms, resulting in inconsistent outcomes from different centers. In another investigation by Tian et al. ^[11], domain shift effects were analyzed to reduce the site effects on grey matter volume maps using a travelling-subject MRI dataset obtained from various sites. They considered several underlying domain shift factors, such as scanner manufacturer, model, phase encoding direction, and channels per coil. Interestingly, they found that the scanner manufacturer is the most significant parameter causing domain shift, followed by the scanner model.

In another study, Lee et al. ^[12] explored the effects of changing MRI scanners on whole-brain volume change estimation at different time point visits. They identified that inter-vendor (e.g., Philips to Siemens) scanner changes led to more significant effects on percentage brain volume change than intra-vendor (e.g., GE Signa Excite to GE Signa HDx) scanner upgrades. Additionally, Glocker et al. ^[13] conducted an empirical study to investigate the impact of scanner effects when using ML on multi-site neuroimaging data. The authors discovered that, even after meticulous pre-processing using advanced neuroimaging tools, a classifier could identify the origin of the data (e.g., scanner) with very high accuracy. Moreover, Panman et al. ^[14] experimented with eight-channel and thirty-two-channel head coil configurations using structural, diffusion, and functional MR images while keeping all other parameters identical. They showed that the variations in the number of head coils could considerably impact the outcomes of analysis methods despite having the acquisition parameters synchronized.

The above studies collectively highlight the pervasive presence of domain shift in multi-center MRI datasets. The observed variations in image characteristics and acquisition parameters across centers pose considerable challenges for analysis and interpretation.

3. Quality Assessment Methods for MRI Data

MRIQC ^[15] is an open-source tool developed to automatically predict the quality of MRI data acquired from unseen sites as manual inspection is subjective and impractical for large-scale datasets. The tool extracts a set of spatial domain features to train an ML classifier and predict whether a scan should be accepted or excluded from the analysis. The authors validated that MRIQC accurately predicted image quality on an unseen dataset of multiple scanners and sites with approximately 76% accuracy. To address the errors and inconsistencies in brain image segmentation, Mindcontrol ^[16], a web-based application, was designed to allow a user to inspect brain segmentation data and manually correct errors visually. The user can view and interact with 3D brain images, including the ability to adjust opacity, slice orientation, and zoom level for data curation and QC.

Osadebey et al. ^[17] presented a quality metric scheme for structural MRI data in multi-site neuroimaging studies. The system evaluates image quality based on factors such as luminance contrast, texture analysis, and lightness and generates a total quality score. The authors demonstrated the system’s effectiveness by applying it to large-scale multi-center MRI data and concluded that it correlates well with human visual judgment. The quality evaluation using multi-directional filters for MRI (QEMDIM) ^[18] is a technique that is capable of detecting various distortions, including Gaussian noise and motion artifacts. The method utilizes mean-subtracted contrast-normalized (MSCN) coefficients to extract image statistics in the spatial domain. Their evaluation showed satisfactory accuracy in identifying low-quality images affected by different artifacts or noises compared to undistorted images.

Esteban et al. ^[19] proposed a crowdsourcing approach for collecting MRI quality metrics and expert quality annotations to train both humans and machines in assessing the quality of MRI data. They revealed that the ML algorithms trained on the crowdsourced data perform comparably to human raters in evaluating image quality. The strategy developed by Oszust et al. ^[20], NOMRIQA, applies high-boost filtering to intensify the high-frequency points, which allows the identification of various distortions. Their method utilizes the fast retina key-point descriptor and the support vector regression classifier to generate a quality score, which assists in detecting distorted T2-weighted images.

Bottani et al. ^[21] introduced an automated QC method for brain T1-weighted MRI in a clinical data warehouse. The technique involves extracting spatial domain features using a convolutional neural network (CNN) to predict scans that need to be excluded. They showed that their method could recognize images with potential quality issues, such as artifacts or motion-related distortions, and detect acquisitions for which gadolinium was injected. Lastly, an overview of various no-reference image quality assessment (NR-IQA) methods designed explicitly for MRI can be found here ^[22]. The authors discussed the challenges associated with evaluating MRI image quality due to the complex and dynamic nature of MRI data, including the influence of various acquisition parameters, image artifacts, and population-related factors.

These QC studies focus mainly on automatically detecting artifacts or poor-quality samples to reduce manual effort and decide whether a particular scan should be accepted or excluded from the analysis. These studies neither emphasize quantifying the degree of domain shift from these QC features nor analyze which features are correlated to domain shift.

4. Existing Domain Shift Analysis Tools

The tools introduced by Sadri et al., MRQy ^[23], and Guan et al., DomainATM ^[24], can be considered the two closest studies related to the proposed framework. MRQy is mainly designed for the QC of MRI data by which manual effort to filter poor-quality data can be automated for clinical and research studies. It uses different spatial-domain-image-quality-related metrics to address different types of noise, shading, inhomogeneity, and motion artifacts.

DomainATM offers visualization of data distribution as well as measures the domain shift distance for the original or synthetic data. Then, they implemented some classical DA methods to show the effectiveness of these methods in reducing the domain shift. However, this tool cannot take raw neuroimaging data, such as NIfTI files, directly as input. To analyze real-world data with DomainATM, the user must process the data with Anatomical Automatic Labeling (AAL) atlas and then extract the grey matter volumes for each region of interest (ROI), making the tool inconvenient for many applications. Most importantly, these grey matter features are not meaningful regarding the domain shift measurement, which is reflected in the experimental section. The proposed framework DSMRI is compared with MRQy and DomainATM to demonstrate the strength of the proposed features in analyzing the domain shift in a multi-center MRI dataset.

References

Kushol, R.; Masoumzadeh, A.; Huo, D.; Kalra, S.; Yang, Y.H. Addformer: Alzheimer’s disease detection from structural Mri using fusion transformer. In Proceedings of the IEEE 19th International Symposium on Biomedical Imaging, Kolkata, India, 28–31 March 2022; pp. 1–5.
El-Latif, A.A.A.; Chelloug, S.A.; Alabdulhafith, M.; Hammad, M. Accurate Detection of Alzheimer’s Disease Using Lightweight Deep Learning Model on MRI Data. Diagnostics 2023, 13, 1216.
Kim, S.; Lee, E.K.; Song, C.J.; Sohn, E. Iron Rim Lesions as a Specific and Prognostic Biomarker of Multiple Sclerosis: 3T-Based Susceptibility-Weighted Imaging. Diagnostics 2023, 13, 1866.
Kushol, R.; Luk, C.C.; Dey, A.; Benatar, M.; Briemberg, H.; Dionne, A.; Dupré, N.; Frayne, R.; Genge, A.; Gibson, S.; et al. SF2Former: Amyotrophic lateral sclerosis identification from multi-center MRI data using spatial and frequency fusion transformer. Comput. Med Imaging Graph. 2023, 108, 102279.
Quinonero-Candela, J.; Sugiyama, M.; Schwaighofer, A.; Lawrence, N.D. Dataset Shift in Machine Learning; MIT Press: Cambridge, MA, USA, 2008.
Botvinik-Nezer, R.; Wager, T.D. Reproducibility in neuroimaging analysis: Challenges and solutions. Biol. Psychiatry: Cogn. Neurosci. Neuroimaging 2023, 8, 780–788.
Guan, H.; Liu, M. Domain adaptation for medical image analysis: A survey. IEEE Trans. Biomed. Eng. 2021, 69, 1173–1185.
Kushol, R.; Frayne, R.; Graham, S.J.; Wilman, A.H.; Kalra, S.; Yang, Y.H. Domain adaptation of MRI scanners as an alternative to MRI harmonization. In Proceedings of the 5th MICCAI Workshop on Domain Adaptation and Representation Transfer, Vancouver, BC, Canada, 12 October 2023.
Gebre, R.K.; Senjem, M.L.; Raghavan, S.; Schwarz, C.G.; Gunter, J.L.; Hofrenning, E.I.; Reid, R.I.; Kantarci, K.; Graff-Radford, J.; Knopman, D.S.; et al. Cross–scanner harmonization methods for structural MRI may need further work: A comparison study. NeuroImage 2023, 269, 119912.
Dadar, M.; Duchesne, S.; CCNA Group and the CIMA-Q Group. Reliability assessment of tissue classification algorithms for multi-center and multi-scanner data. NeuroImage 2020, 217, 116928.
Tian, D.; Zeng, Z.; Sun, X.; Tong, Q.; Li, H.; He, H.; Gao, J.H.; He, Y.; Xia, M. A deep learning-based multisite neuroimage harmonization framework established with a traveling-subject dataset. NeuroImage 2022, 257, 119297.
Lee, H.; Nakamura, K.; Narayanan, S.; Brown, R.A.; Arnold, D.L.; Alzheimer’s Disease Neuroimaging Initiative. Estimating and accounting for the effect of MRI scanner changes on longitudinal whole-brain volume change measurements. Neuroimage 2019, 184, 555–565.
Glocker, B.; Robinson, R.; Castro, D.C.; Dou, Q.; Konukoglu, E. Machine learning with multi-site imaging data: An empirical study on the impact of scanner effects. arXiv 2019, arXiv:1910.04597.
Panman, J.L.; To, Y.Y.; van der Ende, E.L.; Poos, J.M.; Jiskoot, L.C.; Meeter, L.H.; Dopper, E.G.; Bouts, M.J.; van Osch, M.J.; Rombouts, S.A.; et al. Bias introduced by multiple head coils in MRI research: An 8 channel and 32 channel coil comparison. Front. Neurosci. 2019, 13, 729.
Esteban, O.; Birman, D.; Schaer, M.; Koyejo, O.O.; Poldrack, R.A.; Gorgolewski, K.J. MRIQC: Advancing the automatic prediction of image quality in MRI from unseen sites. PLoS ONE 2017, 12, e0184661.
Keshavan, A.; Datta, E.; McDonough, I.M.; Madan, C.R.; Jordan, K.; Henry, R.G. Mindcontrol: A web application for brain segmentation quality control. NeuroImage 2018, 170, 365–372.
Osadebey, M.E.; Pedersen, M.; Arnold, D.L.; Wendel-Mitoraj, K.E.; Alzheimer’s Disease Neuroimaging Initiative, f.t. Standardized quality metric system for structural brain magnetic resonance images in multi-center neuroimaging study. BMC Med. Imaging 2018, 18, 31.
Jang, J.; Bang, K.; Jang, H.; Hwang, D.; Initiative, A.D.N. Quality evaluation of no-reference MR images using multidirectional filters and image statistics. Magn. Reson. Med. 2018, 80, 914–924.
Esteban, O.; Blair, R.W.; Nielson, D.M.; Varada, J.C.; Marrett, S.; Thomas, A.G.; Poldrack, R.A.; Gorgolewski, K.J. Crowdsourced MRI quality metrics and expert quality annotations for training of humans and machines. Sci. Data 2019, 6, 30.
Oszust, M.; Piórkowski, A.; Obuchowicz, R. No-reference image quality assessment of magnetic resonance images with high-boost filtering and local features. Magn. Reson. Med. 2020, 84, 1648–1660.
Bottani, S.; Burgos, N.; Maire, A.; Wild, A.; Ströer, S.; Dormont, D.; Colliot, O.; APPRIMAGE Study Group. Automatic quality control of brain T1-weighted magnetic resonance images for a clinical data warehouse. Med. Image Anal. 2022, 75, 102219.
Stępień, I.; Oszust, M. A Brief Survey on No-Reference Image Quality Assessment Methods for Magnetic Resonance Images. J. Imaging 2022, 8, 160.
Sadri, A.R.; Janowczyk, A.; Zhou, R.; Verma, R.; Beig, N.; Antunes, J.; Madabhushi, A.; Tiwari, P.; Viswanath, S.E. MRQy—An open-source tool for quality control of MR imaging data. Med Phys. 2020, 47, 6029–6038.
Guan, H.; Liu, M. DomainATM: Domain Adaptation Toolbox for Medical Data Analysis. NeuroImage 2023, 268, 119863.

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.

Upload a video for this entry

Information

Subjects: Computer Science, Artificial Intelligence

Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :

Rafsanjany Kushol

Alan H. Wilman

Sanjay Kalra

Yee-Hong Yang

View Times: 159

Update Date: 24 Oct 2023

Table of Contents

Video Upload Options

Confirm