Imaging techniques have revolutionized neuroscience, allowing researchers and physicians to glimpse into the brain’s inner workings without needing to do invasive procedures on a patient or a study subject. Visualizing the brain in clinical settings have been pivotal in clinical settings in many ways, from catching brain tumors to identifying the effects of a stroke. However, with the surging amount of brain scans flooding the hospital environment, the radiologist’s workload has nearly quadrupled in 15 years, potentially compromising the quality and safety of radiological care [1].

Researchers have come up with many ways to combat this issue. One of these is to create automated systems that can help with the analysis and diagnosis using clinical images. However, these systems suffer from a degree of scarcity of brain scans matched to conditions and diseases due to hospitals needing to protect patient privacy as well as neuroimaging techniques being quite costly to run. Growing advances and increased interest in artificial intelligence (AI) have allowed for the generation of synthetic data, offering a promising solution to data scarcity issues in the automated analysis of clinical neural images through general adversarial networks (GANs).

What are GANs, and Why Use Them?

Generative Adversarial Networks (GANs) are an application of deep learning, a kind of artificial intelligence (AI) system that mimics the human learning process. They are computational systems that can generate images from simulated data. GANs are split into two components: the “generator” and the “discriminator” [2]. The generator is designed to create fake data from existing data, such as medical images and scans. These synthetic pieces of data are then relayed to the discriminator, which attempts to distinguish the simulated data from the actual data. If the discriminator is successful, the generator takes this feedback and continues to improve its generation in a process known as training. After sufficient training, the result is a model, which is a system capable of making accurate decisions and classifications when given neuroimaging data. Imagine this system as a relationship between a forgery artist and an art critic. The generator is the forgery artist, who continually tries to improve their craft by imitating paintings done by real artists in order to successfully fool the art critic into buying the forged piece. The discriminator is the art critic whose goal is to buy an authentic piece of art. At first, the forgery artist’s work will be very obvious, and the art critic will be able to point out the areas of the artwork that appear forged. The forgery artist will take this feedback and continue to practice until the critic isn’t able to distinguish the forged piece from the real piece. 

Deep learning methods are not novel to the field of neuroscience. Interest in using deep learning to understand the brain has been around since the late 1900s, and has been pioneered by neuroscientists such as Salk Institute for Biological Studies professor Terrence Sejnowski and University of Toronto professor Geoffrey Hinton [3, 4], and has skyrocketed in attention during the 2010s [5]. Since then, methods for deep learning have evolved, and newer methods such as GANs offer several advantages compared to other classical computational models. For example, random forest algorithms are one of the most prevalent classification systems in machine learning. Random forest algorithms are models that assess multiple distinct features from data, and combine all these individual predictions to give an overall consensus in a label [6]. Supervised deep learning is another common family of algorithms. However, both of these models are trained by feeding lots of paired data to the model in order for it to learn. Labeled data refers to linked data points used to study the relationship between two different variables, and requires large sets of data and patient information that aren’t always readily available, especially in clinical settings [7]. GANs, on the other hand, do not need to use as much starting labeled data, and can therefore work around such limitations.

Synthetic data models allow scientists to work around data scarcity issues for these algorithms, offering an alternative that doesn’t breach valued patient privacy by generating synthetic data. Using GANs saves patients time and reduces operating costs, allowing clinics to funnel their money elsewhere [8]. From diagnosing neurological conditions to providing personalized care, GANs are being researched, built, and applied in many areas of clinical neuroscience.

Using GANs in Clinical Diagnosis

Currently, one of the most studied applications of GANs in diagnosing neurological conditions is for Alzheimer’s Disease (AD). AD is a neurodegenerative condition that is heavily studied within neuroscience due to its increasing prevalence, affecting about 6.9 million Americans aged 65 years and older [9]. Alzheimer’s disease is typically identified and diagnosed through checking for clusters of incorrectly structured proteins. However, this method primarily only works with early onset Alzheimer’s, and may not catch more serious progressions. To cast a wider net on AD diagnosis, a group of researchers based in China designed a GAN model to enhance the identification of brains with AD. Researchers trained their model using resting state functional magnetic resonance imaging (rs-fMRI) scans of AD patients [10]. fMRI is a method of brain imaging commonly used in clinical settings which measures brain activity based on levels of oxygen in blood. The images it generates are maps of the human brain with varying levels of activity. RS-fMRIs, on the other hand, are a specific kind of fMRI used to identify brain activity at rest [11]. After creating the model, the researchers tested their tool using fMRI images from a public image repository from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Their GAN-based model performed much better in classifying rs-fMRI images that had AD characteristics than a non-GAN-based model, being able to identify MRI images of AD at any stage with an accuracy rate of 80% [10]. Researchers from the Kolachama group from the University of Boston were able to design a GAN model that was able to classify MRI scans of patients with AD-induced neurodegeneration and ones without. To do so, this group virtually “upgraded” MRI images captured using their GAN, turning images from lower-quality MRI machines with less magnetic strength to higher-quality images that would typically need to be captured from machines with higher magnetic strength. Then, the model would classify these higher-quality MRI images depending on whether the brain scan had AD or not. They showed that the performance of their framework was better than just the classification of either lower-quality or higher-quality images on their own [8]. This virtual upgrading method would allow clinics that must rely on lower-quality MRI machines to better identify patients with AD on par with newer machines despite budget or space constraints.

GANs can also help identify brain tumors in clinical images in a process known as segmentation. Segmentation is the process of dividing an image into visually meaningful parts and classifying them according to features such as shape, texture, and contour. In the case of brain tumors, segmentation refers to classifying brain tissue as either healthy or part of the tumor. Currently, brain tumors are primarily identified with questionnaires and imaging following a diagnosis, which limits the usage of preventative measures. Moreover, there is only a 33% 5-year survival rate for patients with brain cancer, which makes preventative work and early identification all the more important [12]. In simple MRI images, benign and cancerous tumors can look very similar, especially during the initial stages of a tumor’s development. Using a GAN model trained with over three thousand images of various tumors, a group of researchers from China and the United Kingdom were able to increase the accuracy of identifying brain tumors in MRI images to over 95% when compared to the average of 72% achieved in models without using generative images, offering a much more accurate model to classify brain tumors in a clinical setting [13]. Another GAN model designed to aid with brain tumor identification focused on generating images of brain tumors. This GAN model was able to generate synthetic images of three different kinds of brain tumors, including the most common glioma. When compared to the original images used for training, the generated images were very similar, suggesting that the generated images can serve as accurate representations of real clinical images for other models to use for classification training [14].

While this is a less studied application, these models can also be used to identify brain areas correlated with psychiatric disorders. A GAN-based model designed by a research group in China attempted to distinguish patients suffering from major depressive disorder (MDD) as well as schizophrenia from healthy subjects [15]. The team was able to design a model that could distinguish rs-fMRI scans of patients with depression from healthy ones with an accuracy rate of 70%, which is about 6% higher than the accuracy rates than  conventional methods. Similar levels of accuracy were also achieved with the classification of other mental disorders included in the study [15]. However,GANS used in the detection of psychiatric disorders tend to be a bit less accurate than the models designed for neurological disorders with clear-cut structural abnormalities, such as AD and malignant brain tumors.

It is also possible to use GAN models to diagnose neurological conditions by synthesizing information given across different imaging methods. One research group developed a GAN model that identifies visible signs of AD in patients using two different types of MRI imaging, one for functional neuron connections and the other for detailed structural analysis, while another team developed a GAN model that predicted positron-emission tomography (PET) scans based on MRI scans from patients in order to detect AD [16, 17]. PET scans trace the movement of labeled biomarkers in the brain, providing information about brain metabolism that is not available from MRI scans alone [18]. Making GAN models that can identify patterns across various kinds of brain scans allows for more accurate diagnosis of neurological conditions – after all, two heads (in this case, imaging methods) are better than one.

Using GANs to Predict and Model Disease Progressions

Outside of more efficient and accurate diagnosis using images, GANs also offer predictions to monitor disease progressions of patients with identified neurological conditions. For example, the GAN network designed by a team of researchers led by Dr. Ahmed Elazab has the ability to predict glioma tumor growth for different patients from 3D MRI images [19]. Growing from glial cells in the brain, glioma is one of the most common forms of brain tumors and accounts for approximately 50% of malignant tumor diagnoses as of 2020 [20]. While this particular GAN network was designed to predict the progression of glioma tumors, this technology can be used to aid the diagnosis and monitoring of similar brain tumors. In addition, GANs have the ability to generate series and sets of images, which allows clinicians to predict and simulate how the brain looks at various points in time. One study in 2021 created a GAN model that generates 3D MRI predictions for a brain, while at the same time combining a classification model that recognizes what stage of AD the predicted brain is at. The group was able to create generated images that looked similar to real AD progressions from multiple angles. However, this model mostly attempts to provide a holistic view of the entire brain and does not have as high of a progression accuracy when used to predict specific features or for short periods of time [21].

No two brains are exactly the same, and a GAN’s ability to predict changes in brain structure for any individual provides the potential for more personalized medical care as well as better analysis of a patient's risk of developing neurodegenerative diseases. Using MRI images from a group of older adults aged 45-81 to train their GAN, a research group in the United Kingdom was able to develop a model that predicts how a patient’s brain ages without needing anything more than one MRI image of the patient at any point in time [22]. Being able to generate a predicted progression of an aging healthy brain would allow clinicians to pick up abnormalities that deviate from the progression for any one individual, therefore allowing for preventative interventions.

Using GANs for Image Enhancement

GANs have the potential to aid physicians and other medical staff to make a confident diagnosis when clinical images are not robust enough. There are multiple reasons why medical images may not be robust enough, such as erroneous data points, incomplete images, and faults of the imaging device [23]. Current image reconstruction methods, such as Iterative reconstruction (IR), still stands as one of the most well-established ways to reconstruct computerized tomography (CT) scans, an imaging method that creates individual, cross-sectional slices of the body [24]. However, many of these methods become more computationally demanding the more advanced the data reconstruction needs to be, and require large amounts of data, therefore becoming more costly in terms of operations [25]. GAN models can be designed to improve image quality through deducing information from a given image, and based on patterns learned from a smaller pool of data, attempt to piece together a clearer image of a brain scan. Many of these methods have been tested with MRI scans as well as CT scans.

Sometimes, clinical images may be missing certain parts of the image that would disallow for a concrete diagnosis to be made. Image reconstruction GANs help with producing complete images based upon patterns from similar images. One team of scientists purposely removed certain raw data points from publicly-available MRI datasets in order to develop a GAN model, observing whether it could generate the most plausible guesses based on complete images. This model was able to closely imitate typical features in MRI brain scans of patients suffering from various neurological conditions ranging from strokes to multiple sclerosis, making predictions that closely matched the actual image [26]. Another group instead investigated the reconstruction of CT images. This team was able to develop a GAN model that reconstructs a healthy brain anatomy given a CT image. The reconstructed images would allow for the detection of both brain lesions and hemorrhages in actual CT images, identifying potential deviations from the corresponding generated “healthy” brain at an accuracy comparable to human discernment [27]. Image reconstruction GANs have also been developed to enhance clinical images from other techniques such as PET scans [28].

However, in other cases, while the clinical image is complete, it may be too blurry or otherwise too low in quality to use for diagnosis. In this case, image enhancement GANs can improve the quality of clinical images. For these GANs, their goal is to clean up a clinical image as much as possible, allowing for a much more detailed and clear picture. One method to do so is to design a noise reduction GAN. Strictly speaking, noise reduction is the improvement of an image through eliminating imperfections caused by the imaging system itself [29]. For example, low-dose CT is inherently a noisy imaging method due to its use of lower amounts of radiation than usual, instead opting for safer levels of radiation for the patient in exchange for noisier, lower quality scans. A research group was able to design a GAN that generated a regular dose CT image from low-dose CT scans, increasing the visual readability of a CT image while staying in the range of safe radiation levels [30]. On the other hand, some MRI images may be taken faster in exchange for shorter imaging times, and the resulting images are known as being undersample [31]. These undersampled images carry significantly less data than MRIs taken within a regular timeframe, making them inherently more noisy and blurry. GANs can help reconstruct these undersampled images into high-quality MRI images by learning from paired images, in which high-quality MRI images are correlated with the same image, but manipulated to be at a lower quality [32].These enhanced images would therefore allow accurate decision-making at the clinical level, while reducing the time needed for imaging at the same time.

Limitations

While there have been many advances in the applications of GANs in neuroscience, there are still some drawbacks to using these models. While scientists clearly understand the mathematical and computer science principles underpinning GANS, the way in which these computations lead to classification still continue to be a mystery. Currently, the computational basis of GANs are still not fully understood, and many researchers in fields such as machine learning are still attempting to crack the mystery behind its logic. Because of the unknown nature of  GANs’ inner workings, and to an extent the inner workings of all deep neural networks, clinicians may be suspicious of the results generated by these tools, instead opting to rely on their medical expertise from years of schooling and practice to diagnose these conditions. Additionally, GANs may be susceptible to “mode collapses,” which occur when a GAN fails to capture and create diverse data due to hyper-focusing on the initial dataset provided [33]. This leads to an inaccurate and inflexible model, which may cause more harm than good when deployed in a clinical setting. 

Conclusion

Generative Adversarial Networks, and other deep learning methods in extension, provide exciting new applications for clinical neuroscience. It is still currently a rather new field, but combined with the excitement surrounding generative AI methods and its general versatility, there is sure to be more research done to harness its effectiveness for clinical diagnosis purposes. GANs have shown immense potential to work in tandem with clinical professionals and physicians, providing patients with neurological diseases a more streamlined and balanced experience, while simultaneously allowing for healthcare providers to make more accurate, effective decisions.

References

[1] Bruls, R. J. M., & Kwee, R. M. (2020). Workload for radiologists during on-call hours: Dramatic increase in the past 15 years. Insights into Imaging, 11(1), 121. https://doi.org/10.1186/s13244-020-00925-z

[2] Wolterink, J. M., Mukhopadhyay, A., Leiner, T., Vogl, T. J., Bucher, A. M., & Išgum, I. (2021). Generative adversarial networks: A primer for radiologists. RadioGraphics, 41(3), 840–857. https://doi.org/10.1148/rg.2021200151

[3] Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann Machines. Cognitive Science, 9(1), 147–169. https://doi.org/10.1207/s15516709cog0901_7

[4] Sejnowski, P.J., Rosenberg, C.R. (1987). Parallel Networks that Learn to Produce English Text. Complex Systems, 1(1), 145-168.

[5] Badrulhisham, F., Pogatzki-Zahn, E., Segelcke, D., Spisak, T., & Vollert, J. (2024). Machine learning and artificial intelligence in neuroscience: A primer for researchers. Brain, Behavior, and Immunity, 115, 470–479. https://doi.org/10.1016/j.bbi.2023.11.005

[6] Matsuki, D. K., Kuperman, D. V., & Dyke, D. J. A. V. (2016). The Random Forests statistical technique: An examination of its value for the study of reading. Scientific Studies of Reading : The Official Journal of the Society for the Scientific Study of Reading, 20(1), 20. https://doi.org/10.1080/10888438.2015.1107073

[7] Wang, R., Bashyam, V., Yang, Z., Yu, F., Tassopoulou, V., Chintapalli, S. S., Skampardoni, I., Sreepada, L. P., Sahoo, D., Nikita, K., Abdulkadir, A., Wen, J., & Davatzikos, C. (2023). Applications of generative adversarial networks in neuroimaging and clinical neuroscience. NeuroImage, 269, 119898. https://doi.org/10.1016/j.neuroimage.2023.119898

[8] Zhou, X., Qiu, S., Joshi, P. S., Xue, C., Killiany, R. J., Mian, A. Z., Chin, S. P., Au, R., & Kolachalama, V. B. (2021). Enhancing magnetic resonance imaging-driven Alzheimer’s disease classification performance using generative adversarial learning. Alzheimer’s Research & Therapy, 13(1), 60. https://doi.org/10.1186/s13195-021-00797-5

[9] 2024 Alzheimer’s disease facts and figures. (2024). Alzheimer’s & Dementia, 20(5), 3708–3821. https://doi.org/10.1002/alz.13809

[10] Pan, J., Zuo, Q., Wang, B., Chen, C. L. P., Lei, B., & Wang, S. (2024). Decgan: Decoupling generative adversarial network for detecting abnormal neural circuits in alzheimer’s disease. IEEE Transactions on Artificial Intelligence, 5(10), 5050–5063. https://doi.org/10.1109/TAI.2024.3416420

[11] Lemée, J.-M., Berro, D. H., Bernard, F., Chinier, E., Leiber, L.-M., Menei, P., & Minassian, A. T. (2019). Resting‐state functional magnetic resonance imaging versus task‐based activity for language mapping and correlation with perioperative cortical mapping. Brain and Behavior, 9(10), e01362. https://doi.org/10.1002/brb3.1362

[12] Cancer of the brain and other nervous system—Cancer stat facts. (n.d.). SEER. Retrieved November 14, 2024, from https://seer.cancer.gov/statfacts/html/brain.html

[13] Ahmad, B., Sun, J., You, Q., Palade, V., & Mao, Z. (2022). Brain tumor classification using a combination of variational autoencoders and generative adversarial networks. Biomedicines, 10(2), 223. https://doi.org/10.3390/biomedicines10020223

[14] Mukherkjee, D., Saha, P., Kaplun, D., Sinitca, A., & Sarkar, R. (2022). Brain tumor image generation using an aggregation of GAN models with style transfer. Scientific Reports, 12(1), 9141. https://doi.org/10.1038/s41598-022-12646-y

[15] Zhao, J., Huang, J., Zhi, D., Yan, W., Ma, X., Yang, X., Li, X., Ke, Q., Jiang, T., Calhoun, V. D., & Sui, J. (2020). Functional network connectivity (Fnc)-based generative adversarial network (Gan) and its applications in classification of mental disorders. Journal of Neuroscience Methods, 341, 108756. https://doi.org/10.1016/j.jneumeth.2020.108756

[16] Hassanzadeh, R., Abrol, A., Hassanzadeh, H. R., & Calhoun, V. D. (2024). Cross-modality translation with generative adversarial networks to unveil alzheimer’s disease biomarkers (arXiv:2405.05462). arXiv. https://doi.org/10.48550/arXiv.2405.05462

[17] Zhang, J., He, X., Qing, L., Gao, F., & Wang, B. (2022). BPGAN: Brain PET synthesis from MRI using generative adversarial network for multi-modal Alzheimer’s disease diagnosis. Computer Methods and Programs in Biomedicine, 217, 106676. https://doi.org/10.1016/j.cmpb.2022.106676

[18] Kapoor, M., & Kasi, A. (2024). Pet scanning. In StatPearls. StatPearls Publishing. http://www.ncbi.nlm.nih.gov/books/NBK559089/

[19] Elazab, A., Wang, C., Gardezi, S. J. S., Bai, H., Hu, Q., Wang, T., Chang, C., & Lei, B. (2020). GP-GAN: Brain tumor growth prediction using stacked 3D generative adversarial networks from longitudinal MR Images. Neural Networks: The Official Journal of the International Neural Network Society, 132, 321–332. https://doi.org/10.1016/j.neunet.2020.09.004

[20] Ostrom, Q. T., Price, M., Neff, C., Cioffi, G., Waite, K. A., Kruchko, C., & Barnholtz-Sloan, J. S. (2023). Cbtrus statistical report: Primary brain and other central nervous system tumors diagnosed in the United States in 2016-2020. Neuro-Oncology, 25(12 Suppl 2), iv1–iv99. https://doi.org/10.1093/neuonc/noad149

[21] Zhao, Y., Ma, B., Jiang, P., Zeng, D., Wang, X., & Li, S. (2021). Prediction of alzheimer’s disease progression with multi-information generative adversarial network. IEEE Journal of Biomedical and Health Informatics, 25(3), 711–719. https://doi.org/10.1109/JBHI.2020.3006925

[22] Gadewar, S. P., Ramesh, A., Liu, M., Gari, I. B., Nir, T. M., Thompson, P., & Jahanshad, N. (2023). Predicting individual brain MRIs at any age using style encoding generative adversarial networks. 18th International Symposium on Medical Information Processing and Analysis, 12567, 471–479. https://doi.org/10.1117/12.2669741

[23] Padmapriya, S. T., & Parthasarathy, S. (2024). Ethical data collection for medical image analysis: A structured approach. Asian Bioethics Review, 16(1), 95–108. https://doi.org/10.1007/s41649-023-00250-9

[24] Willemink, M. J., & Noël, P. B. (2019). The evolution of image reconstruction for CT-from filtered back projection to artificial intelligence. European Radiology, 29(5), 2185–2195. https://doi.org/10.1007/s00330-018-5810-7

[25] Patel, P. R., & De Jesus, O. (2024). Ct scan. In StatPearls. StatPearls Publishing. http://www.ncbi.nlm.nih.gov/books/NBK567796/

[26] Shaul, R., David, I., Shitrit, O., & Riklin Raviv, T. (2020). Subsampled brain MRI reconstruction by generative adversarial neural networks. Medical Image Analysis, 65, 101747. https://doi.org/10.1016/j.media.2020.101747

[27] Walluscheck, S., Gerken, A., Galinovic, I., Villringer, K., Fiebach, J. B., Klein, J., & Heldmann, S. (2024). Generative adversarial network–based reconstruction of healthy anatomy for anomaly detection in brain CT scans. Journal of Medical Imaging, 11(4), 044508. https://doi.org/10.1117/1.JMI.11.4.044508

[28] Ouyang, J., Chen, K. T., Gong, E., Pauly, J., & Zaharchuk, G. (2019). Ultra-low-dose PET reconstruction using generative adversarial network with feature matching and task-specific perceptual loss. Medical Physics, 46(8), 3555–3564. https://doi.org/10.1002/mp.13626

[29] Ketteler, R., & Kriston-Vizi, J. (2016). High-content screening in cell biology. In R. A. Bradshaw & P. D. Stahl (Eds.), Encyclopedia of Cell Biology (pp. 234–244). Academic Press. https://doi.org/10.1016/B978-0-12-394447-4.40041-6

[30] Wolterink, J. M., Leiner, T., Viergever, M. A., & Isgum, I. (2017). Generative adversarial networks for noise reduction in low-dose ct. IEEE Transactions on Medical Imaging, 36(12), 2536–2545. https://doi.org/10.1109/TMI.2017.2708987

[31] Kojima, S., Shinohara, H., Hashimoto, T., & Suzuki, S. (2018). Undersampling patterns in k-space for compressed sensing MRI using two-dimensional Cartesian sampling. Radiological Physics and Technology, 11(3), 303–319. https://doi.org/10.1007/s12194-018-0469-y

[32] Upadhyay, U., Sudarshan, V. P., & Awate, S. P. (2021). Uncertainty-aware gan with adaptive loss for robust mri image enhancement (arXiv:2110.03343). arXiv. https://doi.org/10.48550/arXiv.2110.03343

[33] Meor Yahaya, M. S., & Teo, J. (2023). Data augmentation using generative adversarial networks for images and biomarkers in medicine and neuroscience. Frontiers in Applied Mathematics and Statistics, 9. https://doi.org/10.3389/fams.2023.1162760