M1.4 | Use of prior knowledge for interventional MRI

Funding period: Jan 2018 to Jun 2022

Researcher: Soumick Chatterjee

 

Wrap-up

Keywords: Undersampled MRI reconstruction, radial MRI reconstruction, deep learning, complex-valued convolutions, prior knowledge

Background:
Magnetic resonance imaging (MRI) can provide high spatial resolution for detecting minute pathological changes in tissues. However, MRi is an inherently slow process due to consecutive data acquisition, which leads to a long scan time for high-resolution imaging, The acquisition speed can be increased by acquiring less data (undersampling). Consequently, this leads to the degradation of image quality, such as loss of resolution or introduction of image artefacts. 

Objective:
Development of deep learning based techniques for dealing with the problem of undersampled MRIs, which are capable of real-time execution

Methods:
Two different methods have been developed for artefact reduction in the image space - NCC1701 (including ReconResNet model) and PD-UNet, one method has been developed for the super-resolution of undersampled static MRIs - ShuffleUNet, two methods have been developed for super-resolution of undersampled dynamic MRIs - Fine-tuning with prior knowledge and DDoS-UNet, KSPReconResNet has been developed for the reconstruction in the k-space, and finally Fourier-PD and Fourier-PDUNet models for reconstruction in the hybrid space (image and k-space).

Results:
The developed methods have improved the performance of the compared to the state-of-the-art methods. For the task of undersampled MRI reconstruction, the hybrid space models performed better than pure image space model. Incorporation of prior knowledge improved the performance of the models.

Conclusions:
This thesis studied various deep learning techniques for the tasks of undersampled MRI reconstruction and retrospective motion correction. This thesis proposed two different methods for artefact reduction in the image space of undersampled MRIs, while proposing another method aimed at the reconstruction of undersampled radial MRIs, which unifies the problem of sparse CT and undersampled radial MRIs. Furthermore, the problem of undersampled MRI reconstruction has been dealt with as a Super-Resolution problem, and three different methods have been proposed - one of which attempts to learn both spatial and temporal relationships to perform undersampled dynamic MRIs. Eventually, the problem of undersampled reconstruction was addressed directly at k-space and hybrid space (mix of image and k-space). Afterwards, this thesis proposed five different automatic deep learning based MRI processing pipelines and evaluated them while working with undersampled and reconstructed data. Finally, the models proposed for undersampled reconstruction (ReconResNet, Fourier-PD, Fourier-PDUNet) were employed for the task of retrospective motion correction.

Originality:
Development of deep learning architectures - ReconResNet, PO-UNet, KSPReconResNet, ShuffleUNet, Fourier-PD, Fourier-PDUNet; development of processing pipelines for undersampled reconstruction - NCC1701 and DDoS-UNet;

(1) Chatterjee, S., Sarasaen, C., Rose, G., Nürnberger, A., and Speck, O. (2022): DDoS-UNet: Incorporating temporal information using Dynamic Dual-channel UNet for enhancing super-resolution of dynamic MRI.
>> PUBLICATION
>> VIDEO PRESENTATION

(2) Sciarra, A., Chatterjee, S., Dünnwald, M., Placidi, G., Nürnberger, A., Speck, O., and Oeltze-Jafra, S. (2022): Reference-less SSIM Regression for Detection and Quantification of Motion Artefacts in Brain MRIs.
>> PUBLICATION
>> VIDEO PRESENTATION

(3) Ernst, P., Chatterjee, S., Rose, G., and Nürnberger, A. (2022): Primal-Dual UNet for Sparse View Cone Beam Computed Tomography Volume Reconstruction.
>> PUBLICATION


(Medical Imaging with Deep Learning – MIDL 2022,
6-8 July 2022 @Zürich, CH)

(1) Chatterjee, S., Sciarra, A., Dünnwald, M., Talagini Ashoka, A. B., Cheepinahalli Vasudeva, M. G., Saravanan, S., Thirugnana Sambandham, V., Oeltze-Jafra, S., Speck, O., and Nürnberger, A. (2022): Uncertainty quantification for ground-truth free evaluation of deep learning reconstructions.
>> ABSTRACT
>> VIDEO PRESENTATION

(2) Chatterjee, S., Bajaj, H., Siddiquee, I., Nandish, B. S., Simon, S., Shashidhar, S., Speck, O., and Nürnberger, A. (2022): Multi-scale UNet with Self-Constructing Graph Latent for Deformable Image Registration.
>> ABSTRACT
>> CODE
>> VIDEO PRESENTATION

(3) Chatterjee, S., Sciarra, A., Dünnwald, M., Tummala, P., Agrawal, S., Jauhari, A., Kalra, A., Oeltze-Jafra, S., Speck, O., and Nürnberger, A. (2022): StRegA: Unsupervised Anomaly Detection in Brain MRIs using Compact Context-encoding Variational Autoencoder.
>>ABSTRACT
>> CODE
>> VIDEO PRESENTATION

(4) Chatterjee, S., Yassin, H., Dubost, F., Nürnberger, A., and Speck, O. (2022): Learning to segment brain tumours using an explainable classifier.
>>ABSTRACT
>> CODE
>> VIDEO PRESENTATION

 (ISMRM-ESMRMB & ISMRT 31st Annual Meeting,
7-12 May 2022 @London, UK)

VIDEO PRESENTATION by Soumick Chatterjee (MEMoRIAL) on
Ernst, P., Chatterjee, S., Rose, G., Speck, O., and Nürnberger, A. (2022): Sinogram upsampling using Primal-Dual UNet for undersampled CT and radial MRI reconstruction.
Further information...
(IEEE ISBI 2022 / Mar 28-31, 2022)

VIDEO PRESENTATION by Soumick Chatterjee (MEMoRIAL) on
Chatterjee, S., Sciarra, A., Dünnwald, M., Mushunuri, R. V., Podishetti, R., Rao, R. N., Gopinath, G. D., Oeltze-Jafra, S., Speck, O., and Nürnberger, A. (2021): ShuffleUNet: Super resolution of diffusion-weighted MRIs using deep learning.
(IEEE EUSIPCO 2021 / Aug 26, 2021)

  1. Chatterjee, S., Sarasaen, C., Sciarra, A., Breitkopf, M., Oeltze-Jafra, S., Nürnberger, A., and Speck, O. (2021): Going beyond the image space: undersampled MRI reconstruction directly in the k-space using a complex valued residual neural network.
    >> VIDEO PRESENTATION
  2. Chatterjee, S., Das, A., Mandal, C., Mukhopadhyay, B., Vipinraj, M., Shukla, A., Speck, O., and Nürnberger, A. (2021): Interpretability Techniques for Deep Learning based Segmentation Models.
    >> VIDEO PRESENTATION
  3. Chatterjee, S., Bajaj, H., Shashidhar, S. B., Indushekar S. B., Simon, S., Siddiquee, I. H., Subbarayappa, N. B., Speck, O., and Nürnberger, A. (2021): A Comparative Study of Deep Learning Based Deformable Image Registration Techniques.
    >> VIDEO PRESENTATION
  4. Chatterjee, S., Sciarra, A., Dünnwald, M., Agrawal, S. K., Tummala, P., Setlur, D., Kalra, A., Jauhari, A., Oeltze-Jafra, S., Speck, O., and Nürnberger, A. (2021): Unsupervised reconstruction based anomaly detection using a Variational Auto Encoder.
    >> ABSTRACT + DIGITAL POSTER
  5. Sarasaen, C., Chatterjee, S., Saad, F., Breitkopf, M., Nürnberger, A., and Speck, O. (2021): Fine-tuning deep learning model parameters for improved super-resolution of dynamic MRI with prior-knowledge.
    >> ABSTRACT
  6. Nath, V., Pizzolato, M., Palombo, M., Gyori, N., Schilling, K., Hansen, C., Yang, Q., Kanakaraj, P., Landman, B., Chatterjee, S., Sciarra, A., Dünnwald, M., Oeltze-Jafra, S., Nürnberger, A., Speck, O., Pieciak, T., Baranek, M., Bartocha, K., Ciupek, D., and Hutter, J. (2021): Resolving to super resolution multi-dimensional diffusion imaging (Super-MUDI).
    >> ABSTRACT

 (ISMRM & SMRT Virtual Conference & Exhibition / May 15-20, 2021)

VIDEO PRESENTATION by Soumick Chatterjee (MEMoRIAL) on
Chatterjee, S., Prabhu, K., Pattadkal, M., Bortsova, G., Sarasaen, C., Dubost, F., Mattern, H., De Bruijne, M., Speck, O., and Nürnberger, A. (2021):
DS6, Deformation-aware Semi-supervised Learning: Application to Vessel Segmentation with Noisy Data.
Further information ...
(Invited talk at Stanford University, Contrastive & Semi-Supervised Learning Group, CA, US / Mar 12, 2021)

VIDEO PRESENTATION by Soumick Chatterjee (MEMoRIAL) on
Chatterjee, S., Sciarra, A., Dünnwald, M., Oeltze-Jafra, S., Nürnberger, A., and Speck, O. (2020): Retrospective Motion Correction of MR Images using Prior-Assisted Deep Learning.
(Medical Imaging Meets NeurIPS 2020 / Dec 12, 2020)

VIDEO PRESENTATION by Soumick Chatterjee (MEMoRIAL) on
Chatterjee, S., Putti, P., Nürnberger, A., and Speck, O. (2020): Wavelet filtering of undersampled MRI using trainable wavelets and CNN.
(ESMRMB 2020 ONLINE / Sep 30 - Oct 02, 2020)

  1. Mitta, D., Chatterjee, S., Speck, O., and Nürnberger, A. (2020): Unsupervised learning for Abdominal MRI Segmentation using 3D Attention W-Net.
    >> VIDEO PRESENTATION
  2. Sciarra, A., Chatterjee, S., Dünnwald, M., Speck, O., and Oeltze-Jafra, S. (2020): Evaluation of Deep Learning Techniques for Motion Artifacts Removal.
    >>
    ABSTRACT + DIGITAL POSTER
  3. Mattern, H., Sciarra, A., Dünnwald, M., Chatterjee, S., Müller, U., Oeltze-Jafra, S., and Speck, O. (2020): Contrast prediction-based regularization for iterative reconstructions (PROSIT).
    >> ABSTRACT

 (ISMRM & SMRT Virtual Conference & Exhibition / Aug 08-14, 2020)

Last Modification: 24.02.2023 - Contact Person: