bbbbbbzhou / dudornet

DuDoRNet: Learning a Dual-Domain Recurrent Network for Fast MRI Reconstruction with Deep T1 Prior (CVPR 2020)

DuDoRNet: Learning a Dual-Domain Recurrent Network for Fast MRI Reconstruction with Deep T1 Prior

Bo Zhou, S. Kevin Zhou

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020

[Paper]

This repository contains the PyTorch implementation of DuDoRNet.

Citation

If you use this code for your research or project, please cite:

@inproceedings{zhou2020dudornet,
  title={DuDoRNet: Learning a Dual-Domain Recurrent Network for Fast MRI Reconstruction with Deep T1 Prior},
  author={Zhou, Bo and Zhou, S Kevin},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={4273--4282},
  year={2020}
}

Environment and Dependencies

Requirements:

• Python 3.7
• Pytorch 0.4.1
• scipy
• scikit-image
• opencv-python
• tqdm

Our code has been tested with Python 3.7, Pytorch 0.4.1, CUDA 10.0 on Ubuntu 18.04.

Dataset Setup

.
Data
├── TRAIN                   # contain training files
│   ├── T1
│   │   ├── kspace
│   │   │   ├── train_1.mat         
│   │   │   ├── train_2.mat 
│   │   │   ├── ...         
│   │   │   └── train_N.mat 
│   │   └── ...
│   │   
│   ├── T2
│   │   ├── kspace
│   │   │   ├── train_1.mat          
│   │   │   ├── train_2.mat 
│   │   │   ├── ...         
│   │   │   └── train_N.mat 
│   │   └── ...
│   │   
│   ├── FLAIR
│   │   ├── kspace
│   │   │   ├── train_1.mat          
│   │   │   ├── train_2.mat 
│   │   │   ├── ...         
│   │   │   └── train_N.mat 
│   │   └── ...
│   └── ...
│
├── VALI                    # contain validation files
│   ├── T1
│   │   ├── kspace
│   │   │   ├── vali_1.mat          
│   │   │   ├── vali_2.mat 
│   │   │   ├── ...         
│   │   │   └── vali_M.mat 
│   │   └── ...
│   │   
│   ├── T2
│   │   ├── kspace
│   │   │   ├── vali_1.mat          
│   │   │   ├── vali_2.mat 
│   │   │   ├── ...         
│   │   │   └── vali_M.mat 
│   │   └── ...
│   │   
│   ├── FLAIR
│   │   ├── kspace
│   │   │   ├── vali_1.mat          
│   │   │   ├── vali_2.mat 
│   │   │   ├── ...         
│   │   │   └── vali_M.mat 
│   │   └── ...
│   └── ...
│
├── TEST                    # contain test files
│   ├── T1
│   │   ├── kspace
│   │   │   ├── test_1.mat          
│   │   │   ├── test_2.mat 
│   │   │   ├── ...         
│   │   │   └── test_K.mat 
│   │   └── ...
│   │   
│   ├── T2
│   │   ├── kspace
│   │   │   ├── test_1.mat          
│   │   │   ├── test_2.mat 
│   │   │   ├── ...         
│   │   │   └── test_K.mat 
│   │   └── ...
│   │   
│   ├── FLAIR
│   │   ├── kspace
│   │   │   ├── test_1.mat          
│   │   │   ├── test_2.mat 
│   │   │   ├── ...         
│   │   │   └── test_K.mat 
│   │   └── ...
│   └── ...
│            
└── ...

Each .mat should contain a W x W complex value matrix with kspace data in it, where W x W is the kspace size. Please note the variable name should be set as 'kspace_py'. Then, please add the data directory './Data/' after --data_root in the code or scripts.

To Run Our Code

• Train the model

python train.py --experiment_name 'train_DuDoRN_R4_pT1' --data_root './Data/' --dataset 'Cartesian' --netG 'DRDN' --n_recurrent 4 --use_prior --protocol_ref 'T1' --protocol_tag 'T2'

where
--experiment_name provides the experiment name for the current run, and save all the corresponding results under the experiment_name's folder.
--data_root provides the data folder directory (with structure illustrated above).
--n_recurrent defines number of recurrent blocks in the DuDoRNet.
--protocol_tag defines target modality to be reconstruct, e.g. T2 or FLAIR.
--protocol_ref defines modality to be used as prior, e.g. T1.
--use_prior defines whether to use prior as indicated by protocol_ref.
Other hyperparameters can be adjusted in the code as well.

• Test the model

python test.py --experiment_name 'test_DuDoRN_R4_pT1' --accelerations 5 --resume './outputs/train_DuDoRN_R4_pT1/checkpoints/model_259.pt' --data_root './Data/' --dataset 'Cartesian' --netG 'DRDN' --n_recurrent 4 --use_prior --protocol_ref 'T1' --protocol_tag 'T2'

where
--accelerations defines the acceleration factor, e.g. 5 for 5 fold accelerations.
--resume defines which checkpoint for testing and evaluation.
The test will output an eval.mat containing model's input, reconstruction prediction, and ground-truth for evaluation.

Sample training/test scripts are provided under './scripts/' and can be directly executed.

Contact

If you have any question, please file an issue or contact the author:

Bo Zhou: [email protected]

dudornet's Issues

Hi, when will the code published?

Dear author,

As the CVPR is on going, what's your plan to open-source the code?

Bests

Mask generation function

Hello author, two masks radial and spiral are mentioned in the paper, but there is only Cartesian mask generation method in the repository, may I ask how to make these two masks?

how many epochs needed

I have trained your model from the scratch, but at the first 7 epochs (up to now), the images generated are strange, very different from the target images, and i cannot see any tissue structure, is that right?

data processing

Hello author！In the definition of fft2 (the same as ifft2) in the utils module, why do I need ifftshift( ) and fftshift( ) before and after torch. fft( the same as torch.ifft )? I don't quite understand this, can you help me explain it?

About the weight initialization

Hi! Thanks for your great work.

I am currently also working on deep learning-based MRI reconstruction and I am using DuDoRNet as the reconstruction network. I noticed that in the codebase, the network uses a special weight initialization, rather than the default initialization of pytorch. I am wondering why? Cause I found that if I used the default initialization, the convergence of the network became faster.

关于数据问题

作者您好，我想用您的代码仅用来测试，能不能提供一下训练过后的参数文件和测试数据呢？

Dudo

The input data dimensions would be mysteriously altered

As shown in the above picture, the dimensions of x_I that I am going to input into the model are [3, 2, 320, 230]， 3---batch_size, 2---num_channels, 320---height, 230---width

And I find when x_I comes to self.net_G_I, it equals to x comes to self.SFENET1. However, it occurs the strangest thing that the size of x becomes [3, 2, 320, 230, 2] ( at this time, the model has not worked on x), and I am sure there is no obvious operation in these codes to change the size of x_I or x before the model's work.

I look into x[:, :, :, :, 1], and I find x[:, :, :, :, 1] is totally a zero matrix as this picture:

I am so confused by why the dimension increases from x_I to x, and I would appreciate it if you could answer me.