debeshjha / resunetplusplus

Official code for ResUNetplusplus for medical image segmentation (TensorFlow & Pytorch implementation)

ResUNet++: An advanced architecture for medical image segmentation

Tensoflow implementation of ResUNet++

ResUNet++

The ResUNet++ architecture is based on the Deep Residual U-Net (ResUNet), which is an architecture that uses the strength of deep residual learning and U-Net. The proposed ResUNet++ architecture takes advantage of the residual blocks, the squeeze and excitation block, ASPP, and the attention block. More description about the archicture can be in the paper [ResUNet++: An Advanced Architecture for Medical Image Segmentation] (https://arxiv.org/pdf/1911.07067.pdf).

Architecture

Requirements:

os
numpy
cv2
tensorflow
glob
tqdm

Folders:

data: Contains the set of three dataset as mentioned.
files: Contains the csv file and weight file generated during training.
new_data: Contains two subfolder `images` and `masks`, they contains the augmented images and masks.

Files:

1. process_image.py: Augment the images and mask for the training dataset.
2. data_generator.py: Dataset generator for the keras.
3. infer.py: Run your model on test dataset and all the result are saved in the result` folder. The images are in the sequence: Image,Ground Truth Mask, Predicted Mask.
4. run.py: Train the unet.
5. unet.py: Contains the code for building the UNet architecture.
6. resunet.py: Contains the code for building the ResUNet architecture.
7. m_resunet.py: Contains the code for building the ResUNet++ architecture.
8. mertrics.py: Contains the code for dice coefficient metric and dice coefficient loss. 

First check for the correct path and the patameters.

1. python3 process_image.py - to augment training dataset.
2. python3 run.py - to train the model.
3. python3 infer.py - to test and generate the mask.

Pytorch implementation of ResUNet++ can be found here :

https://github.com/rishikksh20/ResUnet/blob/master/core/res_unet_plus.py

Dataset Link

Kvasir-SEG

Results

Qualitative results comparison on the Kvasir-SEG dataset.From the left: image (1), (2) Ground truth, (3) U-Net, (4) ResUNet, (5) ResUNet-mod, and (6) ResUNet++.

Citation

Please cite our paper if you find the work useful:

@INPROCEEDINGS{8959021,
  author={D. {Jha} and P. H. {Smedsrud} and M. A. {Riegler} and D. {Johansen} and T. D. {Lange} and P. {Halvorsen} and H. {D. Johansen}},
  booktitle={Proceedings of the IEEE International Symposium on Multimedia (ISM)}, 
  title={ResUNet++: An Advanced Architecture for Medical Image Segmentation}, 
  year={2019},
  pages={225-230}}

Contact

Please contact [email protected] for any further questions.

resunetplusplus's Issues

RUN.PY

UnboundLocalError: local variable 'logs' referenced before assignment I don't konw how to solve this

Thank you for your contribution to this field

Thank you for your contribution to this field！Could you share the weight file? (resunetplusplus.h5)

resunet++

I want the weight file.

Resunetplusplus

I want the wieght file

about the file infer.py

Thanks for your contribution about the work.
I can't evaluate the results, and it raise an error:
TypeError: '<' not supported between instances of 'function' and 'str', how can I solve the problem?

training time

thanks for your work,how long it will training,I run run.py,it seems take a lot if time to train

I don't know how to use the test set to test the IOU, Precision and loss of the model

Infer. Py can only get the predicted results, but I don't know how to use the test set to test the IOU, Precision and loss of the model. I'm sorry that I'm a rookie.

About the right weight file?

Excuse me, can you upload the right weight file? I want to do an experiment to verify the performance of the network, but because there is no weight file I can't do it. It is very important to me if you can please tell me where to download the file weights，thank you.

Data missing

Hi, I wanted to test your method, however, all the data directories are empty, so I cannot complete your suggested "First check for the correct path and the parameters." initial test. Best, Jon

AttributeError: 'Operation' object has no attribute '_c_op'

tensorflow2.1
dear professor, i met this error,hope you could help me.Thank you.
Traceback (most recent call last):
File "run.py", line 118, in
imgs_mask_test = model.predict(imagestest, verbose=1)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 1013, in predict
use_multiprocessing=use_multiprocessing)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 498, in predict
workers=workers, use_multiprocessing=use_multiprocessing, **kwargs)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 475, in _model_iteration
total_epochs=1)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 128, in run_one_epoch
batch_outs = execution_function(iterator)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 98, in execution_function
distributed_function(input_fn))
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 568, in call
result = self._call(*args, **kwds)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 615, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 497, in _initialize
*args, **kwds))
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 2389, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 2703, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 2593, in _create_graph_function
capture_by_value=self._capture_by_value),
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py", line 978, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 439, in wrapped_fn
return weak_wrapped_fn().wrapped(*args, **kwds)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 85, in distributed_function
per_replica_function, args=args)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 763, in experimental_run_v2
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 1819, in call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 2164, in _call_for_each_replica
return fn(*args, **kwargs)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/autograph/impl/api.py", line 292, in wrapper
return func(*args, **kwargs)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 212, in _predict_on_batch
result = predict_on_batch(model, x)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 556, in predict_on_batch
return predict_on_batch_fn(inputs) # pylint: disable=not-callable
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 778, in call
outputs = call_fn(cast_inputs, *args, **kwargs)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py", line 717, in call
convert_kwargs_to_constants=base_layer_utils.call_context().saving)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py", line 891, in _run_internal_graph
output_tensors = layer(computed_tensors, **kwargs)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 778, in call
outputs = call_fn(cast_inputs, *args, **kwargs)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/keras/layers/convolutional.py", line 209, in call
outputs = self._convolution_op(inputs, self.kernel)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/ops/nn_ops.py", line 1135, in call
return self.conv_op(inp, filter)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/ops/nn_ops.py", line 640, in call
return self.call(inp, filter)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/ops/nn_ops.py", line 616, in _with_space_to_batch_call
block_shape=self.dilation_rate)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/ops/array_ops.py", line 3303, in required_space_to_batch_paddings
const_block_shape = tensor_util.constant_value(block_shape)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/framework/tensor_util.py", line 827, in constant_value
ret = _ConstantValue(tensor, partial)
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/framework/tensor_util.py", line 673, in _ConstantValue
if tensor.op.type == "Const":
File "/home/ma-user/anaconda3/envs/TensorFlow-2.1.0/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 2220, in type
return c_api.TF_OperationOpType(self._c_op)
AttributeError: 'Operation' object has no attribute '_c_op'

Could you upload the weight file?

Hello. Is it possible for you to upload the weight file that is trained with Kvasir-SEG dataset?
This would help a lot.
Thank you so much.

Dimensions must be equal

File "D:\360安全浏览器下载\heatmap-main\heatmap-main\ResUNetPlusPlus\run.py", line 81, in
model.fit(train_gen, #_generator
File "D:\360安全浏览器下载\heatmap-main\venv\Lib\site-packages\keras\src\utils\traceback_utils.py", line 122, in error_handler
raise e.with_traceback(filtered_tb) from None
File "D:\360安全浏览器下载\heatmap-main\heatmap-main\ResUNetPlusPlus\metrics.py", line 16, in dice_loss
return 1.0 - dice_coef(y_true, y_pred)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\360安全浏览器下载\heatmap-main\heatmap-main\ResUNetPlusPlus\metrics.py", line 11, in dice_coef
intersection = tf.reduce_sum(y_true_f * y_pred_f)
~~~~~~~~~^~~~~~~~~~
ValueError: Dimensions must be equal, but are 196608 and 65536 for '{{node compile_loss/dice_loss/mul}} = Mul[T=DT_FLOAT](compile_loss/dice_loss/flatten_8_1/Reshape, compile_loss/dice_loss/flatten_9_1/Reshape)' with input shapes: [?,196608], [?,65536].

I couldn't understand how to do the same work using FCN8 model

Thanks for your contribution about the work. Could you please tell me how can I do the same work using FCN8 model? Where should I change in the code?

The dataset

where is the Kvasir-SEG dataset??
i can't find it anywhere...

process_image.py

I run process_image.py,but I got an empty fold in new_data