milesial / pytorch-unet Goto Github PK
View Code? Open in Web Editor NEWPyTorch implementation of the U-Net for image semantic segmentation with high quality images
License: GNU General Public License v3.0
PyTorch implementation of the U-Net for image semantic segmentation with high quality images
License: GNU General Public License v3.0
How does one alter the eval or training loops to view the precision and recall of the network?
Hello
Thanks for your contribution for making awesome network with pytorch.
I am newbie in this and I have a problem with training this model with one channel image (black and white)
not 3 channels image(RGB)
How can I modify your network to train and test???
Thanks
My OS is Ubuntu 16.04.4 LTS, and Cuda is V7.5.17. My Pytorch is 0.4.
My script to run is:
predict.py --cpu -i ~/Desktop/download.jpg -o output.jpg
But I encounter the following error:
THCudaCheck FAIL file=torch/csrc/cuda/Module.cpp line=51 error=30 : unknown error
Traceback (most recent call last):
File "/projects/segmentation/Pytorch-UNet/predict.py", line 88, in <module>
net.load_state_dict(torch.load(args.model))
File "/anaconda3/lib/python3.6/site-packages/torch/serialization.py", line 303, in load
return _load(f, map_location, pickle_module)
File "/anaconda3/lib/python3.6/site-packages/torch/serialization.py", line 469, in _load
result = unpickler.load()
File "/anaconda3/lib/python3.6/site-packages/torch/serialization.py", line 437, in persistent_load
data_type(size), location)
File "/anaconda3/lib/python3.6/site-packages/torch/serialization.py", line 88, in default_restore_location
result = fn(storage, location)
File "/anaconda3/lib/python3.6/site-packages/torch/serialization.py", line 70, in _cuda_deserialize
return obj.cuda(device)
File "/anaconda3/lib/python3.6/site-packages/torch/_utils.py", line 68, in _cuda
with torch.cuda.device(device):
File "/anaconda3/lib/python3.6/site-packages/torch/cuda/__init__.py", line 225, in __enter__
self.prev_idx = torch._C._cuda_getDevice()
RuntimeError: cuda runtime error (30) : unknown error at torch/csrc/cuda/Module.cpp:51
I tried to input the U-net with different size of the input image. It seems that sometimes when the size number is odd, it will lead to some problem when we upsample, especially when we concat different Xi(x, x1, x2, x3, x4, x5). Is there some way we can change the U-net to fit any size of the input image? Thankyou.
Hi, I ran the code. I used the new Pytorch 0.4.0. I found it needs large GPU memory. Batchsize=4 uses 10G memory. Is it right?
In the code of train.py,I cannot find how to initialize the parameters of Unet.So how to initialize the weight and the bias of the Unet?
Hello!
I am using this project for segmenting teeth, and my dataset has different size that the original one, so that gives me this error.
"Please ensure they have the same size.".format(target.size(), input.size())) Traceback (most recent call last): File "train.py", line 161, in <module> img_scale=args.scale) File "train.py", line 102, in train_net loss = criterion(masks_probs_flat, true_masks_flat) File "/home/miuser/.local/lib/python3.5/site-packages/torch/nn/modules/module.py", line 477, in __call__ result = self.forward(*input, **kwargs) File "/home/miuser/.local/lib/python3.5/site-packages/torch/nn/modules/loss.py", line 486, in forward return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction) File "/home/miuser/.local/lib/python3.5/site-packages/torch/nn/functional.py", line 1597, in binary_cross_entropy "!= input nelement ({})".format(target.nelement(), input.nelement())) ValueError: Target and input must have the same number of elements. target nelement (1267876) != input nelement (1254400)
Does someone know what I have to change and where?, so this could work with my dataset.
Thanks!
hi, @milesial ,
thanks for ur work for every researchers.
In the process of citing ur work, i found somethings is wrong.
In 'utils/load.py', the return value should be '((id, i) for id in ids for i in range(n))' rather than '((id, i) for i in range(n) for id in ids)'.
if i'm correct:
please tell me the that's true
else:
forgive me
I think there is an error in unet_parts.py in the up module after switching to use ConvTranspose2d instead of bilinear upsampling.
self.up = nn.ConvTranspose2d(in_ch, out_ch, 2, stride=2)
should be changed to
self.up = nn.ConvTranspose2d(in_ch//2, in_ch//2, 2, stride=2)
Hi,
Thanks for your open-source code of your project. I have few questions when I use my private data to tain the model, I resize my data to [256, 128] and get negative loss and very large like this.
![image](https://user-images.githubusercontent.com/22422251/40101000-9517981e-5918-11e8-8170-ddaa06def00e.png
Is this normal?If not, how can I fix that?
Looking forward to your reply!
Hi,
Thanks for your codes.
I trained the model with only changing batch size to 4 and I predicted some test images.
The right part in the image from a trained model is distorted and right parts in other results are also similar.
I was wondering why this happened and what your pre-trained parameters or any pre-processing except splitting a image to left and right are. Thanks.
Loading model MODEL.pth
Using CUDA version of the net, prepare your GPU !
Model loaded !
Predicting image ap3.jpg ...
Traceback (most recent call last):
File "predict.py", line 154, in
use_gpu=not args.cpu)
File "predict.py", line 42, in predict_img
output_left = net(X_left)
File "/home/shushi16/miniconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/shushi16/wyz/Pytorch-UNet/unet/unet_model.py", line 26, in forward
x = self.up2(x, x3)
File "/home/shushi16/miniconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/shushi16/wyz/Pytorch-UNet/unet/unet_parts.py", line 68, in forward
x = torch.cat([x2, x1], dim=1)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 84 and 83 in dimension 2 at /opt/conda/conda-bld/pytorch_1524577177097/work/aten/src/THC/generic/THCTensorMath.cu:111
Hi, @changjo , thanks for your great job. I used this code training with my own data. I tried to train with more than labels. I have no idea how to prepare training data, so, can you post any example of data? such like colored images and ground truth images? Thanks again~~~
Hi,When i run the predict,py,it has the following error."RuntimeError: Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 319, 319] to have 3 channels, but got 4 channels instead"
Can you tell me the solution? Thank you very much.
What is the demand for the size and type of the input image?
I found your code can only run on Python3. When I run on Python2, some thing strange happened...
[root@iZuf67ye1557dce2pbylevZ Pytorch-UNet-master]# python2 train.py -g
Starting training:
Epochs: 5
Batch size: 10
Learning rate: 0.1
Training size: 131639
Validation size: 6928
Checkpoints: True
CUDA: True
Starting epoch 1/5.
Killed
Can you extend the compatibility of the code on Python2
Really thank you for your help !!
Hi, thanks for your code, but I have some problems when runing the code.
My dataset has 19k images. When I use the function "to_cropped_imgs" to yield the list of (img, mask), it seems my computer's memory runs out. I think the generation of images list uses a lot of memory. Is anyone has some ways to prevent load so large a list to the memory, so that I can train the model with a great deal of images?
Hi,
I have a training set consisting of greyscale X-ray images where I am trying to find some defects on them. How do I make this implementation work with greyscale images?
Since we are using BatchNorm2d in our U-net Model. Should we switch the network model to evaluation mode when we run net_eval()? Currently, it seems that this code does nothing about this. Should we add net.train() before training and net.eval() before evaluation?
Hello, I’m doing a project about semantic segmentation with 2 class using your code, but my data was unbalanced, so I think maybe use CrossEntropyLoss with weight (just flat the output and label) is a good idea for my project, so changed the final layer from 1 channel to 2 channels,but the loss was unchanged during trainning, I’m so confused ,I give the label 0 or 1 to the mask, I don’t know where it is wrong. Thanks!
criterion = torch.nn.BCELoss() loss = criterion(masks_pred,true_masks)
imgs_size:torch.Size([2, 3, 256, 256])
masks_size:torch.Size([2, 3, 256, 256])
C:\Users\yp\Anaconda3\lib\site-packages\torch\nn\modules\loss.py:512: UserWarning: Using a target size (torch.Size([2, 3, 256, 256])) that is different to the input size (torch.Size([2, 1, 256, 256])) is deprecated. Please ensure they have the same size.
return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction)
Traceback (most recent call last):
File "main.py", line 120, in
train()
File "main.py", line 109, in train
train_model(model,criterion, optimizer, scheduler, train_dataload, valid_dataload,num_epochs)
File "main.py", line 53, in train_model
loss = criterion(masks_pred,true_masks)
File "C:\Users\yp\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "C:\Users\yp\Anaconda3\lib\site-packages\torch\nn\modules\loss.py", line 512, in forward
return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction)
File "C:\Users\yp\Anaconda3\lib\site-packages\torch\nn\functional.py", line 2106, in binary_cross_entropy
"!= input nelement ({})".format(target.numel(), input.numel()))
ValueError: Target and input must have the same number of elements. target nelement (393216) != input nelement (131072)
input size equal image size ? if so , why is target size different to the input size?
when I run the train.py I got this error at imgs = np.array([i[0] for i in b]).astype(np.float32)
(train.py line 65), by the way, my training images are not share the same width and height size.
would you please guide me how to generate MODEL.pth file
Hi!
Your work is great!I am intersted very much.Could you tell me something about your dataset preprocessing?My problems are as follows:
1.Do the dataset images have different sizes?How do you do preprocessing if it does?
Thanks.
Hi,
first of all thanks a lot for open sourcing this repo.
There is an error in the dice loss computation in dice_loss.py.
Your implementation :
self.inter = torch.dot(input.view(-1), target.view(-1)) + 0.0001
self.union = torch.sum(input) + torch.sum(target) + 0.0001
t = 2 * self.inter.float() / self.union.float()
The dice loss is defined by: 2 * intersection / union
, i.e., (2 * dot(A,B)) / (dot(A,A) + dot(B,B))
. A small number ɛ = 0.0001 is added to the intersection and union term in the implementation to avoid division by zero.
While the epsilon is important, it is wrongly placed. The correct version should be to first compute the intersection including the times two factor and then adding the epsilon: (2 * dot(A,B) + ɛ) / (dot(A,A) + dot(B,B)) + ɛ)
.
Usually, this is just a minor flaw that incorrectly computes the dice by the amount of 1*epsilon because the epsilon in the nominator is multiplied by 2. However, for an empty label and prediction map, e.g., in case of correctly deciding that nothing should be segmented in the image, the current form evaluates to 2, which is obviously not defined for the dice score.
To compare, for dot(A,B) = 0
:
Current implementation:
2 * (0 + ɛ)/(0 + 0 + ɛ) = 2 * ɛ/ɛ = 2 * 1
Correct:
((2 * 0) + ɛ)/((0 + 0) + ɛ) = ɛ/ɛ = 1
Best Regards and Thanks again!
Line 22 in 0f45521
Thanks for your work, but i have some questions. why the out of self.up1 is 256 but 512 , and the out of self.down4 is 512 but 1024? Why the in_ch is four times out_ch in self.up?
Hi milesial! Thanks for your great works. I learned much with it.
When I want to train with my own data, i found the image size is really huge (4288*2848) so i resize the image as you mentioned in README. But when I used bilinear upsample instead of transposed convolution, the GPU memory using didn`t change at all(about 5GB). Do u have any suggestions about it?
Thank you.
Hi,
Thanks for your effort first of all.
There is a computation problem when evaluating with function eval_net().
In the first epoch, the dice_val is correct I think like this:
Starting epoch 1/100.
tensor(1.00000e-02 *
8.4271, device='cuda:0')
tensor(0.3177, device='cuda:0')
tensor(0.1262, device='cuda:0')
tensor(0.5119, device='cuda:0')
tensor(0.1107, device='cuda:0')
tensor(0.1193, device='cuda:0')
tensor(1.00000e-02 *
9.0274, device='cuda:0')
tensor(1.00000e-02 *
7.9401, device='cuda:0')
tensor(1.00000e-02 *
9.0199, device='cuda:0')
tensor(0.1371, device='cuda:0')
tensor(0.1182, device='cuda:0')
tensor(0.4859, device='cuda:0')
tensor(1.00000e-02 *
8.5216, device='cuda:0')
tensor(0.2082, device='cuda:0')
tensor(1.00000e-02 *
7.0810, device='cuda:0')
tensor(0.1223, device='cuda:0')
tensor(0.1023, device='cuda:0')
tensor(1.00000e-02 *
8.5076, device='cuda:0')
tensor(1.00000e-02 *
9.6932, device='cuda:0')
tensor(0.1094, device='cuda:0')
tensor(1.00000e-02 *
8.7262, device='cuda:0')
tensor(0.1857, device='cuda:0')
tensor(0.1320, device='cuda:0')
tensor(1.00000e-02 *
8.2069, device='cuda:0')
tensor(0.1559, device='cuda:0')
tensor(0.6490, device='cuda:0')
tensor(0.4299, device='cuda:0')
tensor(0.2339, device='cuda:0')
Validation Dice Coeff: 0.182405274361372
but in the next epochs, the dice_val values are greater than 1, seeming not correct:
Starting epoch 2/100.
tensor(1.9652, device='cuda:0')
tensor(1.9619, device='cuda:0')
tensor(1.9684, device='cuda:0')
tensor(1.9776, device='cuda:0')
tensor(1.9677, device='cuda:0')
tensor(1.9536, device='cuda:0')
tensor(1.9535, device='cuda:0')
tensor(1.9511, device='cuda:0')
tensor(1.9650, device='cuda:0')
tensor(1.9740, device='cuda:0')
tensor(1.9599, device='cuda:0')
tensor(1.9791, device='cuda:0')
tensor(1.9591, device='cuda:0')
tensor(1.9796, device='cuda:0')
tensor(1.9708, device='cuda:0')
tensor(1.9666, device='cuda:0')
tensor(1.9587, device='cuda:0')
tensor(1.9612, device='cuda:0')
tensor(1.9631, device='cuda:0')
tensor(1.9644, device='cuda:0')
tensor(1.9655, device='cuda:0')
tensor(1.9709, device='cuda:0')
tensor(1.9672, device='cuda:0')
tensor(1.9603, device='cuda:0')
tensor(1.9744, device='cuda:0')
tensor(1.9513, device='cuda:0')
tensor(1.9824, device='cuda:0')
tensor(1.9781, device='cuda:0')
Validation Dice Coeff: 1.9660872902188982
I am a newcomer with Pytorch and I cannot fix this problem.
Do you have any idea about this problem?
Thank you!
when i run with my own dataset predicted image i white
Thank you for sharing your code, but I wanted to retrain my data set to make a mistake and run the train.py.
Traceback (most recent call last):
File "train.py", line 129, in
gpu=options.gpu)
File "train.py", line 58, in train_net
val_dice = eval_net(net, val, gpu)
File "/media/Linux/sun/Pytorch-UNet-master/eval.py", line 27, in eval_net
y_pred = net(X)
File "/home/zbw/anaconda2/envs/sun/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "/media/Linux/sun/Pytorch-UNet-master/unet/unet_model.py", line 32, in forward
x = self.up1(x5, x4)
File "/home/zbw/anaconda2/envs/sun/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "/media/Linux/sun/Pytorch-UNet-master/unet/unet_parts.py", line 72, in forward
x = torch.cat([x2, x1], dim=1)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 92 and 91 in dimension 2 at /pytorch/torch/lib/THC/generic/THCTensorMath.cu:111
I hope to get your reply soon.thanks
In the process of training, you don't train the parameter of CRF? Can we directly use the Built - in parameters of dense_crf?
I am trying to use this to train on my own dataset. I couldn't figure out a way to access the data on kaggle without having to download the 24Gb zips. How are the GIF files formed?
In my data I currently have maks with class id's coded into them. I have two classes so my mask is a single channel image with 1's and 0's. But from the error I get when I start the training, it seems that I need a multichannel image with a channel for each class, can you confirm that?
EDIT:
I just set n_classes to 1 instead of 2 and it worked. Can you clarify why for classes 0 and 1 n_classes has to be set to 1?
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 120 and 118 in dimension 2 at /opt/conda/conda-bld/pytorch_1535488076166/work/aten/src/THC/generic/THCTensorMath.cu:87
Thanks for the great repository!
We have a project that partially uses a modification of your u-net model. We're wondering if you'd mind if we release that code. Of course we'll be sure to make it clear that part of that uses a modification of your code.
If you agree, could you please also let me know if you have a preferred way for us to acknowledge your contribution? (e.g. citing your real name and affiliation rather than citing "milesial". ). Thanks! (My email is [email protected] if you'd like to discuss about this through email. )
when I use the pre-trained model to predict as image ,I meet the follow error, the shape of input image is 1920x1080x3
Traceback (most recent call last):
File "predict.py", line 146, in <module>
use_gpu=not args.cpu)
File "predict.py", line 47, in predict_img
left_probs = F.upsample(left_probs, size=(img_height, img_height))
File "/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py", line 1755, in upsample
return torch._C._nn.upsample_nearest2d(input, _scale_factor(2))
File "/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py", line 1727, in _scale_factor
'x'.join(map(str, input.size()))))
RuntimeError: output size specified in UpsamplingNearest (1080x1080) has to be divisible by the input size, but got: 1x1x528x528
Dear Milesial,
I am wondering do you know where has been the crf model learned? I believe this crf python package doesn't include any training method, when I am using it on my segmentation problem it isn't improving the accuracy(instead sometimes it worse the output), could you share your experience of using this pydencrf package?
Best and thank you for your help in advance!
once i run the experiment.
i got 1st epoch accuracy and got this error
cuda runtime error (2) : out of memory at /opt/conda/conda-bld/pytorch_1518244421288/work/torch/lib/THC/generic/THCStorage.cu:58
according to your post, we should turn off all graphical display, how?
Thank you.
Running into an issue when training on my own data set.
I cloned the repo, added folders data/train and data/train_masks,
only part of the code I changed was in load.py
def get_full_img_and_mask(id, dir_img, dir_mask): im = Image.open(dir_img + id + '.jpg') mask = Image.open(dir_mask + id + '_mask.gif') return np.array(im), np.array(mask)
I changed to
def get_full_img_and_mask(id, dir_img, dir_mask): im = Image.open(dir_img + id + '.png') mask = Image.open(dir_mask + id + '.png') return np.array(im), np.array(mask)
because the file names are in png format and the corresponding files are the same name but in a different directory.
here is the error:
Starting epoch 1/5.
Traceback (most recent call last):
File "train.py", line 139, in
img_scale=args.scale)
File "train.py", line 64, in train_net
for i, b in enumerate(batch(train, batch_size)):
File "/home/users/rssadre/RAPIDS-IDREAM/Pytorch-UNet/utils/utils.py", line 37, in batch
for i, t in enumerate(iterable):
File "/home/users/rssadre/RAPIDS-IDREAM/Pytorch-UNet/utils/utils.py", line 17, in hwc_to_chw
return np.transpose(img, axes=[2, 0, 1])
File "/home/users/rssadre/anaconda3/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 639, in transpose
return _wrapfunc(a, 'transpose', axes)
File "/home/users/rssadre/anaconda3/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 56, in _wrapfunc
return getattr(obj, method)(*args, **kwds)
ValueError: axes don't match array
Hi, thanks for your code, it works well on my own dataset.
However, It seems like this project can only support the binary classification problem.
when I try to modify as following:
train.py -124 -> n_classes = 3
this function: loss = criterion(masks_probs_flat, true_masks_flat) ---from train.py 81
goes wrong because the shape of two parameters dismatch.
My mask image is a single 8-bit image with 4 different intensities (0,1,2,3) represent 4 classes(background,class1,class2,class3)
how to modify the code so that it can support a multi-class problem.
thanks advanced
Hello. I am trying to use parts of your code with my data set and whenever i evaluate a model i get values of this sort:
Epoch finished ! Loss: 0.16545738393068313
Validation Dice Coeff: 1.004147481918335
Checkpoint 1 saved !
Is this normal? Shouldn't dice coefficient be a value between 0 and 1? To be mentioned that my masks are not binary but i load them as binary so the validation set should contain binary images. Do you have any idea why this happens ?
Thank you a lot for your code and for sharing it. :)
Hello,
I found this wonderful code and wanted to test on my models:
There was line in your warnings that:
outputs = outputs.permute(2, 3, 0, 1).contiguous().view(-1, 2)
should be added. however, then I get: ValueError: Expected input batch_size (1605632) to match target batch_size (32).
my train size is 1600 images total. and batch is 32, numwork is 2.
Besides, when I removed it, then I get:
invalid argument 3: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1
my length of 4d array is: 32,2,224,224
or the mirrored (no sure)
thanks and best regards,
In http://pytorch.org/docs/0.3.0/nn.html?highlight=pad#torch.nn.functional.pad , "4D input tensor pad should be in form (pad_l, pad_r, pad_t, pad_b )".
but in your code,
Pytorch-UNet/unet/unet_parts.py
Line 68 in 2c17549
when using the class up,it will be wrong when using different image size.If the size of x5 is 21 and the size of x4 is 41 , we will get 42 when using the class up.So the x5'size 42 is not equal to the x4'size 41.
Hi, I use the default learning rate which is 0.1 to train the U-net with my private dataset, but the resulted model is bad. I am wondering if the learning rate is too large. So, what the learning rate should be? Anyone know?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.