Hi ,
I downloaded the VGG_ILSVRC_16_layers.caffemodel manually ,But I don't know where can I set this model? can you tell me ,thanks ~~~

thanks for your work. i note that the generated masks using the code are different from the the original paper.
the generated masks in the paper are as follows:

the generated masks using the code are as follows:

do you follow the method of the paper or just use your own way for generating masks?

Excuse me, if I have trained 1000 times now and got the npz file, how can I set it up if I want to train from the 1000th time?

I have used free-form mask to generate mask which shape is (batchsize,1,256,256),I have set batchsize to 1,and I use batch_postprocess_images to reshpe the size to(256,256,1),the type is 'numpy.ndarray',The mask is like this:

How could I print the mask?

Looking forward to receiving your reply,Thank you very much.

Hi Seitaro, thanks for your interest and efforts in re-producing our work. I noticed there are some difference with our implementation. I just created a Q&A section here: http://masc.cs.gmu.edu/wiki/partialconv

Thanks a lot.
Guilin Liu

Hi,
I produced my own irregular hole masks (more bigger, like objects).
The result is like this:

the content inner the hole is good ,but there is an obvious line around the outline。

And all the Iout images have 2 or 3 pixels offset from the corresponding Igt image. Maybe this is the reason why the uncoordinated occur.

Thank you.

Excuse me, in the place2.py file, def get_example part, you use random to take out 8 pictures, if I want to take out the order from the first to the eighth, how can I change?

Is it because the external cropping loss of the cropping frame is ignored, so the outermost part of the generated picture is not very effective?How to solve it?

when I run the generate_result.py ,and set up correct, it appear The error Unsupported dtype object! How can I solve this problem?

I had trained my model successfully,and as a result,it eventually generated model500000.npz.
When I print "python generate_result.py -g 0 --load_model result/model500000.npz" on my terminal,an error happened.I had tried many ways to solve this problem but in vain.

The question is listed as below:

python generate_result.py -g 0 --load_model result/model500000.npz

Namespace(batch_size=4, crop_to=256, eval_folder='generated_results', gpu=0, load_dataset='place2_test', load_model='result/model500000.npz', resize_to=256)
loading vgg16 ...
ok
Completion model loaded
use gpu 0

Traceback (most recent call last):
File "generate_result.py", line 114, in
main()
File "generate_result.py", line 73, in main
batch = val_iter.next()
File "/home/fl/.local/lib/python3.6/site-packages/chainer/iterators/serial_iterator.py", line 48, in next
self._previous_epoch_detail = self.epoch_detail
File "/home/fl/.local/lib/python3.6/site-packages/chainer/iterators/serial_iterator.py", line 86, in epoch_detail
return self.epoch + self.current_position / len(self.dataset)
ZeroDivisionError: division by zero

What can I do to solve this problem?I will very appreciate if you could help me,thank you!

Excuse me! I'm new to the chainer framework. In your code, I found the convolution operation with the class PConv. But I can't find how you implement upsampling. Can you point it out for me?

Hi ! Very thank you for your excellent job! I am trying to read the paper and run your code.I tools are

python3.6+chainer 4.2.0 +CUDA8.0

but it give me a problem "CUBLAS_STATUS_NOT_SUPPORTED", I tried find methods, but finde none (crying) could you give me some tips?

These are the problem details

 File "/home/cxd/code/code_python/chainer-partial_convolution_image_inpainting-master/chainer-partial_convolution_image_inpainting-master/updater.py", line 133, in update_core
    L_style = calc_loss_style(fs_I_out,fs_I_comp,fs_I_gt) #Loss style out and comp
  File "/home/cxd/code/code_python/chainer-partial_convolution_image_inpainting-master/chainer-partial_convolution_image_inpainting-master/updater.py", line 48, in calc_loss_style
    hout_gram = F.batch_matmul(hout,hout,transb=True)
  File "/root/anaconda3/lib/python3.6/site-packages/chainer/functions/math/matmul.py", line 276, in batch_matmul
    return BatchMatMul(transa=transa, transb=transb).apply((a, b))[0]
  File "/root/anaconda3/lib/python3.6/site-packages/chainer/function_node.py", line 258, in apply
    outputs = self.forward(in_data)
  File "/root/anaconda3/lib/python3.6/site-packages/chainer/functions/math/matmul.py", line 210, in forward
    return _batch_matmul(a, b, self.transa, self.transb, False),
  File "/root/anaconda3/lib/python3.6/site-packages/chainer/functions/math/matmul.py", line 178, in _batch_matmul
    return _matmul(a, b, transa, transb, transout)
  File "/root/anaconda3/lib/python3.6/site-packages/chainer/functions/math/matmul.py", line 49, in _matmul
    return xp.matmul(a, b)
  File "cupy/core/core.pyx", line 3525, in cupy.core.core.matmul
  File "cupy/core/core.pyx", line 3682, in cupy.core.core.matmul
  File "cupy/cuda/cublas.pyx", line 700, in cupy.cuda.cublas.sgemmStridedBatched
  File "cupy/cuda/cublas.pyx", line 718, in cupy.cuda.cublas.sgemmStridedBatched
  File "cupy/cuda/cublas.pyx", line 267, in cupy.cuda.cublas.check_status

I am sorry to trouble you. I want to know what is the meaning of mask_b (https://github.com/SeitaroShinagawa/chainer-partial_convolution_image_inpainting/blob/master/common/net.py#L62) in your code and where did you pixel-wise multiplicate an input image and mask to simulate a broken image?
And the size of the mask is as the same as the feature values x, which is also the same as the cobvolution sliding windows, why did you say that set the mask window as 1(https://github.com/SeitaroShinagawa/chainer-partial_convolution_image_inpainting/blob/master/common/net.py#L55)?
Besides, I can not understand the unmasking rule?
I am sorry that i have so many questions, hope your responce!

would you give us a model to eval

Hello, I ran into some weird issue. I say weird because running the same command gives me 2 different errors each alternative times.
It says,
ValueError: could not broadcast input array from shape (64,3,8,8) into shape (64,3,7,7)
and
KeyError: 'PConv_16/c/b is not a file in the archive'

Can you please help me with this?

Originally posted by @Prudvi01 in #31 (comment)

Could you please add a license to this project? Thank you.

Hi，
I'm also try to implement this paper by pyTorch. Input 512x512 image (celeA-HQ) and 2 batch_size. But the result have much watermark in the hole edge and low-resolution in generate hole area. Do you have this situation?
Thank you!

In https://github.com/SeitaroShinagawa/chainer-partial_convolution_image_inpainting/blob/master/updater.py line 68, Right now it is :
P = Variable(xp.sign(canvas-0.5)*0.5+1.0) #P region (hole mask: 1 pixel dilated region from hole)

while it should probably should be ( I donot use chainer so not sure about this):
P = Variable(xp.sign(canvas-0.5)+1.0)*0.5 #P region (hole mask: 1 pixel dilated region from hole)

Reason because the canvas would have all the values with dilation 1 from the holes with values>0
So P should be 1 for the values>0 and 0 for values =0

Thanks a lot for your excellent work.I have another question for this code,how can we finetune the network in chainer?The original paper seems to use different learning rate(0.00005) and freeze the Batch Normalization in the encoder part of the network ,and in this way,they reduce the color differenes(it means the L1 loss outside the hole).I would appreciate if you could help me,thank you

I have trained the network using the last updated code but produce unwanted boarder surrounding the hole. I used custom mask. look at the following output mask as [example...](url

)

I appreciate if anyone come up with a solution for this problem. Thank you.

Hi,
It seems there is an issue with the perceptual loss.
In https://github.com/SeitaroShinagawa/chainer-partial_convolution_image_inpainting/blob/master/updater.py#L14, it is :

layers = list(hout_dict.keys())
layer_name =  layers[0]
loss = F.mean_absolute_error(hout_dict[layer_name],hgt_dict[layer_name])
loss += F.mean_absolute_error(hout_dict[layer_name],hgt_dict[layer_name])
for layer_name in layers[1:]: 
    loss += F.mean_absolute_error(hcomp_dict[layer_name],hgt_dict[layer_name])
    loss += F.mean_absolute_error(hcomp_dict[layer_name],hgt_dict[layer_name])

while in actual it should be:

layers = list(hout_dict.keys())
layer_name =  layers[0]
loss = F.mean_absolute_error(hout_dict[layer_name],hgt_dict[layer_name])
loss += F.mean_absolute_error(hcomp_dict[layer_name],hgt_dict[layer_name])
for layer_name in layers[1:]: 
    loss += F.mean_absolute_error(hout_dict[layer_name],hgt_dict[layer_name])
    loss += F.mean_absolute_error(hcomp_dict[layer_name],hgt_dict[layer_name])

I'm kind of a newbie, just getting started into deep learning please forgive my amateur question.

I tried implementing your project but I ran into a few issues where I didn't have the required packages. I quickly realized and installed chainer but still ran into some other issues. I'd be really grateful if someone helped me with the requirements I need to install before running train.py

This is the response I got after running train.py

First of all,given an arbitrary broken picture, how to get its mask area?
Furthermore,how to test on a real broken image?I have used your ways to retrain the model but in vain,the generated results still have the broken part.
Last but not the least,what is the meaing of using VGG model,pretraining or caculating the function loss ?
I will very appreciate if you could help me,thank you!

Hi, according to what you mentioned in #6 . I have to set the pixel value of broken parts to 0, which is the mask area like this:


How can I input them into the trained completion model based on my own dataset and get the complete result?
Also, I found that chainer will save the model file（.npz) every 10 epochs so that the file size is quite large. The training process has to be terminated. Therefore, I want to know how to solve this problem.
Last, could you give me some suggestions of a rational training time?
Thanks for your help and looking forward to your reply.

Are there any differences between x/127.5-1 and x/255?

Excuse me! I'm new to the chainer framework and python. When i train with the command "python train.py", it always occur the error ''divsion by zero', i am confused! Hope your responce

I followed the suggestions in this thread but still ran into the same issue but that issue was closed so had to open a new one. I changed my path and double-checked. I used a small dataset the "Small images (256 * 256) with easy directory structure" 21G one. It only has the train and val data set so I copied the val data and named it train_256.
Can you please help me with the dataset structure?

UPDATE: I changed my dataset paths to a different location and now running this

#to check train data path

import glob
from common import paths
train_keys = glob.glob(paths.train_place2+"//*.jpg")
print(len(train_keys))
#if successful, you will get "1434892" as return

returns 1803460, I assume thats because of the different dataset but that's better than 0 which I used to get before. But the ZeroDivisionError: division by zero error persists. Can you please help?

Originally posted by @Prudvi01 in #7 (comment)

Hello @SeitaroShinagawa I don't have an Nvidia GPU so I'm using CPU by excluding (-g 0) but running the command 'python generete_result.py --load_model model500000.npz' still says shows the following error

update

I tried it on google colab with GPU on and received this error

Originally posted by @Prudvi01 in #2 (comment)

Hi, If i have a real broken image, how to test on your model? That means I don’t have the mask as input.

When I try to train the model, in your train.py the code about setting test/val_dataset and test/val_iter was commented out，but it seem doesn't work when I remove the ’#‘ before the code，could you please tell me how to set the test/val_dataset and test/val_iter

As you mentioned in the other issue, I input the broken image as x, and set an ones vector as the mask, but the result is poor, the result is just as the broken image.
The broken image:
The mask:

I_out:

I_composition:

Hi!When I encountered this problem while training, how can I solve it?