minar09 / cp-vton-plus Goto Github PK

I wanted to know how did you calculate Warped Blended (IoU), SSIM, LPIPS, IS metrics for quantitative evaluation and where exactly did you implement in the code for quantitative evaluation?

Hi,

I specified gpus_id = '0,1', but model just run on first device. I checked the code and couldn't find the "DataParallel" to distribute the model to multiple devices. Please let me know if I am missing something

Hi, Great work, and thanks for sharing. I wanted to test your model, are there any pretrained weights available?

got this error with the line: python train.py --name GMM --stage GMM --workers 4 --save_count 5000 --shuffle

and got this with: python train.py --name TOM --stage TOM --workers 4 --save_count 5000 --shuffle

can you guide me, on how to solve these error
downloaded the data, model and did according to the read me

I used the pretrained model and resulting images have a black face and I am getting unexpected result.

Can you please help

I encountered the same problem as shown above on both MacOS and Linux. I also tried downloading the dataset manually from the google drive but I still couldn't untar it.

I am using custom dataset.
More details
Batch_size =1
Segmentation Mask from CHIP_PGN results.
Pose from openpose Coco Model and changed the code that described here
Cloth and cloth mask are from the preprocessed data

When I run dataset_neck_skin_correction.py on the train data I'm getting the following error.

Is it due to image parsing since I'm using https://github.com/Engineering-Course/LIP_JPPNet for parsing ?

Does this model work on cpu version while testing/inference.
if yes,
please tell me changes I would have to make.

Thanks

When I run body_binary_masking.py on the test set some masks seem to consist of two different images. However, many images also appear normal. Is this a known issue?

See below for an example:

Thanks for a great implementation otherwise!

Hi Matiur, @minar09
I tried to run the cp-vton-plus on Google colab but it gave weird result images in the folder "/cp-vton-plus/result/TOM/test/try-on" see below images. Is this the correct folder of the try on results? if not, where is the correct folder to look for try-on look?
can you please help me . I am still beginner in this.

Colab implementation

Result images:

Sample Images of /result/GMM/test/overlayed_TPS :

Sample Images of /result/GMM/test/result_dir :

Sample Images of /result/GMM/test/warp-cloth :

Thanks,

Lgic = gicloss(grid)
# 200x200 = 40.000 * 0.001
Lgic = Lgic / (grid.shape[0] * grid.shape[1] * grid.shape[2])
loss = Lwarp + 40 * Lgic    # total GMM loss

why loss_gic and 200x200=40x0.001 ?

I have been following the tutorial and I am unable to train the algorithm. I have downloaded the dataset via data_download.py. What should I do?

Thank for the work.
I have run the modules according to the read me, but most of results after the TOM stage ('try-on' generated folder) contains significant artifacts. Just attaching here first 5 generated photos. Some are ok, but I would say that most are bad. Don't understand is this a some bug, problem with my actions or current limitation of the approach.

I have read the code carefully, the gic loss calculates the difference of flow value at each grid coordinate, on x and y directions relatively.

And the tv loss seems do the same thing.
https://discuss.pytorch.org/t/implement-total-variation-loss-in-pytorch/55574

Could you please explain the difference between your gic loss and tv loss?

@minar09 what should I do to use this model for slightly high resolution images?

Hi there. I wanted to use your code to create a binary mask for a white cloth on a white background image; however, I only got an absolutely black mask image and I couldn't see the cloth at all. Could you advise how I can create a binary mask for a white cloth on a white background image? Thank you so much!

I really appreciate your work.

i am getting an error like this while testing custom images,

return F.conv2d(input, weight, self.bias, self.stride,
RuntimeError: Given groups=1, weight of size [64, 22, 4, 4], expected input[3, 28, 256, 192] to have 22 channels, but got 28 channels instead

Hi, I am getting this keyword error. I printed dictionary keys and there is no shape_ori key in it.
dict_keys(['c_name', 'im_name', 'cloth', 'cloth_mask', 'image', 'agnostic', 'parse_cloth', 'shape', 'head', 'pose_image', 'grid_image'])

i run pretrained tom weiths and got this all time

RuntimeError: Given groups=1, weight of size [64, 22, 4, 4], expected input[1, 29, 256, 192] to have 22 channels, but got 29 channels instead

I am facing this error when running it for 1 input person and 1 cloth(1 pair), but it worked correctly for the dataset...
what is the reason..
pls help me out as soon as possible..

Hi,

When I tested this model with a long-sleeve shirt to fit a person who was wearing a short-sleeve shirt, the long-sleeve shirt turned into a short-sleeve shirt, and vice versa. Any advice on how to solve this short and long sleeve issue so that the long-sleeve shirt will remain long-sleeve after fitting? Many thanks,

Got error FileNotFoundError: [Errno 2] No such file or directory: 'data/viton_resize/train/image'

I am testing CP-VTON+ on my custom images. I have created masks and parsed images for both cloth and models using some human-body parsing repos like CIHP, JPP-Net but couldn't understand the use of "image-parse-new" folder in test and train directories of VITON dataset. How to create these images inside "image-parse-new" folder. Nothing was mentioned about this in the README file.

I tried to disable the cuda, but the file test.py didn't run. Is there a way to run an inference with a cpu?

File "test.py", line 224, in
main()
File "test.py", line 210, in main
test_gmm(opt, test_loader, model, board)
File "test.py", line 83, in test_gmm
for step, inputs in enumerate(test_loader.data_loader):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 345, in next
data = self._next_data()
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 856, in _next_data
return self._process_data(data)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 881, in _process_data
data.reraise()
File "/usr/local/lib/python3.6/dist-packages/torch/_utils.py", line 395, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/content/drive/My Drive/Colab Notebooks/VirtualTryOn/cp-vton-plus/cp_dataset.py", line 140, in getitem
shape_ori = self.transform(parse_shape_ori) # [-1,1]
File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/transforms.py", line 61, in call
img = t(img)
File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/transforms.py", line 166, in call
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/functional.py", line 208, in normalize
tensor.sub(mean).div(std)
RuntimeError: output with shape [1, 256, 192] doesn't match the broadcast shape [3, 256, 192]

Hello, is it possible to have the code that you used to generate the pose keypoints? I try to generate the keypoints of some image of the cp-vton+ with the code on this github repository: https://github.com/minar09/openpose-pytorch. However, I do not find the same points as the ones in the cp-viton+ dataset.

I'm going to try cp-vton-plus with custom images

In dataset_neck_skin_correction.py, only custom images get the following error

Traceback (most recent call last):
File "dataset_neck_skin_correction.py", line 258, in
main()
File "dataset_neck_skin_correction.py", line 254, in main
image_dir, seg_dir, each[0], mask, updated_seg_dir)# def update_image_segmentation(data_dir, mask_dir, image_name, mask_name, save_dir=None, save_vis=True):
File "dataset_neck_skin_correction.py", line 184, in update_image_segmentation
result = segmentation + neck_mask
ValueError: operands could not be broadcast together with shapes (256,192,3) (256,192)

The dataset image succeeds, the custom image gives me this error. Do you know the cause?

I get the following error.
I would like to know what the solution is.

This is great work!

How do I get rid of the blue hue and blurriness?

I am trying to give inputs from internet like take an image of a person and an image of the cloth..
I tried running the dataset_neck_skin_connection.py
How should I get the image-parse for the input image.. pls guide me.. I am a beginner

why use l1_loss(warped_cloth,c),different from paper said(warped_mask,cm)?

hi,
can you please tell me why is the grid size 5 and what does 5 indicate ?
also, in FeatureRegression the output_dim in 2*opt.grid_size**2 where as generally it should be 6 right because of (2,3 ) parameters in tps transform.
could you please elaborate on grid size and output_dim in FeatureRegression
thank you

When executing the test.py script for TOM model, FileNotFoundError appears.
Line number 56 inside getitem method of cp_dataset.py is causing the error because there is no directory present as such named data/test/warp-cloth.

I found that the pictures I processed at this stage can't be put into the previous CP VTON. Can you help me? The prompt error is"RuntimeError :The size of tensor a (192) must match the size of tensor b(3) at non-singleton dimension 2"

Hey, I am trying to execute the test.py for GMM as mentioned in the repo it is giving the size mismatch error.

RuntimeError: Error(s) in loading state_dict for GMM:
size mismatch for extractionB.model.0.weight: copying a param of torch.Size([64, 3, 4, 4]) from checkpoint, where the shape is torch.Size([64, 1, 4, 4]) in current model.

Kindly help me with this.

There seems to be artifacts when running TOM testing. Does this suggest that the input to the TOM is incorrectly given or are you seeing the same results as below? The input is fixed as reported here: #8 FYI.

I have a question about equation (3) in the paper.

What are Gx and Gy?

Also, I don't see Gy in the equation, is that correct?

Hi. If I would like to train the model with images at different size (e.g. 288 x 512, which is 1.5 times the current image size (192 x 256)), can I do so? If so, do I need to change any data in the model?

Thanks for your help,

In your slide , page 11, cp-vton result and your result have difference in shoulders of the cloth, but the input of upper region is the same, why does it happen?

I have been trying solve it but didnt reach anywhere. The output shape is not able to match the broadcast shape

python train.py --name GMM --stage GMM --workers 4 --save_count 5000 --shuffle

Namespace(batch_size=4, checkpoint='', checkpoint_dir='checkpoints', data_list='train_pairs.txt', datamode='train', dataroot='data', decay_step=100000, display_count=20, fine_height=256, fine_width=192, gpu_ids='', grid_size=5, keep_step=100000, lr=0.0001, name='GMM', radius=5, save_count=5000, shuffle=True, stage='GMM', tensorboard_dir='tensorboard', workers=4)
Start to train stage: GMM, named: GMM!
initialization method [normal]
initialization method [normal]
Traceback (most recent call last):
File "train.py", line 229, in
main()
File "train.py", line 210, in main
train_gmm(opt, train_loader, model, board)
File "train.py", line 72, in train_gmm
inputs = train_loader.next_batch()
File "/content/gdrive/My Drive/virtual mirror/cp_dataset.py", line 228, in next_batch
batch = self.data_iter.next()
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 345, in next
data = self._next_data()
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 856, in _next_data
return self._process_data(data)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 881, in _process_data
data.reraise()
File "/usr/local/lib/python3.6/dist-packages/torch/_utils.py", line 395, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/content/gdrive/My Drive/virtual mirror/cp_dataset.py", line 138, in getitem
shape_ori = self.transform(parse_shape_ori) # [-1,1]
File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/transforms.py", line 61, in call
img = t(img)
File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/transforms.py", line 166, in call
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "/usr/local/lib/python3.6/dist-packages/torchvision/transforms/functional.py", line 208, in normalize
tensor.sub(mean).div(std)
RuntimeError: output with shape [1, 256, 192] doesn't match the broadcast shape [3, 256, 192]

Hey,

Would it be possible for a colab demo or an end to end integrated system run(Single File Run)

Regards

Hi, there is a way to restart the training from a specific checkpoint?
I'm trying this command but it starts always from step 0
python train.py
--stage GMM
--workers 8
--save_count 500
--checkpoint "checkpoints/gmm_train_new/step_000500.pth"

testing gmm network gave this error

python3 test.py --name GMM --stage GMM --workers 4 --datamode test --data_list test_pairs.txt --checkpoint checkpoints/GMM/gmm_final.pth

Namespace(batch_size=4, checkpoint='checkpoints/GMM/gmm_final.pth', data_list='test_pairs.txt', datamode='test', dataroot='data', display_count=1, fine_height=256, fine_width=192, gpu_ids='', grid_size=5, name='GMM', radius=5, result_dir='result', shuffle=False, stage='GMM', tensorboard_dir='tensorboard', workers=4)
Start to test stage: GMM, named: GMM!
initialization method [normal]
initialization method [normal]
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=663 error=11 : invalid argument
Traceback (most recent call last):
File "test.py", line 225, in
main()
File "test.py", line 211, in main
test_gmm(opt, test_loader, model, board)
File "test.py", line 99, in test_gmm
grid, theta = model(agnostic, cm)
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/Desktop/cpvtonplus/cp-vton-plus-master/networks.py", line 522, in forward
correlation = self.correlation(featureA, featureB)
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/Desktop/cpvtonplus/cp-vton-plus-master/networks.py", line 103, in forward
feature_mul = torch.bmm(feature_B, feature_A)
RuntimeError: cublas runtime error : the GPU program failed to execute at /pytorch/aten/src/THC/THCBlas.cu:411

In the make_body_mask function in body_binary_masking.py, I see this line of code

54:  _, seg_mask = cv2.threshold(gray, 1, 255, cv2.THRESH_BINARY)

Shouldn't the first second argument be 0 not 1? Won't the hat segmentation part be lost if 1 is set as the threshold?