mvig-sjtu / wshp

Code for CVPR'18 spotlight "Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer"

Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer

[arXiv]

Transferring human body part parsing labels to raw images by exploiting the anatomical similarity. Some transferred results:

These results are used as extra training samples for the parsing network and can improve the part segmentation results:

Getting Started

Demo video

Check out our demo video here.

Parsing Network

Checkout parsing_network for training\testing\demo code of our parsing network.

Data generation

Checkout data_generation for code of using keypoints similarity to transfer parsing knowledge and generate synthetic training labels.

Feedback

If you get any problems during usage, please open an issue.

Citation

If you use this code for your research, please cite our paper:

@article{fang2018wshp,
  title={Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer},
  author={Fang, Hao-Shu and Lu, Guansong and Fang, Xiaolin and Xie, Jianwen and Tai, Yu-Wing and Lu, Cewu},
  journal={CVPR},
  year={2018}
}

wshp's Issues

How to perform parsing on my own image?

How to perform parsing on my own image to output an image inside a terminal, instead of save it in another files?

hello ,Is there a pre-trained model?

Download link for people outside Mainland China

Hi, i need dataset download link for people outside Mainland China, because I can't login Baidu with my foreign phone number.

Server to download the pretrained weights seems to be down.

https://jbox.sjtu.edu.cn/link/view/80801f89e5a84482842ba6f483584087: download button does not redirect
http://mvig.sjtu.edu.cn/publications/wshp/models.zip: website cannot be reached

inference.py ERROR

No such file or directoryr traceback): dataset/my_test/20180521225238.jpg
[[Node: create_inputs/ReadFile = ReadFile_device="/job:localhost/replica:0/task:0/device:CPU:0"]]

I sure it is exits
donghaoye@amax1:/data1/donghaoye/workspace/WSHP/parsing_network$ ls dataset/my_test/20180521225238.jpg

Lower Level Part Map

Hi there,

Is it possible to get lower level part map where I could detect feet? Because returned predictions are higher level part maps.

Thanks

basestring not defined

As title, when running the code for inferencing, I got an error
\WSHP\parsing_network\kaffe\tensorflow\network.py", line 78, in feed if isinstance(fed_layer, basestring): NameError: name 'basestring' is not defined

I take a quick look at the code, and I indeed did not find any reference to the string from elsewhere. What is this error?

Also can I run the model without specifying the mask in the data_list

How to run real-time-inference.py?

dear,
can you show me how to test video or camera?
Thank you

Run real time video

dear author,
I ran in video and I gained 1 fps, It is so low.
Do you have any idea to lift the performance, such as fps and mIoU?

module 'tensorflow' has no attribute 'ConfigPro'

您好：
我在tensorflow版本为1.5.1、1.9.0和1.12都运行了，都遇到了同样如下的错误
module 'tensorflow' has no attribute 'ConfigPro'
我在下面这个连接中找到了该模块，但是不知道怎么用起来
https://github.com/tensorflow/tensorflow/blob/r1.12/tensorflow/core/protobuf/config.proto
渴望您的回复，谢谢！

FileNotFoundError: [Errno 2] No such file or directory: '/path/to/data/list'

When I run inference.py . The error as follows:

FileNotFoundError: [Errno 2] No such file or directory: '/path/to/data/list'

what is the link to the pretrained model thanks!

why the demo data is not origin_images??? bash demo.sh is not work

Where is the pre-trained model for refinement_network?too

I can not find it. I only find a models(699M）.
But I think it's not suitable for the refinement network.
Could you tell me how can get it?
Thanks a lot.It's really a good work.

inference.py运行报错

你们好，我想请教一个问题如下：

我的路径目录如下：
dataset
dance.txt
deeplab_resnet
kaffe
misc
models
output
VIPeR
cam_a/000_45.bmp
inference.py

我的dance.txt里面是图片的路径：/home/zyk/桌面/WSHP-master/parsing_network/VIPeR/cam_a/000_45.bmp

我在运行inference.py的时候，终端输入的是：
python inference.py /home/zyk/桌面/WSHP-master/parsing_network/VIPeR/cam_a /home/zyk/桌面/WSHP-master/parsing_network/models

但是运行程序的时候总是报错如下：

2018-07-03 21:13:56.618526: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open /home/zyk/桌面/WSHP-master/parsing_network/models: Failed precondition: /home/zyk/桌面/WSHP-master/parsing_network/models; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator?

2018-07-03 21:13:56.620048: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open /home/zyk/桌面/WSHP-master/parsing_network/models: Failed precondition: /home/zyk/桌面/WSHP-master/parsing_network/models; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator?
2018-07-03 21:13:56.620080: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at save_restore_tensor.cc:170 : Data loss: Unable to open table file /home/zyk/桌面/WSHP-master/parsing_network/models: Failed precondition: /home/zyk/桌面/WSHP-master/parsing_network/models; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator?
Traceback (most recent call last):
File "inference.py", line 136, in
main()
File "inference.py", line 110, in main
load(loader, sess, args.model_weights)
File "inference.py", line 56, in load
saver.restore(sess, ckpt_path)
File "/home/zyk/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1802, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/zyk/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/home/zyk/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/home/zyk/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/home/zyk/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.DataLossError: <exception str() failed>

请问我是什么地方有问题呢？非常期待您的回答，谢谢

That repo support code for GPU?

dear,
Your repo can support for running gpu?
thank you!

tensorflow.python.framework.errors_impl.DataLossError: <exception str() failed>

zyk@zyk:~/下载/WSHP-master/parsing_network$ python inference.py /home/zyk/下载/WSHP-master/parsing_network/img/ /home/zyk/下载/WSHP-master/parsing_network/models/final_model/model.ckpt-19315.data-00000-of-00001
WARNING:tensorflow:From inference.py:85: calling expand_dims (from tensorflow.python.ops.array_ops) with dim is deprecated and will be removed in a future version.
Instructions for updating:
Use the axis argument instead
WARNING:tensorflow:From inference.py:96: calling argmax (from tensorflow.python.ops.math_ops) with dimension is deprecated and will be removed in a future version.
Instructions for updating:
Use the axis argument instead
2018-09-05 13:23:32.405096: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open /home/zyk/下载/WSHP-master/parsing_network/models/final_model/model.ckpt-19315.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
2018-09-05 13:23:32.405500: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open /home/zyk/下载/WSHP-master/parsing_network/models/final_model/model.ckpt-19315.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
2018-09-05 13:23:32.405530: W tensorflow/core/framework/op_kernel.cc:1275] OP_REQUIRES failed at save_restore_tensor.cc:170 : Data loss: Unable to open table file /home/zyk/下载/WSHP-master/parsing_network/models/final_model/model.ckpt-19315.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
Traceback (most recent call last):
File "inference.py", line 136, in
main()
File "inference.py", line 110, in main
load(loader, sess, args.model_weights)
File "inference.py", line 56, in load
saver.restore(sess, ckpt_path)
File "/home/zyk/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1725, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/zyk/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 877, in run
run_metadata_ptr)
File "/home/zyk/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1100, in _run
feed_dict_tensor, options, run_metadata)
File "/home/zyk/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1272, in _do_run
run_metadata)
File "/home/zyk/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1291, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.DataLossError: <exception str() failed>

Horse-cow dataset

Can you give me the horse-cow dataset? I want to use it to test my model, but I haven't found it. So I try to ask you. I will look forward to your reply. Thank you very much.

Realtime issue

Hi,
Can I run this code for realtime video from webcam?
Thank you :)

how to test on the image dataset

hi @Fang-Haoshu @jmydurant @noumanriazkhan @luguansong

i want to test on my own image dataset ,what should i do? thanks

Why did you 'Align torso length to the same(50)' ?

As shown in your 'generate_prior_util.py', I don't known why you process the pose with function aligh_torso.

Where is the pre-trained model for refinement_network?

Where is the pre-trained model for refinement_network?
Or say "Where is the pre-trained model for your work?"

'./dataset/dance.txt' can not be found.

hi,
I am very thanks to your contributions. But I have a problem to repetite your wrok. I can not find the file 'dance.txt' , can you help me?
thanks.
best wishs.

你分享的预训练模型已经实效了，可否重新分享一次，非常感谢。

ValueError: operands could not be broadcast together with shapes (88,23) (88,24)

s1@s1:~/Downloads/WSHP/data_generation$ bash demo.sh
length of json_dict 2
length of pascal_img 644
bbox [308, 272, 331, 406]
0 picked pascal 2010_003117_0
Traceback (most recent call last):
File "crop_pose_and_generate_testing_prior.py", line 97, in
prior = generate_prior_single_person(bbox, raw_pose, opt.PASCALMaskImgDir, pascal_poses, pascal_img_names, pascal_pose_dict, opt.n, opt.k)
File "/home/s1/Downloads/WSHP/data_generation/generate_prior_util.py", line 444, in generate_prior_single_person
morphingImg = morphing(pascal_mask_img, pascal_pose, pose, origin_size)
File "/home/s1/Downloads/WSHP/data_generation/generate_prior_util.py", line 351, in morphing
origin_body_part[origin_row_low:origin_row_high, origin_col_low:origin_col_high])
ValueError: operands could not be broadcast together with shapes (88,23) (88,24)

Module 'data_generation' puzzling me. Does it prepare the dataset for 'parsing_network'?

What the data_list stand for?

Hi, I've downloaded the pretrained model and I was trying to segment some images. The problem is that in the repositoryt isn't any data-list example. In the codebase there is a reference to dataset/dance.txt but I didn't find it anywhere.
Where can I find this data list fileor How can I generate it?

TypeError: unsupported operand type(s) for /: 'Dimension' and 'int'

dear,
I had the errors above.
What version did you use python, tensorflow? Tensorflow =1.1?

Error loading pretrained weigths

I have follow error when I try to load pretrained networks in inference.py:

DataLossError (see above for traceback): Unable to open table file D:\Progetti\Tensorflow\GAN\WSHP\parsing_network\models\final_model\checkpoint: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
[[node save/RestoreV2 (defined at D:\Progetti\Tensorflow\GAN\WSHP\parsing_network\inference.py:109) ]]
[[node save/RestoreV2 (defined at D:\Progetti\Tensorflow\GAN\WSHP\parsing_network\inference.py:109) ]]

about pretrained model

Could you tell me where can I download pretrained model? What's meaning of data_list in code "DATA_LIST = './dataset/dance.txt'"? thx.

size not equal in morphing function

File "/Users/zimu/WSHP/data_generation/generate_prior_util.py", line 351, in morphing
origin_body_part[origin_row_low:origin_row_high, origin_col_low:origin_col_high])
ValueError: operands could not be broadcast together with shapes (88,23) (88,24)

how to get the model weights file?

I want to use the model to get the parsing mask as an information to add to my model. So I want to use the pretrained model on my own images.

can you share your trained model?

Cannot find the pretrained model in models/final_model/ ?

I just cannot find the pretrained model since there is not any directory named models/final_model/.