clovaai / ext_portrait_segmentation Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Hi, i used the SINet.json to train the model. After training ends, there are two folders, named 'Dnc_SINet07-12_2344' and 'Enc_SINet07-10_1303'. Both floders have model.pth. And i have try the model.pth in these two folders, but both are wrong. So i just wonder, how can i test the model after i have finishing the training?
Hello,
I am trying to run "main.py", but i get the following error. Could you please provide a little bit explanation.
python main.py --c ExtremeC3Net.json
no tensorflow found. set use_tensorboard = False
no visdom found. set visdom_port = None
Traceback (most recent call last):
File "main.py", line 40, in
with open(args.config) as fin:
FileNotFoundError: [Errno 2] No such file or directory: 'ExtremeC3Net.json'
when i train or test command, there are the same error:
File "/media/cendelian/7CCAEC82CAEC39D2/CV/Project/人像分割/portrait_segmentation/etc/help_function.py", line 25, in val_edge
rand_pick = np.random.randint(0, len(val_loader))
File "mtrand.pyx", line 747, in numpy.random.mtrand.RandomState.randint
File "_bounded_integers.pyx", line 1270, in numpy.random._bounded_integers._rand_int64
ValueError: low >= high
could you help me
According to Visualize_video.py
, i wrote a infer.py
for testing pretrained model located in result
folder. But i found output result is terriable.
Below is the complete code of infer.py
script.
import cv2
import json
import models
import argparse
import torch
import numpy as np
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('-c', '--config', type=str)
parser.add_argument('-ip', '--img_path', type=str)
args = parser.parse_args()
with open(args.config) as fin:
config = json.load(fin)
test_config = config['test_config']
data_config = config['data_config']
if test_config["loss"] == "Lovasz":
test_config["num_classes"] = 1
print("Use Lovasz loss ")
Lovasz = True
else:
print("Use Cross Entropy loss ")
Lovasz = False
if test_config['Model'].startswith('Stage2'):
model = models.__dict__[test_config["Model"]](classes=test_config["num_classes"],
p=test_config["p"], q=test_config["q"])
elif test_config["Model"].startswith('Dnc_SINet'):
model = models.__dict__[test_config["Model"]](
classes=test_config["num_classes"], p=test_config["p"], q=test_config["q"],
chnn=test_config["chnn"])
mean = [107.304565, 115.69884, 132.35703]
std = [63.97182, 65.1337, 68.29726]
model.eval()
image = cv2.imread(args.img_path)
image = cv2.resize(image, (data_config['w'], data_config['h']))
img = image.astype(np.float32)
img = img.astype(np.float32)
for j in range(3):
img[:, :, j] -= mean[j]
for j in range(3):
img[:, :, j] /= std[j]
img /= 255.
img = img.transpose((2, 0, 1))
img_tensor = torch.from_numpy(img)
img_tensor = torch.unsqueeze(img_tensor, 0)
with torch.no_grad():
img_variable = torch.autograd.Variable(img_tensor)
if torch.cuda.is_available():
img_variable = img_variable.cuda()
out = model(img_variable)
classMap_numpy = (out[0].data.cpu() > 0).numpy()[0]
idx_fg = (classMap_numpy == 1)
seg_img = image * idx_fg[:, :, np.newaxis]
cv2.imwrite("test.jpg", seg_img.astype(np.uint8))
And run with the following command
python3 infer.py -c setting/Test_ExtremeC3Net.json -ip /Users/edvardzeng/Dataset/PortraitDataset/Nukki/baidu_V1/input/43.png
Output result:
Did i missing something ?
Looking forward to your replay.
Thx.
I am unable to use the lovasz loss b/c of a channel mismatch:
the output I receive is 2 channels vs lovasz expects 1.
I saw in the comments that we have to adjust this as needed, but unclear where to do it?
I thought the chnn was how to control it but changing that from 1 - 4 does not affect the final tensor output?
Could you kindly clarify how to toggle between lovasz and CE outputs?
Thanks!
@HYOJINPARK Thank you for your hard work,
If I want to train baidu_V2, what exactly should I change?
Because I tried training baidu_V2 dataset, and got error:
FileNotFoundError: [Errno 2] No such file or directory: './baidu_V2//Portrait/train.txt'
So I placed the the files within Portrait
folder, like this:
baidu_V2
└── Portrait
├── input
├── target
├── train.txt
└── val.txt
And now I am getting error:
ValueError: invalid literal for int() with base 10: '07517.jpg\n'
The number of labels is 1887.
The number of pictures is 1630.
I trained the SINet model on EG1800 and the baidu augmented datasets, using the default settings in the code. I tried both cross entropy and lovasz loss functions for training. Both of them acheived mIOU ~ 94.5; but they do not seem to perform well on real world portrait images(even the provided SINet.pth checkpoint). I tried testing the model using the demo video from portrait-net repo and 'Visualize_video.py' script. In both cases there were artefacts on the images in the background regions and sometimes they appear on foreground regions also.
Output Videos:
test_results.zip
Setting --visualize to True doesn't appear to do anything.
Thanks for your share~
I have some problems about the training~
When i try to change the input_image size from 224224 to 448448, it occurs AttributeError :
AttributeError: 'ExtremeC3NetCoarse' object has no attribute '_parameters'
Could you help me?
Thank you for the wonderful library! Wow!
I tried the demo on a sample image, but noticed that the edges on the output are rough. Can you guide me please.....
This is for SINet
Hello @HYOJINPARK ,
when i try to use the result/SINet/SINet.pth
with ./etc/Visualize_video.py
i get the following Error:
RuntimeError: Error(s) in loading state_dict for SINet:
size mismatch for encoder.classifier.conv.weight: copying a param of torch.Size([1, 144, 1, 1]) from checkpoint, where the shape is torch.Size([2, 144, 1, 1]) in current model.
size mismatch for bn_3.weight: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for bn_3.bias: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for bn_3.running_mean: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for bn_3.running_var: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for level2_C.conv.weight: copying a param of torch.Size([1, 48, 1, 1]) from checkpoint, where the shape is torch.Size([2, 48, 1, 1]) in current model.
size mismatch for level2_C.bn.weight: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for level2_C.bn.bias: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for level2_C.bn.running_mean: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for level2_C.bn.running_var: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for level2_C.act.weight: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for bn_2.weight: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for bn_2.bias: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for bn_2.running_mean: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for bn_2.running_var: copying a param of torch.Size([1]) from checkpoint, where the shape is torch.Size([2]) in current model.
size mismatch for classifier.1.weight: copying a param of torch.Size([1, 1, 3, 3]) from checkpoint, where the shape is torch.Size([2, 2, 3, 3]) in current model.
With self trained models i don't get this error. How can i fix this?
Your response would be highly appreciated. Thank you!
hello!
Nice work! I want to train the model with my own dataset, is there a guide on how to start with?
Thanks for your reply in advance
HI, I haven't figure out why a bn is used after unsample, is the bn necessary? I would really appreciate for your reply
Do you have any tip for small object trainning.
I am dealing the small binary segmentation but it doesn';t have good result.
Do you have any tip.
Hello @HYOJINPARK ,
I successfully trained the ExtremeC3Net. Now I want to test new images to see the testing accuracy. I am not sure what to do next. I tried putting images in Link512DATA folder and run Test_model.py. But there is no option to show images results.
Your response would be highly appreciated. Thank you
I am trying to convert the pth model to onnx and I get the following error:
Traceback (most recent call last):
File "", line 1, in
File "/home/rsalasch/miniconda3/envs/tf1_4/lib/python3.6/site-packages/torch/onnx/init.py", line 26, in _export
result = utils._export(*args, **kwargs)
File "/home/rsalasch/miniconda3/envs/tf1_4/lib/python3.6/site-packages/torch/onnx/utils.py", line 488, in _export
fixed_batch_size=fixed_batch_size)
File "/home/rsalasch/miniconda3/envs/tf1_4/lib/python3.6/site-packages/torch/onnx/utils.py", line 351, in _model_to_graph
fixed_batch_size=fixed_batch_size, params_dict=params_dict)
File "/home/rsalasch/miniconda3/envs/tf1_4/lib/python3.6/site-packages/torch/onnx/utils.py", line 154, in _optimize_graph
graph = torch._C._jit_pass_onnx(graph, operator_export_type)
File "/home/rsalasch/miniconda3/envs/tf1_4/lib/python3.6/site-packages/torch/onnx/init.py", line 199, in _run_symbolic_function
return utils._run_symbolic_function(*args, **kwargs)
File "/home/rsalasch/miniconda3/envs/tf1_4/lib/python3.6/site-packages/torch/onnx/utils.py", line 740, in _run_symbolic_function
return op_fn(g, *inputs, **attrs)
File "/home/rsalasch/miniconda3/envs/tf1_4/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 128, in wrapper
args = [_parse_arg(arg, arg_desc) for arg, arg_desc in zip(args, arg_descriptors)]
File "/home/rsalasch/miniconda3/envs/tf1_4/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 128, in
args = [_parse_arg(arg, arg_desc) for arg, arg_desc in zip(args, arg_descriptors)]
File "/home/rsalasch/miniconda3/envs/tf1_4/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 81, in _parse_arg
"', since it's not constant, please try to make "
RuntimeError: Failed to export an ONNX attribute 'onnx::Gather', since it's not constant, please try to make things (e.g., kernel size) static if possible
I think the error is caused by the AveragePool2D
(6)
(d2): S2block(
(down_res): AvgPool2d(kernel_size=4, stride=4, padding=0)
(7)
(d2): S2block(
(down_res): AvgPool2d(kernel_size=2, stride=2, padding=0)
Please let me know if you have converted the model to onnx format before or if this error seems familiar.
@HYOJINPARK Thank you
Upload a pretrained model so that we can test
Thank you for the wonderful library! Wow!
why set ignore_idx==255 in lovasz loss? which is the person label in target .
Can you guide me please.....
============
main.py
if train_config["loss"] == "Lovasz":
from etc.lovasz_losses import lovasz_hinge
criteria = lovasz_hinge(ignore=data_config["ignore_idx"])
else:
from etc.Criteria import CrossEntropyLoss2d
criteria = CrossEntropyLoss2d(weight,ignore=data_config["ignore_idx"]) # weight
SINet.json
"data_config": {
"cash" : "./pickle_file/portrait.p",
"dataset_name": "CVportrait",
"data_dir": "../../Link512DATA/",
"classes" : 2,
"ignore_idx" : 255,
"num_work" : 4,
syn_bg.jpg not found
Hi,
I tried retrain the model on biadu_V2, i am getting training iou as 0.945, but i evaluated on real life images, I am not able to get good results.
Also I can see the loss function you used in here is lovasz whereas the paper SINET has a completely different loss function. Can you provide some information on why the loss function and what could be the problem with these test images?
syn_bg = cv2.imread(os.path.join(savedir,'syn_bg.jpg'))
when i run python main.py, the error accurs as below:
File "/media/cendelian/新加卷/CV/Project/人像分割/portrait_segmentation/data/DataSet.py", line 42, in getitem
label_bool = 255 * ((label > 200).astype(np.uint8))
TypeError: '>' not supported between instances of 'NoneType' and 'int'
could you help me
Thank for your sharing paper, code and dataset.
But I can't download Nukki.zip.
Hello. Thank you for sharing your code and paper.
Could you tell me how to use the raw dataset EG1800 to verify the baseline?
Thank you.
I run the test data by my computer camera with resize image of 224*224. In CPU environment of I5 1.6gGHz, the model predict time is about 350+ms. Is there any problems? I wonder to know why the Flops is 0.128G, and the algorithm speed is so low.
Hello Hyojin Park,
Thanks for the amazing work. I am trying to run the code (ExtPortraitSeg "main.py") on my dataset. However, I have difficulty to do so. Could you please explain the procedure. Your response would be highly appreciated. Thank you
Thank you for your excellent project.I have a question about :
config = [[[3, 1], [5, 1]], [[3, 1], [3, 1]],
[[3, 1], [5, 1]], [[3, 1], [3, 1]], [[5, 1], [3, 2]], [[5, 2], [3, 4]],
[[3, 1], [3, 1]], [[5, 1], [5, 1]], [[3, 2], [3, 4]], [[3, 1], [5, 2]]]
Is there any rule in setting this parameter?
Hello. Thank you for sharing your code and paper.
I am checking your code and there are some confuse what I face.
First in the readme file, if I want to use the custom dataset I need to make new pickle file.
Could you explain which part of the code should I change?
In the train and test config, there is pickle file we need to change.
Is it automatically change or should we make this manually.
Also in the config there is dataset address we need to change
"data_dir": "../../Link512DATA/"
is it assume that the structure of folder is below?
Link512DATA/
└── Nukki
└── baidu_V1
├── input
├── target
├── train.txt
└── val.txt
└── baidu_V2
├── input
├── target
├── train.txt
└── val.txt
└── Portrait
hi,
when I run the command
python main.py --c ./setting/SINet.json
there is a error:
ValueError: invalid literal for int() with base 10: '501.png\n'
my dir tree is
.self.data_dir(../Link512DATA/)
├── Nukki
│ ├── baidu_V1
│ │ ├── input
│ │ ├── target
│ │ ├── train.txt
│ │ └── val.txt
│ └── baidu_V2
│ ├── input
│ ├── target
│ ├── train.txt
│ └── val.txt
└── Portrait
└── train.txt
the Portrait's train.txt is same as the baidu_V1's train.txt
could you help me?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.