win10怎样运行？ about ultra-light-fast-generic-face-detector-1mb

[Linzaer]

@Xuerrui win10推理测试可以直接运行，训练的话可以吧.sh里的参数对应写到启动py文件的arg参数里，多个shell文件只是为了在linux下更方便训练一些。

[TMaysGGS]

@Linzaer 训练时似乎需要把num_workers这个参数设置为0或者不写（以使用默认值0）,否则多线程只会开启多个子进程并且不调用GPU。请参考issue 12831以及Pytorch设置多线程进行dataloader时影响GPU运行。

[Linzaer]

@TMaysGGS num_workers在我理解中就是并行对数据集的图片做预处理，而且这边默认用的cpu做图片预处理，其实训练瓶颈在cpu预处理部分，会占用比较高的cpu，反而由于网络比较轻量，gpu占用上不来。而且实测num_workers越多cpu占用越高，整体训练速度也越快，设置为0的话实测下来和1差不多。不过也也受限于io瓶颈。当然也可以试试Nvidia DALI用gpu加速图片预处理。

[TMaysGGS]

@TMaysGGS num_workers在我理解中就是并行对数据集的图片做预处理，而且这边默认用的cpu做图片预处理，其实训练瓶颈在cpu预处理部分，会占用比较高的cpu，反而由于网络比较轻量，gpu占用上不来。而且实测num_workers越多cpu占用越高，整体训练速度也越快，设置为0的话实测下来和1差不多。不过也也受限于io瓶颈。当然也可以试试Nvidia DALI用gpu加速图片预处理。

谢谢大佬回复。的确瓶颈在图片预处理部分，不过我这边因为是Win10系统下运行，似乎只要num_workers不是0就一定报错……以至于不得不这么设置……

[Linzaer]

嗯，解决问题就好。

[Xuerrui]

这样是什么错误呀？
runfile('D:/2_software/experience/Ultra-Light-Fast-Generic-Face-Detector-1MB-master/train.py', wdir='D:/2_software/experience/Ultra-Light-Fast-Generic-Face-Detector-1MB-master')
2019-10-25 11:01:11,520 - root - INFO - inpu size :320
priors nums:4420
Traceback (most recent call last):
File "", line 1, in
File "D:\2_software\pycharm\PyCharm Community Edition 2019.2.3\helpers\pydev_pydev_bundle\pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "D:\2_software\pycharm\PyCharm Community Edition 2019.2.3\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "D:/2_software/experience/Ultra-Light-Fast-Generic-Face-Detector-1MB-master/train.py", line 112, in
from vision.ssd.data_preprocessing import TrainAugmentation, TestTransform
File "D:\2_software\pycharm\PyCharm Community Edition 2019.2.3\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
File "D:\2_software\experience\Ultra-Light-Fast-Generic-Face-Detector-1MB-master\vision\ssd\data_preprocessing.py", line 1, in
from ..transforms.transforms import *
File "D:\2_software\pycharm\PyCharm Community Edition 2019.2.3\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
File "D:\2_software\experience\Ultra-Light-Fast-Generic-Face-Detector-1MB-master\vision\transforms\transforms.py", line 10, in
from torchvision import transforms
File "D:\2_software\pycharm\PyCharm Community Edition 2019.2.3\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self.system_import(name, *args, **kwargs)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torchvision_init.py", line 1, in
from torchvision import models
File "D:\2_software\pycharm\PyCharm Community Edition 2019.2.3\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self.system_import(name, *args, **kwargs)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torchvision\models_init.py", line 12, in
from . import detection
File "D:\2_software\pycharm\PyCharm Community Edition 2019.2.3\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self.system_import(name, *args, **kwargs)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torchvision\models\detection_init.py", line 1, in
from .faster_rcnn import *
File "D:\2_software\pycharm\PyCharm Community Edition 2019.2.3\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torchvision\models\detection\faster_rcnn.py", line 13, in
from .rpn import AnchorGenerator, RPNHead, RegionProposalNetwork
File "D:\2_software\pycharm\PyCharm Community Edition 2019.2.3\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torchvision\models\detection\rpn.py", line 8, in
from . import _utils as det_utils
File "D:\2_software\pycharm\PyCharm Community Edition 2019.2.3\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self.system_import(name, *args, **kwargs)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torchvision\models\detection_utils.py", line 74, in
@torch.jit.script
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit_init.py", line 364, in script
graph = _script_graph(fn, _frames_up=frames_up + 1)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit_init.py", line 359, in _script_graph
ast = get_jit_ast(fn)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 132, in get_jit_ast
return build_def(SourceRangeFactory(source), py_ast.body[0])
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 151, in build_def
build_stmts(ctx, body))
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 123, in build_stmts
stmts = [build_stmt(ctx, s) for s in stmts]
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 123, in
stmts = [build_stmt(ctx, s) for s in stmts]
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 140, in call
return method(ctx, node)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 205, in build_Assign
rhs = build_expr(ctx, stmt.value)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 140, in call
return method(ctx, node)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 314, in build_Call
func = build_expr(ctx, expr.func)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 140, in call
return method(ctx, node)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 300, in build_Attribute
value = build_expr(ctx, expr.value)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 140, in call
return method(ctx, node)
File "D:\2_software\anaconda\envs\pytorch\lib\site-packages\torch\jit\frontend.py", line 422, in build_Subscript
raise NotSupportedError(base.range(), "slicing multiple dimensions at the same time isn't supported yet")
torch.jit.frontend.NotSupportedError: slicing multiple dimensions at the same time isn't supported yet
proposals (Tensor): boxes to be encoded
"""
# perform some unpacking to make it JIT-fusion friendly
wx = weights[0]
wy = weights[1]
ww = weights[2]
wh = weights[3]
proposals_x1 = proposals[:, 0].unsqueeze(1)
~~~~~~~~~ <--- HERE
proposals_y1 = proposals[:, 1].unsqueeze(1)
proposals_x2 = proposals[:, 2].unsqueeze(1)
proposals_y2 = proposals[:, 3].unsqueeze(1)
reference_boxes_x1 = reference_boxes[:, 0].unsqueeze(1)
reference_boxes_y1 = reference_boxes[:, 1].unsqueeze(1)
reference_boxes_x2 = reference_boxes[:, 2].unsqueeze(1)
reference_boxes_y2 = reference_boxes[:, 3].unsqueeze(1)

[Zx117x]

@Linzaer 训练时似乎需要把num_workers这个参数设置为0或者不写（以使用默认值0）,否则多线程只会开启多个子进程并且不调用GPU。请参考issue 12831以及Pytorch设置多线程进行dataloader时影响GPU运行。

老哥，能给出在win下运行的完整步骤吗，除了num_workers设为0，还有其它需要注意的吗