haoxizhong / pytorch-worker Goto Github PK

I think the framework deserves a cooler name !!!

06/18/2020 16:52:29 - INFO - tools.train_tool - Training start....
Traceback (most recent call last):
File "D:/workspace/CAIL2020/sfks/baseline/train.py", line 59, in
train(parameters, config, gpu_list, do_test)
File "D:\workspace\CAIL2020\sfks\baseline\tools\train_tool.py", line 91, in train
for step, data in enumerate(dataset):
File "D:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 279, in iter
return _MultiProcessingDataLoaderIter(self)
File "D:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 719, in init
w.start()
File "D:\ProgramData\Anaconda3\envs\pytorch\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "D:\ProgramData\Anaconda3\envs\pytorch\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "D:\ProgramData\Anaconda3\envs\pytorch\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "D:\ProgramData\Anaconda3\envs\pytorch\lib\multiprocessing\popen_spawn_win32.py", line 89, in init
reduction.dump(process_obj, to_child)
File "D:\ProgramData\Anaconda3\envs\pytorch\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'init_formatter..train_collate_fn'

请问大佬们运行train时有没有遇到这个问题？如何解决呀？

BasicBert.config中
train_data_path = /data/disk5/private/zhx/pytorcher/data/nlp
bert_path = /data/disk5/private/zhx/bert/chinese
等参数应该如何替换