singlezombie / dl-demos Goto Github PK

您好。
我在使用了你的并行demo之后，同样的模型和batch 比nn.DataParallel 需要更大显存了，这是为什么呢？。环境：pytorch1.12 cuda10.2

》pip show torch
Name: torch
Version: 1.13.0
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: [email protected]
License: BSD-3
Location: c:\python310\lib\site-packages
Requires: typing-extensions
Required-by: torchaudio, torchdata, torchtext, torchvision

》pip show torchtext
Name: torchtext
Version: 0.14.0
Summary: Text utilities and datasets for PyTorch
Home-page: https://github.com/pytorch/text
Author: PyTorch core devs and James Bradbury
Author-email: [email protected]
License: BSD
Location: c:\python310\lib\site-packages
Requires: numpy, requests, torch, tqdm
Required-by:

》python dldemos\Transformer\train.py
Traceback (most recent call last):
File "D:\IDEA_workshop\deep_learning\deep_learning-Demos\dldemos\Transformer\train.py", line 4, in
import torch.nn as nn
File "C:\Python310\lib\site-packages\torch\nn_init_.py", line 1, in
from .modules import * # noqa: F403
File "C:\Python310\lib\site-packages\torch\nn\modules_init_.py", line 1, in
from .module import Module
File "C:\Python310\lib\site-packages\torch\nn\modules\module.py", line 8, in
from ..parameter import Parameter
File "C:\Python310\lib\site-packages\torch\nn\parameter.py", line 2, in
from torch._C import _disabled_torch_function_impl
ModuleNotFoundError: No module named 'torch._C'

No such file or directory: 'dldemos/ddpm/model_unet_res.pth'

您好！请问ddpm/main.py中的权重文件‘’dldemos/ddpm/model_unet_res.pth‘’怎样获取呢

for mini_batch_X, mini_batch_Y in mini_batch_XYs:
mini_batch_Y_hat = model.forward(mini_batch_X)
model.backward(mini_batch_Y)
optimizer.zero_grad()
optimizer.add_grad(model.get_grad_dict())
optimizer.step()
您好：
已知：model.forward()函数用的参数(w,b)是存在列表里面，经过 optimizer.step()更新后的参数是存在字典里。
疑问：在循环时，model.forward()函数为什么能够使用存在字典里的参数呢？
谢谢！