levihsu / ootdiffusion

Official implementation of OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on

OOTDiffusion

This repository is the official implementation of OOTDiffusion

🤗 Try out OOTDiffusion

(Thanks to ZeroGPU for providing A100 GPUs)

OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on [arXiv paper]
Yuhao Xu, Tao Gu, Weifeng Chen, Chengcai Chen
Xiao-i Research

Our model checkpoints trained on VITON-HD (half-body) and Dress Code (full-body) have been released

• 🤗 Hugging Face link for checkpoints (ootd, humanparsing, and openpose)
• 📢📢 We support ONNX for humanparsing now. Most environmental issues should have been addressed : )
• Please also download clip-vit-large-patch14 into checkpoints folder
• We've only tested our code and models on Linux (Ubuntu 22.04)

   

Installation

1. Clone the repository

git clone https://github.com/levihsu/OOTDiffusion

2. Create a conda environment and install the required packages

conda create -n ootd python==3.10
conda activate ootd
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2
pip install -r requirements.txt

Inference

1. Half-body model

cd OOTDiffusion/run
python run_ootd.py --model_path <model-image-path> --cloth_path <cloth-image-path> --scale 2.0 --sample 4

2. Full-body model

Garment category must be paired: 0 = upperbody; 1 = lowerbody; 2 = dress

cd OOTDiffusion/run
python run_ootd.py --model_path <model-image-path> --cloth_path <cloth-image-path> --model_type dc --category 2 --scale 2.0 --sample 4

Citation

@article{xu2024ootdiffusion,
  title={OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on},
  author={Xu, Yuhao and Gu, Tao and Chen, Weifeng and Chen, Chengcai},
  journal={arXiv preprint arXiv:2403.01779},
  year={2024}
}

Star History

TODO List

• Paper
• Gradio demo
• Inference code
• Model weights
• Training code

ootdiffusion's Issues

#Question# Is it normal that the process is so slow?

Is it normal that the process is so slow? 16 minutes in RTX 2060
4 samples
Windows 10

ModuleNotFoundError: No module named 'config'

(ootd) E:\software\OOTDiffusion\run>python run_ootd.py --model_path E:\software\OOTDiffusion\run\examples\model --cloth_path E:\software\OOTDiffusion\run\examples\garment --scale 2.0 --sample 4
Traceback (most recent call last):
File "E:\software\OOTDiffusion\run\run_ootd.py", line 9, in
from preprocess.openpose.run_openpose import OpenPose
File "E:\software\OOTDiffusion\preprocess\openpose\run_openpose.py", line 3, in
import config
ModuleNotFoundError: No module named 'config'

RuntimeError: CUDA error: invalid device ordinal

Traceback (most recent call last):
  File "/mnt/e/project/80297283/OOTDiffusion/run/gradio_ootd.py", line 29, in <module>
    openpose_model_dc = OpenPose(1)
  File "/mnt/e/project/80297283/OOTDiffusion/preprocess/openpose/run_openpose.py", line 32, in __init__
    torch.cuda.set_device(gpu_id)
  File "/home/pub_gpu/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/cuda/__init__.py", line 350, in set_device
    torch._C._cuda_setDevice(device)
RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

how do i fix it?
ps: who run the gradio demo successfully?

running gradio_ootd.py, cuda error

(ootd) root@autodl-container-cf29119c3c-0ecee78c:~/autodl-tmp/OOTDiffusion/run# python gradio_ootd.py
Loading pipeline components...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:01<00:00, 6.08it/s]
text_config_dict is provided which will be used to initialize CLIPTextConfig. The value text_config["id2label"] will be overriden.
text_config_dict is provided which will be used to initialize CLIPTextConfig. The value text_config["bos_token_id"] will be overriden.
text_config_dict is provided which will be used to initialize CLIPTextConfig. The value text_config["eos_token_id"] will be overriden.
Loading pipeline components...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 11.18it/s]
Traceback (most recent call last):
File "/root/autodl-tmp/OOTDiffusion/run/gradio_ootd.py", line 31, in
ootd_model_dc = OOTDiffusionDC(1)
File "/root/autodl-tmp/OOTDiffusion/ootd/inference_ootd_dc.py", line 74, in init
).to(self.gpu_id)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 864, in to
module.to(device, dtype)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1145, in to
return self._apply(convert)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/nn/modules/module.py", line 820, in _apply
param_applied = fn(param)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1143, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Windows部署排坑！！！

系统版本：Win10
显卡：4090

报错一：
subprocess.CalledProcessError: Command ‘[where, cl]‘ returned non-zero exit status 1
解决方案：https://zhuanlan.zhihu.com/p/582856277

报错二：
ninja: build stopped: subcommand failed.
解决方案：

1. pip install inplace-abn
2. 在 preprocess\humanparsing\modules\functions.py 文件中
   加入 import inplace_abn as _backend
3. 在 preprocess\humanparsing\modules\functions.py 文件中
   注释_backend = load(name="inplace_abn",
   extra_cflags=["-O3"],
   sources=[path.join(_src_path, f) for f in [
   "inplace_abn.cpp",
   "inplace_abn_cpu.cpp",
   "inplace_abn_cuda.cu",
   "inplace_abn_cuda_half.cu"
   ]],
   extra_cuda_cflags=["--expt-extended-lambda"])
   

Module not found error: No module named 'cv2'

where can i download the ootd_dc model?

it seems like only the ootd_hd upper body model has been released,when and how can i get the full body ootd_dc model?i appreciate for your hard work

Ask for DC model

thanks for your amazing work! could you please open dc model and train code? thanks again!

Is there a specific time for open training code and the dc model

ImportError: cannot import name 'CaptionProjection' from 'diffusers.models.embeddings'

File "/root/OOTDiffusion/run/run_ootd.py", line 11, in
from ootd.inference_ootd_hd import OOTDiffusionHD
File "/root/OOTDiffusion/ootd/inference_ootd_hd.py", line 24, in
from pipelines_ootd.pipeline_ootd import OotdPipeline
File "/root/OOTDiffusion/ootd/pipelines_ootd/pipeline_ootd.py", line 27, in
from .unet_vton_2d_condition import UNetVton2DConditionModel
File "/root/OOTDiffusion/ootd/pipelines_ootd/unet_vton_2d_condition.py", line 23, in
from .unet_vton_2d_blocks import (
File "/root/OOTDiffusion/ootd/pipelines_ootd/unet_vton_2d_blocks.py", line 23, in
from .transformer_vton_2d import Transformer2DModel
File "/root/OOTDiffusion/ootd/pipelines_ootd/transformer_vton_2d.py", line 29, in
from diffusers.models.embeddings import CaptionProjection, PatchEmbed
ImportError: cannot import name 'CaptionProjection' from 'diffusers.models.embeddings' (/root/anaconda3/envs/ootd/lib/python3.10/site-packages/diffusers/models/embeddings.py)

how to solve it?

ValueError: Cannot load ../checkpoints/ootd/ootd_hd/checkpoint-36000

File "/root/OOTDiffusion/run/run_ootd.py", line 47, in
model = OOTDiffusionHD(args.gpu_id)
File "/root/OOTDiffusion/ootd/inference_ootd_hd.py", line 57, in init
unet_vton = UNetVton2DConditionModel.from_pretrained(
File "/root/anaconda3/envs/ootd/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 812, in from_pretrained
unexpected_keys = load_model_dict_into_meta(
File "/root/anaconda3/envs/ootd/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 155, in load_model_dict_into_meta
raise ValueError(
ValueError: Cannot load ../checkpoints/ootd/ootd_hd/checkpoint-36000 because conv_in.weight expected shape tensor(..., device='meta', size=(320, 8, 3, 3)), but got torch.Size([320, 4, 3, 3]). If you want to instead overwrite randomly initialized weights, please make sure to pass both low_cpu_mem_usage=False and ignore_mismatched_sizes=True.

可以支持儿童模特么，挺多人需要的

pickle_module.load(f, **pickle_load_args)

python run_ootd.py --model_path /root/OOTDiffusion/run/examples/model/01008_00.jpg --cloth_path /root/OOTDiffusion/run/examples/garment/00055_00.jpg --scale 2.0 --sample 4

-:
Traceback (most recent call last):
File "/root/OOTDiffusion/run/run_ootd.py", line 29, in
openpose_model = OpenPose(args.gpu_id)
File "/root/OOTDiffusion/preprocess/openpose/run_openpose.py", line 33, in init
self.preprocessor = OpenposeDetector()
File "/root/OOTDiffusion/preprocess/openpose/annotator/openpose/init.py", line 63, in init
self.body_estimation = Body(body_modelpath)
File "/root/OOTDiffusion/preprocess/openpose/annotator/openpose/body.py", line 26, in init
model_dict = util.transfer(self.model, torch.load(model_path))
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/serialization.py", line 815, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/torch/serialization.py", line 1033, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '\x05'.

模型是哪些，在哪能下？

找不到说明

CUDA OOM

python run_ootd.py --model_path ../assets/m1.png --cloth_path ../assets/t1.png --scale 2.0 --sample 4

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.75 GiB (GPU 0; 23.68 GiB total capacity; 20.97 GiB already allocated; 315.50 MiB free; 21.17 GiB reserved in total by PyTorch)

Can't run with a 24gb memory device?

No module named 'torch._custom_ops'

(ootd) E:\software\OOTDiffusion\run>python run_ootd.py --model_path E:\software\OOTDiffusion\run\examples\model\model_1.png --cloth_path E:\software\OOTDiffusion\run\examples\garment\02305_00.jpg --scale 2.0 --sample 4
Traceback (most recent call last):
File "E:\software\OOTDiffusion\run\run_ootd.py", line 9, in
from preprocess.openpose.run_openpose import OpenPose
File "E:\software\OOTDiffusion\preprocess\openpose\run_openpose.py", line 20, in
from preprocess.openpose.annotator.openpose import OpenposeDetector
File "E:\software\OOTDiffusion\preprocess\openpose\annotator\openpose_init_.py", line 15, in
from .body import Body
File "E:\software\OOTDiffusion\preprocess\openpose\annotator\openpose\body.py", line 14, in
from torchvision import transforms
File "C:\Users\Administrator\pinokio\bin\miniconda\envs\ootd\lib\site-packages\torchvision_init_.py", line 6, in
from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils
File "C:\Users\Administrator\pinokio\bin\miniconda\envs\ootd\lib\site-packages\torchvision_meta_registrations.py", line 4, in
import torch._custom_ops
ModuleNotFoundError: No module named 'torch._custom_ops'

(ootd) E:\software\OOTDiffusion\run>

No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.3'

(ootd) E:\software\OOTDiffusion\run>python run_ootd.py --model_path E:\software\OOTDiffusion\run\examples\model\ --cloth_path E:\software\OOTDiffusion\run\examples\garment\ --scale 2.0 --sample 4
No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.3'
C:\Users\Administrator\pinokio\bin\miniconda\envs\ootd\lib\site-packages\torch\utils\cpp_extension.py:359: UserWarning: Error checking compiler version for cl: [WinError 2] 系统找不到指定的文件。

clip-vit-large-patch14 problem

(ootd) D:\OOTDiffusion\OOTDiffusion\run>python run_ootd.py --model_path D:\OOTDiffusion\checkpoints\ootd --cloth_path D:\OOTDiffusion\images\demo.png --scale 2.0 --sample 4
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00, 16.35it/s]
Traceback (most recent call last):
File "D:\miniconda38\envs\ootd\lib\site-packages\transformers\utils\hub.py", line 389, in cached_file
resolved_file = hf_hub_download(
File "D:\miniconda38\envs\ootd\lib\site-packages\huggingface_hub\utils_validators.py", line 110, in _inner_fn
validate_repo_id(arg_value)
File "D:\miniconda38\envs\ootd\lib\site-packages\huggingface_hub\utils_validators.py", line 158, in validate_repo_id
raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '../checkpoints/clip-vit-large-patch14'. Use repo_type argument if needed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\OOTDiffusion\OOTDiffusion\run\run_ootd.py", line 47, in
model = OOTDiffusionHD(args.gpu_id)
File "D:\OOTDiffusion\OOTDiffusion\ootd\inference_ootd_hd.py", line 78, in init
self.auto_processor = AutoProcessor.from_pretrained(VIT_PATH)
File "D:\miniconda38\envs\ootd\lib\site-packages\transformers\models\auto\processing_auto.py", line 234, in from_pretrained
preprocessor_config_file = get_file_from_repo(
File "D:\miniconda38\envs\ootd\lib\site-packages\transformers\utils\hub.py", line 541, in get_file_from_repo
return cached_file(
File "D:\miniconda38\envs\ootd\lib\site-packages\transformers\utils\hub.py", line 454, in cached_file
raise EnvironmentError(
OSError: Incorrect path_or_model_id: '../checkpoints/clip-vit-large-patch14'. Please provide either the path to a local folder or the repo_id of a model on the Hub.

(ootd) D:\OOTDiffusion\OOTDiffusion\run>

I can't find any about clip-vit-large-patch14 model in your huggingface file. So I download the openai file. but it does't work.

How can I solve this problem?

Any way to avoid hallucination and prevent print changes?

Hello! I hope everyone is well! Firstly, I would like to thank you for making this incredible work available.

I'm doing some tests, and I've noticed some hallucinations. Apparently, the model reconstructs the clothing print, causing some differences in the final result.

I tried adjusting the parameters, but I couldn't resolve it.

Is there any way to get around the problem?

I know that the model must be in the early stages, and there is still a lot to be done, but I would like to know if I can do anything about it right now.

Thank you for your attention!

Congrats for your work and..

This is great job, do you have plans to release model's checkpoints soon?

攻克，Windows不用搭建环境（包括CUDA、ninja编译、VS C++编译等都不用），只要英伟达显卡就行，解压即用！

伟大的项目，但目前环境搭建困难，所以研读修改代码，搞出来了整合包
不用搭建环境（包括CUDA、ninja编译、VS C++编译等都不用），只要英伟达显卡就行，解压即用！
https://github.com/zhaoyun0071/OOTDiffusion-windows-GUI

Training tricks

Really impressive results, thanks for sharing.

I compared the weights of the unet with SD1.5 and if I am not mistaken the whole net was fine-tuned. What training tricks were used to avoid overfiting on such a small dataset like viton-hd?

ImportError: cannot import name 'CaptionProjection' from 'diffusers.models.embeddings'

(ootd) root@autodl-container-cf29119c3c-0ecee78c:~/autodl-tmp/OOTDiffusion/run# python gradio_ootd.py
Traceback (most recent call last):
File "/root/autodl-tmp/OOTDiffusion/run/gradio_ootd.py", line 21, in
from ootd.inference_ootd_hd import OOTDiffusionHD
File "/root/autodl-tmp/OOTDiffusion/ootd/inference_ootd_hd.py", line 24, in
from pipelines_ootd.pipeline_ootd import OotdPipeline
File "/root/autodl-tmp/OOTDiffusion/ootd/pipelines_ootd/pipeline_ootd.py", line 27, in
from .unet_vton_2d_condition import UNetVton2DConditionModel
File "/root/autodl-tmp/OOTDiffusion/ootd/pipelines_ootd/unet_vton_2d_condition.py", line 23, in
from .unet_vton_2d_blocks import (
File "/root/autodl-tmp/OOTDiffusion/ootd/pipelines_ootd/unet_vton_2d_blocks.py", line 23, in
from .transformer_vton_2d import Transformer2DModel
File "/root/autodl-tmp/OOTDiffusion/ootd/pipelines_ootd/transformer_vton_2d.py", line 29, in
from diffusers.models.embeddings import CaptionProjection, PatchEmbed
ImportError: cannot import name 'CaptionProjection' from 'diffusers.models.embeddings' (/root/miniconda3/envs/ootd/lib/python3.10/site-packages/diffusers/models/embeddings.py)

--model_path&--model_clothes

i try below both failed, anyone point right way

python run_ootd.py --model_path "/root/autodl-tmp/OOTDiffusion/run/examples/model/" --cloth_path "/root/autodl-tmp/OOTDiffusion/run/examples/garment/" --scale 2.0 --sample 4

python run_ootd.py --model_path "./examples/model" --cloth_path "./examples/garment" --scale 2.0 --sample 4

请问大佬模型和config.json 是不是没放出来呢

CUDA_HOME environment variable is not set. Please set it to your CUDA install root

Does that mean Silicon M2 is not supported yet? Just gone through all the way here.

How much was training cost? What hardware?

Great work! How to call the checkpoints in script?

After I download these checkpoints, where should I put them and how to call them in the script? Thanks!

There is a problem with the program or documentation.

Should update the following.

1. The following files are required in checkpoints/humanparsing
   exp-schp-201908301523-atr.pth
   exp-schp-201908261155-lip.pth
   exp-schp-201908270938-pascal-person-part.pth

2. Program modification of degradations.py is required
   ・Instead of from torchvision.transforms.functional_tensor import rgb_to_grayscale
   ・from torchvision.transforms.functional import rgb_to_grayscale
   Otherwise, Module not found will occur.

missed dc model ?

This project is really impressive. where is the DC model ?

这种半身裙里用full-body用会穿不上，用穿裤子的模特会穿成裤子

windows下RuntimeError: Ninja is required to load C++ extensions

Traceback (most recent call last):
File "E:\PyCharmProject\OOTDiffusion\run\run_ootd.py", line 10, in
from preprocess.humanparsing.aigc_run_parsing import Parsing
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\aigc_run_parsing.py", line 7, in
from parsing_api import load_atr_model, load_lip_model, inference
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\parsing_api.py", line 11, in
import networks
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\networks_init_.py", line 2, in
from networks.AugmentCE2P import resnet101
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\networks\AugmentCE2P.py", line 22, in
from modules import InPlaceABNSync
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\modules_init_.py", line 1, in
from .bn import ABN, InPlaceABN, InPlaceABNSync
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\modules\bn.py", line 10, in
from .functions import *
File "E:\PyCharmProject\OOTDiffusion\preprocess\humanparsing\modules\functions.py", line 11, in
_backend = load(name="inplace_abn",
File "D:\soft\Miniconda\envs\ootd\lib\site-packages\torch\utils\cpp_extension.py", line 1284, in load
return _jit_compile(
File "D:\soft\Miniconda\envs\ootd\lib\site-packages\torch\utils\cpp_extension.py", line 1509, in _jit_compile
_write_ninja_file_and_build_library(
File "D:\soft\Miniconda\envs\ootd\lib\site-packages\torch\utils\cpp_extension.py", line 1593, in _write_ninja_file_and_build_library
verify_ninja_availability()
File "D:\soft\Miniconda\envs\ootd\lib\site-packages\torch\utils\cpp_extension.py", line 1649, in verify_ninja_availability
raise RuntimeError("Ninja is required to load C++ extensions")
RuntimeError: Ninja is required to load C++ extensions

It seems not well

Thanks for the project, but it didn't work very well when I tried it
This is OOT:

This is Outfit Anyone:

模型张量对不上

ValueError: Trying to set a tensor of shape torch.Size([49408, 768]) in "weight" (which has shape torch.Size([49408, 512])), this look incorrect.

请问大佬这个是什么问题呢

Linux下subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1

Traceback (most recent call last):
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
subprocess.run(
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Data2/wyw/OOTDiffusion/run/run_ootd.py", line 10, in
from preprocess.humanparsing.aigc_run_parsing import Parsing
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/aigc_run_parsing.py", line 7, in
from parsing_api import load_atr_model, load_lip_model, inference
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/parsing_api.py", line 11, in
import networks
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/networks/init.py", line 2, in
from networks.AugmentCE2P import resnet101
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/networks/AugmentCE2P.py", line 22, in
from modules import InPlaceABNSync
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/init.py", line 1, in
from .bn import ABN, InPlaceABN, InPlaceABNSync
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/bn.py", line 10, in
from .functions import *
File "/Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/functions.py", line 11, in
backend = load(name="inplace_abn",
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1284, in load
return jit_compile(
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1509, in jit_compile
write_ninja_file_and_build_library(
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1624, in write_ninja_file_and_build_library
run_ninja_build(
File "/home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension 'inplace_abn': [1/3] /usr/bin/nvcc -DTORCH_EXTENSION_NAME=inplace_abn -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/TH -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/THC -isystem /home/wyw/anaconda3/envs/ootd/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' --expt-extended-lambda -std=c++17 -c /Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/src/inplace_abn_cuda.cu -o inplace_abn_cuda.cuda.o
FAILED: inplace_abn_cuda.cuda.o
/usr/bin/nvcc -DTORCH_EXTENSION_NAME=inplace_abn -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/TH -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/THC -isystem /home/wyw/anaconda3/envs/ootd/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS_ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' --expt-extended-lambda -std=c++17 -c /Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/src/inplace_abn_cuda.cu -o inplace_abn_cuda.cuda.o
nvcc fatal : Value 'c++17' is not defined for option 'std'
[2/3] /usr/bin/nvcc -DTORCH_EXTENSION_NAME=inplace_abn -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/TH -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/THC -isystem /home/wyw/anaconda3/envs/ootd/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' --expt-extended-lambda -std=c++17 -c /Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/src/inplace_abn_cuda_half.cu -o inplace_abn_cuda_half.cuda.o
FAILED: inplace_abn_cuda_half.cuda.o
/usr/bin/nvcc -DTORCH_EXTENSION_NAME=inplace_abn -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/TH -isystem /home/wyw/anaconda3/envs/ootd/lib/python3.10/site-packages/torch/include/THC -isystem /home/wyw/anaconda3/envs/ootd/include/python3.10 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' --expt-extended-lambda -std=c++17 -c /Data2/wyw/OOTDiffusion/preprocess/humanparsing/modules/src/inplace_abn_cuda_half.cu -o inplace_abn_cuda_half.cuda.o
nvcc fatal : Value 'c++17' is not defined for option 'std'
ninja: build stopped: subcommand failed.

No module named 'skimage'

I encountered this issue saying skimage cannot be found.

python run_ootd.py --model_path <examples/model> --cloth_path <examples/garment> --model_type dc --category 2 --scale 2.0 --sample 4
cd: no such file or directory: OOTDiffusion/run
Traceback (most recent call last):
  File "/Users/it6/ootd/run/run_ootd.py", line 9, in <module>
    from preprocess.openpose.run_openpose import OpenPose
  File "/Users/it6/ootd/preprocess/openpose/run_openpose.py", line 20, in <module>
    from preprocess.openpose.annotator.openpose import OpenposeDetector
  File "/Users/it6/ootd/preprocess/openpose/annotator/openpose/__init__.py", line 16, in <module>
    from .hand import Hand
  File "/Users/it6/ootd/preprocess/openpose/annotator/openpose/hand.py", line 10, in <module>
    from skimage.measure import label
ModuleNotFoundError: No module named 'skimage'

So I use pip install skimage and received below errors:

Collecting skimage
  Downloading skimage-0.0.tar.gz (757 bytes)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [3 lines of output]
      
      *** Please install the `scikit-image` package (instead of `skimage`) ***
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

From here I'm in a deadloop.

Comfyui workflow?

Comfyui workflow?

How to train a model on my own dataset？please

I want to create my own dataset to train a model, could you provide the training code？

I made a version for pip that downloads the models/checkpoints automatically

Not sure where to put it, but it's a bit of a hassle to download the models correctly.

You can use pip install git+https://github.com/viktorfa/oot_diffusion.git and use OOTD in any python code.

https://github.com/viktorfa/oot_diffusion

模型训练的损失函数是什么，能直接用两张图片计算MSE，会有效果吗

ModuleNotFoundError: No module named 'config'

where to get this file?

Not able to run on M1 Mac!

Getting below error:

Traceback (most recent call last):
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/run/run_ootd.py", line 10, in
from preprocess.humanparsing.aigc_run_parsing import Parsing
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/aigc_run_parsing.py", line 7, in
from parsing_api import load_atr_model, load_lip_model, inference
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/parsing_api.py", line 11, in
import networks
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/networks/init.py", line 2, in
from networks.AugmentCE2P import resnet101
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/networks/AugmentCE2P.py", line 22, in
from modules import InPlaceABNSync
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/modules/init.py", line 1, in
from .bn import ABN, InPlaceABN, InPlaceABNSync
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/modules/bn.py", line 10, in
from .functions import *
File "/Users/gaurav13.singh/PycharmProjects/OOTDiffusion/preprocess/humanparsing/modules/functions.py", line 8, in
from torch.utils.cpp_extension import load
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 19, in
from .hipify import hipify_python
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/site-packages/torch/utils/hipify/hipify_python.py", line 34, in
from .cuda_to_hip_mappings import CUDA_TO_HIP_MAPPINGS
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/site-packages/torch/utils/hipify/cuda_to_hip_mappings.py", line 34, in
rocm_path = subprocess.check_output(["hipconfig", "--rocmpath"]).decode("utf-8")
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/subprocess.py", line 501, in run
with Popen(*popenargs, **kwargs) as process:
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/subprocess.py", line 966, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/Users/gaurav13.singh/miniconda/envs/ootd/lib/python3.10/subprocess.py", line 1842, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
NotADirectoryError: [Errno 20] Not a directory: 'hipconfig'

不用C++编译、CUDA配置什么的，我改了一下推理的代码，可以直接跑，但

将"preprocess\humanparsing\modules\functions.py"中的内容替换如下，可以跑起来。

from os import path

import torch.distributed as dist
import torch.autograd as autograd

_src_path = path.join(path.dirname(path.abspath(__file__)), "src")


# Activation names
ACT_RELU = "relu"
ACT_LEAKY_RELU = "leaky_relu"
ACT_ELU = "elu"
ACT_NONE = "none"
def normalize_shape(x):
    if x.ndimension() == 1:
        return x.view(1, -1, 1)
    else:
        return x.view(x.size(0), x.size(1), -1)

import torch


def broadcast_to(v, x):
    if x.ndimension() == 2:
        return v
    else:
        broadcast_size = [1, -1]
        for i in range(2, x.ndimension()):
            broadcast_size.append(1)

        return v.view(broadcast_size)


def forward_cpu(x, mean, var, weight, bias, affine, eps):
    gamma = torch.abs(weight) + eps if affine else torch.ones_like(var)
    mul = torch.rsqrt(var + eps) * gamma

    x.sub_(broadcast_to(mean, x))
    x.mul_(broadcast_to(mul, x))
    if affine:
        x.add_(broadcast_to(bias, x))

    return x

def _check(fn, *args, **kwargs):
    success = fn(*args, **kwargs)
    if not success:
        raise RuntimeError("CUDA Error encountered in {}".format(fn))




def _reduce(x):
    if len(x.size()) == 2:
        return x.sum(dim=0)
    else:
        n, c = x.size()[0:2]
        return x.contiguous().view((n, c, -1)).sum(2).sum(0)


def _count_samples(x):
    count = 1
    for i, s in enumerate(x.size()):
        if i != 1:
            count *= s
    return count
def leaky_relu_forward(z, slope):

    return torch.nn.functional.leaky_relu(z, negative_slope=slope)

def _act_forward(ctx, x):
    if ctx.activation == ACT_LEAKY_RELU:
        leaky_relu_forward(x, ctx.slope)



class InPlaceABN(autograd.Function):
    @staticmethod
    def forward(ctx, x, weight, bias, running_mean, running_var,
                training=False, momentum=0.1, eps=1e-05, activation=ACT_LEAKY_RELU, slope=0.01):
        # Save context
        ctx.training = training
        ctx.momentum = momentum
        ctx.eps = eps
        ctx.activation = activation
        ctx.slope = slope
        ctx.affine = weight is not None and bias is not None

        # Prepare inputs
        count = _count_samples(x)
        x = x.contiguous()
        weight = weight.contiguous() if ctx.affine else x.new_empty(0)
        bias = bias.contiguous() if ctx.affine else x.new_empty(0)


        mean, var = running_mean.contiguous(), running_var.contiguous()
        ctx.mark_dirty(x)

        # BN forward + activation
        forward_cpu(x, mean, var, weight, bias, ctx.affine,ctx.eps)
        _act_forward(ctx, x)

        # Output
        ctx.var = var
        ctx.save_for_backward(x, var, weight, bias)
        ctx.mark_non_differentiable(running_mean, running_var)
        return x, running_mean, running_var


class InPlaceABNSync(autograd.Function):
    @classmethod
    def forward(cls, ctx, x, weight, bias, running_mean, running_var,
                training=False, momentum=0.1, eps=1e-05, activation=ACT_LEAKY_RELU, slope=0.01, equal_batches=True):
        # Save context
        ctx.training = training
        ctx.momentum = momentum
        ctx.eps = eps
        ctx.activation = activation
        ctx.slope = slope
        ctx.affine = weight is not None and bias is not None

        # Prepare inputs
        ctx.world_size = dist.get_world_size() if dist.is_initialized() else 1

        # count = _count_samples(x)
        batch_size = x.new_tensor([x.shape[0]], dtype=torch.long)

        x = x.contiguous()
        weight = weight.contiguous() if ctx.affine else x.new_empty(0)
        bias = bias.contiguous() if ctx.affine else x.new_empty(0)


        mean, var = running_mean.contiguous(), running_var.contiguous()
        ctx.mark_dirty(x)

        # BN forward + activation
        forward_cpu(x, mean, var, weight, bias, ctx.affine,ctx.eps)
        _act_forward(ctx, x)

        # Output
        ctx.var = var
        ctx.save_for_backward(x, var, weight, bias)
        ctx.mark_non_differentiable(running_mean, running_var)
        return x, running_mean, running_var



inplace_abn = InPlaceABN.apply
inplace_abn_sync = InPlaceABNSync.apply

__all__ = ["inplace_abn", "inplace_abn_sync", "ACT_RELU", "ACT_LEAKY_RELU", "ACT_ELU", "ACT_NONE"]

但，衣服覆盖的不全，这是为什么呢。
forward_cpu代码是"\preprocess\humanparsing\modules\src\inplace_abn_cpu.cpp"中的C++代码转写成python的

您好，我想要把这个ootdfusion的功能给迁移到comfyui当中——

您好，我想要把这个ootdfusion给迁移到comfyui里，有一些问题想要咨询您，为了方便联系，请问能否方便先留个邮箱的联系方式给我呢，或者可以通过[email protected] 或者[email protected]电邮我~ 辛苦老师

如何在本地启动Gradio demo？

OSError: Error no file named config.json found in directory ../checkpoints/ootd/ootd_hd/checkpoint-36000.

You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Traceback (most recent call last):
File "/root/autodl-tmp/OOTDiffusion/run/run_ootd.py", line 47, in
model = OOTDiffusionHD(args.gpu_id)
File "/root/autodl-tmp/OOTDiffusion/ootd/inference_ootd_hd.py", line 61, in init
unet_vton = UNetVton2DConditionModel.from_pretrained(
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/root/autodl-tmp/OOTDiffusion/diffusers/src/diffusers/models/modeling_utils.py", line 569, in from_pretrained
config, unused_kwargs, commit_hash = cls.load_config(
File "/root/miniconda3/envs/ootd/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/root/autodl-tmp/OOTDiffusion/diffusers/src/diffusers/configuration_utils.py", line 370, in load_config
raise EnvironmentError(
OSError: Error no file named config.json found in directory ../checkpoints/ootd/ootd_hd/checkpoint-36000.


          ![image](https://github.com/levihsu/OOTDiffusion/assets/33121696/63351d0e-1f47-40fb-9bfe-9a1cfa4182e2)

I also downloaded the model of openai, my directory structure is like this, execute the following command 'python run_ootd.py --model_path /home/ysm/tmmt/OOTDiffusion/run/examples/model/01008_00.jpg --cloth_path /home/ysm/tmmt/OOTDiffusion/run/examples/garment/00055_00.jpg --scale 2.0 --sample 4', and it works fine

Originally posted by @nkTmmT in #29 (comment)

No module named 'config'

找不到config文件

(Outfit) D:\Python310\Outfit\OOTDiffusion\run>python run_ootd.py --model_path examples\model --cloth_path examples\garment --scale 2.0 --sample 4
Traceback (most recent call last):
File "D:\Python310\Outfit\OOTDiffusion\run\run_ootd.py", line 9, in
from preprocess.openpose.run_openpose import OpenPose
File "D:\Python310\Outfit\OOTDiffusion\preprocess\openpose\run_openpose.py", line 3, in
import config
ModuleNotFoundError: No module named 'config'

How to train for a swimsuit

How to train on swimsuits that are actually treated as t-shirts when I offer swimsuits, and actually treated as long pants when I offer swim trunks
如何对泳装进行训练，当我提供泳衣的时候，实际上被当成了T恤，当我提供泳裤的时候，实际被当成了长裤

Not run in Windows

Not run in Windows
No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1'