ntad27 / aip490

AIP490 - Virtual Shirt Fitting Model Based on Deep Learning and Computer Vision

Model's architecture

The figures below show the architecture of the model from the data preprocessing step to the main HR-VITON model we used in this project.

Dataset

We train and evaluate our model using the dataset from High-Resolution Virtual Try-On with Misalignment and Occlusion-Handled Conditions, the original dataset was from VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization.

To download the datasets, please check the links below:

• Author's dataset: link
• Our preprocessed dataset: link

After you download the dataset, create a ./data folder and put it under.

If you want to know more about how we preprocess the data, please check the Preprocessing.md

Inference

Here are the download links for each model checkpoint:

• Author's try-on condition generator: link
• Our retrain 30000 steps try-on condition generator: link
• Author's try-on image generator: link
• We assume that you have obtained all the checkpoints and stored them in ./eval_models/weights/v0.1.

We've built a web app demo, please check the following notebook for detailed instructions

Train try-on condition generator

python train_condition.py --name test --cuda {True} --gpu_ids {gpu_ids} --dataroot {dataroot_path} --datamode train --data_list {path_to_trainpairs.txt} --keep_step 300000 --Ddownx2 --Ddropout --lasttvonly --interflowloss --occlusion

Train try-on image generator

python train_generator.py --cuda {True} --name test -b 4 -j 8 --gpu_ids {gpu_ids} --fp16 --tocg_checkpoint {condition generator ckpt path} --occlusion --dataroot {data_path}

To use "--fp16" option, you should install the apex library.

Acknowledgements

• Tran Ngoc Xuan Tin ([email protected])
• Duong Vien Thach ([email protected])
• https://github.com/sangyun884/HR-VITON
• https://github.com/shadow2496/VITON-HD
• https://github.com/plemeri/transparent-background
• https://github.com/CMU-Perceptual-Computing-Lab/openpose
• https://github.com/facebookresearch/detectron2/tree/main/projects/DensePose
• https://github.com/Engineering-Course/CIHP_PGN