Coder Social home page Coder Social logo

unsup3d_pytorch3d's Introduction

(WIP)unsup3D_pytorch3d (CVPR2020 Best paper)

Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi, Visual Geometry Group, University of Oxford. In CVPR 2020 (Best Paper Award).

We propose a method to learn weakly symmetric deformable 3D object categories from raw single-view images, without ground-truth 3D, multiple views, 2D/3D keypoints, prior shape models or any other supervision.

〇 Notice

author: samuel ko

This repo is target to replace the neural_renderer from CVPR 2018 with the sota renderer ICCV 2019 Soft Rasterizer provided by pytorch3d. In order to compatible with the pytorch3d API, we change a little bit in unsup3d/renderer/renderer.py & unsup3d/renderer/utils.py.

However, the effect of this implemention is not good. I will still work on this and try to fix that problem.

SoftRas(trained on celebA)

① Setup (with Anaconda)

1. Install dependencies:

conda env create -f environment.yml

OR manually:

conda install -c conda-forge scikit-image matplotlib opencv moviepy pyyaml tensorboardX

2. Install PyTorch:

conda install pytorch==1.4.0 torchvision cudatoolkit=10.0 -c pytorch

Note: The code is tested with PyTorch 1.4.0 and CUDA 10.0 on Ubuntu 18.04. A GPU version is required for training and testing, since the neural_renderer package only has GPU implementation.

3. Install pytorch3d instead of original repo neural_renderer

conda install --channel https://conda.anaconda.org/pytorch3d pytorch3d

4. (For demo only) Install facenet-pytorch:

This package is optional for the demo. It allows automatic human face detection.

pip install facenet-pytorch

② Datasets

  1. CelebA face dataset. Please download the original images (img_celeba.7z) from their website and run celeba_crop.py in data/ to crop the images.
  2. Synthetic face dataset generated using Basel Face Model. This can be downloaded using the script download_synface.sh provided in data/.
  3. Cat face dataset composed of Cat Head Dataset and Oxford-IIIT Pet Dataset (license). This can be downloaded using the script download_cat.sh provided in data/.

Please remember to cite the corresponding papers if you use these datasets.

③ Demo

python -m demo.demo --input demo/images/human_face --result demo/results/human_face --checkpoint pretrained/pretrained_celeba/checkpoint030.pth

④ Training and Testing

Check the configuration files in experiments/ and run experiments, eg:

python run.py --config experiments/train_gdh.yml --gpu 0 --num_workers 2

⑤ Citation

@InProceedings{Wu_2020_CVPR,
  author = {Shangzhe Wu and Christian Rupprecht and Andrea Vedaldi},
  title = {Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild},
  booktitle = {CVPR},
  year = {2020}
}

unsup3d_pytorch3d's People

Contributors

tomguluson92 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

unsup3d_pytorch3d's Issues

which pytorch3d version do U used

Thanks for your great work!
Unfortunately, I met some problems when run the demo.py
when I used the pretrained celeba model from the original unsup3d project and then run the demo of yours, the render is broken
图片

I used pytorch 1.5.0 cuda 10.1 and pytorch3d 0.2.5 or 0.3.0 (both are broken)

Excellent work!

Have you figured out the reason of poor results of PyTorch3D?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.