Specifying Object Attributes and Relations in Interactive Scene Generation
A PyTorch implementation of the paper Specifying Object Attributes and Relations in Interactive Scene Generation
Paper
Specifying Object Attributes and Relations in Interactive Scene Generation
Oron Ashual1, Lior Wolf1,2
1 Tel-Aviv University, 2 Facebook AI Research
The IEEE International Conference on Computer Vision (ICCV), 2019, (Oral)
Network Architechture
Youtube
Usage
1. Create virtual environment (optional)
All code was developed and tested on Ubuntu 18.04 with Python 3.6 (Anaconda) and PyTorch 1.0.
conda create -n scene_generation python=3.7
conda activate scene_generation
2. Clone the repository
cd ~
git clone [email protected]:ashual/scene_generation.git
cd scene_generation
3. Install dependencies
conda install --file requirements.txt -c conda-forge -c pytorch
- install pytorch which will fit your CUDA TOOLKIT
4. Install COCO API
Note: we didn't trained our models with COCO panoptic dataset, the coco_panoptic.py code is for the sake of the community only.
cd ~
git clone https://github.com/cocodataset/cocoapi.git
cd cocoapi/PythonAPI/
python setup.py install
cd ~/scene_generation
5. Train
$ python train.py
6. Encode the Appearance attributes
python scripts/encode_features --checkpoint TRAINED_MODEL_CHECKPOINT
7. Sample Images
python scripts/sample_images.py --checkpoint TRAINED_MODEL_CHECKPOINT --sample_features 1 --batch_size 32 --output_dir OUTPUT_DIR
8. or Download trained models
Download these files into models/
7. Play with the GUI
The GUI was built as POC. Use it at your own risk:
python scripts/gui/simple-server.py --checkpoint YOUR_MODEL_CHECKPOINT --output_dir [DIR_NAME] --draw_scene_graphs 1
Citation
If you find this code useful in your research then please cite
@InProceedings{Ashual_2019_ICCV,
author = {Ashual, Oron and Wolf, Lior},
title = {Specifying Object Attributes and Relations in Interactive Scene Generation},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019}
}
Acknowledgement
Our project borrows some source files from sg2im. We thank the authors.