Download data and create environment.
git clone https://github.com/JimothyJohn/ARF-svox2
cd ARF-svox2/
. ./download_data.sh
. ./create_env.sh
Extract frames from your 30s video.
ffmpeg -i input.mp4 -vf fps=5 images/%04d.png
Extract pose data from your images.
git clone https://github.com/Fyusion/LLFF
cd LLFF/
docker pull bmild/tf_colmap
docker run --gpus all -it --rm \
--shm-size=1g --ulimit memlock=-1 \
--ulimit stack=67108864 \
-v $(pwd):/home \
-w /home \
bmild/tf_colmap:latest
python imgs2poses.py images/
cd ../
Create model from your images/poses.
cd opt && . ./try_llff.sh images/ [style_id]
Project page: https://www.cs.cornell.edu/projects/arf/
Citation:
@misc{zhang2022arf,
title={ARF: Artistic Radiance Fields},
author={Kai Zhang and Nick Kolkin and Sai Bi and Fujun Luan and Zexiang Xu and Eli Shechtman and Noah Snavely},
year={2022},
booktitle={arXiv},
}
. ./create_env.sh
. ./download_data.sh
cd opt && . ./try_{llff/tnt/custom}.sh [scene_name] [style_id]
- Select
{llff/tnt/custom}
according to your data type. For example, usellff
forflower
scene,tnt
forPlayground
scene, andcustom
forlego
scene. [style_id].jpg
is the style image inside./data/styles
. For example,14.jpg
is the starry night painting.- Note that a photorealistic radiance field will first be reconstructed for each scene, if it doesn't exist on disk. This will take extra time.
The optimized artistic radiance filed is inside opt/ckpt_arf/[scene_name]_[style_id]
, while the photorealistic one is inside opt/ckpt_svox2/[scene_name]
.
Please follow the steps on Plenoxel to prepare your own custom data.
- ARF-TensoRF: to be released; stay tuned.
- ARF-NeRF: to be released; stay tuned.
We would like to thank Plenoxel authors for open-sourcing their implementations.