facebookresearch / deepfocus

DeepFocus: Learned Image Synthesis for Computational Displays

DeepFocus

This repository provides source code, network models, and datasets (~17GB) for the DeepFocus project from Facebook Reality Labs.

The source code and network models were implemented with TensorFlow with 32-bit precision.

If you use this code and/or dataset, please cite the publication: Lei Xiao, Anton Kaplanyan, Alexander Fix, Matt Chapman, Douglas Lanman, "DeepFocus: Learned Image Synthesis For Computational Displays", SIGGRAPH Asia 2018.

The technical paper, video, and more supplementary materials can be found at: https://research.fb.com/publications/deepfocus-siggraph-asia-2018/

Check out our Oculus blog post that introduces DeepFocus: https://www.oculus.com/blog/introducing-deepfocus-the-ai-rendering-system-powering-half-dome/

License

DeepFocus is CC-BY-NC 4.0 (FAIR License) licensed, as found in the LICENSE file.

deepfocus's Issues

Evaluation not consistent with paper

Hi, I run your "Defocus Blur Rendering" code, but the result of PSNR is not consistent with your paper.
Here is the result of "./varifocal/code/sync_defocus_from_rgbd/test/test-lvf.py" code.

#
# mean PSNR for all test images
44.9162
#
# mean PSNR for each focal stack
44.9162
#
# PSNR for each individual image
44.0976 42.7395 43.9045 43.1361 42.4810 42.3215 42.6423 43.1052 43.3841 43.6683 44.0010 44.3423 44.6130 44.8292 45.0373 45.2088 45.3173 45.3792 45.4023 45.4032 45.4001 45.3951 45.3795 45.3702 45.4438 45.5673 45.6528 45.6338 45.6555 45.6959 45.7070 45.7563 45.8386 45.9289 46.0225 46.1052 46.1730 46.2382 46.3046 46.3645

And here is the result of "./varifocal/code/sync_defocus_from_rgbd_fast/test/test-lvf-c.py" code.

#
# mean PSNR for all test images
44.7768
#
# mean PSNR for each focal stack
44.7768
#
# PSNR for each individual image
45.1251 43.4381 44.2045 43.2073 42.6095 42.5099 42.8436 43.3195 43.5656 43.8031 44.0874 44.3799 44.5926 44.7571 44.8416 44.8101 44.7367 44.7145 44.7582 44.8465 44.9493 45.0352 45.0908 45.1372 45.2383 45.3461 45.4033 45.3818 45.4002 45.4394 45.4685 45.5355 45.6263 45.7154 45.7976 45.8584 45.8910 45.8992 45.8810 45.8284

The PSNRs of 0.1D and 3.0D are set to bold and italic. However, the results of two in your paper are 45.6 and 46.1, respectively. I would like to know why the results are different.

Look forward to your reply.

Need environment setting

I am working to reproduce the code but find it hard to satisfy the environment setting. I would appreciate if you could provide a specified tensorflow version or a requirement.txt 0.0

error when loading RGB

hi, an error occurred as follows when I tried to run by python train-lvf.py:
DeepFocus/varifocal/code/sync_defocus_from_rgbd/train/preproces s.py", line 79, in loadData
ValueError: could not broadcast input array from shape (256,452) into shape (512 ,512)

when using other input datasets, this issue was also encountered.
Thanks.

Download issue

We wanna download the whole project but the data set is a bit larger. Could you please split the data set aside in online drives since Github might have some restriction on downloading huge files and ?