pratulsrinivasan / local_light_field_synthesis

Local Light Field Synthesis (Pratul P. Srinivasan, Tongzhou Wang, Ashwin Sreelal, Ravi Ramamoorthi, Ren Ng ICCV 2017)

Learning to Synthesize a 4D RGBD Light Field from a Single Image

Pratul P. Srinivasan, Tongzhou Wang, Ashwin Sreelal, Ravi Ramamoorthi, Ren Ng

In the International Conference on Computer Vision (ICCV) 2017 (Spotlight Oral Presentation)

Paper, Video, Flowers Dataset (~169 GB), Supplementary Material

Example Input 2D Image

Predicted 4D Ray Depths

Synthesized 4D Light Field

Synthesized Synthetic Depth-of-Field (Focused on Flower)

Synthesized Synthetic Depth-of-Field (Focused on Background)

Contents

This repository contains:

1. Local_Light_Field_Synthesis.ipynb Jupyter notebook with an implementation of our algorithm. Note that this code may contain slight updates and modifications to the code used in our paper.

Dependencies

This code depends on a working installation of Tensorflow and basic Python libraries (numpy, scipy, matplotlib).

Acknowledgments

This work was supported in part by ONR grants N00014152013 and N000141712687, NSF grant 1617234, NSF fellowship DGE 1106400, a Google Research Award, the UC San Diego Center for Visual Computing, and a generous GPU donation from NVIDIA.

local_light_field_synthesis's Issues

How to run program

Hello, really thanks for uploading your code, really great work. So, how can I run this code? And how can I put an input image?

are same filters applied across color channels in the occlusion modeling network?

Hi Pratul,
I think the occlusion network implemented in the code is not same as the one in the paper (supplementary material). In Figure 3 of supplementary material, is the depth of intermediate filters 3? are your filters of the size 3,3,3,v*u?

Out_of_range error in RandomShuffleQueue when training

Hi,

I try to run the code and it seems fine in beginning iterations.

However, I got an error when I run some iterations. sometimes 10000+ iterations sometimes thousands of iterations.

2017-10-26 08:50:54.668608: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_1_shuffle_batch_join/random_shuffle_queue' is closed and has insufficient elements (requested 1, current size 0)
	 [[Node: shuffle_batch_join = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch_join/random_shuffle_queue, shuffle_batch_join/n)]]
Traceback (most recent call last):
  File "iccv_tf.py", line 379, in <module>
    trainsummary = sess.run(merged, feed_dict={is_training:True})
  File "/home/mb55411/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/home/mb55411/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1124, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/mb55411/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
    options, run_metadata)
  File "/home/mb55411/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_1_shuffle_batch_join/random_shuffle_queue' is closed and has insufficient elements (requested 1, current size 0)
	 [[Node: shuffle_batch_join = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch_join/random_shuffle_queue, shuffle_batch_join/n)]]
	 [[Node: shuffle_batch_join/random_shuffle_queue_Size/_1001 = _HostRecv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_2352_shuffle_batch_join/random_shuffle_queue_Size", tensor_type=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

Caused by op u'shuffle_batch_join', defined at:
  File "iccv_tf.py", line 323, in <module>
    aif_batch, lf_batch = input_pipeline(train_filenames, lfsize, patchsize, batchsize, num_crops)
  File "iccv_tf.py", line 313, in input_pipeline
    [patchsize[0], patchsize[1], lfsize[2], lfsize[3], 3]])
  File "/home/mb55411/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 1362, in shuffle_batch_join
    name=name)
  File "/home/mb55411/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 828, in _shuffle_batch_join
    dequeued = queue.dequeue_many(batch_size, name=name)
  File "/home/mb55411/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 457, in dequeue_many
    self._queue_ref, n=n, component_types=self._dtypes, name=name)
  File "/home/mb55411/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 1342, in _queue_dequeue_many_v2
    timeout_ms=timeout_ms, name=name)
  File "/home/mb55411/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
    op_def=op_def)
  File "/home/mb55411/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/mb55411/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1204, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

OutOfRangeError (see above for traceback): RandomShuffleQueue '_1_shuffle_batch_join/random_shuffle_queue' is closed and has insufficient elements (requested 1, current size 0)
	 [[Node: shuffle_batch_join = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch_join/random_shuffle_queue, shuffle_batch_join/n)]]
	 [[Node: shuffle_batch_join/random_shuffle_queue_Size/_1001 = _HostRecv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_2352_shuffle_batch_join/random_shuffle_queue_Size", tensor_type=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

I am new in tensorflow and any idea about this error?
Thanks in advance!

Hey

About the toys dataset

Thank you for your excellent work. May I ask for a link to the toy dataset in the paper?