piiswrong / deep3d Goto Github PK
View Code? Open in Web Editor NEWAutomatic 2D-to-3D Video Conversion with CNNs
Automatic 2D-to-3D Video Conversion with CNNs
I wanna ask that how to initialize the weight of deconvolution filter to bilinear interpolati as you did In your newest paper "Deep3D: Automatic 2D-to-3D Video Conversion with CNNs"?
Thanks a lot!
I ran the convert_movie.py file and it generateed this error. It seems that it is because of the version of opencv I used. Can you please check which opencv version are you using? I am using v2.4.8. Thanks a lot!
Hi,
Can someone tell ma what the structure is creating databases for deep3d, and how to use this to train a model.
thanks,
laszlo
Hello,I can't find these:
http://homes.cs.washington.edu/~jxie/download/deep3d-0050.params &&
eep3d-0050.params
I don't know what's the matter.
can you send me your params? Tank you !
mail to : [email protected]
I got this error when add depth-dot code and recomplie mxnet, and change 'type' to 'mode' in softmax
I'd like to train the network using KITTI stereo datasets instead of stereo movies. is that somehow possible - and if yes, can someone give me some pointers? thanks, Ito
Can you provide a more detailed instruction on how to run your code?
I have got an error when I used raw_shape instead of shape as below code
shape = (384, 160)
img = cv2.imread('demo.jpg')
raw_shape = (img.shape[1], img.shape[0])
img = cv2.resize(img, raw_shape)
Thank you piiswrong
what is the mean? i hope to get you help
"Then, append EXTRA_OPERATORS=path/to/deep3d/operators to path/to/mxnet/config.mk and recompile MXNet."
we cann't connect the url https://www.cs.washington.edu/~jxie/.
pls help me.
@piiswrong
I use the parse.py and data.py to create my own database. I just give one prefix. But when I run the data.py, it needs to get the prefix_list,. How can I solve it ? Thanks.
The latest mxnet seems not have the DepthDot operator
Can this algorithm be used to render a 2D video stream into 3D in real time?
would love to try generate the parameters by training module, are you releasing it?
thanks!
Hi ,@piiswrong
[15:44:47] /home/git/mxnet/mxnet/dmlc-core/include/dmlc/logging.h:235: [15:44:47] src/operator/./cudnn_softmax_activation-inl.h:44: Check failed: (in_data[softmax_activation::kData].ndim()) == (2) Input need to have 2 dimensions when mode=instance.
[15:44:47] /home/git/mxnet/mxnet/dmlc-core/include/dmlc/logging.h:235: [15:44:47] src/engine/./threaded_engine.h:306: [15:44:47] src/operator/./cudnn_softmax_activation-inl.h:44: Check failed: (in_data[softmax_activation::kData].ndim()) == (2) Input need to have 2 dimensions when mode=instance.
cuda7.5+cudnn5.0
I get this result when I run "python ./train.py".
How to solve this problem, thank you
When I run the code to the place(in data.py next member function for class Mov3dStack):
for j in range(max(1, self.data_frames)):
sl = txn.get('%09d'%(idx+(j-self.data_frames/2)_self.stride), db=self.ldb)
if sl is None:
pass
else:
_, s = mx.recordio.unpack(sl)
mx.nd.imdecode(s, clip_rect=(p[0], p[1], p[0] + self.data_shape[0], p[1] +self.data_shape[1]),
out=ndleft, index=i_self.data_frames+j, channels=3, mean=self.left_mean_nd)
Then error thrown out as :
File "/media/lqzhu/e/deep3d-master/data.py", line 215, in load_mean
for batch in data_iter:
File "/media/lqzhu/e/deep3d-master/data.py", line 392, in next
out=ndleft, index=i*self.data_frames+j, channels=3, mean=self.left_mean_nd)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.7.0-py2.7.egg/mxnet/ndarray.py", line 1034, in imdecode
out=out)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.7.0-py2.7.egg/mxnet/ndarray.py", line 1172, in generic_ndarray_function
c_array(ctypes.c_char_p, [str(i).encode('ascii') for i in kwargs.values()])))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
I dig in the web and no useful dips can fix this problem, I've tried adding below lines:
import sys
reload(sys)
sys.setdefaultencoding('utf8')
Still doesn't work.
Any suggestion?
I would like to know if it is still available these network parameters?
Best,
Goffredo
I notice there is a variable named 'source' in convert movie.py , what is its meaning actually? What is the test_idx? And if i want to use the code to directly input a video's name, what extra work should be done to produce a 3D movie rather than a simple gif?
if not os.path.exists('deep3d-0050.params'):
urllib.urlretrieve('http://homes.cs.washington.edu/~jxie/download/deep3d-0050.params', 'deep3d-0050.params')
model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))
The error gives:
---------------------------------------------------------------------------
MXNetError Traceback (most recent call last)
<ipython-input-7-9deaeed89c01> in <module>()
1 if not os.path.exists('deep3d-0050.params'):
2 urllib.urlretrieve('http://homes.cs.washington.edu/~jxie/download/deep3d-0050.params', 'deep3d-0050.params')
----> 3 model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))
/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.pyc in load(prefix, epoch, ctx, **kwargs)
832 - ``prefix-epoch.params`` will be saved for parameters.
833 """
--> 834 symbol, arg_params, aux_params = load_checkpoint(prefix, epoch)
835 return FeedForward(symbol, ctx=ctx,
836 arg_params=arg_params, aux_params=aux_params,
/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.pyc in load_checkpoint(prefix, epoch)
361 - parameters will be loaded from ``prefix-epoch.params``.
362 """
--> 363 symbol = sym.load('%s-symbol.json' % prefix)
364 save_dict = nd.load('%s-%04d.params' % (prefix, epoch))
365 arg_params = {}
/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/symbol.pyc in load(fname)
932 raise TypeError('fname need to be string')
933 handle = SymbolHandle()
--> 934 check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
935 return Symbol(handle)
936
/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/base.pyc in check_call(ret)
75 """
76 if ret != 0:
---> 77 raise MXNetError(py_str(_LIB.MXGetLastError()))
78
79 def c_str(string):
MXNetError: Failed loading Op softmax of type SoftmaxActivation: Cannot find argument 'type', Possible Arguments:
----------------
mode : {'channel', 'instance'},optional, default='instance'
Softmax Mode. If set to instance, this operator will compute a softmax for each instance in the batch; this is the default mode. If set to channel, this operator will compute a num_channel-class softmax at each position of each instance; this can be used for fully convolutional network, image segmentation, etc.```
It looks like:
```sym = symbol.load('deep3d-symbol.json')
How to fix it?
Thanks
Hi,
I'm getting this error: Find name bn_pool3_moving_inv_var that is not in the auxiliary states
Would you know how to resolve it?
Thanks!
I had create a database using parse.py and data.py, but when I run train.py I got this error:
$ python train.py [01:08:37] include/dmlc/logging.h:235: [01:08:37] src/io/local_filesys.cc:149: Check failed: allow_null LocalFileSystem: fail to open "vgg16-0001.params" Traceback (most recent call last): File "train.py", line 92, in <module> train(64, 'exp/deep3d') File "train.py", line 65, in train vgg16 = data.load_vgg(data_frames, flow_frames, two_stream=False) File "/home/salim/deep3d/data.py", line 443, in load_vgg vgg16 = {name: arr for name, arr in mx.nd.load('vgg16-0001.params').items() if name.startswith('arg:conv')} File "/usr/local/lib/python2.7/dist-packages/mxnet-0.7.0-py2.7.egg/mxnet/ndarray.py", line 956, in load ctypes.byref(names))) File "/usr/local/lib/python2.7/dist-packages/mxnet-0.7.0-py2.7.egg/mxnet/base.py", line 77, in check_call raise MXNetError(py_str(_LIB.MXGetLastError())) mxnet.base.MXNetError: [01:08:37] src/io/local_filesys.cc:149: Check failed: allow_null LocalFileSystem: fail to open "vgg16-0001.params"
What file is that vgg16-0001.params???? What have to do make it work?
Thanks.
i build the code in window7 with vs2013, and got .dll and .lib , but i con not know how to run it to see the result ,can you give me a favor?
mxnet 2.0.0
Please, include a License.
I was wondering how much VRAM this network requires for the batchsize 64? The reason I ask is that I am working on porting this to tensorflow for a class project, and am currently restricted to a batch size of 32 due to memory issues on a K80 with ~11.2 gb available VRAM.
Also, were all the operations performed on GPU in this mxnet implementation? Any information you could provide on memory usage would be much appreciated!
Dear,
I am using visual studio 2015 with python tools, all the imports in the code are running well except mxnet and leveldb
I think I have an issue in setting up mxnet I followed the official document and still there is an error can you help please
Regards,
I run the code according to the deep3d.ipynb, but i found the result was not as good as your sample which you provided (that is the demo.gif).
can you do me a favor and tell me what can i do to get a result as good as you provided.
thank you!
I'd like to train the network using depth maps as ground truths. is that somehow possible - and if yes, can someone give me some pointers? thanks, laszlo
Hi,
anyone can tell me the right, TESTED and WORKING configuration under Ubuntu?
This config is work???
Ubuntu 16.04LTS -64bit
Cuda 8
DNN 5.1
Nvidia 1080-8gb
mxnet from git
deep3d from git
Best
Moty
Hi,
are there any plans to update the code with the newest version of MxNet and with Cuda 5?
Cheers!
When I run the demo, I encounter the error: /mxnet/dmlc-core/include/dmlc/logging.h:245: [18:04:24] src/operator/operator.cc:19: Cannot find Operator CuDNNBatchNorm in registry.
Thanks a lot!
Hi @piiswrong
Looks the pre-trained model is difficult to run with newer version of cuda and cudnn. Maybe I would try to train a new deep3d model with cuda 8.0 and cudnn 5.1. Can you please give me some suggestions about this task?
Thanks very much!
Hi,
I'm trying to run convert_movie.py using a single jpg as an input file and the default params file, however, I get the above error. Has anyone had this problem before?
thanks
laszlo
edit:
I set my args such as (all my paths and files are correct):
python convert_movie.py C:\dev\mxnet\deep3d\param --ctx 0 --source C:\dev\mxnet\deep3d\image\someImage.jpg --output C:\dev\mxnet\deep3d\output\out.jpg
Although I mansged to run through the test code with my own dataset, the result is very poor, the right eye images are rather blur compared with left one. Could you please share the file mean.npz ? My test data set is very small and may be the mean values generated from it are not very good?
Hi,
I can't find the dataset (3d movies) mentioned in the paper. Can you illustrate how to download the dataset?
when I run the' Y = model.predict(test_iter)' ,which has NameError: name 'model' is not defined. how to solve it ?
MXNetError: [16:38:44] src/operator/cudnn_batch_norm.cc:20: CuDNNBatchNorm is merged into BatchNorm for cudnn version above v5.Use the later instead.
any working solution? or patch?
From your README.md
:
Following each image, there are 4-by-3 maps of depth layers, ordered from near
to far. You can see that objects that are near to you appear in the first depth
maps and objects that are far away appear in the last ones. This shows that the
internal depth representation is learning to infer depth from 2D images without
been directly trained on it.
Where in the code do you generate depth layers from an image?
[12:38:08] /root/mxnet/dmlc-core/include/dmlc/logging.h:245: [12:38:08] src/operator/operator.cc:19: Cannot find Operator CuDNNBatchNorm in registry
Traceback (most recent call last):
File "myMain.py", line 23, in
model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.py", line 822, in load
symbol, arg_params, aux_params = load_checkpoint(prefix, epoch)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.py", line 362, in load_checkpoint
symbol = sym.load('%s-symbol.json' % prefix)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.5.0-py2.7.egg/mxnet/symbol.py", line 886, in load
check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.5.0-py2.7.egg/mxnet/base.py", line 77, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: Failed loading Op bn_pool1 of type CuDNNBatchNorm: [12:38:08] src/operator/operator.cc:19: Cannot find Operator CuDNNBatchNorm in registry
cuda7.5 cudnn4
running by official mxnet docker gpu image
Thank you piiswrong
Can you give me some codes about that? My ability is so poor that I can't find the way to output these maps. I try to output these maps from the deconvolution which is behind some pooling layer, but I failed, I can't find the way to output some values such as pred1, pred2, and so on in sym.py from the module. I just want the method or codes of the output of the 4-by-3 maps from the depth layers in your paper. Can you help me? Thank you.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.