Comments (6)
Hi @NingMingHao ,
I can't find the computer_vision_utils file, so I self-defined the read_image
read_image
refers to this implementation. Probably we should add this kind of utils functions inside the dreyeve repo, thanks for pointing it out.
is the 'dreyeve_mean_frame.png' generated as the mean of the first 37 runs?
It is indeed.
semseg_branch also performs badly
I don't understand if the problem lies in the semantic segmentation network or in the dreyeve branch that predicts the gaze from the segmentation. Anyway, two suggestions:
- Use a more recent pretrained network for segmentation, such as the mapillary model used here.
- Remove the segmentation branch to sanity-check that the rest of your model works properly. From ablation study in the paper (Table 4) you can notice that the segmentation branch is the one that contributes the less to the final result; you can safely disable it to see if things still work.
Best,
A
from dreyeve.
Really thanks for your rapid reply. And I will try your suggestion to remove the segmentation branch.
The reason why I upload my models.py is that I'm using tensorflow as the backend of keras, it's a pity that tensorflow doesn't support to resize a tensor using scale_ratio, so you can find that I have commented some your code, and used
coarse_h = Lambda(lambda x: tf.transpose(tf.image.resize_bilinear(tf.transpose(x,perm=[0,2,3,1]),[_w*4,_w*4],name='{}_4x_upsampling'.format(branch)),perm=[0,3,1,2]))(coarse_h)
Maybe there is something wrong here, and I will check it out.
Thanks again!
from dreyeve.
I have test this code:
coarse_h = Lambda(lambda x: tf.transpose(tf.image.resize_bilinear(tf.transpose(x,perm=[0,2,3,1]),[_w*4,_w*4],name='{}_4x_upsampling'.format(branch)),perm=[0,3,1,2]))(coarse_h)
I'm sure it works properly,
And finally, I have a look at your model weights, I find that all the convolutional bias weights of SaliencyBranch are 0, I'm not sure if this is the reason?
finally, this is the output the im_net gives.
from dreyeve.
@ndrplz Hi, I am trying to run predict_dreyeve_sequence.py file. It requires dreyeve_mean_frame.png
file. How can I generate this file?
from dreyeve.
Hi @varunjammula
you can generate it as the average of all frames of all sequences of the training set.
from dreyeve.
Thanks for clarifying the issue above. I have a new issue now. Where can I get dreyevenet_model_central_crop.h5
model?
from dreyeve.
Related Issues (20)
- re-create the DrEYEve dataset ground truth HOT 2
- to compute optical flow matlab code HOT 1
- Coarse2FineTwoFrames.m source code HOT 1
- SVISINIT problem HOT 5
- Pre-trained coarse weights for segmentation branch? HOT 1
- Where can I get `dreyevenet_model_central_crop.h5` model HOT 4
- semseg predictions HOT 4
- dreyeve_mean_frame.png HOT 1
- Package dependency list HOT 2
- dataset HOT 4
- the use of GPU HOT 2
- Camera calibration HOT 1
- Demo HOT 1
- CUDA_ERROR_OUT_OF_MEMORY: out of memory
- CUDA_ERROR_OUT_OF_MEMORY: out of memory
- how to download the DR(eye)VE data? HOT 2
- how to download the DR(eye)VE data?
- Where is your gt_fix?
- Is it possible to project the images and attention points into BEV?
- how to use GPU? which version of Python, cuda, keras, and Theano should I use
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dreyeve.