alinstein / depth_estimation

Deep learning model to estimate the depth of image.

Monocular Depth Estimation with Transfer Learning pretrained MobileNetV2

This project implements a deep learning neural network model to generate the depth image of a given image. Model is a U-net model with MobileNetV2 as the encoder, and model has utilized skip connection from encoder to decoder. Model generates a depth image of resolution 480x640 for input image of same size.

This project was implemented taking reference from the following paper:

High Quality Monocular Depth Estimation via Transfer Learning (arXiv 2018) [Ibraheem Alhashim] and Peter Wonka

Getting Started

Model is trained using the IPYTHON file "train_mobilenetv2.ipynb".

• Download the dataset and give the location of dataset.
• Change the following according to the needs: batch_size, epochs, lr (learning rate). Load the pretrained model if needed.

IPYTHON file "test_img.ipynb" can be used to generate the depth image on pretrained model.

• Give the location for the dictionary of images to be converted and load the pretrained model

IPYTHON file "test_video.ipynb" can be used to generate the depth video on pretrained model.

• Give the location for the dictionary of images to be converted and load the pretrained model.

Implementation of the Depth estimation using Densenet model is in the folder "Densenet_depth_model".

Dataset

• NYU Depth V2 (50K) (4.1 GB): File is extraced while running the "train_mobilenetv2.ipynb".

Download the pretrained model

• Mobilenet (55 MB). Pretrained model is trained on 2 NVIDIA GeForce GTX 1080 for 6 hours(6 epoches).

Author

Written by Alinstein Jose, University of Victoria.

depth_estimation's Issues

dataset link

Hi @alinstein
can you recheck the link to dataset because it is not working...

Need eval code! Can you provide eval code please, I try to write it by myself , but I can't get the same result as paper provide

[Pretrained model] Wrong URL

Hi there,

Thanks for all your porting to PyTorch!
I tried to download the pretrained model you mention in your README, but there is nothing in the linked Google Drive folder. Would you mind updating the URL please? 🙏

Best

Can I get the real depth value converted from the plasma image?

Hi! I'm really thankful to give your great code.
I'm actually want to get real depth value with my depth prediction image.

For example, I used test_img.ipynb and make depth prediction images.
With your pre-trained model, I put this RGB image.
This RGB image is from rgbd_dataset_freiburg1_xyz.

And I can get a depth prediction image.

I want to get real depth value like this.

I would like to convert it so that it can have a similar value even if it doesn't have an exact depth value.

Are there any formulas or codes that can be applied here?

Link for downing NYU Depth V2 dataset seems to broken

Hi Jose,
Thanks for making this work available. 😊
I am trying to download NYU Depth V2 dataset you mention in your README, but the link seems to be broken. Could you kindly update the URL please?
Cheers

can we get real distance of objects using depth map?

i want to predict whether object is close,moderate, far away distance