Comments (11)
Thanks for your interest! For one specific (rgb, sd, gt) tuple, the supervision is semi-dense and there're many pixels where supervision doesn't exist. However, for the whole dataset, on most of the pixels in the 2D-image coordinate the network receives supervision from different data samples. And we should also notice some pixels where there're truly no supervision, like those in the top of the full image (out of the FoV of the LiDAR).
from penet_icra2021.
However, for the whole dataset, on most of the pixels in the 2D-image coordinate the network receives supervision from different data samples
I am so happy to receive your prompt reply!
I still don't quite understand, as shown in the picture below, how to ensure the accuracy of this depth in the absence of supervision(no GT)? Could you be more detailed? Thank you!
from penet_icra2021.
One of my interpretations is that the network randomly generates dense depth maps and then constrains only the supervised parts, and when the supervised parts are accurate, the rest(no GT) should also be accurate (possibly taking advantage of some hidden properties of the image). I don't know if I'm right.
from penet_icra2021.
However, for the whole dataset, on most of the pixels in the 2D-image coordinate the network receives supervision from different data samples
I am so happy to receive your prompt reply! I still don't quite understand, as shown in the picture below, how to ensure the accuracy of this depth in the absence of supervision(no GT)? Could you be more detailed? Thank you!
In this case, we cannot guaranty the depth values there is accurate as there're no nearby input LiDAR points. But it does not mean that it owes to the lack of supervision that we can't get reliable depth here. The accuracy of depth values of a specific predicted depth is more related to the input sparse depth. It has very minor relationship with its supervision. However, it is related to the overall distribution of supervision data in the whole dataset.
from penet_icra2021.
Hi, haven't seen you for a while.
I'm still interested in this problem, and this Friday, I trained a simple deep completion model using sparse loss functions (paper).
But here's what you get.
As shown in the figure below, no reliable depth is generated in the unsupervised part.
Why is that?
Ask for help,thank you!!!
(GT at the top and results at the bottom)
from penet_icra2021.
show me more detail:
network: input img+depth(HXWX4),simple encoder-decoder
dataset: kitti detection task dataset(not kitti depth dataset)
loss function: your code loss. sparse
result:unsupervised depth is zero?(The origin part)
from penet_icra2021.
Hi, haven't seen you for a while. I'm still interested in this problem, and this Friday, I trained a simple deep completion model using sparse loss functions (paper). But here's what you get. As shown in the figure below, no reliable depth is generated in the unsupervised part. Why is that? Ask for help,thank you!!! (GT at the top and results at the bottom)
How do you get ground truth data in the object detection dataset? In supervised depth completion, models can predict relatively reliable depth maps because the groundtruth data is multiple times denser than the inputs. In unsupervised depth completion, addition loss items such as photometric loss (minimum the re-projection error between sequential frames) and smoothness loss is required. If you take the inputs as the groundtruth data for training a single-frame depth completion model, I think the model will probably not work.
from penet_icra2021.
I projected a depth map (0 < value < 80) through lidar points, and then concated it with RGB and sent it directly to the network for learning. I don't know much about deep completion, and there may be some common sense mistakes.So, how should I make it work? I understand. After downsampling the original depth map(GT)(e.g 12w point decreased 4w point), concated it with RGB and sending it to the network for learning, is that right?
from penet_icra2021.
I have not conducted experiments under a similar setting. But I guess there could be two main reasons: (1) The groundtruth data you used is not dense enough to support the convergence of a single-image depth completion model. (2) Not all pixels are reliable. The noisy pixels should be filtered out before adapting to downstream tasks. You could refer to a previous issue about this.
from penet_icra2021.
You could refer to a previous issue about this.
hello, I read your paper found kitti depth complete task raw datasets and 3d object datasets have large differents.
raw datasets have 86k image and temporal(5% sparse depth and 16% depth GT). 3d object datasets have 7.8k and non-temporal. At this point, I understand your first reply.
Now,I'd like to share with you my latest progress. I seem to have made the network work through sampling and data enhancement in some ways, as shown in the picture, the car's 3D points are starting to dense.But there is still some noise (this seems to be the general noise of depth completion?)..I am interested in confidence map solve noise (I want to try some simple methods first), can you recommend this article, there is a code is best > < thank you!!!
from penet_icra2021.
You could refer to [1]【PNConv-CNN】CVPR2020 Uncertainty Aware CNNs for Depth Completion Uncertainty from Beginning to End. [2] 【PwP】ICCV2019 Depth Completion From Sparse LiDAR Data With Depth-Normal Constraints. for better understanding of predicting a confidence map.
from penet_icra2021.
Related Issues (20)
- Projection result confirmation HOT 3
- Questions about KITTI-odometry HOT 2
- Runtime measurement HOT 4
- How to use the sparse depth?
- How to use the sparse depth? HOT 4
- Modify the backbone network HOT 3
- lightweight deployment of PENet network
- How to infer PENet for KITTI object task? HOT 3
- broken PNG file HOT 1
- the difference intrinsic parameters between train and test HOT 4
- Modify DA-CSPN++ for single branch input HOT 1
- some questions about the implement of CSPN HOT 4
- Model mismatch at inference time HOT 1
- Tensor Dimension Mismatch HOT 1
- How many parameters dose PENet have? HOT 2
- About training HOT 1
- How to implement the model with ROS?
- can't download the pretrained PENet Model
- Pretrained on NYU dataset?
- RuntimeError: CUDA out of memory.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from penet_icra2021.