Lack of masked depth map about stereo_sparse_depth_fusion

[ShreyasSkandanS]

Hi @hu2011, can you post a few examples of the bad fusion results with the raw KITTI data. I used the Stereo 2015 dataset for this work because the "bad" LiDAR measurements have been removed from the Ground Truth measurements, whereas I believe the raw data is not filtered. This could potentially cause poor disparity images. If you post a few examples I'm happy to take a look.

[hu2011]

Hi @ShreyasSkandanS yes, you are right. There is big difference between the obtained disparity from raw lidar and ground truth disparity. For example(2011_09_26_drive_0009_sync frame 384, is 000010_10.png in stereo2015), here are 22 thousand points on left image from lidar, but only 9 thousand efficent points on the same coordinates on ground truth disparity. It is almost impossible to get the same results from the raw lidar data.

The bad fused image (from diffusion )is like this:

[ShreyasSkandanS]

Hi @ShreyasSkandanS , I'm so sorry for getting wrong velodyne frame by pykitti(384). The new result is shown below, and it looks a little better, still need some process.

using the obtained disparity from lidar and the gt_disparity as evaluation reference. the run results:
{EVALUATION A:} -- SEMI GLOBAL MATCHING --
{status} no fusion. Semi global matching only..
Total ground truth points: 111664
{SGM} avg error: 1.2602

{EVALUATION B:} -- NAIVE LIDAR FUSION --
Total ground truth points: 111664
{NAIVE FUSION} avg error: 1.20339

{EVALUATION C:} -- DIFFUSION BASED --
Total ground truth points: 111664
{DB} avg error: 0.953724

{EVALUATION D:} -- NEIGHBORHOOD SUPPORT --
Total ground truth points: 111664
{NS} avg error: 0.9042

This looks more reasonable, but the output of this - is this from SGM only? or is this one Evaluation C or Evaluation D?

[hu2011]

Hi @ShreyasSkandanS , this result is from DIFFUSION BASED FUSION (evaluation C). Do you have any suggestions to get a better disparity image like in your paper? Thanks

[ShreyasSkandanS]

Hi @hu2011,

Could you post pictures of the following: Input RGB image, Sparse Depth input, Semi Global Matching and Diffusion Based fusion?

Best regards,
Shreyas

[hu2011]

@ShreyasSkandanS hi, sorry for late reply.
Input left RGB

right RGB image

Input sparse depth (from lidar, maybe too sparse to see)

Output of sgm

output of diffusion based fusion. It is strange that here is very bright area (disparity values > 240) on the right signal light.

[ShreyasSkandanS]

This might have something to do with the input LiDAR scan. But the fusion seems to be working (if you look at the circular arrow sign). The errors at the top of the image appear to just be problems with SGM in those regions. They aren't improved because there are no LiDAR points there.

[ShreyasSkandanS]

Since this is no longer an issue related to this work, I am proceeding to close this. If you have any other issues, please feel free to raise another issue. Best regards, SS.