feiran-l / structured-light-stereo

3D scanner with a stereo camera and structured light

Structured-light-stereo

This repo uses a stereo camera and gray-code-based structured light to realize dense 3D reconstruction.

.

How to use:

• STEP 0: pip install opencv-python==4.5.4.60 opencv-contrib-python==4.5.4.60 (Version is necessary since the APIs are changed in higher versions.)

• STEP 1: Setup the stereo camera and calibrate it with a Charuco board. An example dataset is provided in ./data/stereo_calib_flir. You can play with it by running ./capture/charuco_calib.py. The calibration results will be saved in './data/stereo_calib_flir/stereo_calib_data.h5.

• STEP 2: Connect your projector with your computer and run gray_code_encoder.py to project gray-code patterns. I used two FLIR cameras to build my stereo system and the driver script is ./capture_calib/double_flir_capture.py. An example dataset is in ./data/bag.

• STEP 3: Run one of the three gray_code_decoder_*.py scripts to get the reconstructed point clouds. Differences among them are:

  • gray_code_decoder_disp.py: First recover a disparity map and rely on it to construct the point clouds. So the matching is column-direction only. This script can be the primary choice with good stereo calibration.
  • gray_code_decoder_tria_all_pts.py: First respectively construct the pixel-level camera-projector correspondences. Then triangulate the left-right camera pixels that correspond to the same projector's pixel. So the matching is 2D.
  • gray_code_decoder_tria_unique_pts.py: Same as above. But only pixels with unique correspondences are triangulated.

Citation

This repo is part of our DP-simulator project. Please consider to cite our paper if you find this repo useful

@inproceedings{li2023learning,
  title={Learning to Synthesize Photorealistic Dual-pixel Images from RGBD frames},
  author={Li, Feiran and Guo, Heng and Santo, Hiroaki and Okura, Fumio and Matsushita, Yasuyuki},
  booktitle={2023 IEEE International Conference on Computational Photography (ICCP)},
  pages={1--11},
  year={2023},
  organization={IEEE}
}

structured-light-stereo's Issues

Scalloped dots in the point cloud!

Hello, I am generating point cloud data with my own images using gray_code_decoder_disp.py. But the point cloud is scalloped as seen in the photo. In other words, it consists of many small squares parallel to the plane of the camera.

Point Cloud Photos:

Full cloud: https://ibb.co/bWGD1DX
Zoom to small squares : https://ibb.co/f1TMFQc
Zoom squares side view: https://ibb.co/tL9xdqw

Even though the surface of the piece I scanned is completely smooth and flat, the point cloud is formed in this way.

[Question] where is the code snippet for stereo unwrapped phase matching?

'disp_filter' is not defined in gray_code_decoder_disp.py

Hi, SILI1994
Thanks for your great work !
There is something to ask for your advice.
When I run "gray_code_decoder_disp.py", I meet a problem is "NameError: name 'disp_filter' is not defined"
I also can't find any function or py file named by 'disp_filter'.
Could you give me any advice? thank you!

decoding problem

I checked your code carefully. In the decoding section, only the gray_code_decoder_disp.py file
Function. The rest of the files show this error after execution.
Traceback (most recent call last):
File "C:/Users/APA/PycharmProjects/pythonProjectOpen3D/Structured-light-stereo-main/gray_code_decoder_tria_unique_pts.py", line 121, in
error, projPixel = graycode.getProjPixel(pattern_list[0], j, i)
TypeError: Required argument 'projPix' (pos 4) not found

Process finished with exit code 1
In addition, and most importantly, the output file cannot be converted into a mesh. My goal is to use your code to convert the decode of the bag into an .obj file and display it in my program through Mashlab.