This repository contains the code for running FlowBased IBVS and the results obtained.
-
Code : This folder contains the main code for running FlowBased IBVS.
This folder contains the code that runs the habitat simulator and ibvs controller.
This folder contains the implementation of the Flownet2 in tensorflow taken from [https://github.com/sampepose/flownet2-tf]. We have made changes to the net.py files so as to use the flownet's output as part of our pipeline.
-
Data : This folder contains the results for various experiments performed.
Please download the FlowNet2 folder from the following link and store it in Code/flownet2-tf/src/flownet2/
https://drive.google.com/open?id=13rvb_HCWc673C43YVZurwlxJTdX42UUd
Final path of accessing the downloaded folder should be Code/flownet2-tf/src/flownet2/FlowNet2
Save the input(initial) and output(desired) images in the folder : Code/flownet2-tf/src/image_baseline_2 as initial_image.png and desired_image.png.
NOTE: For the initial depth image in case of flowdepth, store it as initial_depth.png in the same folder.
Run the below commands in 2 separate terminals in the following order .
[Server Code]
cd Code/flownet2-tf
python -m src.flownet2.test_flow_depth (for flow depth)
(or)
python -m src.flownet2.test_depth_net (for depth network)
[Client Code]
cd Code/habitat_sim_client
python example_me_depth_net.py --width 512 --height 384 --scene path/to/habitat/scene/name.glb --save_png --depth_sensor --max_frames 20
(or)
python example_me_flow_depth.py --width 512 --height 384 --scene path/to/habitat/scene/name.glb --save_png --depth_sensor --max_frames 20
The quantitative results such as photoerror, velocities predicted in every iteration are stored in the folder : Code/flownet2-tf/src/aaaaa
The qualitative results : The images taken in the simulator, depth images and flo images are stored in the folder : Code/flownet2-tf/src/image_baseline_2_output
Tensorflow, Habitat-sim (for running in the simulator, can also send the velocities predicted in a different simulator).
Here is the link to the Habitat-sim repo : https://github.com/facebookresearch/habitat-sim The repo also contains the information to download the dataset used (.glb files from gibson dataset).