hans-ob1 / neural_cam_ros

This package is essentially a ros-wrapper of neural_cam. More features would be added in the future, geared towards mobile robot platform. Eventual goal of this package is to solve the problem of given an image, where is the obstacle with respect to the robot in a 3D space.

neural_cam_ros

neural_cam_ros is essentially an ros wrapper of neural-cam. The backend deep learning framework used is darknet developed by Joseph Redmon. You can visit his site here. This module is designed to quickly test the state-of-the-art obstacle detection/tracking for mobile robot platform. The package is built using rosbuild, I havent test it in catkin but can be easily integrated. No training option is available in this module.

Update

• basic ros-wrapper for obstacle detection is complete
• includes link for pre-trained weights/configuration files
• added cv_bridge for ros image transport
• tested on drive px 2 (detection does not work well with cuDNN! i think you would need to train directly on their platform)
• tested on nvidia TX1 (low fps, wait for TX2!)

Additional Information

• requires pre-trained weights, cfg files to run
• open another terminal and type "rostopic echo /obstacle" to see detected objects been published
• edit/launch multiple cameras using multicam.launch file
• to get a decent framerate, use a good graphic card. preferably to be GTX 950 above and run the network with cuDNN
• i had pre-trained a set of weights to detect road obstacles for my autonomous vehicle project. There are only three classes for the weights i trained (person, bike and vehicle). you can download and try it our yourself from this link. Do take note that is set of weights is trained with cuDNN v5, be sure to download that from nvidia website. Otherwise, there will be no detections!

Recommend System Requirement (Tested and Proven)

• cmake 3.2 above
• ROS Kinetic (due to its preloaded opencv 3, saved alot of hassle)
• ubuntu 16.04
• cuda 8.0 (for real time performance)
• cuDNN v5
• latest nvidia graphic driver 367.48

Minimum System Requirement

• cmake 2.8 or above
• ROS indigo or above
• runs on opencv 3.1
• ubuntu 14.04 above
• cuda 7.5 above (modify the CMakeList.txt if you dont have a GPU!)
• cuDNN v5 (download from nvidia official website, need v5 for cuda 8.0)
• latest nvidia graphic driver 367.48

Usage Detail

requires a connected webcam and ROS installed to run rosmake in a terminal

  $ roscd neural_cam_ros
  $ rosmake
  $ roslaunch neural_cam_ros multicam.launch

Common Error

for "installation" of the package, add this line to ~/.bashrc and source it:

    export ROS_PACKAGE_PATH=/your/path/to/workspace:$ROS_PACKAGE_PATH

if cmake display of not able to find "curand/cublas/cudnn", be sure to add the following symbolic link:

   sudo ln -s /usr/local/cuda/lib64/libcurand.so /usr/lib/libcurand.so
   sudo ln -s /usr/local/cuda/lib64/libcublas.so /usr/lib/libcublas.so
   sudo ln -s /usr/local/cuda/lib64/libcudnn.so /usr/lib/libcudnn.so

Demo Video on YouTube

neural_cam_ros's Issues

Couldn't find executable named detector

Hi~ I have a little problem~
When i execute the rosrun neural-cam-ros detector, I encounter a problem:

and...I have some question.
where are xxx.date, xxx.weights and xxx.cfg file put on when i execute the rosrun neural-cam-ros detector?
Thanks you!