sean85914 / rl_pnp

Hardwares

Hardware	Image
ABB IRB1660ID
RealSense D435/415 One required, one optinal
Vacuum Pumb Should can be controlled by an Arduino and a valve
Tools from XYZ Robotics

Dependencies

• ROS kinetic
• librealsense2
• Serial
• Cuda with version either 9 or 10

How to Start

$ cd && git clone https://github.com/sean85914/flip_object.git
$ cd flip_object && catkin_make
$ source devel/setup.bash # do this every time you open a new terminal
$ roscore
$ roslaunch grasp_suck actuator.launch # Make sure you turn on the robot, run the server and share the same network with your PC
$ roslaunch grasp_suck sensors.launch serial_no:=[D4X5 Serial No.] record_serial_no:=[side-record camera serial no.] # if you don't have side-recording camera, set side_record to false and leave record-serial_no empty
$ roslaunch visualization viz.launch rviz:=true
$ roscd grasp_suck/src && python main.py [integer number]

DLP 2019 Final Project

In this project, we try to use reinforcement learning to do pick and place with custom made gripper. The gripper is composed of Robotiq 2F-85 and a retractable suction cup. The method is inspired by "Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning", from Andy Zeng.

@inproceedings{zeng2018learning,  
  title={Learning synergies between pushing and grasping with self-supervised deep reinforcement learning},  
  author={Zeng, Andy and Song, Shuran and Welker, Stefan and Lee, Johnny and Rodriguez, Alberto and Funkhouser, Thomas},  
  booktitle={2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},  
  pages={4238--4245},  
  year={2018},  
  organization={IEEE}  
}

Master Thesis

Extend from the project above, we try to use three different tools to do bin picking job with ABB IRB1660ID. Specially thanks to XYZ Robotics for sponsoring the robot arm and the tooling system.

rl_pnp's Issues

Add image processing method to depth image from d435

The depth information from D435 is not only too undulated but also includes many nan points. I should add some image processing method, such as mean filter, to solve the issue

Change checking method for suction cup

Since I have changed the input from cropped image to height map,it should revise the method to check if the object is grasped.

Method:
Put prior and posterior depth height map into the service server

service to check if output point is valid

It seems that the depth from d435 is too undulated to use as the condition for checking if valid.
I should add a service, which takes point cloud and output point as input, and return a bool to indicate whether the point is valid

Simplify the code

TODO