Using mobilenetv2 as backbone and 2 prediction head (1 for 16 orientation, 1 for 112x112 location of grasping action)
TODO:
Update evalution script
Using ROS replace V-REP python api
Increase location map to 224x224 to improve precision
Add more 1 prediction head for pushing/grasping
[24/11/2023] Single branch, end-to-end pipeline
End-to-end pipeline, single branch, replace densenet121 with mobilenetv2
No input rotation, modeling action space as a 3D tensor 112x112x16 (height map resolution=4mm, 8 angle rotations)
Only use RGB as input of network, depth information for z position
CoppeliaSim simulation
The simulation scene when training train_twoheadgraspnet.py, robot successfuly learn to find the object and do grasping action. Due to the limit of resolution (4mm instead of 2mm in original work), location prediction is sometime inaccurate. There is no pushing action so robot find difficult to handle complex scenerios. The scene is recorded during training phase so there are random actions in sequence of actions.
Training result
Training result of two head grasp model
Requirements
CoppeliaSim v4.5.1 linux
Pytorch
Setup
Open simulation/simulation.ttt in CoppeliaSim
Run python train_twoheadgraspnet.py
Note
This repository is under experimenting and developing period