GraspFilter::filterGraspsHelper sometimes hangs at OpenMP for loop about moveit_grasps

[Levaru]

I was able to narrow the problem down to this function: Line in Question

Is there maybe a variable that is being modified by multiple threads or is this caused by something else entirely?

[Levaru]

After some debugging and commenting out lines of code I can indeed confirm that searchPositionIK() is the culprit here. I don't know if this is a data race or if the ik solver just hangs for another reason. I solved this issue by replacing KDL with the Trac-Ik plugin. I would really appreciate it, if someone could double check this!

[davetcoleman]

@Levaru thanks for reporting this and updating us on the progress. could you suggest a PR to fix this issue? it sounds like an issue with KDL and multi-threading, correct? so maybe a patch is needed for KDL...

We have used KDL with this project a lot, though, and haven't had this issue. Additionally, TracIK actually just uses two threads under the hood, one uses KDL...

[Levaru]

@davetcoleman i'm not sure how a fix would look like to be honest. This data race issue started happening once I moved to a PC with more cores and thus more threads were created (12) for the multithreaded filtering. A good way to check for this data race is to wrap the filterGrasps() function in a while(true) loop and then just let it run.

I believe that searchPositionIK() doesn't timeout in one of the threads for some reason and thus blocks the whole loop from finishing.

As for TracIK: Because of an increased delay in cartesian path planning, I replaced it by loading the lma_kinematics_plugin/LMAKinematicsPlugin which is a standard IK solver plugin included in MoveIt. It turned out to be thread-safe and was just as fast as KDL.

I hope this information helped!

[JafarAbdi]

Hi @Levaru,

I'm working on solving this issue, I tested the grasp_filter_demo on my PC with 12 thread and it didn't hang, this demo uses Franka Emika’s Panda robotics arm; do you mind sharing the code you are working on right now or anything that would help .? so I could debug it more to see why the hang is happening and to see how I could fix it,

Thanks