nvidia-ai-iot / ros2_trt_pose_hand

ROS2 package for trt_pos_hand, "Real-time hand pose estimation and gesture classification using TensorRT"

ros2_trt_pose_hand

In this repository, we build ros2 wrapper for trt_pose_hand for Real-time hand pose estimation and gesture classification using TensorRT on NVIDIA Jetson Platform.

Package outputs:

• Hand Pose message with 21 key-points
• Hand Pose detection image message
• std_msgs for gesture classification with 6 classes [fist, pan, stop, fine, peace, no hand]
• Visualization markers
• Launch file for RViz

Requirements:

• ROS 2 Eloquent:
  
  • Install Instructions
    
• trt_pose
  • Dependencies for trt_pose
    
  • Install trt_pose
    
• Gesture Classification
  • scikit-learn $ pip3 install -U scikit-learn

Build:

• Clone repository under ros2 workspace
  

$ cd ros2_ws/src/
$ git clone https://github.com/NVIDIA-AI-IOT/ros2_trt_pose_hand.git

• Install requirements using rosdep
  

$ rosdep install --from-paths src --ignore-src --rosdistro eloquent -y

• Build and Install ros2_trt_pose_hand package
  

$ colcon build
$ source install/local_setup.sh

Run

• Change Power Mode for Jetson sudo nvpmodel -m2 (for Jetson Xavier NX)
  sudo nvpmodel -m0 (for Jetson Xavier and Jetson Nano)
  

• Keep trt_pose_hand related model files in base_dir, it should include:
  

  • Model files for resnet18 download link
  • Hand Pose points json file
  • Gesture Classification classes json file
  • Gesture Classification model included in git

• Method 1:
  

  • Input Images are captured using image_tools package
    ros2 run image_tools cam2image
  • Run ros2_trt_pose node
    

    $ ros2 run ros2_trt_pose_hand  hand-pose-estimation --ros-args -p base_dir:='<absolute-path-to-base_dir>'
    

  • Visualize markers
    

  $ ros2 run rviz2 rviz2 launch/hand_pose.rviz
  

• Method 2: Use Launch file to each node:
  

  • Run using Launch file
    

  $ ros2 launch ros2_trt_pose_hand hand-pose-estimation.launch.py
  

  Note: Update rviz file location in launch file in launch/hand_pose_estimation.launch.py
  

• For following use separate window for each:
  

  • See Pose message
    

  $ source install/local_setup.sh
  $ ros2 run rqt_topic rqt_topic
  

Other related ROS 2 projects

• ros2_jetson webpage
• ros2_trt_pose
• ros2_torch_trt : ROS2 Real Time Classification and Detection
  
• ros2_deepstream : ROS2 nodes for DeepStream applications
  
• ros2_jetson_stats : ROS 2 package for monitoring and controlling NVIDIA Jetson Platform resources
  

ros2_trt_pose_hand's Issues

hand-pose-estimation ERROR/ No tf data in RVIZ

Hi I am trying to use 2. method to tun ros2_trt_pose_hand and getting this error:
[ERROR] [hand-pose-estimation-1]: process has died [pid 10259, exit code -9, cmd '/home/robot/Documents/ros2_ws/install/ros2_trt_pose_hand/lib/ros2_trt_pose_hand/hand-pose-estimation --ros-args -r __node:=hand_pose_estimation --params-file /tmp/launch_params_7ffbcjty'].

I see that it is publishing images, but seems like they are not processed anywhere. Also rviz2 is starting fine, but do nothing.
[INFO] [hand-pose-estimation-1]: process started with pid [10259] [INFO] [cam2image-2]: process started with pid [10260] [INFO] [rviz2-3]: process started with pid [10261]
When I am trying to lunch it in separate window (method 1). I can run both:
ros2 run image_tools cam2image
ros2 run ros2_trt_pose_hand hand-pose-estimation --ros-args -p base_dir:='<absolute-path-to-base_dir>'but nothin happens and after a while hand-pose-estimation ends without any log.