This ReadMe contains instructions for the setup and usage of the control interface for the robot monkey, which was started during Jaú Gretlers bachelor's thesis: "Assembly and Programming of a Robot Monkey to Study Imitation Learning in Marmosets".
- Raspberry Pi (RPP)
- Controller device: The device on which you want to run the simulation and interact with the robot
- Turn the power supply on FIRST and then connect it to the robot.
- Connect the Ethernet cable and log in to the RPP:
ssh [email protected]
PW:powderguide
- Start the ROS network by running
roscore
on the RPP.
- Update to the latest version:
cd /home/robot-user/monkey_robot_codebase git pull ./update.bash # moves the relevant updated files to the RPP
- Run
roslaunch monkey demo.launch
to start Rviz. - In a second terminal run
python3 /home/robot-user/monkey_robot_codebase/monkey_interface/src/robot_gui.py
to start the GUI. - Arrange the windows such that you have Rviz on the left side of your screen and the GUI on the right side.
There are three options:
-
Collect waypoints
: Lets you choose waypoints for the left and right arm iteratively until you choose to save. You can choose the left/right arm in Rviz:MotionPlanning/Planning/Planning Group
. Whenever you have moved the EEF of the robot to a desired waypoint you must manipulate the rotation of one of the interactive markers (red/green/blue circles). This ensures that the last interactive marker position recorded by the monkey_interface node corresponds to the position of the hand. -
Load and edit waypoints
Lets you load a previously saved collection of waypoints. You can then add waypoints that will be appended. -
Plan/Execute
When using a collection of waypoints for the first time, you firstLoad Waypoints
, thenPlan Carthesian Path
, thenSave Plan
, then choose the number of interpolation steps between waypoints, thenExecute Plan
. When the robot is ready to start the movement, theStart
button is activated. In consequent usage, it will beLoad Plan
, then choose the number of interpolation steps between waypoints, thenExecute Plan
. There is a video of an example workflow on the Desktop of the controller.
You can get the image from here.
Use the Raspberry Pi Imager to flash a Ubuntu 20.04 Server Distribution to the RPP. Select "Other general-purpose OS/Ubuntu/Ubuntu Server 20.04.5 LTS (64 bit)" from the options. The RPP Imager will offer you to include the SSID and password of a local network in the image to be flashed, which can be useful to already set at this stage.
- SSH into the RPP
- To enable a connection over ethernet:
- (on computer) https://docs.phillycommunitywireless.org/en/latest/device-configs/shared-connection/
- (on RPP) Enable ethernet interface:
sudo ifconfig eth0 up
- Navigate to /etc/netplan
- Create config.yaml and write the following (adapt the wlan0 settings to the connection you choose in the Rasperry Pi Imager):
network:
version: 2
wifis:
renderer: networkd
wlan0:
access-points:
JorisPhone:
password: genti111
dhcp4: no
optional: true
addresses:
- 172.20.10.3/28
gateway4: 172.20.10.1
nameservers:
addresses:
- 8.8.8.8
ethernets:
eth0:
addresses: [10.42.0.2/24] #<RPP_IP>
gateway4: 10.42.0.1 #<PC_IP>
nameservers:
addresses: [8.8.8.8,8.8.4.4]
Follow the steps described here to install ROS on the controller device. Use ros-noetic-desktop-full
Follow the steps described here to install ROS on the RPP.
For the RPP it suffices to install ros-noetic-ros-base instead of ros-noetic-desktop-full. The latter includes Rviz and Gazebo (simulations softwares) which are of no use on the headless RPP.
For the RPP with IP-Address <RPP_IP>, write these lines in the .bashrc file:
source /opt/ros/noetic/setup.bash
export ROS_IP=<RPP_IP>
export ROS_MASTER_URI=http://<RPP_IP>:11311
#e.g.:
source /opt/ros/noetic/setup.bash
export ROS_IP=10.42.0.2
export ROS_MASTER_URI=http://10.42.0.2:11311
For the desktop PC with IP-Address: <PC_IP>, write these lines in the .bashrc file:
source /opt/ros/noetic/setup.bash
export ROS_IP=<PC_IP>
export ROS_MASTER_URI=http://<RPP_IP>:11311
#e.g.:
source /opt/ros/noetic/setup.bash
export ROS_IP=10.42.0.1
export ROS_MASTER_URI=http://10.42.0.2:11311
Note that the ROS_MASTER_URI refers to the device which is the ROS master, i.e who starts and controls the ROS network. With these environment variables you must start the ROS network by running
roscore
on the RPP. If you dont do that the setup assistant and all other nodes you try to launch will fail because they are trying to find the ROS master but can't, since roscore wasn't run on the device whose IP was exported as ROS_MASTER_URI.
If you just want to use Rviz and not control the physical robot, just use the PC_IP in the ROS_MASTER_URI. Then the network will be started by your PC.
Follow the instructions here to install MoveIt on the controller device. From this point onwards I am assuming you have a catkin workspace setup, to which I will refer as "ws_moveit" from now on. Note that this simply means that there is a folder in your /home directory called ws_moveit, in which you have executed all commands listed in the tutorial mentioned above. One command which you don't have to run is this one:
git clone https://github.com/ros-planning/panda_moveit_config.git -b noetic-devel #unnecessary
If you run it "fatal error" will be displayed, but that is irrelevant for our purposes.
The package in question is called "Rospy message converter". You can install it with:
sudo apt install ros-noetic-rospy-message-converter
Note that any other package which turns out to be missing should be installed analogously.
sudo apt install python3-roslaunch
For the servo hat:
sudo apt install python3-smbus
sudo apt-get install i2c-tools
sudo apt install python3-pip
sudo pip install adafruit-circuitpython-servokit
pip install RPi.GPIO
These instructions assume you have setup a catkin workspace.
- Download the folder "monkey_complete" from this repository. Place it in ws_moveit/src.
- In ws_moveit, open a terminal and run
catkin build monkey_complete
to build the monkey_complete package. - Source the workspace by running
source devel/setup.bash
in ws_moveit. - Launch the moveit setup assistant with
roslaunch moveit_setup_assistant setup_assistant.launch
. Note that this will only work if:
roscore
is running on the RPP- you have installed and setup moveit according to this tutorial
- you have sourced your workspace with
source devel/setup.bash
- In the setup assistant, select "Create New Moveit Configuration Package"
- Click on "Browse" and select your URDF file
- Click on "Load Files"
- In Self-Collision: Set "Sampling Density to max" and click "Generate Collision Matrix"
- In Virtual Joints: Create an entry according to the following picture:
- In Planning groups:
a. Add new planning groups by clicking on "Add group".
b. For the "Kinematic Solver" choose "kdl_kinematics_plugin".
c. For "Group Default Planner" choose "RRT".
d. Add components to the group by clicking on "Add Kin. Chain". First select the "Base Link" (in our case base_link) and press "Choose Selected". Second, select the "Tip Link" (e.g L_Hand_Link) and press "Choose Selected".
e. Add single links to a planning group by clicking "Add Links".
With this knowledge you can create the planning groups according to the following picture:
- In Robot Poses: Create entries according to the following pictures:
-
In End Effectors: Create entries according to the following picture:
-
In Author Information, type in a name and a valid email, otherwise we can't save the config
-
In Configuration Files, specify the name of the package and place it in the source folder of your catkin workspace, e.g
/home/<user>/ws_moveit/src/monkey
-
Click "Generate Package"
-
Now we need to build the moveit config package. To do that, open your catkin ws in a terminal and run:
source devel/setup.bash
catkin build monkey
- In order to test the freshly generated config package, run (in the same terminal as before):
source devel/setup.bash
roslaunch monkey demo.launch
Now you should see a window popping up containing the simulation environment Rviz. It should look like this:
Note that when Rviz starts up, it will display a lot of logs, among other things potentially the warning, that the link "base_link" has an inertia specified in the URDF and that this is a problem. Do not try to change the inertia of this link in the URDF, that wont work. This warning means that your URDF is missing the snippet mentioned under 2. or that the snippet contains a mistake.
In order to have a efficient workflow and make use of all the tools and ideas conceived during the thesis, the following modifications have to be made inside of Rviz and saved.
- In the "Displays" panel, click on "Add". In the then appearing panel (named "Rviz"), order the visualizations "By display type", select "MarkerArray" and click "Ok".
![](https://private-user-images.githubusercontent.com/50492539/244377599-c7b48752-9687-4233-907a-78f788a528e0.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM4NjI4MjAsIm5iZiI6MTcyMzg2MjUyMCwicGF0aCI6Ii81MDQ5MjUzOS8yNDQzNzc1OTktYzdiNDg3NTItOTY4Ny00MjMzLTkwN2EtNzhmNzg4YTUyOGUwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE3VDAyNDIwMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWEyNDZhZTAyZDU2ZWExMDNmYmU2NDkxYjc4ZGE3ZjMwY2Q4M2Y0YzMyZTkwMzEzOWExOTJiNTMwNTVlNjk4NmUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.x2V0boKM8EuXRFN2wLFWsnUs_xRPuLbWkpO9G35KeFQ)
-
In the MotionPlanning Panel, tick the tickbox labeled "Approximate IK solutions". Without that enabled, we cannot manipulate the end effectors of the robots arms.
-
In the Displays Panel, go to the MotionPlanning/Planned Path rider and uncheck the tickbox "Show Robot Visual". This will deactivate the animation of the planned robot arm trajectory, which can get annoying.
-
Press "CTRL+S" to save the current Rviz config
In order to have a efficient workflow and make use of all the tools and ideas conceived during the thesis, a few modifications have to be made to the Kinematics.yaml file, which can be found in the "config" folder of your moveit-config-pkg. The file should look as follows:
monkey_left_arm:
kinematics_solver: kdl_kinematics_plugin/KDLKinematicsPlugin
kinematics_solver_search_resolution: 0.01
kinematics_solver_timeout: 0.005
goal_joint_tolerance: 0.0001
goal_position_tolerance: 0.0001
goal_orientation_tolerance: 0.001
position_only_ik: True
monkey_right_arm:
kinematics_solver: kdl_kinematics_plugin/KDLKinematicsPlugin
kinematics_solver_search_resolution: 0.01
kinematics_solver_timeout: 0.005
goal_joint_tolerance: 0.0001
goal_position_tolerance: 0.0001
goal_orientation_tolerance: 0.001
position_only_ik: True
monkey_head:
kinematics_solver: kdl_kinematics_plugin/KDLKinematicsPlugin
kinematics_solver_search_resolution: 0.01
kinematics_solver_timeout: 0.005
goal_joint_tolerance: 0.0001
goal_position_tolerance: 0.0001
goal_orientation_tolerance: 0.001
position_only_ik: True
List of changes made:
- added line
position_only_ik: True
to each planning group - changed "kinematics_solver_search_resolution" to 0.01 for each planning group
If you did everything right up until here, you should be able to drag around the hands of the robot quite freely around in space (of course only inside the space which is reachable by the robot and permitted by his joint limits). If you can't drag around the hands, it might help to untick and retick the "Approximate IK solutions" box. This setting seems to deactivate itself sometimes.
- From this repo, download the folder "monkey_interface" and place it in the "src" folder of your catkin ws
- Run
catkin build monkey_interface
to build the package. - If you haven't done so already, run
caktin build
in the src folder, to build all packages. This will take about 10min (if you never build them before). - Run
chmod +x ~/ws_moveit/src/monkey_interface/src/monkey_interface.py
to allow monkey_interface.py to be executed.