jeanelsner / panda-py

Python bindings for real-time control of Franka Emika robots.

panda-py

Finally, Python bindings for the Panda. These will increase your productivity by 1000%, guaranteed1!

Getting started

To get started, check out the tutorial paper, Jupyter notebooks and other examples you can run directly on your robot. For more details on the API, please refer to the documentation.

Extensions

• dm_robotics_panda Panda model and tools for reinforcement learning in dm_robotics

Install

pip install panda-python

This will install panda-py and all its requirements. The pip version ships with libfranka 0.9.2, the newest version for the Franka Emika Robot. Please refer to the section below if you use an older system version or the more recent Franka Research 3 robot.

libfranka Version

There are currently two robot models available from Franka Emika: the Franka Emika Robot (FER, formerly known as Panda) and the Franka Research 3 (FR3). Depending on the installed firmware, the FER supports version <0.10 while the FR3 requires version >=0.10. For details, refer to this compatibility matrix. If you need a libfranka version different from the default 0.9.2, download the respective zip archive from the release page. Extract the archive and install the wheel for your Python version with pip, e.g., run

pip install panda_python-*libfranka.0.7.1-cp310*.whl

to install the binary wheel for libfranka 0.7.1 and Python 3.10.

Citation

If you use panda-py in published research, please consider citing the original software paper.

@article{elsner2023taming,
title = {Taming the Panda with Python: A powerful duo for seamless robotics programming and integration},
journal = {SoftwareX},
volume = {24},
pages = {101532},
year = {2023},
issn = {2352-7110},
doi = {https://doi.org/10.1016/j.softx.2023.101532},
url = {https://www.sciencedirect.com/science/article/pii/S2352711023002285},
author = {Jean Elsner}
}

Footnotes

1. Not actually guaranteed. Based on a sample size of one. ↩

panda-py's Issues

UDP receive: Timeout

First of all thanks for the fast reply regarding the installation. I went ahead and installed virtualbox and Ubuntu and the installation and everything else worked but...

I made a new python file in PyCharm and copied the first couple of lines from your jupyter tutorial notebook.
The connection to the desk works as you can see from the INFO:.... output. the unlock also works the the Panda robot unlocks all axes and moves them a tiny but as it is supposed to happen and as if I would press the unlock button in the Desk, but then when I want to connect to the Robot and the Hand I ran into the following error:

INFO:desk:Login succesful.
INFO:desk:Legacy desk detected.
Traceback (most recent call last):
  File "/home/vboxuser/PycharmProjects/Panda_Test/Connect2Desk_Unlock.py", line 20, in <module>
    panda = panda_py.Panda(hostname)
RuntimeError: libfranka: UDP receive: Timeout

Process finished with exit code 1

I searched for this error but there are only 2 options i found.

1. I don't have the FCI, but I check in the Desk -> Settings -> System and I have it installed (But I don't have a "Activate FCI Mode" button - maybe thats the reason?)
2. Some issue with iptables but I had nothing in those so i put the IP in INPUT, OUTPUT and FORWARD but with no success..

Do you think the reason is that the robot is on a too old version (4.0.2) and that this is the reason that I cannot connect?
Pinging it works but your communication_test.py always ends at the
if len(sys.argv) < 2: raise RuntimeError(f'Usage: python {sys.argv[0]} 172.16.0.2')

I have also added a static IP and connected to the control unit instead of the Arms base, as it is recommended in the documentation of the Panda..

cartesian_velocity_violation

Hi!
Is there any way of exceeding the 2m/s limit on cartesian velocity of the end effector using panda-py?

Potential issue with indexing of time-optimal trajectory smoothing

Hi, I was testing the code for trajectory smoothing, and in my case I require the code to track densely sampled trajctories rather than a limited number of way-points (for example points along a circle). Currently the code in panda-py forces the max_deviation in the Path constructor to zero (which works fine), but this will result in jerky motions that treat each pair of samples as a segment. This can be avoided by increasing max_deviation to a suitable value. However, doing so results in the Trajectory::getTrajectorySegmentIndex function returning a wrong index for a given timestep. I believe this is due to a mismatch in the length of the smoothed trajectory and the input path.

I'm still testing this, but one way to avoid the issue is to store a segment index into each PathSegment when constructing the Path object, and then using the following code for Trajectory::getTrajectorySegmentIndex:

size_t Trajectory::getTrajectorySegmentIndex(double time) {
  size_t idx = 0;
  auto t = getTrajectorySegment(time);
  double s = t->path_pos_;
  return path_.getPathSegment(s)->index;
}

Porting Active Controllers

First of all really good work Jean!! Thanks a lot for this repo.

Secondly, have you looked at porting the Active Controllers that was introduced with libfranka v0.13.0? It enables user to control without the need for the callback function.

How to consider impedance control and position control at the same time?

I'm conducting some grasping experiments where the gripper may contact the desk a little. But I've noticed that in the grasping process, after the gripper closed, the program just stuck and finally aborted (core dumped).

my code is like:

self.panda.move_to_joint_position(pre_grasp_joints, speed_factor=self.move_speed)
time.sleep(0.03)

self.panda.move_to_joint_position(grasp_joints, speed_factor=self.move_speed)
self.gripper.move(width=0.0, speed=self.gripper_speed)
time.sleep(0.03)

self.panda.move_to_joint_position(after_grasp_joints, speed_factor=self.move_speed)
self.gripper.move(width=self.max_gripper_width, speed=self.gripper_speed)
time.sleep(0.03)

here is the output:

....(previous ignored)
INFO - 2023-11-08 22:57:14,710 - grasp_panda - Stopping active controller (Trajectory).
INFO - 2023-11-08 22:57:14,712 - grasp_panda - Initializing motion generation (moveToJointPosition).
INFO - 2023-11-08 22:57:14,724 - grasp_panda - Computed joint trajectory: 1 waypoint, duration 5.72 seconds.
INFO - 2023-11-08 22:57:14,724 - grasp_panda - Starting new controller (Trajectory).
INFO - 2023-11-08 22:57:20,850 - grasp_panda - Stopping active controller (Trajectory).
INFO - 2023-11-08 22:57:20,853 - grasp_panda - Initializing motion generation (moveToJointPosition).
INFO - 2023-11-08 22:57:20,860 - grasp_panda - Computed joint trajectory: 1 waypoint, duration 2.17 seconds.
INFO - 2023-11-08 22:57:20,861 - grasp_panda - Starting new controller (Trajectory).
  File "grasp_panda.py", line 136, in _execute_pick_and_place
    self.gripper.move(width=0.0, speed=self.gripper_speed)
RuntimeError: libfranka gripper: Command failed!
INFO - 2023-11-08 22:59:13,746 - core - signal_shutdown [atexit]
Aborted (core dumped)

Move command aborted: motion aborted by reflex! ["controller_torque_discontinuity"]

Hi! I want to use panda-py to execute a sequence of gripper pose on fr3. But I find that sometimes the move_to_pose function does not work, especially when the gripper deviates from the starting position. Can you help me with that?
Here is the error:
Control loop interruped: libfranka: Move command aborted: motion aborted by reflex! ["controller_torque_discontinuity"]
control_command_success_rate: 1
Irregular state detected. Attempting automatic error recovery.
Segmentation fault (core dumped)

problem with move_to_pose function

There is some issue with the ik solver in the move_to_pose function. Given a position that is not hard to reach, it will give me ik solver failed error. However, I use another ik solver to get the joint position, and then use the set joint position function, the robot can reach the desired position.

Reading RobotState asynchronously

Hi, I have a separate process that needs to read RobotState continuously at specified frequency. In my main thread I'm also trying to execute robot move to pose commands. However, these two commands seem to conflict with each other and I get UDP timeout issues. What's the proper way to asynchronously read RobotState from a separate process?

Main Process:
while True:
# run some move_to_pose command

Process 1:
robot = libfranka.Robot(hostname).read(test.call)

Joint instability

Thanks for the great work! I've been using this as my main Franka driver and I noticed that the joints will sometimes wildly oscillate when trying to get into a particular motion. Has anyone else encountered this issue? The error I get is "Power Limit Violation"

A BIG THANK YOU FOR THAT

Hi @JeanElsner ,

I recently discovered this repo and it is EPIC!!!!

It makes my life so much simpler!!!

THANKS A LOT FOR THAT!!!

No issue here.

Adding programming vs demonstration mode

Hi, it would be useful to add functionality to the desk interface so it can also set programming vs execution mode. Currently I am doing this externally (using a FR3) with this function:

def set_programming_mode(desk, flag):
        if flag:
            url = '/desk/api/operating-mode/programming'
        else:
            url = '/desk/api/operating-mode/execution'
        desk._request('post',
                  url,
                  files={'force': True},
                  headers={'X-Control-Token': desk._token.token})

With the URLs borrowed from https://github.com/TimSchneider42/franky

some issues about the `ik` functions

hello, thank you very much for your work!
I have tried it and it is so impressive! But I'm a little confused why in your ik function, q_7 is redundant joint? If it is only 6-DoF indeed, some waypoints may generate nan results. It can solve ik successfully if we do not fix the value of q_7.

Quaternion convention

Hi Jean,
Not a bug, but I was testing the cartesian impedance controller and working on extending it to include orientation, and noticed that the expected convention for quaternions is with scalar last, which differs from the one in roboticstoolbox/spatialmath (which I am using quite extensively) and Eigen (in API not memory, confusing!). I wander if it is worth having a configuration flag that determines this? Otherwise I think it is worth pointing this out visibly in the documentation/examples as it can actually lead to some unexpected outcomes:)

Build and install from source

Thanks a lot for your fantastic work!

I want to modify and fine-tune some C++ code but cannot figure out how to build and install from the source.
Could you please write a simple guidance of how to build and install the panda_py from the source code?

update: I successfully build the _core.so and libfranka.so, and use make install to install the .so file, but how to make and install the python library? Currently I can not import panda_py from my own project.

Many thanks.

Runtime Error on simple Cartesian motion

Hi @JeanElsner,

thanks for the great work here. I came across an edge case in the Cartesian motion generation. Executing the following code ...

T_0 = panda_py.fk(constants.JOINT_POSITION_START)
T_1 = T_0.copy()
T_1[0, 3] += 0.35
T_1[2, 3] -= 0.2
# T_1[1, 3] -= 0.2

panda.move_to_joint_position(constants.JOINT_POSITION_START)

panda.move_to_pose(T_1,
                   speed_factor=0.05,
                   stiffness=2 * np.array([600, 600, 600, 600, 250, 150, 50]))

produces the following error:

INFO:panda:Connected to robot (192.168.88.140).
INFO:panda:Panda class destructor invoked (192.168.88.140).
INFO:panda:Initializing motion generation (moveToJointPosition).
INFO:motion:Computed joint trajectory: 1 waypoint, duration 2.10 seconds.
INFO:panda:Starting new controller (Trajectory).
INFO:panda:Stopping active controller (Trajectory).
INFO:panda:Initializing motion generation (moveToPose).
INFO:motion:Computed Cartesian trajectory: 1 waypoint, duration 4.24 seconds.
INFO:panda:Starting new controller (Trajectory).
ERROR:motion:IK failed at time 3.11, prior to waypoint 1. Goal may be outside of workspace.
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[77], line 10
      6 # T_1[1, 3] -= 0.2
      8 panda.move_to_joint_position(constants.JOINT_POSITION_START)
---> 10 panda.move_to_pose(T_1,
     11                    speed_factor=0.05,
     12                    stiffness=2 * np.array([600, 600, 600, 600, 250, 150, 50]))

RuntimeError: IK produced NaN.

When I shift the goal pose in y (see line 5 in the code above), the motion works but the first joint turns around very quickly. Let me know, if you need more data or how we can debug this.

Best,
Jan

Recording positions

Hi, first of all thanks for the work!
I want to record the robot pose while "teaching", is there a suggested way or examples on how to do this with panda-py?
I currenly am getting the the pose with panda.get_robot().read_once().O_T_EE

Problem with installation

Hi, I am relatively new to this topic and I am currently working on my master thesis with the Panda robot.
I want write a program in python for the robot so it build up a structure of some kind of bricks.

My problem is that i cannot install your package.
I am using a windows PC and put the pip install panda-python command in the Terminal of a new project in PyCharm.
When i did this i got the following error:
ErrorCode_Python_panda_install.txt

So i tried to clone this repo first and open the project folder and repeat the install but now the pip command does not work at all with the following error message:
pip : The term 'pip' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Did the same in Visual Studio, cloned the repo, opened a CMake and opened the CMakeList.txt, the installation started but with the same error as in the attached file -> cannot find Eigen3....

Could you please provide some help, I really wish to use Python to manipulate the robot and write some tasks for it.
Do I need to use a Linux OS or what am I doing wrong.

I thank you in advance for your help.

Best regards
Thomas

Moving gripper and robot simultaneously

Hello, and thank you for a very nice library.

I was wondering if there's any way of moving the gripper at the same time as the robot arm moves? I've tried to do some simple threading, where one script controls the arm, and the other controls the gripper, but didn't really get it to work. The actions happened in sequence, as if it was written in one script.

Again, thank you!

Error in API Call for Unlock/Lock on FR3

Hi Jean, thanks for the great package. Upon trying to unlock the FR3, I get the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/python3.10/site-packages/panda_py/__init__.py", line 113, in unlock
    self._request('post',
  File "/path/to/python3.10/site-packages/panda_py/__init__.py", line 312, in _request
    raise ConnectionError(response.text)
ConnectionError: <!DOCTYPE html>
<html>
<head>
<title>Not found</title>
</head>
<body>
<code>No handler accepted "/api/robot/open-brakes"</code>
</body></html>

I've downloaded the wheel associated with libfranka version 0.13.2 for python 3.10. My guess is that the API call might be different between the Panda and the FR3? By examining the API calls sent out from the web GUI, I can successfully unlock the robot with the following change to __init__.py:

  def lock(self, force: bool = True) -> None:
    """
    Locks the brakes. API call blocks until the brakes are locked.
    """
    self._request('post',
                  '/desk/api/joints/lock',
                  files={'force': force})

  def unlock(self, force: bool = True) -> None:
    """
    Unlocks the brakes. API call blocks until the brakes are unlocked.
    """
    self._request('post',
                  '/desk/api/joints/unlock',
                  files={'force': force},
                  headers={'X-Control-Token': self._token.token})

If there is indeed a difference between the FER and FR3 interfaces, then perhaps the Desk object can just take an additional flag that indicates which platform the user has. I've opened a PR with those changes for your review.

Feature request: Shutting down the robot using panda-py

Hi @JeanElsner,

Could you add a function to the 'Desk' class that triggers the robot to shut down? It could also be a parameter for the 'logout(shutdown=True)' function.

Thanks in advance!
Jan