duburcqa / jiminy Goto Github PK

Hi @duburcqa,

I really appreciate seeing Pinocchio's code to be used in external projects.

It should be nice to keep the copyright holders of the files when lines of code have been copy-pasted from original projects.
For instance, https://github.com/Wandercraft/jiminy/blob/master/python/jiminy_py/src/jiminy_py/meshcat/meshcat_visualizer.py, even if it contains some modifications.
I don't think this is the only where it occurs but as a maintener of Pinocchio, I would really appreciate to see the original credits of our work.

In addition, I see you add some features to https://github.com/Wandercraft/jiminy/blob/master/python/jiminy_py/src/jiminy_py/meshcat/meshcat_visualizer.py which can benefit to everybody. I do think you can contribute to the whole community by improving the original version here https://github.com/stack-of-tasks/pinocchio/blob/master/bindings/python/pinocchio/visualize/meshcat_visualizer.py, in the same way we are trying to help you when you encounter issues or bugs.

Best regards,

Justin, on the behalf of the Pinocchio's team.

• For C++, clang-tidy and clang-format would do the job
• For Python, pylint + mypy seems pretty good

There are many other things we could add to this tutorial:

• Adding constants / new variables to a log - doesn't make much sense on a simple pendulum, but would on a more complex system.
• Adding sensors, changing the simulation configuration.
• A multi-robot simulation with interesting coupling

...

I wish to simulate a segway-like system: basically, a wheel turning on the ground. This cannot be exactly simulated in Jiminy, which only supports punctual contacts with an elastic ground: the 'hack' one could do would be to place many points along the rim of the wheel, and set them all as contact points.

Rather, I want to simulate the contact as an analytical constraint: i.e. the wheel is in perfect contact with the ground and does not slip. This can be done by adding the constraint in the dynamics equation as Lagrange multipliers.

Note that I'm only interested in having a constant contact throughout the simulation: I do not mean to implement 'analytical contact' for a walking robot, that would require an added contact detection framework. What I have in mind is a constant contact throughout the simulation. However, this is already a significant first step in that direction if we ever want to do that.

This is how I see the implementation:

• at a 'low-level', contact kinematics is implemented in pinocchio as pinocchio::forwardDynamics(..., J, drift) where J is the jacobian of the constraint, and drift the associated drift. This implementation of pinocchio::forwardDynamics would replace the aba algorithm in Engine.computeSystemDynamics. In practice, pinocchio::forwardDynamics is likely to be much slower than aba (because it requires computing the inertia matrix, angument it with the jacobian, and do the inverse), thus aba would remain the default option, and Engine would switch to using forwardDynamics only if a contact constraint is being used.

• like we did in aba, we will need a custom implementation of forwardDynamics to support armature inertia.

• While I do not need this for my use case, supporting several constraints at once seems a nice feature to have. I think the best way to handle this from the API standpoint is to give a name to each constraint, so adding and removing contact constraints would be like adding and removing sensors or motors.

• As API, to simulate a contact analytical we need a jacobian, with signature matrixN_t jacobian(q), and the corresponding drift term with signature vectorN_t drif(q, v). The main usage would probably be to have a specific frame in contact, however, for a wheel, I want the wheel rim to be in contact, not a fixed point. This requries a 'custom' jacobian to be implemented. So I'm thinking about having two ways to define a constaint: by a simple frame name, or by a callback function.

In the end, here is an overall architecture of what I plan to do:

• Define signatures for jacobian and drift callback functions:

typedef matrixN_t (*JacobianFct_t) (model, data, q);
typedef vectorN_t (*DriftFct_t) (model, data, q, v);

• Add class representing the contact:

class ContactConstraint:
    ContactConstraint(std::string contactFrameName);
    ContactConstraint(JacobianFct_t J, DriftFct_t drift);

• Add add/remove methods to robot, like add / remove of motors, sensors, contact frames... work:
  • Robot::setContactConstraint(std::string name, ContactConstraint)

The engine would then switch between both implements of the dynamics depending of the presence of not of ContactContraint. Note that both contact are not mutually exclusive: you could imagine having both intermitent, elastic contacts, and constant constraints (for instance, fix you robot foot on the ground analytically, then make its hand hit a wall).

Telemetry namespace of controllers should not be hard-coded, but defined at construction or initialization. Uniqueness should be ensured by EngineMultiRobot.

https://github.com/Siviuze/Piper

Hello, I am using Jiminy tag version 1.6.17. I was trying to get the orientation and position of a particular link using a procedure I was using in tag version 1.6.1 however, I am getting segmentation faults. This is a minimal modification of the double pendulum example that reproduces my problem:

void computeCommand(float64_t        const & t,
                    vectorN_t        const & q,
                    vectorN_t        const & v,
                    sensorsDataMap_t const & sensorsData,
                    vectorN_t              & command)
{
    // No controller: energy should be preserved
   auto linkId = robot->pncModel_.getFrameId("PendulumArm");    
}

I made sure to declare "robot" as a global variable (auto robot = std::make_shared();)

Whenever I remove the getFrameId line in computeCommand the code doesn't have any problems running, but adding the line causes the sefgaults. I am using the pip installation method for building the example.

Victor

Codacy detected an issue:

Message: Assigning to function call which doesn't return

Wrong report error. Actually Async methods always return coroutine in Python.

Currently on:

• Commit: b86cf5e
• File: python/jiminy_py/src/jiminy_py/viewer/meshcat/recorder.py
• LineNum: 230

It would be nice to use a standard model for contact and friction if it is not the use, but more importantly to document it.

https://github.com/Wandercraft/jiminy/pull/94/files

It would be nice to be able to compute the collision with a analytical ground function f(x,y) -> (z, normal), and to display it in the viewer. It is not straightforward to do so because computing the collision with a very large non-convex mesh is time-consuming. After some benchmark, just replacing the halfspace ground by a mesh increases the overall computation time by a factor x10 (which means that computing the collision is probably x1000 slower...).

Dear @duburcqa,

I was wondering how does it cost to isolate the pip build for Pinocchio from your current setup? I think it might be useful to do it, in order to let external users rely on Pinocchio using pip.

Thanks in advance for your help,

Justin

This is especially critical for 'dynamics' helpers because some methods are quite complex and it would be nice to validate them. It is pretty "easy" to do it. For instance, it would be possible to run first a simulation, extracting only the motor position/velocity/acceleration, and try the recover missing data.

Currently, the contact force is modelled using a spring-dumper mechanism. Using infinite stiffness without dumping for the ground reaction force should, in theory, replicate a perfectly rigid impact model. However, it would increase dramatically the stiffness of the ode, which in turn would be impossible to integrate using a decent step size. On the contrary, taking advantage of the discrete dynamics tools should do the job pretty well in Pinocchio (impulseDynamics and forwardDynamics under constraints).

It would be cautious to have a basic unit test in Python checking that the Python API is working properly. The idea is just to check that it is possible to run a simulation without segfault, which may happen in practice if Eigenpy and Boost Python are not properly initialized.

Currently, the motors are part of Model itself. This implementation has major flaws:

• It is inconsistent with the sensors, which are dedicated objects.
• The engine has to manage the options of all the motors at the same time. The size of some option parameters associated with the motors depends on the number of motors, therefore user-defined values must be disable whenever a motor is added. This unexpected behaviour is very disturbing for the user. Moreover, adding motor options requires to implement a custom Model class.
• It does not allow to use advanced motor model, for example using a realistic voltage/current-torque dynamics, or gears with backslash. It also enable to instantiate a motor without a model. It could be useful for example to assess their own dynamics, which is possible in the real world.

The idea would be to implement something similar to the sensor: all motors must share a common interface that would be specified by an abstract class. The motors would be part of the input of the command functor called by the engine.

Currently, it is possible to add hppfcl primitive geometries to both Meshcat and Geopetto-Gui, yet it is not possible to update there properties using the same principal. Moreover, their respective API to do so is quite bad. One way to get around this limitation would be to delete and create a new geometry systematically, but it is likely to perform poorly.

• Meshcat needs to be patched manually to be supported by Python 3.8 on Windows :
  In file meshcat\servers\zmqserver.py, replace

# Install ZMQ ioloop instead of a tornado ioloop
# http://zeromq.github.com/pyzmq/eventloop.html
ioloop.install()

by

if sys.platform == 'win32':
    import asyncio as _asyncio
    _asyncio.set_event_loop_policy(_asyncio.WindowsSelectorEventLoopPolicy())

Python native code is properly documented, but it is not the case for the C++ API and the Python bindings (which are using the same documentation in most cases). It is really important to fix this, at least for methods and classes that the user is supposed to use in practice.

Panda3D viewer is very processing. It mitigates most of the pitfalls of Meshcat when used locally. It would be very nice to support it.

There is no default implementation for the internal dynamics in AbstractController. It could be interesting to have one, for example to model the joint frictions. However, it is not straightforward to do so, since it the option parameters may vary depending on the chosen implementation.

Here is the provided original implementation, based on Pinocchio. It would also be nice to increase the accuracy of the already existing integrators by taking into account that the velocity is the derivative of the position.

So far, only continuous joint position sensors, 3D force sensors, and Quat/Gyro/Accelo IMUs are available. It would be useful to have 1D force sensors (z-axis only), Gyzo/Accelero IMUs, joint torque sensors, and discrete joint position sensors (returning int64).

Those features are already available for sensors. Adding them to the motors should be straightforward.

Currently, the log file only contains data. As a result, it is impossible to replay the simulation or to compute some dynamical quantities using only a log file.

It would be great to log the URDF and options as string constants directly in the header of the log file (it may requires to not assume that the header is not split in multiple chunks while reading it).

It is a preliminary step Howard the integration of Piper for pipeline design #239.

telemetrySender_.registerNewEntry should return a handle with updateValue already specialized to this specific variable, so that it avoids having to search it in the whole dictionary of registered variables systematically.

Simulator should be modified to support both Engine and EngineMultirobot. The same goes for Viewer.

In the current implementation of Engine, some base methods are explicitly forbidden, which is a bad practice. Instead, it should be overloaded to raise an exception when appropriate, so that the API remains compatible. In this way, it should be possible to use indifferently Engine or EngineMultirobot in simulator, addSystem instead of initialized...

Once this limitation would be alleviated for Simulator, it should be suite straightfoward to adapt Viewer to support it, even though it is not clear how to deal with multiple robot. Looping over systems coule be done as an inner loop or an outer loop. Indeed, at replay, a multi-robots system is just a set of Independent single-robot systems.

Currently, the tolerance for the unit tests on the CI is pretty high. It is possible to be more accurate on my personal machine but I don't understand why exactly the UT are failing on the CI if I use the same tolerance. I need to reduce the accuracy on the CI by at least 2 order of magnitude. I would be great to understand such discrepancy.

For now, it is impossible to replay flexible simulation using rigid model. Ideally, once should create a new robot based on logged one, rather than alter the current one. Ideally, the viewer should deal the Jiminy::Model directly instead of Jiminy::Robot, so that it is straightfoward to implement a copy method, because otherwise it would requires to copy every sensors/motors and the controller, which may be hazardous. This could be done by implementing a copyModel method. Another way to deal with this issue should be to add pinocchio_model/data_flexible/pinocchio_model/data_rigid attribute to Jiminy::Model, but it does not guarantee that the flexibility model did not changed. Ultimately, a third solution would be to enable instantiating a Jiminy::Model from logged data (stream or file). It seems more reliable and requires minimal refactoring of the existing code base. So it is probably the way to go.

What is expected is to provide the desired type of joint (as input of specialization) and emulate it.

Running the original tutorial .ipynb, there seem to be issues I couldn't solve. First,
AbstractController' object has no attribute 'set_controller_handle
Commenting out the line trying to set the handle causes the next problem:
TypeError: run() missing 1 required positional argument: 'v0'
Adding a v0 variable which mimics x0 finally results in one last error:

<ipython-input-34-90bc00b210d1> in <module>
     20 
     21 # Launch the simulation
---> 22 simulator.run(simulation_duration, x0,v0)

~/anaconda3/envs/joints/lib/python3.6/site-packages/jiminy_py/simulator.py in run(self, tf, q0, v0, is_state_theoretical, log_path, show_progress_bar)
    418                 "[{elapsed}<{remaining}]"))
    419         try:
--> 420             self.simulate(tf, q0, v0, None, is_state_theoretical)
    421         finally:  # Make sure that the progress bar is properly closed
    422             if show_progress_bar:

~/anaconda3/envs/joints/lib/python3.6/site-packages/jiminy_py/simulator.py in simulate(self, t_end, q_init, v_init, a_init, is_state_theoretical)
    384             t_end, q_init, v_init, a_init, is_state_theoretical)
    385         if return_code != jiminy.hresult_t.SUCCESS:
--> 386             raise RuntimeError("The simulation failed.")
    387 
    388     def run(self,

RuntimeError: The simulation failed.```

It doesn't get past the first step.

For instance, it would be great to display the CoM (center of mass) and the CoP (center of pressure) using fixed-size sphere primitives, or contact forces using dynamic-size capsules. This information requires knowledge about the dynamics of the system, which is not the case for the viewer itself. It would be harmful to introduce such dependency in the viewer itself, so it makes sense to enable to user to register a callback function that will be called automatically during video replay. Using the same mechanism than for camera motion should do the job pretty well.

It would be nice to have a step-by-step example of how to run a simulation using Jiminy (real-time and replay) and run a simple learning algorithm on a simple pendulum from scratch using Pytorch.

Intantiating Panda3dQWidget raises the following exception:

Traceback (most recent call last):
  File "/install/bin/logstudio", line 33, in <module>
    sys.exit(load_entry_point('logstudio', 'console_scripts', 'logstudio')())
  File "/src/python/logstudio/src/wdc/logstudio/logstudio.py", line 46, in main
    win.init_gui()
  File "/src/python/logstudio/src/wdc/logstudio/main_window.py", line 149, in init_gui
    self.init_widgets()
  File "/src/python/logstudio/src/wdc/logstudio/main_window.py", line 67, in init_widgets
    self.widgets.viewer3d.init(self.logfile_path, self.data_source)
  File "/src/python/logstudio/src/wdc/logstudio/viewer3d_widget.py", line 136, in init
    Viewer.open_gui(start_if_needed=True)
  File "/install/lib/python3.8/site-packages/jiminy_py/viewer/viewer.py", line 555, in fct_safe
    return fct(*args, **kwargs)
  File "/install/lib/python3.8/site-packages/jiminy_py/viewer/viewer.py", line 802, in open_gui
    Viewer.__connect_backend(start_if_needed)
  File "/install/lib/python3.8/site-packages/jiminy_py/viewer/viewer.py", line 555, in fct_safe
    return fct(*args, **kwargs)
  File "/install/lib/python3.8/site-packages/jiminy_py/viewer/viewer.py", line 1171, in __connect_backend
    client = Panda3dQWidget()
  File "/install/lib/python3.8/site-packages/jiminy_py/viewer/panda3d/panda3d_widget.py", line 22, in __init__
    super(QtWidgets.QWidget, self).__init__(parent=parent)
TypeError: PySide2.QtCore.QObject isn't a direct base class of Panda3dQWidget

The package available on Pypi for windows should bundle every required dynamic libraries, namely Eigen3, Eigenpy, Boost (only the few required components), Urdfdom, and Pinocchio, since it is quite difficult to install them for Windows so far.

Eigen already offers some very handy functionalities to ensure no memory is allocated at runtime using:

#define EIGEN_RUNTIME_NO_MALLOC
Eigen::internal::set_is_malloc_allowed(false);

stable_baselines is not moving forward and rely on tensorflow < 1.15. Additionally, some important features are not supported by Windows.

On the contrary, ray[rllib] is not specific to any backend learning framework (at least in principle) and moving forward very fast. Very promising features are under development. It is very easy to use and highly customizable. Finally, the support of Windows is coming very soon (<1 month). Yet, the learning curve is steeper than stable_baselines .

• SubprocVecEnvLock hangs if gepetto-viewer window is closed during rendering
• Rendering of vector env is not working anymore

Hi,

I am trying to build jiminy with a virtual env : essentially just an other python located on an other folder.
I was able to build everything, jiminy is accessible to python on the venv, but eigenpy and pinocchio were linked to the wrong python (the system-wide python).

Thus, when I try to import jiminy from the venv python, I get the following :

(venv) oscar@oscar-Aspire-A515-54G:~/workspace/jiminy$ python3 -c "import jiminy_py"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/oscar/workspace/jiminy/venv/lib/python3.8/site-packages/jiminy_py/__init__.py", line 1, in <module>
    from .core import *  # noqa: F403
  File "/home/oscar/workspace/jiminy/venv/lib/python3.8/site-packages/jiminy_py/core/__init__.py", line 43, in <module>
    _module = _importlib.import_module(".".join((__name__, module_name)))
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'jiminy_py.core.eigenpy'

I guess jiminy was not build to support venv. But I would like to know where I can choose the linked python for eigenpy, pinocchio (and other I might have missed).

The initial state of the simulation is associated to the rigid model, whereas everywhere else the state corresponds to the current model, which can be flexible or rigid. I think the initial state should be consistent with the current model by default, with an additional flag to specify if is a rigid or flexible.

It would be more consistent, and enable to use a proper telemetry namespace for the engine. In addition, the telemetry namespace should be exposed as a static variable in Python.

It only requires to restore sensors and motors originally attached to the robot when building robot from log, which is not the case for now. This could be done by enabling exporting hardware configuration file from a robot.

fext is not updated based on the value of the lagrangian multipliers of the problems for user-specified constraints.

One application which I find interesting for jiminy would be the ability to simulate more finely interactions between an robot exoskeleton and the person wearing it.

Currently jiminy can only simulate a pair of model (kinematic tree) and controller. I wonder whether it could be extended to handle multiple models along with force interactions between the model respective frames. This would bring jiminy closer to being able to run simulations on a kinematic graph rather than a simple kinematic tree (see for instance gazebo which can handle graphs).

Codacy detected an issue:

Message: multiprocessing.active_children is not callable

Wrong error report. It is callable according to the official doc https://docs.python.org/3.6/library/multiprocessing.html.

Occurred on:

• Commit: 358a804
• File: python/jiminy_py/src/jiminy_py/viewer/viewer.py
• LineNum: 184
• Code: multiprocessing.active_children()

Currently on:

• Commit: b86cf5e
• File: python/jiminy_py/src/jiminy_py/viewer/viewer.py
• LineNum: 185