siemens / ros-sharp

ROS# is a set of open source software libraries and tools in C# for communicating with ROS from .NET applications, in particular Unity3D

ROS# is a set of open source software libraries and tools in C# for communicating with ROS from .NET applications, in particular Unity.

Here are some showcases illustrating what can be done with ROS#. Community provided a variety of other application examples for ROS# here. Please don't hesitate to post yours!

Contents

• Libraries: .NET solution for RosBridgeClient, Urdf and MessageGeneration
• ROS: ROS packages used by ROS#.
• Unity3D: Unity project containing
  • Unity-specific extensions to RosBridgeClient and Urdf and MessageGeneration
  • example scenes and reference code (see Wiki)

Releases

In addition to the source code, Releases contain:

• a Unity Asset Package containing the Unity3D project assets:
  • to be imported in other Unity projects using ROS#.
• binaries of RosBridgeClient and Urdf
  • to be used in other .NET projects using these libraries.

The latest release is also being published in the Unity Asset Store.

Please get the latest development version directly from the tip of this master branch.

Unity Robotics Hub

In 11/2020 Unity launched Unity Robotics Hub and included a major part of ROS#.

• Short Description on Unity Robotics Hub written by Unity developers
• Differences between ROS# and Unity Robotics Hub

We are in close contact with the developers and decided to run both initiatives in parallel.

Licensing

ROS# is open source under the Apache 2.0 license and is free for commercial use.

External Dependencies

RosBridgeClient requires:

• Newtonsoft.Json (MIT License)
• Newtonsoft.Json.Bson (MIT License)
• websocket-sharp (MIT License), required only when using WebSocketSharpProtocol

Platform Support

• ROS# is developed for Windows and has successfully been used on a variety of other platforms community members.

• The RosSharp Visual Studio solution requires .NET Framework 4.6.1 and Visual Studio 2017 or higher.

• The Unity Project Unity3D has been developed with Unity Version 2019.4.18f (LTS) and should be compatible also with older versions. In Versions below 2019.3, make sure to set the scripting runtime version to .NET 4.x Equivalent (see Wiki page).

• Please find a UWP version of ROS# here.

• Please find a .NET Standard 2.0 version of UrdfImporter here.

Further Info

• Read the Wiki
• Contact the project team
• Contributors and Acknowledgements

---

© Siemens AG, 2017-2021

Author: Dr. Martin Bischoff ([email protected])

ros-sharp's Issues

Various problems with the tutorial

Not sure if this is a bug or just me, but I'm trying to follow the tutorial on the wiki. All of the tutorials starting with 1 worked fine, but I get into trouble when doing 2.1. I'm able to start the URDF Asset importer, and it looks like everything is importing fine when pressing the Read Robot Description. button, but...

1. When it's done I'm getting a prompt saying Do you want to generate a null GameObject now?
   
2. The imported null GameObject looks like the TurtleBot, but the buttom (Kobuki) part is turned 90 degrees around the x axis, making it look like a wheel.
   
3. In the Unity console I get a few warnings of the type
   No Joint Component will be created in GameObject "camera_[xxx]_frame" as it has no Rigidbody Component. Please define an Inertial for Link "camera_depth_optical_frame" in the URDF file to create a Rigidbody Component..
   Where [xxx] is rgb, rgb_optical, depth or depth_optical.
   
4. When trying to import the .unity scene from step 2.2 (GazeboSimulationScene.unity) by doing File -> Open Scene, nothing happens. I found the scene in ros-sharp/Unity3D/Assets/RosSharp/Scenes/GazeboSimulationScene.unity since the latest Release (linked to in the tutorial) does not contain it.
5. I tried doing the scene manually, as described in the same tutorial, and it seemed to work fine. But when starting the gazebo-simulation-scene on the Ubuntu VM I get the following error in the terminal: ERROR: cannot launch node of type [gazebo_simulation_scene/joy_to_twist.py]: can't locate node [joy_to_twist.py] in package [gazebo_simulation_scene], and the Gazebo main Window just remains black. I can start Gazebo normally and it seems to work fine.

• My OS is: Windows 10 Pro, version 10.0.16299
• My Unity Version is: 2017.3.1f1
• My Ubuntu (on VirtualBox) Version is: 16.04
• My ROS Distribution is: Kinetic
• My Gazebo version is: 7.12.0 (the one on the ORDF repository).

Thanks in advance, this seems like a great product otherwise and I'm excited to get going :)

Rename Advertize to Advertise in RosSocket

Compare:

ros-sharp/RosBridgeClient/RosSocket.cs

Line 53 in 1423fbb

public int Advertize(string topic, string type)

https://github.com/ros/ros_comm/blob/lunar-devel/clients/roscpp/src/libros/node_handle.cpp#L285

Provide a gentle "getting started"

Non-experts shall be able to try out our code faster and easier.

Thus we should to set up the following two "getting started" example projects:

1 Windows-only Example Project:

• Provide a Unity3D example project with all required binaries from this project (ROSbridgeClient.dll, Urdf.dll) such that the user only has to do the following:

  • include external .dlls (drag and drop in Assets/Plugins folder).
  • import the Robot model from a urdf file (imprted via the Urdf menu)

When clickling play the user can e.g. control the robot with the keyboard.

2 ROS Interaction Example Project:

• Provide a Unity3D example project similar to the one above with the difference that the Robot model is imported from ROS. Additionally to the example above, the user has to establish the Websocket connectinon to ROS (via the ROSbridge menu in Unity3D).

Further we should provide a detailed Wiki page for each example project

For Example Project 1 we should provide download links to:

• Unity3D
• external dlls
• example robot URDF
  and detailed explanations what to do in Unity3D to get the example project running.

For Example Project 2 we should additionaly refer to external Wiki pages explaining:

• how to set up a VM and to install Ubuntu
• how to install ROS
• how to install the example robot on ROS
  What remains to be explained in our Wiki page is the websocket connection to the VM.

Blocked in tutorial 2.1

Hi, I'm having some trouble with the Turtlebot example. I just arrived to the launch step and it gives me this error :
[publish_description_turtlebot2.launch] is neither a launch file in package [file_server] nor is [file_server] a launch file name

I already try to run setup.sh and setup.shell, but i didn't help. Would you have an idea of my problem ?

By the way, there's a misprint in the turtlebot installation, it's not 'ros-kinetic-kobuki-ftd' but 'ros-kinetic-kobuki-ftdi'. So the correct line would be :
sudo apt-get install ros-kinetic-turtlebot ros-kinetic-turtlebot-apps ros-kinetic-turtlebot-interactions ros-kinetic-turtlebot-simulator ros-kinetic-kobuki-ftdi ros-kinetic-ar-track-alvar-msgs ros-kinetic-turtlebot-gazebo

Running rosbag commands from C#

• I am at the right place and my issue is directly related to ROS#. General technical questions I would post e.g. at ROS Answers or Stackoverflow. For library-specific questions I would look for help in the corresponding library forums.
• I have throughly read the Contributing Guideline and writing this issue is the right thing to do in my case.

---

I have a question!

• I searched the Wiki, open and closed issues for an answer. I tried my best in finding the answer by myself without success. I believe that the discussion we will have in this issue, and the solutions we might find, will help me, and likely other community members who have a similar problem.

Here is my question:

---

I found a bug!

• I am using the latest ROS# version available here on the master branch.

• I am adding all required information, code and data files, screenshots and log files so that you can reproduce the problem.

• My OS is:

• My Unity Version is:

• My ROS Distribution is:

• My Build Target Platform is:

Here is my bug description:

Perform the following steps reproduce the bug:

Observed results:

Expected results:

Relevant Code:

<!-- Insert code here, 
-->

---

I have a feature request!

I want to do this:

For being able to do it, I wish ROS# had the following feature:

Talker code on ROS end

• I am at the right place and my issue is directly related to ROS#. General technical questions I would post e.g. at ROS Answers or Stackoverflow. For library-specific questions, I would look for help in the corresponding library forums.
• I have thoroughly read the Contributing Guideline and writing this issue is the right thing to do in my case.

---

I have a question!

• I searched the Wiki, open and closed issues for an answer. I tried my best in finding the answer by myself without success. I believe that the discussion we will have in this issue, and the solutions we might find, will help me, and likely other community members who have a similar problem.

Here is my question:

Hi,

I was looking at this example here.
I am wondering how should the talker look like in order to receive the published
message.data = "Hello!";

If I am not wrong, is this right?

#include "ros/ros.h"
#include "std_msgs/String.h"
#include <sstream>
int main(int argc, char **argv)
{
  ros::init(argc, argv, "my_node");
  ros::NodeHandle n; 
  ros::Publisher chatter_pub = n.advertise<std_msgs::String>("talker", 1000);
 
   std_msgs::String msg;
  ROS_INFO("%s", msg.data.c_str());
  chatter_pub.publish(msg);
  ros::spin();
  return 0;
}

Clarification about the tutorials step 1.4

Hi Devs,

I am completely new to ROS and I am trying to follow the tutorials in your wiki to control a robot in Unity 3D via ROS. Until the stage where I have to build the file_server (tutorial part 1.3), I could follow. I am having issues in stage 1.4 and 3.1:

• especially, if the 'getting_started2.launch' file already exists or should I create one?
• If I created one, in which ROS package's launch folder should I place it, I tried file_server/launch, but the launch command didn't work?
• should I place it in rtmros_nextage/rtmros_nextage/launch(create the launch folder?) ?

Also, should the ros-sharp main package be cloned to catkin_ws/src or just the file_server suffices?

If the questions come across as silly, please do consider that I am totally new to ROS and still trying to understand how the mechanism of packages, wrokspaces, etc. function!

Thanks in advance,
Anirudh

How to subscribe to custom messages using ros-sharp?

Please help me how to define headers for custom messages and subscribe them in Unity. Here are messages defined https://github.com/siemens/ros-sharp/blob/master/RosBridgeClient/Message.cs. I think there should be separate files to define each type of message.

Tutorial 3.1 folder structure needs clarification

In Tutorial 3.1 Importing a URDF on Windows, it suggets that all you have to do is put the shadowhand.urdf file under Assets\Urdf\simox_ros to be able to import the URDF file. But you also have to put the mesh (.dae) files found in simox_ros\sr_grasp_description\meshes in the Assets\Urdf\simox_ros\sr_grasp_description\meshes folder to get it to work.

rosbridge_suite not sending custom message types

Rosbridge status - Client Connected, Subscribed to /topic.
Also 'rostopic echo /topic' is giving the right output.
Under subscriber callback function in Unity, printing Message, parsing as MessageType and printing, both are giving Null Object. Help, please.

URDF Importer does not work with descriptions containing no meshes

I found a bug!

• My OS is: Windows 10
• My Unity Version is: 2017.1.1f1
• My ROS Distribution is: ROS Kinetic

Here is my bug description:
Importing URDF scenes without meshes does not work.

Perform the following steps reproduce the bug:
Try to import URDF scene containing primitives (e.g. box, cylinder) and no meshes.

Observed results:
No imported scene.

Expected results:
The scene should be imported.

Unity Asset folder structure for Urdf file and resources

It seems only the following folder structure is valid:

robot_description_dir
   urdf_dir
      myrobot.urdf
      meshes_dir
         link1.stl
         link2.stl
         etc

Packages with the following structure fail:

robot_description_dir
   urdf_dir
      myrobot.urdf
   meshes_dir
      link1.stl
      link2.stl
      etc

I've tested this with franka and ur5 and get the same result. I can only import when I change the structure of the directory. Was there a reason to follow this structure on the Unity side or could this be followed to allow for various structures of the robot_description directory?

Fix Inertia Import in Unity3D

UnityEngine.rigidbody specifies the Inertia of a rigid body via:

• Vector3 UnityEngine.rigidbody.inertiaTensor
• Quaternion UnityEngine.rigidbody.inertiaTensorRotation
  see: https://docs.unity3d.com/ScriptReference/Rigidbody-inertiaTensor.html

URDF specifies the inertia of a rigid body via:

• the 6 upper-diagonal elements of the 3x3 inertia matrix
  i_xx, i_yy, i_zz, i_xy, i_xz, i_yz,
  see: http://wiki.ros.org/urdf/Tutorials/Adding%20Physical%20and%20Collision%20Properties%20to%20a%20URDF%20Model

In order to import URDF's inertia matrix in Unity3D we have to:

1 diagonalize the matrix D = RIR'

• D is the diagonalized matrix
• R is a transformation matrix with norm 1, i.e. a rotation matrix
• I is the URDF inertia matrix
• R' is the transpose of R

2 fill Unity's inertiaTensor with the 3 diagonal elements of D

3 convert the rotation matrix in a quaternion and store it as inertiaTensorRation

We should try to use simple Linear Algebra and preferably no additional libraries, or at least very common math libraries.

How to import STL files to Unity3D?

Currently UrdfImporter only supports the import of .dea (Collada) files.

URDF also supports STL, whereas Unity3D does not (without plugins).

How shall we realize the STL import in UrdfImporter? I see the following options:

• implement an own STL Importer
• include existing open source STL importer (e.g. https://github.com/karl-/pb_Stl)
• refer to an exisitng importer assets from Unity3D Asset Store (e.g. https://www.assetstore.unity3d.com/en/#!/content/3397). I haven't found a free one there yet.
• give users the advice to convert all stl meshes to dae before importing (http://lmgtfy.com/?q=stl+dae+converter)

how to realize ROS serviceServer in Unity .cs

I have a question!

Here is my question:

hello

I download ros-sharp a few days ago and flowing the Wiki to realize ROS Communication step by step. it's realy helpful. In Wiki Info_Showcases page, here says ROS# can Subscribe and publish topics, call and advertise services, but I have not find the way use RosSocket in unity cs files to realize ROS serviceServer , to be called by local ROS node. How to realize ?
does it relate with rosbridge_suite?

• My OS is: Ubuntu 16.04
• My Unity Version is: Unity-2017.3.0f1
• My ROS Distribution is: kinetic
• My Build Target Platform is: Linux 64bit

Which .NET Platform should be supported by UrdfImporter and RosBridgeClient?

We developed ROS# primarily for communicating with ROS from within Unity3D.
Official Unity3D Versions still support .NET Framework 3.5 only.

For Non-Unity3D-Applications of ROS#, UrdfImporter and RosBridgeClient could be provided in a more recent, "standardized" .NET platform.
Currently discussed versions are:

• .NET Core for RosBridgeClient
• .NET Standard 2.0 for UrdfImporter
  See https://docs.microsoft.com/en-us/dotnet/standard/ for an overview on .NET platforms.

Please use this issue to let us know the platforms you requrire for your project.
Please support us by providing suggestions how to manage multiple Platform releases in one Visual Studio Project as upgrading from .NET Framework 3.5 would complicate the applicabilty in Unity3D.
@blommers

How to include additional standard types (eg Float, Int, etc)

I have ROS running a project and broadcasting a float, but get an error on the Unity side since the MessageTypes.Dictionary doesn't have mapping for it. String is mapped ( { "std_msgs/String", typeof(StandardString) }, ) but I do not understand how this is populated in order to correctly extend it to the other standard types.

My question is how do I extend the RosBridgeClient to support the other standard types and how are they populated?

Thanks

Unity3D Camera Feed to ROS Topic /image.

We get Texture2Ds in Unity which can be converted to Jpeg or Pngs and further base64 encodings. Can these images be transferred over RosBridge to convert it into an image topic.

UrdfJointExtensions not being called to augment GameObject names with joint type

Hi,

I am using the release version of the RosSharp unity package from Release/RosSharp.unitypackage to import .urdf files from a running instance of ROS. I'm able to import without any issues except that the GameObject names do not get augmented with the joint types which must be handled by the RosSharp/Editor/UrdfImporter/UrdfJointExtensions.cs script. I have checked my urdf file and have no other issues such as displaying it in rviz. I can't seem to figure out what I'm doing different, or maybe I'm missing something.

Thanks!

Unity Texture2D to ROS CompressedImage Message

• I am at the right place and my issue is directly related to ROS#. General technical questions I would post e.g. at ROS Answers or Stackoverflow. For library-specific questions I would look for help in the corresponding library forums.
• I have throughly read the Contributing Guideline and writing this issue is the right thing to do in my case.

---

I found a bug!

• I am using the latest ROS# version available here on the master branch.

• I am adding all required information, code and data files, screenshots and log files so that you can reproduce the problem.

• My OS is: Ubuntu 16.04

• My Unity Version is:

• My ROS Distribution is: ROS Kinectic

• My Build Target Platform is:

Here is my bug description:

When sending Texture2D from unity to ROS as SensorCompressedImage Message, rosbridgeserver produces an error.

Perform the following steps reproduce the bug:

https://github.com/siemens/ros-sharp/blob/master/Unity3D/Assets/RosSharp/Scripts/CameraImageManager.cs
https://github.com/siemens/ros-sharp/blob/master/Unity3D/Assets/RosSharp/Scripts/CameraImagePublisher.cs

Observed results:

Expected results:

Texture2D should be received as ROS Messages.

Relevant Code:

<!-- Insert code here, optionally attrach code or data files to this issue. -->

Mouse movements are not being transferred correctly to Unity in Tutorial 2.3

I found a bug!

• My OS is: Windows 10 Pro, version 10.0.16299
• My Unity Version is: 2017.3.1f1
• My Ubuntu (on VirtualBox) Version is: 16.04
• My ROS Distribution is: Kinetic

Bug description

In tutorial 2.3 you are supposed to move the mouse in a Ubuntu Window and watch the robot move. There seems to be a bug with a subscriber in Unity that makes the message not being received correctly.

Steps to reproduce

1. [On Windows machine] Start a new empty Unity project.
2. [On Windows machine] Import RosSharp.unitypackage through Assets -> Import Package -> Custom Package.... Select all Assets for import.
3. [On Windows machine] In the project tab, navigate to RosSharp -> Scenes, and double-click on UnitySimulationScene.
4. [On Ubuntu machine with ROS] Launch the unity simulation scene nodes:
   roslaunch unity_simulation_scene unity_simulation_scene.launch
5. [On Windows machine] Press the Play button to start the scene.

Observed results

On the Ubuntu VM, the terminal outputs the mouse position (seemingly) correctly. The terminal also outputs a message that a client was connected when the Unity scene starts.

On the Windows machine, the simulation starts and the box falls to the ground, but the robot doesn't move when the mouse is moved in the Ubuntu window. The terminal also displays some warnings and an error message,

When stopping the Unity scene, the Ubuntu terminal displays that a client was disconnected.

Update links for GitHub

Check links in the following files and update them for GitHub if necessary:

• CODE_OF_CONDUCT.md
• CONTRIBUTING.md
• LICENSE.md
• README.md
• ROSbridgeClient/README.md
• URDFsharp/README.md
• file_server/README.md

Wiki:

• home.md
• using-rossharp-to-import-urdf-from-ros.md

Cleanup Tutorial Projects

• Move "Getting Started" Projects form "Example" folder to "Tutorial", then"Example" folder
• Update tutorial projects to latest RosSharp.unitypackage

ROS Bridge Client Error

• I am at the right place and my issue is directly related to ROS#. General technical questions I would post e.g. at ROS Answers or Stackoverflow. For library-specific questions I would look for help in the corresponding library forums.
• I have throughly read the Contributing Guideline and writing this issue is the right thing to do in my case.

---

I have a question!

• I searched the Wiki, open and closed issues for an answer. I tried my best in finding the answer by myself without success. I believe that the discussion we will have in this issue, and the solutions we might find, will help me, and likely other community members who have a similar problem.

Here is my question:
I have installed Unity3D in my Linux machine and ROS-Unity3D seems to be communicating properly. I am experimenting Unity Simulation using TurtleBot and ROS seems to be controlling the robot. I would like to know what this error means in that attached screenshot of my simulation. Am I getting this error because I am not using Windows platform since my understanding is that ROS Bridge Client runs on the Windows platform?

---

I found a bug!

• I am using the latest ROS# version available here on the master branch.

• I am adding all required information, code and data files, screenshots and log files so that you can reproduce the problem.

• My OS is: Ubuntu

• My Unity Version is: 2018.2.0b7 Linux Experimental

• My ROS Distribution is: ROS Kinetic

• My Build Target Platform is:

Here is my bug description:

Perform the following steps reproduce the bug:

Observed results:

Expected results:

Relevant Code:

<!-- Insert code here, optionally attrach code or data files to this issue. -->

---

I have a feature request!

I want to do this:

For being able to do it, I wish ROS# had the following feature:

New ROS Message Handling Framework in Unity

I am looking for a simple and general concept, how Subscriber-Processor-Manager classes and also Manager-Processor-Publisher classes should interact, i.e. which class calls the other and when.

Publications:
I typically make the manager class the driving one, calling the processor, which then calls the publisher when ready, The manager class is called every Update().

For publishing a huge amount of data and infrequent changes, it is worthwile to have a parallel process that is not called every Update(). I experimented with calling it every Nth update, implementing an own timing, and also event-driven calls. I used Backgroundworkers in those cases.

Subscriptions:
My typical concept is to process the message as soon as it is received. In the next Update() the Manager checks the Processor for new data to apply on the GameObject.

When doing major updates on a scene (e.g. updating a 3D point cloud camara image), I shift an update over multiple frames. This has both a better performance, and provides a smoother change of the point cloud.

My question is:
Do you have any suggestions on a simple yet general interaction of Publisher & Subscriber classes? Do you have any improvement ideas for my concepts above?

The Class Naming Guidelines provide further info on Publisher, Subscriber, Processor and Manager classes.

SdfImporter Feature Request

I've just read the ROS# announcement, and saw that you use URDF to simulate robots. This format is not suitable for description of simulation environments. The better format to use is http://sdformat.org/ used by Gazebo. It allows for other important specifications like joint limits, friction coefficients etc... And for those left only with a URDF model, there is the gzsdf tool that converts URDF to SDF (which, of course, doesn't contain any of the simulation-specific tags).

But anyways, great work!

Texture2D to CompressedImage ROS lagging on the client.

• I am at the right place and my issue is directly related to ROS#. General technical questions I would post e.g. at ROS Answers or Stackoverflow. For library-specific questions I would look for help in the corresponding library forums.
• I have throughly read the Contributing Guideline and writing this issue is the right thing to do in my case.

---

I found a bug!

• I am using the latest ROS# version available here on the master branch.

• I am adding all required information, code and data files, screenshots and log files so that you can reproduce the problem.

• My OS is: Ubuntu 16.04

• My Unity Version is:

• My ROS Distribution is: ROS Kinectic

• My Build Target Platform is:

Here is my bug description:

Texture2D is being published using these scripts:
https://github.com/siemens/ros-sharp/blob/master/Unity3D/Assets/RosSharp/Scripts/CameraImageManager.cs
https://github.com/siemens/ros-sharp/blob/master/Unity3D/Assets/RosSharp/Scripts/CameraImagePublisher.cs
But the FPS received on the Client from Unity is very low and the video is lagging.

Perform the following steps reproduce the bug:

Observed results:

Lag observed on the client side. Also the server is hosted on localhost and ethernet powered, so most probably not an issue of network.

Expected results:

No Lag.

Relevant Code:

<!-- Insert code here, optionally attrach code or data files to this issue. -->

Missing JointUrdfDataManager class

It seems that in commit 41f545f the concept of JointUrdfDataManager was introduced in some files, but I can't seem to find the actual class definition. Is there a file missing? Unity issues these compile errors:

And doing a content search for "JointUrdfDataManager" in the repository folder yields these results

Split URDFsharp

• URDFsharp.dll shall be made independent from Unity3D. It shall provide the functionality to import URDF files as C# objects.

• The import of the URDF C# in Unity3D shall be realized by separate .cs scripts which can directly be included as Unity3D assets.

• All Unity3D resources required to import URDF files (including the URDFsharp.dll) shall then be assembled as a .unitypackage.

Missing Xlib dependency for Tutorial 2.3

The mouse_to_joy.py (Tutorial 2.3) needs the Xlib module, but it doesn't say in the tutorials how to install it. It can be installed by running:

sudo apt install python-xlib

Meshes missing from tutorial turtlebot scene

I found a bug!

• My OS is: Windows 10 Pro, version 10.0.16299
• My Unity Version is: 2017.3.1f1
• My Ubuntu (on VirtualBox) Version is: 16.04
• My ROS Distribution is: Kinetic
• My Gazebo version is: 7.12.0, running on Ubuntu VM

Bug description

Turtlebot meshes are missing from turtlebot imported through GazeboSimulationScene used in the tutorials.

Steps to reproduce

1. Start a new empty Unity project.
2. Import RosSharp.unitypackage through Assets -> Import Package -> Custom Package.... Select all Assets for import.
3. In the project tab, navigate to RosSharp -> Scenes, and double-click on GazeboSimulationScene.

Observed results

A ground plane, a camera and a sunlight is visible, as well as a magenta rectangle.

Selecting the robot in the Hierarchy, you can see the collision geometries

Zoomed-in picture

If you navigate the Hierarchy of the turtlebot until you get to a visual with a mesh, you can see that the mesh seems to be missing

Communication with Ros bridge fails with built game

Unity Version : 2017.2.0f4
Platform : Linux Ubuntu 16.04

Below I describe an issue encountered when building the game and running the built game. This issue does not appear if the game is executed from the Unity Editor.

Building the application for Linux x86_64 works. Then running the application, the websocket client successfully gets connected to the rosbridge node but there appears to be no message transiting on the topics and a warning is raised by the rosbridge node :

[WARN] [1512640449.098013]: Inbound TCP/IP connection failed: connection from sender terminated before handshake header received. 0 bytes were received. Please check sender for additional details.

NullReferenceException when receiving publication without id

I found a bug!

• My OS is: Windows 10 PRO, version 1803
• My ROS Distribution is: ROS Kinetic

Here is my bug description:
When a RosSocket subscriber receives a message published via rostopic pub, the Subscribe method will throw a NullReferenceException. This seems to be due to a missing "id" tag in the JSON object returned from the RosBridge server.

Perform the following steps reproduce the bug:

1. Compile the program described here: https://github.com/siemens/ros-sharp/wiki/Info_CodeExample
2. Start a rosbridge server on a Ubuntu machine with ROS: roslaunch rosbridge_server rosbridge_websocket.launch
3. Start the compiled program.
4. On the Ubuntu machine, publish a message to the /listener topic: rostopic pub /listener std_msgs/String "data: 'hello'"

Observed results:
The program crashes and throws a NullReferenceException.

I think the error is due to a change in the method GetServiceId in JObjectExtensions.cs introduced with 802f47a. It now throws an exception instead of returning 0 when the id tag was not found.

Add List of Dependencies in ROSbridgeClient/README.md

specify project internal and external dependencies of the ROSbridgeClient solution similar to URDFsharp/README.md.

RosBridgeClient Refactoring

We are just starting a major refactoring of the RosBridgeClient library.

Here are some things that we want to achieve:

• add generic communication protocol interface
  This enables us to easily switch between websocket-sharp, Websocket-Sharp-UWP .NET Websocket and to add other (also non-WebSocket) communication protocols in the future. Thanks to @jaguar243 for suggesting this in issue #21 months ago!
• add NUnit tests
• improve readability, code structure and performance

You can follow the current state of development on my fork.

What are you using ROS# for?

We are very curious what you use ROS# for!

Let us use this issue to present our projects, whether finished, ongoing, or first idea.

I start! Here is our first public project:

A Tutlebot 2 is teleoperated via touchpad controllers HTC Vive.
A 3d point cloud of the camera image is illustrated in VR, as well as the mapping data and the moving turtlebot. The wheels of the turtlebot are animated via JointState messages. Users can teleport to different positions in the scene.
Further, users can switch between teleoperation and simulation mode in real time. In the simulation mode, RosBridgeClient disconnects and Users control the Urdf Simulation model by applying a torque at the wheels. It feels quite similar with the VR glasses on. The only difference is that the camera image is missing and the map does not update anymore. 😏

Problems while trying to run Getting Started 2 tutorial

Hi all,

First of all, thanks for providing this ROS package, it's been very helpful so far. I am also using ROS in combination with Unity for my masters thesis.

When I run your tutorial Getting Started 2, I'm facing two problems.

First, the ros-sharp/UrdfImporter/UrdfSharpUnity3D folder mentioned during "Setting up Unity" section does not exist anymore. But I was able to get everything working by copying and pasting the Unity3D/Assets/RosSharp folder on the Assets folder of my project. Should we change the wiki? Also, I noticed that there is a unitypackage file inside Release folder. I included this file and I was able to successfully see the URDF Model in 3D Object menu.

And second, and more critical, I could successfully import the model of the robot in Unity, but when I clicked on play, the client does not connect and subscribe to the robot model topics. So, I can see my robot moving in Rviz when I click on randomize, but I cannot see it moving in Unity. Any tips on why this is not working as expected?

Cloning rosbridge does not install all dependencies

Hi all,

I know you guys are improving the documentation, but I would like to give a heads up about the current steps.

At least for me, using rosbridge by cloning its repo didn't work. Even after compiling it. I got this error while running the launch file:

Traceback (most recent call last):
  File "/home/lucas/catkin_ws/src/rosbridge_suite/rosapi/scripts/rosapi_node", line 36, in <module>
    from rosapi import proxy, objectutils, params
ImportError: No module named rosapi
Traceback (most recent call last):
  File "/home/lucas/catkin_ws/src/rosbridge_suite/rosbridge_server/scripts/rosbridge_websocket", line 39, in <module>
    from tornado.ioloop import IOLoop
ImportError: No module named tornado.ioloop

and then rosbridge node crashed.

However, installing rosbridge like described in their documentation http://wiki.ros.org/rosbridge_suite works like a charm.

sudo apt-get install ros-kinetic-rosbridge-server

Robot links seperated with UrdfImporter

When I try to import my own robot URDF file into Unity, all the links are separated. This may be due to the wrong transformation between links. However, the model works well after MoveIt! setup and interacting in RViz...any reasons that may cause that issue?

Wrong paths when importing URDF model in linux

I tried to import the Nextageopen URDF in Unity ;

The URDF is correctly imported but actually copied in an inappropriate path ;

My project is named DroneControl and, instead of copying the assets in DroneControl/Assets/Urdf , it copied it into DroneControl\Assets\Urdf and therefore the object does not appear in your project in Unity , because "DroneControl\Assets\Urdf" is not a subdirectory of DroneControl.

If I copy the assets in the appropriate path , the model is correctly displayed within Unity. So , it is just a question of "/" and "" in the path of the UrdfImporter (I suppose) ;

Evaluate ROS - Unity Communication Performance

I have a question!

I'm implementing an application where I want Unity to listen for ROS messages on a topic and, as soon as one is received, publish another message on a separate topic. It's important that the response from Unity is as fast as possible, preferably below 5 ms.

I have devised the following setup:

• A Publisher that launches a new thread containing a function with a while(true) loop. The loop publishes a message and then waits for an AutoResetEvent block to be released.
• A Receiver that, when a message is received, releases the Publisher block and allows the loop to go one more round.
• A regular Subscriber using the Receiver as its MessageReceiver.

I'm using ROS on a separate Ubuntu machine to send messages, and then using Wireshark to measure timings. Some observations I've made:

1. The time it takes from publishing the message from ROS, to when a message is received again by ROS, varies, with the shortest time being ~1 ms.
2. The time from 1 varies with the Fixed Timestep (Edit > Project Settings > Time). For large timesteps (> 20 ms, 0.02 s), the largest time seems to vary between 1 ms and the timestep.
3. Decreasing the Fixed Timestep below 20 ms has no further effect on the time described in 1.
4. I have made the same setup in Simulink, using the ROS capabilities from the Robotics System Toolbox. There I've managed to get the response time down to a steady 4-5 ms, so I don't think my issue has to do with ROS. Although Matlab does not use rosbridge as far as I know, so it could be that.

But my question is: What is it that decides when a Subscriber handles a message, and when a Publisher dispatches it? Does it somehow depend on the Fixed Timestep?

Thanks for a great product!

A few wiki fixes

Fixed a few typos and added some missing steps. Link to my fork.

• Section 1.4.1 - 'git clone' command missing '.git'
• Section 3.2.1 - Spelling, added steps to create the referenced GameObjects, formatting

Link I found useful for merging wikis:

Which Publisher and Subscriber Examples should be provided in the ROS#/Unity3D project?

ROS#/Unity3D currently provides the following Subscription / Publication Examples:

Subscriber-Examples for the Topics:

• /joint_states https://github.com/siemens/ros-sharp/blob/master/Unity3D/Assets/RosSharp/Scripts/JointStateSubscriber.cs
• /odom https://github.com/siemens/ros-sharp/blob/master/Unity3D/Assets/RosSharp/Scripts/OdometrySubscriber.cs
• /cmd_vel https://github.com/siemens/ros-sharp/blob/master/Unity3D/Assets/RosSharp/Scripts/VelocitySubscriber.cs (31b27ad)

Publication Examples for the Topics:

• image_raw/compressed https://github.com/siemens/ros-sharp/blob/master/Unity3D/Assets/RosSharp/Scripts/CameraImagePublisher.cs (31b27ad)

We are planning to include more examples and to exdend above list.

On the one hand we want to keep ROS# generic and easy-to-learn and not overload our library with too much example code. Users should be able to understand ROS# quickly and be able to make their individual extensions and adjustments for their robot’s particularities in Unity3D scripts.
On the other hand we want to have a preferably complete and ready-to use library.

Let us use this issue to discuss ideas which Subscription / Publication Examples are worth including.
@lucascoelhof @jeremyfix

How to fix the menu 'RosBridgeClient' couldn't show after import?

---

I have a question!

• I searched the Wiki, open and closed issues for an answer. I tried my best in finding the answer by myself without success. I believe that the discussion we will have in this issue, and the solutions we might find, will help me, and likely other community members who have a similar problem.

Here is my question:

I try Assets→Import Package→Custom Package
and choose RosSharp.unitypackage to import.
After I import it, I can see the RosSharp Asset, but I can't see the menu has RosBridgeClient.

1. I try run the following code and re-import again, but failed.

cd ~/code/Unity_project/ai2thor/Assets/
mv RosSharp RosSharp.bak
mv RosSharp.meta RosSharp.meta.bak
mv RosSharp.bak ../
mv RosSharp.bak.meta ..

2. I make sure I install the following packages:
   sudo apt-get install ros-kinetic-turtlebot ros-kinetic-turtlebot-apps ros-kinetic-turtlebot-interactions ros-kinetic-turtlebot-simulator ros-kinetic-kobuki-ftdi ros-kinetic-ar-track-alvar-msgs ros-kinetic-turtlebot-gazebo ros-kinetic-rosbridge-*

3. I try to reboot my ubuntu16.04 and rerun Unity but failed.
   My installation code is:

cd ~/code/download
wget https://beta.unity3d.com/download/e685bfa3c483/UnitySetup-2017.4.4f1
chmod +x UnitySetup-2017.4.4f1
./UnitySetup-2017.4.4f1

What can I do next?
Thank you~

RosBridgeClient for UWP

Hello,

I would like to send an array of float values as a std_msgs/String. However, I could not locate many examples for publishing message from Unity to ROS except for CameraImagePublisher.cs. Is it possible using ros-sharp?

My message would look something like this: "[718.23, 234.21, 324.54 . . . . .. ]"

Also, after importing ROS-sharp as given here. The menu items do come, but when I am building it for UWP,

the following errors are thrown:

Assets\RosSharp\Scripts\JointStateSubscriber.cs(21,30): error CS0246: The type or namespace name 'RosSocket' could not be found (are you missing a using directive or an assembly reference?)

Assets\RosSharp\Scripts\JointStateSubscriber.cs(45,40): error CS0246: The type or namespace name 'Message' could not be found (are you missing a using directive or an assembly reference?)

Assets\RosSharp\Scripts\OdometrySubscriber.cs(47,44): error CS0246: The type or namespace name 'NavigationOdometry' could not be found (are you missing a using directive or an assembly reference?)

I don't know why these errors are thrown as in the plugins the ROSBridgeClient.dll is already imported. I am building it as a C# project, to deploy onto my mixed reality device.

Fix Joint Rotation Limits

URDF Joint Limits are already correctly imported to Unity3D.
However Unity3D allows only Joint Limits in the range from -180° to 180°.

A possible solution would be a script that counts the number of rotations and activating a joint limit only if we are in the minimum / maximum rotation number.

Concerns about performance and FixedUpdate

Hi!

Earlier I had some thoughts about ROS - Unity communication performance (see #55), and I've now done some more testing and want to share the results. I also have some concerns about one test which I'd like some feedback on if possible.

Scripts

I am appending the scripts used in this issue: 2 C# Unity scripts using ROS#, 1 C++ and 1 CMakeLists.txt file to create a ROS package.

C#/Unity

Files: unity.zip

• SimplifiedPublisher - Publishes a string. The publication can be set to occur either every time FixedUpdate is called; or to be related to a call to a trigger method. If the PublicationType enum is set to FixedUpdateTrigger, the trigger method will make sure the message is published the next time a FixedUpdate callback is called. If it is set to FastTrigger, the message will be published immediately.
• SimplifiedPublishSubscriber - Calls the trigger method of the SimplifiedPublisher script when a string message is received.

C++/ROS

Files: ros.zip

• listen_and_talk.cpp - ROS node that subscribes to the /unity_talker topic, and publishes "ros" to the /ros_talker topic each time a message is received.
• CMakeLists.txt - CMakeLists

To compile into a ROS node (execute at the top level of a Catkin workspace):

cd src
catkin_create_pkg unity_listener std_msgs roscpp
cp [path to unity_listener.cpp] unity_listener/src
cp [path to CMakeLists.txt] unity_listener
cd ..
catkin_make

and run using

rosrun unity_listener unity_listener

Tests

I think all of these tests should be simple to reproduce using the scripts provided above. All test cases use a Unity scene with a single GameObject, which is described under each subsection. Test cases 2 and 3 also uses the ROS node provided.

All frequencies have been measured using the ROS command:

rostopic hz /topic_name

Test case 1: One-way communication

Unity scene

RosConnector GameObject with the following components

• RosConnector
• SimplifiedPublisher publishing the string "unity" to the /unity_talker topic. Configured to use the FixedUpdate publication type.
  

Results

I ramped down the time step using Edit -> Project Settings -> Time, and the maximum stable frequency achieved was about 4000 Hz.

Test case 2: Two-way communication outside of FixedUpdate

Unity scene

RosConnector GameObject with the following components

• RosConnector
• SimplifiedPublishSubscriber subscribed to the /ros_talker topic.
• SimplifiedPublisher publishing the string "unity" to the /unity_talker topic. Configured to use the FastTrigger publication type.
  

ROS node

Uses the unity_listener node described under the Scripts section.

Notes

You might have to start the communication by manually sending a message to either topic, using for example:

rostopic pub /unity_talker std_msgs/String "data: 'xxx'" --once

Results

The frequency lies around 1000 Hz.

Test case 3: Two-way communication using FixedUpdate

Unity scene

Same as Test case 2, but with the SimplifiedPublisher configured to use the FixedUpdateTrigger publication type.

ROS node

See Test case 2.

Results

The frequency never goes above 60 hz, even when the time step is ramped down. Setting a time step above 1/60 makes the frequency go down accordingly.

Analysis

Test case 1 shows that FixedUpdate is able to publish at an approximate rate of about 4000 messages/second. Test case 2 shows that it is possible to communicate between ROS and ROS# at a rate of about 1000 message/second. Therefore I do not understand why the results look like they do for Test case 3. I also tried to alter the SimplifiedPublisher script to publish to a second topic during Test case 3, and it was able to publish at a rate similar to Test case 1. Do you have any thoughts on this, or if there are other test cases I can make to shed some light?

Import URDF into Unity

Hi Sir,
I've been following your tutorials to learn how to use ROSsharp. My question is could you please show me some other examples of how to import URDF models of other robots(UR5/Baxter/PR2) into Unity? I am less experienced. Thank you so much for your support.

Cheers
Steven

Communicating within linux using Unity linux build

• I am at the right place and my issue is directly related to ROS#. General technical questions I would post e.g. at ROS Answers or Stackoverflow. For library-specific questions I would look for help in the corresponding library forums.
• I have throughly read the Contributing Guideline and writing this issue is the right thing to do in my case.

---

I have a question!

• I searched the Wiki, open and closed issues for an answer. I tried my best in finding the answer by myself without success. I believe that the discussion we will have in this issue, and the solutions we might find, will help me, and likely other community members who have a similar problem.

Here is my question:

---

I found a bug!

• I am using the latest ROS# version available here on the master branch.

• I am adding all required information, code and data files, screenshots and log files so that you can reproduce the problem.

• My OS is:

• My Unity Version is:

• My ROS Distribution is:

• My Build Target Platform is:

Here is my bug description:

Perform the following steps reproduce the bug:

Observed results:

Expected results:

Relevant Code:

<!-- Insert code here, optionally attrach code or data files to this issue. -->

---

I have a feature request!

I want to do this:

For being able to do it, I wish ROS# had the following feature:

Connected but not received in tutorial 2.1

Hello! I am trying to connect ROS node and unity both on a same ubuntu console, and I have tried the tutorial 2.1 to spawn the turtlebot. The picture below is my result. It seems that the ROS node is connected with unity since it responds that a client is connected and disconnected, but unity receives nothing even I increase timeout.

Is there anything I can try to check out what is going wrong? Thank you!!!