intelligentroboticslabs / arviz

ARViz Project

ARViz

ARViz is an RViz implementation for Augmented Reality. This application runs inside the Microsoft Hololens AR device as a UWP application, without the need to use Ros Bridge. This project has as goal to combine:

• TF2 support for ROS C#.
• Secure communications.
• Hololens Positioning in the scene.

This project uses the UWP port of ROS2 done by Esteve Fernández. You can have a look at the talk he gave at ROSCon 2018 about ROS2 for Android, iOS and UWP to understand the underlying code Video Slides.

This project is funded by ROSIN as a focused Technical Project.

Project mantainers:

• David Vargas ([email protected])
• Francisco Martín ([email protected])

Advisor:

• Esteve Fernández ([email protected])

---

Supported by ROSIN - ROS-Industrial Quality-Assured Robot Software Components.
More information: rosin-project.eu

This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement no. 732287.

Development Status

	Task	Status	Progress
Milestone_1
	Hololens positioning. Hololens node which publish TF2 messages with the device position with respect the world frame.	Finished development of TF2 bindings for rcldotnet at IntelligentRoboticsLabs geometry2 repo (branch: rcldotnet)	100%:white_check_mark:
	User AR Interface	Finished initial version	100%:white_check_mark:
	Final Development of C# bindings	Finished development of Nested types and Collections, needed for TF2 bindings, finished. PRs pending in https://github.com/esteve/ros2_dotnet	100%:white_check_mark:
	AR representation of basic types: scalar, images, pointcloud, laser, TFs, Maps, PoseArray and Paths		100%:white_check_mark:
Milestone_2
	Integration of data types in rcldotnet		100%:white_check_mark:
	Acceptance tests and final design of user AR Interface	Work in progress. Acceptance tests pending.	90%:arrows_counterclockwise:
	Multi robot in same scene	Multiple robots identified in same scene with minimal initial setup required.	100%:white_check_mark:
Milestone_3
	Unitary and integrations tests		100%:white_check_mark:
	System final acceptance questionnaire	Work in progress.	50%:arrows_counterclockwise:
	Final Documentation	Wiki	100%:white_check_mark:

Videos & demos (the images link to the youtube video)

Global positioning system

ARViz GUI and Data Visualization

TFs performance demonstration

AR UI - Topics and Types

---

Intelligent Robotics - intelligentrobotics.es - @IntellRobotLabs

arviz's Issues

`System.DllNotFoundException´ in rcldotnet_common.dll

Hi @dvargasfr
I tried out your Unity solution, but unfortunately it did not work for me.

I Used Unity 2018.2.91f,
the ros2-dotnet-uwp-x84 prebuild release from ros2_dotnet (https://github.com/esteve/ros2_dotnet/releases),
and the Hololens Emulator to test it.

I encountered a deviation from the instruction when I tried to add the reference to the rcldotnet_assemblies.dll, rcldotnet_common.dll and the std_msgs_assemblies.dll.
I got an error message telling me that the references cant be included, because they already exist.

After starting the Hololens Emulator, the application was terminated due to an System.DllNotFoundException in rcldotnet_common.dll

I tried adding thercldotnet_assemblies.dll, rcldotnet_common.dll and the std_msgs_assemblies.dll to the project as well but that did not change anything

Possible collaboration?

Hi,

I noticed that there is a lot of overlap between what you guys are working on and what I have been doing with https://github.com/DynoRobotics/unity_ros2

My main goal is to use Unity as a large scale multi-robot simulator with human avatars.
Here are some examples of what I have been using my project for this far:

I was unfortunately not able to get ros2_dotnet working for the crystal release. Since I needed navigation2 and actions I implemented my own message generator and csharp wrapper with a lot the same new functionality you have been working on (lists of nested types, tf, paths, laser scan etc.)

I also have fairly extensive unit tests in my wrapper (https://github.com/DynoRobotics/rclcs/tree/master/src/dotnet/Test)
which seems to be a future milestone of this project as well.

As @esteve already mentioned in ros2/design#84 this amount of effort duplication is pretty unfortunate...

I'm planning to write a guide on how to install my Unity project and run the navigation2 stack with a turtlebot3 in this week.

Do you think there is some way we can merge the result of our projects? I have already offered to contribute my wrapper and generator as a crystal branch of ros2_dotnet, but that could make it difficult to migrate ARViz to crystal.

Not really sure what the best way forward is... Do you have any ideas?