stereolabs / zed-unreal-examples

Stereolabs ZED - UE4 Examples

Please update to our Unreal Engine 5 plugin to use the last features of the ZED SDK

As of the release of the ZED SDK 4.0, this sample is deprecated and will not be maintained anymore. To use the most recent features and capabilities of the ZED SDK, we invite you to use our Unreal Engine 5 plugin.

Thanks for your comprehension.

---

Stereolabs ZED - UE4 Examples

This repository contains Unreal examples projects that demonstrate how to create advanced Mixed Reality application using the ZED SDK and Unreal.

UE4_Examples is a single project with a variety of small samples, each meant to highlight a single feature of the ZED UE4 plugin. PingPongMR is a game combining multiple ZED features, and demonstrating how to have a shared multiplayer experience in AR with multiple headsets or observer cameras.

Before You Start

Our plugin requires our SDK and a custom build of the Unreal engine. Follow the steps in our Getting Started and Building the Engine guides before using these samples.

Then, see the readme file in the UE4_Examples and PingPongMR folders for how to build each project.

UE4_Examples

All samples are in the UE4_Examples project as a level, found in Content -> Levels.

Passthrough

See the real world through an Oculus Rift or HTC Vive, with virtual objects mixed in. Must have a compatible headset plugged in and running when the project is loaded.

Lighting

Casts a spotlight that properly illuminates the real world just as well as the virtual. Also has a directional light that causes virtual spheres to cast shadows on the real floor. See our lighting documentation to learn how it works.

Dynamic Crosshair

Puts a 3D crosshair in the center of the screen that moves itself closer or further to be just in front of the real-world object it overlaps. Explore to see how to use the GetDepthAtScreenPosition function, and experiment with the other GetDepth and GetNormal functions included in the plugin.

HitTest

Left click to shoot virtual balls that will stop when they collide with a real-world object. Demonstrates that you don't need to pre-scan an environment for such interactions thanks to the ZED's real-time depth map. See the BP_HitTestBall blueprint and the HitTestReal function to learn how it works.

Object Placement

Click on a flat, upwards-facing surface to spawn a cyborg standing on top of it. Demonstrates how to use the ZED's Plane Detection feature to place objects without pre-scanning. See the level Blueprint and the GetFloorPlaneAtScreenPosition function to see how it works.

Spatial Mapping

Map your environment into a 3D mesh, then shoot balls at real objects that realistically bounce off using Unreal's collision detection and physics. Optionally, save the mesh, mesh texture and .area spatial memory file for later. Use to pre-scan an environment for when you need persistent geometry, such as when collisions with the real-world need to occur even when you can't see them. See our spatial mapping documentation to learn how it works.

Motion Controllers

Makes virtual motion controllers follow real motion controllers tracked by an Oculus Rift or HTC Vive. Must have a compatible headset plugged in and running when the project is loaded.

Open New Level

Initiates the ZED, then properly closes it before opening a new level (in this case, itself). Follow this procedure (visible in the level Blueprint) any time you wish to change levels while the ZED is running.

Advanced

Shows advanced ways to grab the ZED's image from the camera using two example Blueprints: one that retrieves images directly, and one that retrieves via a texture batch.

PingPongMR

PingPongMR puts a virtual ping pong table in your environment. Bring a friend with their own Vive and ZED Mini to compete against you - they'll see the same ball, table and paddles as you. You can even use another ZED and Vive Tracker to view the match from third person in case you want to stream it live.

PingPongMR has additional equipment requirements for multiplayer, and has a more involved setup. See the readme file in the PingPongMR folder for instructions.

Support

If you need assistance go to our Community site at https://community.stereolabs.com/

zed-unreal-examples's Issues

Build the solution

Hello, Think you for your repositories. Build the solution by pressing Ctrl + F5, but I meet some errors. as following:

1. Unable to instantiate module 'Stereolabs': ZED SDK Minor Version mismatch: found 5 expected 7
2. E:\3Dtelepresence\unrealengine\Unreal_ZED\zed-unreal-examples-master\UE4_Examples\ZEDSamples.uproject" -WaitMutex -FromMsBuild” Please verify that you have sufficient permissions to run this command.

I don't deal with this error, please give me some suggestions. I Build the Engine according to https://www.stereolabs.com/docs/unreal/, before starting to build the solution

Unreal Sample Projects Only Shows Black

Hello, finally done compiling custom Unreal Engine 4.21 in 3 days after many unsuccessful try.
Downloaded Zed Unreal plugin for SDK 3.1, and extracted to Plugin folder inside the Zed Unreal Sample Folder, all runs smoothly, no errors, but when I hit play button, searching camera, looks normal, but in the end, it only shows black. What I'm missing?

A problem when I open UE4

Hello，I try to use your repositories at present, and have some problem too. After I finish to build the solution by pressing Ctrl + F5,I start the project with a double click on ZEDSamples.uproject, UE4 will produce some errors:

Do you know why this happened? Can you give me some advice? Think you very much.
(I use the ZED SDK 2.5, CUDA9.0 , the custom engine for 2.5 and UE4.19)

Persistent GPU memory errors

I'm working with the L_Passthrough project and I can't seem to find a way to provide Unreal with a sufficient amount of GPU memory. I have the same issue on two different machines, one with an 8GB card and another with 16GB. No clear reason as to why it might be lacking resources and Task Manager doesn't show any other applications consuming extraordinary amount of GPU memory? Is there something I can do to allocate more GPU memory to Unreal? Simply executing again will yield a positive result, but this is both frustrating and time consuming. TIA.

ZED SDK missing

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues.
• This issue is not a question, feature request, or anything other than a bug report directly related to this project.

Description

download the ZED_unreal_plugin zip from gitup, I want to use it successfully it did not work

Steps to Reproduce

1.follow the step from git and zed documantation
2.
3.
...

Expected Result

run successfully

Actual Result

ZED Camera model

ZED 2i

Environment

ZED SDK version:3.7
vs version:2022
ue version:4.27

Anything else?

no

the image in UE4

Hello ,your respositories has helped me a lot. But now a simple question has been bothering me for some time.i just can see one image in the UE4 window.

What do I do to see two images like this?

Thanks very much！

How can I download the custom distribution of Unreal Engine 4.21 ? #9

It seems the page is 404.
Thank you.