Video Streaming and WebRTC Samples for Unity.
Unity supports WebRTC, see com.unity.webrtc or github. However, the demos only work in P2P mode, not with remote SFU or SRS.
To work with SFU or WebRTC server, the best practice is to use WHIP for Unity to publish to SFU, such as SRS. Actually, you're also able to play stream by WHIP.
The most common use scenaio for publishing stream, is to covert video game as live streaming. However you can use OBS to capture the window and audio, but it enable you to capture from Unity inside, cool! It works like this:
(Scenario 1)
Unity Game ---WebRTC----> SRS --+-----RTMP--> YouTube/FFmpeg
+---WebRTC--> H5/Unity
And the playing stream can be used for streaming consuming, it's a new feature enabled by Unity WebRTC. It works like this:
(Scenario 2)
OBS/FFmpeg ----RTMP----+--> SRS --WebRTC--> Unity Game
H5/Unity -----WebRTC-----+
It also allows multiple Unity Games to communicate by WebRTC. And there should be a set of new use scenarios for Unity+WebRTC+SRS, that you can finger out and please let us know
We have tested on:
- Unity editor
2020.3.48f1 LTS, other latest LTS versions should work well also. - WebRTC 2.4+, because CaptureStream API changed, see #2 for detail.
The supported versions:
| WebRTC | Supported | Note |
|---|---|---|
| 3.0.0-pre.6 | v1.0.2, v1.0.3 | Stable. Fix #963 |
| 3.0.0-pre.5 | v1.0.1 | Stable |
| 2.4.0-exp.11 to 3.0.0-pre.4 | v1.0.1 | Known issue, see #5 and #882. |
The latest version should work well also, please file an issue if not.
First, please setup you Unity Project. If you're stuck, please get help from Discord.
Step 1: Download and setup Unity Hub.
- Open
Unity Hub. - Click
Installs > Install Editor, please select one release to install. - Check modules for
Unity Editor, (Windows: make sureVisual Studioinstalled).
Step 2: Create a Unity project.
- Open
Unity Hub. - Click
Projects > New Project. - Select
3D Coreor3D URPtemplate. - Set the
Project NametoMy project. - Click
Create project, and anUnity Editorwill be opened.
Step 3: Install dependency package com.unity.webrtc.
- Click
Window > Package Manager. - Click
+ > Add package from git URL. - Input
com.unity.webrtcthen clickAdd.
Step 3.1 Case of Unity 2020.3 or 2021.3:
- Click
Window > Package Manager. - Click
+ > Advanced buttonand enableShow preview packages - Search
webrtcand install the package.
Step 3.2 Case of Unity 2019.4:
- Click
Window > Package Manager. - Click
+ > Add package from git URL. - Input
com.unity.webrtcthen clickAdd.
Step 4: Install srs-unity package.
- Download package SRS.WebRTC.Samples.unitypackage.
- Click
Asserts > Import Package > Custom Package, select the fileSRS.WebRTC.Samples.unitypackage, then clickImport. - From
Projectpanel, openAsserts > io.ossrs > Samples, where you got all samples there.
Step 5: Start SRS WebRTC media server:
CANDIDATE="192.168.1.10"
docker run --rm -it -p 1935:1935 -p 1985:1985 -p 8080:8080 \
--env CANDIDATE=$CANDIDATE -p 8000:8000/udp \
ossrs/srs:5 ./objs/srs -c conf/docker.confNote: Make sure your SRS is
v4.0.264+orv5.0.62+. Please read this guide to setup SRS.
Note: Please remember to replace the
CANDIDATEto your server IP, please read link for details.
Note: For online service, you might need authentication and other features, please read How to Setup a Video Streaming Service by 1-Click or How to Setup a Video Streaming Service with aaPanel to build a service by SRS.
Bellow is detail guide for different use scenarios.
To publish your WebCamera and Microphone using WebRTC. If you're stuck, please get help from Discord.
Please follow Setting up project, then work with Publisher sample.
- From
Projectpanel, openAsserts > io.ossrs > Samples > Publisher, then open theScene. - Click
Edit > Playto play Unity scene, which publish WebRTC stream to SRS. - Open Main camera object in the editor and under the SRS publisher script, change the ip address and stream key to your e.g.
http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream - Play the WebRTC stream by H5.
Note: Note that the
Publisherscene require the WebCamera and Microphone permission, you can try other sample if no device.
The stream flows like this:
Unity ---WebRTC---> SRS --WebRTC--> H5/Chrome
(WebCamera and Microphone)
Note: You could use other WebRTC media server and client to replace SRS and Chrome.
1662719512637159.mp4
Note: The latency is extremely low, which allows you to communicate with another Unity App. For example, you can use this mode to build a metting application by Unity.
To publish your game camera and voice using WebRTC. If you're stuck, please get help from Discord.
Please follow Setting up project, then work with Streamer sample.
- From
Projectpanel, openAsserts > io.ossrs > Samples > Streamer, then open theScene. - Click
Edit > Playto play Unity scene, which publish WebRTC stream to SRS. - Open Main camera object in the editor and under the SRS streamer script, change the ip address and stream key to your e.g.
http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream - Play the WebRTC stream by H5.
Note: Note that the
Streamerscene grab an extra camera in game and voice ofMain Camera, and we add an example audio clip from hls.
The stream flows like this:
Unity ---WebRTC---> SRS --WebRTC--> H5/Chrome
(Game camera and voice)
Note: You could use other WebRTC media server and client to replace SRS and Chrome.
1662715460552658.mp4
Note: A rotating cube is in the video stream, which is also the demostration of WebRTC.
1662727649470018.mp4
Note: Please note that we also capture the audio of game, so when we mute the H5 player there is no audio.
To play stream using WebRTC. If you're stuck, please get help from Discord.
Note: The stream might be published by another WebRTC client, or live stream like OBS or FFmpeg
Please follow Setting up project, then work with Player sample.
- Publish the WebRTC stream by H5.
- From
Projectpanel, openAsserts > io.ossrs > Samples > Player, then open theScene. - Open Main camera object in the editor and under the SRS player script, change the ip address and stream key to your e.g.
http://localhost:1985/rtc/v1/whip-play/?app=live&stream=livestream - Click
Edit > Playto play Unity scene, which publish WebRTC stream to SRS.
Note: You're also able to publish a live stream and play it in Unity using WebRTC, see WebRTC for Live Streaming.
The stream flows like this:
H5/Chrome ---WebRTC---> SRS --WebRTC--> Unity
Note: You could use other WebRTC media server and client to replace SRS and Chrome.
1662719351347655.mp4
Note: By converting live streaming to WebRTC, you're able to play normal stream from OBS or exists stream to Unity. For example, you can view a live sport or music by Unity.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent