omofolarin / client-sdk-rust

Rust SDKs for LiveKit

📹🎙️🦀 Rust Client SDK for LiveKit

Use this SDK to add real-time video, audio and data features to your Rust app. By connecting to a self- or cloud-hosted LiveKit server, you can quickly build applications like interactive live streaming or video calls with just a few lines of code.

⚠️ Warning

This SDK is currently in Developer Preview mode and not ready for production use. There will be bugs and APIs may change during this period.

We welcome and appreciate any feedback or contributions. You can create issues here or chat live with us in the #rust-developer-preview channel within the LiveKit Community Slack.

Features

• Receiving tracks
• Cross-platform ( currently tested on Windows & MacOS )
• Data channels
• Publishing tracks
• Adaptive Streaming
• Dynacast
• Simulcast
• Hardware video enc/dec
  • NvEnc for Windows
  • VideoToolbox for MacOS/iOS

Crates

• livekit-core: LiveKit protocol implementation
• livekit-utils: Shared utilities between our crates
• livekit-ffi: Bindings for other languages. Uses livekit-core.
• livekit-webrtc: Safe Rust bindings to libwebrtc
• webrtc-sys: Unsafe bindings to libwebrtc

Motivation and Design Goals

LiveKit aims to provide an open source, end-to-end WebRTC stack that works everywhere. We have two goals in mind with this SDK:

1. Build a standalone, cross-platform LiveKit client SDK for Rustaceans.
2. Build a common core for other platform-specific SDKs (e.g. Unity, Unreal, iOS, Android)

Regarding (2), we've already developed a number of client SDKs for several platforms and encountered a few challenges in the process:

• There's a significant amount of business/control logic in our signaling protocol and WebRTC. Currently, this logic needs to be implemented in every new platform we support.
• Interactions with media devices and encoding/decoding are specific to each platform and framework.
• For multi-platform frameworks (e.g. Unity, Flutter, React Native), the aforementioned tasks proved to be extremely painful.

Thus, we posited a Rust SDK, something we wanted build anyway, encapsulating all our business logic and platform-specific APIs into a clean set of abstractions, could also serve as the foundation for our other SDKs!

We'll first use it as a basis for our Unity SDK (under development), but over time, it will power our other SDKs, as well.

Getting started

Currently, Tokio is required to use this SDK, however we plan to make the async executor runtime agnostic.

Connecting to a Room and listen to events:

use livekit::prelude::*;

#[tokio::main]
async fn main() -> Result<()> {
   let (room, mut room_events) = Room::connect(&url, &token).await?;

   while let Some(event) = room_events.recv().await {
      match event {
         RoomEvent::TrackSubscribed { track, publication, participant } => {
            // ...
         }
         _ => {}
      }
   }

   Ok(())
}

Receive video frames of a subscribed track

match event {
   RoomEvent::TrackSubscribed { track, publication, participant } => {
      if let RemoteTrackHandle::Video(video_track) => {
          let rtc_track = video_track.rtc_track();
          rtc_track.on_frame(Box::new(move |frame, buffer| {
              // Just received a video frame!
              // The buffer is YuvEncoded, you can decode it to ABGR by using our yuv_helper
              // See the simple_room example for the conversion
          });
      } else {
          // Audio Track..
      }
   }
   _ => {}
}

Examples

We made a simple room demo leveraging all the current SDK features. Videos are rendered using wgpu and egui.

FAQ

Do you plan to offer a C/C++ SDK?

Yes! In fact, we also plan to release an SDK for C++ in the coming months. It, like our other platform-specific SDKs, will use the Rust SDK. 🙂

Did you consider C/C++ as your common core?

Yes. We chose Rust over C++ for a few reasons:

• Rust's ownership model and thread-safety leads to fewer crashes/issues.
• Rust's build system requires less configuration and is easier to work with.
• While we love C/C++, it's a bit nicer to write code in Rust.
• Rust has a rich ecosystem of tools (e.g. websockets, async executor).
• Having the WebAssembly target will be useful down the road, C++ has Emscripten but it's a bit harder to set up and doesn't yet have WebRTC support.

Did you look at Arcas for libwebrtc bindings?

Yes. Our build system is inspired by LBL's work! Given that some of our logic (e.g. hardware decoder code) is in C++ and that we may need to bridge more/different things than Arcas, we decided it was better to have our own bindings for full control.

Did you consider using webrtc.rs instead of libwebrtc?

Yes! As webrtc.rs matures, we'll eventually migrate to a pure Rust stack. For now, we chose libwebrtc for a few reasons:

• Chrome's adoption and usage means libwebrtc is thoroughly battle-tested.
• webrtc.rs is ported from Pion (which our SFU is built on) and a better fit for server-side use.
• libwebrtc currently supports more features like encoding/decoding and includes platform-specific code for dealing with media devices.

LiveKit Ecosystem
Client SDKs	Components · JavaScript · Rust · iOS/macOS · Android · Flutter · Unity (web) · React Native (beta)
Server SDKs	Node.js · Golang · Ruby · Java/Kotlin · PHP (community) · Python (community)
Services	Livekit server · Egress · Ingress
Resources	Docs · Example apps · Cloud · Self-hosting · CLI