houraiteahouse / backroll-rs

A (almost) 100% pure safe Rust implementation of GGPO-style rollback netcode.

backroll-rs

Backroll is a pure Rust implementation of GGPO rollback networking library.

Development Status

This is still in an early beta stage. At time of writing, the public facing API is stable, and has undergone limited testing. There may still be notable bugs that have not been found yet.

Differences with the C++ implementation

• (Almost) 100% pure safe Rust. No unsafe pointer manipulation.
• Type safety. backroll-rs heavily utilizes generics and associated types to avoid serialization overhead and potentially unsafe type conversions when saving and loading game state.
• Abstracted transport layer protocols - integrate and use any transport layer library you need. Comes with a raw UDP socket based implementation.
• Configurable at runtime - Many of the hard-coded constants in GGPO are exposed as configuration parameters during session initialization.
• Reduced memory usage - Backroll's use of generics potentially shrinks down the sizes of many data types.
• Vectorized input compression scheme - Backroll utilizes the same XOR + RLE encoding, but it's written to maximize CPU utilization.
• Multithreaded I/O - All network communications run within an async task pool. I/O polling is no longer manual, nor blocks your game's execution.

Repository Structure

This repo contains the following crates:

• backroll - the main Backroll interface, intended to be used as the original GGPO.
• backroll_transport - An isolated set of transport layer abstractions.
• backroll_transport_udp - A transport layer implementation using raw UDP sockets.
• backroll_transport_steam - A transport layer implementation using the Steam provided networking utilities. Enables access to the Steam Datagram Relay service.
• bevy_backroll - a integration plugin for bevy. (Complete, untested).
• lib - third-party linking dependencies (i.e. Steam) for easier local development

backroll-rs's Issues

Remove requirement of Hashing from GameState

Currently in Backroll, gamestates that get saved and loaded HAVE to be able to be hashed. This leads to a lot of issues, since important gamestate stuff like floats and vectors aren't able to be hashed. This should be replaced with something more universal.
Note: as far as I know I think this issue is with bevy_backroll but it could extend to the original backroll library as well

Proposal: 'Command Buffer' pattern to replace 'Callbacks' structure

Managing a SessionCallbacks-implementing structure that is expected to be able to directly access the current game state can be somewhat non-ergonomic. However, passing control back and forth between Backroll and client code is not necessarily, strictly speaking, necessary. Backroll needs to tell client code "load this state", "advance with these inputs", "save this state to this buffer", etc, but this does not need to be done during Backroll's own processing.

Therefore, rather than immediately passing control back to client code, Backroll can instead create a list of 'commands' to the client, which the client can execute afterwards all at once. A rough mock-up of this use of the Command pattern might look like this:

// backroll definition
enum Command {
    Save(rollback_buffer_idx: usize),
    Advance(inputs, rollback: bool),
    Load(rollback_buffer_idx: usize)
};


// client code
let command_buffer: Vec<Command> = session.advance_frame();

for command in command_buffer {
    match command {
        Command::Advance(inputs, rollback) => {
            self.state.advance(inputs, rollback);
        }
        Command::Save(rollback_buffer_idx) { 
            session.save(self.save_data(), rollback_buffer_idx);
        }
        Command::Load(rollback_buffer_idx) {
            self.load_data(session.load(rollback_buffer_idx));
        }
    }
}

This match command { statement effectively replaces the role of the SessionCallbacks struct in describing the client's performance of Save/Load/etc operations, and it could be easier to understand and implement than a separate struct.

Example for bevy_backroll

Thank you for this, looks awesome!

Could we get a minimalistic bevy app example that uses bevy_backroll? I think that would help getting into the whole networking and backroll topic.

Naming Conventions: "Backroll" prefix redundant?

The library currently has a number of typenames prefixed with "Backroll," which seems to add a lot of name bloat. The examples seem to deal with this name bloat with use backroll::*;. Rather than typing something like

use backroll::*;
// ...
    let local_handle: BackrollPlayerHandle = session_builder.add_player(BackrollPlayer::Local);

would it be more idiomatic to leave the namespace intact and use

    let local_handle: backroll::PlayerHandle = session_builderr.add_player(backroll::Player::Local);

to avoid conflicts?

I'm not super familiar with Rust naming conventions, so consider this a suggestion, but it seems like this is the convention among other Rust libraries, with names like Context, Sprite, etc going un-prefixed.

P2PSession is not polling the Player<T> reciever

P2PSession is not polling the Player reciever, which means that the queues of network events are not being forwarded to the top level. This usually ends in a prediction limit hit.

WebRTC Transport Layer

Adding this here just for posterity, since I already know that this is in the plans. This would be pretty easy to test though, since bevy and other rust game frameworks already support the web, with a few being web-only in fact. Having a game with rollback on the web besides Tough Love Arena would be really cool.

Does the bevy plugin need to be single threaded?

There's a comment saying "to avoid potentially introducing non-deterministic simulation results", but is there a specific unfixable issue? It seems to me that this should be at least configurable, since it should be possible for all the systems to be completely deterministic.

I apologize if this is documented and I just missed it, but I feel that this should be configurable or have a more descriptive comment saying why it has to be single threaded.

Steam Networking Sockets transport implementation

The original reason a I/O abstraction layer via bevy_transport was to support potentially multiple I/O options where direct control over the low level communications may not be possible, particularly with the ISteamNetworkingSockets interface in the Steamworks SDK.

Right now the most mature wrapper around the Steamworks SDK is steamworks-rs, but the ISteamNetworkingSockets interface does not have bindings (see Noxime/steamworks-rs#6), and the existing ones are blocking (see Noxime/steamworks-rs#57). It does contain a Send and Sync client that can do the original ISteamNetworking interface, but according to the official Steamworks documentation, it's considered depercated and may be removed in the future:

NOTE: This API is deprecated and may be removed in a future Steamworks SDK release. Please 
use ISteamNetworkingSockets or ISteamNetworkingMessages instead. 

is it support WebAssembly?

Bevy plugin does not create a checksum for the state anymore

As of 014fd15, bevy_backroll is now saving the provided state without any hashing/checksum. This could lead to silent desyncs as no checksum checks will fail in these cases. We should look for a generic way of handling this without reintroducing the Hash bound on Config::State.

Consider reverting to the C-style ring-buffer for SavedState

Commit cf0b896 fixed #12 but replaced the SaveState ring buffer with a frame-keyed hashmap instead. This adds the overhead of clearing out the HashMap of old states each time a frame is confirmed by a remote, and potentially allocates more on the heap each time a new state is saved. This indirection is likely not particularly cache friendly.

However, given this HashMap should normally never be more than 8-20 entries large at a given time, this may be a moot point in terms of performance.

Leaving this as an issue to revisit this when time permits.

Tracking Issue: Migrate transport level logic into backroll-transport

There are a notable amount of the application level protocol that belongs in a lower level implementation. The heartbeat, the initial synchronization, the magic number management, sequence number, and disconnect timeout are all to establish a unreliable but ordered connection, something Laminar and other reliable UDP implementations already support. This should exist at the backroll-transport layer. A good amount of this is already established in TimonPost/laminar#290, and this change be considered a step towards implementing backroll-transport's connection model in Laminar.

• Create a wrapper around Peer for ordered unreliable sends. OrderedPeer?
• Add a way to notify the main game of incoming connections.
• Alter Peer to notify either end of a network interruption.
• Move "Synchronizing, Running, Interrupted, Disconnected" state machine to the transport layer.
• Move heartbeat task into OrderedPeer

Support multiple local players per peer

Right now Backroll only supports one local player per peer, which GGPO also has but does not publicly document. This seems like a very useful feature to have for a lot of mixed-couch/networked multiplayer.

This will likely require restructuring the full sync, input queue, and protocol message structure.

Make framerate / TPS configurable

See:

backroll-rs/backroll/src/protocol/mod.rs

Line 38 in 3c6bdc0

const TARGET_TPS: u64 = 60;

which is used for estimating the remote player's frame for timesyncing purposes:

backroll-rs/backroll/src/protocol/mod.rs

Lines 740 to 753 in 3c6bdc0

	pub fn set_local_frame_number(&self, local_frame: Frame) {
	let mut stats = self.stats.write();
	// Estimate which frame the other guy is one by looking at the
	// last frame they gave us plus some delta for the one-way packet
	// trip time.
	let remote_frame = self.input_decoder.last_decoded_frame()
	+ (stats.round_trip_time.as_secs() * TARGET_TPS) as i32;
	// Our frame advantage is how many frames *behind* the other guy
	// we are. Counter-intuative, I know. It's an advantage because
	// it means they'll have to predict more often and our moves will
	// pop more frequently.
	stats.local_frame_advantage = remote_frame - local_frame;
	}

On an unrelated note, it that calculation correct? The comment seems to suggest round_trip_time.as_secs() / 2.

Epic Online Services Implementation

Since Steamworks API might get deprecated, maybe there should be Epic Online Services variant. I know HiFight uses EOS for FOOTSIES, and a lot of fighting game devs consider it to be really good as well. Additionally, it works on all platforms, including mobile and console, which is way better than Steamworks coverage.

Desync of game state on PredictionBarrierReached

Whenever the prediction barrier gets reached due to an influx of inputs, there will be a bunch of jitters, which leads to the players' positions in this sample project getting desynced. The current theory is that the inputs during that frame don't get read.