wesmania / faf-aio-replayserver-original Goto Github PK
View Code? Open in Web Editor NEWPrototype for new FAF replay server
License: GNU General Public License v3.0
Prototype for new FAF replay server
License: GNU General Public License v3.0
The old replayserver uses some convoluted way of sending replays - timestamped chunks wrapped up with qt protocol. We'd prefer to just have a regular data stream. If we want to replace replayserver seamlessly, we need to demangle that.
Right now the implementation of replay stream is very simple and just gets the data from a player with the most data. A malicious player can just flood us with garbage to corrupt the replay.
One way to go around it is to do things like the legacy replayserver does - pick a player, then stick with him until he's disconnected. This of course forces us to cache data (or use some asyncio high / low water buffer marks to do it for us). A malicious attacker could try flooding us with data again, but at the same time we want to avoid being help up by a currently selected player for too long. We need some sound logic here.
To be honest, this sounds like a problem that someone, somewhere, should've already solved. We need to look for some existing solution if possible.
I believe that according to legacy replayserver code, after unwrapping, the replay data stream from each player should be the same (barring freak accidents). I'm not 100% sure about it and we'll have to verify if that really behaves like this, in case we need some extra replay merging logic.
Turns out that no (or very little) data gets sent if there are no new commands for a while. This causes some extra & variable delays. If we flush at every write or periodically, that's somewhat of performance penalty probably, if we do nothing that's some extra delays. We'll have to decide what's more important.
Legacy replay server parses and saves replay data in a database (and the file too, probably). We should do that as well.
There are 2 different notions of a game ending:
When 1. happens first (e.g. replay writer connection stalls and we timeout a replay after 5 hours), then we mark the game as in vault, but it has no replay file yet.
When 2. happens first (e.g. a game is left at the scoring screen, so the server keeps the player-game relation), then a game marked as 'live' no longer has a live replay available.
One way to fix that would be to decouple the 'game ended' concept from 'game available as live replay or vault file' concept. The replay server can decide when the replay is available as a file instead of a live replay (and make sure the periods for both overlap so everyone gets one) and notify the server about these changes somehow. The server can add another field to game messages that tell us how to watch them.
There's some buffer juggling going on with replay merging, data reading and all that. memoryview
might help us avoid unnecessary copies.
The server does save replay files where we expect them, and with the expected format. However, the link for grabbing them (http://content.test.faforever.com/faf/vault/replay_vault/replay.php?id=) returns a 404. We need to investigate whether:
Try wget --no-check-certificate 'http://content.test.faforever.com/faf/vault/replay_vault/replay.php?id=8571147'
- the json inside seems all wrong. We'll have to investigate how it came to be.
Right now we take timestamps at each second to get current streamed replay data position, and we await an appropriate timestamp 5 minutes back when sending it. The legacy replayserver seems to use some timestamped data instead. Figure out if our method is good enough (i.e. prevents cheating by watching an up-to-date replay of a game in progress).
One problem might for example be one player freezing a game for 5 minutes so that a streamed game can progress to a current state. If this is unacceptable, then we'll have to figure something out (depends if the legacy replay server is succeptible as well).
I don't know much about asyncio's interaction with multithreading and whether Python's GIL defeats the purpose anyway. Once we get to the testing phase, it would be good to check if we can get some performance improvements by delegating replays to separate event loops somehow.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.