Coder Social home page Coder Social logo

marfusios / crypto-websocket-extensions Goto Github PK

View Code? Open in Web Editor NEW
41.0 8.0 9.0 7.63 MB

šŸ§° Unified and optimized data structures across cryptocurrency exchanges

License: Apache License 2.0

C# 100.00%
cryptocurrency bitcoin exchange websocket websockets extensions client reactive orderbook orderbook-tick-data

crypto-websocket-extensions's Introduction

Logo

Cryptocurrency websocket extensions Build Status NuGet version

This is a library that provides extensions to cryptocurrency websocket exchange clients.

It helps to unify data models and usage of more clients together.

Releases and breaking changes

License:

Apache License 2.0

Features

Supported exchanges

Logo Name Websocket client
bitfinex Bitfinex bitfinex-client-websocket
bitmex BitMEX bitmex-client-websocket
binance Binance binance-client-websocket
coinbase Coinbase coinbase-client-websocket
bitstamp Bitstamp bitstamp-client-websocket

Extensions

Order book

  • efficient data structure, based on howtohft blog post
  • CryptoOrderBook class - unified order book across all exchanges
  • support for L2 (grouped by price), L3 (every single order) market data
  • support for snapshots and deltas/diffs
  • provides streams:
    • OrderBookUpdatedStream - streams on an every order book update
    • BidAskUpdatedStream - streams when bid or ask price changed (top level of the order book)
    • TopLevelUpdatedStream - streams when bid or ask price/amount changed (top level of the order book)
  • provides properties and methods:
    • BidLevels and AskLevels - ordered array of current state of the order book
    • BidLevelsPerPrice and AskLevelsPerPrice - dictionary of all L3 orders split by price
    • FindLevelByPrice and FindLevelById - returns specific order book level

Usage:

var url = BitmexValues.ApiWebsocketUrl;
var communicator = new BitmexWebsocketCommunicator(url);
var client = new BitmexWebsocketClient(communicator);

var pair = "XBTUSD";

var source = new BitmexOrderBookSource(client);
var orderBook = new CryptoOrderBook(pair, source);

// orderBook.BidAskUpdatedStream.Subscribe(xxx)
orderBook.OrderBookUpdatedStream.Subscribe(quotes =>
{
    var currentBid = orderBook.BidPrice;
    var currentAsk = orderBook.AskPrice;

    var bids = orderBook.BidLevels;
    // xxx
});
        
await communicator.Start();

Trades

  • ITradeSource - unified trade info stream across all exchanges

Orders (authenticated)

  • CryptoOrders class - unified orders status across all exchanges with features:
    • orders view and searching - only executed, search by id, client id, etc.
    • our vs all orders - using client id prefix to distinguish between orders

Position (authenticated)

  • IPositionSource - unified position info stream across all exchanges

Wallet (authenticated)

  • IWalletSource - unified wallet status stream across all exchanges

More usage examples:

  • console sample (link)
  • unit tests (link)
  • integration tests (link)

Pull Requests are welcome!

Powerfull Rx.NET

Don't forget that you can do pretty nice things with reactive extensions and observables. For example, if you want to check latest bid/ask prices from all exchanges all together, you can do something like this:

Observable.CombineLatest(new[]
            {
                bitmexOrderBook.BidAskUpdatedStream,
                bitfinexOrderBook.BidAskUpdatedStream,
                binanceOrderBook.BidAskUpdatedStream,
            })
            .Subscribe(HandleQuoteChanged);

// Method HandleQuoteChanged(IList<CryptoQuotes> quotes)
// will be called on every exchange's price change

Multi-threading

Observables from Reactive Extensions are single threaded by default. It means that your code inside subscriptions is called synchronously and as soon as the message comes from websocket API. It brings a great advantage of not to worry about synchronization, but if your code takes a longer time to execute it will block the receiving method, buffer the messages and may end up losing messages. For that reason consider to handle messages on the other thread and unblock receiving thread as soon as possible. I've prepared a few examples for you:

Default behavior

Every subscription code is called on a main websocket thread. Every subscription is synchronized together. No parallel execution. It will block the receiving thread.

client
    .Streams
    .TradesStream
    .Subscribe(trade => { code1 });

client
    .Streams
    .BookStream
    .Subscribe(book => { code2 });

// 'code1' and 'code2' are called in a correct order, according to websocket flow
// ----- code1 ----- code1 ----- ----- code1
// ----- ----- code2 ----- code2 code2 -----

Parallel subscriptions

Every single subscription code is called on a separate thread. Every single subscription is synchronized, but different subscriptions are called in parallel.

client
    .Streams
    .TradesStream
    .ObserveOn(TaskPoolScheduler.Default)
    .Subscribe(trade => { code1 });

client
    .Streams
    .BookStream
    .ObserveOn(TaskPoolScheduler.Default)
    .Subscribe(book => { code2 });

// 'code1' and 'code2' are called in parallel, do not follow websocket flow
// ----- code1 ----- code1 ----- code1 -----
// ----- code2 code2 ----- code2 code2 code2

Parallel subscriptions with synchronization

In case you want to run your subscription code on the separate thread but still want to follow websocket flow through every subscription, use synchronization with gates:

private static readonly object GATE1 = new object();
client
    .Streams
    .TradesStream
    .ObserveOn(TaskPoolScheduler.Default)
    .Synchronize(GATE1)
    .Subscribe(trade => { code1 });

client
    .Streams
    .BookStream
    .ObserveOn(TaskPoolScheduler.Default)
    .Synchronize(GATE1)
    .Subscribe(book => { code2 });

// 'code1' and 'code2' are called concurrently and follow websocket flow
// ----- code1 ----- code1 ----- ----- code1
// ----- ----- code2 ----- code2 code2 ----

Async/Await integration

Using async/await in your subscribe methods is a bit tricky. Subscribe from Rx.NET doesn't await tasks, so it won't block stream execution and cause sometimes undesired concurrency. For example:

client
    .Streams
    .TradesStream
    .Subscribe(async trade => {
        // do smth 1
        await Task.Delay(5000); // waits 5 sec, could be HTTP call or something else
        // do smth 2
    });

That await Task.Delay won't block stream and subscribe method will be called multiple times concurrently. If you want to buffer messages and process them one-by-one, then use this:

client
    .Streams
    .TradesStream
    .Select(trade => Observable.FromAsync(async () => {
        // do smth 1
        await Task.Delay(5000); // waits 5 sec, could be HTTP call or something else
        // do smth 2
    }))
    .Concat() // executes sequentially
    .Subscribe();

If you want to process them concurrently (avoid synchronization), then use this

client
    .Streams
    .TradesStream
    .Select(trade => Observable.FromAsync(async () => {
        // do smth 1
        await Task.Delay(5000); // waits 5 sec, could be HTTP call or something else
        // do smth 2
    }))
    .Merge() // executes concurrently
    // .Merge(4) you can limit concurrency with a parameter
    // .Merge(1) is same as .Concat()
    // .Merge(0) is invalid (throws exception)
    .Subscribe();

More info on Github issue.

Don't worry about websocket connection, those sequential execution via .Concat() or .Merge(1) has no effect on receiving messages. It won't affect receiving thread, only buffers messages inside TradesStream.

But beware of producer-consumer problem when the consumer will be too slow. Here is a StackOverflow issue with an example how to ignore/discard buffered messages and always process only the last one.

Available for help

I do consulting, please don't hesitate to contact me if you have a custom solution you would like me to implement (web, [email protected])

crypto-websocket-extensions's People

Contributors

marfusios avatar shaynevanasperen avatar zsojma avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

crypto-websocket-extensions's Issues

Authenticated wallet / order snapshots via rest

Hi,

Do you have any idea of where to implement authenticated rest calls to get wallet and orders snapshot.

Coinbase only sends orders and wallets on change via Websocket.

We need to add wallets and orders to CryptoWalletCollection and CryptoOrderCollection.

I originally had calls internally, but it may be best to get this data in the individual exchange libraries?

Any thoughts?

FTX Library?

Hi,

Are you planning on adding an FTX Websocket client?

I can try to help if you have a template or could start a new client library?

Regards

OrderBookExample: getting all levels that caused the update

Hello,
I'm trying to capture all levels that caused an OrderBookChange using the sample provided.
I'm not very familiar with c#, so I might be missing something.
I am not able to find any level in the IOrderBookChangeInfo I'm getting: Quotes are updated but OrderBookLevel[] Levels { get; } seems empty.
Would you have any suggestion to get it to work?I'm just using the OrderBookExample provided in the repo.

Many thanks for your kind help and congratulations for the awesome package.

Snapshot reload timer

Hi,

I can see potential issue with snapshot timer.
We could get stale data to trade on while getting snapshots.

Make this optional or add a bool stream to notify not to trade while getting snapshots.

Just an idea..

Collaboration discussion

Hello Mariusz! Thank you for sharing this lovely library. I've been using it for several months now and it has served me quite well. I would like to collaborate with you on improving it, but I'm not sure this is the correct medium for us to be having a conversation about that.

I have several questions I'd like to ask you about it, and I have several ideas for improvement, but I fear that if I just go ahead and do the work first and then submit a PR, you will have valid objections to the changes I propose. So before I waste my time with that, I thought I would reach out to you and see if there's a chance for us to discuss my ideas.

Please let me know if you are open to have a discussion.

Kind regards,
Shayne

Concurrent Dictionary - Debug Vs Release latency

Hi,

Not really an issue, just an observation..
Iā€™m sure you are aware but, the latency between debug and release version of build have a HUGE impact on speed / spin up of concurrent dictionary.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    šŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. šŸ“ŠšŸ“ˆšŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ā¤ļø Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.