Comments (46)
This is a known problem on Bittrex side. See aloysius-pgast/bittrex-signalr-client#17 and https://github.com/Bittrex/bittrex.github.io/issues/50#issuecomment-480890809
from crypto-exchanges-gateway.
Which pair did you have a problem with ?
from crypto-exchanges-gateway.
theres many that intermittently trigger the error. i understand that the issue happens but it would be great if the error could trigger a notification so that my client is aware that the trades it calculates on the pairs that are not receiving current data should be ignored.
from crypto-exchanges-gateway.
How many pairs are you monitoring at the same time ?
from crypto-exchanges-gateway.
100-200. is that a bottleneck?
from crypto-exchanges-gateway.
I think so. 10-15 pairs sounds more realistic.
from crypto-exchanges-gateway.
is the limitation that wouldnt allow me to monitor 200 pairs on bittrex side? would monitoring less pairs guarantee that i wont get that error?
If the chances of the error happening are less but still exist, that would mean that i still risk placing trades based on outdated data :/
from crypto-exchanges-gateway.
Market subscriptions generates a lot of data (because of order books). See aloysius-pgast/bittrex-signalr-client#23. That's the bottleneck. What are you aiming at exactly ? (maybe we can find an alternative)
from crypto-exchanges-gateway.
I am trying to write a neural network that generates short term trades based on high volume and fast price movement.
It makes sense that my trying to monitor too many pairs causes this error and I will try monitoring less. However, if it still happens while I monitor less pairs without the gateway notifying me when it happens, it would basically wipe out the small margin of my potential trades :/
So monitoring less pairs is a good temporary solution but I still wouldn't be able to run such a system live unattended if this error occurs for a pair and the system cannot prevent the affected pairs' trades from going out to the exchange.
How cumbersome would you think it is to emit a notification when a pair subscription is affected?
from crypto-exchanges-gateway.
It should be possible to add a new type of subscription for each exchange, to account for this kind of custom notifications
from crypto-exchanges-gateway.
Wouldn't a general notification for pair subscription interruptions be applicable to all exchanges?
It seems like there could be many reasons why pair subscriptions could have issues in other exchanges as well, and consumers would profit from getting promptly notified.
from crypto-exchanges-gateway.
Problem is that pair subscription interruptions doesn't really make sense per se and cannot really be generalized to all exchanges. It requires context which is only (easily) known to client. Custom exchanges events such as Bittrex QueryExchangeState problem is easier to define on the other end
from crypto-exchanges-gateway.
I agree that subscriptions to pair data feed interruptions don't make sense. But I don't think that a user should get subscribed to interruption notifications in order to get notified when such interruptions happen.
But I get your point that leaving it up to the user is perhaps a more general and flexible approach.
The thing is that I cannot think of a use case where a consumer that subscribes to orderBook or trade data would not want to get a notification when a connection issue, like the Bittrex QueryExchangeState error, happens. It seems to me that anyone who is gathering data either for analysis or live trading would want to know if the data that they have is real-time or not. So I would vote for making the interruption subscription a default for all other subscriptions. On the other hand, a flag could be added for those that don't want to get interruption notifications to opt out of it.
from crypto-exchanges-gateway.
What I meant was that data feed interruptions does not make sense since its very context dependant, and the context is known by the user. 5 minutes without book order data might be perfectly valid for a given pair on a given exchange, while on another exchange for the same pair more than 30s without data might indicate a pb.
from crypto-exchanges-gateway.
Oh. I think I had misunderstood the QueryExchangeState error. I didn't know that the error was the result of a timeout. I thought it was caused by the gateway getting an actual query exchange error from Bittrex, which is why I figured that forwarding such errors as notifications to the client would be useful.
So what are the conditions for the Bittrex QueryExchangeState error warning to be printed on the gateway's console?
from crypto-exchanges-gateway.
Bittrex QueryExchangeState
error can be an error indicating :
- that a pair is not supported by Bittrex anymore. This can be temporary during maintenance window (that's why it is better to always retrieve the list of supported pair using REST API before subscribing
- that something wrong occured on Bittrex side of the connection (see https://github.com/Bittrex/bittrex.github.io/issues/50#issuecomment-480890809)
It is also possible to have timeout where Bittrex just stop emitting data and no error is returned at all
from crypto-exchanges-gateway.
Ok. But then it is a product of the exchange issuing some sort of error. In this case it is Bittrex saying that the pair is temporarily not supported or that their side is having issues with the connection. For other exchanges it could be another sort of problem.
What I was proposing is that when the gateway gets errors from the exchange regarding a specific pair, as it does from Bittrex in this particular case, it emits a notification to the client with the corresponding pair and error.
That way, if and when any exchange has errors with any pair's data feed, the client is made aware right away.
from crypto-exchanges-gateway.
Yes, that's what I meant with the custom notification type, which would be dependant on the exchange. By subscribing to this custom type, a client could receive a notification similar to :
{
"n":"custom",
"d":{
"exchange":"bittrex",
"timestamp":1509986841.91
"type":"QueryExchangeState",
"data":{
"pair":"USDT-BTC"
}
}
}
from crypto-exchanges-gateway.
Ok. I see what you meant. That is fine with me but I think perhaps a more meaningful name for the notification would make it easier to follow for consumers. What about 'error'?
'custom' seems vague.
from crypto-exchanges-gateway.
I meant custom because each event is a custom which might be completely distinct for each exchange but on second thought, it would probably better to use something like streamError which would cover QueryExchangeState
& anything related to websocket endpoints being unreachable (ie: connection error)
from crypto-exchanges-gateway.
that sounds perfect!
from crypto-exchanges-gateway.
Actually, it would also be very useful to get notifications when the gateway has to disconnect and retry the connection. I often get a 1590225908654|INFO|Connection (bittrex|2) will be disconnected
in the gateway's console but not notification through the socket. This means that the order books that I have cached, may no longer be up to date when the socket reconnects and I am getting trades that experience high slippage because the rates that triggered them are no longer available.
Would it be possible to add notifications for these gateway disconnects as well so that I can clear my order books when it happens?
I think when such reconnections happen, the order books that are subscribed get re-synced. Is that correct?
from crypto-exchanges-gateway.
Yes orderbooks will be re-synced and you'll get an orderBook
event first. Bittrex connection is disconnected in following cases :
- no more subscription exists for any pair
- connection has been closed remotely (in such case connection will be reconnected if at least one subscriptions exist for a pair)
The message1590225908654|INFO|Connection (bittrex|2) will be disconnected
seems to indicate this is case 1 and is expected behaviour
from crypto-exchanges-gateway.
What does 'no more subscription exists for any pair' mean exactly.
I am getting the disconnection while I am subscribed to order books of several pairs.
And the problem I am having is that when re-connection happens and the pairs start getting re-synced, the new order books trigger trades that are calculated based on order books of pairs that are outdated because their re-syncing has not happened just yet.
So it would be very useful for my system to become aware of a disconnection so that I can clear all order book cache when the disconnection happens and let the order books get re-synced from scratch when the gateway connects again.
from crypto-exchanges-gateway.
When the socket is reconnected, you should get an orderBook event which means you can forget cached order book.
from crypto-exchanges-gateway.
Yes, and I do flush the cache when an 'orderBook' notification is received. The thing is that in my system, one pair's activity can affect another pair's neural network :/
So when a disconnection happens and the first pair's 'orderBook' notification arrives, it could trigger a trade on another pair, whose order book is outdated because the 'orderBook' for that pair has not arrived yet after the reconnection.
So basically any system that uses other pairs activity as trading indicators, would be affected by not being aware that a disconnection happened.
from crypto-exchanges-gateway.
Yes orderbooks will be re-synced and you'll get an
orderBook
event first. Bittrex connection is disconnected in following cases :* no more subscription exists for any pair * connection has been closed remotely (in such case connection will be reconnected if at least one subscriptions exist for a pair) The message `1590225908654|INFO|Connection (bittrex|2) will be disconnected` seems to indicate this is case 1 and is expected behaviour
I am getting the disconnection even though I am subscribed to some pairs and when I try to subscribe again, I get an WebSocket is already in CLOSING or CLOSED state.
error in the browser console.
Why does the websocket connection with the gateway close?
from crypto-exchanges-gateway.
You need to re-create a Websocket after ws is disconnected in the browser
from crypto-exchanges-gateway.
Ok. But why does the socket close all of a sudden while there are active pair subscriptions?
from crypto-exchanges-gateway.
Could you post gateway log (running with DEBUG) and content of onclose event in browser ?
from crypto-exchanges-gateway.
Sure! How do I run the gateway with DEBUG?
from crypto-exchanges-gateway.
Try the following (both should be done) :
- set
"loglevel":"debug"
inconfig.json
- run gateway using
DEBUG=CEG:Session node gateway.js
from crypto-exchanges-gateway.
I there way to do it using the docker container?
from crypto-exchanges-gateway.
Yes, just define both environment variables :
cfg.logLevel=debug
DEBUG=CEG:Session
from crypto-exchanges-gateway.
Here are the logs. As you can see, the gateway is working fine and receiving order book updates normally and then all of a sudden something happens and it begins to disconnect :/
2020-05-25T06:30:02.678Z CEG:Session Received 'orderBookUpdate' event from exchange 'bittrex' for pair 'BTC-OCEAN' : {"cseq":153026,"buySize":0,"sellSize":3}
2020-05-25T06:30:02.681Z CEG:Session Unregistering socket '1' (172.17.0.1) from session 'rpc.bf099f02-45a9-4301-8ec9-b194f62f2c3f' : 0 sockets remaining
2020-05-25T06:30:02.681Z CEG:Session Removing listeners for session 'rpc.bf099f02-45a9-4301-8ec9-b194f62f2c3f'
2020-05-25T06:30:02.681Z CEG:Session Removing 'ticker (bittrex)' listener for session 'rpc.bf099f02-45a9-4301-8ec9-b194f62f2c3f'
2020-05-25T06:30:02.681Z CEG:Session Removing 'orderBook (bittrex)' listener for session 'rpc.bf099f02-45a9-4301-8ec9-b194f62f2c3f'
2020-05-25T06:30:02.681Z CEG:Session Removing 'orderBookUpdate (bittrex)' listener for session 'rpc.bf099f02-45a9-4301-8ec9-b194f62f2c3f'
2020-05-25T06:30:02.681Z CEG:Session Removing 'trades (bittrex)' listener for session 'rpc.bf099f02-45a9-4301-8ec9-b194f62f2c3f'
2020-05-25T06:30:02.682Z CEG:Session Removing 'kline (bittrex)' listener for session 'rpc.bf099f02-45a9-4301-8ec9-b194f62f2c3f'
1590388202689|INFO|Connection (bittrex|4) will be disconnected
2020-05-25T06:30:02.700Z CEG:Session Session 'rpc.bf099f02-45a9-4301-8ec9-b194f62f2c3f' will be destroyed in 600s
Here is the close event from the onclose callback:
bubbles: false
cancelBubble: false
cancelable: false
code: 1006
composed: false
currentTarget: WebSocket
binaryType: "blob"
bufferedAmount: 0
extensions: ""
onclose: event => {…}
onerror: err => console.error('error:', err)
onmessage: event => {…}
onopen: null
protocol: ""
readyState: 3
url: "ws://127.0.0.1:8101/"
__proto__: WebSocket
defaultPrevented: false
eventPhase: 0
isTrusted: true
path: []
reason: ""
returnValue: true
srcElement: WebSocket
binaryType: "blob"
bufferedAmount: 0
extensions: ""
onclose: event => {…}
onerror: err => console.error('error:', err)
onmessage: event => {…}
onopen: null
protocol: ""
readyState: 3
url: "ws://127.0.0.1:8101/"
__proto__: WebSocket
target: WebSocket
binaryType: "blob"
bufferedAmount: 0
extensions: ""
onclose: event => {…}
onerror: err => console.error('error:', err)
onmessage: event => {…}
onopen: null
protocol: ""
readyState: 3
url: "ws://127.0.0.1:8101/"
__proto__: WebSocket
timeStamp: 236145.6249999901
type: "close"
wasClean: false
from crypto-exchanges-gateway.
Did you also set environment variable cfg.logLevel=debug
? Because I don't see any log line with |DEBUG|
from crypto-exchanges-gateway.
Here is my docker command:
docker run --rm -p 8100:8000 -p 8101:8001 --name crypto-exchanges-gateway -e cfg.exchanges.bittrex.key='xxx' -e cfg.exchanges.bittrex.secret='xxx' -e cfg.logLevel=debug -e DEBUG=CEG:Session apendergast/crypto-exchanges-gateway
from crypto-exchanges-gateway.
And here is the output without DEBUG=CEG:Session
1590397315461|WARN|Starting...
1590397315478|WARN|MarketCap API is enabled
1590397315478|WARN|FxConverter API is enabled
1590397315479|WARN|poloniex exchange (poloniex) is enabled (public API)
1590397315479|WARN|bittrex exchange (bittrex) is enabled (public API & trading API)
1590397315479|WARN|binance exchange (binance) is enabled (public API)
1590397315479|WARN|kucoin exchange (kucoin) is enabled (public API)
1590397315480|WARN|okex exchange (okex) is enabled (public API)
1590397315480|WARN|UI is enabled
1590397315480|WARN|TickerMonitor is enabled
1590397316268|INFO|Database is ok
1590397316273|INFO|0 sessions loaded
1590397316274|INFO|0 TickerMonitor entries loaded
1590397316281|INFO|Data loaded successfully
1590397316302|WARN|HTTP server is alive on *:8000
1590397316303|WARN|WS server is alive on *:8001
1590397319217|INFO|Got new connection for RPC session 'rpc.cab862db-035f-4d79-9e37-01b4d56e5bf1' : ipaddr = '172.17.0.1'
1590397319811|DEBUG|172.17.0.1 GET /exchanges/bittrex/pairs
1590397320164|DEBUG|172.17.0.1 GET /exchanges/bittrex/tickers
1590397324785|DEBUG|172.17.0.1 GET /exchanges/bittrex/balances
1590397340855|INFO|Connection (bittrex|1|bbebb563-8a43-4107-8188-0c2c92921b6f) connected
1590397439224|DEBUG|Got timeout for WS rpc.cab862db-035f-4d79-9e37-01b4d56e5bf1/1
1590397439258|INFO|Connection (bittrex|1) will be disconnected
For some reason the timeout wasn't there last time I posted it. What causes that timeout?
from crypto-exchanges-gateway.
The websocket is disconnected because it didn't answer to ws ping sent by the gateway, exactly 2 minutes after successful connection (Got timeout for WS rpc.cab862db-035f-4d79-9e37-01b4d56e5bf1/1
)
from crypto-exchanges-gateway.
ws ping/pong is usually handled directly by the WebSocket class in browser AFAIK. Which browser are you using ?
from crypto-exchanges-gateway.
I use Chrome.
from crypto-exchanges-gateway.
And how to you open the ws connection ?
from crypto-exchanges-gateway.
const ws = new WebSocket('ws://127.0.0.1:8101');
from crypto-exchanges-gateway.
Could you try to create an html file with below content, check what you get in chrome console and see if you have an indication that ws was terminated after some time ?
<html>
<head>
<script>
document.addEventListener('DOMContentLoaded', () => {
const wsUri = 'ws://127.0.0.1:8101';
const ws = new WebSocket(wsUri);
let connectedTimestamp = Date.now();
let closedTimestamp = Date.now();
ws.onopen = (e) => {
connectedTimestamp = Date.now();
console.log(`${new Date().toISOString()} - ws successfully connected`);
}
ws.onerror = (e) => {
closedTimestamp = Date.now();
const delay = (closedTimestamp - connectedTimestamp);
console.error(`${new Date().toISOString()} - Could not open ws connection to '${wsUri}' (delay = ${delay}ms)`);
}
ws.onclose = (e) => {
closedTimestamp = Date.now();
const delay = (closedTimestamp - connectedTimestamp);
console.error(`${new Date().toISOString()} - ws connection '${wsUri}' was closed (code = '${e.code}'', reason = '${e.reason}') (delay = ${delay}ms)`);
}
ws.onmessage = (e) => {
console.log(`${new Date().toISOString()} - Got ws message from '${wsUri}'`);
console.log(e.data);
}
});
</script>
</head>
<body>
</body>
</html>
from crypto-exchanges-gateway.
No disconnection :/
2020-05-25T11:24:53.127Z - ws successfully connected
2020-05-25T11:24:53.141Z - Got ws message from 'ws://127.0.0.1:8101'
{"hello":{"sid":"rpc.f580c347-2417-4884-b24e-4bb1f9482e1a","isNew":true}}
I guess its a problem with my implementation.
Ill post back if I figure out whats wrong. Thank you so very much for all the help and support, though!!
from crypto-exchanges-gateway.
It turns out that the issue is a function of how many pairs I am subscribed to and happens only on chrome. It would seem like after so many processing threads initiated by different threads, chrome gets overloaded and fails to respond to the gateway's ping :/
Either way, it doesn't happen on firefox so I will continue working there.
from crypto-exchanges-gateway.
Related Issues (20)
- npm install ERROR HOT 3
- Add cfg.exchanges.kucoin.password into environment variables HOT 3
- Tiny issue of the doc HOT 1
- Okex doesn't work HOT 13
- DeFi exchanges (PancakeSwap) and BSC donation address HOT 1
- Completed orders incorrectly sorted in UI when some orders are cancelled (Binance) HOT 2
- Updating pair in url (myOrders view) does not refresh pair chooser HOT 1
- Looking to autosell BTC/ETH HOT 6
- Bug in your websocket connection file + a few fixes HOT 4
- In Localhost installation, can only view "prices", & Order Books HOT 5
- Implement Kucoin V2 API HOT 2
- Crypto-exchange : dummy ethers HOT 1
- Klines for exchanges that only have a trades websocket? HOT 6
- MaxListenersExceededWarning HOT 8
- blocking call WS RPC subscribeToKlines HOT 7
- Question: WS kline or polling history candles? HOT 2
- Setting ports on docker command does not work HOT 1
- Install issue HOT 10
- Filter trade notifications to only emit my exchange's account trades HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from crypto-exchanges-gateway.