bitfinexcom / bfx-hf-data-server Goto Github PK

Issue type

• bug
• missing functionality
• performance
• feature request

Brief description

Why did you removed the websocket server documentation ?

Steps to reproduce

Additional Notes:

Hi. Syncing data does not work. Syncing is detecting time gaps only with mts criterion. So if you backtest for example EOSUSD it will fetch the data correctly, but try to backtest XRPUSD after that for the same timeframe. It will result in 0 candles.

Also if the filtration based on symbol will work the gap detection is somehow broken, it returns very little candles... for whole year just 1500 candles

Currently all trades are fetched at once from the DB for a backtest, whereas in the past they were fetched in chunks. This should be refactored to replicate the legacy behavior and speed up backtests.

lib/cmds/exec_bt.js

  const trades = await Trade.getInRange([
    ['exchange', '=', exchange],
    ['symbol', '=', symbol]
  ], {
    key: 'mts',
    start,
    end
  }, {
    orderBy: 'mts',
    orderDirection: 'asc'
  })

PS E:\bfx\node_modules\bfx-hf-data-server\examples> npm start

[email protected] start E:\bfx\node_modules\bfx-hf-data-server
node examples/server.js

info nems Starting download of mongodb 3.6.6 to a mongodb-download directory under C:\Users\sav17\AppData\Local\Temp
bfx:hf:util:catch-uncaught unhandled rejection: MongoError: could not start mongo process: ���⥬� �� 㤠���� ���� 㪠����� ����.
(+0ms)
Promise {
{ [MongoError: could not start mongo process: ���⥬� �� 㤠���� ���� 㪠����� ����.
]
message:
'could not start mongo process: ���⥬� �� 㤠���� ���� 㪠����� ����.\r\n' } }

The bfx proxy functionality was added to allow bfx-hf-ui to communicate with the bfx API (not possible in the browser due to CORS)

This is no longer needed as it is handled by the new bfx-hf-server

The test folder currently contains a stub so npm test passes; this needs to be filled in/expanded.

Issue type

• bug

  bfx:hf:data-server` error processing message: Error: ER_NO_SUCH_TABLE: Table 'mydb.candles' doesn't exist

Brief description

I am trying to use this module bfx-hf-strategy-py with the dataserver running on another port. I want to use the sql backend for it. I properly set up the connection (it succeeds) but at the first invocation of the WS API it complains that the table doesn't exist.

How do I create such table? Am I missing something?

Steps to reproduce

run the example in bfx-hf-strategy-py using the dataserver as backend

Issue type

• [x ] bug
• missing functionality
• performance
• feature request

Brief description

Anytime we send event exec.str to websocket, bfx-hf-data-server crawl candle again, nothing store in datatabase

Steps to reproduce

Additional Notes:

node examples/server-lowdb.js

bfx:hf:models:adapter-lowdb loading from /home/quenouille/bfx-hf-data-server/db/dev.json +0ms
bfx:hf:data-server websocket API open on port 8899 +0ms
bfx:hf:data-server ws client connected +3s
bfx:hf:data-server error processing message: TypeError: Cannot destructure property exchange of 'undefined' or 'null'.
at module.exports (/home/quenouille/bfx-hf-data-server/lib/bt/sync_data.js:9:7)
at module.exports (/home/quenouille/bfx-hf-data-server/lib/cmds/exec_bt.js:27:11)
at DataServer.onWSMessage (/home/quenouille/bfx-hf-data-server/lib/server.js:345:5)
at WebSocket.emit (events.js:198:13)
at Receiver.receiverOnMessage (/home/quenouille/bfx-hf-data-server/node_modules/ws/lib/websocket.js:825:20)
at Receiver.emit (events.js:198:13)
at Receiver.dataMessage (/home/quenouille/bfx-hf-data-server/node_modules/ws/lib/receiver.js:437:14)
at Receiver.getData (/home/quenouille/bfx-hf-data-server/node_modules/ws/lib/receiver.js:367:17)
at Receiver.startLoop (/home/quenouille/bfx-hf-data-server/node_modules/ws/lib/receiver.js:143:22)
at Receiver._write (/home/quenouille/bfx-hf-data-server/node_modules/ws/lib/receiver.js:78:10)
at doWrite (_stream_writable.js:415:12)
at writeOrBuffer (_stream_writable.js:399:5)
at Receiver.Writable.write (_stream_writable.js:299:11)
at Socket.socketOnData (/home/quenouille/bfx-hf-data-server/node_modules/ws/lib/websocket.js:900:35)
at Socket.emit (events.js:198:13)
at addChunk (_stream_readable.js:288:12)
at readableAddChunk (_stream_readable.js:269:11)
at Socket.Readable.push (_stream_readable.js:224:10)
at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17) +5ms
bfx:hf:data-server:wss:send-error internal error (103) +0ms

Getting this error when running the data_backtest server part of ema_cross.py from bfx-hf-strategy-py and also the online_bt example of bfx-hf-backtest.

Ubuntu 20.04 LTS
npm 6.14.4
Python 3.8.5

Currently we send 0 as the backtest trade count on the bt.start message, as the true count is only known later once the trades are fetched from the DB.

Fix this by fetching the trades first and passing the true trade count along with the rest of the bt.start data.

lib/cmds/exec_bt.js

send(ws, ['bt.start', '', '', start, end,, 0, candleData.length])