tucsky / aggr-server Goto Github PK

Hello Aggr-Server contributors! I have a specific requirement where I need to store all trade data from Binance perpetual and spot markets starting from 2020 up until the present. My goal is to compute, normalize, and calculate percentage deviations between the cumulative volume deltas (CVDs) over this time period and then visualize the results through charts.

I'm interested in leveraging Aggr-Server for this task. Could you please provide guidance on how I can utilize Aggr-Server to store the trade data, compute CVDs, normalize them, calculate percentage deviations, and generate the necessary charts? Additionally, if there are any potential roadblocks or limitations I should be aware of in this process, please let me know.

Thank you for your assistance and expertise!

Not just historicals like with coinalize script, but if server goes down for an hour and recoverMissingTrades doesn't fill in all the holes. Can coinalize script work for this as well? Thanks!

I'm able to connect my aggr client to my aggr-server fine, but am struggling to get the client to fetch historical data from my server. I currently have the server recording to files, rather than Influx, and can see the data being recorded into said files. Do I need to switch over to Influx as the storage method? I saw this line in files.js that made me think this may be the case: "console.error('[storage/file] historical data request not supported by this storage type (raw trade files)')"

A 400 Bad Request error occurred: overflowed duration. The SELECT query, in the WHERE clause, appends 'ms' to the end of both the "from" and "to" values. However, when viewing the data in influxdb using influxdb-cli, the "time" values do not include 'ms' at the end. I see values like 1671323630000000000. When I try to use the SELECT query in influx.js fetch function, I get the same "overflowed duration" error when it tries to use the value 1671323630000000000ms, but I do not get this error when I remove the 'ms'.

I must be overlooking something, but I am not understanding why the ms is in the SELECT query. More broadly, when I am running Aggr-server locally, I cannot find a working API_URL value for Aggr. It is suggested to use http://localhost:3001/historical/{from}/{to}{timeframe}/{market}, but I do not understand what I am supposed to put in for these values. I can workout timeframes and markets, but the "from" and "to" values are confusing me. I have tried a few different ones, but I don't understand why I am hard-coding the from and to values here. It seems to me that a better API_URL would be http://localhost:3001/ or http://localhost:3001/historical.

Thank you for any clarity you can provide.

Let trade and script strategies too ...
that send trade orders to the exchg(s) & show account info etc.

Hi, I'm trying to host the server feature and it's not working for me. I know nothing about js for the record. Could someone help me via Discord? Mine is Chuti#0800

Sorry for bothering and have a good day

Hi!

I'm trying to understand the complexity of backfilling aggr-server with historical data (from e. g. binance & bybit data files and/or public endpoints).

From what I understood, scripts/coinalize/* might be a good reference, and as I understood it's possible to backfill just minutely trades (which might be the only thing available for majority of exchanges outside of binance and bybit), and resample that into needed timeframes, is that correct?

Why not record futures data ，like OKEX- BTC-USD-211231

I was going to create two different requests for this on both the charts and server but decided to just ask here.
Is it possible to pull pairs baed on either a wildcard or Asset+Quote pairing?
Example: I want all pairs on Binance or Binance US that end with USD or USDT or BINANCE:.*/USDT,BINANCE_US:.*/USD
This would also play into the API_SUPPORTED_PAIRS variable in the charts.

Receive the below error when watching about 700 symbols on a single machine. I'm guessing this is way too many, but didn't expect the error to show up in the InfluxDB insert query. Any idea what could be causing this?

write points failed (1st attempt) A 400 Bad Request error occurred: {"error":"partial write: unable to parse 'trades_1s,market=MEXC:PONDUSDT cbuy=0,close=0.009378,csell=1,high=0.009378,low=0.009378,open=0.009378,vbuy=0,vsell=7.943634899999999 NaN': bad timestamp\n

CCXT Pro is now free using the original CCXT library.
Implementing this could open up Exchange support for AGGR.

Hey, after installing the server and Influxdb and running node index, it's impossible to log in!

[storage/influx] connecting to localhost:8086 [storage/influx] A 401 Unauthorized error occurred: {"code":"unauthorized","message":"Unauthorized"}... retrying in 1s

I would like to create a cvd for Binance:btcfdusd, but I can't because there is no historical data. Is it possible to have it added?

Hi, Thank you for sharing such a good project source,
I'm trying to run aggr-server to access data locally, I'm using a fresh ubuntu server 18.04 LTS
Installed nodejs and npm, no issues when running npm build but when I try to run node index I'm getting this error :

using config file config.json
/home/test/aggr-server/node_modules/decamelize/index.js:9
                .replace(/([\p{Lowercase_Letter}\d])(\p{Uppercase_Letter})/gu, `$1${separator}$2`)
                 ^

SyntaxError: Invalid regular expression: /([\p{Lowercase_Letter}\d])(\p{Uppercase_Letter})/: Invalid escape
    at module.exports (/home/test/aggr-server/node_modules/decamelize/index.js:9:4)
    at Object.keys.forEach (/home/test/aggr-server/src/config.js:244:23)
    at Array.forEach (<anonymous>)
    at Object.<anonymous> (/home/test/aggr-server/src/config.js:243:21)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Module.require (module.js:596:17)

I can't find much help about this error on the google so if you know what I'm doing wrong thanks in advance. 👍

Aggr-server cannot connect to influxdb. But i can connect to influxdb with terminal. Maybe it is because of new influxdb server version
Running with docker

st-server | [storage/influx] connecting to localhost:8086
st-server | [storage/influx] connect ECONNREFUSED 127.0.0.1:8086... retrying in 1s

server.js describe away to use aggr-server in broadcast mode, but there's no documentation about it.

1. What's the current status of this feature ?
2. Is it still in use ?
3. What's a typical user case ?
4. Is it still useful in the current state?

Shall we write some documentation about it?

Hi,

I can't seem to use the recover.js script when setting influx as the storage solution.
It seems to be related to a missing "insert" function in influx.js

Am I missing something?

Thanks a lot for the great work!

Hi,
I am currently running on influxdb2 branch and having this error :
throw new Error('No key set vapidDetails.publicKey');

Can you help me to show how I can fix this ?

Thanks~