hastobegood / crypto-bot-commander Goto Github PK

Add moving average calculation to the engine.

• SMA (Simple Moving Average)
• CMA (Cumulative Moving Average)
• EMA (Exponential Moving Average)

A specific repository has been created for DCA : https://github.com/hastobegood/crypto-dca.

Remove it from this repository.

Add Sonar analysis on each PR.

As the project is open source, it should be free.

If for some reason the exchange API is unavailable while updating candlesticks, when it's available again we should retrieve all the missing datapoints.

Currently, datapoints are "lost".

Currently candlesticks are stored in DynamoDB with 1 item per minute. This results having 1440 items to request for a single day of data. This become really inefficient when requesting for example 30 days.

Find a way to improve performance and reduce number of queries to DynamoDB.

Binance has published a swagger for its API. It could be interesting to use it to generate models instead of manually creating it.

See https://github.com/binance/binance-api-swagger.

Add the backtesting feature.

Add a github action :

• Build
• Run tests
• Run linter

It should be triggered on :

• pull requests
• main branch

Add an initial budget to strategy model.

The send order step, when using Account source, should use the available quantity from the strategy budget instead of the Binance spot account.

See https://aws.amazon.com/about-aws/whats-new/2021/09/better-price-performance-aws-lambda-functions-aws-graviton2-processor/ for more details.

Setup a strategy engine that will process all active strategies every minute.

Strategy steps will be executed in sequential order depending on the step type and input data.

Each step will produce an output with a success indicator and additional data depending on the step type.

Each step must be saved in a table as historic data and resume point for the next execution.

Strategy example :

{
  "id": "999",
  "symbol": "BNB#USDT",
  "status": "Active",
  "template": {
    "1": {
      "id": "1",
      "type": "MarketEvolution",
      "nextId": "2",
      "input": {
        "interval": "1m",
        "period": 60,
        "source": "Market",
        "percentageThreshold": -0.01
      }
    },
    "2": {
      "id": "2",
      "type": "SendOrder",
      "nextId": "3",
      "input": {
        "side": "Buy",
        "source": "Account",
        "type": "Market",
        "percentage": 0.1
      }
    },
    "3": {
      "id": "3",
      "type": "MarketEvolution",
      "nextId": "4",
      "input": {
        "percentageThreshold": 0.01,
        "source": "LastOrder"
      }
    },
    "4": {
      "id": "4",
      "type": "SendOrder",
      "nextId": "1",
      "input": {
        "side": "Sell",
        "source": "LastOrder",
        "type": "Market",
        "percentage": 1
      }
    }
  }
}

Strategy step execution example : 

```json
{
  "id": "1",
  "type": "MarketEvolution",
  "strategyId": "999",
  "nextId": "2",
  "creationDate": "2021-09-07T02:21:38.401Z",
  "lastExecutionDate": "2021-09-07T03:29:36.680Z",
  "input": {
    "source": "Market",
    "interval": "1m",
    "period": 60,
    "percentageThreshold": -0.01
  },
  "output": {
    "success": true,
    "currentPrice": 497.8,
    "lastPrice": 504.4,
    "percentage": -0.0131
  }
}

To do :

• Initialize candlesticks by year & month for a specific symbol
• Update candlesticks with schedule events
• Publish active strategies IDs based on updated candlesticks instead of schedule events
• Query candlesticks from DynamoDB instead Binance API during strategy evaluation

Use this package which contains common code for crypto bots : https://www.npmjs.com/package/@hastobegood/crypto-bot-artillery

Add the moving average crossover step.

Input attributes :

• Type (SMA, CMA or EMA)
• Crossover (CurrentPrice or ShortTermPrice)
• Signal (Buy or Sell)
• Short term period
• Long term period

Output attributes :

• Current price
• Short term price
• Long term price

Details on how to detect the crossover : https://www.investopedia.com/articles/active-trading/052014/how-use-moving-average-buy-stocks.asp

Gaps are currently not handled in candlesticks.
It means that if I request 1000 candles but there are gaps, I will receive less than 1000 candles.

It's not a big deal for the API as it depends on the expected behaviour.

But for moving average calculation, it can lead to errors. We should detect gaps and fetch more candles until number of requested candles is reached.

Note: pay attention when there is no more data in database to avoid inifinite loop.

Huge flash crash has been detected by the market evolution step, but the send order sell price was too high.

So I bought BNB at 412.6$. The strategy was waiting for the price to go down by at least -1%.
A huge "flash crash" happened and the price dropped to 312.5$ (see below).

 "input": {
  "interval": "1m",
  "period": 60,
  "source": "Market",
  "percentageThreshold": -0.01
 },
 "output": {
  "success": true,
  "currentPrice": 312.56,
  "lastPrice": 412.6,
  "percentage": -0.2425
 },
 "creationDate": "2021-09-08T16:35:38.100Z",
 "lastExecutionDate": "2021-09-08T17:00:36.371Z"

So the next step kicks in and send a sell market order to Binance 16ms after.
Unfortunately the order has been executed at 418$... Which is much greater than my buy price that was 412$... (see below).

  "input": {
   "side": "Buy",
   "source": "Account",
   "type": "Market",
   "percentage": 0.1
  },
  "output": {
   "externalId": "983405",
   "id": "BNB#USDT/Buy/Market/1631120438292",
   "quantity": 1.41,
   "success": true,
   "price": 418.2758865248228,
   "status": "Filled"
  },
  "creationDate": "2021-09-08T17:00:37.387Z",
  "lastExecutionDate": "2021-09-08T17:00:37.387Z"

Looking at the OHLC (Open High Low Close) candlesticks during this flash crash :

    [ 1631120400000, "416.90000000", "418.50000000", "312.50000000", "418.10000000", ... ], // 2021-09-08T17:00:00.000Z
    [ 1631120460000, "416.31000000", "418.00000000", "311.77000000", "417.40000000", ... ], // 2021-09-08T17:01:00.000Z
    [ 1631120520000, "415.70000000", "417.60000000", "311.57000000", "415.30000000", ... ], // 2021-09-08T17:02:00.000Z
    [ 1631120580000, "416.70000000", "418.40000000", "311.50000000", "417.40000000", ... ], // 2021-09-08T17:03:00.000Z

So it seems that during 4 minutes the price has moved from around 418$ to 311$ every minute and closing price was always around 418$.

That explains why my order has been executed at 418$.

A few questions :

• Is it a "normal" behavior that can happen in the real world ?
• If yes, how to handle it ?

A few ideas :

• Stop using market order but for what ?
• Any other idea ?

Add 2 new order types : StopLossLimit & TakeProfitLimit.

A new attribute should be added to the SendOrder step to set the percentage related to the price threshold to use.

Currently only OHLC are supported. Add volume.

It could be interesting to switch to AWS SDK v3 in order to improve tree-shaking.

More info here : https://github.com/aws/aws-sdk-js-v3.

Add tags to the deployed CF stack :

• Environment
• Application

Below is the exhaustive list that Binance can return when sending an order :

Currently the status is not check when an order is sent even if it is not FILLED (aka order has been successfully processed).

See how to handle the other statuses.

TODO

Add market evolution calculation to the engine.

For example 998.5632 / 2.36 result in 423.12000000000006 instead of 423.12.

Investigate what is the best way to handle this.

Currently only market orders are supported. Add limit orders support.

Since #12 the DCA trading function fails :

   {
    "symbol": "BTCBUSD",
    "requestedQuantity": 25,
    "baseAsset": "BTC",
    "message": "Unable to extract assets, symbol 'BTCBUSD' is invalid",
    "success": false,
    "quoteAsset": "BUSD"
   }

This is due to the symbol format refactoring.

We need to store orders in the database when sending order to exchange.

Update the check order step as well to read database instead of calling exchange API.

Add a Lambda that will call exchange API for all orders with a status Waiting or PartiallyFilled.

When fees are paid with the base asset (for example when trading BNB#XXX and paying fees with BNB), executed quantity returned by Binance API does not include commision quantity.

We need to substract manually the commission quantity from the executed quantity in this case.

Add a Or step between two steps where the first that is a success is the winner.

Add an option to the stack in order to enable/disable X-Ray tracing during deployment.

Targets :

• AWS SDK
• External HTTP requests

Retrieve symbol candlesticks from Binance endpoint GET api/v3/klines.

The current EMA calculation output is different that the one from Binance.

It seems it's because not enough historical data is being used. See https://stackoverflow.com/questions/65727498/different-ema-exponential-moving-average-than-on-binance#comment116237282_65727498

Proposed solution : take 3 times more candles than the long term period requested (ex: for EMA 10d, get 30d and calculate EMA 10d based on 30d historical data)