stav / kingbot Goto Github PK

View Code? Open in Web Editor NEW

2.0 2.0 0.0 792 KB

Deno CLI trading robot

Home Page: https://stav.github.io/kingbot/

TypeScript 82.92% Python 4.52% Rust 12.55%

trading-bot

kingbot's Introduction

K1NGBOT

This is a Deno command line application that

listens for trade signals from Telegram,
opens positions on an exchange,
listens for price events and
moves the stop-loss for the entire position.

Currently the only exchange supported is XTB's XStation 5 appliance via websocket connections.

1.0

The kingbot Proof-of-concept started early October 2021 and within a month it was moving the Break-even on the exchange by:

grouping multi-order positions into "families",
listening for Take-profit events,
moving the Stop-loss for all orders in the family to slightly better than the entry.

We began trading with real money in December, the worst month.

2.0

Kingbot 2.0 is a rewrite of the code switching the execution engine from Node.js to 🦕Deno. It started in December 2021. The plan is to have it in production come the spring with the following features:

monitor Telegram for signals,
create orders on the exchange,
move the Break-even.

With these features the bot can theoretically run fully-automatically.

Components

Bot written in TypeScript
Telegram client written in Python
Log file tools written in Rust

Installation

The Kingbot uses these components:

Git - version control (Not needed in future)
Deno - execution engine runtime
Velociraptor - script runner

Install Deno

Deno works on macOS, Linux, and Windows. Deno is a single binary executable. It has no external dependencies.

On Linux use cURL to download the installation script and run it in a shell:

curl -fsSL https://deno.land/x/install/install.sh | sh

For other platforms see the installation page.

Script Runner

Velociraptor makes it easy to run scripts.

Install Velociraptor:

deno install -qAn vr https://deno.land/x/[email protected]/cli.ts

Add Bash completion:

source <(vr completions bash)

Install the Kingbot

Deno doesn't use any package management files in the project. Modules are cached when they are first encountered in the code; therefore, the bot doesn't need to be installed. Just clone the repository:

git clone [email protected]:stav/kingbot.git
cd kingbot

Configuration

Edit the file .config/exchange.example.yaml with your real information and save it as .config/exchange.yaml.

Edit the file .config/telegram.example.yaml with your real information and save it as .config/telegram.yaml.

Usage

The Telegram client and server are currently decoupled. The server is able to be started from the main deno application but the client must be started manually.

Start the Application

Open a new command-line terminal.

Make sure you are in the kingbot directory.

Start the deno runtime with Velociraptor.

vr start

You will spawn an interactive Kingbot interpreter:

0[-]>

The prompt shows:

The exchange account index number currently active:

Zero (0) index is the Telegram connection.

The prompt also shows the login status of current connection surrounded by square brackets.

Dash (Telegram not connected):

[-]

Finally, the greater-than input pointer character

Start the Telegram Server

The Telegram server listens to HTTP traffic on localhost port 8000.

Enter the connect command to start the server:

0[-]> connect
input "connect" (function) [Function: bound connect]
Listening to localhost:8000 for { id:123456, name:"Demo", type:"demo" }

0[C]>

Send EOF (Ctrl-D End-of-file) to exit.

Start the Telegram Client

Open a second command-line terminal.

Make sure you are in the kingbot directory.

Start the Python script to listen to configured Telegram chats.

vr pyx

Connection to 149.154.175.52:443/TcpFull complete!

Please enter your phone (or bot token): +12345678901

Disconnection from 149.154.167.51:443/TcpFull complete!
Connection to 149.154.175.52:443/TcpFull complete!
Please enter the code you received: 12345
Signed in successfully

Send Interupt signal (Ctrl-C) to exit.

Tools

See the tools documentation to learn how to analyze the log files.

Development

Documentation

XTB https://xstation5.xtb.com/
xAPI http://developers.xstore.pro/documentation/
Deno https://deno.land/
Rhum https://drash.land/rhum/
Velociraptor https://velociraptor.run/docs/

Coverage

deno test -A --import-map=denoPaths.json --coverage=coverage

deno coverage coverage
deno coverage coverage |grep '100.000%'
deno coverage coverage |grep 'cover file'

Dependency Debugging

# cache dependencies locally
DENO_DIR=$PWD/deno deno test tests/profits.test.ts -A --import-map=denoPaths.json

# query the location of the dependency file in question
DENO_DIR=$PWD/deno deno info --unstable https://deno.land/x/[email protected]/src/mock_builder.ts

kingbot's People

Contributors

Stargazers

Watchers

kingbot's Issues

Break-even not working

2022-03-28 22:05:00 UTC INFO [default] check 8 "trades in total"
2022-03-28 22:05:00 UTC INFO [default] check 6 "family of" "US30"
2022-03-28 22:05:00 UTC INFO [default] Updating stop loss for 6 "orders"
2022-03-28 22:05:00 UTC INFO [default] STOP LOSS: 34870 "=" 34880 "+" -10.463999999999999
2022-03-28 22:05:00 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:00 UTC INFO [default] check 8 "trades in total"
2022-03-28 22:05:00 UTC INFO [default] check 6 "family of" "US30"
2022-03-28 22:05:00 UTC INFO [default] Updating stop loss for 6 "orders"
2022-03-28 22:05:00 UTC INFO [default] STOP LOSS: 34870 "=" 34880 "+" -10.463999999999999
2022-03-28 22:05:00 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:00 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.6744786141599206",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}
2022-03-28 22:05:00 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:01 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.304496235653674",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}
2022-03-28 22:05:01 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:01 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.8421582238771113",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}
2022-03-28 22:05:01 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:01 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.9899584534750738",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}
2022-03-28 22:05:01 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:01 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.2522870797833092",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}
2022-03-28 22:05:01 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:01 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.9603172876796604",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}
2022-03-28 22:05:01 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:02 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.7866592985235608",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}
2022-03-28 22:05:02 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:02 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.01855817086105871",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}
2022-03-28 22:05:02 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:02 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.4382449079167623",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}
2022-03-28 22:05:02 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:02 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.5320570453844948",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}
2022-03-28 22:05:02 UTC ERROR [default] setFamilyStoploss: transaction
2022-03-28 22:05:02 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.11285434890359203",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}
2022-03-28 22:05:03 UTC DEBUG [default] setFamilyStoploss: response {
  status: false,
  customTag: "0.9905702365916691",
  errorCode: "BE4004",
  errorDescr: "Invalid price"
}

Telethon process management

The bot should be able to manage the Telethon client.

Add signal group identifier to log line

No Break-even Today

Apparently we didn't get the T/P trade event notification from XTB.

I think the reason might have been that although the bot was logged in to XTB is was not listening for messages.

So today we learned that the bot needs to be both:

Logged in to XTB server
Listening for XTB messages

Daily Orders

Posting of some output of the new Log Analysis Tool representing what orders where created for certain days.

Grammy

If we are using Junction Bot then we can have them forward all signals to our King signals group and then we can get rid of Telethon and therefore get rid of the Python client and just do everything with TypeScript, for example, Grammy.

Rust

Rewrite the bot in Rust.

Telegram Integration

The bot should be able to:

login as a user (or a bot)
listen to any chat the user is a member of
parse messages for specific patterns
create orders for selected exchanges

There are few ways to connect to Telegram:

GrammY library (native deno support, bots only)
GramJS library (Webpack deno hack, user login)
Telethon library (Python, microservice needed)
JunctionBot service (not free, requires bot)

GramJS is not a viable option as the hack is not reliable.

There are a few service configurations:

JunctionBot + GrammY (easiest)
Telethon + Telegram + GrammY (safest)
Telethon + socket (fastest)

Order 361071563

A signal comes in from DowJonesChannel on Telegram:

the signal gets parsed correctly, (<1 second)
the bot opens an xtb connection (1 second)
the bot logs in to the xtb server (1 second)
an order is created on xtb but it is rejected with the message: "Invalid prices(limit)"

Here are the log files in use by the kingbot:

kingbot.log
kingmsg.log
telegram-parser.log
telegram-server.log
telethonx.log

Here are all the relevant log lines in chronological order from the the above log files:

2022-03-10 02:19:42,844 ¦ DEBUG ¦ telethonx.log ¦ Got event message Message(
  peer_id=PeerChannel( channel_id=1699473616 ),
  date=datetime.datetime(2022, 3, 10, 7, 19, 42, tzinfo=datetime.timezone.utc),
  message='US30 DOW JONES 🇺🇸\nGOLD BUY 1978.50\nSL 1968.50\nTP 1981.50\nTP  1997.50\nTP 2020.50\n\n#USE #PROPER #RISK',
)

2022-03-10 02:19:42,845 ¦ DEBUG ¦ telethonx.log ¦ Sending request to http://localhost:8000

2022-03-10 02:19:42 ¦ telegram-parser.log ¦ INFO Parsing {
  cid: 1699473616,
  fid: "PeerUser(user_id=374139448)",
  date: "2022-03-10 07:19:42+00:00",
  eindex: 1,
  msg: "US30 DOW JONES \nGOLD BUY 1978.50\nSL 1968.50\nTP 1981.50\nTP  1997.50\nTP 2020.50\n\n#USE #PROPER #RISK"
}

2022-03-10 02:19:42 ¦ telegram-parser.log ¦ INFO Parser [Function: DowjonesParser] "Parsed" {
  volume: 0.01,
  symbol: "GOLD",
  type: "BUY",
  entry: 1978.5,
  sl: 1968.5,
  tps: [ 1981.5, 1997.5, 2020.5 ]
} "Signal" {
  volume: 0.01,
  symbol: "GOLD",
  type: "BUY",
  entry: 1978.5,
  sl: 1968.5,
  tps: [ 1981.5, 1997.5, 2020.5 ]
}

2022-03-10 02:19:42 ¦ telegram-server.log ¦ INFO ServerTrade 1 {
  volume: 0.01,
  symbol: "GOLD",
  type: "BUY",
  entry: 1978.5,
  sl: 1968.5,
  tps: [ 1981.5, 1997.5, 2020.5 ]
}

2022-03-10 02:19:43 ¦ kingmsg.log ¦ INFO Socket opened "wss://ws.xtb.com/demo" "XapiSocket  wss://ws.xtb.com/demo  13110323|Tom Drake (Stav#4)  OPEN|"

2022-03-10 02:19:43 ¦ kingbot.log ¦ INFO Sending '{"customTag":"0.4597332562883496","command":"login","arguments":{"userId":13110323,"password":"...","appName":"KingBot"}}'

2022-03-10 02:19:43 ¦ kingmsg.log ¦ INFO {"status":true,"streamSessionId":"5cf3fcfffe78123a...","customTag":"0.4597332562883496"}

2022-03-10 02:19:43 ¦ kingbot.log ¦ INFO Sending '{"customTag":"0.12656677586013299","command":"tradeTransaction","arguments":{"tradeTransInfo":{"cmd":4,"customComment":"Kingbot Telegram Signal","expiration":1678432783666,"offset":0,"order":0,"price":1978.5,"sl":1968.5,"symbol":"GOLD","tp":1981.5,"type":0,"volume":0.01}}}'

2022-03-10 02:19:43 ¦ kingmsg.log ¦ INFO {"status":true,"returnData":{"order":361071563},"customTag":"0.12656677586013299"}

2022-03-10 02:19:44 ¦ kingbot.log ¦ INFO Trade { order: 361071563 }

2022-03-10 02:19:44 ¦ kingbot.log ¦ INFO Sending '{"customTag":"0.4211917212317686","command":"tradeTransactionStatus","arguments":{"order":361071563}}'

2022-03-10 02:19:44 ¦ kingmsg.log ¦ INFO {"status":true,"returnData":{"order":361071563,"requestStatus":4,"message":"Invalid prices(limit)","customComment":"Kingbot Telegram Signal","ask":0.0,"bid":0.0},"customTag":"0.4211917212317686"}

2022-03-10 02:19:44 ¦ telegram-server.log ¦ INFO server-trade-result {
  cmd: 4,
  customComment: "Kingbot Telegram Signal",
  expiration: 1678432783666,
  offset: 0,
  order: 0,
  price: 1978.5,
  sl: 1968.5,
  symbol: "GOLD",
  tp: 1981.5,
  type: 0,
  volume: 0.01
} {
  order: 361071563,
  requestStatus: 4,
  message: "Invalid prices(limit)",
  customComment: "Kingbot Telegram Signal",
  ask: 0,
  bid: 0
}

2022-03-10 02:19:45,488 ¦ DEBUG ¦ telethonx.log ¦ Got response: b'[\n  {\n    order: 361071563,\n    requestStatus: 4,\n    message: "Invalid prices(limit)",\n    customComment: "Kingbot Telegram Signal",\n    ask: 0,\n    bid: 0\n  },\n  {\n    order: 361071566,\n    requestStatus: 4,\n    message: "Invalid prices(limit)",\n    customComment: "Kingbot Telegram Signal",\n    ask: 0,\n    bid: 0\n  },\n  {\n    order: 361071569,\n    requestStatus: 4,\n    message: "Invalid prices(limit)",\n    customComment: "Kingbot Telegram Signal",\n    ask: 0,\n    bid: 0\n  }\n]'

Parse close signals

Original signal:

US30 DOW JONES 🇺🇸
🚀XAUUSD SELL 2070/2072
❌SL 2084
💰 TP 2067
✅ TP 2050
🟦 TP 1720

Close signals:

[In reply to Channels] (link to original post)
US30 DOW JONES 🇺🇸
GOLD TRADE CLOSE 1140 PIPS

[In reply to Channels] (link to original post)
US30 DOW JONES 🇺🇸
THIS TRADE CLOSE WAIT FOR NEW TREND

Log: add bid, ask & spot

When we make a trade we should print some prices to the log entry to give some price context.

bid price
ask price
spot price

Additional Parsing Rules

We should teach the bot is how to read what I will call TIP-MODIFIERS, for example when the group says to change the stop-loss:

[Forwarded from NAS100 PRO SIGNALS💰💰]
Sl Change 11300

Another example would be if they are confident the might say... "Use big lot"

[Forwarded from US30 DOW JONES 🇺🇸]
GOLD/XAUUSD BUY 1826/1824
SL 1817 TP 1830 TP 1835 TP 1854
TP 1912 use big lot

Also it is possible that a group can edit their tips: https://t.me/Nas100freepip/19724

https://t.me/c/1570788671/1222

Originally posted by @stav in #28 (comment)

Streamline Start-up

When the app starts it should not require any Deno permissions, for example "read".

error: Uncaught (in promise) PermissionDenied: Requires read access to ".config/local.yaml", run again with the --allow-read flag
const content = await Deno.readTextFile('.config/local.yaml')
                ^
    at async open (deno:runtime/js/40_files.js:51:17)
    at async readFile (deno:runtime/js/40_read_file.js:25:18)
    at async Object.readTextFile (deno:runtime/js/40_read_file.js:43:24)
    at async file:///home/stav/Work/KingBot/kingbot/src/lib/config.ts:5:17

Reopen closed connections

When a connection gets closed by the server we need to reopen it.

Sync Errors

What are these errors:

2022-06-16 14:44:18 UTC ERROR [default] {"status":false,"errorCode":"K1NG","errorDescr":"Sync Connection Closed"}
2022-06-16 14:44:20 UTC ERROR [default] {"status":false,"errorCode":"K1NG","errorDescr":"Sync Connection Closed"}
2022-06-16 14:44:21 UTC ERROR [default] {"status":false,"errorCode":"K1NG","errorDescr":"Sync Timeout"}

1[ll]> list
input "list" (function) 'Function' [Function: bound list]
[
  "CNX 0 [C] TConn",
  "CNX 1 [--] XConn XapiSocket(13477414|Tomas Dhua (Stav#7)) XapiStream(13477414|Tomas Dhua (Stav#7))",
  "CNX 2 [ll] XConn XapiSocket(2088041|Tina) XapiStream(2088041|Tina)",
  "CNX 3 [ll] XConn XapiSocket(2128824|Balam) XapiStream(2128824|Balam)",
  "CNX 4 [--] XConn XapiSocket(2281589|Davor) XapiStream(2281589|Davor)"
]

1[--]>

Xapi connect after timeout looks like it's logged in

1[--]> login
input "login" (function) 'AsyncFunction' [AsyncFunction: bound login]
undefined

1[l-]>
<Timeout>
input "" (undefined) 'undefined' undefined

1[--]> connect
input "connect" (function) 'Function' [Function: bound connect]
undefined

1[lo]>

The problem is the session is not cleared out. It should either be cleared out or reused.

Trading Strategy

Here we brain-storm various successful trading strategies that we may want to integrate into the bot:

Trailing Stop-loss
Trailing Entry-price
20 pips TP1 Stop-loss

Exchange Availability

The bot should be able to check the status for each exchange to determine its health and availability.

Also, scheduled maintenance windows should be schedule-able and observed.

Exit

The bot needs a graceful exit which:

writes state to file (to restore next startup)
closes all open connections
returns back to enclosing environment

Mystery price field in break-even

What is the mystery price field added to the break-even logic?

Break-even working

Happy Monday, here's what happened when the tip came in today.

Volume Algorithm (0.05)

When a certain channel gets a stop-loss we assume they will not have another stop-loss for a while and for just the next trade we want to bump up the volume by 5x.

So we need to detail the particulars of the 0.05 volume change:

WHAT: making volume 0.05
WHEN: always or after certain events?
WHERE: specific signal groups or specific assets?
WHO: certain clients?

Asset replacement

Need to convert some asset symbols for specific exchanges.

Need to allow various levels of granularity:

group,
symbol,
exchange.

For example, say we want to convert the symbol S&P500 to US500.

Maybe we only want to convert it from the Money Bags group.

Maybe we only want to convert it when the exchange target is XTB.

Conversions:

S&P500 => US500 for XTB
NAS100 => US100 for XTB
XAUUSD => GOLD for XTB

Order analysis tool

Use the log files to figure out what happened to a particular order.

Example

Say we have this message in the logs:

2022-04-20 13:34:52 UTC ERROR { "customTag":"0.6548979425131982", "errorCode":"Invalid price" }

We would like to know what order this message applied to and what events took place for it.

Looking at the main log file kingbot.log for that customTag:

2022-04-20 13:34:52 UTC INFO Sending {...order: 377725515,... customTag: "0.6548979425131982" }
2022-04-20 13:34:52 UTC ERROR {...,"customTag":"0.6548979425131982",..."errorDescr":"Invalid price"}
2022-04-20 13:34:52 UTC INFO setFamilyStoploss: response {errorDescr: "Invalid price"}

We can see from the last line above our error has something to do with setFamilyStoploss.

Tool

We need a tool that can look thru this (and other corresponding) log files to determine the life cycle and event list of what happened to order 377725515 so we can figure out what caused the error.

Log file rotation

https://deno.land/[email protected]/log#rotatingfilehandler