5ymph0en1x / heptet

Pair Trading - Reinforcement Learning - with Oanda Trading API

Pair Trading - Reinforcement Learning - with Oanda Trading API

A TensorFlow implemention in Reinforcement Learning and Pairs Trading. The current status of the project covers implementation of RL in cointegration pair trading based on 1-minute forex market data. For the Reinforcement Learning here we use the N-armed bandit approach. The code is expandable so you can plug any strategies, data API or machine learning algorithms into the tool if you follow the style.

Guidance

• Medium - The corresponding article for this project.

Data Source

• HistData - Run 'python formater_piphist.py' (EXAMPLE/Symphoenix) after modifying 'symbol = USDCAD' to match the one you want to retrieve data from (covers 2019).

Examples

See the folder EXAMPLE for more details. Please initiate your own virtual environment and move the scripts you want to the root folder before running them.

Idea of Usage

Start by retrieving data for EURJPY and GBPJPY by running 'formater_piphist.py' with the correct 'symbol' setting. Train a model with 'Training_Model.py' after editing the first two lines of the 'Read prices' section (lines 23-24) to match the data you have just downloaded. Edit 'CONFIG/config_trading.yml' with your personal ids and tokens. Start trading the 2 pairs by running 'Trading_Model.py'. That's all !

Going Deeper...

• You can download multiple pairs historical data and run 'Analysis.py' (EXAMPLE/Symphoenix) to explore correlation and cointegration values and graphs.

• By running 'Training_Model.py', you get a backtest result at the end which allows tuning strategies.

Disclaimer

The article and the relevant codes and content are purely informative and none of the information provided constitutes any recommendation regarding any security, transaction or investment strategy for any specific person. The implementation described in the article could be risky and the market condition could be volatile and differ from the period covered above. All trading strategies and tools are implemented at the users’ own risk.

heptet's Issues

Order size in backtest

Thanks for your project. Great work!

What is the order size used in the backtest?

Forward test live

Hello 5ymph0en1x,

Really good job man ! The backtest sound really good (i test FOREX on 5mins data).

How is the forward test on demo account ? because the basktest sound too good for to be real true lol

Regards
Ludo.

Proxy uses HTTP and not HTTPS

requests.exceptions.ProxyError: HTTPSConnectionPool(host='stream-fxpractice.oanda.com', port=443): Max retries exceeded with url: /v3/accounts/***/pricing/stream?instruments=EUR_JPY%2CGBP_JPY (Caused by ProxyError('Your proxy appears to only use HTTP and not HTTPS, try changing your proxy URL to be HTTP. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#https-proxy-error-http-proxy', SSLError(SSLError(1, '[SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:852)'),)))

Error with formater_piphist.py

While following your suggested example and without performing any edits I get the following error when I run formater_piphist.py

`https://www.histdata.com/download-free-forex-historical-data/?/ascii/1-minute-bar-quotes/usdcad/2018
{'tk': '220375159aaf4276dc3cfb87d9c5c9ac', 'date': '2018', 'datemonth': '2018', 'platform': 'ASCII', 'timeframe': 'M1', 'fxpair': 'USDCAD'}
Wrote to ./DAT_ASCII_USDCAD_M1_2018.zip
https://www.histdata.com/download-free-forex-historical-data/?/ascii/1-minute-bar-quotes/usdcad/2019
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/histdata/api.py", line 110, in download_hist_data
assert len(token) > 0
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "formater_piphist.py", line 30, in
zipfiles = dl(year=j, month=None, pair=symbol, platform=P.GENERIC_ASCII, time_frame=TF.ONE_MINUTE)
File "/usr/local/lib/python3.6/dist-packages/histdata/api.py", line 112, in download_hist_data
raise AssertionError('There is no token. Please make sure your year/month/pair is correct.'
AssertionError: There is no token. Please make sure your year/month/pair is correct.Example is year=2016, month=7, pair=eurgbp`

I am able to download the data, but the token seems to be a required element of the next step, however it does not appear to be the toke that is issued by histdata as it is reported as "missing".

Therefore the script does not continue and no data processing occurs. This seems to be related to my other error with Training_Model.py

What seems to be the root cause?

ModuleNotFoundError: No module named 'histdata'

Hello, nice work, thanks for sharing.
I am trying to follow your instructions and run:

python formater_piphist.py

but I am getting the following error:

Traceback (most recent call last):
  File "formater_piphist.py", line 4, in <module>
    from histdata import download_hist_data as dl
ModuleNotFoundError: No module named 'histdata'

where is this module supposed to be?
is it correct to run this code from EXAMPLES/Symphoenix directory?

cant get data

Hi,
First of all congratulations for this project, brilliant idea.
I've just finished installing everything and I get these errors, I think a step by step which file should be execute will make it a lot easier for everybody. Anyway this is what I get:

Wrote to ./DAT_ASCII_USDCAD_M1_201908.zip
https://www.histdata.com/download-free-forex-historical-data/?/ascii/1-minute-bar-quotes/usdcad/2019/9
{'tk': 'd477a807c601da1827f549e96ecb6c74', 'date': '2019', 'datemonth': '201909', 'platform': 'ASCII', 'timeframe': 'M1', 'fxpair': 'USDCAD'}
Wrote to ./DAT_ASCII_USDCAD_M1_201909.zip
https://www.histdata.com/download-free-forex-historical-data/?/ascii/1-minute-bar-quotes/usdcad/2019/10
{'tk': 'c769445b0c6f2d8bdc61673b2ecdef61', 'date': '2019', 'datemonth': '201910', 'platform': 'ASCII', 'timeframe': 'M1', 'fxpair': 'USDCAD'}
Wrote to ./DAT_ASCII_USDCAD_M1_201910.zip
Traceback (most recent call last):
  File "formater_piphist.py", line 69, in <module>
    ts = ts.reindex(index=index, method='pad')
  File "/usr/lib/python3/dist-packages/pandas/util/_decorators.py", line 127, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pandas/core/frame.py", line 2935, in reindex
    return super(DataFrame, self).reindex(**kwargs)
  File "/usr/lib/python3/dist-packages/pandas/core/generic.py", line 3023, in reindex
    fill_value, copy).__finalize__(self)
  File "/usr/lib/python3/dist-packages/pandas/core/frame.py", line 2870, in _reindex_axes
    fill_value, limit, tolerance)
  File "/usr/lib/python3/dist-packages/pandas/core/frame.py", line 2878, in _reindex_index
    tolerance=tolerance)
  File "/usr/lib/python3/dist-packages/pandas/core/indexes/base.py", line 2993, in reindex
    raise ValueError("cannot reindex a non-unique index "
ValueError: cannot reindex a non-unique index with a method or limit

I do have however some files saved now csv's inside the Symphoenix folder.
Thank you

Errors while Training_model.py

I am running the code in a Ubuntu 18.04 docker container with python3 and tensorflow2 and I have several issues that I kind of overcame, but perhaps not really. The latest one is this while running Training_Model.py:

Traceback (most recent call last): File "Training_Model.py", line 12, in <module> from STRATEGY.Cointegration import EGCointegration File "/tf/Heptet/STRATEGY/Cointegration.py", line 35, in <module> config_trading = FileIO.read_yaml(config_path) File "/tf/Heptet/UTIL/FileIO.py", line 19, in read_yaml with open(file_path, 'r') as stream: OSError: [Errno 22] Invalid argument: 'CONFIG\\config_trading.yml'

I looked at the code for FileIO.py and it does not seem to be doing what it is supposed to be doing. It seems to want to handle the historical data, but for some reason it is referencing the config file for Oanda, and it is attempting to manipulate the Windows convention file path.

Do you have a version of FileIO.py for a Linux environment?

count_spe_trades is not defined

Hi @5ymph0en1x , I am getting this error:

p_value: 0.06075080504069065
Beta: [1.64576441]
Traceback (most recent call last):
  File "Trading_Model.py", line 250, in <module>
    EG_Test.process_trading(index=j, transaction_cost=0.000, **action_dict)
  File "/srv/STRATEGY/Cointegration.py", line 484, in process_trading
    self.gen_trade(order)
  File "/srv/STRATEGY/Cointegration.py", line 409, in gen_trade
    if self.count_spe_trades(instrument_1) < 0:
AttributeError: 'EGCointegration' object has no attribute 'count_spe_trades'

it seems function count_spe_trades is not defined.

Not placing orders

Hi,

I've been testing your solution for a couple of days now, and although I think it might be a great tool for trading, it seems that I can't get it right....

First of all it barely gets a Beta under 1 to trade, and when it does in most cases will be this neutral felling to it.

If somehow will get into action and actually place a trade

p_value: 0.08963013915151219
Beta: [0.35512627]
NEUTRAL - Current holding: 0
Minute update...
Ready to analyze history...
EUR_JPY
C1: 1199
TS: 2019-12-17 21:17:00  /TE: 2019-12-18 18:56:00
GBP_JPY
C2: 1199
TS: 2019-12-17 22:56:00  /TE: 2019-12-18 18:56:00
X: 1201
Y: 1201
p_value: 0.09087981597641864
Beta: [0.35396983]
NEUTRAL - Current holding: 0
Minute update...
Ready to analyze history...
EUR_JPY
C1: 1199
TS: 2019-12-17 21:21:00  /TE: 2019-12-18 18:57:00
GBP_JPY
C2: 1199
TS: 2019-12-17 23:00:00  /TE: 2019-12-18 18:59:00
X: 1198
Y: 1198
p_value: 0.09444624003182783
Beta: [0.34829104]
Traceback (most recent call last):
  File "Trading_Model.py", line 250, in <module>
    EG_Test.process_trading(index=j, transaction_cost=0.000, **action_dict)
  File "/home/ftpuser/ml/STRATEGY/Cointegration.py", line 495, in process_trading
    order = self.gen_trade_signal(start_forward, end_forward, trade_th, stop_loss, transaction_cost)
  File "/home/ftpuser/ml/STRATEGY/Cointegration.py", line 204, in gen_trade_signal
    x, y = self.get_trade_sample(start, end)
  File "/home/ftpuser/ml/STRATEGY/Cointegration.py", line 133, in get_trade_sample
    assert start < end <= len(self.x), 'Error:Invalid Indexing.'
AssertionError: Error:Invalid Indexing.

I'm not sure if I do something wrong...