mhallsmoore / qsforex Goto Github PK
View Code? Open in Web Editor NEWQuantStart Forex Backtesting and Live Trading
Home Page: http://www.quantstart.com
QuantStart Forex Backtesting and Live Trading
Home Page: http://www.quantstart.com
This is the output I get after running python trading.py
Traceback (most recent call last): File "trading/trading.py", line 12, in <module> from qsforex.execution.execution import OANDAExecutionHandler File "/root/venv/qsforex/local/lib/python2.7/site-packages/qsforex/execution/execution.py", line 13, in <module> import urllib3 ImportError: No module named urllib3
This could just be my setup, but I've tried pip install urllib3 --upgrade
and the issue persists
The drawdown curve produced subsequent to the backtest takes a long time to calculate. Profile it and improve the speed (possibly by removing a direct for-loop).
Dear Michael,
If one wants to trade GBPEUR pair using USD nominated account,
he will need GBPEUR, GBPUSD and EURUSD prices to calculate pnl.
So, I think that home/quote and home/base pairs as well as their reciprocals shall be in included in the price_dict.
Thanks,
Maxim
Dear Michael,
when you are inverting prices in the PriceHandler
inv_bid shall be 1/ask and inv_ask shall be 1/bid
since bid is always smaller than ask
Thanks,
Maxim
2016-08-22 20:01:33,803 - qsforex.trading.trading - INFO - Received new order event: Type: ORDER, Instrument: GBPUSD, Units: 2000, Order Type: market, Side: buy
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 763, in run
self.__target(_self.__args, *_self.__kwargs)
File "trading/roc.py", line 45, in trade
execution.execute_order(event)
File "/home/weiwu/.virtualenvs/qsforex/local/lib/python2.7/site-packages/qsforex/execution/execution.py", line 73, in execute_order
response = self.conn.getresponse().read().decode("utf-8").replace("\n","").replace("\t","")
File "/usr/lib/python2.7/httplib.py", line 1051, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 415, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 379, in _read_status
raise BadStatusLine(line)
BadStatusLine: ''
2016-08-22 21:15:34,320 - qsforex.trading.trading - INFO - Received new order event: Type: ORDER, Instrument: GBPUSD, Units: 2000, Order Type: market, Side: sell
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 763, in run
self.__target(_self.__args, *_self.__kwargs)
File "trading/roc.py", line 45, in trade
execution.execute_order(event)
File "/home/weiwu/.virtualenvs/qsforex/local/lib/python2.7/site-packages/qsforex/execution/execution.py", line 71, in execute_order
params, headers
File "/usr/lib/python2.7/httplib.py", line 979, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.7/httplib.py", line 1007, in _send_request
self.putrequest(method, url, **skips)
File "/usr/lib/python2.7/httplib.py", line 877, in putrequest
raise CannotSendRequest()
CannotSendRequest
After this error the trade thread is dead, the program will only show price streaming debug log into the console.
how to solve this? thank you.
Hello. I am running Ubuntu 14.04 and have tried using this program. When I try to run trading.py I obtain the following error message:
Traceback (most recent call last):
File "trading/trading.py", line 68, in
prices, events, equity=equity, backtest=False
File "/home/justin/forexEnv/local/lib/python2.7/site-packages/qsforex/portfolio/portfolio.py", line 31, in init
self.backtest_file = self.create_equity_file()
File "/home/justin/forexEnv/local/lib/python2.7/site-packages/qsforex/portfolio/portfolio.py", line 74, in create_equity_file
out_file = open(os.path.join(OUTPUT_RESULTS_DIR, filename), "w")
File "/home/justin/forexEnv/lib/python2.7/posixpath.py", line 77, in join
elif path == '' or path.endswith('/'):
I would really appreciate your help, and thanks for making such a helpful blog.
Note: I doubt this would make much of a difference, but I was having a lot of difficulty installing scipy in a virtualenv so I used the --system-site-packages options when creating the virtualenv.
The warning is as follows:
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.or...
InsecureRequestWarning)
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 810, in
__bootstrap_inner
self.run()
File "C:\Python27\lib\threading.py", line 763, in run
self.__target(_self.__args, *_self.__kwargs)
File "trading.py", line 40, in trade
execution.execute_order(event)
File "c:\documents\qsforex\execution\execution.py", line 71, in
execute_order
response = self.conn.getresponse().read()
File "C:\Python27\lib\httplib.py", line 1132, in getresponse
response.begin()
File "C:\Python27\lib\httplib.py", line 453, in begin
version, status, reason = self._read_status()
File "C:\Python27\lib\httplib.py", line 417, in _read_status
raise BadStatusLine(line)
Some people have experienced this bug when holding positions for a long time (20 mins+). Investigate!
I would be interested in using qsforex for trading Bitcoins on Kraken and would be willing to implement PriceHandler and ExecutionHandler.
My problem is that I am not sure how these modifications will fit into the overall architecture without making a total mess. If someone could help me or point me to solutions, I would be willing to code the suggested changes if I have time.
Or should I look into qstrader instead?
Add a logging system with timestamps to account for new positions being added/removed, changes to account balance etc.
"I was reading through your code, specially the 'Position' class and something seemed a bit confusing. When you define 'remove_units' method, it doesn't seem that you are using the 'remove_price', shouldn't that be used to update the 'self.cur_price' instead?"
I see that development ceased about two years ago as per the datetime in the code listing.
I ran trading/trading.py but the output to the screen did not occur
(qsforex) gs-macbook-air:qsforex ghbcode$ python trading/trading.py
2017-02-09 10:24:10,978 - qsforex.trading.trading - INFO - Starting trading thread
2017-02-09 10:24:10,978 - qsforex.trading.trading - INFO - Starting price streaming thread
I found a discussion about this but it didn't help. I changed the heartbeat from 0.0 to 0.5 and no luck. So I traced the program and found that 'resp' from trading.py returns
<Response [400]>
In the two years that development ceased the request syntax may have changed.
Please excuse me if this is covered elsewhere. I looked around and via google without much luck.
Thanks!
Hi Michael,
Can I ask a question about the heartbeat?
Does the thread stop as the heartbeat interval if there is no new ticks coming in? And the thread resumes back to work if there's new ticks coming in?
How can I make my trading system working on a 5 or 15 minutes time frame interval instead of tick by tick?
Thank you.
Regards,
Wei
Have useful input instead of direct pasting of server/response messages to the screen.
Hi,
You might use requests instead of urllib
https://github.com/mhallsmoore/qsforex/blob/d191aad64102ebfa113357987f59f61f0667a190/execution/execution.py
http://docs.python-requests.org/en/latest/
So you won't have to maintain boilerplate code to support both Python 2 and Python 3
Kind regards
Hi Mike,
I previously implemented your code that you provided in Trading Diary #1 and #2, and it seems good. However, after I updated the code according to the Github resource, when I run the trading.py there's no feedback from the screen, and no error information neither. Any suggestions?
Best,
the format of pair string is too hard coded. Thus the whole package can only deal with pure currency pairs. But for some new supported product, it doesn't work at all, ex:AU200_AUD, wheat_usd, etc. The format should be changed such that instead of
instrument = "%s_%s" % (event.instrument[:3], event.instrument[3:])
we can simply input the pair string. and we just need pair.split('_') to get the list of the two instrument.
moreover, why not use some oanda api wrapper to do the execution instead of writing our own?
There are well developed python wrappers for v1 and v20 account, we just need to do an conditional import
Looks like all new accounts go to the V20 interface. I don't see an option to use the old legacy v1 interface.
Is there any plans on supporting v20?
Would be fun to toy with LSTMs using tensorflow or something lol. I've already started switching some of the code. Pretty much just stream and execution to use v20 using the abstraction layer https://github.com/hootnot/oanda-api-v20. I'll make a pull request when I am done if anyone want to check that out.
The portfolio class works with the variable 'side' with possible values: 'buy' and 'sell' where the position class except it to be LONG or SHORT (line 21 position.py)
Hello,
I see in your code https://github.com/mhallsmoore/qsforex/blob/master/backtest/backtest.py#L60
usage of string
Why not using Enum
?
For Python 3
https://docs.python.org/3/library/enum.html
For Python 2.7 there is enum34 https://pypi.python.org/pypi/enum34
Kind regards
PS: "buy", "sell", "long", "short" should also be Enum
Here is an example usage
in event.py
from enum import Enum
EventType = Enum("EventType", "TICK BAR SIGNAL ORDER FILL")
in backtest.py
if event.type == EventType.TICK:
self.strategy.calculate_signals(event)
self.portfolio.update_portfolio(event)
self.ticks += 1
elif event.type == EventType.SIGNAL:
self.portfolio.execute_signal(event)
elif event.type == EventType.ORDER:
self.execution.execute_order(event)
Other Enums to define:
in price_handler/__init__.py
PriceHandlerType = Enum("PriceHandlerType", "TICK BAR")
in portfolio/position.py
PositionType = Enum("PositionType", "LONG", "SHORT")
For qstrader
in position/position.py
PositionActionType = Enum("PositionActionType", "BOT", "SLD")
Currently the code is only working for Python 2.7.x, but since Python 3.4.x is now rapidly becoming the standard, the code needs to be updated.
Hi there!
is this project still alive? I see many pull requests still opened.
Hi Mike, Thanks for sharing the code. I noticed the time interval between two ticks from Oanda is not constant. Sometimes it is short, around 0.1s. Sometimes, it can be more than 5 seconds. I am wondering is it my internet problem or code itself?
Jack
When I'm reading the code, I noticed that qsforex can't have position with trailing stop.
https://github.com/mhallsmoore/qsforex/blob/master/execution/execution.py#L62
Is this my misunderstand? If somebody knows how to make order with trailing stop, please teach me.
Hi, Michael, I am getting this error under the virtual environment when executing this command:
python trading/trading.py
can you help? thank you.
Traceback (most recent call last): File "trading/trading.py", line 49, in <module> logging.config.fileConfig('../logging.conf') File "/usr/lib/python2.7/logging/config.py", line 77, in fileConfig formatters = _create_formatters(cp) File "/usr/lib/python2.7/logging/config.py", line 113, in _create_formatters flist = cp.get("formatters", "keys") File "/usr/lib/python2.7/ConfigParser.py", line 607, in get raise NoSectionError(section) ConfigParser.NoSectionError: No section: 'formatters'
Firstly, thank you very much for the great work on qsforex.
I believe one could develop and backtest strategies based on OHLCV data faster (considering calculations necessary, time jumps faster etc..)... Since I thought considering spread, trading would be profitable at least in the scale of half an hour... As such feature is already established in qstrader, do you think it could also be established on qsforex?
Thank you :)
I would also try to do this on my side as it seems to be very important to me :)
"Also, when we are calculating the PnL we always hedge back to home currency. So why aren't we using 'Bid' always? For example, say we are short on GBP_USD and the account is denominated in HKD. If we want to close the position then we buy back GBP_USD however we should still use 'bid' price for 'USD_HKD'."
Hello,
it will be a good idea to add some unit tests and continuous integration (Travis might help)
Kind regards
$ python qsforex/examples/mac.py
/Users/femto/cache/data/random/GBPUSD_20140101.csv
Traceback (most recent call last):
File "qsforex/examples/mac.py", line 27, in <module>
equity=settings.EQUITY
File "/Users/femto/github/femto/qsforex/backtest/backtest.py", line 29, in __init__
self.ticker = data_handler(self.pairs, self.events, self.csv_dir)
File "/Users/femto/github/femto/qsforex/data/price.py", line 107, in __init__
self.file_dates[self.cur_date_idx]
File "/Users/femto/github/femto/qsforex/data/price.py", line 144, in _open_convert_csv_files_for_day
names=("Time", "Ask", "Bid", "AskVolume", "BidVolume")
File "//anaconda/lib/python3.5/site-packages/pandas/io/parsers.py", line 562, in parser_f
return _read(filepath_or_buffer, kwds)
File "//anaconda/lib/python3.5/site-packages/pandas/io/parsers.py", line 315, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "//anaconda/lib/python3.5/site-packages/pandas/io/parsers.py", line 641, in __init__
self.options, self.engine = self._clean_options(options, engine)
File "//anaconda/lib/python3.5/site-packages/pandas/io/parsers.py", line 755, in _clean_options
_validate_header_arg(options['header'])
File "//anaconda/lib/python3.5/site-packages/pandas/io/common.py", line 265, in _validate_header_arg
raise TypeError("Passing a bool to header is invalid. "
TypeError: Passing a bool to header is invalid. Use header=None for no header or header=int or list-like of ints to specify the row(s) making up the column names
According docstring
`````` ?pd.read_csv```
header : int or list of ints, default 'infer'
Row number(s) to use as the column names, and the start of the data.
Default behavior is as if set to 0 if no ``names`` passed, otherwise
``None``. Explicitly pass ``header=0`` to be able to replace existing
names. The header can be a list of integers that specify row locations for
a multi-index on the columns e.g. [0,1,3]. Intervening rows that are not
specified will be skipped (e.g. 2 in this example is skipped). Note that
this parameter ignores commented lines and empty lines if
``skip_blank_lines=True``, so header=0 denotes the first line of data
rather than the first line of the file
so in https://github.com/mhallsmoore/qsforex/blob/master/data/price.py#L140
self.pair_frames[p] = pd.io.parsers.read_csv(
pair_path, header=True, index_col=0,
parse_dates=True, dayfirst=True,
names=("Time", "Ask", "Bid", "AskVolume", "BidVolume")
)
should be replaced by
self.pair_frames[p] = pd.io.parsers.read_csv(
pair_path, header=0, index_col=0,
parse_dates=True, dayfirst=True,
names=("Time", "Ask", "Bid", "AskVolume", "BidVolume")
)
Metatrader 5 provides 2 kind of position accounting
https://www.mql5.com/en/articles/2299
It will be a nice feature to have in qsforex / qstrader
Hi,
Would be useful to add a little GUI (Graphic User Interface) to the module.
=> A few entry forms for key parameters
=> A few entry forms for config settings
=> A button to start and stop trading.
I will be doing some work upon it also for my instance of qsforex.
First run of trading/trading.py:
"Import error: No module named qsforex.execution.execution"
Hi,
I use Windows 10. When I've ran "traiding.py" in Spyder appear the following error: ImportError: No module named 'qsforex'
I setup an account to obtain the API authentication credentials, which you will need to carry out live trading.
I made the changes in "settings.py" according to "Forex Trading Diary 1 - Automated Forex Trading with the AONDA API".
I've downloaded the zip file of the current master branch at https://github.com/mhallsmoore/qsforex/archive/master.zip.
I put all the files in the same directory.
I created two directories in it: "QSFOREX_CSV_DATA_DIR" and "QSFOREX_OUTPUT_RESULTS_DIR".
I don't know set environment variables.
Can anyone help to solve the problem?
Best Regards.
Edilson
Hello,
it will be a great idea to provide a pip package.
Kind regards
HI, I'm looking at the code trying to figure out how hard to make it works to another instrument, like commodity. I'm getting into commodity trading strategy, focusing on intraday pair trading arb.
thanks
When running the backtest for mac.py I get an error as per below.
(qsforex) jbowler@GEN-U-DAE-01:~/Projects/qsforex$ python examples/mac.py
Traceback (most recent call last):
File "examples/mac.py", line 29, in
backtest.simulate_trading()
File "/home/jbowler/venv/qsforex/lib/python2.7/site-packages/qsforex/backtest/backtest.py", line 81, in simulate_trading
self._run_backtest()
File "/home/jbowler/venv/qsforex/lib/python2.7/site-packages/qsforex/backtest/backtest.py", line 57, in _run_backtest
self.ticker.stream_next_tick()
File "/home/jbowler/venv/qsforex/lib/python2.7/site-packages/qsforex/data/price.py", line 182, in stream_next_tick
bid = Decimal(str(row["Bid"])).quantize(
File "/home/jbowler/ProgramData/anaconda2/lib/python2.7/decimal.py", line 547, in new
"Invalid literal for Decimal: %r" % value)
File "/home/jbowler/ProgramData/anaconda2/lib/python2.7/decimal.py", line 3873, in _raise_error
raise error(explanation)
decimal.InvalidOperation: Invalid literal for Decimal: 'Bid'
How is this fixed?
Thank-you
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.