quantsoftware / quantsoftwaretoolkit

QuantSoftwareToolkit

##QSTK QuantSoftware Toolkit

##Website & Documentation wiki.quantsoftware.org

##Source Repository https://github.com/QuantSoftware/QuantSoftwareToolkit

##Dependencies

• numpy >= 1.6.1,
• scipy >= 0.9.0,
• matplotlib >= 1.1.0,
• pandas >= 0.7.3,
• python-dateutil==1.5,
• cvxopt >= 1.1.3,
• scikit-learn >= 0.11

Vagrant Image

Vagrant image with QSTK installed can be found at https://github.com/Wesseldr/vagrant-QSTK

##Author

• Prof. Tucker Balch
• Sourabh Bajaj
• John Cornwell

##License The QuantSoftware ToolKit (QSTK) is released under the New BSD License as follows:

Copyright (c) 2011, 2012, Georgia Tech Research Corporation (GTRC) All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the Georgia Tech Research Corporation nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE GEORGIA TECH RESEARCH CORPORATION BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

quantsoftwaretoolkit's Issues

Automated scripts for Installation

Cannot Find TimeSeries

  File "Validation.py", line 123, in <module>
    import QSTK.qstkutil.tsutil as tsu
  File "/usr/local/lib/python2.7/dist-packages/QSTK-0.2.8-py2.7.egg/QSTK/qstkutil/tsutil.py", line 19, in <module>
    from QSTK.qstkutil import qsdateutil
  File "/usr/local/lib/python2.7/dist-packages/QSTK-0.2.8-py2.7.egg/QSTK/qstkutil/qsdateutil.py", line 38, in <module>
    GTS_DATES = _cache_dates()
  File "/usr/local/lib/python2.7/dist-packages/QSTK-0.2.8-py2.7.egg/QSTK/qstkutil/qsdateutil.py", line 36, in _cache_dates
    return pd.TimeSeries(index=dates, data=dates)
AttributeError: 'module' object has no attribute 'TimeSeries'

Any Ideas on how to solve this?

unable to add python 2 path in Windows installation

Hi,
It would be helpful if an option to specify python2 path during installation instead of it reading from registry file in windows installation.
QSTK v.0.2.6

.

Update setup.py with metadata

Trouble with getNYSEdays function

The documentation of the getNYSEdays function says that it creates a list of timestamps between startday and endday (inclusive). With endday being the last day to consider.

As I see, the list is not inclusive:

import QSTK.qstkutil.qsdateutil as du
import datetime as dt

dt_start = dt.datetime(2012, 1, 3)
dt_end = dt.datetime(2012, 1, 4)
dt_timeofday = dt.timedelta(hours=16)
ldt_timestamps = du.getNYSEdays(dt_start, dt_end, dt_timeofday)
ldt_timestamps

[Timestamp('2012-01-03 16:00:00', tz=None)]

pip install raises "error: Namespace package problem: QSTK is a namespace package, ..."

Hello,

with Ubuntu 14.04 LTS

$ sudo pip install QSTK
Downloading/unpacking QSTK
Downloading QSTK-0.2.7.tar.gz (39.0MB): 39.0MB downloaded
Running setup.py (path:/tmp/pip_build_root/QSTK/setup.py) egg_info for package QSTK

error: Namespace package problem: QSTK is a namespace package, but its
__init__.py does not call declare_namespace()! Please fix it.
(See the setuptools manual under "Namespace Packages" for details.)

Complete output from command python setup.py egg_info:
running egg_info

creating pip-egg-info/QSTK.egg-info

writing requirements to pip-egg-info/QSTK.egg-info/requires.txt

writing pip-egg-info/QSTK.egg-info/PKG-INFO

writing namespace_packages to pip-egg-info/QSTK.egg-info/namespace_packages.txt

writing top-level names to pip-egg-info/QSTK.egg-info/top_level.txt

writing dependency_links to pip-egg-info/QSTK.egg-info/dependency_links.txt

writing manifest file 'pip-egg-info/QSTK.egg-info/SOURCES.txt'

warning: manifest_maker: standard file '-c' not found

error: Namespace package problem: QSTK is a namespace package, but its

init.py does not call declare_namespace()! Please fix it.

(See the setuptools manual under "Namespace Packages" for details.)

---

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/QSTK
Storing debug log for failure in /home/ubuntu/.pip/pip.log

Add _init_ to manifest.

Install on Windows does not come with an init, eclipse has issues recognizing QSTK due to that.

Wiki Update

1. Windows Installation
   • Pandas Version Change
   • QSTK new release
2. Mac Installation
   • Rewrite the whole thing
3. Ubuntu Installation
   • Change version numbers
4. Homepage
   • Remove old tutorials

The wiki website is down.

It seems that the website is down: http://wiki.quantsoftware.org/

Support for different python versions

update change.txt

Update data till 2013

Update to latest data.

Error bars on eventprofiler are hardly visible

The alpha used equals 0.1 combined with the pastel color #AAAFF
is almost invisible on PDF charts (see screenshots). The light color of #AAAFF would probably already sufficient to have a distinction between the error bars and the chart.
Suggestion setting it to 0.9 for a much more clear and visible error bar on the event study charts.

On line 76~77 in the file QuantSoftwareToolkit / QSTK / qstkstudy / EventProfiler.py

    plt.errorbar(li_time[i_lookback:], na_mean[i_lookback:],
                    yerr=na_std[i_lookback:], ecolor='#AAAAFF',
                    alpha=0.1)
```python

Wessel

clean up QSTK and homebrew

Installation does not work with python 2.7.5 on MAC Mavericks

I tried to install it with the installation script but it was throwing me the same error than later ("QSTK is a namespace package...").

I then tried to upload the latest zip from https://pypi.python.org/packages/source/Q/QSTK/, and launched sudo python setup.py install. Here is the result :

> sudo python setup.py install
running install
Checking .pth file support in /Library/Python/2.7/site-packages/
/usr/bin/python -E -c pass
TEST PASSED: /Library/Python/2.7/site-packages/ appears to support .pth files
running bdist_egg
running egg_info
writing requirements to QSTK.egg-info/requires.txt
writing QSTK.egg-info/PKG-INFO
writing namespace_packages to QSTK.egg-info/namespace_packages.txt
writing top-level names to QSTK.egg-info/top_level.txt
writing dependency_links to QSTK.egg-info/dependency_links.txt
error: Namespace package problem: QSTK is a namespace package, but its
__init__.py does not call declare_namespace()! Please fix it.
(See the setuptools manual under "Namespace Packages" for details.)

I tried to launch it with python2.5 and it was not sending this error message... It doesn't seem that there is a lot to fix, just adding a line in a init.py file, but I don't know exactly...

Thanks :)

Documentation

Virtual box image of light weight distro

Command "python setup.py egg_info" failed with error code 1 in ...

Hello:

I am currently trying to install qstk on a mac osx 10.12 by running the following command.
pip install -i https://pypi.anaconda.org/pypi/simple qstk

Here is the error that I get.

I am using Anaconda python 3.5.3

Any help would be appreciated. Thank you :)

Data update

Is there a mode in DataAccess where I can download only the new data in each of the .csv files ?

Basically the code download the whole file up to today or only download the extra dates up to today and add them to the existing data in the corresponding file ?

This is quite useful when the data provider only keeps a certain amount of data, say 4 years for instance.

Verbose param in dataaccess

Verbose to be added to various functions.

Data Path

Hi Guys,

I have received approval to teach the course again. Yay!

I’m double checking that you can help with the course. Brian Hrolenok will also help here at GT.

QSTK issue. Actually 2 of them:

1. Where is the data? Is the data distributed via the github distribution?
2. I don’t like that the Examples.zip depends on a machine I don’t have control over, e.g.

curl -O https://spark-public.s3.amazonaws.com/compinvesting1/QSTK-Setups/Examples.zip

Can/should we move it to github?

Thanks

Tucker Balch

Error : QSTK can not be imported or not installed.

Hello

When I have just installed QSTK, but when I run the Validation.py file, I get an error saying QSTK is not installed.

Below is the terminal output during installation.

Any help is appreciated.

Current Directory :  /home/anton/Documents/software/QSTK/QSTK-0.2.8/Examples

Files in the current directory
EventProfiler
FeatureSearch
DataAccess
Validation.py
KNN
Features
Basic

Numpy is installed and the version used is :  1.8.2
Please make sure you're using version >= 1.6.1

Matplotlib is installed and version is :  1.4.2
Please make sure you're using version >= 1.1.0

Pandas is installed and the version used is :  0.17.1
Please make sure you're using version >= 0.7.3

Scipy is installed and the version used is :  0.14.1
Please make sure you're using version >= 0.9.0

Dateutil is installed and the version used is :  2.2
Please make sure you're using version == 1.5

Setuptools is installed and the version used is :  18.4
Please make sure you're using version >= 0.6

datetime is installed and can be imported

All dependencies are installed and working

Error : QSTK can not be imported or not installed.
anton@anton-K93SV:~/Documents/software/QSTK/QSTK-0.2.8/Examples$ 

Event profiler crashes in case of no event.

Fix the event profiler for cases when it crashes in case of zero events.

Raise an error instead.

brew install matplotlib

pip install pyparsing

matplotlib: Unsatisfied dependency: pyparsing
Homebrew does not provide Python dependencies; install with:
  pip install pyparsing
matplotlib: Unsatisfied dependency: python-dateutil
Homebrew does not provide Python dependencies; install with:
  pip install python-dateutil
Error: Unsatisifed requirements failed this build.

Fill prices in the tutorials

Fill prices in all the tutorials.

Travis integration

DataAccess.get_data hash function for symbols does not consider order of symbol list

Orders of symbols should matter in cacheing. Better way to implement would be to reindex at the end of the program.

Function to check if symbol is valid

Function to check valid symbols.

Mac Installation

Clean it up.

Mac remove VE

EPD Python Path

Traceback (most recent call last):
File "C:\Users\federico\Documents\Formació\Computational Investment\Validation.py", line 19, in
print "Your Python Version is : ", platform.python_version()
File "C:\Python27\lib\platform.py", line 1468, in python_version
return _sys_version()[1]
File "C:\Python27\lib\platform.py", line 1423, in _sys_version
repr(sys_version))
ValueError: failed to parse CPython sys.version: '2.7.3 |EPD_free 7.3-2 (32-bit)| (default, Apr 12 2012, 14:30:37) [MSC v.1500 32 bit (Intel)]'

links in readme

Prune the repository of large files and all distributions

Remove dead tutorials

Remove the tutorials that use Compustat and NAG from the distribution

tutorial5.py does not run

When running the tutorial5.py script it gives the error message below.
I suspect it is because of numpy version 1.8.2 but I'm not sure.

Changing the line
if f_x >= 0:
to
if f_x.item() >= 0:
Seems to fix the problem

Traceback (most recent call last):
File "tutorial5.py", line 92, in
main()
File "tutorial5.py", line 82, in main
i_target_leverage=1, f_rate_borrow=3.5, log="transaction.csv")
File "/usr/local/lib/python2.7/dist-packages/QSTK-0.2.8-py2.7.egg/QSTK/qstksim/tradesim.py", line 304, in tradesim
prediction_shares = prediction_shares.apply(_nearest_interger)
File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 3596, in apply
return self._apply_standard(f, axis, reduce=reduce)
File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 3686, in _apply_standard
results[i] = func(v)
File "/usr/local/lib/python2.7/dist-packages/QSTK-0.2.8-py2.7.egg/QSTK/qstksim/tradesim.py", line 132, in _nearest_interger
if f_x >= 0:
File "/usr/local/lib/python2.7/dist-packages/pandas/core/generic.py", line 698, in nonzero
.format(self.class.name))
ValueError: ('The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().', u'occurred at index AAPL')

Advanced installation methods

Try installation directly using apt-get & homebrew

Examples zip file

Remove everything apart from tutorials

Support for Mavericks

pandas version update

Remove overhead files that are living inside the different directories

Clean up directories to reduce size and remove junk files

Fix the list of S&P5002008

Fix the missing symbols. Try to make it complete 500 syms.

Bugs in the indicators

Bug in Aaroon indicator -- Needs confirmation.

Support Python3?

Since Python2 will end in 2020, could you please update this package for Python3?

Ubuntu Installation

Move to virtual environments

Update and fix the feature code

Test the features code from the distribution for part2 of the class.

tsutil should work on data frames

Event Profiler various bugs

1. Better handling of 0,1 event found.

2. Should not modify the event matrix

FutureWarning: TimeSeries broadcasting along DataFrame index by default is deprecated

I am seeing this warning message when using QSTK 0.2.8 with pandas 0.14.1 on Ubuntu
(I suspect there is some version sensitivity at work here.)

The message led me to believe my old event study code would still work however I was detecting over 200K events where only a few 100s were expected. After modifying my code to eliminate the broadcasting I was still getting the warning for qstkstudy/EventProfiler.py

I found that line 35:
df_rets = df_rets - df_rets[s_market_sym]

needed to be changed to:
df_rets = df_rets.sub(df_rets[s_market_sym], axis=0)

The broadcasting change is explained in pandas docs http://pandas.pydata.org/pandas-docs/stable/dsintro.html?highlight=broadcasting#data-alignment-and-arithmetic and the pink "Warning" box there describes exactly this case.

EventProfiler example does not work

Examples/EventProfiler/tutorial.py does not seem to produce expected result.

The following warning is produced:

/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/QSTK/qstkutil/qsdateutil.py:36: FutureWarning: TimeSeries is deprecated. Please use Series
  return pd.TimeSeries(index=dates, data=dates)

The generated PDF graph is blank
MyEventStudy.pdf

This is the output of pip list

cycler (0.9.0)
matplotlib (1.5.0)
nose (1.3.7)
numpy (1.10.1)
pandas (0.17.1)
pip (7.1.0)
pyparsing (2.0.6)
python-dateutil (1.5)
pytz (2015.7)
QSTK (0.2.8)
scikit-learn (0.17)
scikits.statsmodels (0.3.1)
scipy (0.16.1)
setuptools (18.0.1)
six (1.10.0)
wheel (0.24.0)