Coder Social home page Coder Social logo

mathsci / fecon236 Goto Github PK

View Code? Open in Web Editor NEW
120.0 9.0 50.0 372 KB

Tools for financial economics. Curated wrapper over Python ecosystem. Source code for fecon235 Jupyter notebooks.

Home Page: https://git.io/econ

License: Other

Shell 7.51% Python 90.78% Dockerfile 1.71%
python fecon235 pandas time-series econometrics statistics economics finance interest-rates bonds

fecon236's Introduction

fecon236 :: Tools for financial economics

Curated wrapper over Python ecosystem. Source code for fecon235 Jupyter notebooks.

GitHub repository is at fecon236, see CHANGELOG for revision history. The protected master branch gets released via pip, see our PyPI. The develop branch is where pull requests are currently directed.

Gitter / master Build Status / develop Build Status

fecon236 logo

What is this repository for?

fecon236 provides an interface for financial economics to the Python ecosystem, especially packages for mathematics, statistics, science, engineering, and data analysis. Complex packages such as numpy, pandas, statsmodels, scipy, and matplotlib are seamlessly integrated at a high-level with APIs of various data hosts for:

  • Essential commands which correctly handle annoying low-level pitfalls.

  • Retrieval of economic and financial data, both historical and the most current.

  • Data munging, for example, resampling and alignment of time-series data from hosts using mutually incompatible formats.

  • Analysis using techniques from econometrics, time-series analysis, and statistical machine learning.

  • Abstraction and software optimization of mathematical operators, for example, linear algebra used in portfolio analysis.

  • Visualization of data using graphical packages.

  • Reproducible research which is collaborative and openly accessible at zero cost.

To practically test theoretical ideas interactively, fecon236 can employed with any Python IDE interactive development environment, IPython console, or with a Jupyter notebook. The code has been tested against both python27 and python3 since 2014, and works across major platforms: Linux, Mac, and Windows.

The best way to see the convenience of fecon236 in action is to run the notebooks in the fecon235 nb directory.

How does one get started?

For installation details and FAQ, please first visit our wiki. For the casual user in an Anaconda environment, we recommend: conda update pip, then pip install --pre fecon236

Documentation is currently being served from 236docs. Please start your orientation with this README notebook which shows how most of this project is self-documenting.

Development status: stable

For the developer, we recommend forking the fecon236 repository, then pip install --editable .

  • fecon235 becomes a repository solely of Jupyter notebooks. The old Python source code at fecon235 will remain for archival purposes, while new code development shifts over to fecon236.

Version 10 of fecon236 represents refactoring of the fecon235 v5.18.0312 Python code, not the Jupyter notebooks, with a new architecture depicted in Appendix 1. Function names have been retained, but under fecon236 expect infrequent function calls to be explicit rather than implicit, i.e. modules names and their aliases are significant.

  • After 2019-01-01, our official support for python27 will discontinue (like numpy and pandas), however, straddling code may still continue to work.

Version 11 of fecon236 will signal when our Travis builds under Python 2.7 fail, and at that point we expect to require at least Python 3.6.

Community

Join the chat at Gitter and ping the lead developer @rsvp. Please consider becoming a member of the Mathematical Sciences Group.

MathSci logo

Appendix 1: fecon236 package map

>>> print(fe.map)
Annotated tree map of package directory [with module aliases]
    fecon236
    ├── __init__.py   (Router, sole non-empty __init__.py file herein)
    ├── tool.py       (Tools, low-level essentials)
    ├── top.py        (Top priority, experimental)
    ├── boots   (Bootstrap)
    │   └── bootstrap.py   [bs]
    ├── dst   (Distributions)
    │   └── gaussmix.py   [gmix]
    ├── econ
    │   └── infl.py
    ├── futures
    │   └── cftc.py
    ├── host
    │   ├── fred.py
    │   ├── hostess.py
    │   ├── qdl.py
    │   ├── _ex_Quandl.py
    │   └── stock.py
    ├── math
    │   └── matrix.py   [mat]
    ├── ml   (Machine Learning)
    │   └── learn.py
    ├── oc   (Optimization Control)
    │   └── optimize.py   [op]
    ├── parse
    │   └── sec.py
    ├── prob   (Probability)
    │   └── sim.py   (Simulation)
    ├── prtf   (Porfolio theory)
    │   └── boltzmann.py   [boltz]
    ├── rates  (Fixed Income)
    │   └── fedfunds.py
    ├── tsa    (Time Series Analysis)
    │   └── holtwinters.py   [hw]
    ├── util   (Utilities)
    │   ├── group.py
    │   └── system.py
    └── visual
        └── plots.py

BSD License and TOS / This page, last update : 2018-07-25

fecon236's People

Contributors

gitter-badger avatar rsvp avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fecon236's Issues

Documentation Discussion

fecon documentation

In-progress

Discussion thread for fecon236 documentation (from #4)

Overview

I have an initial Sphinx documentation implementation in the fecon-docs repository, which in part uses the sphinx-autodoc extension to render the docstrings ("auto-documentation"). There are also small sections which I have built out (such as FRED Codes) for additional context.

The repository is setup to use TravisCI with doctr for CI and CD to Github pages. The travis build lints the documentation source, though I do need to explore options for linting the docstrings themselves (there are a few, each of which have pros and cons).

General Structure

The documentation will be structured as a one-to-one mirror of the modules used in fecon236:

image

Implementation

Implementation will begin as auto-documentation of each fecon236 module.

In order to complete this process, it is easiest to convert all docstrings to the numpydoc format. An initial implementation of these conversions has been completed (see pull requests #10 through #24)

ATTN mention: @rsvp

Disruption of Quandl data, esp. CME futures

Data from Quandl failing -- NASDAQ takeover

For example, when retrieving the Dec 2019 Eurodollar futures contact:

>>> tmp = fe.get('CME/EDZ2019')
ValueError: INVALID symbol string or code variable.

Projects fecon235 and fecon236 have many data dependencies on the Quandl API, including but limited to, futures quotes.

  • Bug
  • Enhancement
  • Upstream disruption

Steps which reproduce behavior

  1. Eurodollar futures quotes relies on CME vendor at Quandl.
  2. "PAGE NO LONGER EXISTS." at usual vendor page: https://www.quandl.com/data/CME-Chicago-Mercantile-Exchange-Futures-Data/documentation/data-organization
  3. Last successful Google crawl on said vendor page was December 1, 2018.
  4. Top notification, "Quandl has been acquired by Nasdaq. Learn more" links to a Press Release from Nasdaq: https://business.nasdaq.com/mediacenter/pressreleases/1855930/nasdaq-acquires-quandl-to-advance-the-use-of-alternative-data which includes the following statements:

December 4, 2018, Nasdaq Acquires Quandl

Nasdaq (Nasdaq: NDAQ) announced today it has acquired Quandl, Inc., a leading provider of alternative and core financial data. Quandl from over 350 sources to more than 30,000 active monthly users. The company offers a global database of alternative, financial and public data, including information on capital markets, energy, shipping, healthcare, education, demography, economics and society.

Nasdaq plans to combine Quandl with its existing Analytics Hub business within Global Information Services.

Terms of the deal were not disclosed, but this investment is consistent with both Nasdaq’s strategy to maximize opportunities as a technology and analytics provider to capital markets, as well as its capital deployment and return on investment capital objectives.

Matthew Sheahan
Phone Number: + 1 212 231 5945
Email: [email protected]

  1. Search for "CME" on Quandl's search page https://www.quandl.com/search no longer returns any results for futures contracts (bonds, FX, metals, etc.) directly provided by CME.
  2. Google search leads to link for "CME Dec 2019 Eurodollar Futures", https://www.quandl.com/CME/EDZ2019 -- but currently displays "PAGE NO LONGER EXISTS." Last successful Google crawl was on November 30, 2018.
  3. Disappearance of CME data thus most likely occurred between November 30 and December 7, 2018. Midway, on December 4, Nasdaq announced acquisition of Quandl, a distributor of CME data.

Additional helpful details for bugs

  • Problem started recently, but not in older versions

  • Problem happens with all files, not only some files

  • Problem can be reliably reproduced

  • Problem happens randomly

  • fecon236 version: 10.8.0

  • Python version: 2.7 and 3 series

  • Operating system: All

Until advised otherwise, we will assume that the Quandl data will be behind paywalls (Nasdaq's Analytics Hub).

Migrate to Python 3 > 2019-01-01

[... continued from fecon235, https://github.com/rsvp/fecon235/issues/9]

Home for our Jupyter notebooks shall remain at fecon235 due to their bulky size. Migration to the python3 kernel should present no problems since fecon235 has been compatible with both python27 and python3.

Version 10 of fecon236 represents the refactoring of only the fecon235 v5.18.0312 library code (not the Jupyter notebooks). It shall maintain compatibility with both python27 and python34.

After 2019-01-01, our official support for python27 will discontinue (following policy at numpy and pandas), however, straddling code may still continue to work.

Version 11 of fecon236 will signal when our Travis builds under python27 fail, and at that point we expect to require at least Python 3.6.

Minor: flake8 upgrade introduces "W605 invalid escape sequence" Syntax Error

flake8 upgrade introduces "W605 invalid escape sequence" error

Travis build 118 on 2018-11-09 has failed -- solely due to fecon236/dst/gaussmix.py -- diagnosis and remedy are discussed herein.

  • Bug (Very minor)
  • Enhancement

Expected behavior

No SyntaxError was expected.

Observed behavior, build 118.2

$ flake8 --version
3.6.0 (mccabe: 0.6.1, pycodestyle: 2.4.0, pyflakes: 2.0.0) CPython 2.7.15 on Linux
The command "flake8 --version" exited with 0.

$ flake8 fecon236
fecon236/dst/gaussmix.py:66:-1721: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-1494: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-1483: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-1474: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-1417: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-1319: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-1307: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-1293: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-1284: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-934: W605 invalid escape sequence '\k'
fecon236/dst/gaussmix.py:66:-822: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-810: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-796: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-787: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-703: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-661: W605 invalid escape sequence '\s'
fecon236/dst/gaussmix.py:66:-648: W605 invalid escape sequence '\k'
fecon236/dst/gaussmix.py:66:-553: W605 invalid escape sequence '\m'
The command "flake8 fecon236" exited with 1.

Steps which reproduce or enhance behavior

  1. PASS: Build 117 https://travis-ci.org/MathSci/fecon236/jobs/449914169
$ flake8 --version
3.5.0 (mccabe: 0.6.1, pycodestyle: 2.3.1, pyflakes: 1.6.0) CPython 2.7.15 on Linux
  1. FAIL: Build 118 https://travis-ci.org/MathSci/fecon236/jobs/452950251
$ flake8 --version
3.6.0 (mccabe: 0.6.1, pycodestyle: 2.4.0, pyflakes: 2.0.0) CPython 2.7.15 on Linux

Why would the improvement be useful to most users?

Conformity with latest flake8 lint and pycodestyle guidelines.

Additional helpful details for bugs

What is W605? Prior to Python 3.6, a backslash-character pair that was not a valid escape sequence generated a DeprecationWarning. This has become a SyntaxError as of flake8 version 3.6.0. See https://lintlyci.github.io/Flake8Rules/rules/W605.html

Anti-pattern: regex = '\.png$'

Best practice: regex = r'\.png$'

Thus the best resolution is to use explicit raw string mode.

  • Problem started recently, but not in older versions

  • Problem happens with all files, not only some files

  • Problem can be reliably reproduced

  • Problem happens randomly

  • fecon236 version: 10.7.1

  • pandas version: 0.22

  • Python version: 2.7 and 3

  • Operating system: All

Documentation needed

So far we have relied largely on demonstrative notebooks
at https://github.com/rsvp/fecon235

The source code here at fecon236 is designed to be more
explicit than casual usage, so if you have experience in
setting up a nice documentation framework, please
kindly volunteer.

The current "docs" need a complete redo, one that starts from fresh eyes.
Nobody peeks at the source code unless there is something which
solves their problem or extends their research interest.

As an analogy, Unix-style man pages are really dry,
but the usage examples are most useful in practice.
That's what fecon235 notebooks were intended to do informally,
but as the research topics got more specialized,
formalization of the structure for clarity became necessary,
hence the refactoring into fecon236.

As an example, forecast() used to be called directly,
but now we prefer fe.hw.forecast(). The user may not know
that fe is shorthand for fecon236 (not fecon235), and
that hw is an alias for the holtwinters.py module which
is physically located implicitly under the tsa directory
for time-series analysis.

This is the sort of thing which needs documentation
which should be a skilled blend of unit-based and
integrative explanations.

Let's us know if you would like to join a MathSci team.
Please discuss further at https://gitter.im/MathSci/fecon236

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.