Quandl integration about pandas-datareader

[bashtage]

Quandl already exports to Pandas. I don't see much value add in duplicating existing packages.

[femtotrader]

pinging @briancappello and @davidastephens

maybe we should consider a sort of "delegation" mechanism.

[femtotrader]

I would to tell you more about this "delegation" concept.

I don't think most DataReader should have Pandas as dependency.
A DataReader should be able to perform a request and get a result.

If Pandas is installed and user want to use it we should import it and use it
if pandas is not installed or user don't want to use it we should return "raw" Python data (string, list, dict, ...)

Here is how I think delegation could work.
I'm taking Yahoo Daily as it's an easy datareader
I volontary add requests as dependency... I'm not considering to add it in pandas-datareader (neither click... that's just an example) !

# yahoo_daily.py

import click

import requests

from pandas.compat import StringIO
from pandas_datareader.utils import _sanitize_dates
from delegate import Delegator

_URL = 'http://ichart.finance.yahoo.com/table.csv'


def get(sym, start, end, interval, delegate):
    """
    Get historical data for the given name from yahoo.
    Date format is datetime

    Returns a DataFrame.
    """
    start, end = _sanitize_dates(start, end)
    params = {
        's': sym,
        'a': start.month - 1,
        'b': start.day,
        'c': start.year,
        'd': end.month - 1,
        'e': end.day,
        'f': end.year,
        'g': interval,
        'ignore': '.csv'
    }
    response = requests.get(_URL, params=params)
    status_code = response.status_code
    if status_code == 200:
        dat = response.text

        with Delegator('pandas', delegate) as pd:
            if pd is not None:
                # Pandas is installed and user want to use it
                df = pd.read_csv(StringIO(dat))
                df = df.set_index("Date")
                return df
            else:
                # Pandas is not installed or user doesn't want to use it
                return dat
        del pd

    else:
        raise Exception("status_code=%d - it should be 200" % status_code)

@click.command()
@click.option('--delegate/--no-delegate', default=True)
def main(delegate):
    result = get("AAPL", "2015-04-01", "2015-08-01", "d", delegate)
    print(result)
    print(type(result))

if __name__ == "__main__":
    main()


# delegate.py

class Delegator(object):
    def __init__(self, s_module, delegate):
        self.s_module = s_module
        self.delegate = delegate

    def __enter__(self):
        try:
            if self.delegate:
                i = __import__(self.s_module, fromlist=[''])
                return i
            else:
                return None
        except ImportError:
            return None

    def __exit__(self, *args):
        pass



# =======


$ python yahoo_daily.py --no-delegate

This example is just with Yahoo Daily.
I don't think we should do this here (in pandas-datareader) but I think this is typically how others packages (maybe like quandl-python) should do in order not to have Pandas as dependency but to be able to output Pandas DataFrame eventually.

The Delegator should be an other package, with an other github repository, ...
This package should have no dependency and be usable with most Python version still available.

[bashtage]

from pandas.compat import StringIO

requires pandas. I don't think this package should aspire to drop pandas, especially in consideration of the same.

I also think you are making thigs too hard to have optional packages. Most people do things like

try:
    import pandas as pd
    _has_pandas = True
except:
        _has_pandas = False

and then later

# Read data to lists of lists
if _has_pandas: 
    return pd.DataFrame(list_of_list)

Of course, the big downside is that you lose all of pandas functionality, including dates, etc.

[femtotrader]

Sorry

    with Delegator('pandas', delegate) as pd:
        if pd is not None:
            with Delegator('pandas.compat', delegate) as compat:
                df = pd.read_csv(compat.StringIO(dat))
                df = df.set_index("Date")
                return df
        else:
            return dat

Hum... it could become tricky with a lot of package to import

Yes _HAS_PANDAS is probably less tricky.

[bashtage]

Quandl is supported. Closing.