rxavier / econuy

Wrangling Uruguayan economic data so you don't have to.

Hi! I'm Rafael Xavier

Summary

I am a data scientist holding a Masters degree in Economics. I have 7+ years of experience working with statistical and machine learning models, involving projects in public policy research as well as in consulting.

I work at Mercado Libre, Latin America's largest e-commerce site, as a Data Science Technical Leader. Prior to that I built and led a data science team at one of Uruguay's largest consulting firms.

Current projects

I enjoy building tooling for data scientists. My team built Embedding Encoder, a scikit-learn compatible transformer that leverages neural network embedding layers to process categorical variables, and I'm currently working on Poniard, a scikit-learn companion library that streamlines model evaluation.

Past projects

I built econuy, a data retrieval and processing library for Uruguayan economic statistics. I also built a Flask/Plotly Dash frontend hosted at econ.uy that's been online since mid-2020.

econuy's Issues

econuy.utils.lstrings is kind of a mess

This one needs a complete overhaul.

One option is to extract the urls dict into its own file, rename it to sources (since it's more than just URLs).

Whatever is left should be organized. Some of the objects could be placed back in their own functions or even files (like the CPI measures weights).

If possible, this file should disappear.

cannot import name 'Session' from 'econuy'

despues de realizar el pip install, cuando voy a correr:

from econuy import Session sess = Session(location="/Users/fededeleon/econuy") df = sess.get(dataset="cpi").dataset

me aparece el warning:
cannot import name 'Session' from 'econuy'

cómo lo puedo solucionar?

Muchas gracias!!

Option to save to Excel and save metadata in another sheet

First part is super easy (through Session().save() and utils.ops._io()).

Second part is not complicated. In fact, in order to save and load to SQL something of the sort is already done (see _io()).

Some retrieval functions return dicts

economic_activity.national_accounts, external_sector.trade, fiscal_accounts.balance and fiscal_accounts.public_debt in econuy.retrieval return dicts containing dataframes instead of a single dataframe.

That was an old design decision (national_accounts was the first retrieval function I wrote) that looks out of place nowadays. Basically, there's two reasons for this:

1. The download file has multiple sheets, many of which are useful and cannot be logically merged into a single dataframe. That's the case of fiscal data.
2. Similar data is held in similar download files, which makes it easier to share a single retrieval function. That's the case of the other two.

Case 2 is easily solved. Just split the current functions in as many as needed, while dealing with their name parameter (which should point to the correct sources in utils.lstrings.urls).

Case 1 is trickier since it's a single file. An option is to have a function which downloads the file, tidies it and saves it. Then add one function per sheet of interest which just loads the downloaded file and does the rest of the processing.

Missing files in sdist

It appears that the manifest is missing at least one file necessary to build
from the sdist for version 0.7.3. You're in good company, about 5% of other
projects updated in the last year are also missing files.

+ /tmp/venv/bin/pip3 wheel --no-binary econuy -w /tmp/ext econuy==0.7.3
Looking in indexes: http://10.10.0.139:9191/root/pypi/+simple/
Collecting econuy==0.7.3
  Downloading http://10.10.0.139:9191/root/pypi/%2Bf/d30/e8e6efdb40edc/econuy-0.7.3.tar.gz (42 kB)
    ERROR: Command errored out with exit status 1:
     command: /tmp/venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-wheel-ooi5dob3/econuy/setup.py'"'"'; __file__='"'"'/tmp/pip-wheel-ooi5dob3/econuy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-wheel-ooi5dob3/econuy/pip-egg-info
         cwd: /tmp/pip-wheel-ooi5dob3/econuy/
    Complete output (7 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-wheel-ooi5dob3/econuy/setup.py", line 18, in <module>
        pipfile_lock_requirements = get_requirements_from_pipfile_lock()
      File "/tmp/pip-wheel-ooi5dob3/econuy/setup.py", line 12, in get_requirements_from_pipfile_lock
        lock_data = json.load(open(pipfile_lock))
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-wheel-ooi5dob3/econuy/Pipfile.lock'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.