Coder Social home page Coder Social logo

philsv / myeia Goto Github PK

View Code? Open in Web Editor NEW
14.0 1.0 4.0 78 KB

Python wrapper for the U.S. Energy Information Administration (EIA) API v2

Home Page: https://pypi.org/project/myeia/

License: MIT License

Python 100.00%
eia eia-api python energy statistics

myeia's Introduction

myeia

PyPI version License: MIT Weekly Downloads Monthly Downloads Downloads

myeia is a simple Python wrapper for the U.S. Energy Information Administration (EIA) APIv2. It is designed to be simple to use and to provide a consistent interface for accessing EIA data.

Installation

pip install myeia

Requirements

  • backoff
  • pandas
  • python-dateutil
  • python-dotenv
  • requests

eia OPEN DATA Registration

To obtain an API Key you need to register on the EIA website.

eia API Query Browser

To find all EIA Datasets visit API Dashboard.

How to use

from myeia.api import API

eia = API()

Environment Variables

# Create your the .env file in your projects root directory
touch .env

By Default the EIA class will look for your API EIA_TOKEN.

If you have registered for an API key you can set it in your .env file.

EIA_TOKEN=YOUR_TOKEN_HERE

Get Series

Lets look at an example of how to get the EIA Natural Gas Futures. You can use the simpler v1 API method where you only need to pass the series_id or you can use the newer v2 API method where you need to pass the route, series, and frequency.

df = eia.get_series(series_id="NG.RNGC1.D")

df = eia.get_series_via_route(
    route="natural-gas/pri/fut",
    series="RNGC1",
    frequency="daily",
)

df.head()

Output Example:

            Natural Gas Futures Contract 1 (Dollars per Million Btu)
Date
2022-09-13                                              8.284
2022-09-12                                              8.249
2022-09-09                                              7.996
2022-09-08                                              7.915
2022-09-07                                              7.842
...                                                       ...

Different Facets

Lets look at another example the Total OPEC Petroleum Supply where the facet is available as seriesId. By Default it is set as series but we can define the facet as seriesId.

df = eia.get_series(series_id="STEO.PAPR_OPEC.M")

df = eia.get_series_via_route(
    route="steo",
    series="PAPR_OPEC",
    frequency="monthly",
    facet="seriesId",
)

df.head()

Output Example:

            Total OPEC Petroleum Supply
Date
2023-12-01                    34.517314
2023-11-01                    34.440397
2023-10-01                    34.376971
2023-09-01                    34.416242
2023-08-01                    34.451823
...                                 ...

Filter by multiple facets

You can also filter by multiple facets. Lets look at the UAE Crude oil, NGPL, and other liquids where the facets we choose are countryRegionId and productId. The difference here is that both facet columns are present in the dataframe, unlike the previous examples where only one facet was present.

df = eia.get_series_via_route(
    route="international",
    series=["ARE", 55],
    frequency="monthly",
    facet=["countryRegionId", "productId"],
)

df.head()

Output Example:

           countryRegionId productId  Crude oil, NGPL, and other liquids
Date                                                                  
2024-03-01             ARE        55                         4132.394334
2024-02-01             ARE        55                         4132.394334
2024-01-01             ARE        55                         4142.394334
2023-12-01             ARE        55                         4082.394334
2023-11-01             ARE        55                         4082.394334
...                    ...       ...                                 ...

Get Multiple Series

For multiple series you have to loop through the series and append the data to a list.

data = []
for item in ["RNGC1", "RNGC2"]:
    df = eia.get_series_via_route(
    route="natural-gas/pri/fut",
    series=item,
    frequency="daily",
    facet="series",
    )
    data.append(df)

df = pd.concat(data, axis=1)
df.head()

Output Example:

            Natural Gas Futures Contract 1 (Dollars per Million Btu)  Natural Gas Futures Contract 2 (Dollars per Million Btu)
Date                                                                                                                        
2023-08-29                                              2.556                                                     2.662     
2023-08-28                                              2.579                                                     2.665     
2023-08-25                                              2.540                                                     2.657     
2023-08-24                                              2.519                                                     2.636     
2023-08-23                                              2.497                                                     2.592     
...                                                       ...                                                       ...

Define a Start and End Date

You can define a start and end date for your query.

df = eia.get_series(
    series_id="NG.RNGC1.D",
    start_date="2021-01-01",
    end_date="2021-01-31",
)

df.head()

Output Example:

            Natural Gas Futures Contract 1 (Dollars per Million Btu)
Date                                                              
2021-01-29                                              2.564     
2021-01-28                                              2.664     
2021-01-27                                              2.760     
2021-01-26                                              2.656     
2021-01-25                                              2.602     
...                                                       ...     

This also works for the get_series_via_route method.

df = eia.get_series_via_route(
    route="natural-gas/pri/fut",
    series="RNGC1",
    frequency="daily",
    start_date="2021-01-01",
    end_date="2021-01-31",
)

df.head()

Output Example:

            Natural Gas Futures Contract 1 (Dollars per Million Btu)
Date
2021-01-29                                              2.564
2021-01-28                                              2.664
2021-01-27                                              2.760
2021-01-26                                              2.656
2021-01-25                                              2.602
...                                                       ...

myeia's People

Contributors

philsv avatar snyk-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

myeia's Issues

First example throws error

Hi, I was just trying the first example you have and I get an error:

HTTPError: 400 Client Error: Bad Request for url: https://api.eia.gov/v2/natural-gas/pri/fut/data/?api_key=c4wW96H5wHjJf0C9RKda6nseO5gjZV8VqHz8zdzB&data%5B%5D=value&frequency=daily&start=1963-09-23&end=2073-08-26&facets%5BNone%5D%5B%5D=RNGC1&sort%5B0%5D%5Bcolumn%5D=period&sort%5B0%5D%5Bdirection%5D=desc

{"error":"Invalid facet 'None' provided. The only valid facets are 'duoarea', 'product', 'process', and 'series'.","code":400}.

The second works fine. Just FYI. Cheers.

ModuleNotFoundError: No module named 'myeia'

Greetings,

I am trying to implement myeia into my scripts. Starting with a test script replicating your instructions, the error message is:
ModuleNotFoundError: No module named 'myeia'

When examining the location of the files, they are placed in:
/usr/local/lib/python3.11/site-packages/src

Changing /usr/local/lib/python3.11/site-packages/src to /usr/local/lib/python3.11/site-packages/myeia, the following error is returned:
from myeia import API
ImportError: cannot import name 'API' from 'myeia' (/usr/local/lib/python3.11/site-packages/myeia/init.py)

The file init.py is empty. It should contain something like
from .myeia import API, version

No?

Much thanks for your effort and attention.

Max

p.s. my operating environment is Fedora Linux 37; except for two python modules (entsog and entsoe) installed wity pip, all the rest are installed via dnf/rpm.

Feature Request - multiple series and start-time

Hi,

Thanks Phil for this great package. I was wondering if there was a way to have multiple series from the same route, and as an added bonus specify where the start time should begin. Right now I am using pd.merge_asof to merge columns together, but it would be extremely helpful to have it all done in one pull. Let me know if its possible with your package.

start_date

Have updated to the latest version, but now it seems start_date is not working properly..)
WLFC = eia.get_series(series_id="STEO.PATC_WORLD.M",start_date="2023-01-01")
WLFC.tail()
data is from 1990...

Thank you!

Version 0.2.3 errors

Encountering errors:
Running ./GetMyEIA.py
gets
Traceback (most recent call last):
File "/home/pyz/projects/Commodities/DataSources/EIA/Development/API/./GetMyEIA.py", line 5, in
from myeia import API
ImportError: cannot import name 'API' from 'myeia' (/usr/local/lib/python3.11/site-packages/myeia/init.py)

init.py modified to:
from .api import API, version

added two lines to api.py
title = "api.py"
version = "0.2.3"

But, I've put my EIA-API Token in an .env file, but the script still can't find it:
File "/usr/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.eia.gov/v2/seriesid/NG.RNGC1.D?api_key=None

Documentation Update

Hi Phil,

Looks like your repo is just about the only up-to-date Python wrapper for the EIA API after the March 2023 change which nullified most projects. I believe getting things by SeriesID is not longer working since v1 was sunset. See: mra1385/EIA-python#9

I could be wrong about this - but I think the docs need to be updated.

UnboundLocalError: local variable 'df' referenced before assignment

Hi,

I am getting the following error below for the code below. Could you please assist? I am trying to pull the wells drilled from the monthly-energy review

https://www.eia.gov/opendata/browser/total-energy?frequency=monthly&data=value;&facets=msn;&msn=PATWPUS;&sortColumn=period;&sortDirection=desc;

Error: UnboundLocalError: local variable 'df' referenced before assignment

list = ['PATWPUS']
df = eia.get_series_via_route(
route="total-energy",
facet='msn',
series="PATWPUS",
frequency="monthly"
)

df= df.apply(lambda x: x.astype(float) if x.dtype == 'object' else x)

df['Vintage']= vintage
df['Type']= "Crude"
crude = df.copy()
crude

Initial import not working

Hi,

I installed the latest myeia package and followed the initial prompts and getting an error. Please advise. Thanks!!

Here is my code:

from myeia.api import API

eia = api.API(token='*****')

Error:

ImportError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_31796\2078088430.py in
----> 1 from myeia.api import API
2
3 eia = api.API(token='*****')

~\Anaconda3\lib\site-packages\myeia\api.py in
7 import requests
8 from dotenv import load_dotenv
----> 9 from pandas.errors import SettingWithCopyWarning
10
11 warnings.simplefilter(action="ignore", category=SettingWithCopyWarning)

ImportError: cannot import name 'SettingWithCopyWarning' from 'pandas.errors' (C:\Users\mlr\Anaconda3\lib\site-packages\pandas\errors_init_.py)

How to support?

Hi Phil, this tool was very useful for me. Is there a way I can help support you/buy you a cup of coffee (or 2)?

Friendly implementation of API token imports

Hello, I just started using this API. I noticed that this strictly requires one to have a .env file and a token named EIA_TOKEN.
I was wondering if a more flexible and convenient way would be to also allow declaring the variables inside our script, which would be much more easier for use in notebooks.

I was thinking something like a parameter passed with the class API, eg: API(api_key = EIA_token)
Something like this would be great. I would love to work on this, I actually tried doing this but I don't exactly know how. If this is something you'd be interesting in adding, I would love some help to make it happen.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.