openclimatefix / pvitaly

Forecast for PV energy systems

PVItaly

Forecast for PV energy systems

Data downloading

Create a .env file with your UCAR login details:

UCAR_EMAIL='[email protected]'
UCAR_PASS='your-password'

Then:

sudo apt-get install libeccodes-dev libeccodes-tools
pip install -r requirements.txt -r requirements-dev.txt
pre-commit install

Then this will output individual zarrs for each timestamp and time-step into the provided output dir:

python scripts/download.py 2021-01-01 2021-02-01 zarrs/

These zarrs can then be merged:

python scripts/merge.py zarrs/ merged/

Running inference

Install requirements:

pip install -r requirements-ml.txt

Check the help on the infer script:

python infer.py --help

It will output a wide- and a long- DF to the specified output_dir.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Peter Dudfield 💻

Chris Arderne 💻

This project follows the all-contributors specification. Contributions of any kind welcome!

pvitaly's Issues

Train model

Detailed Description

Using ocf data pipes, train model to use

• PV
• NWP
• Sun features

Start with a simple model of 4 fully connection layers with ~ 256 nodes.
see for good example - https://github.com/openclimatefix/PV-Challenge

Context

• Pilot model needs to work and have enough things to add t in the future. Like more satellite + NWP

Downloaded NWP data for PVsites in Italy

Detailed Description

Get NWP ERA5 data for specific sites at Italy. Get forecasts from 0 to 12 hours.
Get the following variables

• TODO from https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-land?tab=form

Context

need NWP data for training ML model for sites in italy

Download PVoutput.org data from Italy

Detailed Description

Download data from Italien sites from pvoutput.org.
Good to start with data from 2020 to now.

Context

• Useful for training our model
• Metadata has already been downloaded by @jacobbieker (could you add where this has been downloaded)

Possible Implementation

Find out more information from @JackKelly how data has been previously downloaded

Run Inference

Detailed Description

Given a trained model run inference with the follow objectives

• Produce csv ready to hand over to client.
• run metrics on test set, understand the MAE

Context

Useufl to run on test set, so we are not running on training set

Possible Implementation

CSV shouwl ahve the following columns

• ocf_id
• target_time_utc, with timezone to be clear. The time the forecast is for.
• creation_time_utc, the time at which the forecast was made
• solar_generation_forecast_kw, the forecast value.
• solar_generation_actual_kw, the actual value.

[meta] Italy Pilot

• #3
• #4
• #5
• #6
• #7

Format Client data

Detailed Description

Reformat the client data into the following strucutre

pandas dataframe with the following structure

• datetime_utc: should include timezone
• id: make id of site. Might be good to make googlesheet / csv of all our PV sites with and ocf_id
• solar_generation_kw

and save as parquet file.

Context

• Good to have data in standard format
• good to have a format where other sites can easily be added

[meta] Italy Pilot

• Format Client data
• Download PVoutput.org Italy data
• Get NWP global data
• Train model
• Run inference