ecmwf-projects / mooc-machine-learning-weather-climate

MOOC Machine Learning in Weather & Climate - Jupyter notebook exercises

This repository hosts the Jupyter notebook based exercises of the Massive Open Online Course (MOOC) on Machine Learning in Weather & Climate https://www.ecmwf.int/mlwc-mooc.

The notebook files can be found in the subdirectories corresponding to each tier of the MOOC. These include the following:

Tier 1 notebooks (ML in Weather & Climate)

In this tier there is only one notebook that demonstrates how to build a simple neural network on the WeatherBench dataset.

Tier 2 notebooks (Concepts of Machine Learning)

In this tier there are notebooks for each module that provide practical guidance on key concepts of Machine Learning.

Tier 3 notebooks (Practical ML Applications in Weather & Climate)

Each module of this tier contains notebooks that demonstrate practical applications of Machine Learning in the various stages of Numerical Weather and Climate prediction.

How to run the notebooks

The notebooks can either be downloaded and run on participants' own computers, or they can be run directly in various cloud environments. The advantage of the latter is that no software needs to be installed locally. In each notebook a number of options are provided where the notebook can be run. These may include the following:

Colab	Kaggle	Deepnote
Colab requires a Google account, which can easily be set-up for free.	Requires (free) registration with Kaggle. Once in, "switch on the internet" via settings.	Requires (free) registration. Deepnote is a good platform also for collaboration.

License

Unless otherwise stated, the notebooks fall under Apache License 2.0. In applying this licence, ECMWF does not waive the privileges and immunities granted to it by virtue of its status as an intergovernmental organisation nor does it submit to any jurisdiction.

mooc-machine-learning-weather-climate's Issues

MOOC Tier 2, Module 1, Exercise 4

The default command for the exercise after creating and stalling the plugin is:

ds = cml.load_dataset('my-plugin', parameter = 'soil_temperature')
However the created function requires "year" as an argument. Also, when I attempt to use any year as in:

ds = cml.load_dataset('my-plugin', parameter = 'soil_temperature', year=2000)
I receive the following error:

ValueError: Invalid value 'soil_temperature', possible values are ['tp', 't2m'] (EnumSingleOrListType)

Then, if I try to change the parameter, for example to 't2m', I would get:

HTTPError: 404 Client Error: Not Found for url: https://storage.ecmwf.europeanweather.cloud/climetlab/test-data/0.5/fixtures/plugin_create_dataset_example_2000_t2m.grib

I've changed the 'year' entry and received the same error for any year between 1980-2010

Tier 3 Post-processing training. A bug?

Is the second line in

mooc-machine-learning-weather-climate/tier_3/post_processing/precipitation_downscaling_gan.ipynb

Lines 428 to 429 in 97d5190

	" combined_samples_hr = tf.concat([samples_gen, samples_hr], axis=0)\n",
	" combined_samples_lr = tf.concat([samples_lr, samples_lr], axis=0)\n",

right (samples_lr duplicated)?

Tier3 - E3CI requirements

I got an error while installing the default requirements and I thought it should be 'scikit-learn' instead of 'sklearn'.

tier_2 data handling

Hello,
I am working on tier_2/data_handling/04-dataset-plugin.ipynb, and noticed a possible typo in one of the text cell.
plugin_plugin_dataset to be changed to plugin_create_dateset or plugin_create_source?

Run from a shell terminal:
$ climetlab
(climetlab) plugin_plugin_dataset
Answer questions...

Tier 3 - Module Ocean & Climate, E3CI Python notebook

Hi,

I run the E3CI Python notebook of the Ocean & Climate module and got three HTTP 404 errors.

The URLs are incomplete. Everything works if the three URLs are changed from

wget https://raw.githubusercontent.com/ecmwf-projects/mooc-machine-learning-weather-climate/main/tier_3/ocean_climate/requirements.txt -O requirements.txt
wget https://github.com/ecmwf-projects/mooc-machine-learning-weather-climate/raw/main/tier_3/ocean_climate/data/e3ci_dataset_workshop.xlsx -O data/e3ci_dataset_workshop.xlsx
wget https://raw.githubusercontent.com/ecmwf-projects/mooc-machine-learning-weather-climate/main/tier_3/ocean_climate/data/europe.geojson -O data/europe.geojson

to

wget https://raw.githubusercontent.com/ecmwf-projects/mooc-machine-learning-weather-climate/main/tier_3/ocean_climate/e3ci/requirements.txt -O requirements.txt
wget https://github.com/ecmwf-projects/mooc-machine-learning-weather-climate/raw/main/tier_3/ocean_climate/e3ci/data/e3ci_dataset_workshop.xlsx -O data/e3ci_dataset_workshop.xlsx
wget https://raw.githubusercontent.com/ecmwf-projects/mooc-machine-learning-weather-climate/main/tier_3/ocean_climate/e3ci/data/europe.geojson -O data/europe.geojson

It is missing e3ci after /ocean_climate/.