ipl-uv / gauss4eo

Gaussianization for Earth Observation Data.

RBIG for Spatial-Temporal Exploration of Earth Science Data

This repo has the experiments and reproducible code for all of my experiments using RBIG for analyzing Earth science data. I am primarily focused on using RBIG to measure the information content for datasets with spatial-temporal features. I look at IT measures such as Entropy, Mutual Information and Total Correlation. The strength of RBIG lies in it's ability to handle multivariate and high dimensional datasets.

I will be periodically updating this repo as I finish more experiments. I will also make the code even more reproducible as I learn some of the best practices. For now, I would advise you to look at the notebooks.

---

Example Experiments

I have included some example experiments in the notebooks folder including the following experiments:

• Global Information Content (TODO)
• Spatial-Temporal Analysis of variables
• Temporal analysis of Drought Indicators
• Climate Model Comparisons

---

Installation Instructions

1. Firstly, you need to clone the following RBIG repo and install/put in PYTHONPATH

git clone https://github.com/jejjohnson/rbig

2. Secondly, you can create the environment from the .yml file found in the main repo.

conda env create -f environment.yml -n myenv
source activate myenv

---

Conferences

• Estimating Information in Earth Data Cubes - Johnson et. al. - EGU 2018
• Multivariate Gaussianization in Earth and Climate Sciences - Johnson et. al. - Climate Informatics 2019 - repo
• Climate Model Intercomparison with Multivariate Information Theoretic Measures - Johnson et. al. - AGU 2019 - slides

Journal Articles

• Iterative Gaussianization: from ICA to Random Rotations - Laparra et. al. (2011) - IEEE Transactions on Neural Networks (arxiv)
• Gaussianizing the Earth – Multidimensional Information Measures for Earth Data Analysis - Johnson et. al. (2020)- (Submitted)
• Information Theory Measures via Multidimensional Gaussianization - Laparra et. al. (2020)- (Submitted, arxiv)

---

External Toolboxes

RBIG (Rotation-Based Iterative Gaussianization)

This is a package I created to implement the RBIG algorithm. This is a multivariate Gaussianization method that allows one to calculate information theoretic measures such as entropy, total correlation and mutual information. More information can be found in the repository rbig.

Earth Science Data Cube Tools

These are a collection of useful scripts when dealing with datacubes (datasets in xarray Dataset format). I used a few preprocessing methods as well as a Minicuber implementation which transforms the data into spatial and/or temporal features. More information can be found in the repository esdc_tools.

---

Data Resources

Earth System Data Lab

This is sponsered by the earthsystemdatalab. They host a cube on their servers which include over 40+ variables including soil moisture and land surface temperature. They also feature a free-to-use JupyterHub server for easy exploration of th data.

SMADI Data

This is a database of SMADI index which is a drought indicator. I use this dataset. There are some additional instructions to install this package which requires registration and agreeing to some terms of use for each dataset.

Climate Data Store

This is a database of climate models implemented by the ECMWF and sponsored by the Copernicus program. I use a few climate models from here by using the CDSAPI. There are some additional instructions to install this package which requires registration and agreeing to some terms of use for each dataset.

---

Contact Information

Links to my co-authors' and my information:

• J. Emmanuel Johnson - Website | Scholar | Twitter | Github
• Maria Piles - Website | Scholar | Twitter
• Valero Laparra - Website | Scholar
• Gustau Camps-Valls - Website | Scholar | Twitter

Acknowledgements

This work was supported by the European Research Council (ERC) Synergy Grant “Understanding and Modelling the Earth System with Machine Learning (USMILE)” under Grant Agreement No 855187.