sylvainschmitt / getcordex

Get projection data from CORDEX

getCordex - Get projection data from CORDEX

Sylvain Schmitt April 18, 2023

• Data
• Installation
• Usage
  • Locally
  • HPC
• Workflow

singularity & snakemake workflow to download projection data from CORDEX.

Data

CORDEX: Coordinated Regional Climate Downscaling Experiment

The CORDEX vision is to advance and coordinate the science and application of regional climate downscaling through global partnerships.

Data are focused here on South-America (SAM), but WGET scripts can be added in the wget folder.

We particularly focused on domain SAM-22 with following variables: Near-Surface Air Temperature (tas) Near-Surface Relative Humidity (hurs), Near-Surface Wind Speed (sfcWind), Precipitation (pr), Surface Downwelling Shortwave Radiation (rsds), & Surface Upwelling Shortwave Radiation (rsus). We restricted outputs at a 3hr time-frequency and used the last available downscaling (v1). We used 3 types of experiments: one historical (historical) and two Representative Concentration Pathway RCP 2.6 & 8.5 (rcp26 & rcp85). We have 3 driving models available (MOHC-HadGEM2-ES, MPI-M-MPI-ESM-(LR or MR), and, NCC-NorESM1-M) coupled with 2 RCM model (REMO2015 & RegCM4-7).

Data are retrieve from the Institut Pierre-Simon Laplace node. You need first to create an account on this page (create account link at corner right).

Then you’ll need to register credentials locally to use the workflow. A help page is available here: https://esgf.github.io/esgf-user-support/user_guide.html?highlight=credentials%20pem#access-data-with-the-command-line-via-opendap. In linux, you need myproxy-logon installed and to run this command line (with your user name):

myproxy-logon -sesgf-node.ipsl.upmc.fr -l {user_name} -b -T -t 72 -o ~/.esg/credentials.pem

To run the workflow on a cluster, you can simply copy your local credentials to the cluster. For instance:

cp ~/.esg/credentials.pem /my_cluster/

Installation

• Python ≥3.5
• Snakemake ≥5.24.1
• Golang ≥1.15.2
• Singularity ≥3.7.3
• This workflow

# Python
sudo apt-get install python3.5
# Snakemake
sudo apt install snakemake`
# Golang
export VERSION=1.15.8 OS=linux ARCH=amd64  # change this as you need
wget -O /tmp/go${VERSION}.${OS}-${ARCH}.tar.gz https://dl.google.com/go/go${VERSION}.${OS}-${ARCH}.tar.gz && \
sudo tar -C /usr/local -xzf /tmp/go${VERSION}.${OS}-${ARCH}.tar.gz
echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
source ~/.bashrc
# Singularity
mkdir -p ${GOPATH}/src/github.com/sylabs && \
  cd ${GOPATH}/src/github.com/sylabs && \
  git clone https://github.com/sylabs/singularity.git && \
  cd singularity
git checkout v3.7.3
cd ${GOPATH}/src/github.com/sylabs/singularity && \
  ./mconfig && \
  cd ./builddir && \
  make && \
  sudo make install
# detect Mutations
git clone [email protected]:sylvainschmitt/getCordex.git
cd getCordex

Usage

Locally

snakemake -np -j 1 --resources mem_mb=10000 # dry run
snakemake --dag | dot -Tsvg > dag/dag.svg # dag
snakemake -j 1 --resources mem_mb=10000 # run

HPC

Muse

module load snakemake # for test on node
snakemake -np # dry run
sbatch job_muse.sh # run

Genologin

module load bioinfo/snakemake-5.25.0 # for test on node
snakemake -np # dry run
sbatch job_genologin.sh # run

Workflow

download

Download data from the Institut Pierre-Simon Laplace node using wget scripts from the wget/ folder.

extract_var

• Script: extract_var.R

Extract variables from the netcdf files in parallel into one tsv table per experiment and variable.

join_exp

• Script: join_exp.R

Join variables from the same experiment into one single tsv table.

extrapolate

• Script: extrapolate.R

Extrapolate variables at defined time frequency (config: extrapolation_time_freq, defined at 0.5hr for TROLL) with splines.

reformat

• Script: reformat.R

Reformat experiment variables into variables used in the TROLL model:

• rainfall (rainfall in mm) = 60 * 60 * time_freq_hr * pr (Precipitation in kg/m/s)
• snet (absorbed short-wave radiation in w/m2) = rsds (Surface Downwelling Shortwave Radiation in W/m2) - rsus (Surface Upwelling Shortwave Radiation in W/m2)
• temperature (temperature in degrees C) = tas (Near-Surface Air Temperature in Kelvin) - 273.15
• vpd (vapour pressure deficit in Pa) = rh_to_vpd[ hurs(Near-Surface Relative Humidity in %), temperature ]
• ws (wind speed in m/s) = sfcWind (Near-Surface Wind Speed in m/s)

plot

• Script: plot.R

Plot TROLL variables into hourly, daily, monthly and yearly variation for each variable and experiment.