Coder Social home page Coder Social logo

climateimpactlab / downscalecmip6 Goto Github PK

View Code? Open in Web Editor NEW
121.0 10.0 28.0 86.88 MB

Downscaling & bias correction of CMIP6 tasmin, tasmax, and pr for the R/CIL GDPCIR project

License: MIT License

Jupyter Notebook 99.80% Python 0.14% HCL 0.05%
cmip6 climate-impacts climate-data downscaling downscale

downscalecmip6's Introduction

Global Downscaled Projections for Climate Impacts Research

image

image

The World Climate Research Programme's 6th Coupled Model Intercomparison Project (CMIP6) represents an enormous advance in the quality, detail, and scope of climate modeling.

The Global Downscaled Projections for Climate Impacts Research dataset makes this modeling more applicable to understanding the impacts of changes in the climate on humans and society with two key developments: trend-preserving bias correction and downscaling. In this dataset, the Climate Impact Lab provides global, daily minimum and maximum air temperature at the surface (tasmin and tasmax) and daily cumulative surface precipitation (pr) corresponding to the CMIP6 historical, ssp1-2.6, ssp2-4.5, ssp3-7.0, and ssp5-8.5 scenarios for 25 global climate models on a 1/4-degree regular global grid.

Contents:

Additional links:

Accessing the data

GDPCIR data can be accessed on the Microsoft Planetary Computer: planetarycomputer.microsoft.com/dataset/group/cil-gdpcir

The dataset is made of collections distinguished by data license at the time of publication:

Each modeling center with bias corrected and downscaled data in this collection falls into one of these license categories - see the table below to see which model is in each collection, and see the section below on Citing, Licensing, and using data produced by this project for citations and additional information about each license. For examples of how to browse the collections and load the data using python, see the example use section below.

Data format & contents

The data is stored as partitioned zarr stores (see https://zarr.readthedocs.io), each of which includes thousands of data and metadata files covering the full time span of the experiment. Historical zarr stores contain just over 50 GB, while SSP zarr stores contain nearly 70GB. Each store is stored as a 32-bit float, with dimensions time (daily datetime), lat (float latitude), and lon (float longitude). The data is chunked at each interval of 365 days and 90 degree interval of latitude and longitude. Therefore, each chunk is (365, 360, 360), with each chunk occupying approximately 180MB in memory.

Historical data is daily, excluding leap days, from Jan 1, 1950 to Dec 31, 2014; SSP data is daily, excluding leap days, from Jan 1, 2015 to either Dec 31, 2099 or Dec 31, 2100, depending on data availability in the source GCM.

The spatial domain covers all 0.25-degree grid cells, indexed by the grid center, with grid edges on the quarter-degree, using a -180 to 180 longitude convention. Thus, the “lon” coordinate extends from -179.875 to 179.875, and the “lat” coordinate extends from -89.875 to 89.875, with intermediate values at each 0.25-degree increment between (e.g. -179.875, -179.625, -179.375, etc).

Available institutions, models, and scenarios by license collection

Modeling institution Source model Available experiments License collection
CAS FGOALS-g31 SSP2-4.5, SSP3-7.0, and SSP5-8.5 Public domain datasets
INM INM-CM4-8 SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 Public domain datasets
INM INM-CM5-0 SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 Public domain datasets
BCC BCC-CSM2-MR SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0
CMCC CMCC-CM2-SR5 ssp1-2.6, ssp2-4.5, ssp3-7.0, ssp5-8.5 CC-BY-4.0
CMCC CMCC-ESM2 ssp1-2.6, ssp2-4.5, ssp3-7.0, ssp5-8.5 CC-BY-4.0
CSIRO-ARCCSS ACCESS-CM2 SSP2-4.5 and SSP3-7.0 CC-BY-4.0
CSIRO ACCESS-ESM1-5 SSP1-2.6, SSP2-4.5, and SSP3-7.0 CC-BY-4.0
MIROC MIROC-ES2L SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0
MIROC MIROC6 SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0
MOHC HadGEM3-GC31-LL SSP1-2.6, SSP2-4.5, and SSP5-8.5 CC-BY-4.0
MOHC UKESM1-0-LL SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0
MPI-M MPI-ESM1-2-LR SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0
MPI-M/DKRZ2 MPI-ESM1-2-HR SSP1-2.6 and SSP5-8.5 CC-BY-4.0
NCC NorESM2-LM SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0
NCC NorESM2-MM SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0
NOAA-GFDL GFDL-CM4 SSP2-4.5 and SSP5-8.5 CC-BY-4.0
NOAA-GFDL GFDL-ESM4 SSP1-2.6, SSP2-4.5, SSP3-7.0, and SSP5-8.5 CC-BY-4.0
NUIST NESM3 SSP1-2.6, SSP2-4.5, and SSP5-8.5 CC-BY-4.0
EC-Earth-Consortium EC-Earth3 ssp1-2.6, ssp2-4.5, ssp3-7.0, and ssp5-8.5 CC-BY-4.0
EC-Earth-Consortium EC-Earth3-AerChem ssp370 CC-BY-4.0
EC-Earth-Consortium EC-Earth3-CC ssp245 and ssp585 CC-BY-4.0
EC-Earth-Consortium EC-Earth3-Veg ssp1-2.6, ssp2-4.5, ssp3-7.0, and ssp5-8.5 CC-BY-4.0
EC-Earth-Consortium EC-Earth3-Veg-LR ssp1-2.6, ssp2-4.5, ssp3-7.0, and ssp5-8.5 CC-BY-4.0
CCCma CanESM5 ssp1-2.6, ssp2-4.5, ssp3-7.0, ssp5-8.5 CC-BY-4.0

Notes:

Example Use

See the following examples on github: github.com/microsoft/PlanetaryComputerExamples

You can try these out in a live example on Binder:

image

Project methods

This project makes use of statistical bias correction and downscaling algorithms, which are specifically designed to accurately represent changes in the extremes. For this reason, we selected Quantile Delta Mapping (QDM), following the method introduced by Cannon et al. (2015), which preserves quantile-specific trends from the GCM while fitting the full distribution for a given day-of-year to a reference dataset (ERA5).

We then introduce a similar method tailored to increase spatial resolution while preserving extreme behavior, Quantile-Preserving Localized-Analog Downscaling (QPLAD).

Together, these methods provide a robust means to handle both the central and tail behavior seen in climate model output, while aligning the full distribution to a state-of-the-art reanalysis dataset and providing the spatial granularity needed to study surface impacts.

A publication providing additional detail is in review for publication in Geoscientific Model Development and a pre-print can be accessed in EGUsphere: https://egusphere.copernicus.org/preprints/2023/egusphere-2022-1513/.

The downscaleCMIP6 Repository

The ClimateImpactLab/downscaleCMIP6 repository contains infrastructure setup, argo workflows, and validation notebooks which together produce the bias corrected and downscaled daily 1/4-degree CMIP6 tasmin, tasmax, and pr data for the Climate Impact Lab Global Downscaled Projections for Climate Impacts Research (CIL GDPCIR) project.

See also:

Citing, licensing, and using data produced by this project

Projects making use of the data produced as part of the Climate Impact Lab Global Downscaled Projections for Climate Impacts Research (CIL GDPCIR) project are requested to cite both this project and the source datasets from which these results are derived. Additionally, the use of data derived from some GCMs requires citations. See each GCM's license info in the links below for more information.

CIL GDPCIR

Users are requested to cite this project in derived works.

Gergel, D. R., Malevich, S. B., McCusker, K. E., Tenezakis, E., Delgado, M. T., Fish, M. A., and Kopp, R. E.: Global Downscaled Projections for Climate Impacts Research (GDPCIR): preserving quantile trends for modeling future climate impacts, Geosci. Model Dev., 17, 191–227, https://doi.org/10.5194/gmd-17-191-2024, 2024.

ERA5

Additionally, we request you cite the historical dataset used in bias correction and downscaling, ERA5. See the ECMWF guide to citing a dataset on the Climate Data Store:

Hersbach, H, et al. The ERA5 global reanalysis. Q J R Meteorol Soc.2020; 146: 1999–2049. https://doi.org/10.1002/qj.3803

Muñoz Sabater, J., (2019): ERA5-Land hourly data from 1981 to present. Copernicus Climate Change Service (C3S) Climate Data Store (CDS). (Accessed on June 4, 2021), 10.24381/cds.e2161bac

Muñoz Sabater, J., (2021): ERA5-Land hourly data from 1950 to 1980. Copernicus Climate Change Service (C3S) Climate Data Store (CDS). (Accessed on June 4, 2021), 10.24381/cds.e2161bac

GCM-specific citations & licenses

The CMIP6 simulation data made available through the Earth System Grid Federation (ESGF) are subject to Creative Commons BY-SA 4.0 or BY-NC-SA 4.0 licenses. We have reached out to each of the modeling institutions to request waivers from these terms so the outputs of this project may be used with fewer restrictions, and have been granted permission to release our data using the licenses listed here.

Public Domain Datasets

The following bias corrected and downscaled model simulations are available in the public domain using a CC0 1.0 Universal Public Domain Declaration. Access the collection on Planetary Computer at https://planetarycomputer.microsoft.com/dataset/cil-gdpcir-cc0.

  • FGOALS-g3

    License description: data_licenses/FGOALS-g3.txt

    CMIP Citation:

    Li, Lijuan (2019). CAS FGOALS-g3 model output prepared for CMIP6 CMIP. Version 20190826. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1783

    ScenarioMIP Citation:

    Li, Lijuan (2019). CAS FGOALS-g3 model output prepared for CMIP6 ScenarioMIP. SSP1-2.6 version 20190818; SSP2-4.5 version 20190818; SSP3-7.0 version 20190820; SSP5-8.5 tasmax version 20190819; SSP5-8.5 tasmin version 20190819; SSP5-8.5 pr version 20190818. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.2056

  • INM-CM4-8

    License description: data_licenses/INM-CM4-8.txt

    CMIP Citation:

    Volodin, Evgeny; Mortikov, Evgeny; Gritsun, Andrey; Lykossov, Vasily; Galin, Vener; Diansky, Nikolay; Gusev, Anatoly; Kostrykin, Sergey; Iakovlev, Nikolay; Shestakova, Anna; Emelina, Svetlana (2019). INM INM-CM4-8 model output prepared for CMIP6 CMIP. Version 20190530. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1422

    ScenarioMIP Citation:

    Volodin, Evgeny; Mortikov, Evgeny; Gritsun, Andrey; Lykossov, Vasily; Galin, Vener; Diansky, Nikolay; Gusev, Anatoly; Kostrykin, Sergey; Iakovlev, Nikolay; Shestakova, Anna; Emelina, Svetlana (2019). INM INM-CM4-8 model output prepared for CMIP6 ScenarioMIP. Version 20190603. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.12321

  • INM-CM5-0

    License description: data_licenses/INM-CM5-0.txt

    CMIP Citation:

    Volodin, Evgeny; Mortikov, Evgeny; Gritsun, Andrey; Lykossov, Vasily; Galin, Vener; Diansky, Nikolay; Gusev, Anatoly; Kostrykin, Sergey; Iakovlev, Nikolay; Shestakova, Anna; Emelina, Svetlana (2019). INM INM-CM5-0 model output prepared for CMIP6 CMIP. Version 20190610. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1423

    ScenarioMIP Citation:

    Volodin, Evgeny; Mortikov, Evgeny; Gritsun, Andrey; Lykossov, Vasily; Galin, Vener; Diansky, Nikolay; Gusev, Anatoly; Kostrykin, Sergey; Iakovlev, Nikolay; Shestakova, Anna; Emelina, Svetlana (2019). INM INM-CM5-0 model output prepared for CMIP6 ScenarioMIP. SSP1-2.6 version 20190619; SSP2-4.5 version 20190619; SSP3-7.0 version 20190618; SSP5-8.5 version 20190724. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.12322

CC-BY-4.0

The following bias corrected and downscaled model simulations are licensed under a Creative Commons Attribution 4.0 International License. Note that this license requires citation of the source model output (included here). Please see https://creativecommons.org/licenses/by/4.0/ for more information. Access the collection on Planetary Computer at https://planetarycomputer.microsoft.com/dataset/cil-gdpcir-cc-by.

  • ACCESS-CM2

    License description: data_licenses/ACCESS-CM2.txt

    CMIP Citation:

    Dix, Martin; Bi, Doahua; Dobrohotoff, Peter; Fiedler, Russell; Harman, Ian; Law, Rachel; Mackallah, Chloe; Marsland, Simon; O'Farrell, Siobhan; Rashid, Harun; Srbinovsky, Jhan; Sullivan, Arnold; Trenham, Claire; Vohralik, Peter; Watterson, Ian; Williams, Gareth; Woodhouse, Matthew; Bodman, Roger; Dias, Fabio Boeira; Domingues, Catia; Hannah, Nicholas; Heerdegen, Aidan; Savita, Abhishek; Wales, Scott; Allen, Chris; Druken, Kelsey; Evans, Ben; Richards, Clare; Ridzwan, Syazwan Mohamed; Roberts, Dale; Smillie, Jon; Snow, Kate; Ward, Marshall; Yang, Rui (2019). CSIRO-ARCCSS ACCESS-CM2 model output prepared for CMIP6 CMIP. Version 20191108. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.2281

    ScenarioMIP Citation:

    Dix, Martin; Bi, Doahua; Dobrohotoff, Peter; Fiedler, Russell; Harman, Ian; Law, Rachel; Mackallah, Chloe; Marsland, Simon; O'Farrell, Siobhan; Rashid, Harun; Srbinovsky, Jhan; Sullivan, Arnold; Trenham, Claire; Vohralik, Peter; Watterson, Ian; Williams, Gareth; Woodhouse, Matthew; Bodman, Roger; Dias, Fabio Boeira; Domingues, Catia; Hannah, Nicholas; Heerdegen, Aidan; Savita, Abhishek; Wales, Scott; Allen, Chris; Druken, Kelsey; Evans, Ben; Richards, Clare; Ridzwan, Syazwan Mohamed; Roberts, Dale; Smillie, Jon; Snow, Kate; Ward, Marshall; Yang, Rui (2019). CSIRO-ARCCSS ACCESS-CM2 model output prepared for CMIP6 ScenarioMIP. Version 20191108. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.2285

  • ACCESS-ESM1-5

    License description: data_licenses/ACCESS-ESM1-5.txt

    CMIP Citation:

    Ziehn, Tilo; Chamberlain, Matthew; Lenton, Andrew; Law, Rachel; Bodman, Roger; Dix, Martin; Wang, Yingping; Dobrohotoff, Peter; Srbinovsky, Jhan; Stevens, Lauren; Vohralik, Peter; Mackallah, Chloe; Sullivan, Arnold; O'Farrell, Siobhan; Druken, Kelsey (2019). CSIRO ACCESS-ESM1.5 model output prepared for CMIP6 CMIP. Version 20191115. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.2288

    ScenarioMIP Citation:

    Ziehn, Tilo; Chamberlain, Matthew; Lenton, Andrew; Law, Rachel; Bodman, Roger; Dix, Martin; Wang, Yingping; Dobrohotoff, Peter; Srbinovsky, Jhan; Stevens, Lauren; Vohralik, Peter; Mackallah, Chloe; Sullivan, Arnold; O'Farrell, Siobhan; Druken, Kelsey (2019). CSIRO ACCESS-ESM1.5 model output prepared for CMIP6 ScenarioMIP. Version 20191115. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.2291

  • BCC-CSM2-MR

    License description: data_licenses/BCC-CSM2-MR.txt

    CMIP Citation:

    Xin, Xiaoge; Zhang, Jie; Zhang, Fang; Wu, Tongwen; Shi, Xueli; Li, Jianglong; Chu, Min; Liu, Qianxia; Yan, Jinghui; Ma, Qiang; Wei, Min (2018). BCC BCC-CSM2MR model output prepared for CMIP6 CMIP. Version 20181126. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1725

    ScenarioMIP Citation:

    Xin, Xiaoge; Wu, Tongwen; Shi, Xueli; Zhang, Fang; Li, Jianglong; Chu, Min; Liu, Qianxia; Yan, Jinghui; Ma, Qiang; Wei, Min (2019). BCC BCC-CSM2MR model output prepared for CMIP6 ScenarioMIP. SSP1-2.6 version 20190315; SSP2-4.5 version 20190318; SSP3-7.0 version 20190318; SSP5-8.5 version 20190318. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1732

  • CanESM5

    License description: data_licenses/CanESM5.txt

    CMIP Citation:

    Swart, Neil Cameron; Cole, Jason N.S.; Kharin, Viatcheslav V.; Lazare, Mike; Scinocca, John F.; Gillett, Nathan P.; Anstey, James; Arora, Vivek; Christian, James R.; Jiao, Yanjun; Lee, Warren G.; Majaess, Fouad; Saenko, Oleg A.; Seiler, Christian; Seinen, Clint; Shao, Andrew; Solheim, Larry; von Salzen, Knut; Yang, Duo; Winter, Barbara; Sigmond, Michael (2019). CCCma CanESM5 model output prepared for CMIP6 CMIP. Version 20190429. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1303

    ScenarioMIP Citation:

    Swart, Neil Cameron; Cole, Jason N.S.; Kharin, Viatcheslav V.; Lazare, Mike; Scinocca, John F.; Gillett, Nathan P.; Anstey, James; Arora, Vivek; Christian, James R.; Jiao, Yanjun; Lee, Warren G.; Majaess, Fouad; Saenko, Oleg A.; Seiler, Christian; Seinen, Clint; Shao, Andrew; Solheim, Larry; von Salzen, Knut; Yang, Duo; Winter, Barbara; Sigmond, Michael (2019). CCCma CanESM5 model output prepared for CMIP6 ScenarioMIP. Version 20190429. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1317

  • CMCC-CM2-SR5

    License description: data_licenses/CMCC-CM2-SR5.txt

    CMIP Citation:

    Lovato, Tomas; Peano, Daniele (2020). CMCC CMCC-CM2-SR5 model output prepared for CMIP6 CMIP. Version 20200616. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1362

    ScenarioMIP Citation:

    Lovato, Tomas; Peano, Daniele (2020). CMCC CMCC-CM2-SR5 model output prepared for CMIP6 ScenarioMIP. SSP1-2.6 version 20200717; SSP2-4.5 version 20200617; SSP3-7.0 version 20200622; SSP5-8.5 version 20200622. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1365

  • CMCC-ESM2

    License description: data_licenses/CMCC-ESM2.txt

    CMIP Citation:

    Lovato, Tomas; Peano, Daniele; Butenschön, Momme (2021). CMCC CMCC-ESM2 model output prepared for CMIP6 CMIP. Version 20210114. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.13164

    ScenarioMIP Citation:

    Lovato, Tomas; Peano, Daniele; Butenschön, Momme (2021). CMCC CMCC-ESM2 model output prepared for CMIP6 ScenarioMIP. SSP1-2.6 version 20210126; SSP2-4.5 version 20210129; SSP3-7.0 version 20210202; SSP5-8.5 version 20210126. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.13168

  • EC-Earth3-AerChem

    License description: data_licenses/EC-Earth3-AerChem.txt

    CMIP Citation:

    EC-Earth Consortium (EC-Earth) (2020). EC-Earth-Consortium EC-Earth3-AerChem model output prepared for CMIP6 CMIP. Version 20200624. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.639

    ScenarioMIP Citation:

    EC-Earth Consortium (EC-Earth) (2020). EC-Earth-Consortium EC-Earth3-AerChem model output prepared for CMIP6 ScenarioMIP. Version 20200827. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.724

  • EC-Earth3-CC

    License description: data_licenses/EC-Earth3-CC.txt

    CMIP Citation:

    EC-Earth Consortium (EC-Earth) (2020). EC-Earth-Consortium EC-Earth-3-CC model output prepared for CMIP6 CMIP. Version 20210113. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.640

    ScenarioMIP Citation:

    EC-Earth Consortium (EC-Earth) (2021). EC-Earth-Consortium EC-Earth3-CC model output prepared for CMIP6 ScenarioMIP. Version 20210113. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.15327

  • EC-Earth3-Veg-LR

    License description: data_licenses/EC-Earth3-Veg-LR.txt

    CMIP Citation:

    EC-Earth Consortium (EC-Earth) (2020). EC-Earth-Consortium EC-Earth3-Veg-LR model output prepared for CMIP6 CMIP. Version 20200217. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.643

    ScenarioMIP Citation:

    EC-Earth Consortium (EC-Earth) (2020). EC-Earth-Consortium EC-Earth3-Veg-LR model output prepared for CMIP6 ScenarioMIP. SSP1-2.6 version 20201201; SSP2-4.5 version 20201123; SSP3-7.0 version 20201123; SSP5-8.5 version 20201201. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.728

  • EC-Earth3-Veg

    License description: data_licenses/EC-Earth3-Veg.txt

    CMIP Citation:

    EC-Earth Consortium (EC-Earth) (2019). EC-Earth-Consortium EC-Earth3-Veg model output prepared for CMIP6 CMIP. Version 20200225. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.642

    ScenarioMIP Citation:

    EC-Earth Consortium (EC-Earth) (2019). EC-Earth-Consortium EC-Earth3-Veg model output prepared for CMIP6 ScenarioMIP. Version 20200225. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.727

  • EC-Earth3

    License description: data_licenses/EC-Earth3.txt

    CMIP Citation:

    EC-Earth Consortium (EC-Earth) (2019). EC-Earth-Consortium EC-Earth3 model output prepared for CMIP6 CMIP. Version 20200310. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.181

    ScenarioMIP Citation:

    EC-Earth Consortium (EC-Earth) (2019). EC-Earth-Consortium EC-Earth3 model output prepared for CMIP6 ScenarioMIP. Version 20200310. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.251

  • GFDL-CM4

    License description: data_licenses/GFDL-CM4.txt

    CMIP Citation:

    Guo, Huan; John, Jasmin G; Blanton, Chris; McHugh, Colleen; Nikonov, Serguei; Radhakrishnan, Aparna; Rand, Kristopher; Zadeh, Niki T.; Balaji, V; Durachta, Jeff; Dupuis, Christopher; Menzel, Raymond; Robinson, Thomas; Underwood, Seth; Vahlenkamp, Hans; Bushuk, Mitchell; Dunne, Krista A.; Dussin, Raphael; Gauthier, Paul PG; Ginoux, Paul; Griffies, Stephen M.; Hallberg, Robert; Harrison, Matthew; Hurlin, William; Lin, Pu; Malyshev, Sergey; Naik, Vaishali; Paulot, Fabien; Paynter, David J; Ploshay, Jeffrey; Reichl, Brandon G; Schwarzkopf, Daniel M; Seman, Charles J; Shao, Andrew; Silvers, Levi; Wyman, Bruce; Yan, Xiaoqin; Zeng, Yujin; Adcroft, Alistair; Dunne, John P.; Held, Isaac M; Krasting, John P.; Horowitz, Larry W.; Milly, P.C.D; Shevliakova, Elena; Winton, Michael; Zhao, Ming; Zhang, Rong (2018). NOAA-GFDL GFDL-CM4 model output. Version 20180701. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1402

    ScenarioMIP Citation:

    Guo, Huan; John, Jasmin G; Blanton, Chris; McHugh, Colleen; Nikonov, Serguei; Radhakrishnan, Aparna; Rand, Kristopher; Zadeh, Niki T.; Balaji, V; Durachta, Jeff; Dupuis, Christopher; Menzel, Raymond; Robinson, Thomas; Underwood, Seth; Vahlenkamp, Hans; Dunne, Krista A.; Gauthier, Paul PG; Ginoux, Paul; Griffies, Stephen M.; Hallberg, Robert; Harrison, Matthew; Hurlin, William; Lin, Pu; Malyshev, Sergey; Naik, Vaishali; Paulot, Fabien; Paynter, David J; Ploshay, Jeffrey; Schwarzkopf, Daniel M; Seman, Charles J; Shao, Andrew; Silvers, Levi; Wyman, Bruce; Yan, Xiaoqin; Zeng, Yujin; Adcroft, Alistair; Dunne, John P.; Held, Isaac M; Krasting, John P.; Horowitz, Larry W.; Milly, Chris; Shevliakova, Elena; Winton, Michael; Zhao, Ming; Zhang, Rong (2018). NOAA-GFDL GFDL-CM4 model output prepared for CMIP6 ScenarioMIP. Version 20180701. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.9242

  • GFDL-ESM4

    License description: data_licenses/GFDL-ESM4.txt

    CMIP Citation:

    Krasting, John P.; John, Jasmin G; Blanton, Chris; McHugh, Colleen; Nikonov, Serguei; Radhakrishnan, Aparna; Rand, Kristopher; Zadeh, Niki T.; Balaji, V; Durachta, Jeff; Dupuis, Christopher; Menzel, Raymond; Robinson, Thomas; Underwood, Seth; Vahlenkamp, Hans; Dunne, Krista A.; Gauthier, Paul PG; Ginoux, Paul; Griffies, Stephen M.; Hallberg, Robert; Harrison, Matthew; Hurlin, William; Malyshev, Sergey; Naik, Vaishali; Paulot, Fabien; Paynter, David J; Ploshay, Jeffrey; Reichl, Brandon G; Schwarzkopf, Daniel M; Seman, Charles J; Silvers, Levi; Wyman, Bruce; Zeng, Yujin; Adcroft, Alistair; Dunne, John P.; Dussin, Raphael; Guo, Huan; He, Jian; Held, Isaac M; Horowitz, Larry W.; Lin, Pu; Milly, P.C.D; Shevliakova, Elena; Stock, Charles; Winton, Michael; Wittenberg, Andrew T.; Xie, Yuanyu; Zhao, Ming (2018). NOAA-GFDL GFDL-ESM4 model output prepared for CMIP6 CMIP. Version 20190726. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1407

    ScenarioMIP Citation:

    John, Jasmin G; Blanton, Chris; McHugh, Colleen; Radhakrishnan, Aparna; Rand, Kristopher; Vahlenkamp, Hans; Wilson, Chandin; Zadeh, Niki T.; Dunne, John P.; Dussin, Raphael; Horowitz, Larry W.; Krasting, John P.; Lin, Pu; Malyshev, Sergey; Naik, Vaishali; Ploshay, Jeffrey; Shevliakova, Elena; Silvers, Levi; Stock, Charles; Winton, Michael; Zeng, Yujin (2018). NOAA-GFDL GFDL-ESM4 model output prepared for CMIP6 ScenarioMIP. Version 20180701. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1414

  • HadGEM3-GC31-LL

    License description: data_licenses/HadGEM3-GC31-LL.txt

    CMIP Citation:

    Ridley, Jeff; Menary, Matthew; Kuhlbrodt, Till; Andrews, Martin; Andrews, Tim (2018). MOHC HadGEM3-GC31-LL model output prepared for CMIP6 CMIP. Version 20190624. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.419

    ScenarioMIP Citation:

    Good, Peter (2019). MOHC HadGEM3-GC31-LL model output prepared for CMIP6 ScenarioMIP. SSP1-2.6 version 20200114; SSP2-4.5 version 20190908; SSP5-8.5 version 20200114. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.10845

  • MIROC-ES2L

    License description: data_licenses/MIROC-ES2L.txt

    CMIP Citation:

    Hajima, Tomohiro; Abe, Manabu; Arakawa, Osamu; Suzuki, Tatsuo; Komuro, Yoshiki; Ogura, Tomoo; Ogochi, Koji; Watanabe, Michio; Yamamoto, Akitomo; Tatebe, Hiroaki; Noguchi, Maki A.; Ohgaito, Rumi; Ito, Akinori; Yamazaki, Dai; Ito, Akihiko; Takata, Kumiko; Watanabe, Shingo; Kawamiya, Michio; Tachiiri, Kaoru (2019). MIROC MIROC-ES2L model output prepared for CMIP6 CMIP. Version 20191129. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.902

    ScenarioMIP Citation:

    Tachiiri, Kaoru; Abe, Manabu; Hajima, Tomohiro; Arakawa, Osamu; Suzuki, Tatsuo; Komuro, Yoshiki; Ogochi, Koji; Watanabe, Michio; Yamamoto, Akitomo; Tatebe, Hiroaki; Noguchi, Maki A.; Ohgaito, Rumi; Ito, Akinori; Yamazaki, Dai; Ito, Akihiko; Takata, Kumiko; Watanabe, Shingo; Kawamiya, Michio (2019). MIROC MIROC-ES2L model output prepared for CMIP6 ScenarioMIP. Version 20200318. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.936

  • MIROC6

    License description: data_licenses/MIROC6.txt

    CMIP Citation:

    Tatebe, Hiroaki; Watanabe, Masahiro (2018). MIROC MIROC6 model output prepared for CMIP6 CMIP. Version 20191016. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.881

    ScenarioMIP Citation:

    Shiogama, Hideo; Abe, Manabu; Tatebe, Hiroaki (2019). MIROC MIROC6 model output prepared for CMIP6 ScenarioMIP. Version 20191016. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.898

  • MPI-ESM1-2-HR

    License description: data_licenses/MPI-ESM1-2-HR.txt

    CMIP Citation:

    Jungclaus, Johann; Bittner, Matthias; Wieners, Karl-Hermann; Wachsmann, Fabian; Schupfner, Martin; Legutke, Stephanie; Giorgetta, Marco; Reick, Christian; Gayler, Veronika; Haak, Helmuth; de Vrese, Philipp; Raddatz, Thomas; Esch, Monika; Mauritsen, Thorsten; von Storch, Jin-Song; Behrens, Jörg; Brovkin, Victor; Claussen, Martin; Crueger, Traute; Fast, Irina; Fiedler, Stephanie; Hagemann, Stefan; Hohenegger, Cathy; Jahns, Thomas; Kloster, Silvia; Kinne, Stefan; Lasslop, Gitta; Kornblueh, Luis; Marotzke, Jochem; Matei, Daniela; Meraner, Katharina; Mikolajewicz, Uwe; Modali, Kameswarrao; Müller, Wolfgang; Nabel, Julia; Notz, Dirk; Peters-von Gehlen, Karsten; Pincus, Robert; Pohlmann, Holger; Pongratz, Julia; Rast, Sebastian; Schmidt, Hauke; Schnur, Reiner; Schulzweida, Uwe; Six, Katharina; Stevens, Bjorn; Voigt, Aiko; Roeckner, Erich (2019). MPI-M MPIESM1.2-HR model output prepared for CMIP6 CMIP. Version 20190710. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.741

    ScenarioMIP Citation:

    Schupfner, Martin; Wieners, Karl-Hermann; Wachsmann, Fabian; Steger, Christian; Bittner, Matthias; Jungclaus, Johann; Früh, Barbara; Pankatz, Klaus; Giorgetta, Marco; Reick, Christian; Legutke, Stephanie; Esch, Monika; Gayler, Veronika; Haak, Helmuth; de Vrese, Philipp; Raddatz, Thomas; Mauritsen, Thorsten; von Storch, Jin-Song; Behrens, Jörg; Brovkin, Victor; Claussen, Martin; Crueger, Traute; Fast, Irina; Fiedler, Stephanie; Hagemann, Stefan; Hohenegger, Cathy; Jahns, Thomas; Kloster, Silvia; Kinne, Stefan; Lasslop, Gitta; Kornblueh, Luis; Marotzke, Jochem; Matei, Daniela; Meraner, Katharina; Mikolajewicz, Uwe; Modali, Kameswarrao; Müller, Wolfgang; Nabel, Julia; Notz, Dirk; Peters-von Gehlen, Karsten; Pincus, Robert; Pohlmann, Holger; Pongratz, Julia; Rast, Sebastian; Schmidt, Hauke; Schnur, Reiner; Schulzweida, Uwe; Six, Katharina; Stevens, Bjorn; Voigt, Aiko; Roeckner, Erich (2019). DKRZ MPI-ESM1.2-HR model output prepared for CMIP6 ScenarioMIP. Version 20190710. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.2450

  • MPI-ESM1-2-LR

    License description: data_licenses/MPI-ESM1-2-LR.txt

    CMIP Citation:

    Wieners, Karl-Hermann; Giorgetta, Marco; Jungclaus, Johann; Reick, Christian; Esch, Monika; Bittner, Matthias; Legutke, Stephanie; Schupfner, Martin; Wachsmann, Fabian; Gayler, Veronika; Haak, Helmuth; de Vrese, Philipp; Raddatz, Thomas; Mauritsen, Thorsten; von Storch, Jin-Song; Behrens, Jörg; Brovkin, Victor; Claussen, Martin; Crueger, Traute; Fast, Irina; Fiedler, Stephanie; Hagemann, Stefan; Hohenegger, Cathy; Jahns, Thomas; Kloster, Silvia; Kinne, Stefan; Lasslop, Gitta; Kornblueh, Luis; Marotzke, Jochem; Matei, Daniela; Meraner, Katharina; Mikolajewicz, Uwe; Modali, Kameswarrao; Müller, Wolfgang; Nabel, Julia; Notz, Dirk; Peters-von Gehlen, Karsten; Pincus, Robert; Pohlmann, Holger; Pongratz, Julia; Rast, Sebastian; Schmidt, Hauke; Schnur, Reiner; Schulzweida, Uwe; Six, Katharina; Stevens, Bjorn; Voigt, Aiko; Roeckner, Erich (2019). MPI-M MPIESM1.2-LR model output prepared for CMIP6 CMIP. Version 20190710. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.742

    ScenarioMIP Citation:

    Wieners, Karl-Hermann; Giorgetta, Marco; Jungclaus, Johann; Reick, Christian; Esch, Monika; Bittner, Matthias; Gayler, Veronika; Haak, Helmuth; de Vrese, Philipp; Raddatz, Thomas; Mauritsen, Thorsten; von Storch, Jin-Song; Behrens, Jörg; Brovkin, Victor; Claussen, Martin; Crueger, Traute; Fast, Irina; Fiedler, Stephanie; Hagemann, Stefan; Hohenegger, Cathy; Jahns, Thomas; Kloster, Silvia; Kinne, Stefan; Lasslop, Gitta; Kornblueh, Luis; Marotzke, Jochem; Matei, Daniela; Meraner, Katharina; Mikolajewicz, Uwe; Modali, Kameswarrao; Müller, Wolfgang; Nabel, Julia; Notz, Dirk; Peters-von Gehlen, Karsten; Pincus, Robert; Pohlmann, Holger; Pongratz, Julia; Rast, Sebastian; Schmidt, Hauke; Schnur, Reiner; Schulzweida, Uwe; Six, Katharina; Stevens, Bjorn; Voigt, Aiko; Roeckner, Erich (2019). MPI-M MPIESM1.2-LR model output prepared for CMIP6 ScenarioMIP. Version 20190710. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.793

  • NESM3

    License description: data_licenses/NESM3.txt

    CMIP Citation:

    Cao, Jian; Wang, Bin (2019). NUIST NESMv3 model output prepared for CMIP6 CMIP. Version 20190812. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.2021

    ScenarioMIP Citation:

    Cao, Jian (2019). NUIST NESMv3 model output prepared for CMIP6 ScenarioMIP. SSP1-2.6 version 20190806; SSP2-4.5 version 20190805; SSP5-8.5 version 20190811. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.2027

  • NorESM2-LM

    License description: data_licenses/NorESM2-LM.txt

    CMIP Citation:

    Seland, Øyvind; Bentsen, Mats; Oliviè, Dirk Jan Leo; Toniazzo, Thomas; Gjermundsen, Ada; Graff, Lise Seland; Debernard, Jens Boldingh; Gupta, Alok Kumar; He, Yanchun; Kirkevåg, Alf; Schwinger, Jörg; Tjiputra, Jerry; Aas, Kjetil Schanke; Bethke, Ingo; Fan, Yuanchao; Griesfeller, Jan; Grini, Alf; Guo, Chuncheng; Ilicak, Mehmet; Karset, Inger Helene Hafsahl; Landgren, Oskar Andreas; Liakka, Johan; Moseid, Kine Onsum; Nummelin, Aleksi; Spensberger, Clemens; Tang, Hui; Zhang, Zhongshi; Heinze, Christoph; Iversen, Trond; Schulz, Michael (2019). NCC NorESM2-LM model output prepared for CMIP6 CMIP. Version 20190815. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.502

    ScenarioMIP Citation:

    Seland, Øyvind; Bentsen, Mats; Oliviè, Dirk Jan Leo; Toniazzo, Thomas; Gjermundsen, Ada; Graff, Lise Seland; Debernard, Jens Boldingh; Gupta, Alok Kumar; He, Yanchun; Kirkevåg, Alf; Schwinger, Jörg; Tjiputra, Jerry; Aas, Kjetil Schanke; Bethke, Ingo; Fan, Yuanchao; Griesfeller, Jan; Grini, Alf; Guo, Chuncheng; Ilicak, Mehmet; Karset, Inger Helene Hafsahl; Landgren, Oskar Andreas; Liakka, Johan; Moseid, Kine Onsum; Nummelin, Aleksi; Spensberger, Clemens; Tang, Hui; Zhang, Zhongshi; Heinze, Christoph; Iversen, Trond; Schulz, Michael (2019). NCC NorESM2-LM model output prepared for CMIP6 ScenarioMIP. Version 20191108. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.604

  • NorESM2-MM

    License description: data_licenses/NorESM2-MM.txt

    CMIP Citation:

    Bentsen, Mats; Oliviè, Dirk Jan Leo; Seland, Øyvind; Toniazzo, Thomas; Gjermundsen, Ada; Graff, Lise Seland; Debernard, Jens Boldingh; Gupta, Alok Kumar; He, Yanchun; Kirkevåg, Alf; Schwinger, Jörg; Tjiputra, Jerry; Aas, Kjetil Schanke; Bethke, Ingo; Fan, Yuanchao; Griesfeller, Jan; Grini, Alf; Guo, Chuncheng; Ilicak, Mehmet; Karset, Inger Helene Hafsahl; Landgren, Oskar Andreas; Liakka, Johan; Moseid, Kine Onsum; Nummelin, Aleksi; Spensberger, Clemens; Tang, Hui; Zhang, Zhongshi; Heinze, Christoph; Iversen, Trond; Schulz, Michael (2019). NCC NorESM2-MM model output prepared for CMIP6 CMIP. Version 20191108. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.506

    ScenarioMIP Citation:

    Bentsen, Mats; Oliviè, Dirk Jan Leo; Seland, Øyvind; Toniazzo, Thomas; Gjermundsen, Ada; Graff, Lise Seland; Debernard, Jens Boldingh; Gupta, Alok Kumar; He, Yanchun; Kirkevåg, Alf; Schwinger, Jörg; Tjiputra, Jerry; Aas, Kjetil Schanke; Bethke, Ingo; Fan, Yuanchao; Griesfeller, Jan; Grini, Alf; Guo, Chuncheng; Ilicak, Mehmet; Karset, Inger Helene Hafsahl; Landgren, Oskar Andreas; Liakka, Johan; Moseid, Kine Onsum; Nummelin, Aleksi; Spensberger, Clemens; Tang, Hui; Zhang, Zhongshi; Heinze, Christoph; Iversen, Trond; Schulz, Michael (2019). NCC NorESM2-MM model output prepared for CMIP6 ScenarioMIP. Version 20191108. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.608

  • UKESM1-0-LL

    License description: data_licenses/UKESM1-0-LL.txt

    CMIP Citation:

    Tang, Yongming; Rumbold, Steve; Ellis, Rich; Kelley, Douglas; Mulcahy, Jane; Sellar, Alistair; Walton, Jeremy; Jones, Colin (2019). MOHC UKESM1.0-LL model output prepared for CMIP6 CMIP. Version 20190627. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1569

    ScenarioMIP Citation:

    Good, Peter; Sellar, Alistair; Tang, Yongming; Rumbold, Steve; Ellis, Rich; Kelley, Douglas; Kuhlbrodt, Till; Walton, Jeremy (2019). MOHC UKESM1.0-LL model output prepared for CMIP6 ScenarioMIP. SSP1-2.6 version 20190708; SSP2-4.5 version 20190715; SSP3-7.0 version 20190726; SSP5-8.5 version 20190726. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.1567

Acknowledgements

This work is the result of many years worth of work by members of the Climate Impact Lab, but would not have been possible without many contributions from across the wider scientific and computing communities.

Specifically, we would like to acknowledge the World Climate Research Programme's Working Group on Coupled Modeling, which is responsible for CMIP, and we would like to thank the climate modeling groups for producing and making their model output available. We would particularly like to thank the modeling institutions whose results are included as an input to this repository (listed above) for their contributions to the CMIP6 project and for responding to and granting our requests for license waivers.

We would also like to thank Lamont-Doherty Earth Observatory, the Pangeo Consortium (and especially the ESGF Cloud Data Working Group) and Google Cloud and the Google Public Datasets program for making the CMIP6 Google Cloud collection possible. In particular we're extremely grateful to Ryan Abernathey, Naomi Henderson, Charles Blackmon-Luca, Aparna Radhakrishnan, Julius Busecke, and Charles Stern for the huge amount of work they've done to translate the ESGF CMIP6 netCDF archives into consistently-formattted, analysis-ready zarr stores on Google Cloud.

We're also grateful to the xclim developers (DOI: 10.5281/zenodo.2795043), in particular Pascal Bourgault, David Huard, and Travis Logan, for implementing the QDM bias correction method in the xclim python package, supporting our QPLAD implementation into the package, and ongoing support in integrating dask into downscaling workflows. For method advice and useful conversations, we would like to thank Keith Dixon, Dennis Adams-Smith, and Joe Hamman.

Financial support

This research has been supported by The Rockefeller Foundation and the Microsoft AI for Earth Initiative.


  1. At the time of running, no ssp1-2.6 precipitation data was available for the FGOALS-g3 model. Therefore, we provide tasmin and tamax for this model and experiment, but not pr. All other model/experiment combinations in the above table include all three variables.

  2. The institution which ran MPI-ESM1-2-HR’s historical (CMIP) simulations is MPI-M, while the future (ScenarioMIP) simulations were run by DKRZ. Therefore, the institution component of MPI-ESM1-2-HR filepaths differ between historical and SSP scenarios.

downscalecmip6's People

Contributors

brews avatar delgadom avatar dependabot[bot] avatar dgergel avatar emileten avatar jgerardsimcock avatar kemccusker avatar merfish avatar pyup-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

downscalecmip6's Issues

add validation to global bias correction prototype workflow

This might include comparing a timeseries of holdout versus predicted for one grid cell, a map of annual/seasonal average differences between holdout and predicted, etc.

A separate workflow will also validate with a NASA-NEX downscaled model and its corresponding CMIP5 model.

cc @kemccusker

also see #59 comments

check that historical model run (2004-2014) is concatenated to future SSPs

for bias correcting the first ten years of the projection period (2015 - 2024) for each SSP, we want the historical model run to be concatenated onto the SSP such that the full 21 years of data are going into each rolling CDF. This is likely already what is happening given xarray date selection functionality but we need to verify it.

time handling updates needed in clean-cmip6-workflow

As a follow-up to #93, we need to clean up the following time handling aspects of the CMIP6 cleaning workflow:

  • update histslice-from-time to 1950
  • update histslice-to-time to 2014 (pending @merfish check that xclim's QDM implementation can handle different time slices in training and ref CDFs)
  • update referenceslicehist-from-time to 1994 and only include last 15 days from that year (add month/day to time slice)
  • update from_time and to_time to be 1979-12-17 etc.

cc @brews (feel free to add anything I've missed)

updates needed in dc6-workflow

As a follow-up to #93, we need to clean up the following aspects of the main bias correction and downscaling workflow:

  • update biascorrect-firstfutureyear to 1950
  • currently the workflow only adjusts one run, a historical and future projection concatenated together. Need to update to have a "train QDM" step and then branch out into "apply historical" (1950-2014) and "apply future" (2015 - 2100) for each SSP, and those need to have the correct time slices available (e.g. future SSPs need to have historical data, historical model run needs SSP245 if that's the decision we make in terms of including/excluding future years)
  • update biascorrect-lastfutureyear to 2100

cc @brews (feel free to add anything I've missed)

GISS-E2-1-G tasmax

Currently NASA-GISS GISS-E2-1-G model output is only available for SSP2-4.5 and historical in GCS and AWS, not SSP1-2.6 and 3-7.0. Given how widely used this model is, we likely need to get it migrated from ESGF to GCS/AWS.

cc @kemccusker for discussion

Transition established Argo Workflows into WorkflowTemplates

We have some raw Workflows in the workflows/ directory. We've been transitioning these into reusable, testable WorkflowTemplates in workflows/templates.

Once the transition is complete, we need to remove the raw, legacy workflows in the workflows/ directory.

As part of this, we're also dropping workflows/intake-catalog.yaml. Also workflows/cmip6run-targets.json in favor of the per-run files in workflows/parameters.

Fix dateline artifact in downscaled output

Prototype runs have output ACCESS tasmin and tasmax with a dateline artifact. Looks like it's coming from ERA-5 regridding and specifically the "coarse" reference file input the the train-aiqpd step. Specifically, the artifact appears in input data after the reference data is regrid from 1x1 degree to 0.25x0.25 -- the second regridding.

Obvious solution is to have the coarse reference template add cyclic pixels before regridding 1x1 to 0.25x0.25.

concatenate SSP3-7.0 to historical GCM output

Currently when we bias correct the historical model run from 2004 - 2014, the window length for the 21-yr rolling CDF decreases. We want to concatenate 2015 - 2025 of the SSP3-7.0 run to the historical so that those years are included in bias correcting 20014 - 2014.

Combine biascorrection ("dc6") and downscaling into a single workflow step

Right now we run biascorrect and downscaling as separate workflow steps. These two steps can be kept as separate workflow templates but they feed into one-another and many of their steps can run in parallel - reducing wall time for each run.

Let's try to have a biascorrect and a downscale workflow template — defining the core logic for each step, as we have now. But then let's have a biascorrectdownscale template that merges this work together in a more efficient way.

add example workflow using rechunker package on CMIP6

One of the key parts of our workflow will involve rechunking CMIP6 model output from time chunks to space chunks. Because this is a computationally intensive step (and we may need it multiple times), we want the speedup of rechunker. After writing this up as a notebook, we'll containerize it.

post-processing step for bias correction needed

With the additional diagnostics option in sdba that we are now leveraging for downscaling so that we have the quantiles computed during bias correction, we need one additional step so that xclim can ingest the quantiles correctly. We need to update the quantiles to be a coordinate of scen and need to do this either in dodola after adjusting or in the dc6 workflow. Right now the quantiles are a data variable and not a coordinate of scen, so xclim can't correctly ingest them. I think that putting this into dodola makes more sense, in the qdm adjust function. It would look something like this:

ds_biascorrected = ds_biascorrected['scen'].assign_coords(sim_q=ds_biascorrected.sim_q)

cc @brews @kemccusker

Migrate storage account containers to specialized expanded containers

With PR #151 we added several containers for Azure blob storage to accommodate an expanded workflow. The current workflows should be updated to use these new containers before the old containers can be mothballed.

Biascorrected and downscaled output be directed to:

  • biascorrected
  • downscaled

Paths in the workflows/intake-catalog.yaml should also be updated.

clean up notebooks directory

The organization of the notebooks directory has gotten out of hand - need to come up with a more reasonable directory structure.

Output QDM quantiles with qdm-adjust bias-corrected output

Right now, our qdm-adjust step outputs the bias-corrected GCM run data.

We need the quantiles (sim_q) from the QDM adjustment included in output data. These quantiles are required for the subsequent downscaling step with AIQPD.

Ideally, sim_q would be a coordinate variable to the bias-corrected output. That said, I'd be happy if we could get this data out as an entirely separate (non-corrdinate variable) or even a separate zarr store.

downscale DTR using tasmax and tasmin

As we've discussed, we'll be running tasmax and diurnal temperature range (DTR, e.g. tasmax - tasmin), along with daily precip through our pipeline. DTR will be treated as a multiplicative type of variable (like precip) as opposed to additive.

This means that we need to add two steps to our pipeline:

  1. make sure that DTR is being computed at the beginning after cleaning tasmax and tasmin (this might already exist?)
  2. after downscaling, we'll need a step to compute tasmin from tasmax and DTR, which is what we'll be using/sharing

add support for 360-day calendars

Currently our pipeline only supports leap day and no-leap day GCM calendars. The HadGEM and KACE models (Hadley Center and NIMS-KMA, respectively) both have 360_day calendars, thus we need to add support for those calendars in our pipeline. Current proposal is to use the xclim convert_calendar function for now.

Create an `e2e-tasmax` workflow and time it, record node usage

Right now the steps to process a GCM for a single variable (e.g. tasmax) are broken out into separate workflows:

  1. download-cmip6
  2. clean-cmip6
  3. dc6/biascorrect/downscale

Each of these steps is represented by a workflow template. Folks are curious about runtime and node usage for all these to run a single GCM on a single variable for multiple SSPs.

Make sense to have a single e2e-tasmax workflow that runs all these steps. Then we can get the information we need by timing this e2e workflow run.

Potential collaboration

Hi, we (Ouranos) have a similar interest in building a library of empirical downscaling methods built on xarray. Recently I've added the quantile method to groupby objects in xarray, which simplifies the implementation of quantile-based correction methods (see Ouranosinc/xclim#246 for a first rough draft).

Let us know if this is something you are still considering, as we'd be glad to join efforts.

add bias correction workflow

Was just trying to provide an example of this to one of our collaborators and realized we don't have the .yaml specs of our bias correction argo workflow up just yet. I know we're still fine tuning this, but @brews is it possible to add an early rendition even though we'll be making quite a few more updates?

add precip wet day frequency correction post-downscaling

We have already added the wet day frequency correction in to the workflow pre-bias correction, but we need to add it in post-downscaling as well. This already exists in dodola as the post flag option, just need to add it to our pipeline.

global bias correction workflow cleanup

Currently the global bias correction workflow needs some overall cleanup:

  • packagizing of functions that are in the notebook and need to be taken out
  • add metadata for netcdf files created in the workflow
  • add further validation beyond what's already been done (see also #62)

prototype global bias correction

Design global prototype of daily downscaling (using the BCSD NASA-NEX daily method). Use CMIP6 model from CMIP6 GCS archive along with 1/4 deg obs dataset. Run for limited number of timesteps.

add QDM/QPLAD demo integration notebook

In addition to the QPLAD and QDM notebooks thus far, we need a notebook that integrates QDM and QPLAD using ERA-5 and CMIP6 data over a small geographical area to use as a demo notebook.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.