pcmdi / input4mips_cvs Goto Github PK

Datasets have the following filenames:
solarforcing-ref-day_input4MIPs_solar_CMIP_SOLARIS-HEPPA-4-1_gn_18500101-20231231.nc
solarforcing-ref-mon_input4MIPs_solar_CMIP_SOLARIS-HEPPA-4-1_gn_18500101-20231231.nc
solarforcing-picontrol-fx_input4MIPs_solar_CMIP_SOLARIS-HEPPA-4-1_gn_18500101-18730128.nc
There will also be an alternative SSI dataset which we shall call
solarforcing-alternative-ssi-day_input4MIPs_solar_CMIP_SOLARIS-HEPPA-4-1_gn_18500101-20231231.nc

Current global attributes (for solarforcing-ref-day_input4MIPs_solar_CMIP_SOLARIS-HEPPA-4-1_gn_18500101-20231231.nc):

:title = "CMIP7 solar forcing historic (1850-2023)";
:institution_id = "SOLARIS-HEPPA";
:institution = "APARC SOLARIS-HEPPA";
:activity_id = "input4MIPs";
:comment = "SSI data are taken from NRL v03r00_preliminary. Sub-annual variability has been added for the period before 1874; m. TSI in this file is from source data as integral over SSI between 0 and 10,000nm";
:time_coverage_start = "1850-01-01";
:time_coverage_end = "2023-12-31";
:frequency = "day";
:source = "nrlssi_v03r00_preliminary (Odele Coddington et al., pers. comm.); Ap, Kp, F10.7 from ftp.ngdc.noaa.gov until 2014, afterwards from GFZ Potsdam (https://kp.gfz-potsdam.de), P-IPR from SEP-II (Ilya Usoskin et al., pers. comm.), MEE-IPR from APEEP apeep_v2024b_cmip7 (Max van de Kamp et al., pers. comm.), GCR-IPR from CRII v2024-02 (Ilya Usoskin et al., pers. comm.)";
:source_id = "SOLARIS-HEPPA-CMIP-4-1";
:realm = "atmos";
:further_info_url = "http://solarisheppa.geomar.de/cmip7";
:metadata_url = "see http://solarisheppa.geomar.de/solarisheppa/sites/default/files/data/cmip7/CMIP7_metadata_description_4.1.pdf";
:contributor_name = "Bernd Funke, Timo Asikainen, Stefan Bender, Thierry Dudok de Wit, Illaria Ermolli, Margit Haberreiter, Doug Kinnison, Sergey Koldoboskiy, Daniel R. Marsh, Hilde Nesse, Annika Seppaelae, Miriam Sinnhuber, Ilya Usoskin, Max van de Kamp, Pekka T. Verronen";
:references = "Funke et al., Geosci. Model Dev., 17, 1217–1227, https://doi.org/10.5194/gmd-17-1217-2024, 2024";
:contact = "[email protected]";
:dataset_category = "solar";
:dataset_version_number = "4.1";
:grid_label = "gn";
:mip_era = "CMIP7";
:target_mip = "CMIP";
:variable_id = "multiple";
:license = "Solar forcing data produced by SOLARIS-HEPPA is licensed under a Creative Commons Attribution "Share Alike" 4.0 International License (http://creativecommons.org/licenses/by/4.0/). The data producers and data providers make no warranty, either expressed or implied, including but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law.";
:Conventions = "CF-1.6";
:creation_date = "2024-05-06T14:57:54Z";

From @MartinaSt

From: Martina Stockhause
Date: Friday, May 19, 2023 at 2:52 AM
To: Durack, Paul J.
Subject: CMIP6Plus - input4MIPs
Hi Paul,
I have thought about CMIP6Plus and input4MIPs a bit more:

1. DOI granularity/-ies: For CMIP6-input4MIPs DOIs on two granularities were registered. The easiest is to do it for CMIP6Plus in the same way and have two new DOIs for your AMIP-1-1-9 data. However, if the CMIP6Plus period is to be short, it might be better to have only a DOI on the finer granularity (with the possibility to relate it to the coarser one for CMIP6). I guess an example makes these options clearer:

• 2 new DOIs on data collections 'input4MIPs.CMIP6Plus.CMIP.PCMDI' and 'input4MIPs.CMIP6Plus.CMIP.PCMDI.PCMDI-AMIP-1-1-9'
• 1 new DOI on data collection 'input4MIPs.CMIP6Plus.CMIP.PCMDI.PCMDI-AMIP-1-1-9' with the possibility to relate it to 'input4MIPs.CMIP6.CMIP.PCMDI'

2. Data Access Links: I will use links into the CoG and change them together with all the others for CMIP6 when ESGF moves to MetaGrid.

3. CV/citation service workflow: Currently, the ESGF publication of the input4MIPs data is first, then the insert into the citation DB and the DOI publication. For some cases, this leads to DOI registrations before the citation manager could adjust the citation metadata. I don't want to change this for a rather short period of CMIP6Plus but rather live with the inconvenience: writing an email to me in case this has happened.

4. Errata/Versions: Yes, that is something we should improve. There will be a delay for displaying this information on the DOI landing pages.

Which option under 1. would you prefer?
Best wishes,
Martina

See code in src/getInput4MIPsESGF.ipynb - consider that deprecated datasets do not list files, which leads to a discrepancy if deprecated=false is not selected

Obtain updated file from https://github.com/santisoler/cc-licenses

@mjevanmarle just creating an issue as a placeholder for discussions in finalizing the registration of the biomass burning institution_id and source_id.

Note the CMIP6 contribution had institution_id: VUA (here) and a couple of versioned releases, so source_id entries: VUA-CMIP-BB4CMIP6-1-0, 1-1 and 1-2 (here). I wonder if we want to maintain any consistency with the previous version, or just start again? I see that you've used the previous template so VUA-CMIP-BB4CMIP6-1-0 becomes DRES-CMIP-BB4CMIP7-1-0.

Just a note, while we start to coordinate the collation of these prototype (v0) datasets, and gather feedback, we're aiming to catch these in the "CMIP6Plus" project, in preparation for CMIP7 in a couple of years. This will allow a clean split between the CMIP7 "endorsed" forcing collection, and those that we are working out the kinks on (caught in CMIP6Plus).

We have updated the institution registration a little moving beyond CMIP6; these now depend on the RoR registry (see here), and, as an example, Deltares is already registered - https://ror.org/01deh9c76

@wolfiex @matthew-mizielinski @taylor13 @vnaik60 @znichollscr ping

This should be added to dataset metadata/CVs

Checa-Garcia, R., 2018: CMIP6 Ozone forcing dataset: supporting information (Initial). Zenodo. https://doi.org/10.5281/zenodo.1135127

@vnaik60

Replicate registration from PCMDI/input4MIPs-cmor-tables#130 in this repo to enable publication of data imminently. Data has been produced and is being downloaded to LLNL/PCMDI to enable local publication into input4MIPs.

For CMIP6, we had the following institution_id entries registered - input4MIPs_institution_id.json:

{
    "institution_id":{
        "CCCma":"Canadian Centre for Climate Modelling and Analysis, Victoria, BC V8P 5C2, Canada",
        "CNRM-Cerfacs":"CNRM (Centre National de Recherches Meteorologiques, Toulouse 31057, France), CERFACS (Centre Europeen de Recherche et de Formation Avancee en Calcul Scientifique, Toulouse 31100, France)",
        "IACETH":"Institute for Atmosphere and Climate, ETH Zurich, Zurich 8092, Switzerland",
        "IAMC":"Integrated Assessment Modeling Consortium (see www.globalchange.umd.edu/iamc/membership for complete membership). Mailing address: International Institute for Applied Systems Analysis (IIASA), Schlossplatz 1, A-2361 Laxenburg, Austria",
        "ImperialCollege":"Imperial College London, South Kensington Campus, London SW7 2AZ, UK",
        "MOHC":"Met Office Hadley Centre, Fitzroy Road, Exeter, Devon, EX1 3PB, UK",
        "MPI-B":"Max Planck Institute for Biogeochemistry, Jena 07745, Germany",
        "MPI-M":"Max Planck Institute for Meteorology, Hamburg 20146, Germany",
        "MRI":"Meteorological Research Institute, Tsukuba, Ibaraki 305-0052, Japan",
        "NASA-GSFC":"NASA Goddard Space Flight Center, Greenbelt, MD 20771, USA",
        "NCAR":"National Center for Atmospheric Research, Boulder, CO 80307, USA",
        "NCAS":"National Centre for Atmospheric Science, University of Reading, Reading RG6 6BB, UK",
        "PCMDI":"Program for Climate Model Diagnosis and Intercomparison, Lawrence Livermore National Laboratory, Livermore, CA 94550, USA",
        "PNNL-JGCRI":"Pacific Northwest National Laboratory - Joint Global Change Research Institute, College Park, MD 20740, USA",
        "SOLARIS-HEPPA":"SOLARIS-HEPPA, GEOMAR Helmholtz Centre for Ocean Research, Kiel 24105, Germany",
        "UCI":"Department of Earth System Science, University of California Irvine, Irvine, CA 92697, USA",
        "UColorado":"University of Colorado, Boulder, CO 80309, USA",
        "UReading":"University of Reading, Reading RG6 6UA, UK",
        "UoM":"Australian-German Climate & Energy College, The University of Melbourne (UoM), Parkville, Victoria 3010, Australia",
        "UofMD":"University of Maryland (UofMD), College Park, MD 20742, USA",
        "VUA":"Vrije Universiteit Amsterdam, De Boelelaan 1105, 1081 HV Amsterdam, Netherlands"
    }
}

Of these CNRM-CERFACS, IACETH, IAMC, ImperialCollege, and MOHC are not registered in PCMDI/mip-cmor-tables/MIP_institutions.json. We'll also need to register a new institution CR (@znichollscr).

@wolfiex how would you recommend we proceed, and should I open an issue in PCMDI/mip-cmor-tables?

..PCMDIobs/obs4MIPs_input/MOHC
HadISST1-1 (high, greens function - here and other sources) - updated to 2023-04-12
..PCMDIobs/obs4MIPs_input/NOAA-PSL
COBE1 (high) - 2023-06-01
ERSST5 (low) - 2023-10-01
OISST2-1 (low) - 2023-10-01
..PCMDIobs/obs4MIPs_input/RSS
RSS-MW5-1
RSS-MW-IR5-1

@eguil @davidhassell this issue has been generated following the email correspondence regarding https://es-doc.org/cmip6-ensembles-conformance/

It will be useful to iterate over the format of the json version info within this issue

Can be closed as this is a duplicate of #8

There doesn't appear to be an obvious place to register institutions. In the source IDs, they are a field, but there is no *instution_IDs.json file in this repo.

Are they instead meant to be registered instead in https://github.com/PCMDI/mip-cmor-tables/blob/main/MIP_institutions.json or https://github.com/PCMDI/input4MIPs-cmor-tables/blob/master/input4MIPs_institution_id.json#L23?

@durack1 have I understood correctly or am I missing something?

In short, it's not clear to me what the rules for the controlled vocabularies are. For example, which fields are compulsory, which can be inferred from data, which are only required for ESGF (i.e. are things that data providers shouldn't worry about, but tooling after submission does need to handle).

To get the conversation started, I've made a google doc here: https://docs.google.com/document/d/1oLK4mWW6TX2YPrhGoLdMLcrK7vX1flU3BjiheTb2Hwk/edit?usp=sharing

Once we've got something a bit more concrete, I will pull everything back into issues that we can track across the various repositories.

@thomasaubry just creating an issue as a placeholder for discussions in finalizing the registration of the volcanic forcing institution_id and source_id.

Note the CMIP6 contribution had institution_id: IACETH (here) and a couple of versioned releases, so source_id entries: IACETH-SAGE3lambda-2-1-0 and 3-0-0 (here).

We have updated the institution registration a little moving beyond CMIP6, these now depend on the RoR registry (see here), and, as an example UExeter is already registered - https://ror.org/03yghzc09

@wolfiex @matthew-mizielinski @taylor13 @vnaik60 @znichollscr ping

@mzelinka just creating an issue as a placeholder for discussions in finalizing the registration of these prototype AMIP SST datasets. These will all be published against the input4MIPs mip_era CMIP6Plus target_mip CMIP - as per PCMDI-AMIP-1-1-9

@taylor13 ping as we're discussing identifiers

Notes to self:
Status of E3SM sims here
The path to Frankenstein ~/scripts/examine_SST_datasets.ipynb

@durack1 moving discussion from #12 here.

Summary of @durack1's comment here:

• excel spreadsheet where attributes was discussed is here: https://docs.google.com/spreadsheets/d/1vMlMVAnP25GpHcS0B8cyvFvUoTZxlUvxBmFB-PbLDG0/edit#gid=108565686
• idea is to move as much info as possible into the source ID JSON file in this repo, with the suggestion being as below

        "PCMDI-AMIP-1-1-9":{
            "calendar":"gregorian",
            "comment":"Based on Hurrell SST/sea ice consistency criteria applied to merged HadISST (1870-01 to 1981-10) & NCEP-0I2 (1981-11 to 2022-12)",
            "contact":"PCMDI ([email protected])",
            "dataset_category":"SSTsAndSeaIce",
            "further_info_url":"https://pcmdi.llnl.gov/mips/amip",
            "grid":"1x1 degree longitude x latitude",
            "grid_label":"gn",
            "institution":"Program for Climate Model Diagnosis and Intercomparison, Lawrence Livermore National Laboratory, Livermore, CA 94550, USA",
            "institution_id":"PCMDI",
            "license":"AMIP boundary condition data produced by PCMDI is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0; https://creativecommons.org/licenses/by/4.0). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing input4MIPs output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file). The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law",
            "mip_era":"CMIP6Plus",
            "nominal_resolution":"1x1 degree",
            "product":"observations",
            "references":"Taylor, K.E., D. Williamson and F. Zwiers, 2000: The sea surface temperature and sea ice concentration boundary conditions for AMIP II simulations. PCMDI Report 60, Program for Climate Model Diagnosis and Intercomparison, Lawrence Livermore National Laboratory, 25 pp. Available online: https://pcmdi.llnl.gov/report/pdf/60.pdf",
            "region":[
                "global_ocean"
            ],
            "release_year":"2023",
            "source":"PCMDI-AMIP 1.1.9: Merged SST based on UK MetOffice HadISST and NCEP OI2",
            "source_description":"Sea surface temperature and sea-ice datasets produced by PCMDI (LLNL) for the AMIP (DECK) experiment of CMIP6Plus",
            "source_id":"PCMDI-AMIP-1-1-9",
            "source_type":"satellite_blended",
            "source_variables":[
                "areacello",
                "sftof",
                "siconc",
                "siconcbcs",
                "tos",
                "tosbcs"
            ],
            "source_version":"1.1.9",
            "target_mip":"CMIP",
            "title":"PCMDI-AMIP 1.1.9 dataset prepared for input4MIPs"
        },

Summary of my reply here:

• I think I understand
• Does it make sense to remove some of the data file specific keys (e.g. grid) to avoid having to make new source IDs for each data file? That would mean the source ID info could be applied to all data files in a data set, with some data file level metadata being captured elsewhere (probably in the file I guess)

Discussion can continue below.