ipcc-wg1 / atlas Goto Github PK
View Code? Open in Web Editor NEWThis project forked from santandermetgroup/atlas
Repository supporting the implementation of FAIR principles in the IPCC-WGI Atlas
License: Other
This project forked from santandermetgroup/atlas
Repository supporting the implementation of FAIR principles in the IPCC-WGI Atlas
License: Other
To improve the findability, I think the existence of the ATLAS github repository should be made more visible in the Interactive Atlas itself. I could only find a link to this resource in the http://ipcc-atlas.ifca.es/about section of the IA (the link is broken, by the way).
Thank you for the invitation to review the IPCC AR6 WGI Atlas. I believe this effort is fantastic, and I am sure it will improve the accessibility and understanding of the projected changes due to anthropogenic climate change. My comments and suggestions are to, what I consider, an improvement in the FAIR data practices. My overall suggestions are:
1. ATLAS vs. Atlas. I consider that we should be consistent and use one of those options. I would choose "Atlas" because, at first sight, "ATLAS" seems to be an acronym.
2. Include Python in addition to R (?). Despite the popularity of R, I believe that including Python could improve the accessibility and reproducibility of the Atlas. We could, for example, wrappers to run the R-based scripts in Python. At least in my institution, many users do not have experience using R but with Python.
3. Model Validation. I consider that it would be great to include observational climate data in addition to model data so that users can compare and validate, for example, trends in precipitation during the observed historical period (e.g., 1850-2015) its simulated counterpart. These comparisons are critical for assessments of anthropogenic climate change uncertainties at regional scales.
4. Accessing the Atlas online. For some reason, the links provided to access the Atlas (e.g., http://ipcc-atlas.ifca.es and ipcc-atlas.ifca.es/about) do not work.
I believe it would be appropriate to change the name of the inventory files included in the folder ESGF -inventory to better reflect the realm (atmos, land, ocean, etc) and the frequency of the subset.
The option "Export PDF" in the IA returns a file with a wrong extension (".png"). Besides, in my opinion, the name of the files are a little bit ambiguous since they contains several reference regions that have not been selected by the user:
CMIP6 - Accumulated precipitation (PR) Relative change (mm_month) - Long Term (2081-2100) SSP5 8.5 AR5 (1986-2005) - Annual (Greenland_Iceland, E.Europe, S.Asia, Sahara, N.E.Canada, N.W.North-America, S.Central-America, C.Nor-1.png
Description of the problem
the dynamic sliding scale at the left of the display does not correspond to where the mouse is pointing
To Reproduce
Write below the steps to reproduce the problem:
4 float mouse of dark red areas (texas) or NE US (all greyed out)..the number of days above C40 is higher in NE (~15-20) than dark red (where scale sits around 10)
Expected behavior
I would say that was obvious
:-)
Screenshots
Desktop (please complete the following information):
Hi, I've gone through all the items in this repo and I will give a high level review because I am neither an R user nor familiar with the pieces that go into global scale climate work (I am a coastal oceanographer). However, I am an Associate Editor in Chief with the Journal for Open Source Software (JOSS) and I have related experience through that (as well as my own research).
Generally I really like the approach you are taking. It is clear you are working very hard to be clear, open, and reproducible. This is excellent! Using GitHub as a tool for this is great since it is all tracked and open, and additionally there are a number of tools built in to help, for example displaying notebooks is really helpful. I think that Python has a larger user base for this sort of work, but mine is an outside perspective and maybe isn't true for climate modeling. The readme's are nice for guidance through the directories, and the notebooks are helpful for more controlled explanation. A full readthedocs sort of comprehensive documentation would be better.
All this said, I would like to suggest that for future, more formal reviews, you could consider the JOSS review approach. Or, perhaps it would be appropriate to initiate a formal JOSS-IPCC partnership for this (I am happy to initiate a conversation about this between groups if there is interest on your part).
For a review, an editor recruits at least 2 reviewers to actually install, test, and use the code, using the available software installation instructions and other documentation. The review process is done in github, but in a specific issue (in the JOSS repo) in which the reviewers summarize their reviews and then link to other issues in the software repo for more details. This helps to keep the reviews organized and have a basic train of thought and conversation from the reviewers while still allowing for lots of details in the separate issues.
I didn't try out your JupyterHub, but in your case it might make more sense to use that environment since probably all the model output is available there which is important, though if by the time of the next IPCC the model output and data is easily available in the cloud, it would be best for reviewers to test everything totally independently. The goal in your case might be that the reviewers can reasonably reproduce the final plots, products, or analysis, but that would need to be specified.
You can get a good feel for the review approach from the review checklist.
The Review Guiding document specifies that the datasets mentioned in the inventory support provenance. Could the inventory include a link to the dataset and/or especially to the provenance records associated with it, in a way that this could be validated?
I have selected CMIP6, CDD, 3°C GWL, baseline 1995-2014. the error ref
https://interactive-atlas.ipcc.ch/permalink/yf8eyPRB
is reported and no map is displayed.
Selecting other baselines the problem is solved. It seems to be related to this specific baseline
Thanks for your help
Piero Lionello
Description of the problem
To Reproduce
Write below the steps to reproduce the problem:
Expected behavior
Screenshots
Desktop (please complete the following information):
Additional context
First, this is a great project and will come very useful to many researchers.
Interactive web http://ipcc-atlas.ifca.es : time series graphs have many lines and shades but no legends saying what they are, adding legends will greatly improve user experience.
Git Warming levels: Some guide explaining how to interpret tables would help someone who is just starting with the subject.
Git Scripts: Codes are well written with comments, so easy to read. A little more information (reference literature/papers) in the README.md about methods used in calculations would be useful.
This is more a question/curiosity than a comment: Why does the repo focus only on model (CMIP5/6, CORDEX) simulations and does not provide access to aggregated observations? Having these data would be very useful for many users.
When entering the interactive atlas / regional information, the default scenario that is selected is the "worst-case" scenario SS5 8.5, which is unlikely, as far as I know. Wouldn't it be better to select, by default, the most likely scenario, rather than the worst case one?
Although find the general structure of the repo is OK, the main README (at the root directory level) may not be really descriptive of the contents that are actually hosted. Considering this is the first impression a user gets when he/she lands on the repo, it may be worth to polish a bit this page. A simple improvement would be to include at this level one small paragraph briefly describing what is included in each directoy (aggregated-datasets, reference-grids, etc.). I don't see it's a problem to have this information duplicated in the corresponding README of each directory.
Firstly, this is such a great effort. The portal has a nice look and feel with a plethora of information one can access without having to even download the datasets or write a script. Great work. Just some knit picking in the best interest of Atlas, can be found below. Thanks for the oppurtunity.
R is great and the use of Jupyter notebooks is fantastic too. The scope could be expanded by including languages like Python. This would also fit under the “Reusability” and “Interoperability” aspects in FAIR. This also sets a future vision to containerize the scripts to promote reusability.
The csvs in https://github.com/IPCC-WG1/Atlas/tree/master/ATLAS-inventory are great ways to establish some level of provenance. The version identifier is extremely important, nicely captured for CMIP6. For CMIP5 though, it’s not specified in the .csv. I believe version strings did exist for CMIP5 as well.
The CMIP6 siconc inventory from ESGF also seems to be missing version identifiers https://github.com/IPCC-WG1/Atlas/tree/master/ESGF-inventory/CMIP6.
For the datasets used, it’s nice to see the observation datasets listed. It might help to specify versions for all of the datasets, provide a data DOI or at least a pointer to the data in the public domain.
The notebook examples themselves would be great to have more markup documentation, especially the sections where the data is ingested to let users know how it can be expanded.
Also, what does one need to do in order to reproduce the notebook figures? This question should be addresses in simple words to promote reusability.
It will be nice tooltips while hovering over different locations on the atlas which then lists the products such as time series, etc options. These type of actions could also be documented in the README or user documentation links.
The maps downloaded from the atlas portal do not carry enough metadata. This could potentially be used in presentations and papers and it will be great to see the provenance info on the maps, unless I am missing it. Once again,the data DOIs or acknowledgements are essential to make sure users continue to track provenance.
Possible typos in Help & Instructions and missing links.
In particular, the SOD incldues
Climatic Impact Drivers as defined in Chapter 12 SOD (in is missing)
Climatic Impact Drivers - map legend, confidence. In direction of change
About page:
The inventory of model runs, scenarios and variables used in the SOD is available at the IPCC WG1 Atlas repository . But, the link https://github.com/IPCC-WG1/Atlas/tree/master/AtlasHub-inventory is not a valid reference. Perhaps you’re referring to https://github.com/IPCC-WG1/Atlas/tree/master/ATLAS-inventory
I think the purpose of Atlas and the maps shown in atlas could be documented to be more specific For example, if "key figures" are shown in Atlas, how are the key figures determined? Are they from the chapter figures? If yes, how does this effort compare to the ESMValTool effort.
Consider renaming master branch to main branch. https://github.com/github/renaming
In terms of findability and accessibility, one could consider inventorying the scripts used. Sorry if I overlooked this, the README is a great place to start.
In terms of regridding, the regridding algorithms and source code could be references. Basically, at whatever possible granularity, the tools that encompass Atlas should be archived, along with a DOI, especially since DOIs can support different versions should that be needed.
Dad data retraction elements, pointers to errata pages (if any) for the datasets used also should be documented-- if there be a situation like that. I think this is a general problem that has no direct solution. Your csv listing is a great start. Some basic documentation or prompts for users to check data versions before running the notebooks, say few years from now on a data that later has a newer version - may be valuable. DOIs can also be very helpful here to trace back the steps.
For future work, it will be great to see more generalized data cataloguing utilities that can readily help with analysis. Some python based utilities exist today. To help users work with big data, example notebooks for use with xarray, dask etc exploring on and off the cloud capabilities would be cool. Overall, this is a great initiative.
in the README
https://github.com/IPCC-WG1/Atlas/blob/master/reference-regions/README.md
The link to the AtlasHub-inventory gives 404. Does it exist/require special Access rights? if yes it should be explained how to get that. How is this related to the The Atlas Hub mentioned in the review guidelines? (https://hub.ipcc.ifca.es)
The CSV files in the aggregated-datasets/data
are probably the most important for the non (climate) data scientists, in the end for many applications a time-series is enough to produce meaningful insights. To this end, I would add other fields in the metadata provided in the header of the CSV files:
In my opinion, with those info, the files might have more value for many users.
Thank you very much for the fantistic effort putting the Atlas together. Here are some comments/question:
IPCC-WGI-reference-regions-v4_coordinates.csv
: add header with column titlesThis repository is a timely contribution to the need of a transparent and reproducible science and evidence-informed policy making. To this end, I think we should expect many different types of users, not only climate scientists but in general data practitioners and scientists from other disciplines.
Then I think the README should be more informative and providing details on:
This directory is probably the most obscure one. The README is pretty thin and overall, it feels very much like internal stuff. Not clear exactly how to review this.
The ATLAS interactive tool is looking really good! However, there are some accessibility problems that will prevent users navigating the page using a keyboard or a screen-reader (including users with vision problems). I think there are some small changes that can be made to fix this.
The only parts of the page that are focusable using a keyboard are the primary navigation elements and the IPCC logo along the top. This means that users without a pointing device cannot locate other page elements, and users with a screen-reader cannot focus on them in order to hear descriptions of them.
This can be fixed by adding a tabindex="0"
attribute to those elements (which will allow them to be focused on in the order they appear in the HTML). See MDN's page on tabindex
for more details.
I've included a list of elements that require attention at the bottom of the issue (although many have classes but no unique ID, so there might be some hunting required). The content of the map (including individual regions) would ideally also be tabbable, but I'm less sure about how this is accomplished when it's inside a canvas
. The charts included when you click on the regions should, I think, be enhanceable; I know other Recharts users have requested this.
A few navigation elements and menus also require the attribute role="list"
, so that screen-readers can correctly identify the size of their contents (for example, to tell a user that they are focused on a list of 9 radio buttons, without reading each radio button out unless they want to). Other elements might benefit from other role
attribute values (there's a full list here), but I'm not entirely sure which ones have been subsumed into HTML and which are still in active use.
I strongly recommend using a screen-reader browser extension to test the page. Use Tab and Enter to focus on elements that have been augmented as described above, and ensure that they are correctly described to users. Those elements that aren't can be further enhanced with the aria-label
attribute—for example, aria-label="Button: select map projection"
or aria-label="Region: Southern Australia"
.
Element description | ID | Class |
---|---|---|
Home logo | .ipcc-logo |
|
Page navigation | .ipcc-app__link-container |
|
--Links x 4 | .ipcc-header-link |
|
Mobile "hamburger" menu | .ipcc-menu-button |
|
--Links x 4 | .ipcc-mpbile-link |
|
--Language selector | .lang-select |
|
Map navigation (also applies to left and right mirrored views) | .map_container_options |
|
--Top-level options x 4 (Dataset, Variable, ...) | .option__dropdown-box |
|
----Radio set x 3 | .radio-list-selector |
|
----Radio buttons | .radio |
|
Region set selector | .region-set-selector |
|
Map | #map__[id] |
.map |
--Controls | .ipcc-conrols__container |
|
----Control x 8 | .ipcc-base control |
|
----Projection buttons x 3 | .ipcc-base control |
|
----Download options x 2 | .ipcc-base control |
|
--Legend | .ipcc-html-legend-container |
|
----Collapse legend button | .resize-legend |
|
--Map content (canvas element) | ||
--Caption | #primary-map-caption |
|
Permalink button | .permalink-button |
|
Region tab sizing wrapper | .resize-handler-wrapper |
|
--Resize handle | .resize-handler |
|
--Maximize window | ||
--Close window | ||
Region tab | .ipcc-tabs |
|
--Tabs navigation | .ipcc-tabs__header |
|
----Tabs x 5 | .ipcc-tabitem |
|
--Tabs body: Recharts container | .recharts-wrapper , .recharts-responsive-container |
|
--Tabs body: summary table | .summary-table |
|
----Table rows (not sure if these need to be addded?) | ||
--Tabs body: climate stripes | .climate_stripe |
Looking forward to seeing the tool go live! I think these improvements will open the tool up to broader audiences 🥳
The "Virtual Workspace: The Atlas Hub" section in the root README file points to the climate4R GitHub repository to start working with it, but this seems a dead end. No mention of the Atlas Hub is found by following the link provided (a climate4R Hub is mentioned, but no access information is provided). I think that, even if the hub relies on climate4R, a direct link should be added to this section of the README file.
The Atlas Hub is a great idea for a fast start in working with the data, scripts and notebooks. It seems just an example of the more general climate4R Hub. The contents of the github Atlas repository can be found under shared/github/ATLAS
. No specific mention to the Atlas Hub is found when one arrives at the home directory, though. I would suggest to add a README file to guide the user through the Hub contents. A direct link (e.g. github-IPCC-WG1-Atlas -> shared/github/ATLAS) would also be helpful to easily arrive at the github Atlas directory structure.
Also, the contents of the shared/github/ATLAS must be a frozen version (a given commit) of the github Atlas repository. Is it currently commit #73961c4 tag:v1.6-review? The new climate4R hub root README file should make clear which Atlas github code version is available for use in the hub.
Description of the problem
To Reproduce
Write below the steps to reproduce the problem:
Expected behavior
Screenshots
Desktop (please complete the following information):
Additional context
The guided tour help for Time Series is incorrectly positioned and the 'Time Series' tab isn't highlighted
When taking the tour through the atlas a modal window (rather a layer) appears containing a help message explaining a UI element of the atlas. In all cases, the UI element is highlighted, except for the 'Time Scale' tab.
The respective help message appears in the middle of the screen not next to the 'Time Series' tab'
To Reproduce
Write below the steps to reproduce the problem:
Expected behavior
The blue help message should appear next to the Time Series tab, and the latter should be highlighted, like it is for other tabs. Screenshot shows how it is for the 'GWL Plot' tab.
Expected appearance (but as an example for 'GWL Plot'):
Desktop:
I am noticing that point inofrmation for the cursor position is sometimes (in some spots) not giving accurate values relative to point color and corresponding color bar value.
Description of the problem
This is just a test problem report.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Screenshots
Desktop (please complete the following information):
Additional context
When the map projection is set to 'Robinson (Pacific-centric)' and then the map is exported to a PNG file, polygon edges from continents touching the bounding box cross from one edge to the other.
To Reproduce
Expected behavior
For the exported map to look as it does in the browser viewer.
Desktop (please complete the following information):
Description of the problem
To Reproduce
Write below the steps to reproduce the problem:
Expected behavior
When loading the map it stops and doesn't load the full map (see screenshot)
Screenshots
Desktop (please complete the following information):
Additional context
--
First of all, let me highlight that the scripts and notebooks provided are extremely useful to promote transparency, reproducibility and re-usability! That said, I'd like to make a comment on this regard. My view differs from other previous review comments, which suggest to store all the scripts/notebooks into a unique, dedicated directory. Instead, I find useful to include a "scripts" and a "notebook" directory within each main directory (e.g. "aggregated-datasets", "reference-regions"), since it'll be easier for a user to find what he/she's looking for. Following from this, I wonder if keeping the current "scripts" and "notebooks" directories (which are rather empty), at the root level, makes much sense. Perhaps it could be appropriate to create a new "figs" directory at this level, and include there the scripts/notebooks which allow to reproduce the key figures of the Atlas chapter.
Jupyter notebooks are a great way to explain in detail examples of use of the available data and scripts. However, I think that scripts could be further commented to improve reusability. At least the main sections and some specific actions. E.g. there are hardcoded specific model runs excluded from the computations here:
Also, given that the scripts are devoted to reusability, all of them should include a license text and author(s). I also miss the author(s) in notebooks.
CMIP6 datasets are usually provided with a DOI, which is returned by the ESGF search service.
In order to support FAIRness, could the DOIs be resolved and the landing page stored in the inventory? Is this maybe already implemented in the Interactive Atlas tool?
For instance, taking as example the dataset
CMIP6.CMIP.EC-Earth-Consortium.EC-Earth3.historical.r1i1p1f1.Amon.gr
The associated DOI would be
doi:10.22033/ESGF/CMIP6.4700
which resolves to the landing page which contains many of information on the model, such as its parametrisation, characteristic, execution, attribution (provenance).
General question: Is there any higher granularity supported for CMIP6 DOIs? For instance until the variable?
I find the structure OK. Just the man
directory is a bit odd. I'm not sure of the origin of this directory name. This is a standard directory name for Unix man pages. However, it contains some instructions on the Atlas Hub, along with coastline data files and some images of the reference regions and grids. A clearer name could be found for this directory, or the contents moved to the existing ones (e.g. to reference-regions, reference-grids, ...).
For all regions, it's weird that the 95th percentile maximum of daily maximum temperatures is lower for 4 C global warming than 3 C global warming - it should be the opposite! Why does this occur?
When you mouse over the time series, a small window appears which shows the year (the x value). From a communication perspective, the pop up should show the y value because (1) the y value is important, and (2) the y value is squished, so I can't see what the y-value is, whereas the x value (the year) is easily readable.
Some scripts have a GPL license in the headers. Can these be removed since there is already a license file in the top directory ?
Also, can the scripts be made available under CC-BY ?
The "esgf-utils @ 30edcfb", "publisher @ 0e4b1bb" and "tds-content @ b6f5dc3" directories seem to be mostly for internal use, I'm not sure if it's worth to keep them in the repo.
testing
This is a minor comment, but it'd be nice to carefully check the terminology used along the repo for the sake of consistency. For instance, the terms "Climate Change Atlas", "ATLAS" (capital letters), "Atlas" (small letters) could be easily homogeneized.
Description of the problem
The global temperature colormap plot initially shows the temperature distribution for 1.5degC but shows 2degC selected in the legend.
To Reproduce
Expected behavior
I expected to see a change in the colormap image when I initially select 1.5 degC (changing from the initial default of 2 degC), but the image stays the same. If I then select 2degC the colormap changes to a hotter image.
PS the tool looks very approachable. Thanks for the awesome work.
Australasia data: In the Interactive Atlas, when you select "Regional information" then "WGII continental", this produces a global map - a drop-down menu of region names would be handy. When you click over Australia or New Zealand, both Australia and New Zealand are highlighted on the map. Therefore, the region is Australasia, consistent with the name of WGII Chapter 11. If you then select any of the menu options such as "Time series" or "Table Summary", the plots or data are incorrectly labeled Australia rather than Australasia.
I would like to find all the notebooks in the same place. Maybe use symlinks if you want to keep the original source in separate directories ? Notebooks include a lot of warnings and white space around figures which distract from the content.
CMIP6Amon_Atlas_WarmingLevels.csv
and CMIP6_Atlas_WarmingLevels.csv
It seems that the word "Artic" is used sometimes (CORDEX Artic, Title for some of the figures...) instead of "Arctic"
[Reported by Samuel Somot: A colleague of mine has found a mistake concerning the Arctic:]
Thanks for the invitation to participate in the IPCC AR6 WGI Atlas FAIR Review - it's exciting to see the IPCC get involved in the open/reproducible science space.
My feedback relates primarily to the issue of orienting readers of the Atlas to the information/metadata available to them. From the README in the root directory and the READMEs in each sub directory it is possible to figure out everything that's available, but I think the main README in the root directory would benefit from a quick summary/overview. Something along the lines of:
For any data visualisation in the Interactive Atlas or Atlas Chapter, you can get the following information/metadata from the various directories in this repository:
aggregated-datasets/
: the corresponding data (i.e. the values shown in the visualisation)scripts/
: the code that generated the visualisationATLAS-inventory/
: a list of the models/runs (and their version number) shown in the visualisationreference-grids/
: details of the common ensemble grid that the models were interpolated toreference-regions/
: details of the region boundariesIf you'd like to dig further into the data on your own, you can also access:
man/
: a cloud environment (the "Atlas Hub") to analyse the datanotebook/
: user friendly descriptions (in Python and R) of how to work with the dataOther information included included in the repository includes:
warming_levels/
: data for particular warming levels as opposed to time slices (not shown in the Atlas)ESGF_inventory/
: a list of all the model data available on the ESGF (this is really for planning purposes only)Description of the problem
CDD results map won't appear for global warming level 2C SSP5-8.5 anomaly relative to 1850-1900. The global map in the interactive atlas is blank (all white). The maps for the other global warming levels are fine - 1.5C, 3C, and 4C.
I came here after visiting the main page https://www.ipcc.ch/
the atlas is great, so it should not be hidden in deep links. People coming to the site need to see
eye-catching button labeled Interactive Atlas
which they can click without thinking and get to the map.eye-catching input to change the warming value
.That way it would just take one search && 2 clicks to see the own location fire🔥.☠️
People are lazy (that includes me too), if they weren't, we wouldnt need an atlas. I think this is crucial.
PS: honestly, at first i thought with interactive atlas that rotating globe was meant, it would be a great pity when people get only so far!
Description of the problem
To Reproduce
Write below the steps to reproduce the problem:
Expected behavior
Screenshots
Desktop (please complete the following information):
Additional context
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.