enram / aloftdata.eu Goto Github PK

See reverse alphabetical sorting on https://aloftdata.eu/browse/?prefix=baltrad/hdf5/bewid/2017/12/27/

• Write content for homepage
• Provide link for "radar aeroecology community"
• Choose home banner (originally home-banner)
• Save home banner locally
• Add favicon?
• Add banner logo?
• Add images on homepage for quick navigation? (see template below)

It is not guaranteed that data-repository is where the documentation will remain.

• Dataset metadata (see also https://enram.github.io/globam-dmp/2_pvol_europe.html and https://enram.github.io/globam-dmp/8_open_data.html)
  • Source
  • License
  • Geographical scope
  • Temporal scope
  • Frequency
  • Format
  • File naming conventions
  • Quality assurance
  • Where data are stored
• How to cite the data
• Acknowledgement:

We acknowledge the European Operational Program for Exchange of Weather Radar Information (EUMETNET/OPERA) for providing access to European radar data, faciliated through a research-only license agreement between EUMETNET/OPERA members and ENRAM.

• How to download data (see #2)
  • In your browser
  • With bioRad
  • With AWS CLI
• How to visualize data (CROW)
• How to report issues (at https://github.com/enram/data-repository/issues)

The opera DB json now contains an empty property for each radar, that was not there before:

{
        "number": "1209",
        "country": "Austria",
        "countryid": "LOWM41",
        "oldcountryid": "OS41",
        "wmocode": "11038",
        "wigosid": "",
        "odimcode": "atrau",
        "location": "Wien/Schwechat",
        "status": "1",
        "latitude": "48.074",
        "longitude": "16.536",
        "heightofstation": "",
        "band": "C",
        "doppler": "Y",
        "polarization": "D",
        "maxrange": "224",
        "startyear": "1978",
        "heightantenna": "224",
        "diametrantenna": "",
        "beam": "0.88",
        "gain": "45.5",
        "frequency": "5.625",
        "stratus": "",
        "cirusnimbus": "",
        "wrwp": "",
        "": "" <----
    },

@BarVodSur can this be removed?

Here are download instructions I wrote for BALTRAD. Should be integrated in the website

Data can be downloaded using the following methods.

bioRad

Amazon S3 command line interface

Data are stored in the Amazon S3 bucket enram-vp, which is publicly accessible. The Amazon S3 Command Line Interface (AWS S3 CLI) provides functionalities to efficiently list and download files.

Installation:

python -m pip install awscli

List files for a directory with ls (add --recursive to see the content of subdirectories):

$ aws s3 ls s3://enram-vp/baltrad/hdf5/bejab/
PRE 2018/
PRE 2019/
PRE 2020/
PRE 2021/

Sync/download files to a local directory with sync (add --dryrun to test):

$ aws s3 sync s3://enram-vp/baltrad/hdf5/bejab/ ~/Downloads/baltrad/
download: s3://enram-vp/baltrad/hdf5/bejab/2018/05/22/bejab_vp_20180522T102000Z_0x5.h5 to 2018/05/22/bejab_vp_20180522T102000Z_0x5.h5
download: s3://enram-vp/baltrad/hdf5/bejab/2018/05/22/bejab_vp_20180522T102500Z_0x5.h5 to 2018/05/22/bejab_vp_20180522T102500Z_0x5.h5
download: s3://enram-vp/baltrad/hdf5/bejab/2018/05/22/bejab_vp_20180522T103000Z_0x5.h5 to 2018/05/22/bejab_vp_20180522T103000Z_0x5.h5

Sync is generally more useful than copy (cp), because it will browse subdirectories by default and skip files that are already downloaded. So you can stop the command at any point (Ctrl+c) and it will pick up where it left off if you run it again.

• Link to new s3 bucket
• Review JS code (maybe update from https://github.com/rufuspollock/s3-bucket-listing) keep current
• Decide whether title should be "Browse", "Browse aloft data", "Browse BALTRAD data" Browse data
• Decide whether to show coverage info (e.g. in sidebar, see attempt at browse.txt)
• Decide whether to show radar info (e.g. in sidebar)
• Decide whether to show download instructions (probably easier to link to dataset page)

Rather than showing the coverage, the browser could show the radar metadata when a user navigations to a radar directory:

Buttons could then lead the person to two other pages:

• A coverage page, where coverage is shown in context with other radars. See also enram/data-repository#67
• A CROW page, where data can be visualized.

• Derive from OPERA
• Provide as CSV
• Provide link to CSV
• Section per country
• TOC
• Deep anchors to radar codes
• Link out the coordinates to a map

As discussed with @CeciliaNilsson709 this is the envisioned roadmap for the aloftdata.eu website

Website

• Homepage: explains scope "Bird movement data from European weather radars" and has quick links to visualize (see #3 for more tasks)
• Visualize: CROW visualization for BALTRAD data (external page)
• Browse: file navigation for BALTRAD data (JS on top of S3 repo)
• Datasets: More technical instructions on how to download BALTRAD data. Can be extended with other technical pages for other datasets (see #6 for more tasks)
• About: About page for the website, includes FAQ for non-technical users (see #7 for more tasks)
• Radars (optional): Overview of European radars + metadata TODO: useful? yes

GitHub repositories

This is where the code would live for different components

• enram/aloftdata.eu (Jekyll): home of the aloftdata website
• enram/data-repository (Python): (same as current situation): code for getting data onto the data repository + issues regarding data quality. Landing index.html page would move to enram/aloftdata.eu
• enram/crow (Vue): home of the crow visualization for aloft (currently BALTRAD data only). Would mimic aloftdata style
• enram/vpts (Jekyll): keep has home for vpts csv format, but remove website and integrate directly in aloftdata.eu/format/vpts

URLs and names

• http://enram.github.io/aloftdata.eu redirect to (CNAME) https://aloftdata.eu
• http://enram.github.io/data-repository redirects to https://aloftdata.eu/browse
• Test if http://enram.github.io/data-repository/?prefix=se/osd/ has a meaningful redirect
• http://enram.github.io/crow (new) redirects to https://aloftdata.eu/crow or https://crow.aloftdata.eu
• ENRAM data repository -> Aloft data repository

See https://aloftdata.eu/about/ source at https://github.com/enram/aloftdata.eu/blob/main/pages/about.md

Should describe website "Aloft data" and FAQ, not necessarily the data repository itself ("Aloft data repository")

Could also include a section "about us" (the radar aeroecology community that developed this website).