Coder Social home page Coder Social logo

unidata / python-gallery Goto Github PK

View Code? Open in Web Editor NEW
87.0 18.0 31.0 39.76 MB

Repository of examples showing off the use of Python for meteorology.

Home Page: http://unidata.github.io/python-gallery

Python 34.81% Smarty 0.96% Jupyter Notebook 31.95% HTML 1.43% Makefile 14.39% Batchfile 14.81% Shell 1.65%
unidata python python3 metpy meteorology atmospheric-science atmospheric-sciences

python-gallery's Introduction

Unidata Python Gallery

THIS REPOSITORY IS NO LONGER MAINTAINED It has been replaced by Unidata's Python Training repository.

This is a collection of examples published in the Unidata blogs or just contributed. In addition to viewing them on the blog you can view them rendered at the web gallery, where you can also download the examples as Jupyter notebooks.

Installation Instructions

The current list of dependencies for this collection is:

The easiest way to install these libraries is with conda.

  1. Install Miniconda (Python 3.4) from Continuum Analytics. (Determine if your OS 32 or 64 bit)
  2. Once Miniconda is installed, from the command line (e.g., OS X terminal, cmd.exe), run these instructions to clone the repository and create the environment:
git clone https://github.com/Unidata/python-gallery

cd python-gallery

conda env create -f environment.yml

From a Unix command line (e.g., OS X terminal)

If your default shell is NOT bash, first type bash. To activate or switch to a conda environment, you can conda activate <environment>. For example,

conda activate gallery

To switch and/or deactivate environments:

conda deactivate
conda activate <environment>

From a Windows command line (e.g., cmd.exe)

To activate or switch to a conda environment, you can activate <environment>. For example,

activate gallery

To switch and/or deactivate environments:

deactivate
activate <environment>

python-gallery's People

Contributors

aodhansweeney avatar blaylockbk avatar dopplershift avatar jrleeman avatar julienchastang avatar kgoebber avatar kpozsonyi avatar lago8103 avatar sgdecker avatar tjwixtrom avatar zbruick avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

python-gallery's Issues

atm test

Could we run python-gallery against atm.ucar.edu/thredds to see what shakes out? Thanks!

Miller composite needs update

Current image has wind barbs on top of legend and is not filtering them. Legend issue at least was probably broken by Matplotlib.

NEXRAD on AWS

This might not be the best place to raise this issue because I don't think this notebook is here, but I was trying to run the @dopplershift example of accessing NEXRAD data from AWS here:
http://nbviewer.jupyter.org/gist/dopplershift/356f2e14832e9b676207
and having issues.

The 1st issue is that I can't even open the AWS endpoint, even using a .edu domain. Are there some kind of credentials required?

I tried this:

from siphon.radarserver import RadarServer
url = 'http://thredds-aws.unidata.ucar.edu/thredds/radarServer/nexrad/level2/S3/'
rs = RadarServer(url)

and nothing crashed, but no response either, so after 25 minutes I interrupted the kernel, which then reported:

KeyboardInterruptTraceback (most recent call last)
<ipython-input-1-05811291f6a7> in <module>()
      2 #url = 'http://thredds.ucar.edu/thredds/radarServer/nexrad/level2/IDD/'
      3 url = 'http://thredds-aws.unidata.ucar.edu/thredds/radarServer/nexrad/level2/S3/'
----> 4 rs = RadarServer(url)

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/siphon/radarserver.pyc in __init__(self, url)
     74         if url.endswith(xmlfile):
     75             url = url[:-len(xmlfile)]
---> 76         super(RadarServer, self).__init__(url)
     77 
     78     def _get_metadata(self):

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/siphon/http_util.pyc in __init__(self, url)
    351         self._base = url
    352         self._session = create_http_session()
--> 353         self._get_metadata()
    354 
    355     def get_query(self, query):

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/siphon/radarserver.pyc in _get_metadata(self)
     77 
     78     def _get_metadata(self):
---> 79         ds_cat = TDSCatalog(self.url_path('dataset.xml'))
     80         self.metadata = ds_cat.metadata
     81         self.variables = set(k.split('/')[0] for k in self.metadata['variables'].keys())

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/siphon/catalog.pyc in __init__(self, catalog_url)
     62 
     63         # get catalog.xml file
---> 64         resp = session.get(self.catalog_url)
     65         resp.raise_for_status()
     66 

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/requests/sessions.pyc in get(self, url, **kwargs)
    485 
    486         kwargs.setdefault('allow_redirects', True)
--> 487         return self.request('GET', url, **kwargs)
    488 
    489     def options(self, url, **kwargs):

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/requests/sessions.pyc in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    473         }
    474         send_kwargs.update(settings)
--> 475         resp = self.send(prep, **send_kwargs)
    476 
    477         return resp

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/requests/sessions.pyc in send(self, request, **kwargs)
    583 
    584         # Send the request
--> 585         r = adapter.send(request, **kwargs)
    586 
    587         # Total elapsed time of the request (approximately)

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/requests/adapters.pyc in send(self, request, stream, timeout, verify, cert, proxies)
    401                     decode_content=False,
    402                     retries=self.max_retries,
--> 403                     timeout=timeout
    404                 )
    405 

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.pyc in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, **response_kw)
    576                                                   timeout=timeout_obj,
    577                                                   body=body, headers=headers,
--> 578                                                   chunked=chunked)
    579 
    580             # If we're going to release the connection in ``finally:``, then

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.pyc in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    383         try:
    384             try:  # Python 2.7, use buffering of HTTP responses
--> 385                 httplib_response = conn.getresponse(buffering=True)
    386             except TypeError:  # Python 2.6 and older
    387                 httplib_response = conn.getresponse()

/home/usgs/miniconda/envs/ioos/lib/python2.7/httplib.pyc in getresponse(self, buffering)
   1134 
   1135         try:
-> 1136             response.begin()
   1137             assert response.will_close != _UNKNOWN
   1138             self.__state = _CS_IDLE

/home/usgs/miniconda/envs/ioos/lib/python2.7/httplib.pyc in begin(self)
    451         # read until we get a non-100 response
    452         while True:
--> 453             version, status, reason = self._read_status()
    454             if status != CONTINUE:
    455                 break

/home/usgs/miniconda/envs/ioos/lib/python2.7/httplib.pyc in _read_status(self)
    407     def _read_status(self):
    408         # Initialize with Simple-Response defaults
--> 409         line = self.fp.readline(_MAXLINE + 1)
    410         if len(line) > _MAXLINE:
    411             raise LineTooLong("header line")

/home/usgs/miniconda/envs/ioos/lib/python2.7/socket.pyc in readline(self, size)
    478             while True:
    479                 try:
--> 480                     data = self._sock.recv(self._rbufsize)
    481                 except error, e:
    482                     if e.args[0] == EINTR:

The 2nd issue is that when I switched the endpoint to thredds.unidata instead of AWS, the data loaded fine (notebook here), but the animation didn't work, producing this output:

IOErrorTraceback (most recent call last)
/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/IPython/core/formatters.pyc in __call__(self, obj)
    341             method = _safe_get_formatter_method(obj, self.print_method)
    342             if method is not None:
--> 343                 return method()
    344             return None
    345         else:

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/matplotlib/animation.pyc in _repr_html_(self)
    979         fmt = rcParams['animation.html']
    980         if fmt == 'html5':
--> 981             return self.to_html5_video()
    982 
    983 

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/matplotlib/animation.pyc in to_html5_video(self)
    953                                 bitrate=rcParams['animation.bitrate'],
    954                                 fps=1000. / self._interval)
--> 955                 self.save(f.name, writer=writer)
    956 
    957             # Now open and base64 encode

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/matplotlib/animation.pyc in save(self, filename, writer, fps, dpi, codec, bitrate, extra_args, metadata, extra_anim, savefig_kwargs)
    808                     # TODO: Need to see if turning off blit is really necessary
    809                     anim._draw_next_frame(d, blit=False)
--> 810                 writer.grab_frame(**savefig_kwargs)
    811 
    812         # Reconnect signal for first draw if necessary

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/matplotlib/animation.pyc in grab_frame(self, **savefig_kwargs)
    228             # frame format and dpi.
    229             self.fig.savefig(self._frame_sink(), format=self.frame_format,
--> 230                              dpi=self.dpi, **savefig_kwargs)
    231         except RuntimeError:
    232             out, err = self._proc.communicate()

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/matplotlib/figure.pyc in savefig(self, *args, **kwargs)
   1563             self.set_frameon(frameon)
   1564 
-> 1565         self.canvas.print_figure(*args, **kwargs)
   1566 
   1567         if frameon:

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/matplotlib/backend_bases.pyc in print_figure(self, filename, dpi, facecolor, edgecolor, orientation, format, **kwargs)
   2230                 orientation=orientation,
   2231                 bbox_inches_restore=_bbox_inches_restore,
-> 2232                 **kwargs)
   2233         finally:
   2234             if bbox_inches and restore_bbox:

/home/usgs/miniconda/envs/ioos/lib/python2.7/site-packages/matplotlib/backends/backend_agg.pyc in print_raw(self, filename_or_obj, *args, **kwargs)
    517             close = False
    518         try:
--> 519             fileobj.write(renderer._renderer.buffer_rgba())
    520         finally:
    521             if close:

IOError: [Errno 32] Broken pipe

Bring in some materials from workshop

I just noticed that we don't have an example, in the gallery itself, of loading upper air data with siphon and plotting a Skew-T. This has likely been overlooked because we do that in our workshop.

We need to look over what we have in the workshop and make sure we bring over those same concepts/demos into the gallery, minus the exercises and probably some of the explanatory verbiage.

Readme instructions

The README should include instructions on how to create a conda environment for running the notebooks. That probably means creating an environment.yml file, which needs to be kept up-to-date.

ipywidgets

An interactive example using IPython's widgets would be really cool. Specific examples:

  • Satellite viewer where you can select GOES East vs. West and what channel
  • Model viewer to look at GFS/HRRR/NAM 12 hour forecast for some specific variables

Not sure if this can be serialized in a way that would run on the blog, though.

Magic numbers in temperature advection example

The 850-mb temperature example is great, but it is quite mysterious how you know the grid spacing is exactly 42583.44283066879 m in one direction and 55588.7366760194 m in the other. Shouldn't these values be calculated from the underlying data?

Make first notebook

Reminder to me that I need to work on the first notebook for this series.

Multiple Unidata Logos

Now that I have fixed the sphinx-gallery issue with transparent backgrounds #43 the logo on the main page needs alpha. We'll need two versions and to upstream this to sphinx-gallery.

Make This Project Easier to Use?

Is there anyway to make this project easier to use? If I want to experiment with a notebook, say the Miller Composite Chart I currently clone

git clone https://github.com/Unidata/python-gallery

At this point, I don't know where to look so I run find . -name '*.ipynb' -print and find nothing.

I then I hop over to gh-pages branch hoping to find the notebooks. Sure enough this time find reveals:

./_downloads/2efb9f7572d606baf7a7ad2b7cf0391b/Ageostrophic_Wind_Example.ipynb
...
./_downloads/af0a8029589d8efd9084299025c8265a/miller_composite.ipynb

If I want to build the conda environment to run these notebooks, I hop back over to master because that is where environment.yml is located.

Is there anyway to have a top-level notebooks directory and a co-located environment.yml from the branch that gets checked out by default?

Add categories

This can be punted until we have more notebooks. Easiest way would be to group them by directory names, and then use those names in the web view.

Fix no image placeholder

The Unidata logo used as a placeholder for examples with no graphical output have a black background because of the alpha. We need a version with a white background so the gallery page looks decent.

GFS with NCSS

Just a basic multi-panel example of GFS forecast using NCSS to simplify getting data for only CONUS. Would also show plotting lon/lat on a projection using CartoPy.

Need a gallery view

So we can give visitors images of the cools things hiding in here. Similar to https://metpy.readthedocs.io/en/latest/examples/index.html.

This will likely use some kind of python script to convert the notebooks (only to extract one of the images) and generate a markdown file that jekyll can handle. Then we can use gh-pages to host the image gallery, which can link to nbviewer to display the actual notebook.

Unidata page link

Need to link/advertise this repository and site from the main Unidata Python page.

Animation

Show how to make an animation in the notebook. Sources:

  • Follow up on GINI
  • GFS/HRRR
  • MRMS

It might also be good to start with some simpler data first--like a simple timeseries plot.

One question is what format we can embed in the blog. We might be limited to html5 video, though that would mean we only need functionality present in matplotlib mainline.

Update Examples with New dim_order Argument

Found this in preparing an example. Examples using advection, vorticity, and geostrophic/ageostrophic wind, etc. should be updated to use the new dim_order argument rather than transposing the data.

Meteogram Example - Move to Canned Data

Looks like the METAR data is giving us all NaN for pressure data and that's causing the y-limit setting to go sideways. Hazards of using "live" data I suppose. Maybe we should point the gallery at the Irma datasets? That would be interesting and consistent while still showing how to access data from THREDDS.

This looks to be a consequence of a regression in mpl matplotlib/matplotlib#9361

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.