unidata / metpy

MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data.

MetPy

MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data.

MetPy follows semantic versioning in its version number. This means that any MetPy 1.x release will be backwards compatible with an earlier 1.y release. By "backward compatible", we mean that correct code that works on a 1.y version will work on a future 1.x version.

For additional MetPy examples not included in this repository, please see the Unidata Python Gallery.

We support Python >= 3.9.

Need Help?

Need help using MetPy? Found an issue? Have a feature request? Checkout our support page.

Important Links

• HTML Documentation
• Unidata Python Gallery
• "metpy" tagged questions on Stack Overflow
• Gitter chat room

Dependencies

Other required packages:

• Numpy
• Scipy
• Matplotlib
• Pandas
• Pint
• Xarray

There is also an optional dependency on the pyproj library for geographic projections (used with cross sections, grid spacing calculation, and the GiniFile interface).

See the installation guide for more information.

Code of Conduct

We want everyone to feel welcome to contribute to MetPy and participate in discussions. In that spirit please have a look at our Code of Conduct.

Contributing

Imposter syndrome disclaimer: We want your help. No, really.

There may be a little voice inside your head that is telling you that you're not ready to be an open source contributor; that your skills aren't nearly good enough to contribute. What could you possibly offer a project like this one?

We assure you - the little voice in your head is wrong. If you can write code at all, you can contribute code to open source. Contributing to open source projects is a fantastic way to advance one's coding skills. Writing perfect code isn't the measure of a good developer (that would disqualify all of us!); it's trying to create something, making mistakes, and learning from those mistakes. That's how we all improve, and we are happy to help others learn.

Being an open source contributor doesn't just mean writing code, either. You can help out by writing documentation, tests, or even giving feedback about the project (and yes - that includes giving feedback about the contribution process). Some of these contributions may be the most valuable to the project as a whole, because you're coming to the project with fresh eyes, so you can see the errors and assumptions that seasoned contributors have glossed over.

For more information, please read the see the contributing guide.

Philosophy

The space MetPy aims for is GEMPAK (and maybe NCL)-like functionality, in a way that plugs easily into the existing scientific Python ecosystem (numpy, scipy, matplotlib). So, if you take the average GEMPAK script for a weather map, you need to:

• read data
• calculate a derived field
• show on a map/skew-T

One of the benefits hoped to achieve over GEMPAK is to make it easier to use these routines for any meteorological Python application; this means making it easy to pull out the LCL calculation and just use that, or reuse the Skew-T with your own data code. MetPy also prides itself on being well-documented and well-tested, so that on-going maintenance is easily manageable.

The intended audience is that of GEMPAK: researchers, educators, and any one wanting to script up weather analysis. It doesn't even have to be scripting; all python meteorology tools are hoped to be able to benefit from MetPy. Conversely, it's hoped to be the meteorological equivalent of the audience of scipy/scikit-learn/skimage.

metpy's Issues

Color map that more closely matches the NWS mapping

Hey Ryan,

There doesn't seem to be a color map available that closely matches the level II image outputs from the NWS. Is this something that is easily modified?

Thanks!

skew.ax.fill_betweenx shading not shading entire region

When using the fill_betweenx() function per the sounding example ipython notebook, shading does not properly fill in CAPE/CIN regions on the profile. Examples on this vary based on the sounding.

Relevant code:
skew.ax.fill_betweenx(p, t, prof, where=t >= prof, facecolor='#5D8C53', alpha=0.7)

Screenshot:

Pint compatibility issues

Somehow we need to make the pint.Quantity instances be numpy subclasses. This would:

• Fix the problems with various numpy functions
• Fix matplotlib's unit support

In the best case, this gets contributed upstream to pint.

Save data out as transparent .gif

Is there a way to do this? I know about the savefig('foo.png') but it doesn't support .gif. That would be a neat feature.

Thanks!

Extra developer requirements

I think we should document the requirements for developers, such as what is required to build the docs.

sphinx => 1.3.1
sphinxcontrib-napoleon => 0.3.3

Maybe document it in @mmorello1 install reqs doc?

Fix aspect ratio for Skew-Ts

The following code:

from datetime import datetime, timedelta
from siphon.catalog import TDSCatalog
from siphon.ncss import NCSS
import numpy as np
import metpy.calc as mcalc
from metpy.units import units, concatenate
from metpy.plots import SkewT

units.define('degrees_north = 1 degree')
units.define('degrees_east = 1 degree')
unit_remap = dict(inches='inHg', Celsius='celsius')
def metpy_units_handler(vals, unit):
    arr = np.array(vals)
    if unit:
        unit = unit_remap.get(unit, unit)
        arr = arr * units(unit)
    return arr

cat = TDSCatalog('http://thredds-jumbo.unidata.ucar.edu/thredds/catalog/grib/HRRR/CONUS_3km/wrfprs/catalog.xml?dataset=grib/HRRR/CONUS_3km/wrfprs/Best')

best_ds = list(cat.datasets.values())[0]

ncss = NCSS(best_ds.access_urls['NetcdfSubset'])
ncss.unit_handler = metpy_units_handler

query = ncss.query().accept('csv')
query.lonlat_point(-105, 40).time(datetime.utcnow())
query.variables('Temperature_isobaric', 'Dewpoint_temperature_isobaric',
                'u-component_of_wind_isobaric', 'v-component_of_wind_isobaric')

data = ncss.get_data(query)

T = data['Temperature_isobaric'].to('degC')
Td = data['Dewpoint_temperature_isobaric'].to('degC')
p = data['vertCoord'].to('mbar')

import matplotlib.pyplot as plt
%matplotlib inline
fig = plt.figure(figsize=(9, 9))
skew = SkewT(fig=fig, rotation=36)
skew.plot(p, T, 'r')
skew.plot(p, Td, 'g')
skew.ax.set_ylim(1000, 100)
skew.ax.set_xlim(-20,40)
skew.plot_mixing_lines()
skew.plot_dry_adiabats()
skew.plot_moist_adiabats()

generates the following plot:

Notice the 0C dry adiabat crosses 700 mb directly above where -20C isotherm intersects 1000 mb.

However, changing the plotting code to:

fig = plt.figure(figsize=(9, 9))
skew = SkewT(fig=fig, rotation=36)
skew.plot(p, T, 'r')
skew.plot(p, Td, 'g')
skew.ax.set_ylim(1000, 100)
skew.ax.set_xlim(-50,40)
skew.plot_mixing_lines()
skew.plot_dry_adiabats()
skew.plot_moist_adiabats()

produces this plot:

where the same dry adiabat is now above where the -20C isotherm intersects 1000mb at ~650 mb. This difference seems to be a result of the physical distance between the isotherms changing. It may be nice if there was a way to change the figure width automatically if the xlim is increased or decreased to maintain both the rotation angle of the isotherms and the slope of the adiabats.

Demo of radar using VisPy

Based on vispy/vispy awesomeness in the notebook, we should be able to do a really cool PPI demo using:

• siphon to pull from radarserver
• metpy to parse raw file
• vispy to make PPI

Current test coverage

Current test coverage is only 55%, most of which comes from nexrad, which is covered at that level, but is ~2000 lines. Need to get:

• full coverage of basic.py
• more tests for nexrad.py
• Some kind of testing for skewt.py

Simplify skewT plotting

• Default construct figure if necessary
• Refactor API a bit to enable multipanel SkewT
• Add method default_plot()
  • Add mixing ratio lines, dry and moist adiabats
  • Set good default plot limits
    • x: -40 to 45 C
    • y: 1020 to 100 mb
  • Set solid black grid lines
• Use this to simplify examples
• Simplify non 3 May example further -- eliminate calculations
• Add example using simple wrapper for Wyoming to show shortest possible example

Add data interface for Wyoming Sounding archive

Data here: http://weather.uwyo.edu/upperair/sounding.html

• Will need to start creation of "standard" data object
• Need some solution to lookup numeric station ID from name

Xkcd example

Just because.

Skew-T subplots

There's no way to use the SkewT class as a subplot, since you can't (as far as I know) assign a projection to an Axes after creation. Need (essentially) as second constructor to do so. Might need to refactor the API a bit then, so that the ideal case of one constructor using the other can be achieved.

Documentation generation

Need to automatically generate documentation. Is Sphinx still the best option?

Could we do this from Travis? Where would we deploy? Do we need to exclude PRs?

`dry_lapse()` vs. `moist_lapse()`

The interfaces of dry_lapse() and moist_lapse() differ, when they really shouldn't. dry_lapse() takes a separate starting pressure, while moist_lapse() requires it to be the first pressure value given. If at all possible, these should be aligned better.

Python 2.6

Python 2.6 is not currently considered supported by MetPy, but the question is: should it be?

Currently, the only problem caused by Python 2.6 support is that the NEXRAD I/O code doesn't work on it. This is because you cannot pass a bytearray to the re (and maybe other) libraries. We could add some simple wrapper code to handle this on 2.6 (it just involves copying to a bytes() object). Is this worth it? Is 2.6 useful to potential users?

Examples

Is it better to have examples as notebooks, or as standalone scripts? Python files are easier to quickly run from the command line. Notebooks can be viewed with the images online simply using nbviewer. They can also readily be turned into Python (at least from the server) and be converted into an html page that could go into a user doc.

Full API documentation

Get some docstrings on the remaining public classes:

• SkewT
• Level2File
• Level3File

Chop down nexrad.py

nexrad.py has a lot of general data structures and utilities for binary I/O.

• Move out general utilties to promote re-use
• Base IOBuffer on the built-in io module.
• Add unit tests

Add standard data object

• This is probably just a dictionary of arrays right now.
• Utilize standard names from CF to make things work automatically.
  • Might need to provide short names or something like it too
• Provide unit capabilities as well
• At least mimic iris API

This could also just be replaced wholesale by IRIS if that's deemed better.

Add Level 2 Example

Moved over from a comment on my blog:

"
First, I would like to thank you for creating MetPy. It is awesome. I looked everywhere for a python module that would work in python 3 and read these level II files I have.

I'm still intermediate in Python and a lot of MetPy is a bit above my head. You posted an example of how to parse and display data from a level III file that was very helpful, but the variables available in level II files are different. Could you please post an example of how to do this with a level II file? That would be awesome.

Thanks!
"

NHC archive reader

Pull out I/O code from: https://github.com/Unidata/HurricaneTracker

CAPE/CIN calculation

Add a CAPE and CIN calculation:

CAPE/CIN Calculation Options:

• Source parcel

  • Surface
  • Most Unstable
  • Mixed Layer

• Mixed layer depth

  • I think we should allow user to specify it in meters or hPa and do the conversion internally

• Adiabat used

  • pseudo-adiabatic, liquid only
  • reversible, liquid only
  • pseudo-adiabatic with ice
  • reversible, with ice

• Virtual temperature correction

Technical Questions:

• If there is no LFC, is CIN 0 or integrated from surface to the top of the sounding?
• Are multiple crossings counted or just the first positive/negative areas in the integration bounds?
• Should virtual temperature be used for LFC/EL calculations?
• What should the default mixed layer thickness be?

"Standard" CAPE calculation according to Doswell and Rasmussen, 1994:

• Use the simple pseudo-adiabatic, pure parcel theory calculation for the lifted parcel ascent.
• Make the virtual temperature correction
• Pick the most unstable parcel in the lowest 300 mb (Bryan's code uses lowest 500 mb)

Reference Material

• George Bryan’s Code (FORTRAN)
• A Reevaluation of Ice–Liquid Water Potential Temperature
• The Effect of Neglecting the Virtual Temperature Correction on CAPE Calculations
• The Intricacies of Instabilities

Station Plots

Need to get station plots. First cut can be simple, no symbols:

• Simple station plots
• Add weather symbols

It would be nice to base this on: matplotlib/matplotlib#4063

Hodographs

Add a hodograph to the skewT:

• Add inset axes to the upper right
• Make configurable
• Separate plot call with U,V data

Add four panel plot example.

Add example based on Dr. Kevin Goebbert's four_panel.py script. Part of this involves reducing data access (which will be provided by NCSS wrappers in siphon). The other part is to add helpers in MetPy to improve colormap handling.

`get_with_steps` broken

Get with steps does not account for starting point when calculating end with the number of steps.

Currently bounds are:

arange(start, step * num_steps, step)

Should be:

arange(start, start + step * num_steps, step)

Don't have time to add tests and fix, so this is tracking it.

Gempak Colormaps

Add parser for gempak colormap files. Not sure how best to integrate this with matplotlib's colormap + norm machinery.

Add `viridis` colormap

I want to make sure we have this new colormap available regardless, so let's just add an internal copy from here:
https://github.com/BIDS/colormap/blob/master/option_d.py

Automatic field calculation

I have visions of being able to have certain fields automatically calculated on datasets. By providing a decorator for calculation functions (and utilizing unit handling), the calculations could specify its requirements, and a graph of relationships between the calculations could be generated. Using this graph, we could plot a course to calculate one variable from what's available.

Synoptic Examples

Add some examples of more advanced synoptic analysis:

• Ageostrophic winds, advection
• 500mb Vorticity advection
• Differential vorticity advection?
• Thermal wind
• Temperature advection
• Height tendencies

These are just ideas. Need to add any missing calculations for these terms.

Put MetPy version on main page of docs

Seems odd to have to dig to API to see the version. Need to make sure latest doesn't look terrible, though.

Documentation Improvements

• Denote returns from geostrophic_wind as u and v

Upper Air Parsing

Add parsing of TTAA/TTBB messages. This will facilitate replacing the netcdf-perl decoders TDS uses for the (unhosted) upper air netcdf files.

Clean-up colormaps

ctables.py is way too big. Several things need to happen:

• Remove less-useful tables
• Implement reading of simple colortables from file instead of embracing full matplotlib capabilities
• Add registry for our colortables

Improve NEXRAD API

Right now Level2File and Level3File are really low-level APIs; they provide complete implementation of their respective specifications, but they don't provide a nice interface to the user (just a bunch of attributes with ad-hoc data structures).

Should:

• Add NetCDF-like API (variables, attributes, etc.)
• Provide CF-Radial-like data model
• Add layer with CDM-like sweeps, etc.

Question whether these should be separate objects, or just add functions to the existing classes (possibly using a mix-in).

Add `environment.yml`

This would codify the process of installing using conda.

Using logging rather than warnings

See here: https://docs.python.org/3/howto/logging.html

Add class to help with creating colorbars

Creating a colorbar that sits along an edge of a CartoPy axes is non-trivial. We should provide a class to do so. This is the crux of the correct code:

def resize_colobar(event):
    # Tell matplotlib to re-draw everything, so that we can get
    # the correct location from get_position.
    plt.draw()

    posn = ax.get_position()
    colorbar_ax.set_position([posn.x0 + posn.width + 0.01, posn.y0,
                             0.04, axpos.height])

fig.canvas.mpl_connect('resize_event', resize_colobar)

Taken from here: https://stackoverflow.com/questions/30030328/correct-placement-of-colorbar-relative-to-geo-axes-cartopy

Problems with `heat_index`

• Relative humidity are 0 to 100. Elsewhere we use [0, 1]. We should make them consistent.
• Returning masked values breaks units

Default colormaps for parameters

In the spirit of viridis and:

https://www.youtube.com/watch?v=XjHzLUnHeM0

Maybe we should develop some solid default colormaps for meteorological parameters?

Clarify scope for this project

From the face of it, MetPy looks very ambitious: "reading, visualizing and performing calculations with weather data" covers a lot of ground. On the other hand, it's scope is currently much more limited.

Some questions that I tried and failed to figure out from the README/docs:

• What are your goals for this project? What is the unmet need?
• Who is your intended audience?
• How do you see MetPy fitting into the broader Python ecosystem?

Tracks (and other attributes) on NIDS broken on Python 3

So zip on Python 3 returns an iterator (zip object) on Python 3. So this broke reading NIDS products that rely on that behavior. I observed this when trying to plot storm track information from NIDS products.

Just need to look over all the uses of zip in the NIDS reader to make sure that when a list is needed (i.e. not when iterating over the results) we explicitly call list. Also would be a good time to start adding some tests that actually check the results of parsing. Not suggesting exhaustive tests, just making sure there's a test that catches each bad use of zip before we change the code.

Automate release process

Based on scipy 2015 talks, it would be possible and really nice to automate the release process. General concept:

• Only on tagged builds
• Use issues/PRs to make release notes
• PyPI
  • Build:
    • src tarball
    • universal wheel
  • upload
• Build conda package
  • Build linux package
  • use conda convert to make for all platforms (since we're pure python)
  • upload

units in get_wind_components

In metpy.calc.basic.get_wind_components, the docstring says to input wind direction in degrees. However, looking at the corrections in revision #35, the calculation gets rid of the degrees to radians. When I made a plot using u,v after passing in degrees, it showed that the wind direction is indeed quite off from what is expected.

METAR parsing

Add METAR parsing to facilitate replacing the old netcdf-perl decoders for NOAAPORT METAR messages.

Support for units (internal array type)

We need to work out a plan for built-in use of units. Rough sketch of the benefits:

• Automatic appropriate labels for plots
• Better sanity checking on calculations

There exists the quantities package that provides a great numpy subclass that provides unit-handling. Unfortunately, this does not work with masked arrays, which are a separate numpy subclass. Also, I'm not sure we want to rely on this as an external dependency.

I lean towards making an internal fork of quantities that gives us masked values, unit support, as well as support for arbitrary metadata.

Add 'automagic' plotting interface module

Add a separate module for plotting that does more data management, provided the right data object/interface is being used:

• Use whatever data interface we define in data module
  • Variable naming, etc. done by data object
  • Units as done there
• Store data object, and then use this instead of passing data in
• Start with skewT
  • Keep name, just subclass from basic plotting module
  • Wrap methods from base class, just pass in data
  • default_lines() (?) method probably needs better name, but could add barbs and hodograph automatically
  • Add example

Split `get_speed_dir`

The only reason to have both in one function now is because they may be commonly used together. We should split them.

List of potential calculations to support

This is just a running list of potential calculations to support.

Calculations removed as they have been broken out into separate issues.

Objective Analysis / Smoothing

- [ ] Horizontal smoothing using normally distributed weights
- [ ] Smooth a grid using a 5-point smoother
- [ ] Smooth a grid using a 9-point smoother
- [ ] Circular aperture smoother
- [ ] Rectangular aperture smoother
- [ ] Cressman smoother

Meteorological Calculations:

- [ ] Absolute vorticity

• Advection
  - [ ] ageostrophic wind
  - [ ] Circulation (for cross sections)
• Coriolis force at each point
• Divergence
• Equivalent potential temperature in Kelvin
  - [ ] Fosberg index, also called Fire Weather Index.
  - [ ] Frontogenesis
  - [ ] Geopotential (GP) to Height (Z)
• Geostrophic wind
  - [ ] Inertial advective wind
• Isallobaric wind
  - [ ] Potential vorticity
  - [ ] Q-vectors
  - [ ] Relative humidity
  - [ ] Richardson stability number in a layer
  - [ ] Rossby number
• Shear deformation
• Stretching deformation
  - [ ] Solve the Poisson equation of a forcing function
  - [ ] Thermal wind
• Total deformation
  - [ ] Vorticity

Cloud Physics (cloudphysics)

Thermodynamics (thermo)

- [ ] Wet bulb potential temperature in Celsius
- [ ] Wet bulb temperature in Kelvin
- [ ] Thermodynamic stability within a layer (lapse rate)

• Temperature of the lifting condensation level
  - [ ] Saturated equivalent potential temperature in Kelvin
• Mixing ratio
• Parcel temperature in Kelvin along a moist adiabat
• Pressure of the lifting condensation level
• Potential temperature in Kelvin
  - [ ] Brunt-Vaisala frequency squared in a layer

Basic (basic)

• Wind direction from components

Tools (tools)

Kinematics (kinematics)

Turbulence (turbulence)

Radiation (radiation)

Severe (severe, indicies, derived ?)

- [ ] WINDEX (index for microburst potential)

Synoptic?

Skew-T barbs y limits

Since clipping is disabled on wind barbs on the skewT, they do not follow the set Y limits on the axes. This completely messes up the plot.