metoffice / catnip

Climate Analysis Tool: Now In Python

CATNIP

Climate Analysis Tools: Now In Python

This library is a collection of routines to make frequently used climate data analysis and visualisation tasks in Iris easier and quicker to write.

Complete documentation can be found at: CATNIP Docs

Requirements

Please ensure you have the following dependencies installed:

iris numpy matplotlib six scipy

Usage

To get started make sure all the requirements are installed, then clone the repository and set your Python environment to point to your copy, for example:

git clone [email protected]:MetOffice/CATNIP.git

export PYTHONPATH=<path/to/CATNIP/lib>

CATNIP is also available via pypi and conda-forge:

pip install mo-catnip

conda install -c conda-forge mo-catnip

Contributing

If you want to contribute to CATNIP be sure to review the contribution guidelines.

Contact

For any suggestion, contribution and bug/error reporting, contact us at [email protected]

License

BSD-3 License

catnip's Issues

Apply black to whole code base

Apply black and add test for black compliance to pytest. Check carefully if it breaks any doctests

Define and add the requirement.txt for dependencies

list all of the libraries that your project depends on in a file called requirement.txt and specify versions of those packages that are known to work. One line for each package e.g.

iris==2.2.0
improver==0.10.0

feature test add coord system

Feature test add time coords cats

Testing: doc tests for visualisation.vector_plot

Write a doc tests for plot functions

feature test remove forecast coordinates

update documentation pages with recent changes

documentation pages need updating and rebuilding to reflect the recent changes made to the function regrouping.

feature test umstash to pystash

Testing: test data

for the Doc tests, test data should reside in the repo.
a separate directory "test data" should contain the data and change the paths in every doc test accordingly

Feature test convert from um stamp

create target cube for interpolation from a rotated pole grid

Given a 2D field on a rotated pole grid, calculate a regular lat-lon grid that has the same resolution but on a regular lat-lon grid. There should be two options:

• return a cube on a grid that completely encloses the rotated pole grid
• return a cube on a grid entirely inside the rotated pole grid

reinstate preparation.extract_rot_cube

#18 removed ASCEND and to make sure that the doctests pass extract_rot_cube was also removed from preparation.py

This should be reintroduced using a different approach which does not use ASCEND

add unittest of analysis.test_calculate_dewpoint

feature test precisD2

new sections in documentation

Write new "how to" sections

test sort cube

cdsapi retrieval helper

Description

A new function to help populate fields within the cdsapi retrieve() function. Mainly to make dealing with date ranges quicker. The function would take a from and to date, and options to specify variables and time interval (frequency). This could be extended to use dask for parallel retrievals.

Example syntax

VARS = ['10m_u_component_of_wind', '10m_v_component_of_wind']
getera5(from='2019-05-23', to='2019-05-30', var=VARS, freq='H')

or for a parallel version

FROMS=['2019-05-23', '2008-12-25']
TOS = ['2019-05-30', '2009-01-05']
VARS = ['10m_u_component_of_wind', '10m_v_component_of_wind']
getera5(from=FROMS, to=TOS, var=VARS, freq='H')

to retrieve different time means for the same period

getera5(from=['2019-05-23', '2019-05-23'], to= ['2019-05-30', '2019-05-30'], 
        var=VARS, freq=['H', 'M'])

Notes

• In the case of multiple retrievals, the function would assume that the same VARS are retrieved in each case. I can't particularly see a use case where different vars would be wanted for different time periods within code relating to one project...
• Dependencies: pandas to deal with date range wrangling
• I'd suggest sticking with the pandas frequency aliases: https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#timeseries-offset-aliases
• Need to consider what other args might be need to facilitate the full range of cdsapi retrievals

Make precisyy and precisD2 internal functions

create .gitignore file and removing files which should not be under version control

.pyc files (and others) should not be under version control and to make sure they are not added, we need a .gitignore file. We should also consider deleting any .pyc files etc that are already under verison control

feature test common timeperiod

Remove code which uses ASCEND

as we cannot source ASCEND from any public repo, we should remove the function extract_rot_cube from preparation.py and then at a later date replace this with an approach which doesn't use ascend.shape

Parallelisation helper function

With reference to George Ford's ASKS presentation (attached below), I wonder if there is something that can be incorporated here...

Need to check that his suggested method is still up-to-date with respect to Dask etc. but I think the principal still holds... could be used in #17

20181107_GFord_parallel_python.pptx

feature test precisyy

Feature test convert to un stamp

feature add aux unrotated coords

Add return values to the function and update the unittests

Data access module

Implement a separate data access module i.e. config.py

Further documentation update

Updates required to documentation after the 20/05/20 meeting:

• update reference to PEP8 and flake8 instead of pylint
• in the Some good Git best practices section move up the Use branches to first bullet point
• ...

feature test um file list

MO specific documentation section

identifying and adding any MO specific instructions to a separate section in the documentation titled

• "Met Office specific instructions"

feature test add bounds

Functions regrouping

Functions need to regroup and filenames needs to change accordingly.

add unittest of analysis.test_ci_interval

feature test add aux unrotated coords

feature add coord system

add return values to the function coord_system and update the unittests

feature add time coord cats

add return values to the function add_time_coord_cats()

Follow pep8 rules on naming conventions

PEP8 states:
https://www.python.org/dev/peps/pep-0008/#naming-conventions
"The naming conventions of Python's library are a bit of a mess, so we'll never get this completely consistent -- nevertheless, here are the currently recommended naming standards. New modules and packages (including third party frameworks) should be written to these standards, but where an existing library has a different style, internal consistency is preferred."

This issue will review our existing code and where it does not meet the naming conventions change the code to align it with those standards

feature get date range

Develop unittests for utils.get_date_range()

unittests should include all possible positive and negative tests

Define and add setup.py file

To enable people to install CATNIP we need to create a setup.py file and use the setuptools package to tell Python how to install the code.

add unittest of analysis.test_linear_regress

External library/package dependecies

There are currently dependencies in the code to some external libraries/packages as mentioned in : #14 (comment) we need to list these and then decide what is the best way to include them. So far I have found these:

• Iris
• Improve
• Ascend

But let me know if you know of anymore!

There are some suggestions in this git article

Feature: test framework

Implement a doc test to be run with single command in command line

Documentation

Get the basic set of documentation of the current function.
This issue should be solved after getting regrouping of functions

Add unitest for analysis.set_regrider

feature add bounds

Add return value to the function add_bounds and update the unittests

Add unittest for analysis.test_regrid_to_target

feature remove forecast coordinates

Add return value to the function remove_forecast_coords() and update the unittests

feature test date chunk

Implement unittest for utils.date_chunk

Directories hierarchy

Make a overall structure of library.
for reference: see iris repo

Testing: git hooks

Make use of git hooks

feature test rim remove