jkreklow / radproc

Radproc - RADOLAN composite processing, analysis and data exchange with ArcGIS

Radproc - A GIS-compatible Python-Package for automated RADOLAN Composite Processing and Analysis

Radproc is an open source Python library intended to facilitate precipitation data processing and analysis for GIS-users. It provides functions for processing, analysis and export of RADOLAN (Radar Online Adjustment) composites and rain gauge data in MR90 format. The German Weather Service (DWD) provides the RADOLAN RW composites for free in the Climate Data Center but the data processing represents a big challenge for many potential users. Radproc's goal is to lower the barrier for using these data, especially in conjunction with ArcGIS. Therefore, radproc provides an automated ArcGIS-compatible data processing workflow based on pandas DataFrames and HDF5. Moreover, radproc's arcgis module includes a collection of functions for data exchange between pandas and ArcGIS.

Note

Please cite radproc as Kreklow, J. (2018): Radproc - A GIS-compatible Python-Package for automated RADOLAN Composite Processing and Analysis. Zenodo. http://doi.org/10.5281/zenodo.1313701

Radproc's Main Features

Raw Data Processing

• Support for the reanalyzed RADOLAN products RW (60 min), YW and RY (both 5 min. resolution)
• Automatically reading in all binary RADOLAN composites from a predefined directory structure
• Optionally clipping the composites to a study area in order to reduce data size
• Default data structure: Monthly pandas DataFrames with full support for time series analysis and spatial location of each pixel
• Efficient data storage in HDF5 format with fast data access and optional data compression
• Easy downsampling of time series
• Reading in DWD rain gauge data in MR90 format into the same data structure as RADOLAN.

Data Exchange with ArcGIS

• Export of single RADOLAN composites or analysis results into projected raster datasets or ESRI grids for your study area
• Export of all DataFrame rows into raster datasets in a new file geodatabase, optionally including several statistics rasters
• Import of dbf tables (stand-alone or attribute tables of feature classes) into pandas DataFrames
• Joining DataFrame columns to attribute tables
• Extended value extraction from rasters to points (optionally including the eight surrounding cells)
• Extended zonal statistics

Analysis

• Calculation of precipitation sums for arbitrary periods of time
• Heavy rainfall analysis, e.g. identification, counting and export of rainfall intervals exceeding defined thresholds
• Data quality assessment
• Comparison of RADOLAN and rain gauge data
• In preparation: Erosivity analysis, e.g. calculation of monthly, seasonal or annual R-factors

Documentation

The full documentation for the latest radproc version is available at http://www.pgweb.uni-hannover.de/

Most of the docs are also hosted at https://radproc.readthedocs.io which will provide support for docs of older versions in future, but unfortunately Readthedocs doesn't seem to support sphinx autodocs for the arcpy module which is not hosted at PyPI. Consequently, the docs for the radproc.arcgis module are missing here. If you have any idea how to fix this issue, please let me know.

radproc's Issues

does it work with python 3

I was trying to use radproc but I can not able to install it using pypi for python 3.10 (setup.py error cannot find the "requirements.txt") and even if I use the source code with python3, there are a lot of errors. I assume this was made for python2 and has not been updated. Is there a plan to support python3?

Result comparison with Radolan Java Library

For the http://wiki.bitplan.com/index.php/Sprinkler automatic sprinkler control software project I was looking for a source of raindata history that
is fit to be used in this application. I purposely did not want to use a local rain sensor.

In the past few days I came up with the idea to use RADOLAN data for the purpose. Since i couldn't find any Open Source Java Library and i assumed that accessing python would be too much effort I decided to port Jonny Schäfers golang library https://gitlab.cs.fau.de/since/radolan to Java see

Radolan Java Library

Since i do not have any Meteorology background I do not know how well the port works regarding the accuracy and validity of the results.
I'd like to get some feedback where things need to be fixed and I'd appreciate help (even to the point to recruit new committers).

For my Sprinkler app I'll get into a local view of just a few square km around a given location. The "big picture" is not so interesting for me personally.
That is why I believe that involving people from the meteorology community is necessary to make the Java library a useful asset for interested people.

If you comment, give feedback , create issues, help or otherwise participate that would be very much appreciated.

Try using GitHub Pages

Maybe only that one line

radproc/docs/Makefile

Line 9 in bba7178

BUILDDIR = _build

needs to be changed to .., or maybe the _build folder contents can be moved up one level with an additional line in the makefile, so that the index.html is located in docs already. That is the crucial point for Pages.