tobifinn / torch-assimilate Goto Github PK

View Code? Open in Web Editor NEW

Python package based on PyTorch and Dask for efficient and parallelized data assimilation of weather model data and observations on GPUs.

License: GNU General Public License v3.0

Python 99.54% Makefile 0.46%

torch-assimilate's Introduction

torch-assimilate

docs
pipeline
coverage
quality
package

Data assimilation based on PyTorch

torch-assimilate is a python package for data assimilation of meteorological observations into numerical weather model data.

This package is constructed for efficient and parallelized data assimilation in python. The central entity of this package are the data assimilation methods optimized in PyTorch¹. Furthermore, some algorithms are parallelized with dask² and allow a distributed computing with many cores. For data in- and output xarray³ is used. Originally, this package is designed for offline data assimilation via io-operations.

In the future, different data assimilation methods, like ensemble Kalman filters, particle filters, variational data assimilation and neural assimilation will be added.

This package is developed for a PhD-thesis about nonlinear methods in coupled data assimilation at the "Universität Hamburg", "Universität Bonn" and the Max Planck Institute for Meteorology.

Installation

We highly recommend to create a virtual environment for this package to prevent package collisions. At the moment this package is only available at pypi-test.

This package is programmed in python 3.6 and should be working with all python versions > 3.3. Additional requirements are pytorch and xarray.

PyTorch needs to be additionally installed because of different possible versions. In following CPU-based installation for linux is shown.

via conda (recommended):

git clone [email protected]:tobifinn/torch-assimilate.git
cd torch-assimilate
conda env create -f environment.yml
source activate pytassim
conda install pytorch torchvision cpuonly -c pytorch
pip install .

via pip (latest pypi-test):

pip install --index-url https://test.pypi.org/simple/ torch-assimilate
pip install torch==1.6.0+cpu torchvision==0.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html