pyswmm / swmm-python

python wrappers for the Stormwater Management Model (SWMM5)

Getting started

• Project Website: www.pyswmm.org

• Official PySWMM Documentation

• PySWMM YouTube Channel

• PySWMM Example Bundles

Introducing the SWAG STORE! All Proceeds go toward the hosting/service fees related to maintaining the PySWMM Project!!! Get yourself a hoodie or coffee cup!

• PySWMM SWAG Store

🆘Do you need HELP?🆘

GitHub Discussions to answer support questions related to PySWMM.

Cite our Paper

McDonnell, Bryant E., Ratliff, Katherine M., Tryby, Michael E., Wu, Jennifer Jia Xin, & Mullapudi, Abhiram. (2020). PySWMM: The Python Interface to Stormwater Management Model (SWMM). Journal of Open Source Software, 5(52), 2292, https://doi.org/10.21105/joss.02292

YouTube Training Vidoes

Setting a manhole inflow during a running simulation!

Overview

PySWMM is a Python language software package for the creation, manipulation, and study of the structure, dynamics, and function of complex networks.

With PySWMM you can load and manipulate USEPA Stormwater Management Models. With the development of PySWMM, control algorithms can now be developed exclusively in Python which allows the use of functions and objects as well as storing and tracking hydraulic trends for control actions.

As of version v1.1.0, PySWMM includes new features to process metadata and timeseries stored in SWMM binary output file.

Who uses PySWMM?

PySWMM is used by engineers, modelers, and researchers who want to streamline stormwater modeling optimization, controls, and post-processing results.

Goals

PySWMM is intended to provide

• tools for the study of the structure and dynamics within USEPA SWMM5,
• a standard programming interface and graph implementation that is suitable for many applications,
• a rapid development environment for collaborative, multidisciplinary projects,
• an interface to USEPA SWMM5,
• development and implementation of control logic outside of native EPA-SWMM Controls,
• methods for users to establish their own node inflows,
• a coding interface to binary output files,
• new modeling possibilities for the SWMM5 Community.

Install

Get the latest version of PySWMM from PyPI See the Quick Guide!

$ pip install pyswmm

As of version 1.3.1, pyswmm can be installed with specific versions of the SWMM engine ranging from 5.1.14 to 5.2.4 using pip extras:

$ pip install pyswmm[swmm5.2.4]

SWMM and Python Compatibility Table

pyswmm version	swmm-toolkit compatibility (SWMM engine)	python compatibility
1.0.0 - 1.1.1	0.8.2 (SWMM 5.1.13)	3.6 - 3.9
1.2.0 - 1.5.0	0.9.1 - 0.15.0 (SWMM 5.1.14 - 5.2.4) Note: 0.11.0 (SWMM 5.2.0) only supported on windows	3.7 - 3.12

Usage

A quick example that steps through a simulation:

Examples:

See the Latte Example

from pyswmm import Simulation, Nodes, Links

with Simulation(r'Example1.inp') as sim:
    Node21 = Nodes(sim)["21"]
    print("Invert Elevation: {}". format(Node21.invert_elevation))

    Link15 = Links(sim)['15']
    print("Outlet Node ID: {}".format(Link15.outlet_node))

    # Launch a simulation!
    for ind, step in enumerate(sim):
        if ind % 100 == 0:
            print(sim.current_time,",",round(sim.percent_complete*100),"%",\
                  Node21.depth, Link15.flow)

Opening a SWMM binary output file and accessing model metadata and timeseries.

from pyswmm import Output, SubcatchSeries, NodeSeries, LinkSeries, SystemSeries

with Output('model.out') as out:
    print(len(out.subcatchments))
    print(len(out.nodes))
    print(len(out.links))
    print(out.version)

    sub_ts = SubcatchSeries(out)['S1'].runoff_rate
    node_ts = NodeSeries(out)['J1'].invert_depth
    link_ts = LinkSeries(out)['C2'].flow_rate
    sys_ts = SystemSeries(out).rainfall

Bugs

Our issue tracker is at https://github.com/pyswmm/pyswmm/issues. Please report any bugs that you find. Or, even better, fork the repository on GitHub and create a pull request. All changes are welcome, big or small, and we will help you make the pull request if you are new to git (just ask on the issue).

Contributing

Please check out our Wiki https://github.com/pyswmm/pyswmm/wiki for more information on contributing, including an Author Contribution Checklist.

License

Distributed with a BSD2 license; see LICENSE.txt:

Copyright (C) 2014-2024 PySWMM Developers
Community-Owned See AUTHORS and CITATION.cff

swmm-python's Issues

Unit tests for Cannonical API

Legacy API is comprised of the original API functions for out of the box SWMM.

Support for unicode filenames

Passing a filename containing unicode characters causes ERROR 303.

Bug reported by pyswmm user.

Python 3.8 Wheel

Is there a python 3.8 wheel available for this project? (Windows)

Update swmm-toolkit to wrap OWA/SWMM Toolkit API

swmm-toolkit currently wraps the US EPA SWMM API. I am looking for two volunteers interested in learning SWIG to work on wrapping the OWA/SWMM Toolkit API and ultimately create a package for use by pyswmm.

The SWIG generated wrappers will standardize the packaging of the swmm-solver and help accelerate pyswmm. This task involves learning SWIG and modifying the solver.i interface description file. We will be reusing the package structure and scikit-build based build already established in the swmm-toolkit package.

If you have 24 hours to spare over the next two weeks and are interested in learning SWIG this is an excellent opportunity. If you have been involved in maintaining pyswmm ctypes based wrapper learning SWIG will be essential for maintaining these packages going forward.

I'm looking forward to working with you. Let's do this!

Strip prefixes and suffixes from function names

This task involves removing these prefixes using the SWIG %rename.

The modules in this package are contained within the swmm namespace. Consequently, prefixes such as swmm are no longer required in the python wrapper. Removing them will improve conformance of the function names with python PEP 8 naming conventions.

Add typemap for double * as return

Automate wheel packaging on CI build workers

Move wheel build from OWA/swmm repository to swmm-toolkit package.

Move Project Dev Zone

Planning on Archiving this project and moving development over here:

https://github.com/SWMM-Project/swmm-python

Add support for metadata (parameter/attribute names and units)

Add support for metadata describing swmm parameters and attributes. Create API for providing units.

Configure wheel build to run tests

Add a build step to run tests

Create virtual environment
Install pytest
Install wheel
run pytest

Turn on CI

I need an administrator to turn on Appveyor for this repo. Thanks!

Check license for OpenMP

May need to include license when bundled into wheel. Check license and comply with terms.

Add typemap for int * as return

error when running python setup.py file

@michaeltryby

Hi! Totally new to SWIG in general, so apology in advance if I asked something really simple/silly, but I tried following the readme to setup swmm-python, but I'm stuck with the following errors:

swmm/toolkit/toolkit_wrap.c(3310): error C2065: 'SWMM_ProjectHandle': undeclared identifier
swmm/toolkit/toolkit_wrap.c(3310): error C2065: 'arg1': undeclared identifier
swmm/toolkit/toolkit_wrap.c(3310): error C2059: syntax error: ')'
swmm/toolkit/toolkit_wrap.c(3311): error C2065: 'SWMM_ProjectHandle': undeclared identifier
swmm/toolkit/toolkit_wrap.c(3311): error C2146: syntax error: missing ';' before identifier 'retval1'
swmm/toolkit/toolkit_wrap.c(3311): error C2065: 'retval1': undeclared identifier
swmm/toolkit/toolkit_wrap.c(3316): error C2065: 'retval1': undeclared identifier
swmm/toolkit/toolkit_wrap.c(3316): warning C4047: '=': 'int' differs in levels of indirection from 'void *'
swmm/toolkit/toolkit_wrap.c(3317): error C2065: 'arg1': undeclared identifier
swmm/toolkit/toolkit_wrap.c(3317): error C2065: 'retval1': undeclared identifier
swmm/toolkit/toolkit_wrap.c(3317): warning C4047: '=': 'int' differs in levels of indirection from 'int *'
swmm/toolkit/toolkit_wrap.c(3320): warning C4013: 'swmm_alloc_project' undefined; assuming extern returning int
swmm/toolkit/toolkit_wrap.c(3320): error C2065: 'arg1': undeclared identifier
swmm/toolkit/toolkit_wrap.c(3327): error C2065: 'retval1': undeclared identifier
swmm/toolkit/toolkit_wrap.c(3337): error C2065: 'SWMM_ProjectHandle': undeclared identifier
swmm/toolkit/toolkit_wrap.c(3337): error C2065: 'arg1': undeclared identifier
swmm/toolkit/toolkit_wrap.c(3337): error C2059: syntax error: ')'
swmm/toolkit/toolkit_wrap.c(3338): error C2065: 'SWMM_ProjectHandle': undeclared identifier

These are the steps I used:

1. Clone owa swmm develop branch
2. Built the swmm following the instructions from owa swmm
3. Copied header files (swmm5.h and toolkitAPI.h) to include folder within \swmm-python\toolkit\swmm\toolkit
4. Copied swmm5.dll, swmm5.exp, swmm5.lib to lib folder within \swmm-python\toolkit\swmm\toolkit
5. tried python setup.py build but got the following errors

Is SWIG trying to find the SWMM_ProjectHandle within the header files?

Target multiple versions of Python

Python Windows Platform Compilers:
Python 2.7 - Visual Studio 9 2008
Python 3.3, 3.4 - Visual Studio 10 2010
Python 3.5, 3.6, 3.7 - Visual Studio 14 2015

Current SWMM benchmark builds on Visual Studio 15 2017
PyQGIS targets Python 3.6
Should we support Python 2.7, 3.3, 3.4?

Update swmm-toolkit to use aenum 3.0.0

Aenum has been updated to 3.0.0 and it breaks the enum definitions in our package.

Establish new python package "swmm-pandas"

Purpose of this package would be to marshal swmm data into and out of pandas data frames.

import broken after merging PR61

Hej swmm-python team,
i had been compiling swmm-python from source , using the dev-workflow branch (PR61) yesterday.
Everything worked fine.
Now after the merge, compiling from the dev branch, I get an import error when trying to import the output module.

I didnt change anything in my code , so i fear that some little error occured from the merge?

Best regards :)
Andre

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/swmm_toolkit-0.8.1-py3.6-linux-x86_64.egg/swmm/toolkit/output.py", line 20, in swig_import_helper
    return importlib.import_module(mname)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 658, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 922, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: libswmm-output.so: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 7, in <module>
    from make_result_geojson import get_result_geojson
  File "/app/make_result_geojson.py", line 4, in <module>
    from swmm.toolkit import output, shared_enum, output_metadata
  File "/usr/local/lib/python3.6/dist-packages/swmm_toolkit-0.8.1-py3.6-linux-x86_64.egg/swmm/toolkit/output.py", line 23, in <module>
    _output = swig_import_helper()
  File "/usr/local/lib/python3.6/dist-packages/swmm_toolkit-0.8.1-py3.6-linux-x86_64.egg/swmm/toolkit/output.py", line 22, in swig_import_helper
    return importlib.import_module('_output')
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_output'

Trouble with swmm-toolkit install

I am getting various errors after installing on windows and linux. I also tried building and installing from source and still get the same errors. Not sure if I am missing something.

Windows

after pip install swmm-toolkit executing from swmm.toolkit import output, output_enum yields:

>>> from swmm.toolkit import output, output_enum
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "~\Miniconda\envs\swmm\lib\site-packages\swmm\toolkit\output.py", line 18, in <module>
    from . import _output
ImportError: DLL load failed while importing _output: The specified module could not be found.

Linux

after pip install swmm-toolkit executing from swmm.toolkit import output, output_enum yields:

>>> from swmm.toolkit import output, output_enum
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'output_enum' from 'swmm.toolkit' (~/miniconda3/envs/swmm/lib/python3.9/site-packages/swmm/toolkit/__init__.py)

Prepare for release swmm-toolkit v0.9.0

OWA SWMM is ready for a release. Need to update wrapper with OWA SWMM v5.1.14. Any other open issues we should address prior to finalizing?

Add typemap for array struct * as return? (discussion first)

get_system_attribute function seg default

int EXPORT_OUT_API SMO_getSystemAttribute(SMO_Handle p_handle, int periodIndex,
    SMO_systemAttribute attr, float **outValueArray, int *length)
//
//  Purpose: For the system at given time, get a particular attribute.
//
{
    int     errorcode = 0;
    float   temp;
    data_t *p_data;

    p_data = (data_t *)p_handle;

    if (p_data == NULL)
        errorcode = -1;
    else if (periodIndex < 0 || periodIndex >= p_data->Nperiods)
        errorcode = 422;
    else {
        // don't need to loop since there's only one system
        temp = getSystemValue(p_data, periodIndex, attr);

        *outValueArray = &temp;
        *length        = 1;
    }

    return set_error(p_data->error_handle, errorcode);
}

Reviewing this section of code from swmm solver, I think float temp should be float* temp instead because outValueArray is a double pointer.

@michaeltryby @cbuahin

This error is not caught because there isn't an unit test for get_system_attribute

Make swmm-toolkit package citable

swmm.output SMO_sys enum type is missing from python 37 whl

I just installed the latest swmm.output whl (swmm.output-0.4.0.dev0-cp37-cp37m-win_amd64.whl), it install correctly, but failed at runtime. It is missing the SMO_sys enum type, I saw SMO_link, SMO_node, and SMO_subcatch, but didn't see SMO_sys type

Add typemap for array double * as return

Proposal to relicense swmm-toolkit from CC0 to CC BY 4.0

If you are a contributor or user of the packages found here please consider the implications of relicensing on your work and feel free to comment here. Thank you for supporting our project!

uploading wheel to pypi

Is there any plan to upload the last packages to the Python Package Index? The package already has an entry. This will facilitate testing and integration with other packages.

swmm.toolkit open input file error

Here is the input file:
https://github.com/mel-meng/SewerAnalysis/blob/master/bugs/data/swmm.toolkit.bug.inp

Running the same code on the example1.inp works fine.

Here is how to reproduce the bug:
https://github.com/mel-meng/SewerAnalysis/blob/master/bugs/swwm.toolkit%20error%20in%20input%20file.ipynb

I installed the latest release python 3.7 64bit *.whl.

thanks.

Mel

import of binary fails

Hi,

Importing pyswmm fails with the following error:

In [1]: import pyswmm
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-2215b2c7feca> in <module>
----> 1 import pyswmm

~/miniforge3/envs/hidrologia/lib/python3.8/site-packages/pyswmm/__init__.py in <module>
      9 
     10 # Local imports
---> 11 from pyswmm.links import Link, Links
     12 from pyswmm.lidcontrols import LidControls, LidControl
     13 from pyswmm.lidgroups import LidGroups, LidGroup, LidUnit

~/miniforge3/envs/hidrologia/lib/python3.8/site-packages/pyswmm/links.py in <module>
     10 
     11 # Local imports
---> 12 from pyswmm.swmm5 import PYSWMMException
     13 from pyswmm.toolkitapi import LinkParams, LinkResults, LinkPollut, LinkType, ObjectType
     14 

~/miniforge3/envs/hidrologia/lib/python3.8/site-packages/pyswmm/swmm5.py in <module>
     18 
     19 # Third party imports
---> 20 from swmm.toolkit import solver
     21 
     22 # Local imports

~/miniforge3/envs/hidrologia/lib/python3.8/site-packages/swmm/toolkit/solver.py in <module>
     16 # Import the low-level C/C++ module
     17 if __package__ or "." in __name__:
---> 18     from . import _solver
     19 else:
     20     import _solver

ImportError: libswmm5.so: cannot open shared object file: No such file or directory

Ubuntu 20.10, in a conda env, pyswmm 1.0.1 installed with pip.

Make python a required dependency

Add REQUIRED to find_package() python call.

That way we can fail fast for a specific reason and make the build a little easier to troubleshoot.

Fix aenum issue

Aenum, a dependency to swmm-toolkit, was recently updated. The update broke a feature we rely on.

swmm-toolkit v0.8.1 needs aenum v2.2.4 but this version is not specified in the setup.py file.

The dev branch contains bug fix for shared_enum.py to get around the problem in aenum 3.0.

Automate library and python package build

The package build is currently performed manually. It requires the swmm-toolkit and swmm-output libraries to be built a priori. The required .dll, .lib, and .h files are then manually copied to the source directory for the module being built. Then the python packaging system is used to build the swig wrapper, link with the library, and create the package. This process needs to be automated for easy packaging and distribution of the python modules.

Packaging and distribution

networkx is an example of a python library that combines multiple namespace packages into one distribution package. Perhaps it can serve as a template for swmm-python.

SWMM output interface and plugin infrastructure

Update nrtest-swmm to use latest version of swmm-toolkit

We are currently using an older version of swmm-toolkit for regression testing. This issue involves refactoring nrtest-swmm to use latest version of swmm-toolkit.

change namespace from toolkit to swmm

Update nrtest-swmm to use latest swmm-toolkit and add it to build

Binary incompatibility libomp on Mac OS

On Mac OS we currently satisfy libomp system requirement using a precompiled binary from brew. This binary targets Mac OS v10.15 but needs to target v10.9 for binary compatibility with Python.

As a result the wheel doesn’t work because libomp doesn’t load on older versions of Mac OS.

Need to figure out how best to accomplish this.

build .whl for python 3.9

Write up workaround for UTF-8 support on Windows

Build wheels on CI server

Configure code coverage

Add Doc strings to interface file

sample code for output

I was able to download and install the following,
swmm.output-0.4.0.dev1-cp37-cp37m-win_amd64.whl

Not sure how to get started, tried the following from the test and got an error.

Add Readme

Should describe what this is for.

readme.rst

rst so that python package manager renders it nicely.

Higher Level OutputAPI

I have occasionally had a need for fast, programmatic access to the binary SWMM output files, and as a python lover have turned to modules written by bemcdonnell or timcera.

However, lately I've been yearning for a more full fledged outputAPI package that integrates with the numpy/pandas ecosystem, takes care of the boilerplate parsing of output meta data to allow for a more object-oriented experience, and perhaps provides some statistical functionality as well. To my knowledge, there isn't really anything our there like that yet (is there?). I can see there was a solid attempt from the developers of the stale SWMM python gui applicaiton. But the file no longer integrates with the current swmm-python, and requires some overhaul.

I really like the implementation of swmm-python because it uses C for very fast access to the binary file (orders of magnitude faster than the pure python implementation), but the existing API is too low-level for my liking. I'd like to develop something new (probably inspired by what was developed for the SWMM gui), but I am not sure if it deserves it's own repo, or perhaps should be integrated with this one. Thoughts on this?

I think whatever I'd develop would be heavily dependent on the swmm toolkit output swig api provided by this library. How mature to feel the api is here? Do you see any major overhaul on the horizon? I don't want to develop anything based on an API that has incoming breaking changes.

I'd appreciate your thoughts on this idea.

Thanks!

Move relocate.zsh script into swmm-toolkit package

Wheels are Python binary packages. They are meant to be installed directly on an end users machine bypassing the package build process. When delivering wheels containing python extensions that have library dependencies care must be taken to relocate dependent libraries into the wheel otherwise the user will experience runtime errors because the libraries haven't been installed on their machine.

MacOS does not come with native support for OpenMP. A custom script relocate.zsh has been written to relocate the OpenMP library into the swmm-toolkit wheel. The script is currently located here:

"https://github.com/SWMM-Project/swmm-solver/blob/dev-wrapper/tools/before-nrtest.zsh"

Except the script doesn't actually belong there. It should be moved to the swmm-toolkit package itself.

The relocate.zsh script should be moved to swmm-toolkit/tools and the CMakeLists.txt should be updated to call the script from its new location.

version 1.0.1 cannot read inp files with "õ, ä, ö, ü" in the name or path

Describe the bug
When opening input files where the letters "õ", "ä", "ö" or "ü" are in the name of the file or in the path (catalogue) pyswmm returns error message "ERROR 303: cannot open input file."

The issue was solved when I downgraded back to version 0.6.2

To Reproduce
For example file name "Söderhamn.inp" resulted that error message while "Soderhamn.inp" resulted expected behaviour (no errors when opening the file)

Expected behavior
For example the following path throws the error:
C:\Users\somename\models\Söderhamn

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Windows 10
• Python Version Python 3.8
• PySWMM Version 1.0.1

Additional context
Add any other context about the problem here.

Linux wheels are NOT manylinux compliant

When I try to import solver from swmm-toolkit in linux, I get missing .dll file error.

Python 3.6.9 (default, Nov  7 2019, 10:44:02)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyswmm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/pyswmm/__init__.py", line 11, in <module>
    from pyswmm.links import Link, Links
  File "/usr/local/lib/python3.6/dist-packages/pyswmm/links.py", line 12, in <module>
    from pyswmm.swmm5 import PYSWMMException
  File "/usr/local/lib/python3.6/dist-packages/pyswmm/swmm5.py", line 20, in <module>
    from swmm.toolkit import solver
  File "/usr/local/lib/python3.6/dist-packages/swmm/toolkit/solver.py", line 33, in <module>
    _solver = swig_import_helper()
  File "/usr/local/lib/python3.6/dist-packages/swmm/toolkit/solver.py", line 29, in swig_import_helper
    _mod = imp.load_module('_solver', fp, pathname, description)
  File "/usr/lib/python3.6/imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/lib/python3.6/imp.py", line 343, in load_dynamic
    return _load(spec)
ImportError: libswmm5.so: cannot open shared object file: No such file or directory

Update nrtest-swmm with mincdd_compare

Discussing need for a more "rational" comparison criteria for benchmark results. epanet_mincdd_compare can be found here:

https://github.com/EPANET-Project/epanet-python/blob/dev/nrtest-epanet/nrtest_epanet/__init__.py

So if we wanted to run the mincdd we would need to add it to nrtest-swmm and rebuild. Then in the swmm-nrtestsuite repo update the test description files to call the new comparison routine. Then update the helper scripts with the tolerance we want probably 2 or 3 decimal places of precision.