umep-dev / supy

SUEWS that speaks Python

SuPy

This project is merged into SUEWS.

---

SUEWS that speaks Python

Installation

SuPy requires 64-bit python 3.7+ and can be installed with pip in command line prompt:

python3 -m pip install supy --upgrade

Quick Demo

Once installed, supy can be quickly started to get SUEWS simulations done:

import supy as sp
import matplotlib.pyplot as plt

#load sample data
df_state_init, df_forcing = sp.load_SampleData()
grid = df_state_init.index[0]

#run supy/SUEWS simulation
df_output, df_state_end = sp.run_supy(df_forcing, df_state_init)

#plot results and save figure
res_plot = df_output.SUEWS.loc[grid, ['QN', 'QF', 'QS', 'QE', 'QH']]
ax=res_plot.loc['2012 6 4':'2012 6 6'].resample('30T').mean().plot()
plt.show()
ax.figure.savefig('sample_plot.png')

The above code will produce a plot of surface energy balance components as follows:

Tutorial

Please check out more SuPy tutorials here!

Installation of the development version of SuPy

The development version supy includes both the supy wrapper and its kernel supy-driver - the calculation kernel written in Fortran.

All below is supposed to be in a directory named supy-dev - which however you can change.

1. Get both the source code of SUEWS and SuPy

the development version of SuPy.

# get the source code of SUEWS and SuPy
git clone --recurse-submodules [email protected]:UMEP-dev/SUEWS.git

git clone [email protected]:UMEP-dev/SuPy.git

2. Set up the conda environment

conda env create -f SuPy/env.yml
conda activate supy

3. Install the dev version of SuPy

pip install -e SuPy/src

4. Compile the kernel supy-driver

cd SUEWS/supy-driver
make dev

Here you can check if supy-driver is installed correctly:

pip show supy-driver

note the location field in the output, which should point to the local supy-driver directory.

Note: after installing the development version of supy-driver, you might get the warning about incompatibility, which you can ignore and carry on. However, in case of any issue, please report it here.

5. Check the version info of installed supy

pip show supy

note the location field in the output, which should point to the local supy directory.

supy's Issues

Setting different irrigation settings for different years

Dear Ting

Since last weekend most of my issues with running the model have been solved. I did come across a new issue, which could come up for other researches in the future as well regarding irrigation settings.

I am currently modelling irrigation over a period of ten years in Vancouver. Over this time period there have been different water use restrictions in place depending on the severity of drought and heat that year, e.g. watering only on odd days, watering only during certain hours or, for short periods of time, complete watering bans.

I am wondering if there is a way to set up multiple watering restriction scenarios in SuPy current state?

note: I have found the options, to set beginning and end of irrigation, fractions of properties that use irrigation and to turn on and of irrigation depending on weekdays, but this only applies to the entire simulation period and not for a specific short term period.

Kind regards

[Docs]data-structure/df_state

Issue

I have come across several df_state parameters which are not consistent with what is on the documentation:
https://supy.readthedocs.io/en/latest/data-structure/df_state.html

'sfr' should be 'sfr_surf'
'soilstorecap' should be 'soilstorecap_surf'
'soilstore_id' should be 'soilstore_surf'

SuPy versions I am using:

supy: 2022.9.22
supy_driver: 2021a14.post3

Thank you,
Russell

Links

source doc
RTD page

epw file header not updated

gen_epw for input files

Would it be possible to add functionality to convert standard input SUEWS/UMEP met forcing files to epw-files? To me it looks like a small adjustment of gen_epw could make this happen.

ERA5 pressure data

Hi!

As a finger practice, I've download and generate standard SuPy forcing from ERA5 data for a given location but for different elevations (2m, 10m, 50m and 100m) using SuPy's utility function (gen_forcing_era5). I get a pressure data that increases with elevation. It's either the heatwave or a bug of some sort. :)

Environment info:
macOS 11.5.1
supy: 2022.4.7
supy_driver: 2021a5

Here is what I see (hope it goes through):

Let me know if additional info is needed.
Thank you and stay hydrated!
Csilla

Number limitation to SUEWS_txt-file codes

Describe the Issue
When running sp.init_supy() it seems to not recognize codes longer than 10 digits.

so for a code like

2019353657265

it only look for
2019353657

see code:

df_state_init = sp.init_supy(path_runcontrol)

2022-04-20 15:37:24,825 - SuPy - INFO - All cache cleared.
2022-04-20 15:37:24,898 - SuPy - ERROR - Entries missing from SUEWS_NonVeg.txt
Traceback (most recent call last):
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\supy_load.py", line 835, in build_code_df
df_code = df_code0.loc[list_code, list_keys]
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 925, in getitem
return self._getitem_tuple(key)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1100, in _getitem_tuple
return self._getitem_lowerdim(tup)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 822, in _getitem_lowerdim
return self._getitem_nested_tuple(tup)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 906, in _getitem_nested_tuple
obj = getattr(obj, self.name)._getitem_axis(key, axis=axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1153, in _getitem_axis
return self._getitem_iterable(key, axis=axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1093, in _getitem_iterable
keyarr, indexer = self._get_listlike_indexer(key, axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1314, in _get_listlike_indexer
self._validate_read_indexer(keyarr, indexer, axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1374, in _validate_read_indexer
raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Int64Index([-2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648],\n dtype='int64', name='Code')] are in the [index]"
2022-04-20 15:37:24,900 - SuPy - ERROR - missing code:
[-2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648, -2147483648]
Traceback (most recent call last):
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\supy_load.py", line 835, in build_code_df
df_code = df_code0.loc[list_code, list_keys]
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 925, in getitem
return self._getitem_tuple(key)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1100, in _getitem_tuple
return self._getitem_lowerdim(tup)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 822, in _getitem_lowerdim
return self._getitem_nested_tuple(tup)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 906, in _getitem_nested_tuple
obj = getattr(obj, self.name)._getitem_axis(key, axis=axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1153, in _getitem_axis
return self._getitem_iterable(key, axis=axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1093, in _getitem_iterable
keyarr, indexer = self._get_listlike_indexer(key, axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1314, in _get_listlike_indexer
self._validate_read_indexer(keyarr, indexer, axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1374, in _validate_read_indexer
raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Int64Index([-2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648],\n dtype='int64', name='Code')] are in the [index]"
2022-04-20 15:37:24,901 - SuPy - ERROR - df_code0 has been dumped into C:\temp\suewstests\df_code0.pkl for debugging!
Traceback (most recent call last):
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\supy_load.py", line 835, in build_code_df
df_code = df_code0.loc[list_code, list_keys]
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 925, in getitem
return self._getitem_tuple(key)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1100, in _getitem_tuple
return self._getitem_lowerdim(tup)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 822, in _getitem_lowerdim
return self._getitem_nested_tuple(tup)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 906, in _getitem_nested_tuple
obj = getattr(obj, self.name)._getitem_axis(key, axis=axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1153, in _getitem_axis
return self._getitem_iterable(key, axis=axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1093, in _getitem_iterable
keyarr, indexer = self._get_listlike_indexer(key, axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1314, in _get_listlike_indexer
self._validate_read_indexer(keyarr, indexer, axis)
File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1374, in _validate_read_indexer
raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Int64Index([-2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,\n -2147483648, -2147483648, -2147483648, -2147483648, -2147483648],\n dtype='int64', name='Code')] are in the [index]"

If i reduce the amount of digits to below 10 it works. Perhaps it is a problem in the sp.init_supy() only?

Best,
Oskar

Generate version info from git tag/commit

similarly as SUEWS/supy-driver, use git describe to get the version info.

Windows fatal exception: code 0xc0000028-Error2 -when Running SUEWS Advance in QGIS

Hello UMEP community,

shortly after running advanced succesfully with one month meteodata and 121 grid cells, I needed to extend my area to 1x1km , 400 grids with 50m resolution. I did the analysis again and run the SUEWS prepare succesfully, later for the Advanced QGIS keep crashing on different computers. This is the error I have, never had this one before:)
is it maybe about the area lenght, what is the recommended lenght of the area and resolution for SUEWS?

my meteo data is for a month, I even tried with a week to reduce the calculation time and effort.

---

Python Stack Trace
Windows fatal exception: code 0xc0000028
 
Current thread 0x00001b60 (most recent call first):
  File "C:\Users\ozgey\AppData\Roaming\Python\Python39\site-packages\supy\_run.py", line 121 in suews_cal_tstep_multi
    res_suews_tstep_multi = sd.suews_cal_multitsteps(**dict_input)
  File "C:\Users\ozgey\AppData\Roaming\Python\Python39\site-packages\supy\_run.py", line 370 in 
    suews_cal_tstep_multi(input_grid, df_forcing)
  File "C:\Users\ozgey\AppData\Roaming\Python\Python39\site-packages\supy\_run.py", line 369 in run_supy_ser
    list_res = [
  File "C:\Users\ozgey\AppData\Roaming\Python\Python39\site-packages\supy\_supy_module.py", line 370 in run_supy
    df_output, df_state_final = run_supy_ser(
  File "C:\Users/ozgey/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\suewsmodel\suews_wrapper.py", line 68 in wrapper
    df_output, df_state_final = sp.run_supy(df_forcing, df_state_init, check_input=True, serial_mode=True)
  File "C:\Users/ozgey/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\SUEWS\suews.py", line 314 in start_progress
    suews_wrapper.wrapper(self.model_dir, self.iface)
  File "C:\Users/ozgey/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\SUEWS\suews.py", line 158 in run
    self.dlg.exec_()
  File "C:\Users/ozgey/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\UMEP\UMEP.py", line 440 in SUEWS_advanced
    sg.run()
 
 
Stack Trace
No stack trace is available.

QGIS Info
QGIS Version: 3.26.1-Buenos Aires
QGIS code revision: b609df9ed4d
Compiled against Qt: 5.15.3
Running against Qt: 5.15.3
Compiled against GDAL: 3.5.1
Running against GDAL: 3.5.1

Screenshots

Additional context
the input data created with UMEP-SUEWS in QGIS:
ber1km_2019_data_60.txt
SUEWS_WithinGridWaterDist.txt
int_2019_data_60.txt
SUEWS_AnthropogenicEmission.txt
SUEWS_BiogenCO2.txt
SUEWS_Conductance.txt
SUEWS_ESTMCoefficients.txt
SUEWS_Irrigation.txt
SUEWS_NonVeg.txt
SUEWS_OHMCoefficients.txt
SUEWS_Profiles.txt
SUEWS_SiteSelect.txt
SUEWS_Snow.txt
SUEWS_Soil.txt
SUEWS_Veg.txt
SUEWS_Water.txt

"Invalid cross-device link" error for the command "suews-convert"

Describe the bug
When I used the command "suews-convert" in order to make the input files for SUEWS v2017b compatible for SUEWS v2020a (which I knew I should not do so because v2017b was not included in the version options for this command but I just tried), an error occurred during the process "2019b --> 2020a" as the following:

To Reproduce

File "/home/yingqiz/anaconda3/lib/python3.8/site-packages/supy/util/_converter.py", line 243, in SUEWS_Converter_single convert_utf8(file_dst)
File "/home/yingqiz/anaconda3/lib/python3.8/site-packages/supy/util/_converter.py", line 478, in convert_utf8 path_dst.rename(path_src)
File "/home/yingqiz/anaconda3/lib/python3.8/pathlib.py", line 1366, in rename self._accessor.rename(self, target)
OSError: [Errno 18] Invalid cross-device link: '/tmp/tmp4a5e9382/out-UTF8.txt' -> '/mnt/c/SUEWS/SUEWS_2020a/Input_K/InitialConditionsBe1_2007.nml'

Expected behavior
There seems to be some problems related to path_dst.rename(path_src) in supy/util/_converter.py

I saw someone reported that "os.rename only works if source and destination are on the same file system. You should use shutil.move instead." when I was looking for solution.

So I opened site-packages/supy/util/_converter.py and added
import shutil
at the beginning to shutil which is already in the python standard library, and I modified line 478 from "path_dst.rename()" to
shutil.move(path_dst, path_src)

This seems working and I obtained the new converted .nml files, and initial condition and site description .txt files to my specified repository.

However, a new problem arose which was reported in problems.txt as the following:

Problem: RunControl.nml

ERROR! Program stopped: Something wrong in the rows of the file

There seems to be something to do with the RunControl.nml. Next, I checked this file, and compared it with the converted RunControl.nml file with the template RunControl.nml presented in SUEWS_2020a. They actually differed in content and indentation.

Anyway, I just tried to force SUEWS v2020a with the replacement of forcing meteorological data, and it worked. Later, I just modified the initial data, parameters and the RunControl.nml files based on the template files one by one, which worked efficiently in this situation where I was trying to run SUEWS v2020a with SUEWS v2017b data.

Version and platform (please complete the following information):

• SUEWS version: 2020a
• OS: Ubuntu 20.04

Additional context
Just sharing my experience in case someone encounters the "Invalid cross-device link" error.
And in case someone's trying to convert the SUEWS v2017b to SUEWS v2020a with "suews-convert" commad, this seems not working.

SuPy: SUEWS that speaks Python — SuPy 2021.1.22 documentation

SuPy: SUEWS that speaks Python — SuPy 2021.1.22 documentation

https://supy.readthedocs.io/en/latest/

Converted files not working properly

3) Converted files not working properly

When using the converted SUEWS-output from all three above mentioned methods, the same error message appear

2021-02-22 16:51:49,871 - SuPy - INFO - All cache cleared.
2021-02-22 16:51:49,999 - SuPy - ERROR - Entries missing from SUEWS_Profiles.txt
Traceback (most recent call last):
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py", line 843, in build_code_df
    df_code = df_code0.loc[list_code, list_keys]
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 889, in __getitem__
    return self._getitem_tuple(key)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1060, in _getitem_tuple
    return self._getitem_lowerdim(tup)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 791, in _getitem_lowerdim
    return self._getitem_nested_tuple(tup)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 865, in _getitem_nested_tuple
    obj = getattr(obj, self.name)._getitem_axis(key, axis=axis)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1113, in _getitem_axis
    return self._getitem_iterable(key, axis=axis)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1053, in _getitem_iterable
    keyarr, indexer = self._get_listlike_indexer(key, axis, raise_missing=False)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1266, in _get_listlike_indexer
    self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1308, in _validate_read_indexer
    raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Int64Index([801, 801], dtype='int64', name='Code')] are in the [index]"
2021-02-22 16:51:50,000 - SuPy - ERROR - missing code:
 [801, 801]
Traceback (most recent call last):
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py", line 843, in build_code_df
    df_code = df_code0.loc[list_code, list_keys]
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 889, in __getitem__
    return self._getitem_tuple(key)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1060, in _getitem_tuple
    return self._getitem_lowerdim(tup)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 791, in _getitem_lowerdim
    return self._getitem_nested_tuple(tup)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 865, in _getitem_nested_tuple
    obj = getattr(obj, self.name)._getitem_axis(key, axis=axis)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1113, in _getitem_axis
    return self._getitem_iterable(key, axis=axis)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1053, in _getitem_iterable
    keyarr, indexer = self._get_listlike_indexer(key, axis, raise_missing=False)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1266, in _get_listlike_indexer
    self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1308, in _validate_read_indexer
    raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Int64Index([801, 801], dtype='int64', name='Code')] are in the [index]"
2021-02-22 16:51:50,001 - SuPy - ERROR - missing key:
 ['P', 'o', 'p', 'P', 'r', 'o', 'f', 'W', 'D']
Traceback (most recent call last):
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py", line 843, in build_code_df
    df_code = df_code0.loc[list_code, list_keys]
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 889, in __getitem__
    return self._getitem_tuple(key)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1060, in _getitem_tuple
    return self._getitem_lowerdim(tup)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 791, in _getitem_lowerdim
    return self._getitem_nested_tuple(tup)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 865, in _getitem_nested_tuple
    obj = getattr(obj, self.name)._getitem_axis(key, axis=axis)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1113, in _getitem_axis
    return self._getitem_iterable(key, axis=axis)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1053, in _getitem_iterable
    keyarr, indexer = self._get_listlike_indexer(key, axis, raise_missing=False)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1266, in _get_listlike_indexer
    self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1308, in _validate_read_indexer
    raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Int64Index([801, 801], dtype='int64', name='Code')] are in the [index]"
2021-02-22 16:51:50,003 - SuPy - ERROR - `df_code0` has been dumped into C:\Script\SaraSUEWSTG\df_code0.pkl for debugging!
Traceback (most recent call last):
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py", line 843, in build_code_df
    df_code = df_code0.loc[list_code, list_keys]
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 889, in __getitem__
    return self._getitem_tuple(key)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1060, in _getitem_tuple
    return self._getitem_lowerdim(tup)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 791, in _getitem_lowerdim
    return self._getitem_nested_tuple(tup)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 865, in _getitem_nested_tuple
    obj = getattr(obj, self.name)._getitem_axis(key, axis=axis)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1113, in _getitem_axis
    return self._getitem_iterable(key, axis=axis)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1053, in _getitem_iterable
    keyarr, indexer = self._get_listlike_indexer(key, axis, raise_missing=False)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1266, in _get_listlike_indexer
    self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
  File "C:\Users\xbacos\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py", line 1308, in _validate_read_indexer
    raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Int64Index([801, 801], dtype='int64', name='Code')] are in the [index]"
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-25-973e41b826d6> in <module>
      3 path_runcontrol_umepcmd = "C:\Script\SaraSUEWSTG\output_new\RunControl.nml"
      4 path_runcontrol = 'C:\Script\SuPy\src\supy\sample_run\RunControl.nml'
----> 5 df_state_init = sp.init_supy(path_runcontrol_umep)
      6 

~\Anaconda3\envs\supy_env\lib\site-packages\supy\_supy_module.py in init_supy(path_init, force_reload, check_input)
    101         if path_init_x.suffix == ".nml":
    102             # SUEWS `RunControl.nml`:
--> 103             df_state_init = load_InitialCond_grid_df(
    104                 path_init_x, force_reload=force_reload
    105             )

~\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py in load_InitialCond_grid_df(path_runcontrol, force_reload)
   1589     # load base df of InitialCond
   1590     logger_supy.debug("loading base df_init...")
-> 1591     df_init = load_SUEWS_InitialCond_df(path_runcontrol)
   1592 
   1593     # add Initial Condition variables from namelist file

~\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py in load_SUEWS_InitialCond_df(path_runcontrol)
   1325     path_input = path_runcontrol.parent / dict_ModConfig["fileinputpath"]
   1326     logger_supy.debug("loading df_gridSurfaceChar")
-> 1327     df_gridSurfaceChar = load_SUEWS_SurfaceChar_df(path_input)
   1328     # df_gridSurfaceChar.to_pickle("df_gridSurfaceChar.pkl")
   1329     # only use the first year of each grid as base for initial conditions

~\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py in load_SUEWS_SurfaceChar_df(path_input)
   1098 @functools.lru_cache(maxsize=16)
   1099 def load_SUEWS_SurfaceChar_df(path_input):
-> 1100     df_gridSurfaceChar_exp = gen_df_gridSurfaceChar_exp(path_input)
   1101     dict_var_ndim = {
   1102         "ahprof_24hr": (24, 2),

~\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py in gen_df_gridSurfaceChar_exp(path_input)
   1082 @functools.lru_cache(maxsize=16)
   1083 def gen_df_gridSurfaceChar_exp(path_input):
-> 1084     df_siteselect_exp = gen_df_siteselect_exp(path_input)
   1085     dict_var_tuple = exp_dict_full(dict_var2SiteSelect)
   1086     df_gridSurfaceChar_exp = pd.concat(

~\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py in gen_df_siteselect_exp(path_input)
    986     # nested code: code -> code -> value
    987     df_code_exp_nest = pd.concat(
--> 988         [build_code_exp_df(path_input, code) for code in list(dict_code_var_nest)],
    989         axis=1,
    990     )

~\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py in <listcomp>(.0)
    986     # nested code: code -> code -> value
    987     df_code_exp_nest = pd.concat(
--> 988         [build_code_exp_df(path_input, code) for code in list(dict_code_var_nest)],
    989         axis=1,
    990     )

~\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py in build_code_exp_df(path_input, code_x)
    935     # print(list_code_x)
    936     df_all_code_x = pd.concat(
--> 937         [build_code_df(code, path_input, df_base_x) for code in list_code_x], axis=1
    938     )
    939     df_all_code_x = pd.concat([df_all_code_x], axis=1, keys=[code_x])

~\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py in <listcomp>(.0)
    935     # print(list_code_x)
    936     df_all_code_x = pd.concat(
--> 937         [build_code_df(code, path_input, df_base_x) for code in list_code_x], axis=1
    938     )
    939     df_all_code_x = pd.concat([df_all_code_x], axis=1, keys=[code_x])

~\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py in build_code_df(code, path_input, df_base)
    858         )
    859 
--> 860         raise e
    861 
    862     df_code.index = df_base.index

~\Anaconda3\envs\supy_env\lib\site-packages\supy\_load.py in build_code_df(code, path_input, df_base)
    841 
    842     try:
--> 843         df_code = df_code0.loc[list_code, list_keys]
    844     except Exception as e:
    845         logger_supy.exception(f"Entries missing from {lib_code}")

~\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py in __getitem__(self, key)
    887                     # AttributeError for IntervalTree get_value
    888                     return self.obj._get_value(*key, takeable=self._takeable)
--> 889             return self._getitem_tuple(key)
    890         else:
    891             # we by definition only have the 0th axis

~\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py in _getitem_tuple(self, tup)
   1058     def _getitem_tuple(self, tup: Tuple):
   1059         with suppress(IndexingError):
-> 1060             return self._getitem_lowerdim(tup)
   1061 
   1062         # no multi-index, so validate all of the indexers

~\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py in _getitem_lowerdim(self, tup)
    789         # we may have a nested tuples indexer here
    790         if self._is_nested_tuple_indexer(tup):
--> 791             return self._getitem_nested_tuple(tup)
    792 
    793         # we maybe be using a tuple to represent multiple dimensions here

~\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py in _getitem_nested_tuple(self, tup)
    863 
    864             current_ndim = obj.ndim
--> 865             obj = getattr(obj, self.name)._getitem_axis(key, axis=axis)
    866             axis += 1
    867 

~\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py in _getitem_axis(self, key, axis)
   1111                     raise ValueError("Cannot index with multidimensional key")
   1112 
-> 1113                 return self._getitem_iterable(key, axis=axis)
   1114 
   1115             # nested tuple slicing

~\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py in _getitem_iterable(self, key, axis)
   1051 
   1052         # A collection of keys
-> 1053         keyarr, indexer = self._get_listlike_indexer(key, axis, raise_missing=False)
   1054         return self.obj._reindex_with_indexers(
   1055             {axis: [keyarr, indexer]}, copy=True, allow_dups=True

~\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py in _get_listlike_indexer(self, key, axis, raise_missing)
   1264             keyarr, indexer, new_indexer = ax._reindex_non_unique(keyarr)
   1265 
-> 1266         self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
   1267         return keyarr, indexer
   1268 

~\Anaconda3\envs\supy_env\lib\site-packages\pandas\core\indexing.py in _validate_read_indexer(self, key, indexer, axis, raise_missing)
   1306             if missing == len(indexer):
   1307                 axis_name = self.obj._get_axis_name(axis)
-> 1308                 raise KeyError(f"None of [{key}] are in the [{axis_name}]")
   1309 
   1310             ax = self.obj._get_axis(axis)

KeyError: "None of [Int64Index([801, 801], dtype='int64', name='Code')] are in the [index]"

I did try to replace SUEWS_Profiles.txt with a SUEWS_Profiles.txt from the sample data, but then a similar error message- SuPy - ERROR - Entries missing from SUEWS_BiogenCO2.txt appeared. I looked in BiogenCO2.txt and found that it only had columns

1	2	3	4	5	6	7	8	9
800	alpha	beta	theta	alpha_enh	beta_enh	resp_a	resp_b	min_respi

I have not checked any other .txt files

Originally posted by @gusbacos in UMEP-dev/UMEP#237 (comment)

[SuPy Testing] Impact Studies Using SuPy - NameError: name 'df_res_alb_test' is not defined

Running Tutorial "Impact Studies Using SuPy" using same Ubuntu 20.04 OS etc as described in earlier posts.
Asciidoc output attached.
https://supy.readthedocs.io/en/latest/tutorial/impact-studies.html

Choose results of July 2012 for analysis:

df_res_alb_test_july = df_res_alb_test.SUEWS.unstack(0).loc["2012 7"]
df_res_alb_T2_stat = df_res_alb_test_july.T2.describe()
df_res_alb_T2_diff = df_res_alb_T2_stat.transform(
    lambda x: x - df_res_alb_T2_stat.iloc[:, 0]
)
df_res_alb_T2_diff.columns = list_alb_test - list_alb_test[0]

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Input In [11], in <module>
      1 # choose results of July 2012 for analysis
----> 2 df_res_alb_test_july = df_res_alb_test.SUEWS.unstack(0).loc["2012 7"]
      3 df_res_alb_T2_stat = df_res_alb_test_july.T2.describe()
      4 df_res_alb_T2_diff = df_res_alb_T2_stat.transform(
      5     lambda x: x - df_res_alb_T2_stat.iloc[:, 0]
      6 )

NameError: name 'df_res_alb_test' is not defined

Error_Impact Studies Using SuPy.zip

improve the sample data management using `pooch`

This package seems very useful for managing sample data:

https://github.com/fatiando/pooch

Timezone correction for ERA5

@hamidrezaomidvar commented on Jun 15, 2020, 12:19 PM UTC:

Here are my thoughts to correct the forcing data generated using ERA5 for timezone:

Depending on positive or negative time difference, we can modify gen_forcing_era5(...) to change the start or end date (1 day before or after). Then after downloading the files, we change df_forcing variable to correct the time. Specifically, we need to add a function between these lines:

    df_forcing = grp_grid.apply(
        lambda df: format_df_forcing(
            df.reset_index(["latitude", "longitude"], drop=True)
        )
    )

##### Here we need to modify the time of df_forcing

    # save results as SUEWS met input files
    list_fn = save_forcing_era5(df_forcing, dir_save)

This issue was moved by sunt05 from sunt05/SuPy#157.

utility for manipulating externally sourced data for underlying grids

@sunt05 commented on Jun 17, 2020, 5:00 PM UTC:

we need a tool to merge/manipulate externally sourced data for creating SiteSelect or df_state:

• re-gridding, re-projection.
• land cover.
• population density.
• building/tree info.

(more to add ...).

This issue was moved by sunt05 from sunt05/SuPy#159.

docs couldn't be built at RTD

Describe the Issue
see log here: https://readthedocs.org/projects/supy/builds/14248424/

provide an easy way for users to report their input settings/files and version info when reporting an issue

to help users report issues with correct version info and specific input settings, some utilities should be implemented inside supy:

• save df_forcing and df_init as pickle files
• include version info - both that of supy and driver/kernel - as a separate file.

ValueError: Columns must be same length as key

Describe the Issue
Hi Ting,
The above error occurs when running supy in a Linux environment. The example is for loading SampleData, but it also occurs with my own data, e.g. with df_state_init = sp.init_supy(rctrl). supy runs fine with the same input data in a Windows environment.
Location

Screenshots
feige@met-ws1:~/py$ python3
Python 3.8.10 (default, Jun 2 2021, 10:49:15)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import supy as sp
df_state_init, df_forcing = sp.load_SampleData()

error message:

/users/staff/geogra/feige/.local/lib/python3.8/site-packages/supy/_load.py:1361: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider using pd.concat instead.  To get a de-fragmented frame, use `newframe = frame.copy()`
  df_init[(k, "0")] = dict_InitCond_default[k]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/supy/_supy_module.py", line 261, in load_SampleData
    df_state_init = init_supy(path_runcontrol, force_reload=False)
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/supy/_supy_module.py", line 103, in init_supy
    df_state_init = load_InitialCond_grid_df(
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/supy/_load.py", line 1591, in load_InitialCond_grid_df
    df_init = load_SUEWS_InitialCond_df(path_runcontrol)
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/supy/_load.py", line 1369, in load_SUEWS_InitialCond_df
    df_init[("waterstate", "0")] = df_init["waterdepth"]
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/pandas/core/frame.py", line 3599, in __setitem__
    self._set_item_frame_value(key, value)
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/pandas/core/frame.py", line 3724, in _set_item_frame_value
    raise ValueError("Columns must be same length as key")
ValueError: Columns must be same length as key

Additional context
Thanks for checking
Christian

Latest version os Suews requesting for SUEWS_SPARTACUS.nml and gridlayout files

Hello,
Before now I have got no problem with running SuPy, but with the introduction of version supy: 2022.9.22, it keeps requesting for a SUEWS_SPARTACUS file as well as a gridlayout file. Could anyone please clarify this new additional files and what they provides. I could see sample SUEWS spartacus file here (https://suews.readthedocs.io/en/latest/input_files/SS_input/SS_input.html#suews-spartacus-nml) but I can find as sample gridlaout file.

here is the error

1. /scratch/20202815/SUEWS/ICUC_run/prepare/SUEWS_SPARTACUS.nml does not exists!
2. SuPy - ERROR - /scratch/20202815/SUEWS_Dublin/prepare/GridLayout.nml does not exists!
   Thank you

What package versions are required for newest supy version?

Is this documented anywhere?

Wrong first time step if date is in sp.util.gen_forcing_era5()

@biglimp commented on May 19, 2020, 9:08 AM UTC:

Describe the Issue
If the below inputs are used, the first timestep is 2012-12-01T23:00:00 last timestep is 2012-12-31T23:00:00. I am nut sure if ERA5 data is previous hour or on the hour. If it is previous hour the output should shift one hour forward in time.

lat = 51.518758196192444
lon = -0.10994087999367352
Start = 2012-01-01T00:00:00
End = 2012-12-31T00:00:00

Location
Used the function through UMEP

Screenshots

Additional context

This issue was moved by sunt05 from sunt05/SuPy#151.

RuntimeError: SUEWS kernel error!

I've got a RuntimeError: SUEWS kernel error!

It was still working in August, so maybe it is related to something in the recent upgrade(?). I will check it out.
In any case, I am running SuPy v2022.9.22 on macOS 11.5.1, using python 3.8.
The zipped files are attached.

I hope I provided all the necessary info.
Thank you for the help in advance!

Cs.

supy_debug-3e70c486.zip

Error when Running SUEWS Advance in QGIS

Describe the bug

Dear SUEWS Community,

I want to use the SUEWS for my study area in Berlin. Before that I am trying to run a test version so I know that my data is accurate. So it is only 500mx500m area with 121 Grids. SUEWS prepared worked, but for SUEWS advanced either my QGIS crashed or it gave an

Error:
Unable to allocate 460. MiB for an array with shape (105120,573) and data type float64

To Reproduce
the following output files are from SUEWS Prepare. So in the Advance window, as an input, the folder containing these files should be given and run.

SUEWS-Advance settings are as recommended:

SUEWS Prepare Output files:
SUEWS_AnthropogenicEmission.txt
SUEWS_BiogenCO2.txt
SUEWS_Conductance.txt
SUEWS_ESTMCoefficients.txt
SUEWS_Irrigation.txt
SUEWS_NonVeg.txt
SUEWS_OHMCoefficients.txt
SUEWS_Profiles.txt
SUEWS_SiteSelect.txt
SUEWS_Snow.txt
SUEWS_Soil.txt
SUEWS_Veg.txt
SUEWS_Water.txt
SUEWS_WithinGridWaterDist.txt
ber_2019_data_60.txt

Screenshots
ERROR screen

Additional context
I did create my own land cover according to the SUEWS fractions, all rasters I had are with 1m resolution.

• System Info
  CPU Type: x86_64
  Kernel Type: winnt
  Kernel Version: 10.0.22000

Version and platform (please complete the following information):

• SUEWS version: v2020a
• OS: Windows 11
• QGIS Version: 3.16.16 Hannover

check_forcing: relax ranges of variables?

Describe the Issue
I am testing supy over Mexico City, using input from ERA5. When running applying check_forcing, I got this warning:
"pres" should be between [800, 1300] but "758.7" is found at 2020-04-30 23:05:00

Perhaps the range should be a bit more relaxed, as such low pressure values might be valid, especially for high-altitude regions?

Moreover, as you can see from the timeseries below, seems that only one timestep is reported, even though the whole time series has pressure values < 800 hPa? Not sure if this behaviour is intended?

Location
Referring to the following:

SuPy/src/supy/checker_rules_indiv.json

Lines 996 to 1004 in 100d93b

	"pres": {
	"cat": "grid",
	"logic": "range",
	"param": {
	"max": 1300,
	"min": 800
	},
	"unit": "hPa"
	},

Screenshots

Additional context
Not applicable

29 Feb should be removed from the generated TMY

@sunt05 commented on Mar 9, 2020, 8:48 PM UTC:

Describe the Issue

If 29 Feb exists in TMY, EnergyPlus would complain and stop running.

This issue was moved by sunt05 from sunt05/SuPy#131.

Daily maximum temperature from DailyState

Describe the Issue
Incorrect daily maximum temperatures from DailyState.

Model was run with 5 grids (each with different albedo: 0.1, 0.3, 0.5, 0.7 and 0.9). When looking at the difference between max daily temperatures (Tmax from DailyState) the temperature difference was 0 for all albedo values.

I then extracted maximum daily temperature from the SUEWS output (using max T2 value and resampling for daily values). This showed that there was a temperature difference between the different albedo's maximum temperatures. The max daily temperatures were hotter for low albedo and cooler for high albedo as expected so I am sure that the albedo values were set-up correctly in df_state_init.

Location

Screenshots

Max daily temperature when extracted from SUEWS:

Max daily temperature from Daily state:

GLA_study.zip

Additional context

I have attached the .pkl for the df_state_init and df_forcing I used in the model

improve log file treatment

Describe the Issue
When running supy, SuPy.log is created in the directory where the simulation is executed.

This seems to be defined in the _env.py module.

Is there no option to set another logpath in the beginning @sunt05 ? Or manipulate its filename? As I'd like to keep a log per simulation I do, sensitivity simulations will likely be done at the same time on the same machine. So a more unique log filename or location would be useful.

Supy version
supy.version: 2022.9.22
supy.version_driver: 2021a14.post3

Is supy_debug-xxxxxxxx.zip generated?
NA

Screenshots
NA

Additional context
NA

enhance the input pre-processor

@sunt05 commented on Apr 16, 2020, 7:53 PM UTC:

• an intuitive way to visualise the input info in df_state
• a decision-tree-like workflow to populate proper attributes into df_state

This issue was moved by sunt05 from sunt05/SuPy#140.

ram

Issue

Links

source doc
RTD page

I am experiencing a quiet failure of the simulation due to a spike in ram usage. I have logging set to 10 but no error logs are been written. Does anyone have any idea what may be causing this?

Timestep issue when running SuPy with 1min forcing data

When running SUEWS with input data at 60s resolution, the model timestep is set to 60s, and the output is set to 60s, the resulting output is given for every 30s. Same happens when running the model for a 5min timestep with input data at 5min resolution. The results are then given every 2.5min. The results of these half timesteps looks very strange, with extremely high values of longwave out .

The figure show results while running SUEWS with 5min input data, with a timestep of 5min and the output is set to be given at 5min but are given at every 2.5min.

Error invalid continuation byte when loading input files

issue originally submitted here: Urban-Meteorology-Reading/Urban-Meteorology-Reading.github.io#12 (comment) by @MatthewPaskin

Describe the Issue
Hi, I am trying to do a SUEWS run using supy and when using the input files from the 2020a download, upon running
df_state_init = sp.init_supy(path_runcontrol)

I get this error:

UnicodeDecodeError                        
Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_24724/777218402.py in <module>
----> 1 df_state_init = sp.init_supy(path_runcontrol)

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\supy\_supy_module.py in init_supy(path_init, force_reload, check_input)
    101         if path_init_x.suffix == ".nml":
    102             # SUEWS `RunControl.nml`:
--> 103             df_state_init = load_InitialCond_grid_df(
    104                 path_init_x, force_reload=force_reload
    105             )

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\supy\_load.py in load_InitialCond_grid_df(path_runcontrol, force_reload)
   1588     # load base df of InitialCond
   1589     logger_supy.debug("loading base df_init...")
-> 1590     df_init = load_SUEWS_InitialCond_df(path_runcontrol)
   1591 
   1592     # add Initial Condition variables from namelist file

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\supy\_load.py in load_SUEWS_InitialCond_df(path_runcontrol)
   1325     path_input = path_runcontrol.parent / dict_ModConfig["fileinputpath"]
   1326     logger_supy.debug("loading df_gridSurfaceChar")
-> 1327     df_gridSurfaceChar = load_SUEWS_SurfaceChar_df(path_input)
   1328     # df_gridSurfaceChar.to_pickle("df_gridSurfaceChar.pkl")
   1329     # only use the first year of each grid as base for initial conditions

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\supy\_load.py in load_SUEWS_SurfaceChar_df(path_input)
   1098 @functools.lru_cache(maxsize=16)
   1099 def load_SUEWS_SurfaceChar_df(path_input):
-> 1100     df_gridSurfaceChar_exp = gen_df_gridSurfaceChar_exp(path_input)
   1101     dict_var_ndim = {
   1102         "ahprof_24hr": (24, 2),

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\supy\_load.py in gen_df_gridSurfaceChar_exp(path_input)
   1082 @functools.lru_cache(maxsize=16)
   1083 def gen_df_gridSurfaceChar_exp(path_input):
-> 1084     df_siteselect_exp = gen_df_siteselect_exp(path_input)
   1085     dict_var_tuple = exp_dict_full(dict_var2SiteSelect)
   1086     df_gridSurfaceChar_exp = pd.concat(

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\supy\_load.py in gen_df_siteselect_exp(path_input)
    956 def gen_df_siteselect_exp(path_input):
    957     # df with all code-references values
--> 958     df_all_code = gen_all_code_df(path_input)
    959 
    960     # retrieve all `Code`-relaed names

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\supy\_load.py in gen_all_code_df(path_input)
    875 @functools.lru_cache(maxsize=16)
    876 def gen_all_code_df(path_input):
--> 877     dict_libs = load_SUEWS_Libs(path_input)
    878     df_siteselect = dict_libs["lib_SiteSelect"]
    879     list_code = [code for code in df_siteselect.columns if to_exp_Q(code)]

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\supy\_load.py in load_SUEWS_Libs(path_input)
    268         # lib_path = os.path.join(path_input, lib_file)
    269         lib_path = path_input / lib_file
--> 270         dict_libs.update({lib: load_SUEWS_table(lib_path)})
    271     # return DataFrame containing settings
    272     return dict_libs

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\supy\_load.py in load_SUEWS_table(path_file)
    247         # fileX = path_insensitive(fileX)
    248         str_file = str(path_file)
--> 249         rawdata = pd.read_csv(
    250             str_file,
    251             delim_whitespace=True,

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\pandas\util\_decorators.py in wrapper(*args, **kwargs)
    309                     stacklevel=stacklevel,
    310                 )
--> 311             return func(*args, **kwargs)
    312 
    313         return wrapper

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\pandas\io\parsers\readers.py in read_csv(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, encoding_errors, dialect, error_bad_lines, warn_bad_lines, on_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options)
    584     kwds.update(kwds_defaults)
    585 
--> 586     return _read(filepath_or_buffer, kwds)
    587 
    588 

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\pandas\io\parsers\readers.py in _read(filepath_or_buffer, kwds)
    480 
    481     # Create the parser.
--> 482     parser = TextFileReader(filepath_or_buffer, **kwds)
    483 
    484     if chunksize or iterator:

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\pandas\io\parsers\readers.py in __init__(self, f, engine, **kwds)
    809             self.options["has_index_names"] = kwds["has_index_names"]
    810 
--> 811         self._engine = self._make_engine(self.engine)
    812 
    813     def close(self):

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\pandas\io\parsers\readers.py in _make_engine(self, engine)
   1038             )
   1039         # error: Too many arguments for "ParserBase"
-> 1040         return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
   1041 
   1042     def _failover_to_python(self):

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py in __init__(self, src, **kwds)
     67         kwds["dtype"] = ensure_dtype_objs(kwds.get("dtype", None))
     68         try:
---> 69             self._reader = parsers.TextReader(self.handles.handle, **kwds)
     70         except Exception:
     71             self.handles.close()

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\pandas\_libs\parsers.pyx in pandas._libs.parsers.TextReader.__cinit__()

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\pandas\_libs\parsers.pyx in pandas._libs.parsers.TextReader._get_header()

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\pandas\_libs\parsers.pyx in pandas._libs.parsers.TextReader._tokenize_rows()

~\Anaconda3\envs\my_dev_env_supy\lib\site-packages\pandas\_libs\parsers.pyx in pandas._libs.parsers.raise_parser_error()

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 1330: invalid continuation byte

Problem with SuPy QuickStart tutorial

OS: Ubuntu 20.04
Installed new conda environment to test supy with the tutorial "Quickstart of SuPy"
Python v3.9
SuPy versions: supy: 2021.11.22 and supy_driver: 2021a2
SUEWS V2020a and accompanying test files (e.g. RunControl.nml) extracted to: /home/l1nux/Documents/SUEWS

Tutorial started OK in Jupyter Notebook
Then threw an error at this command:
df_state_init = sp.init_supy(path_runcontrol)

I do not understand the error output (exported as asciidoc; attached inside zip file).
Any help much appreciated.

Looking again at the error output, I see that "None" is listed for certain python packages (e.g. pandas_datareader)
Is this part of the problem?

Quickstart_SuPy.asciidoc.zip

ESTM support

@sunt05 commented on Mar 4, 2020, 2:09 PM UTC:

support ESTM in SuPy.

• add rules for ESTM variables
• add initialisation procedure for ESTM

This issue was moved by sunt05 from sunt05/SuPy#130.

improve the logic checking for input

@sunt05 commented on Jun 17, 2020, 12:53 PM UTC:

We are currently using these predefined rules to check the validity of inputs to simulations.
More sophisticated logic rules may be introduced to better guarantee the input validity and thus inform users of data preparation.

This issue was moved by sunt05 from sunt05/SuPy#158.

Supy vs SUEWS advanced outputs

Hello!
I am having issues replicating the same results using SUEWS advanced from umep and SuPy from a jupyter notebook.
In the zipped file i have included i) the outputs of SUEWS Prepare from UMEP which are used to run suews and ii) the outputs of the model using SUEWS Advanced in UMEP (version 3.17.3).

Then, using the RunControl.nml file located in the output folder, i use SuPy (version 2021.1.12) through a Jupyter Notebook without making any changes in the inputs. After examining the model runs observe differences in the UEB components ('QN','QF','QS','QE','QH') which i am interested in.

For example using the command
df_output.loc[1]['SUEWS'][['QN','QF','QS','QE','QH']].resample('60min').mean().iloc[721:-743].describe().round(1)
i get the table shown in image 1, while the original output file from suews in umep gives the table shown in image 2.
I have double checked that the intervals (721:-743) are the same for the outputs.

What am i missing?
in_out_suews.zip
image1

image2

Thanks in advance!

how to export simulation configurations and results

@sunt05 commented on Jul 27, 2019, 7:58 PM UTC:

• pickle
• csv
• netCDF

This issue was moved by sunt05 from sunt05/SuPy#101.

SuPy throws a KeyError when trying to load initial model states

Hi UMEP Team,

Below is the issue description.
Thank you for your help in advance!

Best,
Cs.

Describe the bug
Supy is throwing a KeyError when trying to load initial model states with supy.init_supy. Init model state was assembled with the SUEWS converter.

To Reproduce
Run df_state_init = supy.init_supy(path_runcontrol) to reproduce the following error.

KeyError: "Passing list-likes to .loc or [] with any missing labels is no longer supported. The following labels were missing: MultiIndex([('AnthropogenicCode', 'BaseT_HC')],\n ). See https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#deprecate-loc-reindex-listlike"

The extended error report is attached.
error extended.txt

Desktop (please complete the following information):
Same exact error is produced by 3 different configurations.

• OS: OS Win 10 EDU | Win 8.1 Pro | macOS 10.14.6
• QGIS: 3.10.13 | 3.10.5 | 3.10.6
• UMEP: 3.17 | 3.15.5 | 3.15.5
• SuPy: 2021.3.30 | 2021.3.30 | 3.15.5
• Python: 3.8.8 | 3.8.8 | 3.8.8

Data
Attached are the initial model states files.
preps.zip

Additional context
The code was working with previous Supy (v2021.1.x) and Python (v3.6.x) versions.

supy.util.gen_epw: AttributeError

Hi Developers,

I've tried to generate an epw file with supy.util.gen_epw, but I get an AttributeError: 'MultiIndex' object has no attribute 'year'. Attached are input and output files from my test run.
R11a01.zip

If somebody could take a look at it, what would be great. Thank you in advance!
Csilla

sp.util.gen_forcing_era5 -- AttributeError: 'Dataset' object has no attribute 'z'

Describe the Issue
AttributeError: 'Dataset' object has no attribute 'z'

Location
-> 1096 da_gph_sfc = ds_sfc.z

Screenshot

how about the above regarding compilation languages (e.g. Fortran)

waiting for further advice from @dstansby and his colleagues.

proposal for analyser

@sunt05 commented on Feb 7, 2020, 10:50 AM UTC:

We are moving the code-base for SUEWS analyser in UMEP into SuPy python package.
Here we discuss the required features and pathways for such transition and enhancement.

Below are some features to add in the near future:

• time series plot
• spatial map
• stat calculation

(More details to add for the above features.)

This issue was moved by sunt05 from sunt05/SuPy#124.

run_supy() RuntimeError: SUEWS kernel error

I am running supy (version 2022.2.17) on an AWS instance using 2 CPU and 8GB RAM and system version:

PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"

My attempt to run run_supy() in either serial or parallel mode using the following lines produces "RuntimeError: SUEWS kernel error":

sp.check_state(df_state_spinup)
sp.check_forcing(df_forcing_spinup)
df_spinup_res, df_spinup_final = sp.run_supy(df_forcing_spinup, df_state_spinup, serial_mode=False)

2022-10-26 12:40:20,873 - SuPy - INFO - SuPy is validating `df_state`...
2022-10-26 12:40:22,510 - SuPy - INFO - All checks for `df_state` passed!
2022-10-26 12:40:22,511 - SuPy - INFO - SuPy is validating `df_forcing`...
2022-10-26 12:40:22,535 - SuPy - INFO - All checks for `df_forcing` passed!
2022-10-26 12:40:22,536 - SuPy - INFO - ====================
2022-10-26 12:40:22,538 - SuPy - INFO - Simulation period:
2022-10-26 12:40:22,540 - SuPy - INFO -   Start: 2019-10-16 00:05:00
2022-10-26 12:40:22,541 - SuPy - INFO -   End: 2019-12-31 23:00:00
2022-10-26 12:40:22,543 - SuPy - INFO - 
2022-10-26 12:40:22,544 - SuPy - INFO - No. of grids: 15
2022-10-26 12:40:22,545 - SuPy - INFO - SuPy is running in parallel mode
---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
ValueError: failed to initialize intent(inout) array -- input 'O' not compatible to 'd'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/supy/_run.py", line 121, in suews_cal_tstep_multi
    res_suews_tstep_multi = sd.suews_cal_multitsteps(**dict_input)
ValueError: failed in converting 70th argument `icefrac' of suews_driver.suews_driver.suews_cal_multitsteps to C/Fortran array

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/supy/_run.py", line 371, in run_supy_ser
    for input_grid in list_input
  File "/opt/conda/lib/python3.7/site-packages/supy/_run.py", line 371, in <listcomp>
    for input_grid in list_input
  File "/opt/conda/lib/python3.7/site-packages/supy/_run.py", line 126, in suews_cal_tstep_multi
    with open("problems.txt", "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'problems.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py", line 47, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/opt/conda/lib/python3.7/site-packages/supy/_run.py", line 417, in run_save_supy
    df_forcing_tstep, df_state_init_m, save_state, n_yr
  File "/opt/conda/lib/python3.7/site-packages/supy/_run.py", line 376, in run_supy_ser
    raise RuntimeError("SUEWS kernel error")
RuntimeError: SUEWS kernel error
"""

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
<ipython-input-27-de98ae1444a9> in <module>
      2 sp.check_state(df_state_spinup)
      3 sp.check_forcing(df_forcing_spinup)
----> 4 df_spinup_res, df_spinup_final = sp.run_supy(df_forcing_spinup, df_state_spinup, serial_mode=False)

/opt/conda/lib/python3.7/site-packages/supy/_supy_module.py in run_supy(df_forcing, df_state_init, save_state, chunk_day, logging_level, check_input, serial_mode)
    364         logger_supy.info(f"SuPy is running in parallel mode")
    365         df_output, df_state_final = run_supy_par(
--> 366             df_forcing, df_state_init, save_state, chunk_day
    367         )
    368     else:

/opt/conda/lib/python3.7/site-packages/supy/_run.py in run_supy_par(df_forcing_tstep, df_state_init_m, save_state, n_yr)
    448                     list_save_state,
    449                     list_n_yr,
--> 450                     list_dir_temp,
    451                 ),
    452             )

/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py in starmap(self, func, iterable, chunksize)
    274         `func` and (a, b) becomes func(a, b).
    275         '''
--> 276         return self._map_async(func, iterable, starmapstar, chunksize).get()
    277 
    278     def starmap_async(self, func, iterable, chunksize=None, callback=None,

/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py in get(self, timeout)
    655             return self._value
    656         else:
--> 657             raise self._value
    658 
    659     def _set(self, i, obj):

/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py in worker()
    119         job, i, func, args, kwds = task
    120         try:
--> 121             result = (True, func(*args, **kwds))
    122         except Exception as e:
    123             if wrap_exception and func is not _helper_reraises_exception:

/opt/conda/lib/python3.7/site-packages/multiprocess/pool.py in starmapstar()
     45 
     46 def starmapstar(args):
---> 47     return list(itertools.starmap(args[0], args[1]))
     48 
     49 #

/opt/conda/lib/python3.7/site-packages/supy/_run.py in run_save_supy()
    415     # run supy in serial mode
    416     df_output, df_state_final = run_supy_ser(
--> 417         df_forcing_tstep, df_state_init_m, save_state, n_yr
    418     )
    419     # save to path_dir_temp

/opt/conda/lib/python3.7/site-packages/supy/_run.py in run_supy_ser()
    374 
    375             except:
--> 376                 raise RuntimeError("SUEWS kernel error")
    377 
    378             # collect output arrays

RuntimeError: SUEWS kernel error

Screenshots

(from attempt to run in serial mode)

Thanks!

Install via pip failed with error on version of supy-driver==2020b14

I attempted to install the supy package according to instructions, using the command line
python -m pip install supy --upgrade

This failed with the following output. I am on a Linux system (Mint), 64 bit, running Python 3.7.2 (64 bit)

Collecting supy Using cached https://files.pythonhosted.org/packages/08/f0/6bc05e9dff90bbe12d797a74641809bdc219e671f8c8134a28eaa37fd87c/supy-2021.2.26-py3-none-any.whl Collecting atmosp (from supy) Downloading https://files.pythonhosted.org/packages/d8/46/9c75ffc6e57cd30d1ed9d99019a6c14e414750dbdc73e53ce03e94350d76/atmosp-0.2.9-py3-none-any.whl (104kB) 100% |████████████████████████████████| 112kB 4.3MB/s Collecting supy-driver==2020b14 (from supy) Could not find a version that satisfies the requirement supy-driver==2020b14 (from supy) (from versions: 2018b11, 2018b12, 2018b13, 2018b14, 2018b15, 2018b17, 2018b19, 2018b20, 2018b22, 2018b25, 2018b26, 2018b27, 2018b28, 2018b29, 2018b30, 2018rc1, 2018rc2, 2018rc3, 2018rc4, 2018rc5, 2018rc6, 2018rc7, 2018rc8, 2019a1, 2019a2, 2019a3, 2019a4, 2019a5, 2019a6, 2019a7, 2019a9, 2019a10, 2019a11, 2019a12, 2019a13, 2019a14, 2019a16, 2019a17, 2019a18, 2019b1, 2019b2, 2019b3, 2019b5, 2020a1, 2020a2, 2020a3, 2020a4, 2020a5, 2020a6, 2020a7, 2020a8, 2020a9, 2020a10, 2020b1, 2020b4, 2020b5, 2020b6, 2020b7, 2020b8, 2020b9, 2020b10, 2020b11, 2020b13) No matching distribution found for supy-driver==2020b14 (from supy)

I did a little further digging and tried manually installing supy_driver, this showed that the supy_driver on the pypi database appears to be 2020b13. I guess that is the issue.

Varying results / NaN for 'T2' in different grid cells

Dear all,
I run SuPy for multiple grid cells (> 200) and I am interested in the 'T2' output. However, SuPy returns only NaN for 'T2' for some grid cells. The initial grid cell states differ in surface fraction, building and tree heights, roughness and zero displacement length, frontal area index, and population density. Forcing data are the same for all grid cells. I attached the corresponding pickle files with two exemplary grid cells as zip file.

SuPy runs without any errors.

Thank you.
Ferdi

Screenshots
Initial state differences of grid cells:

## surface fraction
>> df_state.loc[:, 'sfr'] 

ind_dim   (0,)   (1,)  (2,)   (3,)   (4,)  (5,)   (6,)
grid                                                  
170      0.353  0.106     0  0.247  0.294   0.0  0.000
171      0.241  0.335     0  0.294  0.128   0.0  0.002

## tree and building heights
>> df_state.loc[:, ['dectreeh','evetreeh','bldgh']]

var     dectreeh evetreeh  bldgh
ind_dim        0        0      0
grid                            
170         7.18     7.18  11.67
171         8.33     8.33  13.77

## roughness and zero displacement length
>> df_state.loc[:, ['z0m_in','zdm_in']]

var     z0m_in zdm_in
ind_dim      0      0
grid                 
170       1.18  10.68
171       1.36  20.78

## frontal area index
>> df_state.loc[:, ['faibldg','faidectree','faievetree']]

var     faibldg faidectree faievetree
ind_dim       0          0          0
grid                                 
170        0.12       0.09       0.09
171        0.21       0.22       0.22

Output:

df_output_suews.loc[:, 'T2'].isna()
Out[41]: 
grid  datetime           
170   2006-01-01 02:00:00     True
      2006-01-01 03:00:00     True
      2006-01-01 04:00:00     True
      2006-01-01 05:00:00     True
      2006-01-01 06:00:00     True
 
171   2006-12-31 19:00:00    False
      2006-12-31 20:00:00    False
      2006-12-31 21:00:00    False
      2006-12-31 22:00:00    False
      2006-12-31 23:00:00    False
Name: T2, Length: 17516, dtype: bool

Additional context
Version:

SuPy versions
-------------
supy: 2021.7.22 
supy_driver: 2021a2

=================
SYSTEM DEPENDENCY

INSTALLED VERSIONS
------------------
commit           : 2cb96529396d93b46abab7bbc73a208e708c642e
python           : 3.7.10.final.0
python-bits      : 64
OS               : Windows
OS-release       : 10
Version          : 10.0.17763
machine          : AMD64
processor        : Intel64 Family 6 Model 158 Stepping 13, GenuineIntel
byteorder        : little
LC_ALL           : None
LANG             : en
LOCALE           : None.None

pandas           : 1.2.4
numpy            : 1.20.1
pytz             : 2021.1
dateutil         : 2.8.1
pip              : 21.0.1
setuptools       : 52.0.0.post20210125
Cython           : 0.29.23
pytest           : 6.2.3
hypothesis       : None
sphinx           : 4.0.1
blosc            : None
feather          : None
xlsxwriter       : 1.3.8
lxml.etree       : 4.6.3
html5lib         : 1.1
pymysql          : None
psycopg2         : None
jinja2           : 2.11.3
IPython          : 7.22.0
pandas_datareader: None
bs4              : 4.9.3
bottleneck       : 1.3.2
fsspec           : 0.9.0
fastparquet      : None
gcsfs            : None
matplotlib       : 3.3.4
numexpr          : 2.7.3
odfpy            : None
openpyxl         : 3.0.7
pandas_gbq       : None
pyarrow          : None
pyxlsb           : None
s3fs             : None
scipy            : 1.6.2
sqlalchemy       : 1.4.7
tables           : None
tabulate         : None
xarray           : 0.17.0
xlrd             : 2.0.1
xlwt             : 1.3.0
numba            : 0.53.1

GitHub_Issue.zip

Incorrect units for barometric pressure and RH in forcing data generated from ERA5 multilevel data

Units of barometric pressure and RH in the forcing data generated from ERA5 multilevel data are incorrect. The issue is that barometric pressure has been loaded in hPa, but atoms.calcaulte requires Pa to calculate RH.

Tutorials — SuPy 2021.1.22 documentation

Tutorials — SuPy 2021.1.22 documentation

https://supy.readthedocs.io/en/latest/tutorial/tutorial.html

use `importlib` to load data files

Describe the issue

os.path.dirname(__file__) is not a good way to load data files. It is not portable and it is not a good way to load data files from a package.

Describe the solution you'd like

from importlib.resources import files is a new feature in Python 3.9. It allows us to load data files from a package.

References

1. https://docs.python.org/3/library/importlib.html#module-importlib.resources
2. https://setuptools.pypa.io/en/latest/userguide/datafiles.html#accessing-data-files-at-runtime

create a master file/template to document meta-info of all variables

a YAML file meant to be a central place to store all meta-info about variable exposed to users, which includes:

• key in short name : used in code and input table headers
• values with details:
  • type: input or output (detailed sub-groups will be shown in examples below).
  • description: full description with unit (if applicable).
  • suews-related variables: SUEWS related variables as in the headers of input tables (e.g. those of SUEWS_veg.txt).
  • physics schemes: where this variable is used (e.g. radiation scheme coded radiationmethod = 3).
  • data dimensions: dimension info about this variable.
  • loading path: logic used internally by supy to load values from input tables (similar to this file).

an example file can be found here, which is generated using this python script with some info filled using other files in this repo - details refer to the code.

SMD and Irrigation at beginning of simulation run exceptionally high

Dear Ting

Thank you for your answer on my previous question, it works really well!

Something else came up a week ago, but I didn't have to time to post it until now.

When SMD is modelled, the initial SMD value is very high, no matter when the simulation period starts.
I did two runs, one run with ten years and one run with only the year 2015. Please find below the ten year figure, where SMD starts at ~30 this is in June. (note 2015, where SMD seems normal, after SuPy calibrated)

and then a figure from the second run plus the first few rows of the df_output['SMD'], in the year 2015. Again SMD starts out at 30, which is too high for my data (Sorry, I turned the Precipitation upside down, I think it looks better :D)

Is there a df_state variable I missed to adjust or is this something in your code?

Lastly I also attached a ten year figure for irrigation, which seems to have the same issue as SMD (might be related?)

Best wishes
Oliver

questions regarding documentation under OSSS project

SuPy will be merged into SUEWS soon, so future discussions will be based on the merged version.

---

• best practice to integrate inline comments (e.g. docstrings) into documentation
• best practice in maintaining documentation
• #85
• how to run the open-source community?