samuelduchesne / archetypal Goto Github PK
View Code? Open in Web Editor NEWarchetypal: Retrieve, construct, simulate, convert and analyse building simulation templates
Home Page: https://archetypal.readthedocs.io/
License: MIT License
archetypal: Retrieve, construct, simulate, convert and analyse building simulation templates
Home Page: https://archetypal.readthedocs.io/
License: MIT License
Another JOSS review issue! Once again, just a minor one.
You should check your use of acronyms. M
in BEM
is given as Models
but at one point is used in a way that means modelling
. UBEM
is given in full form twice in the document, once with the M
referring to Models
and once referring to Modelling
. I would also not say UBEM models
as it sounds weird when you think it means Urban Building Energy Modelling models
.
In fact, my preference would be to avoid using both BEM and UBEM in your paper, just use the phrases in full form. It doesn't add much to the paper length, but does aid in clarity.
Note: this issue comes to you as part of my wider JOSS review
I've attempted to read and run an EnergyPlus file following the instructions given in the documentation. Very few steps worked, although many issues seem to be a fault of dependencies. Below I describe what hasn't worked. archetypal version: v1.3.0-dev (56f95d0), E+ version: 9.2.0, installation method: installing using conda, as per installation instructions.
Loading only an IDF file works fine, but loading an IDF and weather file at once fails with the following error trace:
ValueError Traceback (most recent call last)
<ipython-input-8-493961940195> in <module>
----> 1 idf2 = load_idf(eplus_file, weather)
~/miniconda3/envs/archetypal/lib/python3.7/site-packages/archetypal/idfclass.py in load_idf(eplus_file, idd_filename, output_folder, include, weather_file, ep_version)
774 include=include,
775 epw=weather_file,
--> 776 ep_version=ep_version if ep_version is not None else settings.ep_version,
777 )
778 log("Eppy load completed in {:,.2f} seconds\n".format(time.time() - start_time))
~/miniconda3/envs/archetypal/lib/python3.7/site-packages/archetypal/idfclass.py in _eppy_load(file, idd_filename, output_folder, include, epw, ep_version)
825 IDF.setiddname(idd_filename, testing=True)
826 # load the idf object
--> 827 idf_object = IDF(file, epw=epw)
828 # Check version of IDF file against version of IDD file
829 idf_version = idf_object.idfobjects["VERSION"][0].Version_Identifier
~/miniconda3/envs/archetypal/lib/python3.7/site-packages/archetypal/idfclass.py in __init__(self, *args, **kwargs)
57 **kwargs:
58 """
---> 59 super(IDF, self).__init__(*args, **kwargs)
60 self._sql_file = None
61 self.schedules_dict = self.get_all_schedules()
~/miniconda3/envs/archetypal/lib/python3.7/site-packages/eppy/modeleditor.py in __init__(self, idfname, epw)
548 if idfname != None:
549 self.idfname = idfname
--> 550 self.read()
551 if epw != None:
552 self.epw = epw
~/miniconda3/envs/archetypal/lib/python3.7/site-packages/geomeppy/patches.py in read(self)
254 raise IDDNotSetError(errortxt)
255 self.idfobjects, block, self.model, idd_info, idd_index, versiontuple = idfreader1(
--> 256 self.idfname, self.iddname, self, commdct=self.idd_info, block=self.block
257 )
258 self.__class__.setidd(idd_info, idd_index, block, versiontuple)
~/miniconda3/envs/archetypal/lib/python3.7/site-packages/geomeppy/patches.py in idfreader1(fname, iddfile, theidf, conv, commdct, block)
88
89 """
---> 90 versiontuple = iddversiontuple(iddfile)
91 block, data, commdct, idd_index = readdatacommdct1(
92 fname, iddfile=iddfile, commdct=commdct, block=block
~/miniconda3/envs/archetypal/lib/python3.7/site-packages/eppy/idfreader.py in iddversiontuple(afile)
49 return (0,)
50 vers = line.split()[-1]
---> 51 return versiontuple(vers)
52
53
~/miniconda3/envs/archetypal/lib/python3.7/site-packages/eppy/idfreader.py in versiontuple(vers)
34 def versiontuple(vers):
35 """version tuple"""
---> 36 return tuple([int(num) for num in vers.split(".")])
37
38 try:
~/miniconda3/envs/archetypal/lib/python3.7/site-packages/eppy/idfreader.py in <listcomp>(.0)
34 def versiontuple(vers):
35 """version tuple"""
---> 36 return tuple([int(num) for num in vers.split(".")])
37
38 try:
ValueError: invalid literal for int() with base 10: 'Ap,IL,USA,TMY3,725300,41'
Once an IDF is loaded (using the successful method idf = load_idf(eplus_file)
), the run command (idf.run_eplus(weather_file)
) causes an exception: TypeError: run_eplus() takes 1 positional argument but 2 were given
.
If I run the file without first loading it (archetypal.idfclass.run_eplus(eplus_file, weather)
) I get an EnergyPlus error:
EnergyPlusProcessError Traceback (most recent call last)
<ipython-input-27-9a3f51f38d18> in <module>
----> 1 run_eplus(eplus_file, weather)
~/miniconda3/envs/archetypal/lib/python3.7/site-packages/archetypal/idfclass.py in run_eplus(eplus_file, weather_file, output_directory, ep_version, output_report, prep_outputs, simulname, keep_data, annual, design_day, epmacro, expandobjects, readvars, output_prefix, output_suffix, version, verbose, keep_data_err, include, process_files, custom_processes, return_idf, return_files)
1751 }
1752
-> 1753 _run_exec(**runargs)
1754
1755 log(
~/miniconda3/envs/archetypal/lib/python3.7/site-packages/archetypal/idfclass.py in _run_exec(tmp, eplus_file, weather, output_directory, annual, design_day, idd, epmacro, expandobjects, readvars, output_prefix, output_suffix, version, verbose, ep_version, keep_data_err, output_report, include)
1977 tmp.copytree(failed_dir / output_prefix)
1978 raise EnergyPlusProcessError(
-> 1979 cmd=cmd, idf=eplus_file.basename(), stderr=stderr_r
1980 )
1981
EnergyPlusProcessError: AdultEducationCenter.idf:
Program Version,EnergyPlus, Version 9.2.0-921312fa1d, YMD=2020.03.02 14:23,
** Severe ** Line: 1056 You must run the ExpandObjects program for "HVACTemplate:Thermostat"
** Severe ** Line: 1065 You must run the ExpandObjects program for "HVACTemplate:Zone:IdealLoadsAirSystem"
** Fatal ** Errors occurred on processing input file. Preceding condition(s) cause termination.
...Summary of Errors that led to program termination:
..... Reference severe error count=2
..... Last severe error=Line: 1065 You must run the ExpandObjects program for "HVACTemplate:Zone:IdealLoadsAirSystem"
************* Warning: Node connection errors not checked - most system input has not been read (see previous warning).
************* Fatal error -- final processing. Program exited before simulations began. See previous error messages.
************* EnergyPlus Warmup Error Summary. During Warmup: 0 Warning; 0 Severe Errors.
************* EnergyPlus Sizing Error Summary. During Sizing: 0 Warning; 0 Severe Errors.
************* EnergyPlus Terminated--Fatal Error Detected. 0 Warning; 2 Severe Errors; Elapsed Time=00hr 00min 0.21sec
Note: this issue comes to you as part of my wider JOSS review
It stores the complete dict of windows, rather than a ref:
archetypal/tests/input_data/umi_samples/BostonTemplateLibrary_2.json
Lines 5487 to 5538 in 66dc6dc
This means that when loading building templates, it will create a new window object every time, since L336 will always error out:
archetypal/archetypal/template/building_template.py
Lines 335 to 341 in 66dc6dc
This is normally handled fine, but in the realtime-graphs setup that I am working on right now, this leads to errors.
I'm pretty sure it's just an artifact of an old call to save()
a template library, since current template libraries appear to save windows at the building template layer as refs, so it's just a matter of fixing the json.
ar.settings paths do not refer to the paths given in config. For Example :
ar.settings.data_folder returns 'data' instead of 'tests/temp/data'
The same problem happens for ar.settings.imgs_folder ; ar.settings.cache_folder ; ar.settings.logs_folder.
Use PredictedLoadtoSetpoint (+/-) to specify:
In the trnsys.py file, we have several function to translate an IDF file to an input file for TRNBuild. From line 417 to line 436 (region Get schedules from IDF) we get the used schedules in the IDF file.
However, this action takes around 90% of the total time of all the procedures.
After investigation, it seems to be at line 429 where we use s.to_year_week_day() that the major part of the time is consumed.
The docs specify E+ v.8.9.0, when v.9.2.0 is already available (I can safely say that archetypal
doesn't work with v.9.2.0!). Version issues crop up relatively late on in the use of archetypal
, and are not particularly explicit in providing advice on how to solve them in the error messages. Could you check for E+ version on python interactive import of archetypal
, then raise errors then about any user E+ version issues, instead of waiting for one of the archetypal
functions to fail later on? It might also help to make the version dependency even more explicit in e.g. the package README.
Note: this issue comes to you as part of my wider JOSS review
Run as E+ v=9.2; the error is: "Cooling setpoint (90) exceeds maximum value (50)" which appears to be a transition isssue.
RefBldgStand-aloneRetailPost1980_v1.4_7.2_3A_USA_GA_ATLANTA.zip
CI tests take more than 1 hour to run. Long tests must be refactored.
archetypal/archetypal/template/umi_base.py
Line 536 in d0aed03
This recursion call sometimes raises a RecursionError: maximum recursion depth exceeded while calling a Python object
I have not yet been able to reliably repro - error is happening in umiverse flask backend.
Sometimes the following steps execute successfully, sometimes, the final step fails with an recursion error.
["MediumOffice_Post1980_2A", "MediumOffice_New2004_2A", "MediumOffice_Pre1980_2A"]
from MongoDB (though I have not yet found a single template or collection of templates which always causes the error - my guess is it has to do with the order that the templates arrive as results from the db, if that is not the same every time?)bld.to_template(...)
template_lib
with UmiTemplateLibrary(BuildingTemplates=[<templates>])
template_lib.unique_components()
template_lib.to_dict()
When it fails, this stack trace occurs:
Traceback (most recent call last):
--
File "/usr/src/app/project/routes.py", line 1104, in construct_umi_file_task
template_lib_as_dict = template_lib.to_dict()ย # as dict
File "/usr/local/lib/python3.8/site-packages/archetypal/umi_template.py", line 579, in to_dict
data.update({"Name": UniqueName(data.get("Name"))})
File "/usr/local/lib/python3.8/site-packages/archetypal/template/umi_base.py", line 511, in __new__
return str.__new__(cls, cls.create_unique(content))
File "/usr/local/lib/python3.8/site-packages/archetypal/template/umi_base.py", line 536, in create_unique
return cls.create_unique(name)
File "/usr/local/lib/python3.8/site-packages/archetypal/template/umi_base.py", line 536, in create_unique
return cls.create_unique(name)
File "/usr/local/lib/python3.8/site-packages/archetypal/template/umi_base.py", line 536, in create_unique
return cls.create_unique(name)
[Previous line repeated 934 more times]
File "/usr/local/lib/python3.8/site-packages/archetypal/template/umi_base.py", line 528, in create_unique
match = re.match(r"^(.*?)(\D*)(\d+)$", name)
File "/usr/local/lib/python3.8/re.py", line 191, in match
return _compile(pattern, flags).match(string)
File "/usr/local/lib/python3.8/re.py", line 291, in _compile
if isinstance(flags, RegexFlag):
RecursionError: maximum recursion depth exceeded while calling a Python object
Seems like there is some sort of issue with the regex matching when it shouldn't, causing the recurse call to keep happening.
It's possible to increase the python recursion limit (default is 1k I believe) if it is actually possible for the count to go past 1k, but that seems like it shouldn't be necessary, and anyways doesn't seem like the most performant solution if it is. I wonder if it makes sense to use a method like keeping track of the count for each base name, and then just incrementing that, rather than trying to determine the current count with the regex and then recursing.
When an idf file (that does not contain an sql object statement) is run with the output_report="sql" parameter, the eprun will not produce an sql file.
Complexity reduction algorithm should merge a core template into a perimeter template if it's core zone represents less than a certain floor area percentage
Analogue to OpenStudio Measures, an umi measure api could be used to build scripts that modify UMI Templates
Run as E+ v=9.2; something is funny in the transition and this error occurs for several idf files:
"Heating setpoint cannot be higher than the cooling setpoint. "
When reducing a large list of IDFs files, I got an error "AssertionError: Heating setpoint must be lower than the cooling setpoint" but have no way of telling which IDF file is causing this error. The last entry above the error was: "Reducing perimeter zones in <_io 0% | | 0/68
It would be nice if the flag reported which file caused the error.
There is a way of defining windows in E+ that uses to Window object instead of the FenestrationSurface:Detailed. The IDF.wwr() does not seem to capture these cases.
Use this file as an example: SFH_pre_1980.idf
When reducing an IDF file, we ignore the attic characterictics.
Problem: for the roof, the attic construction should be used! Now, it is the ceiling construction that is used for the roof (usually less insulated than a roof construction).
UMI version 2.6.16, a building template now has en entry for the default WWR.
GasMaterial
constructor uses the name input rather than the type input to set the gas type. This causes errors when loading a library if the name is not a valid GasMaterial type.
archetypal/archetypal/template/materials/gas_material.py
Lines 22 to 40 in 4d734c8
repro:
from archetypal import UmiTemplateLibrary
lib = UmiTemplateLibrary.open("<path>/BostonTemplateLibrary_2.json")
gas = lib.GasMaterials[0]
gas.Name = "My Gas"
lib.save("<path>/Library_with_bad_gas.json")
UmiTemplateLibrary.open("<path>/Library_with_bad_gas.json")
Output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/src/local/archetypal/archetypal/umi_template.py", line 332, in open
t = cls.loads(f.read(), name)
File "/usr/src/local/archetypal/archetypal/umi_template.py", line 342, in loads
t.GasMaterials = [
File "/usr/src/local/archetypal/archetypal/umi_template.py", line 343, in <listcomp>
GasMaterial.from_dict(store, allow_duplicates=True)
File "/usr/src/local/archetypal/archetypal/template/materials/gas_material.py", line 107, in from_dict
return cls(id=_id, **data, **kwargs)
File "/usr/src/local/archetypal/archetypal/template/materials/gas_material.py", line 37, in __init__
self.Type = Name.upper()
File "/usr/src/local/archetypal/archetypal/template/materials/gas_material.py", line 51, in Type
assert value.lower() in self._GASTYPES, (
AssertionError: Invalid value 'MY GAS' for material gas type. Gas type must be one of the following:
('air', 'argon', 'krypton', 'xenon', 'sf6')
Non breaking fix would be to try/except using the name first, and then fall back to type? Or maybe to use the type if it exists in kwargs, and if it doesn't, fall back to name?
invoking the archetypal convert
command in the terminal on a fresh docker container with miniconda preinstalled (and no EnergyPlus installation) triggers an endless loop in eppy_load() > while idf_object is None > except Exception as e
This is the command used :
archetypal -v convert --trnsidf-exe docker/trnsidf/trnsidf.exe "tests/input_data/trnsys/ASHRAE90.1_Warehouse_STD2004_Rochester.idf"
The command should warn if no EnergyPlus distribution is found (before entering the loop)
I ran archetypal on branch 123-conditioning-settings-are-not-the-same and while the umi template succesfully created, there is no heating schedule in the resulting umi template and as a result no heating energy use in UMI.
When the output_directory
attribute is specified in run_eplus and that multiple idfs are run sequentially, the first result is return for all runs.
eg.:
idfs = [
Path("archetypes/NECB 2011/NECB 2011 - Retail Standalone.idf"),
Path("archetypes/NECB 2011/NECB 2011 - Large Office.idf"),
Path("archetypes/NECB 2011/NECB 2011 - MidRise Apartment.idf"),
]
res = {
name.basename(): ar.run_eplus(
eplus_file=name,
weather_file=weather_file,
output_directory=output,
prep_outputs=True,
design_day=True,
output_report="htm",
)
for name in idfs
}
will produce de same values for each element of res
This example code from Convert IDF to UMI
doesn't work:
from archetypal import BuildingTemplate
template_obj = BuildingTemplate.from_idf(
x.idf, sql=x.idf.sql, DataSource=x.idf.name
)
^ Variable x
is not yet defined elsewhere in the page
from archetypal import UmiTemplate
template_json = UmiTemplate(
name="my_umi_template",
BuildingTemplates=template_obj
).to_json()
^ template_obj
is expected to be iterable. In the docs example, only one file is loaded, so template_obj
(assuming x.
is removed) is only one, uniterable object.
Note: this issue comes to you as part of my wider JOSS review
Just two points on your JOSS paper (otherwise, I think it is sufficiently clear):
Line 62: striped -> stripped
Lines 79-81: Could you add links to the references to DOE benchmark models, to allow interested users to easily access those models?
Note: this issue comes to you as part of my wider JOSS review
On running archetypal convert "SimpleOfficeBuilding.idf" "output"
I get the following error:
Traceback (most recent call last):
File "my_conda_env/bin/archetypal", line 8, in <module>
sys.exit(cli())
File "my_conda_env/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "my_conda_env/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "my_conda_env/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "my_conda_env/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "my_conda_env/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "my_conda_env/lib/python3.7/site-packages/archetypal/cli.py", line 303, in convert
if paths:
UnboundLocalError: local variable 'paths' referenced before assignment
It looks like in cli.py
there is a with
statement which produces a variable paths
which is then expected to exist later on (if paths:
). paths
might need to be set as e.g. None before the with
statement?
Note: this issue comes to you as part of my wider JOSS review
@zberzolla, The comparison of B_Off_0 template with it's eqivalent after being passed through the archetypal pipeline (ran the B_Off_0 template, opened the SAMPLE0.idf in archetypal (UmiTemplateLibrary.from_idf()
) gives the following differences in the Conditioning settings.
Create YearSchedule for Umi template from DaySchedule and WeekSchedule classes
Installing on MacOS fails to complete due to a pip wheel not being found for 'regex' (one of the dependencies of a dependency). This isn't an issue with archetypal, per se, but was solved by a conda installation workflow (i.e. creating a conda environment, installing all dependencies from conda / conda forge, then installing archetypal with pip). Would it be possible to:
a) have a working conda environment requirements file in this repository, to minimise direct pip dependency, and
b) update the docs to include information on installing archetypal using a conda workflow, instead of directly with pip?
Note: this issue comes to you as part of my wider JOSS review
Edit: I see there is already a requirements file, which is used in the Docker instance, but it would need updating to work with MacOS too...
For some buildings, we encounter an "adjacency" error ("adjacent wall not found" in TRNBuild) only for some adjacent wall. Not all adjacent wall are not found.
For example when translating "NECB - Warehouse.idf", we find in the BUI file (attached to this issue) at lines 604 and 627 that "ADJACENT" and "ADJ_SURF" are empty:
line 604: FLOOR =c_000019 : SURF= 59 : AREA= 236.881 : ADJACENT= : ADJ_SURF= : : GEOSURF=0.102000
line 627: CEILING=c_000017 : SURF= 81 : AREA= 236.881 : ADJACENT= : ADJ_SURF= :
Whereas in line 604 it should be:
line 604: FLOOR =c_000019 : SURF= 59 : AREA= 236.881 : ADJACENT=z_000001 : ADJ_SURF=81 : : GEOSURF=0.102000
And line 627:
line 627: CEILING=c_000017 : SURF= 81 : AREA= 236.881 : ADJACENT=z_000002 : ADJ_SURF=59 :
Moreover, we do not understand why this kind of error happens while for other adjacent walls there is no errors, for example line 602 and 625 of the file:
line 602: WALL =c_000021 : SURF= 57 : AREA= 110.544 : ADJACENT=z_000001 : ADJ_SURF=79 : FRONT : GEOSURF=0.034000
line 625: WALL =c_000021 : SURF= 79 : AREA= 110.544 : ADJACENT=z_000002 : ADJ_SURF=57 : BACK : GEOSURF=0.147827
Add an option to overwrite the idf file used to generate the trnbuild file (.bui)
Error in modeleditor from IDF class when getting schedules in convert_idf_to_trnbuild(), when the idf was loaded from cache (see error below):
commdct = None, key = 'SCHEDULE:DAY:HOURLY', block = None, defaultvalues = True
def newrawobject(data, commdct, key, block=None, defaultvalues=True):
"""Make a new object for the given key.
Parameters
----------
data : Eplusdata object
Data dictionary and list of objects for the entire model.
commdct : list of dicts
Comments from the IDD file describing each item type in `data`.
key : str
Object type of the object to add (in ALL_CAPS).
Returns
-------
list
A list of field values for the new object.
"""
dtls = data.dtls
key = key.upper()
key_i = dtls.index(key)
> key_comm = commdct[key_i]
E TypeError: 'NoneType' object is not subscriptable
Hi Samuel and Louis, this issue relates to the JOSS review of your paper. Here are the detailed comments mentioned in the review page.
Paper summary:
I would suggest writing just a few words to give more high-level context to the non-expert, for example:
All of that could fit in 3-4 sentences, no need to give over-detailed explanations.
Misc comments on the paper:
Documentation:
Generally speaking it seems incomplete: there are examples for using the E+ to UMI conversion, E+ to TRNBuild, but it is very vague on using archetypal as a simulation environment, the modification of UMI Templates is not covered by examples, the "getting started" section is left blank and the tutorials are incomplete or blank.
The user is left to finding out on their own how to use archetypal for any task other than converting models.
You seem to have done a great deal of work, nevertheless some more work on the documentation is in my opinion necessary if You wish to release the software.
Let me know what are your thoughts on these comments.
Kind regards
Thibault
For example, downloading a idf element using the data portal might need EnergyPlus 7.1
Embodied energy is currently ignored because it is not included in the material description within an idf file.
One solution could be to feed archetypal an external file with embodied energy parameters for each of the materials included in the EnergyPlus model (see image bellow). The structure of this file is to be discussed here.
Both heating and cooling COPs are set to 0 when reducing from ClimateStudio IDFs. ClimateStudio uses ZoneHVAC:IdealLoadsAirSystem.
archetypal_issue.zip
There seems to be some issues with the way COPs are calculated.
I am trying to convert an E+ idf file to UMI .json template. I am a new Python user and followed the installation instruction (installing using pip) in Python 3.6. I have installed archetypal successfully, but when I am trying to convert the IDF file following "Using the Python Console" I am getting this error: 'UmiTemplateLibrary' is not defined". Please see the attached screenshot from Python IDE.
Please help to figure this problem out!
Thanks,
8 tests fail and 22 error when running tests locally on MacOS. The 22 errors seem to all be trnsys related, but could be handled more gracefully by skipping a test if the operating system is not Windows (see here). Full test log is given here - enjoy!
Note: this issue comes to you as part of my wider JOSS review
issueFiles.zip
When running through Archetypal these IDFs are E+ v=9.5 and it gives an error that it wants v=9.2. Using as-version also does not work.
Example in sample0.idf
InternalMass,
CoreInternalMass, !- Name of internal mass surface
B_Off_Thm_0, !- Construction Name
Core, !- Name of Associated Thermal Zone
7.71749999999999; !- Total area exposed to Zone [m2]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.