ie3-institute / powersystemdatamodel Goto Github PK

Currently we are using Optional as constructor and filed parameters in some places. While this isn't cause a problem for now (and we already discussed that this makes sense in our case), I just discovered that this prevents the classes using it from being Serializable. As we will depend on serialization in several places this might become a problem in the future. Hence, finding another solution might be unavoidable.

Add a method StandardUnits which provides a convenient method with error checking for conversion to StandardUnits

With the introduction of the revised data model structure, the old .csv writer won't work anymore. Hence, we need new ones. Based on newly introduced and currently being reviewed model processors, implementing a .csv writer for at least the result models should be very straight forward.

Currently, a lot of the entities (e.g. LoadInput, PvResult, ...) are missing their own implementation of toString(). Would be nice if we could provide a proper implementation of them to make them easier to be printed in logs and exceptions.

Currently the uml diagrams do not fit the represented class implementations.

Currently, the phase-to-ground elements in lines and transformers are required to non-zero. However, as this is a typical assumption, we will lower this burden.

... that is able to generate a structure of single grid models or aggregate different single grid models to a joint data set.

... analogously to input models.

Currently it's n(S/km), where µS/km would be desired.

Currently all classes of the data model are located @ edu.ie3. In previous versions we had all data model related stuff located @edu.ie3.datamodel. Is there any specific reason that we don't do this anymore?
Otherwise I would suggest the following:

• move everything into sub-package edu.ie3.datamodel
• move ValidationTools -> edu.ie3.datamodel.utils and rename it to ValidationUtils
• move InvalidEntityExceptionand UnsafeEntityException into new to create edu.ie3.datamodel.exceptions
• delete edu.ie3.datamodel.validation

Adding multiple sample connectors for databases for performance comparison

Some of our models are nested e.g. a 2 winding transformer holds an instance of its node, while in a database or a file only the primary key is referenced in the corresponding field. For simplification and to avoid missing elements when persisting a model, it would be nice to have an extractor class that ensures that all nested models are extracted in their own collection to be passed to the persistence unit.

e.g a collection of Transformer2W can be "extracted" into collections of Transformer2W, OperatorInput and NodeInput

I looks like the new changes introduced by @ckittl break the SystemParticipantTypeInputFactory in the sense that the factory no longer can produce ChpTypeInput entities at it requires not a simple, but a complex factory.

@sebastian-peter please consult @ckittl or me when you're back in office so that we can talk about this asap or add any suggestions for a solution below.

I just discovered, that AggregatedGridInput as a setVoltLvl()-method that allows setting a string. In contrast, NodeInput also contains a field voltLvl which is of type VoltageLevel. @mia-krause can you explain me why we distinguish between this and why this isn't harmonized? I think this is a little bit confusing or even not correctly implemented?!

The old implementation of the reader was able to read data in PostGIS format, the new factories are now only supporting geoJSON. This is totally fine for file based stuff, but not if it comes to databases with PostGIS support. Hence, the capabilities of the factories need to be extended to also support the old PostGIS standard beside the new geoJSON standard.

Add functionalities to read and write ResultEntities from .csv files. Should be easily extendable for all other entities.

All classes in edu.ie3.datamodel.models.input.system.type are using Quantity<> instead of ComparableQuantity<>. That causes type-issues at the moment you call them within edu.ie3.simona.model.participants. Thats because there is type of ComparableQuantity<> required.

For now, the CsvFileSink is only able to process subclasses of ResultEntity. The reason is missing implementations of Processors that are able to 'de-serialize' all kinds of UniqueEntity to make them usable by the CsvFileSink.

Through creation of more Processors we can leverage the full capabilities of the CsvFileSink to write all types of UniqueEntity.

In former versions of the algorithm splitting a total grid model into galvanically separated sub grids, the transformer models have been duplicated as well as their corresponding nodes in other sub grids.

As we intend to make objects immutable (especially with 64e7629), I think that this is not necessary anymore. Therefore, if some freaky things happen, we may have to alter this once again.

Add the classes DistanceWeightedGraph and -Edge again (copy from the old datamodel), since they are needed for OSMoGrid.

Currently, the result factories are only able to build a ResultEntity based on a fieldName -> valueMapping which allows reading and creating of POJOs and can be used to read data into the data model from the outside. The other way around e.g. for writing operations is not supported yet. Hence to support writing, the factories need to be extended to allow for a creating a mapping based on a POJO.

transformerCtoG in GridTestData had a wrong node in the test data (was nodeF instead of nodeG). I changed this, but strangely(?) all tests are still passing. This could indicate that a) either one of the tests is not checking grid extraction properly or b) there is something broken with the logic of grid disassembling as this test data is also used for grid disassembling.

This needs to be investigated carefully the ensure that everything works correctly. Additionally, I would recommend to extend the tests that they break if nodes are not set correctly!

... to enable overriding the JavaDoc of what you gonna receive there.

It should be possible to give detailed model parameters as well as fixed provided time series for each model. Therefore, a mapping from model to time series is provided.

The former implementation of a topology graph edge did not reveal anything about what actual asset connects two sub grids. Added this and some more information.

When parsing, the id itself is not accounted for. Fix this.

At the moment, each NodeInput holds a non-optional geoPosition constructor field. This enforces utilization of null entries when no geo information is available. The application of Optional is also a not-that-nice way to go as this would cause issues with serialization. Hence, we need a better solution to address cases, when no geo data is available avoiding while avoiding null and Optional.

Furthermore, LineInput still contains an optional for overhead line monitoring. This should be removed asap as well.

... as suggested by https://github.com/diffplug/spotless/tree/master/plugin-gradle.

Currently an OperatorInput is required by all input entities, but it might be the case, that input entities don't have an operator. In that case putting null into the OperatorInput field in the corresponding input entity would be doable, but not indicating what exactly the null stands for (error in entity creation, no operator, ...). Hence, some default fields for the OperatorInput might make sense.

At the moment there are unused constructor parameters in the ChpTypeInput class and ChpInput class.
I'm not sure about their usage/if we need them, hence I cannot address this issue. :-(

If someone could provide more information on this I can try to fix this as well.

Remove Characteristic Input Types
Implement OlmCharacteristic as a separate class and replace the previous string in LineInput
Implement qCharacteristic as a separate class and replace the previous string in the corresponding input

Replace existing characteristics available as strings with classes

Removals:
pMin from StorageTypeInput -- not necessary
Deprecated
StorageStrategy.java

Additions:
into StorageTypeInput
cpRate = Charge/Discharge Rate (constant power)

Changes in description:
in StorageTypeInput
dod = Maximum permissible DOD (not minimum)
pMax = Maximum permissible charging/discharging power (also reference power)

I just started working with the AggregatedGridInput and discovered that there is

• no default constructor with input parameters
• fields like gridName, subnet, and voltLvl are writable by setters

Question 1:
I see the point in making grid assets e.g. measurement units, graphics, system participant etc. writable but does this hold true for gridName, subnet, and voltLvl as well? Wouldn't it be better to make these fields final and add a corresponding constructor instead.

Question 2:
Furthermore, I would question that measurement units should be placed in the AggregatedGridInput. Is there any specific reason why this has been placed here instead of AggregatedRawGridInput? Do you have any thoughts on this @ckittl?

Question 3:
@mia-krause maybe you can answer this one: we currently have setters for all our fields in the InputEntity part but we don't have an empty default constructor to make use of the setters (except from modifying objects after creation which might not be reasonable. Is there any reason, why

• these fields are not final (and correspondingly getters can be removed)?
• the empty constructor is not there?

I know that setters + empty constructors are needed for JavaBeans and Generic operations but do we want/need this at this place? And if yes, there is still an empty constructor missing. If no, we can get rid of the setters and make the fields final.

Currently, the content of system participants is not checked.

In the old data model implementation we had a so called .csv naming strategy that has been used to map entity classes to a specific output file name. I would suggest that we re-introduce this naming strategy again as it is helpful for several tasks considering file i/o operations.

I will draft a proposal for this asap.

Several InputEntity, such as AssetInput, require more complex input data types than just native types and Quantities.
A different type of factory may be useful here.

Part of # 16.

edu.ie3.models.StandardUnits does not contain any documentation. While most fields can be guessed based on their name, it would be helpful to have a javadoc-alike documentation for each field.

Actual information related to a voltage level are now distributed throughout different different places. Therefore, a joint concept is implemented.

Currently, factories and processors rely on different sources to determine the field names of serialized objects. Make both use a single point of truth for this information.

Currently, especially the IndividualTimeSeries "only" offers the possibility to query a value at a certain instant in time and giving it back or null, if it not exists. Anyways, the user does not have any possibility to determine, if the query is totally out of scope or what the next available value would be.

Subclasses of InputEntity

Currently, there are ValidationUtils and validate methods in each model. As of now, the models are (partly) as well validated during instantiation and later during composing to a grid container.

Let's develop a thorough validation strategy. My suggestions:

1. Validate all single models during instantiation (without their interplay in a bigger grid)
2. Move validation logic from ValidationUtils to the single classes, as the classes should hold the information on what is valid and what not.
3. During composition of the single containers, only validate the interplay between the models (are the nodes apparent?, does a line connect different voltage levels? etc.)

Currently, it is not possible to attach information on which standard load profile may be applied to a certain node. Add this possibility.

Let all related input models have sRated (rated apparent power) and cosphiRated (rated power factor) instead of active power and others.

The graphic input relies on package com.vividsolutions.jts.geom whereas the rest uses org.locationtech.jts.io.geojson. Search for occurences of com.vividsolutions.jts.geom throughout the project.

For PR #26 I created two new StandardUnits. It would be amazing if someone could check the naming, the unit, and whether this makes sense in general.

Checking the TODO markers in StandardUnits.java should suffice.

Currently, measurementUnits are placed in the wrapper class for a complete grid (AggregatedGridInput) instead of the class with all raw grid elements AggregatedRawGridInput. Maybe it makes sense to put the measurement units also in the raw grid elements class as they logically belong there?!