simontindemans / rangl-zeepkist Goto Github PK

I am a little concerned that the time of the second peak remains effectively constant after instantiation of the agent. So all training is done using the knowledge that the peak occurs at that specific (but random) instant of time.

Should we train over a large number of scenarios where the peak occurs at different times? This could be done by modifying the environment reset() function (in a wrapper) to resample the peak time.

As an intermediate step, we could run evaluations on evaluations with different initialisations.

To do: reduce observation time series from 96 to roughly 25 (time needed to fully ramp up and down the slow generator). That should be a conservative action horizon.

We'll need to modify the plotting function (inside PlotWrapper) to show other relevant aspects. For example, also plot the sum of generator outputs, and the difference with realised demand.

Only if there is time: compare different RL architectures and related parameters (learning rate, gamma, ...). Currently SAC (soft actor critic) is used with gamma=0.85 (to provide exponential averaging on a short-ish time scale)

I have made a bit of an effort to include the current total outturn, but it's not really relevant. Can be removed for simpler code and without loss of performance