Min/max values for continuous actions about tensorforce

[AlexKuhnle]

The beta distribution is now implemented and will be used instead of Gaussian if min_value and max_value are given for an action.

[michaelschaarschmidt]

Min max values are not supported yet, this does not work well in a Gaussian - Alex wrote this not as a bug but as a task for himself.

We will implement a Beta distribution for this purpose - see this paper: http://proceedings.mlr.press/v70/chou17a.html

I will try to get to this in the coming week

[michaelschaarschmidt]

Will do this via Beta distribution

[AdamStelmaszczyk]

I also ran on this issue with PPO and TRPO. Passed actions in Config to the algorithms is:

{'max_value': 1.0, 'shape': (18,), 'min_value': 0.0, 'continuous': True}

Yet, the action returned by TensorForce Model get_action() is:

{'action': array([  1.37754471e+02,   1.57470112e+01,   6.00896423e+02,
        -1.48294473e+00,  -2.35775032e+01,   1.75852025e+00,
         1.50085914e+00,   1.04522383e+00,   9.40244770e+00,
        -5.31497070e+02,  -7.35334206e+00,   6.55987244e+01,
        -6.53353786e+00,   4.18444443e+00,  -1.60262108e-01,
         1.29556608e+00,   1.71527648e+00,   9.04080963e+01], dtype=float32)

[AlexKuhnle]

Note that the extreme values close to min_value and max_value cannot reliably be learned (at least in the current implementation).