Question about Deep Q implementation about l2rpn-baselines

[Tezirg]

Hello @AlexTo

It seems the BaseDeepQ implements the DoubleQ update, where the target is defined as follows:

Y = R(t+1) + γ * Q(S(t+1), argmax[Q(S(t+1), a; θt)]; θ′t). (4)

Just made a quick check: it looks correct, let us know if you find the formula is not implemented correctly.

van Hasselt, Hado, et al. “Deep Reinforcement Learning with Double Q-Learning.” ArXiv:1509.06461 [Cs], Dec. 2015, http://arxiv.org/abs/1509.06461.

[AlexTo]

Ah, I see, at first, I thought the formula is from Mnih, et al Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533. https://doi.org/10.1038/nature14236

Looks like it is correct according to DoubleQ
Thank you