Performance with QR prioritization on Space Invaders about rainbow

[Kaixhin]

I got Pong working after 1.0, but yes I recently observed this drop in Space Invaders. I don't think the epsilon greedy value is the problem (I think that was needed for getting correct scores on Pong, because with too much noise the agent would miss the ball and lose points). The other changes between 1.0 and master are a) adding gradient clipping b) using the prioritised loss for the prioritised weights. I'm going to try Space Invaders and disable gradient clipping, so if you're able to revert b) and run with default settings (no QR) then that would help find the problem.

[marintoro]

Hum... I did a sanity check with your release v1.0 this week-end (commit 952fcb4).
The performance I got are really different from the one you are showing.
I opened a new issue there #26

[Kaixhin]

Closing this as I don't know what performance changing to QR in Rainbow should cause, but leaving the other issue open until the issue there is resolved.