q_networks.AC_net_keras, q_networks.q_network_keras and q_networks.q_network_theano only use 95mb of GPU about deer

[VinF]

It doesn't use much memory in the GPU because it loads a different mini-batch of tuples for every update from the replay memory (but there is indeed no reason that it explicitly says that the GPU has no more space).
Note that if you don't work with images (first four examples in the doc), you will probably be at the same speed (or even higher) with cpu only since the neural network would be quite simple.