Do the results vary significantly between Tonic + TensorFlow vs. Pytorch? And are the default hyperparams in Tonic the same as the ones used to generate this data? about tonic_data

[fabiopardo]

The TensorFlow and PyTorch sections of the library are written to be as similar as possible. For example the default arguments for tf.keras.layers.Dense and tf.keras.optimizers.Adam are slightly modified to match PyTorch. When I compared results for both frameworks I got very similar results but I could not afford to run the same large scale benchmark for PyTorch because the off-policy agents are much slower to run as shown in the paper.

The hyperparameters used for the benchmark are the default ones, except for the number of steps per epoch in tonic.Trainer. I used 500,000 for the benchmark to get only 100 points per graph, making the benchmark lighter. The defaut hyperparameters for both frameworks should be identical.