ilyasu123 / rlntm Goto Github PK

First of all, thanks for sharing source code for your awesome work.
I am trying to apply REINFORCE to train my model, and trying to use idea of reducing variance of gradient.

In the Appendix A - 'ONLINE BASELINE PREDICTION' section, paper derives closed form solution for b_t, which results in vector with size #theta.

However, the last line says, practically b_t is chosen as scalar (which seems expected future rewards).
Can you provide any reference or explanation how this can substitute closed form solution of b_t (as a vector) ?

I really appreciate your help 👍

when I run the cmd "torch rlntm_pkgs/RL_check_grad.lua " , "python exprs/train_reverse.py" and "python exprs/train_repeat_copy.py", I come across the same error as fllows:

/home/wangzhg/torch/install/bin/luajit: /home/wangzhg/torch/install/share/lua/5.1/trepl/init.lua:389: /home/wangzhg/torch/install/share/lua/5.1/trepl/init.lua:389: /home/wangzhg/torch/install/share/lua/5.1/strict.lua:13: another global metatable exists
stack traceback:
[C]: in function 'error'
/home/wangzhg/torch/install/share/lua/5.1/trepl/init.lua:389: in function 'require'
rlntm_pkgs/RL_check_grad.lua:64: in main chunk
[C]: in function 'dofile'
...gzhg/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00405e90

Is there any solution? thanks!

P.S - Idk what all this code about but I am getting started with ML and when I wanted to run this code by copying it into my VScode it didn't ran.