teddykoker / learning-to-learn-jax Goto Github PK

Thanks a lot for the elegant and clean reimplementation of the paper.

I noticed that the meta parameters saved are only from the first model in each episode. Therefore not much learning is tested. The reasoning is as follows:

1. meta_loss is initialized to zero in each episode, here. meta_loss = 0.
2. In each outer loop step, meta_loss accumulates the loss, here. meta_loss += loss.
   Based on that, the least and best meta_loss is only at the first iteration of the outer loop, or when the meta parameters are not updated.
3. The meta parameters are saved when meta_loss < best_loss, here.

A quick fix could be to change the comparison from ( if meta_loss < best_loss) to ( if loss < best_loss).

I hope I didn't miss something. Thank you again for sharing your elegant code.