First order approximate typo? about few_shot_meta_learning

[cnguyen10]

The one mentioned is in the task-adaptation (inner loop) step, where gradient descent is used to optimize the task-specific parameter (denoted as q in the code). Hence, we have to calculate the gradient w.r.t. q. The first order or higher order is at the argument retain_graph or create_graph. According to the document of PyTorch, retain_graph keeps the graph used to compute the grad, while create_graph will construct the graph of the derivative. In the case of create_graph, higher order derivative will be calculated when a backward() is subsequently called.

I think I know what went wrong here. This is a bug caused by argparse, where boolean variables are always set to True no matter what. Therefore, when you change the first_order to False, it would still be the same. I corrected it in the new code. Please make a git pull to get the latest update. You can give higher order a try by setting the first_order as False. Note that this will lead to a higher amount of memory and longer computation time.