Problem with implementation of optimizers. about ensmallen

[rcurtin]

Hey there @SinghKislay, I think that this is not specific to ensmallen but instead mlpack. In any case, I'd suggest taking a careful look at src/mlpack/tests/feedforward_network_test.cpp and src/mlpack/tests/convolutional_network_test.cpp and related files to get an idea of how to build neural networks with mlpack. The strategy you described for building a network is pretty straightforward, but you don't need a special ResNet struct---you can just build an FFN<> and add the right layers.

[SinghKislay]

Hi, @rcurtin I have read the examples from test/convolutional_network_test.cpp, model/kaggle/DigitRecogniser/src/DigitRecogniser.cpp I have also read the example from vae in model reository. All of them use Add() method to add the layer and then they just use Train() method with input and target. I understad this and have also built one. But what I want to do is modify the outputs of each layer which is not done in any of the examples. Could u you point me to an example Where the Feedforward() method is used so that I could do f(x)+x. I don't know why is this so hard am I missing something here. Thankyou

[zoq]

@SinghKislay Sorry for the slow response, in this case you have to manually step through each layer or implement a new one that you could add right after each layer you like to modify. Here is the code to get the output after each layer:

https://github.com/mlpack/mlpack/blob/6faf0b782705c29267fbfeb1eca527b6006e5a04/src/mlpack/methods/ann/ffn_impl.hpp#L134-L146

in this case network[begin + i].OutputParameter() does contain the result. Let me know if I should clarify anything.

[SinghKislay]

@zoq thanks for reply, I understand we have to add layers manually, which is why I made the resnet struct on top of ffn class. Where I am facing problem is propagating the gradients back and the loss function. Say I push the input all the way through to the last layer and then push it into the loss fuction. How do I backprop. All the examples that I saw regarding back prop were using simple train method which we can't use in this case.

[mlpack-bot]

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! 👍