Reimplement does not give idea result about waifu2x

[Zotikus1001]

It looks like you need to feed it high quality images for training..

[xhwang]

@Brokensilence Thanks for your reply. The question is that the training data fed to Caffe and waifu are the same(generated from VOC 2007 dataset), however waifu can get better result.

[nagadomi]

I tried SGD at first but I didn't get good result.
In this task, Adam is pretty better than other optimizers, I think.

Original SRCNN uses SGD. Probably initial weights and layer-wise learning rate settings is important. (I was not able to reproduce this paper with SGD)

The filter weights of each layer are initialized by drawing randomly from a Gaussian distribution with zero mean and standard deviation 0.001 (and 0 for biases). The learning rate is 10^4 for the first two layers, and 10^5 for the last layer. We empirically find that a smaller learning rate in the last layer is important for the network to converge (similar to the denoising case [22]).

[xhwang]

@nagadomi Many thanks! I will try to use Adam to train a network. The result will be updated as soon as possible. :-)

[nagadomi]

The loss(MSE) of waifu2x(in 2x scaling) is 0.000350.00028. not 0.0020.
EDIT: RGB values is scaled 0.01.0.

[xhwang]

Got it, RGB values has been scaled.

[xhwang]

Adam Solver really helps. Now it achieves comparable results. 👍