qthen / candle Goto Github PK

SGD is unable to optimize the baseline neural network (a simple shallow two hidden layer fully connected layer with ReLU). The training loss more or less stays the same and changing the learning rate seems to have no effect after trying values from 0.1 to 1e-6.

It seems the SGD with momentum is also no better, strangely, adaptive learning optimizers like Adam don't have this problem. Furthermore, this problem only starts to happen when the number of PCA components is above 30, which seems strange since the error signal should be stronger.

Occurs when loading in fits file for the spectra data of APOGEE, perhaps make it synchronous?

OSError: [Errno 24] Too many open files: '/Users/philiptsang/Documents/Y4/CSC495/astro/data/dr14/apogee/spectro/redux/r8/stars/l31c/l31c.2/5018/aspcapStar-r8-l31c.2-2M19023984+3838476.fits'