ducha-aiki / lsuv-keras

Simple implementation of the LSUV initialization in keras

Layer-sequential unit-variance (LSUV) initialization for Keras

This is sample code for LSUV and initializations, implemented in python script within Keras framework.

Usage:

from lsuv_init import LSUVinit
...
batch_size = 32
model = LSUVinit(model, train_imgs[:batch_size,:,:,:])

LSUV initialization is described in:

Mishkin, D. and Matas, J.,(2015). All you need is a good init. ICLR 2016 arXiv:1511.06422.

Original Caffe implementation https://github.com/ducha-aiki/LSUVinit

Torch re-implementation https://github.com/yobibyte/torch-lsuv

PyTorch implementation https://github.com/ducha-aiki/LSUV-pytorch

New! Thinc re-implementation LSUV-thinc

lsuv-keras's Issues

need to remove line #66 #67 of "lsuv_init.py"

as title

Initialization failure

D:\jupyter notebook\lsuv_init.py in LSUVinit(model, batch, verbose, margin, max_iter)
64 weights_and_biases = layer.get_weights()
65 weights_and_biases[0] /= np.sqrt(variance) / np.sqrt(needed_variance)
---> 66 layer.set_weights(weights_and_biases)
67 weights /= np.sqrt(variance) / np.sqrt(needed_variance)
68 layer.set_weights([weights, biases])

UnboundLocalError: local variable 'weights' referenced before assignment

Runtime Error in TensorFlow 2.4.1

If I run your code, the following error arises:

RuntimeError: The layer has never been called and thus has no defined input.

It is due to model.get_input_at(0) in def get_activations(model, layer, X_batch):, but I can not manage how to fix it...

Thanks in advance.