Batch training vs online training about somoclu

[peterwittek]

This is related to issue #13. Basically, a brand-new object is created every time you call train. I decided yesterday that this issue will not be resolved for a while, because I am creating a C wrapper around the training methods to interface Somoclu with Julia (issue #11). Since release 1.6.2, there is very little memory copy done in single-process execution, which makes calling train repeatedly more efficient.

The second problem is that Somoclu was initially designed for supercomputers and GPU clusters, and the wrappers were an afterthought. So it is quintessentially batch-oriented.

In short, your hack is bound to be somewhat inefficient, but it should work in principle. Let me do some tests to see if it is a bug, because it looks like one.

[oliviaguest]

Oh, I guess it works if I create a matrix from the pattern vector and send that.
So if all my patterns are called patterns but I just want to train on patterns[0] if I send train() np.matrix(train_patterns[0]) it works.

I think this is not a bug. Indeed it's a feature as perhaps only I would like to do online training! 😉 So it stops you sending a non-matrix to train with, which is sensible.

Do you think there is any use for an option to train internally in an online pattern-wise manner? I assume me doing it in a loop outside the SOM object is not efficient for larger SOMs than the one I am currently playing with.

[oliviaguest]

Hmm, so I can't actually tell if it works. I am having trouble using the standard code to get the BMUs, i.e., it doesn't seem to think there are any, which makes me think hack is not working. I'll let somebody who knows more get back to me.

[peterwittek]

This works for me:

new_data = np.array([[0.2, 0.3]])
som.update_data(new_data)
som.train(epochs=1, scale0=0.001, radius0=5)
som.view_umatrix(bestmatches=True)

[oliviaguest]

Thanks for that. Mine code is similar. It loops through every pattern and does a single epoch like yours above, but mine basically returns no BMUs/has not managed to train meaningfully in most cases. I just tried running train(epochs=1, scale0=0.001, radius0=5) with those parameters and it actually trains it, so obviously you know what good values to choose!

I am doing something like this now, but my U-matrix and quantization errors look terrible (close to untrained):

for pattern in patterns:
    som.update_data(data=np.matrix(pattern))
    som.train(epochs=1, scale0=0.001, radius0=5)

[oliviaguest]

This is what U-matrix looks like after above:

This is what it should look like, I think:

[oliviaguest]

Edit: Actually no - below does not work.

Holy crap! I got it. Looks bad(?) but this works:

radii = range(min(n_columns, n_rows)/2, 0, -1)
for pattern in patterns:
    som.update_data(data=np.matrix(pattern))
    for radius in radii:
        som.train(epochs=1, scale0=0.001, radius0=radius)

[peterwittek]

Getting the parameters right for a continued training is an incredible pain. Try setting scaleN too, perhaps with two epochs, with a very small starting radius. I never really understood why a trained map is so fragile, but it is.

[oliviaguest]

Argh, yeah, I am seeing this. Very small implies below 0.001?

[oliviaguest]

Haha OK, I am only making it worse each time. I'll switch to batch for now and try again tomorrow or soon.

[peterwittek]

Hi, do you think we can add something to your questions? A release is planned for 30 Sept, and I would like to close some pending issues till then. Thanks.

[oliviaguest]

I can't comment on this further as I haven't has time to try anything out – but please feel free to close this issue!