My networks have a 53% error on the training set about neuralnetworks

[yeison]

I got this from the IristTest.java DBN test.

    DBN dbn = NNFactory.dbn(new int[]{30, 121, 2}, true);
    dbn.setLayerCalculator(NNFactory.lcSigmoid(dbn, null));

    SimpleInputProvider trainInputProvider = new SimpleInputProvider(input, target, input.length, 1);
    SimpleInputProvider testInputProvider = new SimpleInputProvider(input, target, input.length, 1);


    // rbm trainers for each layer
    AparapiCDTrainer firstTrainer = TrainerFactory.cdSigmoidTrainer(dbn.getFirstNeuralNetwork(), null, null, null, new NNRandomInitializer(new MersenneTwisterRandomInitializer(-0.01f, 0.01f)), 0.01f, 0.5f, 0f, 0f, 1, true);
    AparapiCDTrainer lastTrainer = TrainerFactory.cdSigmoidTrainer(dbn.getLastNeuralNetwork(), null, null, null, new NNRandomInitializer(new MersenneTwisterRandomInitializer(-0.01f, 0.01f)), 0.01f, 0.5f, 0f, 0f, 1, true);

    Map<NeuralNetwork, OneStepTrainer<?>> map = new HashMap<>();
    map.put(dbn.getFirstNeuralNetwork(), firstTrainer);
    map.put(dbn.getLastNeuralNetwork(), lastTrainer);

    // deep trainer
    DBNTrainer deepTrainer = TrainerFactory.dbnTrainer(dbn, map, trainInputProvider, null, null);

    Environment.getInstance().setExecutionMode(Kernel.EXECUTION_MODE.SEQ);

    // layer pre-training
    deepTrainer.train();

    // fine tuning backpropagation
    BackPropagationTrainer<?> bpt = TrainerFactory.backPropagation(dbn, trainInputProvider, testInputProvider, new MultipleNeuronsOutputError(), new NNRandomInitializer(new MersenneTwisterRandomInitializer(-0.01f, 0.01f)), 0.01f, 0.5f, 0f, 0f);

    // log data
    bpt.addEventListener(new LogTrainingListener(Thread.currentThread().getStackTrace()[1].getMethodName(), true, true));

    // training
    bpt.train();

    // testing
    bpt.test();

[ivan-vasilev]

Is it possible to provide the full source code of your solution so I can be able to debug it? I presume you are using it with custom input and target.

[yeison]

This is the full source code:
https://gist.github.com/yeison/7af98cdb69a213f32cae#file-testclassifier-java

And this is the input file that corresponds to that source code:
https://drive.google.com/file/d/0BynBBBeEroDZQ3EwMFJCS2FYc1U/edit?usp=sharing

The SimpleInputProvider is from your project, of course.

[ivan-vasilev]

Although I haven't run your sample yet, here are some initial comments:

• I noticed that the data is not normalized, i.e. there are values that are not in the 0..1 range. I would suggest to normalize the input first. I just committed modified ScalingInputFunction, which allows you to do that. This leads me to the second point.
• By your code snippet, I noticed that you are using older version of the library. Is it possible to migrate to the latest sources?

[yeison]

Oh cool, thanks for that. I thought normalization might be a concern but wasn't certain. Let me try it out with the new source.

On May 15, 2014, at 2:19 PM, Ivan Vasilev [email protected] wrote:

Although I haven't run your sample yet, here are some initial comments:

I noticed that the data is not normalized, i.e. there are values that are not in the 0..1 range. I would suggest to normalize the input first. I just committed modified ScalingInputFunction, which allows you to do that. This leads me to the second point.

By your code snippet, I noticed that you are using older version of the library. Is it possible to migrate to the latest sources?

—
Reply to this email directly or view it on GitHub.

[yeison]

Thanks for the help.

The error is down to 15% on the training set. That's certainly a huge improvement. Does it sound like an appropriate value to you? The DBN dimension for those results is 30 neurons at the input layer and 10 neurons in the hidden layer.

On the other hand, the program is running about two or three orders of magnitude slower. Perhaps even slower than that. What do you think?

[ivan-vasilev]

Thanks a lot for the feedback. I'm not sure about the error. Perhaps you might try adding another layer? If you prefer to use the current configuration (one hidden layer) you can try simple multilayer perceptron also.

I'll see what I can do about the performance.