Hello,

I am facing a problem when I start my trainer and I can't figure out the cause.

My input data is of dimension 42 and my output should be one value out of 4.

This is the shape of my training and test set:

Training set: 
input = (1152, 42) target = (1152,)

Training set: input = (1152, 42) target = (1152,)
Test set: input = (384, 42) target = (384,)

This is the construction of my network:

ls = MS.GradientDescent(lr=0.01)
cost = MC.CrossEntropy()

i = ML.Input(42, name='inp')
h = ML.Hidden(23, activation=MA.Sigmoid(), initializations=[MI.GlorotTanhInit()], name="hid")
o = ML.SoftmaxClassifier(4, learningScenario=ls, costObject=cost, name="out")

mlp = i > h > o

And this is the construction of the datasets, trainers and recorders:

trainData = MDM.RandomSeries(distances = train_set[0], next_state = train_set[1])
trainMaps = MDM.DatasetMapper()
trainMaps.mapInput(i, trainData.distances)
trainMaps.mapOutput(o, trainData.next_state)

testData = MDM.RandomSeries(distances = test_set[0], next_state = test_set[1])
testMaps = MDM.DatasetMapper()
testMaps.mapInput(i, testData.distances)
testMaps.mapOutput(o, testData.next_state)

earlyStop = MSTOP.GeometricEarlyStopping(testMaps, patience=100, patienceIncreaseFactor=1.1, significantImprovement=0.00001, outputFunction="score", outputLayer=o)
epochWall = MSTOP.EpochWall(1000)

trainer = MT.DefaultTrainer(
        trainMaps=trainMaps,
        testMaps=testMaps,
        validationMaps=None,
        stopCriteria=[earlyStop, epochWall],
        testFunctionName="testAndAccuracy",
        trainMiniBatchSize=MT.DefaultTrainer.ALL_SET,
        saveIfMurdered=False
    )

recorder = MREC.GGPlot2("MLP", whenToSave = [MREC.SaveMin("test", o.name, "score")], printRate=1, writeRate=1)

trainer.start("MLP", mlp, recorder = recorder)

But the following error is being produced:

Traceback (most recent call last):
  File "nn-mariana.py", line 82, in <module>
    trainer.start("MLP", mlp, recorder = recorder)
  File "SUPRESSED/Mariana/Mariana/training/trainers.py", line 226, in start
    Trainer_ABC.start( self, runName, model, recorder, trainingOrder, moreHyperParameters )
  File "SUPRESSED/Mariana/Mariana/training/trainers.py", line 110, in start
    return self.run(runName, model, recorder, *args, **kwargs)
  File "SUPRESSED/Mariana/Mariana/training/trainers.py", line 410, in run
    outputLayers
  File "SUPRESSED/Mariana/Mariana/training/trainers.py", line 269, in _trainTest
    res = modelFct(output, **kwargs)
  File "SUPRESSED/Mariana/Mariana/network.py", line 47, in __call__
    return self.callTheanoFct(outputLayer, **kwargs)
  File "SUPRESSED/Mariana/Mariana/network.py", line 44, in callTheanoFct
    return self.outputFcts[ol](**kwargs)
  File "SUPRESSED/Mariana/Mariana/wrappers.py", line 110, in __call__
    return self.run(**kwargs)
  File "SUPRESSED/Mariana/Mariana/wrappers.py", line 102, in run
    fres = iter(self.theano_fct(*self.fctInputs.values()))
  File "SUPRESSED/Theano/theano/compile/function_module.py", line 871, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "SUPRESSED/Theano/theano/gof/link.py", line 314, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "SUPRESSED/Theano/theano/compile/function_module.py", line 859, in __call__
    outputs = self.fn()
ValueError: Input dimension mis-match. (input[0].shape[1] = 1152, input[1].shape[1] = 4)
Apply node that caused the error: Elemwise{Composite{((i0 * i1) + (i2 * log(i3)))}}[(0, 1)](InplaceDimShuffle{x,0}.0, LogSoftmax.0, Elemwise{sub,no_inplace}.0, Elemwise{sub,no_inplace}.0)
Toposort index: 18
Inputs types: [TensorType(int32, row), TensorType(float64, matrix), TensorType(int32, row), TensorType(float64, matrix)]
Inputs shapes: [(1, 1152), (1152, 4), (1, 1152), (1152, 4)]
Inputs strides: [(4608, 4), (32, 8), (4608, 4), (32, 8)]
Inputs values: ['not shown', 'not shown', 'not shown', 'not shown']
Outputs clients: [[Sum{axis=[1], acc_dtype=float64}(Elemwise{Composite{((i0 * i1) + (i2 * log(i3)))}}[(0, 1)].0)]]

Versions:

Mariana (1.0.1rc1, /media/guilhermevrs/Data/Documentos/Academico/TCC-code/Mariana)
Theano (0.8.0.dev0, SUPRESSED/Theano)

This code was produced having as base the tutorial code from the mnist example.

Could you please help me to figure out what's going on?

Thank you in advance

The current version shows layer connectivity but not the hyper-parameters values. An improved visualization that contains information about hyper-parameters would allow the sharing of an entire model through a visual and small HTML file.

Training crashes with TypeError when using Binomial Dropout.

  File "mlp.py", line 94, in train
    res_train = mlp.train(o, inp=list(train_set['expressions']), target=train_set['class'])
  File "build/bdist.linux-x86_64/egg/Mariana/network.py", line 202, in __getattribute__
  File "build/bdist.linux-x86_64/egg/Mariana/network.py", line 129, in init
  File "build/bdist.linux-x86_64/egg/Mariana/layers.py", line 384, in _setTheanoFunctions
  File "build/bdist.linux-x86_64/egg/Mariana/wrappers.py", line 51, in __init__
  File "/u/seguinj/repos/gvhd-ml/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/compile/function.py", line 310, in function
    output_keys=output_keys)
  File "/u/seguinj/repos/gvhd-ml/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/compile/pfunc.py", line 499, in pfunc
    no_default_updates=no_default_updates)
  File "/u/seguinj/repos/gvhd-ml/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/compile/pfunc.py", line 215, in rebuild_collect_shared
    raise TypeError(err_msg, err_sug)
TypeError: ('An update must have the same type as the original shared variable (shared_var=<CudaNdarrayType(float32, matrix)>, shared_var.type=CudaNdarrayType(float32, matrix), update_val=Softmax.0, update_val.type=TensorType(float64, matrix)).', 'If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.')

• L1 or L2 @ 0.0

Epoch 0
    Train score: 0.735989272594
    Validation score: 0.718354940414 +best+

• L1 or L2 @ 0.01

Epoch 0
    Train score: 0.915873587132
    Validation score: 0.704464316368 +best+

• L1 or L2 @ 0.1

Epoch 0
    Train score: 2.71569609642
    Validation score: 0.699838221073 +best+

• L1 or L2 @ 1.0

Epoch 0
    Train score: 20.8013763428
    Validation score: 0.702481269836 +best+

Hi,
I hope your well.
I was looking for the above file.
It comes with an error when i run the examples.
Many thanks,
Best,
Andrew

Downsampling has changed place in theano. Mariana will be updated very soon

Using save() on models with many layers may crash due to pickle limitations. This is also true for the auto-saves performed by the trainer and the recorder. It will be fixed in the upcoming version.

Hello @tariqdaouda,

First of all, thank you for this framework. It works very well!

I was wondering how does exactly work every observation made during the training phase.

I mean, supposing I am interested in getting the percentage error of my network on a given test set. What data should I look in this case?

Thank you in advance

Add support for tensor flow.

ubgpu@ubgpu:~/github/Mariana$ sudo python Mariana/examples/mnist_mlp.py
Using gpu device 0: GeForce GTX 970

|/| /-\ |-> | /-\ || /->

Run device:

I will use the [-GPU-] to run function 'train' of layer 'out'!

Cheers :),

Mariana

Run device:

I will use the [-GPU-] to run function 'test' of layer 'out'!

Cheers :),

Mariana

Run device:

I will use the [-GPU-] to run function 'propagate' of layer 'out'!

Cheers :),

Mariana

ERROR (theano.gof.opt): Optimization failure due to: local_argmax_pushdown
ERROR (theano.gof.opt): TRACEBACK:
ERROR (theano.gof.opt): Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/gof/opt.py", line 1493, in process_node
replacements = lopt.transform(node)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/nnet/nnet.py", line 1448, in local_argmax_pushdown
return tensor._max_and_argmax(pre_x, axis)
File "/usr/local/lib/python2.7/dist-packages/theano/gof/op.py", line 507, in call
node = self.make_node(_inputs, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/basic.py", line 1252, in make_node
raise TypeError("MaxAndArgmax needs a constant axis")
TypeError: MaxAndArgmax needs a constant axis

ERROR (theano.gof.opt): Optimization failure due to: local_argmax_pushdown
ERROR (theano.gof.opt): TRACEBACK:
ERROR (theano.gof.opt): Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/gof/opt.py", line 1493, in process_node
replacements = lopt.transform(node)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/nnet/nnet.py", line 1454, in local_argmax_pushdown
('x', 0))(pre_bias), axis)
File "/usr/local/lib/python2.7/dist-packages/theano/gof/op.py", line 507, in call
node = self.make_node(_inputs, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/basic.py", line 1252, in make_node
raise TypeError("MaxAndArgmax needs a constant axis")
TypeError: MaxAndArgmax needs a constant axis

==>rec: ggplot2, commit 1, pid: 14344:
|-test set
|->out: 1.62696 +best+
|-train set
|->out: 1.96462 +best+
|-validation set
|->out: 1.61709 +best+

==>rec: ggplot2, commit 2, pid: 14344:
|-test set
|->out: 1.48826 +best+
|-train set
|->out: 1.5747 +best+
|-validation set
|->out: 1.48133 +best+

==>rec: ggplot2, commit 3, pid: 14344:
|-test set
|->out: 1.38383 +best+

==>rec: ggplot2, commit 1001, pid: 14344:
|-test set
|->out: 0.119731 +best+
|-train set
|->out: 0.0630581 +best+
|-validation set
|->out: 0.124834 (best: 0.124825 @ commit: 974)

Dying gracefully from NameError, and saving myself to:

...dx-xb_MLP_death_by_NameError_Sat_Jun_13_21:45:02_2015

Traceback (most recent call last):
File "Mariana/examples/mnist_mlp.py", line 74, in
trainer.start("MLP", mlp)
File "/home/ubgpu/github/Mariana/Mariana/training/trainers.py", line 195, in start
Trainer_ABC.start( self, runName, model, recorder, trainingOrder, shuffle, datasetName )
File "/home/ubgpu/github/Mariana/Mariana/training/trainers.py", line 103, in start
return self.run(runName, model, recorder, _args, *_kwargs)
File "/home/ubgpu/github/Mariana/Mariana/training/trainers.py", line 340, in run
raise EndOfTraining(crit)
NameError: global name 'EndOfTraining' is not defined
ubgpu@ubgpu:~/github/Mariana$ git remote -v

Make Mariana compatible with python 3.

I am working on it. But the rest of Mariana is fine

The gradient returned by the cost is 3D (because of the convolution), but the parameters of the embedding layer are 2D. This causes theano to crash.
The next update will have an Embedding layer for conv nets.

Hello again @tariqdaouda,

Looks like I am facing a challenge here hehe.

What I am trying to do is actually use the already trained model in my system.

So I trained my model with the parameters I experimented to get me the best results and your trainer saved the parameters into the pkl file.

To load the model I used the code in the documentation:
`import cPickle

mlp = cPickle.load(open("myMLP.mariana.pkl"))

But mlp is a dict object, don't have the propagate function.

So I looked it up in the code and found the LoadModel method. I tried to load the model with the following code:

mlp_state = MN.loadModel("mlp_state/state-config")

Worked! Now I have the method propagate. To get the output layer result, I used this code (from test.py in the repository):

o = mlp_state.outputs.values()[0] mlp_state.classify(o, inp = numpy.asarray(inputSet))
But first I was getting an error that the dimensions of the input are wrong:
TypeError: ('Bad input argument to theano function with name "/media/guilhermevrs/Data/Documentos/Academico/TCC-code/Mariana/Mariana/wrappers.py:58" at index 1(0-based)', 'Wrong number of dimensions: expected 2, got 1 with shape (42,).')

My input is of 42 nodes. I even trained the model with 42.

So I placed my input data as the first element of an array. Now the error I am getting this one:

TypeError: Tried to provide value for implicit input: <TensorType(float64, matrix)>

Can you give me a light here, please?

Decorators may not be applied to layers. This issue has been fixed in dev branch and will be part of the next merge.

Composite layer is broken and may disappear in the future to be replaced by an automatic layer concatenation.