abyoussef / drbm_project

Learning joint representation with a Multimodal Deep Boltzmann Machine

Multimodal Deep Boltzmann Machine:

This is the code of my project joint representation using multimodal Deep Boltzmann Machines. It is realized in the course of Probabilistic Graphical Model and the course Object Recognition and Computer Vision in the MVA Master.

The goal is to implement a Multi-DBM model with Tensorflow. This is inspired from the paper of Nitish Srivastava et al appeared at NIPS2012.

In the code directory, you can find :

• MDBMCode is an implementation of the Multimodal Deep Boltzmann Machine on the top of Deep_Learning_Tensorflow library.

• txtGen is the code for generating tags to images without annotations.

• TrainingLog.txt is the log of my training.

drbm_project's Issues

some problems in 'Code/MDBMCode/run_flickr_multimodal_dbm.py'

line77: binary_output = utilities.sample_prob(hprobs, np.random.rand(data.shape[0], joint_representation_units))
the variable 'hprobs' and 'data' are not defined.

line118: labeled = np.concatenate((labeled1, labeled2, labeled3), axis=0)
the variable 'labeled' is not used in the code.

line123: shuff = zip(train_set, train_ref)
the variable 'train_set' and 'train_ref' are not defined.

Could you tell me how to solve these problem? Thank you!

rbm.py run error

Hi, when run project, I found some self parameter have no definition in RBM.py , about three

1. In def _train_mode function, "self._run_validation_error_and_summaries" can not definition

2. In build_model function, "self.compute_regularization" and " self._create_cost_function_node"
   can not find definition.

I will appreciate for your reply.