meljan / pydeep Goto Github PK

The following link gives an error.

Can it be updated?

Is centering supposed to be a delicate operation? I tested out the mnist example with
update_offsets = 0.01
the vis prob immediately diverges

Epoch Recon. Error Log likelihood Expected End-Time
vis = (-0.9625000342860309, 1.077954957023838)
vis = (-1069.8595390064602, 1327.4006511194739)
vis = (-53567281.491539374, 66476566.5927559)
vis = (-4938184234886.023, 6128247022497.816)
vis = (-4.5004117525694554e+17, 5.584974883656288e+17)
vis = (-6.1943772698263585e+22, 7.687172501964909e+22)
vis = (-1.128616196508766e+28, 1.4006036463642153e+28)
vis = (-2.0463202112046235e+33, 2.5394669669882995e+33)
vis = (-4.604357086551687e+38, 5.713970209302338e+38)
vis = (-1.0258660997795336e+44, 1.2730916005612099e+44)
vis = (-2.7208673874176135e+49, 3.3765746016041543e+49)
vis = (-7.146736158979544e+54, 8.869042243796985e+54)
vis = (-2.1715736576845397e+60, 2.6949054893151322e+60)
vis = (-7.46573948398941e+65, 9.264922811161907e+65)

soon the values are all nan.

without update_offset, the model runs just fine with really good sampling results. But, there is a whole paper on the merit of centering RBM, so why doesn't it work with MNIST data?

Add DBN example to tutorial page and repo

I used the gaussian binary model to train a single class of mnist images.

plotting the prob of the samples generated from random input

You can make out the number 4, but the images are not sharp at all.

sampling the states instead of prob gets pure noise

can you offer any insights into how to sharpen the synthesized images?

Hi Jan,

Thanks for your nicely written code!
I have a question regarding the unnormalized log probability of visible units of GRBM in line 906 of model.py in rbm folder.

It seems that you are not computing it following the Eq. (15 - 20), i.e., the marginal distribution of visible units, in your PLOS one paper.
According to the paper, it should be the log of the sum of two exponential functions.
But it seems not to be the case in the current implementation.
Could you help verify if it is the case?

Best,

There is an error while installing the package

I really appreciated the examples and well-written package documented at https://pydeep.readthedocs.io/
I am running:

• iMac: Retina 5K, 27-inch, Late 2015
  -- OS: macOS BigSur 11.6
  -- Processor: 3.2 GHz Quad-Core Intel Core i5
  -- Memory: 32 GB 1867 MHz DDR3
  -- Graphics: AMD Radeon R9 M390 2 GB
• Anaconda: 4.10.3
• Python 3.8.11
• Jupyter Notebook Server: 4.10.3

I run deep learning codes.
I installed pydeep according to the instructions:

# Download PyDeep from GitHub/MelJan
unzip PyDeep-master.zip
cd PyDeep-master
python setup.py install

but it will not run the unit tests:

python -m unittest discover -s testunits

It rather produced the following output:

Traceback (most recent call last):
  File "/Applications/anaconda3/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
...
  File "/Applications/anaconda3/lib/python3.8/unittest/loader.py", line 346, in discover
    raise ImportError('Start directory is not importable: %r' % start_dir)
ImportError: Start directory is not importable: 'testunits'

Before attempting the direct code install I had attempted to load PyDeep into my Anaconda environment using

conda install pydeep

but it is not recognized by conda, so I used pip

pip install pydeep

but. this created serious compile-time conflicts with pyobjc that took a night of debugging and a complete reinstall of anaconda to solve.
This may have been due to the recent Big Sur upgrade of October 11, 2021 by Apple.
I eventually solved this issue using a beta of pyobjc as follows:

pip install pyobjc-core==8.0b1
pip install pyobjc-framework-Cocoa==8.0b1
pip install pyobjc-framework-Quartz==8.0b1

Anyway I am presenting the obstacles I encountered in reverse order in the hope that your 2017 package can remain viable for its quality construction and clarity.
In the meantime I will use sklearn PCA, which is much less elegant than your streamlined implementation.

The paper mentions
GRBM-2-4, GRBM-2-4, and MoG-2-3 model, and report of results for MoG-9 in table 4,
as if there was a separate MoG model trained with eqn 24.

is there implementation code for that? I don't find it anywhere.

There is an overall question how the derivations on PoE and MoG come into the implementation at all. where are these concepts used? it seems we are just doing normal CD-k after all. Is the main lesson learned simply parameter initialization and gradient restriction?

Hi!

Could you please make the code for https://arxiv.org/pdf/1905.12937.pdf available?