igitugraz / spore-nest-module

Synaptic Plasticity with Online Reinforcement learning

NEST module SPORE:
Synaptic Plasticity with Online Reinforcement learning

SPORE is a custom-module for NEST that provides a simulation framework for reward-based learning with spiking neurons. SPORE utilizes MUSIC for communication with other software components.

• A detailed installation guide for SPORE and all its dependencies can be found on the SPORE wiki page.

• A detailed documentation of the module can be found in the SPORE class reference.

• We also provide a first example that demonstrates a typical use case of SPORE.

Quick Installation Guide

SPORE requires NEST installed with MUSIC (-Dwith-music=ON) and Python bindings. Please make sure your version of NEST corresponds to or is newer than version 2.14. A detailed installation guide that also shows how to set up the dependencies is provided below and on the SPORE wiki page.

git clone https://github.com/IGITUGraz/spore-nest-module
cd spore-nest-module
mkdir build && cd build
cmake ..
make
make install
make test

Detailed Installation Guide

This guide describes how to install the SPORE module in version 2.14 for the NEST simulator in version 2.14. Combinining this version of SPORE with other versions of NEST may work but has not been tested. We used MUSIC version 1.1.15 in revision 8e0a609b298 and MPI (Open MPI 1.6.5). Furthermore we need python with the packages pyzmq, numpy, ujson and matplotlib. A guide to install these dependencies is provided below. Finally we used SPORE (version 2.14, or later). The installation procedure was tested on Debian GNU/Linux 8.7 (jessie).

This guide assumes that you want to install everything into your local home folder $HOME/opt/. It is further assumed that you checkout the software into a local folder that is used for development, which we refer to as devel folder.

Preparation

Add the following lines to your ~/.bashrc (or ~/.zshrc or ~/.profile etc.)

export TARGET_DIR=$HOME/opt/
export PATH=$PATH:$TARGET_DIR/bin
export LIBRARY_PATH=$LIBRARY_PATH:$TARGET_DIR/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$TARGET_DIR/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$TARGET_DIR/lib/nest  # required in some cases
export PYTHONPATH=$PYTHONPATH:$TARGET_DIR/lib/python2.7/site-packages
export PYTHONPATH=$PYTHONPATH:$TARGET_DIR/lib64/python2.7/site-packages
export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so  # required in some cases (path only works for debian jessie)
export NUM_CORES=3  # number of CPU cores

Now run source ~/.bashrc or close the current terminal now and start a new session such that the changes get applied.

Now install all dependencies (for Debian Jessie).

Notes:

• Use pip install --user <package name> to install the python packages locally.
• Use sudo pip install <package name> to install the python packages globally.
• The python packages can also be installed in a virtualenv.
• For python3, use pip3 to install python packages and install the python3 versions of the packages for NEST dependencies.
• The use of multiple different python3 versions is discouraged. Make sure to consistently use one version for installing libraries and actually running experiments. When installing MUSIC, the specification of the precise executable (e.g. python3.4), rather than python3, may be necessary in ambigous cases (see #9).

# For NEST and MUSIC
apt-get install build-essential cmake
apt-get install python python-dev python-pip
apt-get install libreadline-dev gsl-bin libgsl0-dev libncurses5-dev openmpi-bin

apt-get install automake libtool # BUILD Dependencies
apt-get install libopenmpi-dev libopenmpi1.6 # RUN Dependencies
apt-get install freeglut3-dev # Optional, for viewevents
pip install cython
pip install mpi4py # RUN Dependencies

# For SPORE
apt-get install libzmq3  # For realtime plotting
pip install pyzmq # For real-time plotting
pip install numpy  # For testing
pip install ujson
pip install --upgrade matplotlib  # (Optional, If you want to have nice plotting, you should upgrade `matplotlib` to the newest version)

Installing MUSIC

In your devel folder, check out the latest version of MUSIC from https://github.com/INCF/MUSIC

git clone https://github.com/INCF/MUSIC.git

Note that at the moment --disable-isend is required because of a critical problem in the MUSIC scheduler.

In the folder ./MUSIC/ :

./autogen.sh
PYTHON=/usr/bin/python ./configure --prefix=$TARGET_DIR --disable-isend  # Replace with python3[.x] if desired
make -j$NUM_CORES
make install

Installing NEST

In your devel folder, check out the latest version of NEST from https://github.com/nest/nest-simulator

# use the latest developmental version of NEST. Release versions don't currently work with SPORE
git clone https://github.com/nest/nest-simulator.git

Then in the folder ./nest-simulator :

mkdir build
cd build/
cmake -DCMAKE_INSTALL_PREFIX:PATH=$TARGET_DIR -Dwith-music=ON -Dwith-mpi=ON -Dwith-python=2 ..  # Change python version to 3 for Python 3
make -j$NUM_CORES
make install

Installing SPORE

In your devel folder, check out the latest version of SPORE from https://github.com/IGITUGraz/spore-nest-module

git clone https://github.com/IGITUGraz/spore-nest-module.git

Then in the folder ./spore-nest-module :

mkdir build
cd build/
cmake -Dwith-python=2 ..  # Change python version to 3 for Python 3, or provide a path to a python binary
make -j$NUM_CORES
make install
make test

In ipython (or ipython3) running import nest and then nest.Install("sporemodule") should now yield the following:

In [1]: import nest
[INFO] [.../rng_manager.cpp:238 @ Network::create_rngs_] : Creating default RNGs
[INFO] [.../rng_manager.cpp:233 @ Network::create_rngs_] : Deleting existing random number generators
[INFO] [.../rng_manager.cpp:238 @ Network::create_rngs_] : Creating default RNGs
[INFO] [.../rng_manager.cpp:284 @ Network::create_grng_] : Creating new default global RNG

              -- N E S T --

  Copyright (C) 2004 The NEST Initiative
  Version 2.14.0 Oct 30 2017 13:06:41

This program is provided AS IS and comes with
NO WARRANTY. See the file LICENSE for details.

Problems or suggestions?
  Visit http://www.nest-simulator.org

Type 'nest.help()' to find out more about NEST.

In [2]: nest.Install("sporemodule")

Oct 31 13:26:30 Install [Info]: 
    loaded module SPORE (version 2.14.0)

Running a first experiment

That should be it.

Now you should be able to execute the script experiment.py in

spore-nest-module/examples/pattern_matching_showcase.

Cite SPORE

Please cite the current version of SPORE as

David Kappel, Michael Hoff, & Anand Subramoney. (2017, November 7).
IGITUGraz/spore-nest-module: SPORE version 2.14.0 (Version v2.14.0).
Zenodo. http://doi.org/10.5281/zenodo.1043486

License

SPORE is open source software and is licensed under the GNU General Public License v2 or later.

spore-nest-module's Issues

Make logos more portable

Our custom NEST logos use a specific font (Courier New) which is not necessarily installed on every system. We should convert those strings to plain SVG shapes to provide better portability.

Create a requirements.txt for python dependencies

Make SPORE compatible with NEST 2.16

This is just a reminder that a new version of NEST is about to come up, see:

https://github.com/nest/nest-simulator/milestone/5

A new release of SPORE (2.16) should follow timely after the new NEST release.

shebang in test_reward_in_proxy

The reward_in_proxy for python2 failed for me in my virtualenv, as the shebang of the py2 files in unittests/test_reward_in_proxy link to the system python install.

spore-nest-module fails to compile

With error:

/.../spore-nest-module/src/diligent_connector_model.h:284:5: error: no matching function for call to ‘nest::ModelManager::register_connection_model(const string&, bool&)
/.../spore-nest-module/src/diligent_connector_model.h:284:5: error: wrong number of template arguments (2, should be 1)

Built with cmake ..

Building off master branch.

Fix python style issues raised by flake8

In the process, update ignored rules if necessary.

Test and merge python3 branch with master

Tracking/reminder to do the merge for myself.

Take default Python version from NEST installation

I would suggest to change the default behavior of the installation script to take the Python version from the nest installation. nest-config recently introduced the feature to access the python version and path, so it should be quite easy to adapt the cmake script. Since in all practical cases the python version of SPORE and NEST should be the same this is more convenient for most users. We can still allow to overwrite the default using `-Dwith-python'.

see:

nest-config  --python-executable # print full path to Python interpreter used
nest-config  --python-version     # print Python version string for interpreter

NEST 5g kernel breaks SPORE

The NEST 5g kernel that will be introduced in one of the next release version is not compatible with SPORE.

nest/nest-simulator#920

The connection model has been significantly re-factored. These changes need to be included in SPORE. Current version does not compile with the 5g branch.

Make SPORE citable

As of this tutorial, GitHub repositories can be turned into DOI references using zenodo.

For this, I think the repository owner (IGITUGraz) has to allow zenodo to access its data.

@anandtrex Could you give that a look? If so, I would also like to assist you in this.

Building help information of external module fails "unicode expected"

When building SPORE for NEST 2.14 on a Ubuntu machine (artful) with python 3.6.3, I get the following error when generating the help information of the module:

Traceback (most recent call last):
  File "generate_helpindex.py", line 39, in <module>
    write_helpindex(help_dir)
  File "/home/david/opt/share/nest/help_generator/writers.py", line 244, in write_helpindex
    f_helphlpindex.write('\u\n'.join(hlp_list))
TypeError: write() argument 1 must be unicode, not str
src/CMakeFiles/generate_help.dir/build.make:57: recipe for target 'generate_help' failed
make[2]: *** [generate_help] Error 1
CMakeFiles/Makefile2:196: recipe for target 'src/CMakeFiles/generate_help.dir/all' failed
make[1]: *** [src/CMakeFiles/generate_help.dir/all] Error 2
Makefile:162: recipe for target 'all' failed
make: *** [all] Error 2

Building the latest release of NEST worked without errors, this only occurs in our custom module.

There is a simple fix by enforcing unicode strings by adding an 'u' in
line 244 of writers.py:

--- a/extras/help_generator/writers.py
+++ b/extras/help_generator/writers.py
@@ -241,7 +241,7 @@ def write_helpindex(helpdir):
     # Todo: using string template for .hlp
     f_helphlpindex = io.open(os.path.join(helpdir, 'helpindex.hlp'), mode='w',
                              encoding='utf-8')
-    f_helphlpindex.write('\n'.join(hlp_list))
+    f_helphlpindex.write(u'\n'.join(hlp_list))
     f_helphlpindex.close()

With this modification the custom module compiles and runs without
problems.

So, my take on this is that the behavior of write has recently changed, making it more picky on the data encoding, which should be accounted for on the NEST side.

Install instructions

The wiki says to install:

apt-get install libzmp3 # For realtime plotting

But this should actually be "libzmq3"

Assertion fails in CopyModel when using multi-threading

import nest
nest.SetKernelStatus({"local_num_threads": 10})
nest.Install("sporemodule")
nest.CopyModel("synaptic_sampling_rewardgradient_synapse", "reward_synapse")

fails with

python: /home/hoff/.sources/nest-simulator/sli/name.cc:89: unsigned int Name::insert(const string&): Assertion `not omp_in_parallel()' failed.
[node16:04418] *** Process received signal ***
[node16:04418] Signal: Aborted (6)
[node16:04418] Signal code:  (-6)
python: /home/hoff/.sources/nest-simulator/sli/name.cc:89: unsigned int Name::insert(const string&): Assertion `not omp_in_parallel()' failed.
python: /home/hoff/.sources/nest-simulator/sli/name.cc:89: unsigned int Name::insert(const string&): Assertion `not omp_in_parallel()' failed.
python: /home/hoff/.sources/nest-simulator/sli/name.cc:89: unsigned int Name::insert(const string&): Assertion `not omp_in_parallel()' failed.

This appears to be a consequence of nest/nest-simulator#739 in combination with multi-threading (local_num_threads > 1).

---

NEST: nest/nest-simulator@d66456c
SPORE: b6e03f4

Time constant of mean reaward is incorrect

The time constant to compute the mean reward in the pattern matching example seems wrong. The update rate is computed here wrongly with hardcoded 1 ms instead of using the update step of the control node as defined in config.py

https://github.com/IGITUGraz/spore-nest-module/blob/master/examples/pattern_matching_showcase/python/experiment_utils/reward.py#L16

So the effective time constant deviates from the value in seconds (is 750 seconds instead of 50 seconds).

The line should read

self._gamma = np.exp(-music_node_timestep / self._normalization_window)

instead.

@mhoff can you confirm that?

The same problem might also be there in other examples.

set_status does not handle gradient_scale anymore

With e8d7c6c, using set_status on the synapse can not set the parameter gradient_scale anymore. The showcase fails because of this.

SPORE v2.10.0 License/Readme

• LICENSE headers are missing in all source files.
• The outdated README should probably be deleted in favour of the README in the master branch.

Problems when installing for python3

We are facing problems while setting up spore with python3. We follow the guidelines
from Pattern Matching Experiment readme with following changes:

export PYTHONPATH=$PYTHONPATH:$TARGET_DIR/lib/python3.4/site-packages
export PYTHONPATH=$PYTHONPATH:$TARGET_DIR/lib64/python3.4/site-packages

We use pip3 instead of pip like instructed.

For MUSIC we change the following line:

PYTHON=/usr/bin/python3 ./configure --prefix=$TARGET_DIR --disable-isend

For NEST we change:
cmake -DCMAKE_INSTALL_PREFIX:PATH=$TARGET_DIR -Dwith-music=ON -Dwith-mpi=ON -Dwith-python=3 ..

For SPORE:
cmake -Dwith-python=3 ..

We tested with both master and py3 branches of the spore repository. Also with NEST revision 58fd190f5e4 and latest master. When performing make test we fail two tests:

9/10 Test  #9: test_reward_in_proxy .............***Failed    0.23 sec
      Start 10: test_music_integration
10/10 Test #10: test_music_integration ...........***Failed    0.31 sec

We encounter no problems for nest.Install("sporemodule") but again for pattern mattching experiment script it fails with:

Traceback (most recent call last):
  File "python/zmq_proxy_node.py", line 6, in <module>
    import snn_utils.comm.music
  File "/home/sinisa/Uni/Project_CI/spore-nest-module/examples/pattern_matching_showcase/python/snn_utils/comm/music/__init__.py", line 4, in <module>
    import music
  File "/home/sinisa/opt/lib/python3.4/site-packages/music/__init__.py", line 25, in <module>
    from .pymusic import *
  File "pybuffer.pxd", line 49, in init pymusic (pymusic.cpp:11666)
ImportError: /home/sinisa/opt/lib/python3.4/site-packages/music/pybuffer.so: undefined symbol: PyString_Type
Traceback (most recent call last):
  File "python/control_node.py", line 6, in <module>
    import music
  File "/home/sinisa/opt/lib/python3.4/site-packages/music/__init__.py", line 25, in <module>
    from .pymusic import *
  File "pybuffer.pxd", line 49, in init pymusic (pymusic.cpp:11666)
ImportError: /home/sinisa/opt/lib/python3.4/site-packages/music/pybuffer.so: undefined symbol: PyString_Type
--------------------------------------------------------------------------
mpirun has exited due to process rank 1 with PID 30396 on
node sinisa-Lenovo-V110-15ISK exiting improperly. There are two reasons this could occur:

1. this process did not call "init" before exiting, but others in
the job did. This can cause a job to hang indefinitely while it waits
for all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.

2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"

This may have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
--------------------------------------------------------------------------

Create 2.14.0 release

Create the SPORE release corresponding to the NEST 2.14.0 release.

As a precondition, all other issues in this milestone are to be solved before.

SPORE Show Cases

This thread is not an actual issue but a collection of show cases SPORE was used for so far. Follow up to this thread to add a demo of your own project. I make a start with our little pattern matching experiment, which you also find here:

Another version with higher resolution and longer recording:

Release Version 2.12.0

After issue #8 is closed and #710 of NEST has been merged into their master, it would be a good time to create an official release version of SPORE. This thread is here as a reminder and for collecting minor changes that should be made before creating the release version.

Installation instructions location

It might be a good idea to keep all documentation (including installation instructions) part of the code so that it can be versioned with the code. The website (doxygen) could display these instructions perhaps?

Verify functionality with NEST release 2.14.0