incf / music Goto Github PK
View Code? Open in Web Editor NEWMUSIC, the MUltiSimulation Coordinator
License: GNU General Public License v3.0
MUSIC, the MUltiSimulation Coordinator
License: GNU General Public License v3.0
* What is MUSIC? MUSIC is an API allowing large scale neuron simulators using MPI internally to exchange data during runtime. MUSIC provides mechanisms to transfer massive amounts of event information and continuous values from one parallel application to another. Special care has been taken to ensure that existing simulators can be adapted to MUSIC. In particular, MUSIC handles data transfer between applications that use different time steps and different data allocation strategies. This is the MUSIC pilot implementation. The two most important components built from this software distribution is the music library `libmusic.a' and the music utility `music'. A MUSIC-aware simulator links against the C++ library and can be launched using mpirun together with the music utility as described below. MUSIC can also be used from a C program using the API in music-c.h. MUSIC is distributed under the GNU General Public License v3. * Required external packages MUSIC is written in C++ and makes use of the MPI message passing library. MUSIC has been tested with versions 1.2 and 1.3 of OpenMPI, mpich and on BG/L. A visualization tool `viewevents' can be built if the GLUT toolkit is installed. MUSIC version 1.0 makes use of the "long long" data type supported by, for example, the GNU C++ compiler g++. * Getting started MUSIC can be built and installed using the generic installation instructions found in the file INSTALL. At the top of the source distribution directory, do: ./autogen.sh ./configure --prefix=<PREFIX> make make install (On some UNIX distributions you might need to run `ldconfig' or define LD_LIBRARY_PATH: export LD_LIBRARY_PATH=/usr/local/lib for the music utility to find its library.) A MUSIC co-simulation is specified in a configuration file given as the first argument to the music utility. The test subdirectory contains a collection of examples, for example: cd test mpirun -np 4 /usr/local/bin/music demo.music On some systems, this way of invoking a MUSIC co-simulation isn't supported. Se blow for an alternative method. * Compilation and linking flags configure.ac contains mechanisms which try to identify which MPI implementation/environment is in use and, based on this, decide automagically which compiler, compilation flags and linking flags to use. To circumvent, pass variable definitions ro configure, e.g.: ./configure MPI_CXXFLAGS="-g -O3" The variables are: MPI_CXX The compiler to use for MPI code MPI_CXXFLAGS C++ compilation flags MPI_CFLAGS C compilation flags MPI_LDFLAGS Linking flags * Python support MUSIC comes with Python bindings. By default, the configure script looks for a Python interpreter in the path. If you want to use a particular Python interpreter, you can specify the path to it using the option --with-python: ./configure --with-python=/my/bin/python Python support can be disabled using: ./configure --without-python * Porting When launching a set of applications in a co-simulation, MUSIC needs to go outside the MPI standard in two respects: 1. It needs to know the MPI process rank of the running process before calling MPI::Init. 2. It needs to extract the first argument given to the music utility before calling MPI::Init. The configuration script tries to figure out which MPI distribution is installed and how to compiler MUSIC for that distribution. The subdirectory mpidep contains all non-standard code. If you are trying to port the MUSIC library to a new version of MPI, please supply new definitions for the functions getRank and getConfig in mpidep/mpidep.c. For more details, see the file PORTING in this directory. * Alternative way to invoke MUSIC co-simulations In case the method of launching MUSIC described under "Getting started" above fails, there is an alternative method: Let's assume that we want to launch application A with binary a and Na processes, and application B with binary b and Nb processes. These are listed in a standard MUSIC config file ab.music with block labels A and B. The standard way to launch these would be: mpirun -np <Na+Nb> music ab.music (where <Na+Nb> is a number). These can alternatively be launched in MPMD style as: mpirun -np Na ./a --music-config ab.music --app-label A : -np Nb ./b --music-config ab.music --app-label B * Where to find more information The MUSIC manual and other information can be found on the MUSIC wiki: https://github.com/INCF/MUSIC/wiki Questions and requests can be sent to [email protected]. * Submitting bug reports Bug reports should be submitted as issues on the MUSIC github page: https://github.com/INCF/MUSIC/issues
Im trying to follow the procedure described by Weidel, Djurfeldt, Duarte and Morrison in "Closed Loop Interactions betweenSpiking Neural Network and RoboticSimulators Based on MUSIC and ROS"
I have NEST, ROS and MUSIC installed and openmpi and other dependencies.
When running my MUSIC configuration file i recieve errors, and I have tried different both relative and absolute paths to the "ros_sensor_adapter" that is installed in the /opt/MUSIC-master/ros/adapters
Error during launching of binary ros_sensor_adapter
MUSIC: No such file or directory
Error during launching of binary connect_adapter
MUSIC: No such file or directory
Error during launching of binary rate_encoder
MUSIC: No such file or directory
`
The fable.music contains the MUSIC configuration code:
stoptime=1000.
music_timestep=0.05
rtf=1
[sensor]
binary=ros_sensor_adapter
args=
np=1
ros_topic=/fableassem/joint_states
message_type=Float64MultiArray
sensor_update_rate=20
[converge]
binary=connect_adapter
args=
np=1
weights_filename=braitenberg3_converge_weights.dat
[encoder]
binary=rate_encoder
args=
np=1
rate_min=1
rate_max=200
[nest]
binary=./pynesttest.py
args=-s 0.05 -t 1000 -n 2
np=1
[decoder]
binary=linear_readout_decoder
args=
np=1
tau=0.03
weights_filename=braitenberg3_readout_weights.dat
[command]
binary=ros_command_adapter
args=
np=1
ros_topic=/fableassem/joint1_torque_controller
message_mapping_filename=braitenberg3_twist_mapping.dat
command_rate=10
sensor.out->converge.in[20]
converge.out->encoder.in[2]
encoder.out->nest.in[2]
nest.out->decoder.in[2]
decoder.out->command.in[2]
For ease of use, and (possibly) weak guarantees for synchronization between processes, it would be great if MUSIC could also start executables which do not use MUSIC internally. This would allow for example starting of ROS directly from a MUSIC config file instead of having to launch it manually in another process. Currently MUSIC will only launch the executable without MUSIC support, while the other processes are not started. This is probably caused by some barrier not hit.
@mdjurfeldt please advise on how this could potentially be addressed
@weidel-p please comment whether you'd find this useful
Headers should be placed beside their compilation unit counterpart to facilitate development work.
There are some syntax issues with "../testsuite/sanitytests/clocksource.cc" file
../../../testsuite/sanitytests/clocksource.cc: In function ‘int main(int, char*)’:
../../../testsuite/sanitytests/clocksource.cc:107:3: error: ‘Setup’ is not a member of ‘MUSIC’
../../../testsuite/sanitytests/clocksource.cc:107:17: error: ‘setup’ was not declared in this scope
../../../testsuite/sanitytests/clocksource.cc:107:29: error: expected type-specifier
../../../testsuite/sanitytests/clocksource.cc:107:29: error: expected ‘;’
../../../testsuite/sanitytests/clocksource.cc:114:3: error: ‘ContOutputPort’ is not a member of ‘MUSIC’
../../../testsuite/sanitytests/clocksource.cc:114:26: error: ‘out’ was not declared in this scope
../../../testsuite/sanitytests/clocksource.cc:123:3: error: ‘ArrayData’ is not a member of ‘MUSIC’
../../../testsuite/sanitytests/clocksource.cc:123:20: error: expected ‘;’ before ‘dmap’
../../../testsuite/sanitytests/clocksource.cc:129:16: error: ‘dmap’ was not declared in this scope
../../../testsuite/sanitytests/clocksource.cc:131:16: error: ‘dmap’ was not declared in this scope
../../../testsuite/sanitytests/clocksource.cc:139:3: error: ‘Runtime’ is not a member of ‘MUSIC’
../../../testsuite/sanitytests/clocksource.cc:139:19: error: ‘runtime’ was not declared in this scope
../../../testsuite/sanitytests/clocksource.cc:139:33: error: expected type-specifier
../../../testsuite/sanitytests/clocksource.cc:139:33: error: expected ‘;’
../../../testsuite/sanitytests/clocksource.cc:150:10: error: type ‘’ argument given to ‘delete’, expected pointer
delete runtime;
^
make[3]: ** [clocksource-clocksource.o] Error 1
@michael: I had already reported this issue last year. Could you please have a look at it?
Hello,
Is pymusic not meant to be used without MPI at all? I see that configure.ac file has a conditional, and then the pymusic Makefile tries to run python3 tests.py
which imports mpi4py
?
Cheers!
Hi,
I just tried to compile Monsteer (which uses MUSIC) against the most recent MUSIC version but the compiler complains as follows:
error: ‘class SpikesHandler’ has virtual functions and accessible non-virtual destructor [-Werror=non-virtual-dtor]
In src/music/event.hh it seems that a virtual destructor is missing (I just learned why its good to have one: http://stackoverflow.com/questions/5827719/class-has-virtual-functions-and-accessible-non-virtual-destructor)
Compiler: gcc 4.8.3
This is a reminder to further investigate a problem discovered in issue #35.
With a linear chain of applications, there seems to be a synchronization problem when MUSIC is compiled with OpenMPI 1.6.5. With the standard configuration, p2p communication does not properly synchronize with the receiver such that expected semantics for maxBuffered=1 is broken.
A workaround is to configure MUSIC with --disable-isend.
If in the config file, one section is missing the np
enttry, in ApplicationMapper::mapApplications ()
it is assigned a random(?) value. The resulting error reads something like the following:
Error in MUSIC library: configuration file specifies 32579 MPI processes but MUSIC was given 3
Instead, MUSIC should raise a proper error message complaining that np
is missing.
It seems like there should in general be a function right after (cfile_->load (*configFile))
in ApplicationMapper::map
that checks the validity of the config file. That way one could also check for all other errors arising from wrong specifications in the config file. Is there something like that already and I'm missing it?
When compiling with g++ 6, compilation fails with
../../music/src/connectivity.cc: In member function 'MUSIC::ConnectivityInfo* MUSIC::Connectivity::info(std::__cxx11::string)':
../../music/src/connectivity.cc:110:14: error: invalid conversion from 'int' to 'MUSIC::ConnectivityInfo*' [-fpermissive]
return NO_CONNECTIVITY;
This seems to be due to stricter typing rules in C++11. As a work-around, one can compile this file with -std=c++98
, but I suppose this should be fixed properly.
PyMusic supports auto detection of primitive datatypes of the buffer passed by the user. There are use cases where complex data types could also be desired as for example in NEST with the music_cont_out_proxy when observing multiple quantities at once to save additional proxies and function calls.
@mdjurfeldt There are a few questions now: do you think its worth it to implement this support in the PyMusic interface? I put some thoughts in how to accomplish auto detection by the given buffer but I think the most straight forward solution would be to have a helper function that creates a proper numpy-dtype with an api like create_numpy_dtype( primitive_dtype, multiplicity) and possibly an alternative for a pure python array.
Hi all,
I recently found this issue. If nest is imported, music.Setup() complains that Setup constructor was already called.
#!/usr/bin/env python3
import music
import nest
import ctypes
ctypes.CDLL("libmpi.so", mode=ctypes.RTLD_GLOBAL)
setup = music.Setup()
running this simple code throws the error:
Error in MUSIC library: rank #0: Setup constructor was called a second time.
Only one Setup object can exist at any instance of time.
This is independent of the order of the two imports, and the issue disappears by commenting out import nest
.
As mentioned in nest/nest-simulator#958 MUSIC does not finish it's simulation if there is only one process defined in the MUSIC configuration file. Apparently, the function pushForward
in the scheduler runs into an endless loop in this case.
This is not a major error as there is no reason to run music with only one process, but I guess it would be nice to fix it anyways.
We should learn from NEST and let configure print out a summary of what main features have been found on the new system.
When I am trying to install MUSIC, with the command './configure', I encountered "-bash: ./configure: No such file or directory".
Consider the following MUSIC topology:
V, P, C, D are nodes from the ros_music_adapters, written in CPP; R is a python (pymusic) node; and N is a pynest node using SPORE.
The problem arises when the red edge (R, N) is inserted into the MUSIC configuration file. Running MUSIC does then result in all nodes functioning normally, except for D, which is getting stuck somewhere before entering the runtime.
This bug appears to be related to #37 (#35), because essentially the same error symptoms occur. However, --disable-isend
does not resolve the issue, nor is the MPI version 1.6.5.
Let me emphasize: slightly different topologies are functional in the sense, that no single node is getting stuck before runtime.
Debug information:
$ python3 --version
Python 3.5.2
$ music --version
MUSIC 1.1.15
# MUSIC build configuration:
$ PYTHON=/usr/bin/python3.5 ./configure --prefix=$HOME/.local --disable-isend
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for mpiCC... mpiCC
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether mpiCC accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of mpiCC... gcc3
checking which MPI system we think we are using... SYSGUESS=openmpi
checking MPI_CXXFLAGS... -I/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent -I/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/include -I/usr/lib/openmpi/include -I/usr/lib/openmpi/include/openmpi -pthread
checking MPI_CFLAGS... -I/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent -I/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/include -I/usr/lib/openmpi/include -I/usr/lib/openmpi/include/openmpi -pthread
checking MPI_LDFLAGS... -pthread -Wl,-rpath -Wl,/usr/lib/openmpi/lib -Wl,--enable-new-dtags -L/usr/lib/openmpi/lib -lmpi_cxx -lmpi
checking for python version... 3.5
checking for python platform... linux
checking for python script directory... ${prefix}/lib/python3.5/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python3.5/site-packages
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... yes
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for ANSI C header files... (cached) yes
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking for size_t... yes
checking for long long... yes
checking for strrchr... yes
checking for mallinfo... yes
checking GL/glut.h usability... yes
checking GL/glut.h presence... yes
checking for GL/glut.h... yes
checking for rts_get_personality... no
checking for ompi_comm_free... yes
checking for MPI::Init_thread method... yes
checking whether /usr/bin/python3 version is >= 2.6... yes
checking for /usr/bin/python3 version... (cached) 3.5
checking for /usr/bin/python3 platform... (cached) linux
checking for /usr/bin/python3 script directory... (cached) ${prefix}/lib/python3.5/site-packages
checking for /usr/bin/python3 extension module directory... (cached) ${exec_prefix}/lib/python3.5/site-packages
checking for "/usr/include/python3.5m/Python.h"... yes
checking whether to build PyMUSIC... yes
checking whether C compiler accepts -fno-strict-aliasing... yes
checking whether to build ROS Toolchain... no
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating pymusic/Makefile
config.status: creating pymusic/setup.py
config.status: creating Makefile
config.status: creating mpidep/Makefile
config.status: creating src/Makefile
config.status: creating src/music/music-config.hh
config.status: creating src/music/version.hh
config.status: creating rudeconfig/Makefile
config.status: creating utils/Makefile
config.status: creating examples/Makefile
config.status: creating testsuite/Makefile
config.status: creating testsuite/music_tests.sh
config.status: creating testsuite/unittests/catch/Makefile
config.status: creating testsuite/sanitytests/Makefile
config.status: creating music-config/Makefile
config.status: creating music-config/predict_rank.py
config.status: creating doc/Makefile
config.status: creating extras/Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
config.status: executing libtool commands
$ mpirun --version
mpirun (Open MPI) 1.10.2
$ lsb_release -d
Description: Ubuntu 16.04.3 LTS
$ cat run.music
music_timestep=0.003
rtf=1.0
stoptime=50000.0
sensor_update_rate=333.3333333333333
command_rate=333.3333333333333
[nest]
binary=python/ffn_node.py
np=1
[position]
ros_node_name=position_adapter
binary=ros_sensor_adapter
message_type=Vector3
ros_topic=/plugging_camera_offset
np=1
[dvs]
music_timestep=0.001
binary=dvs_adapter
ros_topic=/dvs/events
np=1
ros_node_name=dvs_adapter
message_type=EventArray
sensor_update_rate=1000.0
[reward]
binary=python/reward_node.py
np=1
[command_gen]
message_mapping_filename=res/map_float_cmd.dat
ros_topic=/motor/activity
np=1
binary=ros_command_adapter
[decoder]
np=1
binary=linear_readout_decoder
weights_filename=res/activity_to_velocity_translation_weights.dat
tau=0.1
dvs.out -> nest.visual [16384]
nest.motor -> decoder.in [8]
decoder.out -> command_gen.in [2]
position.out -> reward.in [3]
reward.out -> nest.reward [1]
The following link in the README: http://software.incf.org/software/music/home appears to be broken.
aclocal.sh and autogen.sh should not try to find older versions of automake
(The old behavior was intended to try to avoid compatibility issues with newer versions of automake, but now it instead tends to create more problems than it solves.)
Hi there,
I'm using MUSIC to run a multi-simulation with NEST. I'm currently working on ubuntu 14.04 where I was able to install the library without any particular issues. I'd like to reproduce the same environment also on a mac laptop in order to keep working without my lab's desktop. I've been able to install pretty much everything but MUSIC. I got this error:
ld: unknown option: -z
collect2: error: ld returned 1 exit status
make[2]: *** [libmusic.la] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
Since the default compiler clang coming with the os does not support openmp I wrapped the openmpi installation with gcc4.8 compiler installed with homebrew and I'm using it to compile MUSIC. It should use the same GNU ld of ubuntu so I don't understand why I keep getting this error.
Is there a solution for this problem?
Thanks
Hello
while trying to install MUSIC in a virtual env using
I noted that PYTHON value given to configure is not honored.
see
./configure MPI_CXXFLAGS="-g -O3" MPI_CFLAGS="-I$MPI_ROOT/include -pthread" MPI_LDFLAGS="-pthread -Wl,--enable-new-dtags -L$MPI_ROOT/lib -lmpi_cxx -lmpi" PYTHON=/opt/gensoft/exe/MUSIC/1.1.16/venv/bin/python
and
[gensoft@38a8dee6f6d5 MUSIC-1.1.16]$ grep '^PYTHON =' Makefile
PYTHON = /opt/gensoft/adm/bin/python
I was expecting PYTHON =/opt/gensoft/exe/MUSIC/1.1.16/venv/bin/python
this makes cd $(PYTHON} tests.py
to fail in pymusic/Makefile, more precisely in pyconfig.pxi
rule
regards
Eric
I built MUSIC from master with changes from #26 merged (see also #28) and also the changes in #24 merged. I then hit the following problem:
/bin/sh ../libtool --tag=CXX --mode=link g++-6 -I../../music/src -I../src -I../../music -I/usr/local/Cellar/open-mpi/2.0.1/include -g -O3 -o music music-music.o ../src/libmusic.la ../mpidep/libmpidep.la -L/usr/local/Cellar/open-mpi/2.0.1/lib/ -lmpi
libtool: link: g++-6 -I../../music/src -I../src -I../../music -I/usr/local/Cellar/open-mpi/2.0.1/include -g -O3 -o .libs/music music-music.o -Wl,-bind_at_load ../src/.libs/libmusic.dylib -L/usr/local/Cellar/open-mpi/2.0.1/lib/ ../mpidep/.libs/libmpidep.a -lmpi
Undefined symbols for architecture x86_64:
"MPI::Win::Free()", referenced from:
vtable for MPI::Win in music-music.o
"MPI::Comm::Comm()", referenced from:
MPI::Intracomm::Clone() const in music-music.o
MPI::Graphcomm::Clone() const in music-music.o
MPI::Cartcomm::Sub(bool const*) const in music-music.o
MPI::Intracomm::Create_graph(int, int const*, int const*, bool) const in music-music.o
MPI::Cartcomm::Clone() const in music-music.o
MPI::Intracomm::Create_cart(int, int const*, bool const*, bool) const in music-music.o
MPI::Intercomm::Merge(bool) const in music-music.o
...
"MPI::Datatype::Free()", referenced from:
vtable for MPI::Datatype in music-music.o
"_ompi_mpi_cxx_op_intercept", referenced from:
MPI::Op::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool) in music-music.o
ld: symbol(s) not found for architecture x86_64
Does anyone have an idea what might be going wrong? I am linking against OpenMPI 2.0.1 from Homebrew under OSX 10.11.6 with gcc 6.2.1, also from Homebrew.
It might be desirable, within a Python script, to mix ports handled at the C++ level in a simulator with ports created through the PyMUSIC API.
One way to achieve this is to allow the MUSIC::Setup object to be created multiple times. There could be one instance in the simulator and one at the Python level. Internally, they would refer to the same datastructure(s).
When running a music script in which x executables are defined with x + 1 MPI processes, the following error message results:
music: ../../src/application_mapper.cc:187: void MUSIC::ApplicationMapper::mapApplications(): Assertion `config_->getDict().size() != 0' failed.
This should be replaced with a more informative error message, as it is the case when not enough MPI processes are passed.
Error in MUSIC library: configuration file specifies 3 MPI processes but MUSIC was given 2
I this a genuine bug or just me misinterpeting the output?
I haven't looked at the code yet, so I can't promise a PR.
Hi,
I got this error after typing 'make install':
Making install in mpidep
make[1]: Entering directory '/home/le/Installation/MUSIC/mpidep'
make[2]: Entering directory '/home/le/Installation/MUSIC/mpidep'
make[2]: Nothing to be done for 'install-exec-am'.
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/home/le/Installation/MUSIC/mpidep'
make[1]: Leaving directory '/home/le/Installation/MUSIC/mpidep'
Making install in rudeconfig
make[1]: Entering directory '/home/le/Installation/MUSIC/rudeconfig'
make[2]: Entering directory '/home/le/Installation/MUSIC/rudeconfig'
make[2]: Nothing to be done for 'install-exec-am'.
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/home/le/Installation/MUSIC/rudeconfig'
make[1]: Leaving directory '/home/le/Installation/MUSIC/rudeconfig'
Making install in src
make[1]: Entering directory '/home/le/Installation/MUSIC/src'
make[2]: Entering directory '/home/le/Installation/MUSIC/src'
/bin/mkdir -p '/usr/local/lib'
/bin/bash ../libtool --mode=install /usr/bin/install -c libmusic.la libmusic-c.la '/usr/local/lib'
libtool: install: /usr/bin/install -c .libs/libmusic.so.1.0.0 /usr/local/lib/libmusic.so.1.0.0
/usr/bin/install: cannot remove '/usr/local/lib/libmusic.so.1.0.0': Permission denied
Makefile:533: recipe for target 'install-libLTLIBRARIES' failed
make[2]: *** [install-libLTLIBRARIES] Error 1
make[2]: Leaving directory '/home/le/Installation/MUSIC/src'
Makefile:1090: recipe for target 'install-am' failed
make[1]: *** [install-am] Error 2
make[1]: Leaving directory '/home/le/Installation/MUSIC/src'
Makefile:440: recipe for target 'install-recursive' failed
make: *** [install-recursive] Error 1
Then I used 'sudo make install', seems it was installed but when I run test files, it fails:
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:
Process name: [[4272,1],0]
Exit code: 127
--------------------------------------------------------------------------_
I have no idea how to solve this:(
best,
luke
If an application has not config entry, as in the following:
stoptime=1.
rtf=1.
[sender]
binary=./sender.py
np=1
sender.out->receiver.in
music crashes with a segmentation fault:
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 0 on node jordan exited on signal 11 (Segmentation fault).
It would be more helpful for the user to detect the broken config file and raise an exception in which the cause for the error is explained.
I am trying to execute the simple MUSIC demo example from the nest tutorial, but I get the following error:
$ mpirun -np 4 music python.music
MUSIC: Unable to determine process rank.
MUSIC: Did you launch music using mpirun?
MUSIC: If so, check the comments about porting in README.
I get a little more information by trying to run the example without using the music binary:
$ mpirun -np 4 ./send.py --music-confing python.music --app-label from -np 4 ./receive.py --music-config python.music --app-label to
[...]
Fatal error in PMPI_Intercomm_create: Invalid rank, error stack:
PMPI_Intercomm_create(330): MPI_Intercomm_create(comm=0x84000004, local_leader=0, MPI_COMM_WORLD, remote_leader=-1, tag=0, newintercomm=0x7ffcb7760938) failed
PMPI_Intercomm_create(288): Error specifying remote_leader; rank given was -1 but must be in the range 0 to 3
The README says this is likely due to an unexpected version of MPI, but I am not sure how to solve it. I am running mpich 3.3~a2-4, and nest-2.18.0 (installed with music and mpi) on Ubuntu 16.04. Thanks in advance.
I've compiled the recent music version and tried to import it with Ipython, the first error Ive got is that when it enteres the init.py of the music package in the build path, it cannot find pymusic as it is imported 'absolute'. So I just modified the line to from .pymusic import *
After that another issue arises where I am not sure if I am using the package in a wrong manner. Ive tried it also to use with mpiexec ipython on a cluster (openmpi 1.8.7, mpi4py 2.0.0). The python interpreter is from a virtual environment for python3.4 (freshly setup with Cython and mpi4py), MUSIC ENV's are set as follows:
export MUSIC_ROOT_DIR=/gpfs/bbp.cscs.ch/home/bbpnrsoa/martin.setup/builds/MUSIC_1_1_15
export MUSIC_ROOT=${MUSIC_ROOT_DIR}
MUSIC_PATH=${MUSIC_ROOT_DIR}
export LD_LIBRARY_PATH=${MUSIC_PATH}/lib:$LD_LIBRARY_PATH
export PATH=${MUSIC_PATH}/bin:$PATH
export CPATH=${MUSIC_PATH}/include:$CPATH
export PYTHONPATH=${MUSIC_PATH}/lib/python3.4/site-packages:$PYTHONPATH
`
iPython 3.4.3 (default, Jan 18 2016, 16:48:08)
Type "copyright", "credits" or "license" for more information.
IPython 4.1.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
ValueError Traceback (most recent call last)
in ()
----> 1 import music
/gpfs/bbp.cscs.ch/home/bbpnrsoa/martin.setup/builds/MUSIC_1_1_15/lib/python3.4/site-packages/music/init.py in ()
23 else:
24 setrc()
---> 25 from .pymusic import *
26
27 #import DLFCN as dl
MPI.pxd in init pymusic (pymusic.cpp:12172)()
ValueError: mpi4py.MPI.Win has the wrong size, try recompiling
In [2]:
`
As far as I can recall from the documentation MPI.Win was the windows size for buffers (mpi3 features?) but no idea why
I would like to ask if there are specific restrictions not to use a modern C++ standard.
This is because the code of the Setup object is pretty fragile with all the pointers and not very obvious when Ports/Connections etc. are free'd and which class is responsible for that. In c++11 there are shared pointers and move semantic to overcome such problem. Right now I try to emulate a similar behaviour like std::move such that the runtime "steals" ports and connections from the setup instances such that the user only has an empty object left.
Hi, are there any plans to upgrade the existing Debian/Ubuntu packages with version 1.0.7 to the current version 1.1.15?
Sorry to ask this question here in the issues, but I haven't found another channel yet.
When I run ./autogen.sh
it triggers aclocal.sh
and it fails eventually.
dirname: missing operand
Try 'dirname --help' for more information.
dirname: missing operand
Try 'dirname --help' for more information.
aclocal
./aclocal.sh: 21: ./aclocal.sh: aclocal: not found
MessageInputSubconnector::receive assumes that sets of messages are aligned with MESSAGE_BUFFER_MAX. Also, a message larger than MESSAGE_BUFFER_MAX can't be handled.
I would like to propose to collect feature requests/improvement-proposals in this issue;
To start with some suggestions are:
Improvements:
Feature request
Moving MUSIC to support service oriented architectures/interactive 'master/slave' use-cases such as the HBP-Neuro robotics platform and interactive visualization tools. Technically speaking this would include:
All of this implies a bunch of work and needs to be discussed of course
Hello, I am facing a problem with pymusic, while testing MUSIC/pymusic/examples. While MUSIC/examples working fine.
Following is my environment detail.
OS: Ubuntu 20.04.1 LTS
Python 3.8.5
Cython 0.29.21
Open MPI 4.0.3
MUSIC 1.1.16
Error after running hello world example.
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:
Process name: [[49895,1],0]
Exit code: 1
Also attached log of make install.
log_make_install.txt
Please check if I missed something.
Thanks
Hi all,
I would like to install MUSIC on an HPC. Despite loading all necessary modules for compiling MUSIC, I get this error and many errors like this during make: ./music/data_map.hh:46:13: error: ‘MPI’ does not name a type
virtual MPI::Datatype type () = 0;
On my PC the compilation goes without any problem. I have tried with different versions of mpi on PC as well as on HPC, including 3.1.1 and 4.0.3, however I get the same error for all of the versions only on HPC.
Could someone help me on this issue?
Best,
Mohammad
Hi,
I get the following issue when compiling MUSIC
Makefile:336: warning: ignoring old commands for target predict_rank.py' make[2]: Nothing to be done for
all'.
make[2]: Leaving directory /home/fzi/Packages/MUSIC/music-config' Making all in pymusic make[2]: Entering directory
/home/fzi/Packages/MUSIC/pymusic'
cd ../pymusic; /usr/bin/python -c
"from Cython.Build import cythonize;
cythonize('pymusic.pyx', verbose=1)"
cd ../pymusic; /usr/bin/python -c
"from Cython.Build import cythonize;
cythonize('pybuffer.pyx', verbose=1)"
missing cimport in module 'mpi4py.mpi_c': pybuffer.pxd
missing cimport in module 'mpi4py.mpi_c': pymusic.pxd
Compiling pybuffer.pyx because it changed.
[1/1] Cythonizing pybuffer.pyx
Compiling /home/fzi/Packages/MUSIC/pymusic/pybuffer.pyx
missing cimport in module 'mpi4py.mpi_c': ./music/pybuffer.pxd
Compiling pymusic.pyx because it changed.
[1/1] Cythonizing pymusic.pyx
Compiling /home/fzi/Packages/MUSIC/pymusic/pymusic.pyx
...
cdef extern from "mpi_compat.h":
pass
cimport mpi4py.MPI as MPI
from mpi4py.mpi_c cimport *
pybuffer.pxd:7:0: 'mpi4py/mpi_c.pxd' not found
...
cdef extern from "mpi_compat.h":
pass
cimport mpi4py.MPI as MPI
from mpi4py.mpi_c cimport *
pymusic.pxd:7:0: 'mpi4py/mpi_c.pxd' not found
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 877, in cythonize
cythonize_one(args)
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 997, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: pybuffer.pyx
make[2]: ** [pybuffer.cpp] Error 1
make[2]: *** Waiting for unfinished jobs....
...
void tick()
cdef extern void cython_callback(PyObject*, double, IndexType, int)
cdef extern from "music/music_c.h" namespace "MUSIC":
cdef inline MPI_Comm communicator(CSetup*)
pymusic.pxd:136:16: 'MPI_Comm' is not a type identifier
...
cdef extern void cython_callback(PyObject*, double, IndexType, int)
cdef extern from "music/music_c.h" namespace "MUSIC":
cdef inline MPI_Comm communicator(CSetup_)
cdef inline MPI_Comm communicator(CRuntime_)
pymusic.pxd:137:16: 'MPI_Comm' is not a type identifier
...
cdef Args r = argv_toc(argv if argv is not None
else sys.argv)
try:
if required is None:
self.ptr = new CSetup(r.argc, r.argv)
self.provided = MPI_THREAD_SINGLE
pymusic.pyx:377:49: undeclared name not builtin: MPI_THREAD_SINGLE
...
index (0): if index_map is None, this is the base offset for constructing
a linear index
"""
self.buf = buf
self.ptr = new CArrayData(buf.pybuf.buf,
buf.dtype.ob_mpi,
pymusic.pyx:601:43: Cannot convert 'MPI_Datatype' to Python object
...
self.ptr = new CArrayData(buf.pybuf.buf,
buf.dtype.ob_mpi,
index, buf.items)
if index_map is None else
new CArrayData(buf.pybuf.buf,
buf.dtype.ob_mpi,
pymusic.pyx:605:43: Cannot convert 'MPI_Datatype' to Python object
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 877, in cythonize
cythonize_one(args)
File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 997, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: pymusic.pyx
make[2]: ** [pymusic.cpp] Error 1
make[2]: Leaving directory /home/fzi/Packages/MUSIC/pymusic' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory
/home/fzi/Packages/MUSIC'
make: *** [all] Error 2
I am not sure if this depends on my MPI version but mpi4py runs without issues
Hello everyone,
while debugging a more complex experiment using MUSIC I recognized that two of my pymusic-nodes simply never return from calling music_setup.runtime(..)
, whereas music_setup = music.Setup()
.
I have been able to reduce the code to a very minimal not working example.
When I execute the music configuration attached to this issue via mpirun -np 1 music test.music
nothing more than "before runtime"
will get printed. The process appears to enter an endless loop when calling runtime
on the setup object.
In contrast, mpirun -np 1 ./test_node.py
works fine.
I'm well aware that this trivial node combined with this trivial configuration might be a special case for MUSIC, but this issue also arises with a more complex setup and nodes which actually consume data via incoming connections.
music --version
: MUSIC 1.1.15
(8e0a609)mpirun --version
: mpirun (Open MPI) 1.6.5
Minimal example (download: test.zip):
test_node.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import music
music_setup = music.Setup()
print("before runtime")
runtime = music_setup.runtime(0.02)
print("after runtime -- not called")
for current_time in runtime:
print(current_time)
test.music
[test]
binary=./test_node.py
args=
np=1
The C++ interface to MPI has been deprecated.
On OpenMPI installations >= 2.0.1 under OSX, the C++ interface is disabled.
MUSIC should switch to using the C interface.
Notifying @heplesser.
Running music without mpirun (or equivalent) produces this output:
music: application_mapper.cc:187: void MUSIC::ApplicationMapper::mapApplications(): Assertion `config_->getDict().size() != 0' failed.
As far as I remember, the error message used to be clear. This message is not informative to identify the problem.
Use-case: I am trying to adapt the PyNN MUSIC branch towards PyNN 0.8.1 and NEST 2.10. While the PyNN part was no problem a segmentation fault occurs within MUSIC. This happens independently of PyNN-MUSIC.
The error can be reproduced when launching MUSIC with _MUSIC_CONFIG=POSTPONE:0 either with 'python <scriptname.py>' or 'mpiexec -np 1 <scriptname.py>' (launching music as single process)
@mdjurfeldt
From what I can observe is that the old Python-config API sets POSTPONE: but the configuration parser actually expects an ApplicationMap-section within the ENV. Maybe I have missed something to do but in the current state it looks like that either the music-config/config.py must provide a full application-map in the ENV (=I need to change the way PyNN-Music/multisim.py assembles this) or that the MUSIC C++ code must be adapted.
Another question is what the runtime actually does when postpone is true? I mean it calls maybePostponedSetup but I dont see where the updated ENV's are actually parsed, maybe I do miss something?
`void
Setup::maybePostponedSetup ()
{
if (postponeSetup_)
{
delete config_;
config_ = new Configuration ();
fullInit ();
}
}
void
Setup::fullInit ()
{
errorChecks ();
if (!config ("timebase", &timebase_))
timebase_ = MUSIC_DEFAULT_TIMEBASE; // default timebase
string binary;
config_->lookup ("binary", &binary);
string args;
config_->lookup ("args", &args);
argv_ = parseArgs (binary, args, &argc_);
temporalNegotiator_ = new TemporalNegotiator (this);
}
`
There seem to be changes to the local copy of rudeconfig used in MUSIC which are not present in upstream (https://github.com/mflood/rudeconfig). These should be sent upstream---most downstream distributions do not permit private copies of libraries (bundling).
If these are improvements, I can diff the two sources (upstream and the local copy) and open PRs upstream?
Currently Music is not flexible enough for some (more or less special) use-cases. Especially in order to set up ports, they must be known a-priori on all communication sides. I've tried to use MPI4PY to send messages via the global communicator but it did not work out of the box. The "usual" message in/out ports are also not very suitable for this because they can only be used when calling tick(), which is only available after the Setup has been done.
For the moment its fine, we just want to get our neuro-robotics platform working with distributed NEST instances. In the long term, however, we would like to support all features and, thus, we would like to add a bit more flexibility. I could spent some time on doing this if the maintainers of music are interested.
@mdjurfeldt I recall that you were talking in Geneva about something like "out of order" ports, maybe this was exactly for this use-case?
There should be a SparseData alternative to ArrayData for cases where state variables in the application are not stored in a contiguous array.
Seem the Travis setup is failing for MUSIC. The tests fail already during setup:
$ sudo python get-pip.py
You're using an outdated location for the get-pip.py script, please use the one available from https://bootstrap.pypa.io/get-pip.py
The command "sudo python get-pip.py" failed and exited with 1 during .
Your build has been stopped.
I am installing MUSIC on Ubuntu 16.04 (with mpich 3.3~a2-4). I get the following issue:
$ autogen.sh
$ ./configure
$ make
$ make install
[...]
running install_egg_info
Writing /home/xis/HBP/MUSIC-master/build/lib/python3.6/site-packages/music-1.1.16-py3.6.egg-info
install-lib=/home/xis/HBP/MUSIC-master/build/lib/python3.6/site-packages \
--install-scripts=/home/xis/HBP/MUSIC-master/build/bin \
--install-data=/home/xis/HBP/MUSIC-master/build/share/music
/bin/bash: line 1: install-lib=/home/xis/HBP/MUSIC-master/build/lib/python3.6/site-packages: No such file or directory
Makefile:881: recipe for target 'install-exec-hook' failed
make[5]: [install-exec-hook] Error 127 (ignored)
The installation routine seems to ignore this problem. Any ideas on how to fix this? Thanks!
I'm trying to read the positions of 2 joints from the topic /joint_states as known from http://wiki.ros.org/joint_state_publisher.
These positions i want to feed to my neurons in NEST and they will respond by publishing a torque to topic /jointgroupeffort/command.
In this effort I'm having issues with converging from arrays generated by the publisher /jointstates into just 2 inputs to NEST. In doing so I have followed the example with the braitenberg vehicle where the "braitenberg3_converge_weights.dat"
braitenberg3_converge_weights.txt
converges 20 elements into 2 elements, by splitting the 20 elements into 1/20'th parts.
Either I have misunderstood the procedure from the Braitenberg example, or I assume a wrong structure of data published in the /jointstates.
This is my converging weight file
fable_converge_weights2.txt
Where i try to single out the position of the first joint and the position of the second joint.
music file:
fablenew5.txt
When running the simulations I get this error, which I don't receive when using the "braitenberg3_converge_weights.dat" in the same simulation.
martin@martin-X75VD:~/catkin_ws/src/ros-music-adapters/examples/braitenberg3/scripts$ mpirun -np 6 music fablenew5.music
ContROSAdapter init
[INFO] [2017.10.26 17:58:32 /opt/nest-2.12.0/nestkernel/rng_manager.cpp:226 @ Network::create_rngs_] : Creating default RNGs
ContContAdapter init
LinearDecoderAdapter init
RateEncoderAdapter init
ROSContAdapter init
[INFO] [2017.10.26 17:58:32 /opt/nest-2.12.0/nestkernel/rng_manager.cpp:221 @ Network::create_rngs_] : Deleting existing random number generators
[INFO] [2017.10.26 17:58:32 /opt/nest-2.12.0/nestkernel/rng_manager.cpp:226 @ Network::create_rngs_] : Creating default RNGs
[INFO] [2017.10.26 17:58:32 /opt/nest-2.12.0/nestkernel/rng_manager.cpp:272 @ Network::create_grng_] : Creating new default global RNG
-- N E S T --
Copyright (C) 2004 The NEST Initiative
Version 2.12.0 Oct 16 2017 20:28:41
This program is provided AS IS and comes with
NO WARRANTY. See the file LICENSE for details.
Problems or suggestions?
Visit http://www.nest-simulator.org
Type 'nest.help()' to find out more about NEST.
Oct 26 17:58:34 ModelManager::clear_models_ [Info]:
Models will be cleared and parameters reset.
Oct 26 17:58:34 Network::create_rngs_ [Info]:
Deleting existing random number generators
Oct 26 17:58:34 Network::create_rngs_ [Info]:
Creating default RNGs
Oct 26 17:58:34 Network::create_rngs_ [Info]:
Deleting existing random number generators
Oct 26 17:58:34 Network::create_rngs_ [Info]:
Creating default RNGs
Oct 26 17:58:34 Network::create_grng_ [Info]:
Creating new default global RNG
Oct 26 17:58:34 MusicEventHandler::publish_port() [Info]:
Mapping MUSIC output port 'out' with width=2.
Oct 26 17:58:34 NodeManager::prepare_nodes [Info]:
Preparing 8 nodes for simulation.
Oct 26 17:58:34 MusicEventHandler::publish_port() [Info]:
Mapping MUSIC input port 'in' with width=2 , acceptable latency=0 ms.
Oct 26 17:58:34 MUSICManager::enter_runtime [Info]:
Entering MUSIC runtime with tick = 1 ms
Oct 26 17:58:34 SimulationManager::resume [Info]:
Number of local nodes: 8
Simulaton time (ms): 1000
Number of OpenMP threads: 1
Number of MPI processes: 1
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
[martin-X75VD:07505] *** Process received signal ***
[martin-X75VD:07505] Signal: Aborted (6)
[martin-X75VD:07505] Signal code: (-6)
[martin-X75VD:07505] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x7f983bb5f4b0]
[martin-X75VD:07505] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f983bb5f428]
[martin-X75VD:07505] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f983bb6102a]
[martin-X75VD:07505] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x16d)[0x7f983c19984d]
[martin-X75VD:07505] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d6b6)[0x7f983c1976b6]
[martin-X75VD:07505] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d701)[0x7f983c197701]
[martin-X75VD:07505] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d919)[0x7f983c197919]
[martin-X75VD:07505] [ 7] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8debc)[0x7f983c197ebc]
[martin-X75VD:07505] [ 8] /usr/local/lib/libmusic.so.1(_ZN5MUSIC4FIBO4growEi+0xa1)[0x7f983cc694a1]
[martin-X75VD:07505] [ 9] /usr/local/lib/libmusic.so.1(_ZN5MUSIC4FIBO6insertEv+0x27)[0x7f983cc695f7]
[martin-X75VD:07505] [10] /usr/local/lib/libmusic.so.1(_ZN5MUSIC17OutputRoutingData7processEdi+0x16)[0x7f983cc3de46]
[martin-X75VD:07505] [11] /usr/local/lib/libmusic.so.1(_ZN5MUSIC24IntervalProcessingRouterINS_17OutputRoutingDataENS_12IntervalTreeIiNS_8IntervalES1_EEE12processEventEdi+0xd4)[0x7f983cc392f4]
[martin-X75VD:07505] [12] /opt/music-adapters/libmusicadapter.so(_ZN12EventOutPort4sendEid+0xd)[0x7f983d0f3dad]
[martin-X75VD:07505] [13] music(_ZN11RateEncoder4tickEv+0x109)[0x408b99]
[martin-X75VD:07505] [14] /opt/music-adapters/libmusicadapter.so(_ZN7Adapter3runEb+0x88)[0x7f983d0f4f08]
[martin-X75VD:07505] [15] music(main+0x35)[0x408935]
[martin-X75VD:07505] [16] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f983bb4a830]
[martin-X75VD:07505] [17] music(_start+0x29)[0x4089b9]
[martin-X75VD:07505] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 2 with PID 7505 on node martin-X75VD exited on signal 6 (Aborted).
Hereby I would like to propose some minor changes to the interface;
@mdjurfeldt
Currently, there is no way to get info about the available ports in the config at an application which could be used to automatically instantiate it in a factory.
Use cases are any kind of plotting libraries/closed loop systems etc. But actually the info is available, its just accessible via an environment variable in a serialized format (a user library should not deal with that). Also info about the port type is missing. I think the port type should also be part of the config.
The second point is the Event callback for Event input ports. Would it be possible to make the buffer transparent to python or at least to have a callback that just returns two lists (times, senders) instead of one function call for each single spike? I guess this would be a great performance improvement!
For now I do all of this outside of MUSIC but it would be great if you agree on this and we can implement it at some point. Other features requests regarding the interface could also be linked here to develop an idea of how the interface in the next versions could look like.
Hi all,
I am running into the following issue with a very simple music example. I have a non-nest script sending spikes to a nest simulation. The latter simply consists of a parrot neuron that should just replicate the received events, and a spike detectors that should write the spikes to a file. Unfortunately, the file remains empty as if no spikes are received.
I debugged it as follows:
The only case that does not work is when I have a non-nest sender and a nest-receiver. Do you have a clue why that could be the case? Code is as follows...
Non-nest sender
import music
import sys
timestep = 0.001
stoptime = 1.0 # This is the actual execution time, not the simulated time span
setup = music.Setup()
outp = setup.publishEventOutput("out")
firstId = 0
size = 1
buf = 1
outp.map (music.Index.GLOBAL,
base=firstId,
size=size,
maxBuffered=buf)
runtime = setup.runtime(timestep)
tickt = runtime.time()
while tickt <= stoptime:
# Send an event for each tick, util stoptime
outp.insertEvent(tickt, firstId, music.Index.GLOBAL)
runtime.tick()
tickt = runtime.time()
Nest receiver
import nest
N = 1
nest.SetKernelStatus({"overwrite_files": True})
meip = nest.Create ('music_event_in_proxy', N)
nest.SetStatus (meip, [{'port_name': 'spikes_in', 'music_channel': 0}])
delay=2.0
nest.SetAcceptableLatency('spikes_in', delay)
# Parrot neuron
n_in = nest.Create ("parrot_neuron", N)
nest.Connect(meip, n_in, 'one_to_one', {"weight":1.0, "delay": delay})
# Create a spike detecor
sdetector = nest.Create("spike_detector")
nest.SetStatus(sdetector, {"withgid": True, "withtime": True, "to_file": True,
"label": "receive", "file_extension": "spikes"})
nest.Connect(n_in, sdetector)
nest.Simulate (1000.0)
Music config file
[from]
binary=./sender.py
np=1
[to]
binary=./receiver_nest.py
np=1
from.out -> to.spikes_in [1]
configure should have an option --disable-python
mpirun -np 2 music run.music
for the attached testcase leads to:
out1: connected
in1: connected
(0, 0.0)
[node16:13393] *** Process received signal ***
[node16:13393] Signal: Segmentation fault (11)
[node16:13393] Signal code: Address not mapped (1)
[node16:13393] Failing at address: 0x8
[node16:13393] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xf890) [0x7f087af1c890]
[node16:13393] [ 1] /home/hoff/.local/lib/libmusic.so.1(_ZN5MUSIC15EventOutputPort15insertEventImplEdi+0) [0x7f08791b2520]
[node16:13393] [ 2] /home/hoff/.local/lib/python2.7/site-packages/music/pymusic.so(+0x26178) [0x7f0879c3f178]
[node16:13393] [ 3] /usr/bin/python(PyEval_EvalFrameEx+0x6a0a) [0x4f60ca]
[node16:13393] [ 4] /usr/bin/python(PyEval_EvalCodeEx+0x7fb) [0x4f696b]
[node16:13393] [ 5] /usr/bin/python(PyEval_EvalFrameEx+0x573b) [0x4f4dfb]
[node16:13393] [ 6] /usr/bin/python(PyEval_EvalCodeEx+0x7fb) [0x4f696b]
[node16:13393] [ 7] /usr/bin/python(PyEval_EvalCode+0x19) [0x4f6a89]
[node16:13393] [ 8] /usr/bin/python(PyRun_FileExFlags+0x134) [0x51e6a4]
[node16:13393] [ 9] /usr/bin/python(PyRun_SimpleFileExFlags+0xdf) [0x52016f]
[node16:13393] [10] /usr/bin/python(Py_Main+0xb51) [0x537981]
[node16:13393] [11] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f087a260b45]
[node16:13393] [12] /usr/bin/python() [0x41859e]
[node16:13393] *** End of error message ***
The testcase does contain two simple nodes with a unidirectional event connection.
I initially build this testcase to demonstrate another faulty behaviour: Having multiple unconnected event outputs fails. I will go into detail as soon as we could solve the above (it should be just a minor thing, I suppose).
[send]
np = 1
binary = ./send_node.py
[recv]
np = 1
binary = ./recv_node.py
send.out1 -> recv.in1 [1]
#send.out2 -> recv.in2 [1]
#send.out3 -> recv.in3 [1]
#! /usr/bin/python
import music
import itertools
setup = music.Setup()
def make_out_proxy(port_name):
proxy = setup.publishEventOutput(port_name)
print("{}: {}".format(port_name, ["not connected", "connected"][proxy.isConnected()]))
if proxy.isConnected():
assert proxy.width() == 1
proxy.map(music.Index.GLOBAL, size=1, base=0, maxBuffered=1)
return lambda time: proxy.insertEvent(time, 0, music.Index.GLOBAL)
return lambda time: None
outs = map(make_out_proxy, ["out1"])
for index, time in zip(itertools.cycle(range(len(outs))),
itertools.takewhile(lambda t: t < 1000.0, setup.runtime(0.01))):
print(index, time)
outs[index](time + 0.5)
#! /usr/bin/python
import music
import itertools
import functools
setup = music.Setup()
def print_spike(port, time, _, __):
print("Received spike from port {}: {}".format(port, time))
def make_in_proxy(port_name):
proxy = setup.publishEventInput(port_name)
print("{}: {}".format(port_name, ["not connected", "connected"][proxy.isConnected()]))
if proxy.isConnected():
assert proxy.width() == 1
proxy.map(functools.partial(print_spike, port_name), music.Index.GLOBAL,
size=proxy.width(), base=0, maxBuffered=1, accLatency=0)
return proxy
ins = map(make_in_proxy, ["in1"])
for time in itertools.takewhile(lambda t: t < 1000.0, setup.runtime(0.01)):
pass
Runtime::tick () calls postCommunication every tick. For each connector, it should only be called if communication has occurred.
pyconfig.pxi should not exist in the repo but be generated locally.
It should also be removed at distclean.
Otherwise, people with mpi4py 2.0.0 will get problems building pymusic from a newly checked out repo.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.