nest / nest-docker Goto Github PK

Hi. During some testing of Singularity on JUSUF, MPICH appears to be required in the container as Slurm/srun calls the Intel MPI executables. Would it make sense to provide both openMPI and MPICH variants here?

Just a discussion point - the recipes present here incorporates a whole bunch of redundant software which is not needed to compile and run NEST. Why is that the case?

Using buildpack-deps:focal as base image, only a handful of additional installs are needed (taken from https://github.com/LFPy/hybridLFPy_EBRAINS/blob/main/Dockerfile):

FROM buildpack-deps:focal

RUN apt-get update && \
    apt-get install -y \
    cmake \
    libmpich-dev \  # or libopenmpi-dev
    mpich \  # or openmpi-bin
    doxygen \
    libboost-dev \
    libgsl-dev \
    cython3 \
    python3-dev \
    python3-pip

RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10 && \
    update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 10

RUN pip install mpi4py  # or apt-get install -y python3-mpi4py if openmpi is used.

# ----- Install NEST -----
RUN git clone https://github.com/nest/nest-simulator.git && \
    cd nest-simulator && \
    # git checkout master && \
    # git checkout 24de43dc21c568e017839eeb335253c2bc2d487d && \
    cd .. && \
    mkdir nest-build && \
    ls -l && \
    cd  nest-build && \
    cmake -DCMAKE_INSTALL_PREFIX:PATH=/opt/nest/ \
        -Dwith-ltdl=ON \
        -Dwith-gsl=ON \
        -Dwith-readline=ON \
        -Dwith-python=ON \
        -Dwith-mpi=ON \
        -Dwith-openmp=ON \
        ../nest-simulator && \
    make && \
    make install && \
    cd /

RUN rm -r nest-simulator
RUN rm -r nest-build

# Add NEST binary folder to PATH
ENV PATH /opt/nest/bin:${PATH}

# Add pyNEST to PYTHONPATH
ENV PYTHONPATH /opt/nest/lib/python3.8/site-packages:${PYTHONPATH}

If all examples and tests are to be run, and make things a bit more user friendly, a few additional dependencies may be needed, e.g.,:

# ---- additional requirements
RUN apt-get install -y \
    python3-numpy \
    python3-scipy \
    python3-matplotlib \
    python3-pandas \
    ipython3 \
    jupyter

RUN update-alternatives --install /usr/bin/ipython ipython /usr/bin/ipython3 10

I know MUSIC etc. may require some additional dependencies, but it can't possibly be that many, no?

In aspect to prepare NEST Desktop on HBP infrastructure, I also build nest docker there.
However, it failed in adding user during the entrypoint.

I found that the OpenShift (the system of HBP infrastructure) restricts the permission to add user.

A possible solution could be removing adduser from the entrypoint.
Other solution is to ask the admin of the HBP infrastructure to add user initially before building docker images.

What do you think?

In the CI of the development Docker image, an error occurs during the installation of NESTML.
The development version of NESTML is installed with:

RUN python3 -m pip install https://github.com/nest/nestml/archive/refs/heads/master.zip --upgrade

This leads to the error:

AssertionError: Python 3.9 or higher is required to run NESTML

Full error message:

=> ERROR [stage-1 4/6] RUN python3 -m pip install https://github.com/nest/nestml/archive/refs/heads/master.zip --upgrade 6.5s

[stage-1 4/6] RUN python3 -m pip install https://github.com/nest/nestml/archive/refs/heads/master.zip --upgrade:
#11 1.095 Collecting https://github.com/nest/nestml/archive/refs/heads/master.zip
#11 1.735 Downloading https://github.com/nest/nestml/archive/refs/heads/master.zip
#11 6.354 ERROR: Command errored out with exit status 1:
#11 6.354 command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-qv50vk09/setup.py'"'"'; file='"'"'/tmp/pip-req-build-qv50vk09/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-req-build-qv50vk09/pip-egg-info
#11 6.354 cwd: /tmp/pip-req-build-qv50vk09/
#11 6.354 Complete output (5 lines):
#11 6.354 Traceback (most recent call last):
#11 6.354 File "", line 1, in
#11 6.354 File "/tmp/pip-req-build-qv50vk09/setup.py", line 26, in
#11 6.354 assert sys.version_info.major >= 3 and sys.version_info.minor >= 9, "Python 3.9 or higher is required to run NESTML"
#11 6.354 AssertionError: Python 3.9 or higher is required to run NESTML
#11 6.354 ----------------------------------------
#11 6.378 ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

executor failed running [/bin/sh -c python3 -m pip install https://github.com/nest/nestml/archive/refs/heads/master.zip --upgrade]: exit code: 1

As suggested by @jougs , I open this issue to discuss a problem with the execution of the nest-server on PizDaint with Sarus. On that machine the execution of the command

sarus run nestsim/nest:latest nest-server

generates the following output:

id: 'nest': no such user
UID : 9001
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
adduser: Only root may add a user or group to the system.

The main issue is the one reported in the last line, that refers to the execution of the adduser command currently located in the entrypoint file: with Sarus there are some limitations to the operations that can be performed inside a container. I have tried to fix the above issue for our case moving the adduser operation inside the Docker file (please see the attached .zip), basically copying an old version of the Dockerfile/entrypoint pair we were using before the inclusion of the server in the nest code, but I am not sure it is a good idea. The image built with the attached Dockerfile/entrypoint seems to work with Sarus on Daint but we would like to find a better solution and avoid to branch from the official version.

Cheers
Christopher

Files.zip

This is a follow up to PR #74.
docker-compose.yml and README.md need an update because of the new environment variable NEST_CONTAINER_MODE.

Dear all,
I created an instalation of NEST simulator v3.6 using lubuntu for new users. My build his available here. I strongly believe that this build will help with the user experience. Therefore, I would like to share this with community unnder Apache 2.0 license.
Kind regards,
Pedro

⚠️ The mentioned container is not related to NEST. Check for unwanted content first! ⚠️ – @terhorstd

Latest version of sphinx_gallery (v0.11.1) do not work. Error message below.

Checked other versions. v0.10.1 is ok

Possible fix: use python v3.9.

#12 3.183 Extension error (sphinx_gallery.gen_gallery):
#12 3.183 Handler <function generate_gallery_rst at 0x7f321bf99940> for event 'builder-inited' threw an exception (exception: sequence item 1: expected str instance, PosixPath found)
#12 3.502 make[3]: *** [doc/CMakeFiles/html.dir/build.make:72: doc/CMakeFiles/html] Error 2
#12 3.502 make[2]: *** [CMakeFiles/Makefile2:569: doc/CMakeFiles/html.dir/all] Error 2
#12 3.502 make[1]: *** [CMakeFiles/Makefile2:576: doc/CMakeFiles/html.dir/rule] Error 2
#12 3.503 make: *** [Makefile:231: html] Error 2

After a build on Docker Hub there is no ./data directory inside the image. Because of this, the start of the Docker container ends with an error;

/home/nest/entrypoint.sh: line 9: cd: ./data: No such file or directory

The page https://hub.docker.com/r/nestsim/nest points to the personal repository, but should probably point here.