Scripts to automate package builds on RC Platforms
ucl-rits / rcps-buildscripts Goto Github PK
View Code? Open in Web Editor NEWScripts to automate package builds on RC Platforms
License: MIT License
Scripts to automate package builds on RC Platforms
License: MIT License
IN01325476
http://www.mesa3d.org/install.html
Should be something like this:
./configure --prefix=/path/to/install/dir --enable-osmesa
make
make install
May need to load xorg-utils/X11R7.7
For .so.6 I am assuming you might need this version of the source: ftp://ftp.freedesktop.org/pub/mesa/older-versions/6.x/
(We have a libOSMesa.so.8 in the VisIt module with the libraries that VisIt builds)
Legion install requested. IN00874137
Add libXdmcp to the xorg-utils/X11R7.7 module - required by Amber12 which some people have personal installs of. IN00775845
ftp://ftp.x.org/pub/X11R7.7/src/lib/libXdmcp-1.1.1.tar.bz2
IN01315017
In particular these:
BRASS (depends on grass, VAGrENT, cgpVcf)
CaVEMan and cgpCaVEManPostProcessing
gcpIndel
ascatNgs
There are a great many dependencies needed for all these, some of which we have as modules already.
For IN00950423
https://github.com/mattcieslak/easy_lausanne
Prerequisites:
IN00937227
http://www.recamo.cz/en/software/hammock-cluster-peptides/
This program works in combination with clustal-omega, hmmer and HHsuite. (Comes with compiled versions, but strongly recommends you to build your own).
Jobs which complete successfully or end on an error are going back into the queue to be rerun rather than being removed from it. This is a known issue and we are working on it.
It is trying to use the dummy Pstream library instead of the correct one.
mpirun -np 4 interFoam -parallel
--> FOAM FATAL ERROR:
Trying to use the dummy Pstream library.
This dummy library cannot be used in parallel mode
From function UPstream::init(int& argc, char**& argv)
in file UPstream.C at line 37.
FOAM exiting
Scripts relying on #Local2Scratch
won't copy anything back until we implement it on the new system. You'll need to put a manual tar line in your script, like this
#9. Preferably, tar-up (archive) all output files onto the shared scratch area
tar zcvf $HOME/Scratch/output/files_from_job_$JOB_ID.tgz $TMPDIR
If I try to install kmc locally, the install script does not download the package in the temp dir, as in the messages below, but the same wget from the command line works.
$ bash rcps-buildscripts/kmc-2.1.1_install
--2015-11-16 09:33:11-- https://github.com/marekkokot/KMC/archive/2.1.1.tar.gz
Resolving github.com (github.com)... 192.30.252.130
Connecting to github.com (github.com)|192.30.252.130|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/marekkokot/KMC/tar.gz/2.1.1 [following]
--2015-11-16 09:33:11-- https://codeload.github.com/marekkokot/KMC/tar.gz/2.1.1
Resolving codeload.github.com (codeload.github.com)... 192.30.252.145
Connecting to codeload.github.com (codeload.github.com)|192.30.252.145|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 819958 (801K) [application/x-gzip]
Saving to: '2.1.1.tar.gz'
0% [ ] 0 --.-K/s in 0s
Cannot write to '2.1.1.tar.gz' (Success).
We don't have a standalone ScaLAPACK in the new software stack, only MKL's. Related to IN00902683 for individual installs of CP2K.
IN00979987
Request for new version.
http://www.ks.uiuc.edu/Development/Download/download.cgi?PackageName=NAMD
Not on new Legion. IN00777097
STAR-CD using Intel MPI works within a single node but doesn't work across nodes on either
Legion or Grace.
We've received a request via ISD Purchasing to upgrade Amber on Legion and Grace to version 16.
IN00824302
Looks like it was missing some parts of X11 when compiled, so trying to run gmx view gives
Compiled without X-Windows - can not run viewer.
Not on new Legion. IN00774818
IN01157939
Requires Python 2.7 with numpy, also ghostscript and pdf2svg.
http://weblogo.threeplusone.com/manual.html
https://github.com/WebLogo/weblogo
The GPU nodes haven't been moved yet either.
If you need to use the GPU nodes immediately, email rc-support and we can allow you to submit jobs on old Legion.
IN01261049
A parallelised version of AutoDock Vina, molecular docking.
Licensed under Apache License 2.0.
http://mvirdb1.llnl.gov/static_catsid/vina/
Requires boost, mentions boost-mpi as a possibility but says that it currently uses MPI directly.
Beside the standard installation, you can also turn on the Boost MPI binding. Currently the program uses MPI library directly instead of Boost MPI
Blast+ in not installed due to two errors; the following line has a typo (VERSION) and does not point to $VERSION
ftp://ftp.ncbi.nlm.nih.gov/blast/executables/LATEST/ncbi-blast-$(VERSION)+-src.tar.gz
so it should be changed to
ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/${VERSION}/ncbi-blast-${VERSION}+-src.tar.gz
Also, "../configure" should be "./configure" (configure is in the current dir)
These fixes are available in the fork ICONIC-UCL/rcps-buildscripts
Myself, and at least two of my PhD students require an online service called R.E.D. RESP (Restrained Electrostatic Potential). It essentially derives point charges of an atomistic structure for AMBER by first optimising the structure using Gaussian (Gaussian Inc. gave them a 'public' license to dump out the final structure but not any of the energetic or frequency data) and then running the structure through a series of python scripts according to some key-value parameter values provided in two input files. The final product is a zip file full of charge data and coordinate geometry.
Unfortunately, a week ago the R.E.D. Server had gone down. The AMBER community has no idea what's happened. Thankfully, the service providers give away the source code under the GNU license so institutes can establish their own private R.E.D. RESP servers. The link to the scripts is below (rescued by someone before the service died) - as you can guess, any further information has vanished along with the website.
http://lysozyme.medchem.utah.edu/ros/public_html/RED/RED-III.52-Tools-Files/
Investigate if we can install this as a module on Legion. IN00799592
Molpro 2015 on Legion to solve problems with multi-core runs within a node.
(Note: no binaries for Infiniband available yet).
IN00856134
IN01800984
A three-dimensional finite element mesh generator with built-in pre- and post-processing facilities
License: GPL
If we try to run either the python 2 or python 3 bundle scripts, we get this error:
Installing collected packages: numpy
Running setup.py install for numpy
Complete output from command /shared/ucl/apps/python/bundles/1451921561-3/venv/bin/python3.4 -c "import setuptools, tokenize;__file__='/tmp/pip-build-7yjz732n/numpy/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file
__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-5wekvzaj-record/install-record.txt --single-version-externally-managed --compile --install-headers /shared/ucl/apps/python/bundles/1451921561-3/venv/i
nclude/site/python3.4/numpy:
blas_opt_info:
blas_mkl_info:
libraries mkl,vml,guide not found in ['/shared/ucl/apps/python/bundles/1451921561-3/venv/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
NOT AVAILABLE
openblas_info:
libraries openblas not found in ['/shared/ucl/apps/python/bundles/1451921561-3/venv/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
NOT AVAILABLE
atlas_3_10_blas_threads_info:
Setting PTATLAS=ATLAS
libraries tatlas not found in ['/shared/ucl/apps/python/bundles/1451921561-3/venv/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64/sse2', '/usr/lib64', '/usr/lib/sse2', '/usr/lib', '/usr/lib/sse2', '/usr/lib/']
NOT AVAILABLE
atlas_3_10_blas_info:
libraries satlas not found in ['/shared/ucl/apps/python/bundles/1451921561-3/venv/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64/sse2', '/usr/lib64', '/usr/lib/sse2', '/usr/lib', '/usr/lib/sse2', '/usr/lib/']
NOT AVAILABLE
atlas_blas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in ['/shared/ucl/apps/python/bundles/1451921561-3/venv/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64/sse2', '/usr/lib64', '/usr/lib/sse2', '/usr/lib', '/usr/lib/sse2', '/
usr/lib/']
NOT AVAILABLE
atlas_blas_info:
libraries f77blas,cblas,atlas not found in ['/shared/ucl/apps/python/bundles/1451921561-3/venv/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64/sse2', '/usr/lib64', '/usr/lib/sse2', '/usr/lib', '/usr/lib/sse2', '/usr/
lib/']
NOT AVAILABLE
blas_info:
Replacing _lib_names[0]=='blas' with 'openblas'
Replacing _lib_names[0]=='openblas' with 'openblas'
C compiler: cc
creating /tmp/tmpxedlndus/tmp
creating /tmp/tmpxedlndus/tmp/tmpxedlndus
compile options: '-I/shared/ucl/apps/openblas/0.2.14/gnu-4.9.2/lib -I/usr/local/include -I/usr/include -I/shared/ucl/apps/python/bundles/1451921561-3/venv/include -c'
cc: /tmp/tmpxedlndus/source.c
cc /tmp/tmpxedlndus/tmp/tmpxedlndus/source.o -L/shared/ucl/apps/openblas/0.2.14/gnu-4.9.2/lib -lcblas -o /tmp/tmpxedlndus/a.out
/bin/ld: cannot find -lcblas
collect2: error: ld returned 1 exit status
/bin/ld: cannot find -lcblas
collect2: error: ld returned 1 exit status
cc /tmp/tmpxedlndus/tmp/tmpxedlndus/source.o -L/shared/ucl/apps/openblas/0.2.14/gnu-4.9.2/lib -lblas -o /tmp/tmpxedlndus/a.out
/bin/ld: cannot find -lblas
collect2: error: ld returned 1 exit status
Running from numpy source directory.
/tmp/pip-build-7yjz732n/numpy/numpy/distutils/system_info.py:1651: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
warnings.warn(AtlasNotFoundError.__doc__)
Traceback (most recent call last):
File "/shared/ucl/apps/python/3.4.3/gnu-4.9.2/lib/python3.4/distutils/unixccompiler.py", line 194, in link
self.spawn(linker + ld_args)
File "/tmp/pip-build-7yjz732n/numpy/numpy/distutils/ccompiler.py", line 28, in <lambda>
m = lambda self, *args, **kw: func(self, *args, **kw)
File "/tmp/pip-build-7yjz732n/numpy/numpy/distutils/ccompiler.py", line 72, in CCompiler_spawn
raise DistutilsExecError('Command "%s" failed with exit status %d%s' % (cmd, s, msg))
distutils.errors.DistutilsExecError: Command "cc /tmp/tmpxedlndus/tmp/tmpxedlndus/source.o -L/shared/ucl/apps/openblas/0.2.14/gnu-4.9.2/lib -lcblas -o /tmp/tmpxedlndus/a.out" failed with exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-build-7yjz732n/numpy/numpy/distutils/system_info.py", line 1727, in has_cblas
extra_postargs=info.get('extra_link_args', []))
File "/shared/ucl/apps/python/3.4.3/gnu-4.9.2/lib/python3.4/distutils/ccompiler.py", line 734, in link_executable
debug, extra_preargs, extra_postargs, None, target_lang)
File "/shared/ucl/apps/python/3.4.3/gnu-4.9.2/lib/python3.4/distutils/unixccompiler.py", line 196, in link
raise LinkError(msg)
distutils.errors.LinkError: Command "cc /tmp/tmpxedlndus/tmp/tmpxedlndus/source.o -L/shared/ucl/apps/openblas/0.2.14/gnu-4.9.2/lib -lcblas -o /tmp/tmpxedlndus/a.out" failed with exit status 1
During handling of the above exception, another exception occurred:
...
Followed by attempts to link against various blas lib names (-lblas etc) all of which fail.
This is despite us explicitly setting $LAPACK and $BLAS to the path to our openblas .so file.
Switching back to Numpy 1.9.2 resolves this issue, but obviously this is not sustainable, so a solution needs to be found. Ominously, the readme for 1.10.2 states:
- gh-6675 Optimized blas detection broken in master and 1.10.
http://sourceforge.net/projects/numpy/files/NumPy/1.10.2/
Which implies they tried to fix it and have broken it for us or that for some reason it is still broken for us. The error suggests that it's not searching for BLAS in the path we've set as it's searching in '/shared/ucl/apps/python/bundles/1451921561-3/venv/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/'].
And we have set $BLAS/$LAPACK thus:
LAPACK=${LAPACK:-/shared/ucl/apps/openblas/0.2.14/gnu-4.9.2/lib/libopenblas.so}
BLAS=${BLAS:-/shared/ucl/apps/openblas/0.2.14/gnu-4.9.2/lib/libopenblas.so}
export LAPACK
export BLAS
There are various options for fixing this, one of which might be to create sym-links in the openblas lib directory to libopenblas.so called "libblas.so" and "liblapack.so".
cceamda@login09 ~]$ module show hdf/5-1.8.15-p1-impi
-------------------------------------------------------------------
/shared/ucl/apps/modulefiles/libraries/hdf/5-1.8.15-p1-impi/intel-2015-update2:
module-whatis Adds hdf5 1.8.5-p1 (built with Fortran and IntelMPI options) to your environment.
prereq compilers/intel/2015/update2
prereq mpi/intel/2015/update3/intel
prereq mpi/openmpi/1.8.4/gnu-4.9.2
but
module show mpi/intel/2015/update3/intel
-------------------------------------------------------------------
/shared/ucl/apps/modulefiles/libraries/mpi/intel/2015/update3/intel:
module-whatis adds Intel MPI to your environment variables
conflict mpi
and
[cceamda@login09 ~]$ module show mpi/openmpi/1.8.4/gnu-4.9.2
-------------------------------------------------------------------
/shared/ucl/apps/modulefiles/libraries/mpi/openmpi/1.8.4/gnu-4.9.2:
module-whatis adds OpenMPI 1.8.4 for GCC 4.9.2 compilers to your environment variables
conflict mpi
In the meantime, can run du -h --max-depth=1
to see the totals for all the top-level directories relative to where you are. (Run separately in Scratch to see things in there).
Sequence analysis. IN00778573
IN01847774
https://openfoamwiki.net/index.php/Contrib/waves2Foam
waves2Foam is an additional library and solver for OpenFOAM.
Was going to suggest building it for OpenFOAM 3, but it appears OpenFOAM 4 just came out two days ago - check on waves2Foam compatibility when we start this, as 4 is not currently mentioned.
We've had a request for a 1.2.x version of CESM.
I'll take a look at this.
CERNLIB on Legion for use with GNU 4.9.2 and NAG Mark 24.
Was on Old Legion.
IN01032834
The shipping group need PostGIS installed:
Most of the dependencies are already installed but we'll want to do a central install of postgres.
IN01814544
OpenSees is a software framework for developing applications to simulate the performance of structural and geotechnical systems subjected to earthquakes.
http://opensees.berkeley.edu
http://opensees.berkeley.edu/WebSVN/listing.php?repname=OpenSees&path=%2Ftrunk%2FSRC
They provide Windows and Mac binaries only, we'd need to compile from source (and edit a Makefile.def as they don't use configure at present).
IN01193402
https://data.broadinstitute.org/alkesgroup/EIGENSOFT/
Info:
http://www.hsph.harvard.edu/alkes-price/software/
http://www.hsph.harvard.edu/alkes-price/eigensoft-frequently-asked-questions/
Does not appear to use the Broad license so shouldn't need a separate application group:
SOFTWARE COPYRIGHT NOTICE AGREEMENT
This software and its documentation are copyright (2010) by Harvard University
and The Broad Institute. All rights are reserved. This software is supplied
without any warranty or guaranteed support whatsoever. Neither Harvard
University nor The Broad Institute can be responsible for its use, misuse, or
functionality. The software may be freely copied for non-commercial purposes,
provided this copyright notice is retained.
Registration is voluntary and for update information not license purposes. (From http://www.hsph.harvard.edu/alkes-price/software/)
SOFTWARE REGISTRATION: we encourage EIGENSOFT users to register here. Registration is voluntary, but will allow us to send information about software updates.
It appears they didn't find the sqlite3 headers when configuring.
IN00824441
Sequence data analysis. http://www.bioinformatics.babraham.ac.uk/projects/fastqc/
IN01255189
http://www.msg.ameslab.gov/gamess/download.html
Need to clarify license:
That page says this:
A site license for GAMESS is available at no cost to both academic and industrial users. You will be asked to agree to this license during the process of downloading GAMESS. The license acknowledges the Gordon group's exclusive distribution rights to GAMESS, and also prohibits you from making copies of the GAMESS code for any purpose except use at your own institution.
Actual license says this:
This license is considered a "Research Group" license. Thus anyone directly associated with your research group is covered by the license and may share your copy of GAMESS on all of your group's computer resources.
Sequence aligner. http://www.clustal.org/omega/
Also requires http://argtable.sourceforge.net/
IN00778573
Not on new Legion yet. IN00772718
MATLAB 2015a is available on new Legion, but the DCS is still being configured. I will update here when it is ready.
On Legion and Grace. Spglib is a C library for finding and handling crystal symmetries.
http://spglib.sourceforge.net/
IN00865871
IN01315433
The version contains a fix for a particular bug that I've been experiencing whilst running the current version (v1.0.3) on Grace, to do with restarting vibrational frequency calculations.
IN01051232
Currently still in beta - waiting for first full release before a central install, assuming this happens relatively soon.
Request for the AFNI software suite to be installed on Legion, available at afni.nimh.nih.gov.
Ticket IN00770976.
(Website currently not loading).
Currently we have Plumed 2.1.2
IN00804561
https://github.com/BioInf-Wuerzburg/proovread/releases
Depends on Log::Log4perl, File::Which, blast+ and samtools.
blast+/2.2.30/intel-2015-update2 and samtools/1.2/gnu-4.9.2 are available as modules and Log4perl is installed with perl/5.22.0 so just needs File::Which
IN00852851
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.