tmcclintock / cluster_toolkit Goto Github PK
View Code? Open in Web Editor NEWTools for analyzing galaxy clusters.
Home Page: http://cluster-toolkit.readthedocs.io/en/latest/
License: MIT License
cluster_toolkit's People
Contributors
Stargazers
Watchers
Forkers
rafaelgm9 jhod0 combet dylancromer matthewkirby chenzhiwei-171 marcpaterno cristobal-sifon evakosova luoneptunia wolffemcluster_toolkit's Issues
Move docs from readthedocs to github.io
The docs are currently hosted on readthedocs here. However, this is non-ideal because one needs to do some magic in a conf.py file to do magic imports of the toolkit before reading the docstrings. A better solution is to take advantage of the fact that github hosts docs now.
We can now host the docs on clutser_toolkit.githhub.io (or something like that) and provide pre-compiled documentation. We can compile the docs locally and then upload them, without having to deal with any RTD problems.
I will also take this chance to move away from autodocs, which have been a little finicky.
Change the Sigma(R) call to not need NFW parameters
Currently the function Sigma_at_R() takes in NFW parameters, despite taking a whole correlation function as input. The NFW parameters are not necessary unless R_perp < r, in which case a model must be assumed (such as NFW) below r[0].
The default behavior should get rid of needing NFW parameters and a method should be implemented to chose the model below the small scale.
Objectify the toolkit and add Pk attributes
Make the cluster-toolkit module itself an object that I can put attributes on. The first thing to do will be to add attributes for the linear and nonlinear power spectra.
Port boost factor models into this repo
Boost factors are written by themselves in the DES Y1 analysis repo. I should add them here. It won't be hard because boost factor models are very simple.
Write an example of how to use CLASS
Write a very short example of how to use CLASS to get the linear and nonlinear matter power spectra.
Cannot install cluster_toolkit currently in a NERSC Anaconda environment
Hi, the group working on clmassmod requested we add this package to the NERSC python conda environment, but my initial attempts failed. All I did was create a conda environment that cloned my root env, then cloned master from github, and tried to run python setup.py install
(clmassmod) desc@cori17:/global/common/software/lsst/common/miniconda/temp/cluster_toolkit> python setup.py install
ln: failed to create symbolic link 'cluster_toolkit/include/include': File exists
/tmp/easy_install-jji4hs6v/pytest-runner-4.2/.eggs/setuptools_scm-3.0.2-py3.6.egg/setuptools_scm/version.py:181: UserWarning: meta invoked without explicit configuration, will use defaults where required.
zip_safe flag not set; analyzing archive contents...
Installed /global/common/software/lsst/common/miniconda/temp/cluster_toolkit/.eggs/pytest_runner-4.2-py3.6.egg
running install
running bdist_egg
running egg_info
creating cluster_toolkit.egg-info
writing cluster_toolkit.egg-info/PKG-INFO
writing dependency_links to cluster_toolkit.egg-info/dependency_links.txt
writing requirements to cluster_toolkit.egg-info/requires.txt
writing top-level names to cluster_toolkit.egg-info/top_level.txt
writing manifest file 'cluster_toolkit.egg-info/SOURCES.txt'
reading manifest file 'cluster_toolkit.egg-info/SOURCES.txt'
writing manifest file 'cluster_toolkit.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/cluster_toolkit
copying cluster_toolkit/peak_height.py -> build/lib.linux-x86_64-3.6/cluster_toolkit
copying cluster_toolkit/averaging.py -> build/lib.linux-x86_64-3.6/cluster_toolkit
copying cluster_toolkit/massfunction.py -> build/lib.linux-x86_64-3.6/cluster_toolkit
copying cluster_toolkit/density.py -> build/lib.linux-x86_64-3.6/cluster_toolkit
copying cluster_toolkit/__init__.py -> build/lib.linux-x86_64-3.6/cluster_toolkit
copying cluster_toolkit/boostfactors.py -> build/lib.linux-x86_64-3.6/cluster_toolkit
copying cluster_toolkit/xi.py -> build/lib.linux-x86_64-3.6/cluster_toolkit
copying cluster_toolkit/concentration.py -> build/lib.linux-x86_64-3.6/cluster_toolkit
copying cluster_toolkit/bias.py -> build/lib.linux-x86_64-3.6/cluster_toolkit
copying cluster_toolkit/miscentering.py -> build/lib.linux-x86_64-3.6/cluster_toolkit
copying cluster_toolkit/deltasigma.py -> build/lib.linux-x86_64-3.6/cluster_toolkit
creating build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/C_peak_height.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/C_bias.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/C_averaging.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/C_miscentering.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/C_boostfactors.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/C_deltasigma.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/C_density.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/C_power.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/C_concentration.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/C_xi.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/C_massfunction.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
copying cluster_toolkit/include/constants.h -> build/lib.linux-x86_64-3.6/cluster_toolkit/include
running build_ext
building 'cluster_toolkit._cluster_toolkit' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/src
Traceback (most recent call last):
File "setup.py", line 35, in <module>
tests_require=['pytest']
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/site-packages/setuptools/command/install.py", line 67, in run
self.do_egg_install()
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/site-packages/setuptools/command/install.py", line 109, in do_egg_install
self.run_command('bdist_egg')
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/site-packages/setuptools/command/bdist_egg.py", line 169, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/site-packages/setuptools/command/bdist_egg.py", line 155, in call_command
self.run_command(cmdname)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/site-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/command/install_lib.py", line 107, in build
self.run_command('build_ext')
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 75, in run
_build_ext.run(self)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 185, in run
_build_ext.build_ext.run(self)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/site-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
self.build_extension(ext)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
_build_ext.build_extension(self, ext)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
depends=ext.depends)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/ccompiler.py", line 574, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
extra_postargs)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/global/common/software/lsst/common/miniconda/py3-4.3.21-env/envs/clmassmod/lib/python3.6/distutils/spawn.py", line 89, in _spawn_posix
log.info(' '.join(cmd))
TypeError: sequence item 15: expected str instance, bytes found
```
Make C variables static
Some routines can be greatly accelerated by making various variables in the C code static. For instance, spline accelerators, workspaces, integration tables, and integrand parameters.
Properly cast arrays before calling C code
This is actually two issues.
-
Arrays in Python can sometimes not be C-ordered and require a
numpy.ascontiguousarray()call. This can be handled internally in the toolkit. -
Similarly, the C code in the toolkit assumes double precision (float64). If a numpy array is floating point precision (float32) then interpolation errors can occur. This can be handled with proper casting at the same time that issue (1) is handled.
Python 3 library file mismatch
Another error specific to Python 3 when I try to import cluster_toolkit:
$ python example.py
Traceback (most recent call last):
File "example.py", line 12, in <module>
import setup_files
File "/home/msyriac/repos/redmapper-lensing/tomfit/setup_files.py", line 12, in <module>
import cluster_toolkit as ct
File "/home/msyriac/.local/lib/python3.7/site-packages/cluster_toolkit-0.0.0-py3.7-linux-x86_64.egg/cluster_toolkit/__init__.py", line 19, in <module>
_lib = _ffi.dlopen(lib_file)
File "/home/msyriac/.local/lib/python3.7/site-packages/cffi/api.py", line 141, in dlopen
lib, function_cache = _make_ffi_library(self, name, flags)
File "/home/msyriac/.local/lib/python3.7/site-packages/cffi/api.py", line 802, in _make_ffi_library
backendlib = _load_backend_lib(backend, libname, flags)
File "/home/msyriac/.local/lib/python3.7/site-packages/cffi/api.py", line 797, in _load_backend_lib
raise OSError(msg)
OSError: cannot load library '/home/msyriac/.local/lib/python3.7/site-packages/cluster_toolkit-0.0.0-py3.7-linux-x86_64.egg/cluster_toolkit/_cluster_toolkit.so': /home/msyriac/.local/lib/python3.7/site-packages/cluster_toolkit-0.0.0-py3.7-linux-x86_64.egg/cluster_toolkit/_cluster_toolkit.so: cannot open shared object file: No such file or directory. Additionally, ctypes.util.find_library() did not manage to locate a library called '/home/msyriac/.local/lib/python3.7/site-packages/cluster_toolkit-0.0.0-py3.7-linux-x86_64.egg/cluster_toolkit/_cluster_toolkit.so'
This happens because _cluster_toolkit.so actually has the filename _cluster_toolkit.cpython-37m-x86_64-linux-gnu.so. Working around by renaming for now.
Add the massfunction to the docs
The Tinker 08 mass function is implemented and in the API. I need to make a RST document for it for the RTD pages.
Installation difficulties without c99/gnu99 as default.
When I try to install cluster toolkit I am getting the following error. Is this something that is seen before?
running install
running bdist_egg
running egg_info
writing cluster_toolkit.egg-info/PKG-INFO
writing dependency_links to cluster_toolkit.egg-info/dependency_links.txt
writing requirements to cluster_toolkit.egg-info/requires.txt
writing top-level names to cluster_toolkit.egg-info/top_level.txt
reading manifest file 'cluster_toolkit.egg-info/SOURCES.txt'
writing manifest file 'cluster_toolkit.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.7-x86_64/egg
running install_lib
running build_py
running build_ext
building 'cluster_toolkit._cluster_toolkit' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/matthewkirby/anaconda3/include -arch x86_64 -I/Users/matthewkirby/anaconda3/include -arch x86_64 -Iinclude -I/Users/matthewkirby/anaconda3/include/python3.6m -c src/C_profile_derivatives.c -o build/temp.macosx-10.7-x86_64-3.6/src/C_profile_derivatives.o -I/usr/local/include
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/matthewkirby/anaconda3/include -arch x86_64 -I/Users/matthewkirby/anaconda3/include -arch x86_64 -Iinclude -I/Users/matthewkirby/anaconda3/include/python3.6m -c src/C_exclusion.c -o build/temp.macosx-10.7-x86_64-3.6/src/C_exclusion.o -I/usr/local/include
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/matthewkirby/anaconda3/include -arch x86_64 -I/Users/matthewkirby/anaconda3/include -arch x86_64 -Iinclude -I/Users/matthewkirby/anaconda3/include/python3.6m -c src/C_xi.c -o build/temp.macosx-10.7-x86_64-3.6/src/C_xi.o -I/usr/local/include
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/matthewkirby/anaconda3/include -arch x86_64 -I/Users/matthewkirby/anaconda3/include -arch x86_64 -Iinclude -I/Users/matthewkirby/anaconda3/include/python3.6m -c src/C_boostfactors.c -o build/temp.macosx-10.7-x86_64-3.6/src/C_boostfactors.o -I/usr/local/include
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/matthewkirby/anaconda3/include -arch x86_64 -I/Users/matthewkirby/anaconda3/include -arch x86_64 -Iinclude -I/Users/matthewkirby/anaconda3/include/python3.6m -c src/C_deltasigma.c -o build/temp.macosx-10.7-x86_64-3.6/src/C_deltasigma.o -I/usr/local/include
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/matthewkirby/anaconda3/include -arch x86_64 -I/Users/matthewkirby/anaconda3/include -arch x86_64 -Iinclude -I/Users/matthewkirby/anaconda3/include/python3.6m -c src/C_sigma_reconstruction.c -o build/temp.macosx-10.7-x86_64-3.6/src/C_sigma_reconstruction.o -I/usr/local/include
src/C_sigma_reconstruction.c: In function ‘Sigma_REC_from_DeltaSigma’:
src/C_sigma_reconstruction.c:38:3: error: ‘for’ loop initial declarations are only allowed in C99 mode
for(int i = 0; i < N-1; i++){
^
src/C_sigma_reconstruction.c:38:3: note: use option -std=c99 or -std=gnu99 to compile your code
src/C_sigma_reconstruction.c:43:5: error: ‘for’ loop initial declarations are only allowed in C99 mode
for(int j = i; j < N; j++){
^
error: command 'gcc' failed with exit status 1
I am working on coming up with a fix and will pass it on if I come across it. I know @cavestruz was also having this problem but I am not sure how she fixed it. If we come up with a fix that doesn't break the install for anyone else it might be worth PRing it in. Thoughts?
Finish unit tests and regression tests
Write unit tests for all modules:
- averaging
- bias
- boostfactors
- deltasigma
- density
- mass function
- miscentering
- xi
Write regression tests for all modules:
- averaging
- bias
- boostfactors
- deltasigma
- density
- mass function
- miscentering
- xi
Write python documentation
In PEP8 format.
Verify the xi_mm routine against fftlog
Install fftlog and test our algorithm against it to find the correct default settings.
error when using the user option to install
This issue is found by Marc Paterno -- only reported by yyzhang
When tying to install cluster_toolkit, here is the failure I get:
$ git clone https://github.com/tmcclintock/cluster_toolkit.git
$ cd cluster_toolkit/
$ python setup.py install --user
running install
error: can't combine user with prefix, exec_prefix/home, or install_(plat)base
Using the --user flag in order to reduce the likelihood of a collision between a system-wide installation and one's own account's stuff, and it would be nice if we can keep doing that.
Make the GSL requirement easier to deal with to prevent issues on CosmoSIS
As written, the GSL config flags aren't used in the compilation from the setup.py file. I can reproduce the code from cyflann in my setup.py file to properly use the gsl-config file. This will make the installation more stable in the future on Cosmosis.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.