libsharp / libsharp

Library for fast spherical harmonic transforms, see http://arxiv.org/abs/1303.4945

Development has moved

This repository has been archived and is only kept so that packages depending on it have a cacnonical place to download the last version.

For new projects using spherical harmonic transforms we recommend to use

• libsharp2 (https://gitlab.mpcdf.mpg.de/mtr/libsharp) if you need MPI functionality, or
• the sht component of https://gitlab.mpcdf.mpg.de/mtr/ducc if your project is written in modern C++ or Python, and you don't need MPI support within the transforms.

Both of the above libraries are successors to libsharp which are significantly faster than the last libsharp version available here. Since the switch to the new algorithms required slight changes to the API, we decided to develop them in separate repositories and keep the original libsharp repository unchanged.

Libsharp

IMPORTANT NOTE: It appears that the default branch upon cloning from github.com/dagss/libsharp was an outdated 'dagss' branch instead of the 'master' branch. To get the latest copy, please do git checkout master; git pull. New clones are no longer affected.

Paper

https://arxiv.org/abs/1303.4945

Compilation

GNU make is required for compilation.

Libsharp compilation has been successfully tested with GNU and Intel compilers. When using gcc, version 4.x is required [1]. Since libsharp was written in standard C99, other compilers should work fine, but SSE2/AVX support will most likely be deactivated.

If you obtained libsharp directly from the git repository, you will also need a copy of the GNU autotools. In this case, run "autoconf" in libsharp's main directory before any other steps. For libsharp releases distributed as a .tar.gz file, this step is not necessary.

Afterwards, simply run "./configure"; if this fails, please refer to the output of "./configure --help" for additional hints and, if necessary, provide additional flags to the configure script. Once the script finishes successfully, run "make" (or "gmake"). This should install the compilation products in the subdirectory "auto/".

Documentation can be created by the command "(g)make doc". However this requires the doxygen application to be installed on your system. The documentation will be created in the subdirectory doc/.

[1] Some versions of the gcc 4.4.x release series contain a bug which causes the compiler to crash during libsharp compilation. This appears to be fixed in the gcc 4.4.7 release. It is possible to work around this problem by adding the compiler flag "-fno-tree-fre" after the other optimization flags - the configure script should do this automatically.

libsharp's Issues

Installation on NERSC

Hi all,

I would like to get the code to run on NERSC cori as a first step towards being able to use pixell.

So far, I have been able to successfully follow the instructions provided in the README file, culminating in the creation of a libsharp/auto folder with the installation. However, trying import sharp from a python interpreter fails at this stage. To try to remedy this, I have tried python setup.py install from the libsharp/python directory, obtaining the following output and error message:

(delensing_modfsht) ab2368@cori06:~/Software/libsharp/python> python setup.py install Compiling libsharp/libsharp.pyx because it changed. Compiling libsharp/libsharp_mpi.pyx because it changed. [1/2] Cythonizing libsharp/libsharp.pyx /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /global/u1/a/ab2368/Software/libsharp/python/libsharp/libsharp.pxd tree = Parsing.p_module(s, pxd, full_module_name) [2/2] Cythonizing libsharp/libsharp_mpi.pyx /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /global/u1/a/ab2368/Software/libsharp/python/libsharp/libsharp_mpi.pyx tree = Parsing.p_module(s, pxd, full_module_name) running install running bdist_egg running egg_info creating libsharp.egg-info writing requirements to libsharp.egg-info/requires.txt writing libsharp.egg-info/PKG-INFO writing top-level names to libsharp.egg-info/top_level.txt writing dependency_links to libsharp.egg-info/dependency_links.txt writing manifest file 'libsharp.egg-info/SOURCES.txt' reading manifest file 'libsharp.egg-info/SOURCES.txt' writing manifest file 'libsharp.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/libsharp copying libsharp/__init__.py -> build/lib.linux-x86_64-2.7/libsharp creating build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_smoothing_noise_pol_mpi.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_legendre_table.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_sht.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/__init__.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_legendre.py -> build/lib.linux-x86_64-2.7/libsharp/tests running build_ext building 'libsharp.libsharp' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/libsharp gcc -pthread -B /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/global/homes/a/ab2368/Software/libsharp/auto/include -I/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include -I/global/homes/a/ab2368/.conda/envs/delensing_modfsht/include/python2.7 -c libsharp/libsharp.c -o build/temp.linux-x86_64-2.7/libsharp/libsharp.o In file included from /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1824:0, from /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12, from /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4, from libsharp/libsharp.c:630: /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp] #warning "Using deprecated NumPy API, disable it with " \ ^~~~~~~ gcc -pthread -shared -B /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat -L/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib -Wl,-rpath=/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-2.7/libsharp/libsharp.o -L/global/homes/a/ab2368/Software/libsharp/auto/lib -L/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib -lsharp -lfftpack -lc_utils -lpython2.7 -o build/lib.linux-x86_64-2.7/libsharp/libsharp.so -fopenmp /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_geomhelpers.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_core.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_legendre.o): relocation R_X86_64_32S against .rodata' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_legendre_roots.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_legendre_table.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_ylmgen_c.o): relocation R_X86_64_32 against .rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
/global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libfftpack.a(fftpack.o): relocation R_X86_64_32S against .rodata' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libc_utils.a(c_utils.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: final link failed: nonrepresentable section on output
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1`

I don't know if this is relevant information, but I made sure to be using gnu compilers before I started by entering
module unload PrgEnv-intel module load PrgEnv-gnu

Any ideas on how to solve this would be much appreciated.

Thank you very much,

Anton

Update libsharp

is it possible to update this repository to have the last version of libsharp?

Harmonic decomposition of spin 2 fields

Hi,

I am interested in harmonic decomposition of spin 2 fields (such as the combination Q+iU of Stokes parameters of microwave maps).

According to what I have seen, the function libsharp.analysis of the python wrapper outputs the E and B-modes harmonic coefficients when the Q and U maps are given as an input. Am i wrong?

If this is the case, is there a way to obtain directly the spin 2 harmonic coefficients through the libsharp python wrapper?

Thank you

Equivalent of rotate_alm

I'd like to do a distributed coordinate system transformation in alm space with libsharp, the equivalent of the HEALPix rotate_alm functionality, is there a way to do it?
If it is not exposed to Python, I can help with that

Tag/Release versions

Hi!

Thanks for the great library. We're looking at using it in our code SpECTRE and were wondering if it would be possible to have versions tagged. This would make libsharp easier to distribute in package managers like spack.

Cheers,

Nils

Normalisation of analysis/adjoint_synthesis

Hello,

I am trying to use libsharp and find the following problem:
When sampling a white noise pixel map and then using adjoint_synthesis, the resulting a_lm do not have variance 1. See my code below:

`

nside = 2
lmax = 2 * nside
alm_struct = libsharp.triangular_order(lmax)
print(alm_struct.local_size())
order = libsharp.packed_real_order(lmax)
grid = libsharp.healpix_grid(nside)
list_alms = []
list_maps = []
for i in range(10000):
    map = np.random.normal(loc=0, scale=1, size=12*nside**2)
    list_maps.append(map)
    alm_back = libsharp.adjoint_synthesis(grid, order, map[None, None, :])
    list_alms.append(alm_back[0, 0, :])

a1 =np.array(list_alms).T
print("VARIANCES")
print(np.var(np.array(list_alms), axis=0))
print("Covariance example")
print(np.cov(a1[[10,20], :]))

`
I get the following output:

VARIANCES [3.81882475 3.75802199 3.44927839 3.46005672 3.51739053 3.81658222 3.8200262 4.01142113 3.98576517 3.88529729 3.86343938 3.65896153 3.81771057 3.79704237 3.92211808 3.69347394 4.09028853 3.44506011 4.55710171 3.78255003 3.81358878 3.91577468 3.88357503 3.95017779 3.73340248] Covariance example [[3.86382576 0.04520761] [0.04520761 3.81397017]]

This is surprising: since the synthesis has othonormal rows, I expect the resulting a_lm to be distributed as a standard multivariate normal. But it seems that there is a normalization constant missing.

I also note that this normalisation constant is dependent on nside (but it seems it is not on lmax), for example:

`

nside = 4
lmax = 2 * nside

alm_struct = libsharp.triangular_order(lmax)
print(alm_struct.local_size())
order = libsharp.packed_real_order(lmax)
grid = libsharp.healpix_grid(nside)

list_alms = []
list_maps = []
for i in range(10000):
    map = np.random.normal(loc=0, scale=1, size=12*nside**2)
    list_maps.append(map)
    #alm_back = libsharp.analysis(grid, order, map[None, None, :])
    alm_back = libsharp.adjoint_synthesis(grid, order, map[None, None, :])
    list_alms.append(alm_back[0, 0, :])

a1 =np.array(list_alms).T
print(a1.shape)
print("VARIANCES")
print(np.var(np.array(list_alms), axis=0))
print("Covariance example")
print(np.cov(a1[[10,20], :]))`

`

Gives the following output:

VARIANCES [15.34317324 15.0167094 15.07927127 15.07570511 14.90184939 14.69584301 14.58601057 14.34606608 14.35603575 14.73322693 15.4021652 15.05573824 15.65920864 15.25276373 15.48269064 15.08624325 15.65170701 15.23608656 15.23706992 15.34283595 15.29359419 15.30573534 15.19867691 15.37181722 15.25184572 15.13565876 15.66069759 15.36983521 15.78268925 15.29116691 15.70773601 15.23845845 14.84480398 14.97435351 15.44235431 15.21879073 16.02206478 14.19712375 16.27293992 15.41835735 15.29608637 15.56380313 15.16174313 15.41224861 15.51884327 15.63161893 15.42497608 15.19419003 15.02857808 15.0974164 15.45329414 15.44897246 15.16990345 15.01904046 15.15164379 15.0892065 15.79061855 14.99200887 15.58967013 14.69410246 15.60494403 15.60950105 15.23071353 15.09269387 15.2101855 15.44184661 15.48837648 15.82919287 15.53281393 15.2291897 15.22795933 15.27638512 15.54393736 14.80034968 15.7228248 15.10979903 14.82560766 15.34004596 15.37910799 15.22486287 15.27545131] Covariance example [[15.40370557 0.04103769] [ 0.04103769 15.2951237 ]]

Am I missing something ? How is it possible to normalize this transformation ?
Note also that the adjoint_synthesis function seems to induce non negligible correlations between two different components...

Thank you.

Equivalent of HEALPix iter

map2alm in healpy/HEALPix accepts a iter argument, is there anything equivalent in libsharp/MPI?

Unexpected mvstart in packed_real_order

If I run this code:

lmax = 2
order = libsharp.packed_real_order(lmax, stride=1)  
order.mval()
order.mvstart()

I expect the ordering to have m=0 first, that should be 3 elements, then m=1 2 elements and finally m=2 with only ell=2.
So I expect mvstart to be: [0, 3, 5]

Instead I get:

array([0, 1, 2], dtype=int32)
array([0, 1, 3], dtype=int64)

Am I missing something?

./configure

This may seem like a silly issue, but according to the README file there should be a file called ./configure. However, I cannot locate it. I tried the ./configure.ac file however it fails on the first line do to an unexpected syntax error. I would appreciate any help pointing me to how to properly install libsharp. Thank you.

GCC-4.4 issues with Internal Compiler Error

Hi ,
You specified gcc-4xx as good in Readme.compilation.
I want just to point out that gcc-4.4 gives an internal compiler error.
However, gcc-4.6.3 is ok !

---In function 'calc_map2alm_1_1':
$HOME/local/src/libsharp-code/libsharp/sharp_core_inc2.c:222: erreur interne du compilateur: dans extract_component, à tree-complex.c:616

Cheers!

Normalization of alms

I am trying to read alms I created with healpy from disk and transform them in the format expected by libsharp.
However it seems there is a different normalization between the alm I create with healpy and with libsharp.
Investigating this issue I even find some strange behavior, could you please take a look at this notebook and check if you find any mistake/algorithmic errors?

https://nbviewer.jupyter.org/gist/zonca/a481471543cd6f658dea5bd48d0f94e9

make fails with mysterious error

Hello,

While attempting to install libsharp, I first ran the configure script with seemingly no issues:

student4@astro4:~/libsharp-master> ./configure --prefix=/home/student4/.local
configure: loading site script /usr/share/site/x86_64-unknown-linux-gnu
checking for uname... 1
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for gcc option to support OpenMP... -fopenmp
Using gcc version 8
configure: creating ./config.status
config.status: creating config/config.auto

But make then fails with a strange error:

student4@astro4:~/libsharp-master> make
using configuration 'auto'
compiling libsharp/sharp_testsuite.c
In file included from /usr/include/features.h:428,
from /usr/local/include/bits/libc-header-start.h:33,
from /usr/include/stdio.h:27,
from /data/astro4_5/student4/libsharp-master/libsharp/sharp_testsuite.c:31:
/usr/local/include/bits/math-finite.h:57:1: error: conflicting types for ‘acos_MSUF_’
__MATH_REDIRCALL (acos, , (Mdouble));
^~~~~~~~~~~~~~~~
compilation terminated due to -Wfatal-errors.
make: *** [/data/astro4_5/student4/libsharp-master/config/rules.common:19: /data/astro4_5/student4/libsharp-master/build.auto/libsharp/sharp_testsuite.o] Error 1

My gcc compiler and C libraries are up to date, and the web has been no help with this issue. I would really appreciate any assistance.

Felipe Maldonado
Florida State University

GPU support

Is there any plan to have a CUDA version of libsharp?