casacore / python-casacore

Python bindings for casacore, a library used in radio astronomy

Casacore

A suite of C++ libraries for radio astronomy data processing.

Installation

Debian / Ubuntu

Casacore is now part of Debian and Ubuntu, use apt to search for the package you need:

$ apt-get update
$ apt-get search casacore

MacOS

Use Homebrew with the ska-sa tap to install casacore:

$ brew tap ska-sa/tap
$ brew install casacore

Docker

We now also publish docker images for each github branch on quay.io:

$ docker pull quay.io/casacore/casacore:master

Building from source

Getting the source code

The casacore source code is maintained on github.

You can obtain it using:

$ git clone https://github.com/casacore/casacore

Requirements

To compile casacore you need to meet the following requirements:

• cmake
• gfortran
• g++
• flex
• bison
• blas
• lapack
• cfitsio (3.181 or later)
• wcslib (4.20 or later)
• sofa (optional, only for testing casacore measures)
• fftw3
• hdf5 (optional)
• numpy (optional)
• boost-python (optional)
• ncurses (optional)

On Debian / Ubuntu you can install these with:

$ sudo apt-get install build-essential cmake gfortran g++ libncurses5-dev \
   libreadline-dev flex bison libblas-dev liblapacke-dev libcfitsio-dev \
   wcslib-dev libfftw3-dev

and the optional libraries:

$ sudo apt-get install libhdf5-serial-dev python-numpy \
    libboost-python-dev libpython3.4-dev libpython2.7-dev

On CentOS7 you can install these with:

$ sudo yum install cmake cmake-gui gcc-gfortran gcc-c++ flex bison \
       blas blas-devel  lapack lapack-devel cfitsio cfitsio-devel \
       wcslib wcslib-devel ncurses ncurses-devel readline readline-devel\
       python-devel boost boost-devel fftw fftw-devel hdf5 hdf5-devel\
       numpy boost-python 

Obtaining measures data

Various parts of casacore require measures data, which requires regular updating. You can obtain the WSRT measures archive from the ASTRON FTP server:

ftp://ftp.astron.nl/outgoing/Measures/

Extract this somewhere on a permanent location on your filesystem.

Compilation

In the casacore source folder run:

mkdir build
cd build
cmake ..
make 
make install

there are various flags available to cmake to enable and disable options:

$ cmake -DDATA_DIR=/usr/share/casacore/data -DUSE_OPENMP=ON \
    -DUSE_HDF5=ON -DBUILD_PYTHON=ON -DUSE_THREADS=ON

The DATA_DIR should point to the location where you extracted the measures data. Special variables %CASAROOT% and %CASAHOME% can be used here, which can be set at run time through the .casarc file.

You can build python3 support using -DBUILD_PYTHON3=ON. Note that CMake may have problems detecting the correct python3 libraries and headers, so probably you need to set them manually. For example:

-DPYTHON3_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.4m.so
-DPYTHON3_INCLUDE_DIR=/usr/include/python3.4

To configure Python2 specific settings use:

PYTHON2_EXECUTABLE
PYTHON2_LIBRARY
PYTHON2_INCLUDE_DIR

To configure Python3 specific settings use:

PYTHON3_EXECUTABLE
PYTHON3_LIBRARY
PYTHON3_INCLUDE_DIR

If you run into problems with boost libraries, try setting -DBoost_NO_BOOST_CMAKE=True. This will be necessary if you have the libraries from NRAO casa in your PATH or LD_LIBRARY_PATH.

Documentation

http://casacore.github.io/casacore

Problems & bugs

If you have any issues compiling or using casacore, please open an issue on the issue tracker on github.

If you have patches please open a pull request. Your contributions are more than welcome! But to maintain a high code quality we have written a contribution manual, please read that first.

python-casacore's Issues

RuntimeError: Table DataManager error: Data Manager class LofarStMan is not registered

This occurs with both my python-casacore installs, a Fedora 22 "manual" install - using casacore-2.0.3 and python-casacore-2.1.0 - and an Ubuntu 14.04 automatic install.

[hspreeuw@localhost MS]$ python -c "import casacore.tables as pt; t = pt.table('L64371_SAP000_SB000_uv.MS')"

Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python2.7/site-packages/python_casacore-2.1.0-py2.7-linux-x86_64.egg/casacore/tables/table.py", line 311, in init
Table.init (self, tabname, lockopt, opt);
RuntimeError: Table DataManager error: Data Manager class LofarStMan is not registered

Check version of casacore during install

Python-casacore may require some recent version of casacore, as we encountered in #76 and #72.

I think it's fine to enforce such a dependency (otherwise python-casacore will never be able to wrap or use recent functionality in casacore), but we need to check during install (setup.py) whether the available casacore is recent enough. I'm not sure how to do this though..

Make test suite python3 compatible

.tofits() looses coordinates on some systems?

Continued from lofar-astron/factor#191

On some systems, calling .tofits() seems to lose the coordinates, as reported by @botteon

pyrap.tables.addImagingColumns() not compatible with lwimager

See ratt-ru/lwimager#3.

Underlying issue seems to be that lwimager seems to want a CHANNEL_SELECTION keyword in MODEL_DATA of the form {'CHANNEL_SELECTION': array([[ 0, 32]], dtype=int32)}, while addImagingColumns() adds something that omits the dtype attribute.

Workaround is to add these via lwimager. Remove the CHANNEL_SELECTION keyword using pyrap.tables, then

lwimager ms=foo.MS/ operation=empty fillmodel=1

A small typo-error in doc

In this page,

print a([0, 0.5], derivative=True)

should be

print a([0, 0.5], derivatives=True)

Build python-casacore if only some casacore packages are added

Add travis-ci automated test build

OSX travis time out due to casacore bottle install triggers source build

I've made a bottle for casacore, but when I install it brew installs casacore from source, resulting in a timeout of the build. not sure why this is happening, but I think this i not a problem with python-casacore or the travis scripts.

quanta example not working with python 2.7.6

from the example at http://python-casacore.readthedocs.org/en/latest/casacore_quanta.html

from casacore import quanta
boltzmann = quanta.constants['k']
print 'the next line fails with python 2.7.6'
print 'Boltzmann constant is ', boltzmann
print 'the next line seems to work'
print ('Boltzmann constant is ', boltzmann)

gives

the next line fails with python 2.7.6
Boltzmann constant is 0.00000 J/K
the next line seems to work
('Boltzmann constant is ', 1.38066e-23 J/K)

adjust casacore include paths in source files

Currently in the C (header) files the casacore files are included without the top-level casacore folder, this is why you need to add the casacore include path manually:

python setup.py build_ext -I/usr/include/casacore

Probably it is better to just prepend cacacore in the includes, so the compiler will find them automatically. Problem is that some header files in casacore itself don't have the cacacore folder prefix.

python3 compilations fails

building 'casacore.fitting._fitting' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/casacore -I/usr/include/python3.4m -c src/fit.cc -o build/temp.linux-x86_64-3.4/src/fit.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
In file included from src/fit.cc:31:0:
/usr/include/casacore/python/Converters/PycBasicData.h: In static member function ‘static void* casa::python::casa_string_from_python_str::convertible(PyObject*)’:
/usr/include/casacore/python/Converters/PycBasicData.h:84:34: error: ‘PyString_Check’ was not declared in this scope
       if (!PyString_Check(obj_ptr)) return 0;
                                  ^
/usr/include/casacore/python/Converters/PycBasicData.h: In static member function ‘static void casa::python::casa_string_from_python_str::construct(PyObject*, boost::python::converter::rvalue_from_python_stage1_data*)’:
/usr/include/casacore/python/Converters/PycBasicData.h:92:52: error: ‘PyString_AsString’ was not declared in this scope
       const char* value = PyString_AsString(obj_ptr);
                                                    ^
/usr/include/casacore/python/Converters/PycBasicData.h: In static member function ‘static void* casa::python::from_python_sequence<ContainerType, ConversionPolicy>::convertible(PyObject*)’:
/usr/include/casacore/python/Converters/PycBasicData.h:385:26: error: there are no arguments to ‘PyInt_Check’ that depend on a template parameter, so a declaration of ‘PyInt_Check’ must be available [-fpermissive]
    || PyInt_Check(obj_ptr)
                          ^
/usr/include/casacore/python/Converters/PycBasicData.h:385:26: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
/usr/include/casacore/python/Converters/PycBasicData.h:389:29: error: there are no arguments to ‘PyString_Check’ that depend on a template parameter, so a declaration of ‘PyString_Check’ must be available [-fpermissive]
    || PyString_Check(obj_ptr)) {
                             ^
/usr/include/casacore/python/Converters/PycBasicData.h: In static member function ‘static void casa::python::from_python_sequence<ContainerType, ConversionPolicy>::construct(PyObject*, boost::python::converter::rvalue_from_python_stage1_data*)’:
/usr/include/casacore/python/Converters/PycBasicData.h:431:26: error: there are no arguments to ‘PyInt_Check’ that depend on a template parameter, so a declaration of ‘PyInt_Check’ must be available [-fpermissive]
    || PyInt_Check(obj_ptr)
                          ^
/usr/include/casacore/python/Converters/PycBasicData.h:435:29: error: there are no arguments to ‘PyString_Check’ that depend on a template parameter, so a declaration of ‘PyString_Check’ must be available [-fpermissive]
    || PyString_Check(obj_ptr)
                             ^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

mock the compiled modules so the doc generation of readthedocs works

at the moment we auto generate the doc on readthe docs:

http://python-casacore.readthedocs.org/

Problem is that some autodoc logic does not work, since it requires the compiled python-casacore modules installed, something impossible to do on readthedocs. We can solve this by mocking the boost modules, but this is less straight forward than I was hoping.

python-casacore ephemeris information

How can I identify which ephemeris data python-casacore uses, how can I force it to use a specific version and how can I update it? This information is not covered in the python-casacore documentation.

Basically, I am having an issue with the ephemeris being out of date in casacore and I am trying to identify if this is because the ephemeris data needs updating or if the update is not working. See issue here: transientskp/tkp#542

I have installed casacore using the latest measures data from the ftp site following the instructions here: https://github.com/casacore/casacore

I then installed python-casacore using pip install python-casacore

I then ran the following code in ipython, after checking it was calling the correct python-casacore. See the instructions here: http://tkp.readthedocs.io/en/latest/getstart/casacore-measures-data.html
from casacore.measures import measures dm = measures() dm.do_frame(dm.epoch('UTC', '2015/01/01')) dm.separation(dm.direction('SUN'), dm.direction('SUN'))
But this still fails because the ephemeris is still out of date. So I want to check if this failure is because python-casacore is not looking at the correct ephemeris or if the ephemeris is out of date.

I have tried this on a virtual machine, but it is possible it is still calling an old version of the ephemeris. Unfortunately, I am having issues on my laptop so not been able to get python-casacore installed correctly.

Thanks!

intel compiler error

Hi,

i am trying to compile the master branch with the intel compiler
icc --version
icc (ICC) 15.0.3 20150407
Copyright (C) 1985-2015 Intel Corporation. All rights reserved.

Has anyone tried this successfully?

Cheers,
Stefan

Getting the following error:

icc -fno-strict-aliasing -fPIC -O3 -xHost -ftz -fp-speculation=safe -fp-model source -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/software/jureca/Stage2/software/Toolchain/intel-para/2015.07/Python/2.7.10/include/python2.7 -c src/fit.cc -o build/temp.linux-x86_64-2.7/src/fit.o
In file included from /homea/htb00/htb003/local_jureca_intel/include/casacore/python/Converters/PycArray.h(36),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/python/Converters/PycBasicData.h(34),
from src/fit.cc(31):
/homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Array.h(328): error: too few arguments for class template "vector"
vector< T > tovector() const;
^

In file included from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Vector2.tcc(32),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Vector.h(325),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/ArrayUtil.h(34),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/ArrayLogical.tcc(32),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/ArrayLogical.h(453),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/MaskedArray.tcc(33),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/MaskedArray.h(738),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Array.tcc(32),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Array.h(842),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/python/Converters/PycArray.h(36),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/python/Converters/PycBasicData.h(34),
from src/fit.cc(31):
/homea/htb00/htb003/local_jureca_intel/include/casacore/casa/stdvector.h(39): error: "vector" has already been declared in the current scope
using std::vector;
^

In file included from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Vector.h(325),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/ArrayUtil.h(34),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/ArrayLogical.tcc(32),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/ArrayLogical.h(453),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/MaskedArray.tcc(33),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/MaskedArray.h(738),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Array.tcc(32),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Array.h(842),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/python/Converters/PycArray.h(36),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/python/Converters/PycBasicData.h(34),
from src/fit.cc(31):
/homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Vector2.tcc(38): error: no instance of overloaded function "casa::Vector::Vector" matches the specified type
Vector::Vector(const vector<U, V> &other)
^

In file included from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Vector.h(325),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/ArrayUtil.h(34),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/ArrayLogical.tcc(32),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/ArrayLogical.h(453),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/MaskedArray.tcc(33),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/MaskedArray.h(738),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Array.tcc(32),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Array.h(842),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/python/Converters/PycArray.h(36),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/python/Converters/PycBasicData.h(34),
from src/fit.cc(31):
/homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Vector2.tcc(58): error: no instance of overloaded function "casa::Array::tovector" matches the specified type
void Array::tovector(vector<T, U> &out) const {
^

In file included from /homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Array.h(842),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/python/Converters/PycArray.h(36),
from /homea/htb00/htb003/local_jureca_intel/include/casacore/python/Converters/PycBasicData.h(34),
from src/fit.cc(31):
/homea/htb00/htb003/local_jureca_intel/include/casacore/casa/Arrays/Array.tcc(1057): error: no instance of overloaded function "casa::Array::tovector" matches the specified type
vector Array::tovector() const {
^

compilation aborted for src/fit.cc (code 2)
error: command 'icc' failed with exit status 2

set_coordinate() NameError

I am trying to update a spectralcoordinate within a coordinatesystem object of casacore.images.coordinates via the set_coordinate() function. However, I get a NameError, seems like a typo in the code.

I extract the spectralcoordinate from an image opened in python-casacore:

  im = images.image ('filename.image')
  imcor = im.coordinates()
  imcor.get_names()
  # Out[53]: ['stokes', 'spectral', 'direction']
  spw = imcor.get_coordinate ('spectral')
  spw.set_referencevalue (60000000)
  spw.set_restfrequency (60000000)
  imcor.set_coordinate ('spectral', spw)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-50-c37585b728be> in <module>()
----> 1 imcor.set_coordinate ('spectral', spw);

/usr/lib/python2.7/dist-packages/casacore/images/coordinates.pyc in __setitem__(self, name, val)
    100         i = self._names[::-1].index(name)
    101         assert isinstance(val, eval("%scoordinate" % name))
--> 102         self._csys[key+str(i)] = val._coord
    103
    104     # alias

python-casacore==2.1.2

add coverage reporting using travis and coveralls

https://coveralls.io

need to include pyrap.tables.tableutil

Not shipped in current package for some reason.

Consider out parameters for getter methods

In the old pyrap wrappers we have this kind of functionality

import pyrap.table as pt
table = pt.table('MS.MS')
uvw = table.getcol('UVW')

which creates a uvw ndarray array. I'd like to suggest the following functionality be available

import pyrap.table as pt
table = pt.table('MS.MS')
uvw = np.array(shape=(..),dtype=(...))
table.getcol('UVW',out=uvw)

as it can cut down on array creation, and is useful for cases where the ndarray is backed by scarce pinned (page-locked) memory. This use-case is primarily motivated by interaction with PyCUDA, where pinned memory is required for fast DMA transfers between CPU and GPU.

This is low-priority, but may be useful.

tables.tablecopy() ignores many of its arguments

tablecopy doesn't pass dminfo, endian, memorytable, copynorows in its call to table.copy()

Open table fails on OSX 10.10.2

Attempt to open a table fails with:

File "/Library/Python/2.7/site-packages/python_casacore-2.0.0-py2.7-macosx-10.10-intel.egg/casacore/tables/table.py", line 311, in __init__
    Table.__init__ (self, tabname, lockopt, opt);
Boost.Python.ArgumentError: Python argument types in
    Table.__init__(table, str, dict, int)
did not match C++ signature:
    __init__(_object*, casa::String, casa::String, casa::String, bool, casa::IPosition, casa::String, casa::String, int, int, casa::Vector<casa::String>, casa::Vector<casa::String>)
    __init__(_object*, casa::String, casa::Record, casa::String, casa::String, int, casa::Record, casa::Record)
    __init__(_object*, std::__1::vector<casa::TableProxy, std::__1::allocator<casa::TableProxy> >, casa::Vector<casa::String>, int, int, int)
    __init__(_object*, casa::Vector<casa::String>, casa::Vector<casa::String>, casa::Record, int)
    __init__(_object*, casa::String, casa::Record, int)
    __init__(_object*, casa::String, std::__1::vector<casa::TableProxy, std::__1::allocator<casa::TableProxy> >)
    __init__(_object*, casa::TableProxy)
    __init__(_object*)

pyrap images crashes when image data is float64

I get this error when the image data is float64

RuntimeError: The lattice does not appear to be an image

It goes away after after I typecast the data as float32

Remove old Converter code since we use libcasa_python now.

Opening a list of subtables with :: notation does not work

Specifying a list of subtables with :: notation for concatenation does not work.
Eg:
vtab = table(["a.ms::ANTENNA", "b.ms::ANTENNA"])
does not work.

vtab = table(["a.ms/ANTENNA", "b.ms/ANTENNA"])
works fine.

pyrap.images not compatible with pyrap.tables (on 14.04)

With the current package versions:

python -c "import pyrap.tables; tab=pyrap.tables.table('lala.MS')"

runs fine (given an MS named lala.MS), but this fails:

$ python -c "import pyrap.images; import pyrap.tables; tab=pyrap.tables.table('lala.MS')" -v
/usr/lib/python2.7/dist-packages/pyrap/tables/table.py:44: RuntimeWarning: to-Python converter for casa::String already registered; second conversion method ignored.
  from _tables import Table
/usr/lib/python2.7/dist-packages/pyrap/tables/table.py:44: RuntimeWarning: to-Python converter for casa::IPosition already registered; second conversion method ignored.

(... output skipped...)

/usr/lib/python2.7/dist-packages/pyrap/tables/table.py:44: RuntimeWarning: to-Python converter for casa::Record already registered; second conversion method ignored.
  from _tables import Table
Segmentation fault (core dumped)
$

Versions are:

$ apt-show-versions -a python-pyrap python-casacore libcasacore1 libcasacore2
libcasacore1:amd64 1.7.0-2trusty install ok installed
libcasacore1:amd64 1.7.0-2trusty trusty ppa.launchpad.net
libcasacore1:amd64/trusty 1.7.0-2trusty uptodate
libcasacore2:amd64 2.0.3-2trusty install ok installed
libcasacore2:amd64 2.0.3-2trusty trusty ppa.launchpad.net
libcasacore2:amd64/trusty 2.0.3-2trusty uptodate
python-casacore:amd64 2.0.0-3trusty install ok installed
python-casacore:amd64 2.0.0-3trusty trusty ppa.launchpad.net
python-casacore:amd64/trusty 2.0.0-3trusty uptodate
python-pyrap:amd64 1.1.0+svn358-1trusty install ok installed
python-pyrap:amd64 1.1.0+svn358-1trusty trusty ppa.launchpad.net
python-pyrap:amd64/trusty 1.1.0+svn358-1trusty uptodate

pip install python-casacore fails

I have the same issue from the sources.

In file included from /usr/local/include/casacore/fits/FITS/hdu.h:33:0,
from /usr/local/include/casacore/images/Images/FITS2Image.tcc:38,
from /usr/local/include/casacore/images/Images/FITSImage.h:329,
from src/images.cc:36:
/usr/local/include/casacore/fits/FITS/blockio.h:35:50: fatal error: fitsio.h: No such file or directory
#include <fitsio.h> //# header file from cfitsio
^
compilation terminated.
error: command 'gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python2 -u -c "import setuptools, tokenize;file='/tmp/pip-build-PnSpqY/python-casacore/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-kLJwjg-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-PnSpqY/python-casacore/

pip --version
pip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)

lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.2.1511 (Core)
Release: 7.2.1511
Codename: Core

tree /usr/local/include/casacore/fits/
/usr/local/include/casacore/fits/
├── FITS
│   ├── BasicFITS.h
│   ├── BinTable.h
│   ├── blockio.h
│   ├── CopyRecord.h
│   ├── FITS2.h
│   ├── FITS2.tcc
│   ├── FITSDateUtil.h
│   ├── FITSError.h
│   ├── FITSFieldCopier.h
│   ├── fits.h
│   ├── FITSHistoryUtil.h
│   ├── fitsio.h
│   ├── FITSKeywordUtil.h
│   ├── FITSMultiTable.h
│   ├── FITSReader.h
│   ├── FITSSpectralUtil.h
│   ├── FITSTable.h
│   ├── FITSTimedTable.h
│   ├── hdu.h
│   ├── hdu.tcc
│   └── SDFITSTable.h
└── FITS.h

1 directory, 22 files

Set restfrequency in images

In #67 , @hsuyeep wrote:

how is one expected to actually set the restfrequency in an existing spectralcoordinate type?

I don't know. A method set_restfrequency in images/coordinates.py would be nice I suppose.

Docs on readthedocs are wrong

The docs on readthedocs only contain the static .rst files, not the auto generated documentation. @gijzelaerr , how do I access the settings on readthedocs? Are you the owner of this project?

Link to casa_python3 in case of Python3

Malte is working on a python2 and python3 specific casa_python shared library. We should link to the correct version for each python.

include requirements (numpy, setuptools) in setup.py

Python 3.5/ubuntu test suit fails on travis

Probably because it is linked against system the system boost python which is linked against the system python 3.4, while travis manually installs a python 3.5.

No easy fix here I think...

python-casacore segfaults on faulty image data and calling tofits()

Sure it should not process faulty data, but it should not crash or segfault. calling .getdata() doesnt't crash.

>>> from casacore.images import image
>>> x.getdata()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-8-8afe7a814c4c> in <module>()
----> 1 x.getdata()

/home/gijs/Work/tkp/.virtualenv/local/lib/python2.7/site-packages/casacore/images/image.pyc in getdata(self, blc, trc, inc)
    296         return self._getdata (self._adjustBlc(blc),
    297                               self._adjustTrc(trc),
--> 298                               self._adjustInc(inc));
    299 
    300     # Negate the mask; in numpy True means invalid.

RuntimeError: BucketCache::getBucket: bucket 29 exceeds nr of buckets

>>> x.tofits('bla')
terminate called after throwing an instance of 'casa::AipsError'
  what():  FiledesIO /home/gijs/S301_R0-62_T02-06-2016_21-27-41.image/table.f0_TSM0is not writable
    5671 abort (core dumped)  ~/Work/tkp/.virtualenv/bin/ipython

argparse not included in python2.6

setup.py uses argparse which is not included in python2.6. We can work arround this by adding the argparse module as a dependency, which should not harm a python 2.7 or 3+ install.

add auto upload to pypi on release

can be done with travis.

pyrap.measures not correctly linked with casacore 2.2.0 in python-casacore 2.1.2

casacore 2.2.0-2kern1 installed:

sperkins@simon-Y700:~$ apt show casacore-dev
Package: casacore-dev
Version: 2.2.0-2kern1
Priority: optional
Section: libdevel
Source: casacore
Maintainer: Debian Astronomy & Astrophysics Maintainers <[email protected]>
Installed-Size: 12.9 MB
Depends: libcasa-casa2 (= 2.2.0-2kern1), libcasa-coordinates2 (= 2.2.0-2kern1), libcasa-derivedmscal2 (= 2.2.0-2kern1), libcasa-fits2 (= 2.2.0-2kern1), libcasa-images2 (= 2.2.0-2kern1), libcasa-lattices2 (= 2.2.0-2kern1), libcasa-meas2 (= 2.2.0-2kern1), libcasa-measures2 (= 2.2.0-2kern1), libcasa-mirlib2 (= 2.2.0-2kern1), libcasa-ms2 (= 2.2.0-2kern1), libcasa-msfits2 (= 2.2.0-2kern1), libcasa-python2 (= 2.2.0-2kern1), libcasa-scimath-f2 (= 2.2.0-2kern1), libcasa-scimath2 (= 2.2.0-2kern1), libcasa-tables2 (= 2.2.0-2kern1)
Download-Size: 1,522 kB
APT-Manual-Installed: yes
APT-Sources: http://ppa.launchpad.net/kernsuite/kern-dev/ubuntu xenial/main amd64 Packages
Description: CASA core library development files
 The casacore package contains the core libraries of the old
 AIPS++/CASA (Common Astronomy Software Applications) package. This
 split was made to get a better separation of core libraries and
 applications.
 .
 This package contains the files for application development.

Create and activate a virtual environment

sperkins@simon-Y700:~$ virtualenv venv/pyrap
New python executable in /home/sperkins/venv/pyrap/bin/python
Installing setuptools, pip, wheel...done.
sperkins@simon-Y700:~$ source ~/venv/pyrap/bin/activate
(pyrap) sperkins@simon-Y700:~$ pip install python-casacore
Collecting python-casacore
Collecting numpy (from python-casacore)
  Using cached numpy-1.12.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: setuptools in ./venv/pyrap/lib/python2.7/site-packages (from python-casacore)
Requirement already satisfied: appdirs>=1.4.0 in ./venv/pyrap/lib/python2.7/site-packages (from setuptools->python-casacore)
Requirement already satisfied: packaging>=16.8 in ./venv/pyrap/lib/python2.7/site-packages (from setuptools->python-casacore)
Requirement already satisfied: six>=1.6.0 in ./venv/pyrap/lib/python2.7/site-packages (from setuptools->python-casacore)
Requirement already satisfied: pyparsing in ./venv/pyrap/lib/python2.7/site-packages (from packaging>=16.8->setuptools->python-casacore)
Installing collected packages: numpy, python-casacore
Successfully installed numpy-1.12.0 python-casacore-2.1.2

Try to import pyrap.measures

(pyrap) sperkins@simon-Y700:~$ python -c 'import pyrap.measures'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/sperkins/venv/pyrap/local/lib/python2.7/site-packages/pyrap/measures.py", line 1, in <module>
    from casacore.measures import *
  File "/home/sperkins/venv/pyrap/local/lib/python2.7/site-packages/casacore/measures/__init__.py", line 30, in <module>
    import casacore.quanta as dq
  File "/home/sperkins/venv/pyrap/local/lib/python2.7/site-packages/casacore/quanta/__init__.py", line 1, in <module>
    from .quantity import quantity, is_quantity
  File "/home/sperkins/venv/pyrap/local/lib/python2.7/site-packages/casacore/quanta/quantity.py", line 2, in <module>
    from ._quanta import Quantity
ImportError: /home/sperkins/venv/pyrap/local/lib/python2.7/site-packages/casacore/quanta/_quanta.so: undefined symbol: _ZN4casa16throwArrayShapesEPKc
(pyrap) sperkins@simon-Y700:~$ 

Expose MeasurementSet functionality

It would be helpful if requiredTableDesc was exposed in the python wrappers so that access to the basic table configuration was available for creating new tables.

Had a quick go at adding it in pytables.cc but requiredTableDesc doesn't live on TableProxy (looks like its a static method on MeasurementSet).

Move docs to github

Readthedocs has troubles building python-casacore. We should probably have travis make the documentation and push it to github. Could be done with something like https://github.com/Syntaf/travis-sphinx or https://github.com/teracyhq/sphinx-deployment

Probable bug: Zero-valued DATA for large MS depending on how you read

With python-casacore 2.1.0 (see bellow for the linking)

Description: The data read at once gives a numpy array that has zeros values after a certain row. That does not happen if the data is not read at once (so it's not the data that contains zeros). This is done on a system with 256Gb of RAM, so that's enough to host the full data array.

`

get the full data as an array

d=t.getcol("DATA")
print d.shape

(27203365, 64, 4)

select subsample from full array and directly from disc

incr=2179
dsel_FromArray=d[1::incr,10,0]
dsel_FromDisk=t.getcol("DATA",1,rowincr=incr)[:,10,0]

Plot

pylab.plot(dsel_FromDisk.real)
pylab.plot(dsel_FromArray.real)
pylab.legend(("From sliced file on disk","From sliced Array"))
pylab.xlabel("Bin number")
pylab.ylabel("Real part of visibility")
pylab.show()
`

ldd /cep/lofar/python-casacore/lib/python2.7/site-packages/casacore/tables/_tables.so
linux-vdso.so.1 => (0x00007ffe4e7c1000)
libcasa_tables.so.2 => /cep/lofar/casacore/lib/libcasa_tables.so.2 (0x00007fefae12c000)
libboost_python-py27.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0 (0x00007fefadedf000)
libcasa_python.so.2 => /cep/lofar/casacore/lib/libcasa_python.so.2 (0x00007fefadbfc000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fefad8f8000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fefad6e2000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fefad31d000)
libcasa_casa.so.2 => /cep/lofar/casacore/lib/libcasa_casa.so.2 (0x00007fefacdad000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fefacb8f000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fefac889000)
libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007fefac67a000)
libpython2.7.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007fefac116000)
/lib64/ld-linux-x86-64.so.2 (0x00007fefaed13000)
libhdf5.so.7 => /usr/lib/x86_64-linux-gnu/libhdf5.so.7 (0x00007fefabc7a000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fefaba76000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fefab85d000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fefab65a000)

from casacore import tables problem

I did two installs, on Ubuntu 14.04.2 LTS - VirtualBox guest, with sudo apt-get install python-casacore etc. - and on Fedora 22, using casacore-2.0.3 and python-casacore-2.1.0:
as user I did:
python setup.py build_ext -I/usr/local/include/casacore/ -I/usr/include/cfitsio
and as root:
python setup.py install

However, the Fedora install showed problems at a later stage:
[hspreeuw@localhost ~]$ python
Python 2.7.10 (default, Sep 24 2015, 17:50:09)
[GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import casacore as casac
from casacore import tables as tbls
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python2.7/site-packages/python_casacore-2.1.0-py2.7-linux-x86_64.egg/casacore/tables/init.py", line 60, in
from table import table
File "/usr/lib64/python2.7/site-packages/python_casacore-2.1.0-py2.7-linux-x86_64.egg/casacore/tables/table.py", line 44, in
from _tables import Table
ImportError: libcasa_tables.so.2: cannot open shared object file: No such file or directory

What I did is to compare the contents fo the tables directory on Fedora and the Ubuntu VirtualBox guest to try to solve the problem, but I am not sure if I can conclude anything.
Anyway, on Ubuntu:
ls /usr/lib/python2.7/dist-packages/casacore/tables/
init.py msutil.py tablecolumn.py tablehelper.py tableindex.py tableiter.py table.py tablerow.py _tables.so tableutil.pyc wxtablebrowser.pyc
init.pyc msutil.pyc tablecolumn.pyc tablehelper.pyc tableindex.pyc tableiter.pyc table.pyc tablerow.pyc tableutil.py wxtablebrowser.py

On Fedora:
ls /usr/lib64/python2.7/site-packages/python_casacore-2.1.0-py2.7-linux-x86_64.egg/casacore/tables/
init.py msutil.py tablecolumn.py tablehelper.py tableindex.py tableiter.py table.py tablerow.py _tables.py _tables.so tableutil.pyc wxtablebrowser.pyc
init.pyc msutil.pyc tablecolumn.pyc tablehelper.pyc tableindex.pyc tableiter.pyc table.pyc tablerow.pyc _tables.pyc tableutil.py wxtablebrowser.py

On Fedora, there are two extra files, _tables.py (underscoretables.py) and _tables.pyc (underscoretables.pyc), which are not made in the Ubuntu install.
This is _tables.py:
def bootstrap():
global bootstrap, loader, file
import sys, pkg_resources, imp
file = pkg_resources.resource_filename(name, '_tables.so')
loader = None; del bootstrap, loader
imp.load_dynamic(name,file)
bootstrap()

I guess this is probably not the cause of the problem, but I wanted to mention it.

There is a similar problem when importing the images module and possibly also when importing other casacore modules.

What is an elegant way to fix this?

Update Pypi with new README.rst

since it looks like 💩 at the moment on pypi

https://pypi.python.org/pypi/python-casacore/2.1.2

Test python3 in travis

As mentioned by @brentjens in #31, we should test python3 in travis. This needs python3 functionality in libcasacore2-dev package in ppa:radio-astro/main. I'm not sure that is the case yet, since the python3 stuff is currently only in casacore master.

setup.py doesn't detect changes in casacore

The interface of one of the functions in libcasa_casa had changed (from 'casa::ArrayBase::baseReform(casa::ArrayBase&, casa::IPosition const&) const' to
'casa::ArrayBase::baseReform(casa::ArrayBase&, casa::IPosition const&, bool) const'), but setup.py did not force a recompile of the sources using this changed file; it bluntly copied the so-files from the cache ('build/lib.linux-x86_64-2.7') into the install directory. As a result I got the following error when trying to import a pyrap module:

$ python -c 'import pyrap.tables'
Traceback (most recent call last):
File "", line 1, in
File "pyrap/tables.py", line 1, in
from casacore.tables import *
File "casacore/tables/init.py", line 60, in
from table import table
File "casacore/tables/table.py", line 44, in
from _tables import Table
ImportError: casacore/tables/_tables.so: undefined symbol: _ZNK4casa9ArrayBase10baseReformERS0_RKNS_9IPositionE

The workaround is to ditch the build directory every time, but that's a bit of a pity, because most of the time a recompile is not needed.

set src folder as default include path

at the moment you need to manually set it:

python setup.py build_ext -Isrc

Build within anaconda on OSX 10.11

I solved this issue, but will record it here in the hope that it will save someone else the hours that I wasted on it.
I wanted to run python-casacore within an ipython on OSX, and chose to use Anaconda for ipython (instead of for example brew). Anaconda is a flexible system for switching python environments.

To install python-casacore in a conda package, I used (with a lot of stuff that is only necessary because I installed casacore in a non-standard location):

pip install ~/opt/python-casacore/src/ --global-option=build_ext --global-option="-L/Users/dijkema/opt/casacore//lib" --global-option="-I/Users/dijkema/opt/casacore/include:/Users/dijkema/anaconda/envs/itaql/include/python2.7" --global-option="-R/Users/dijkema/opt/casacore/lib"

This failed on an error

error: no template named 'shared_ptr' in namespace 'std'

while funnily the same command from outside the conda environment did succeed.

The solution was to explicitly use libc++:

CFLAGS="-stdlib=libc++ -mmacosx-version-min=10.8" LINKFLAGS="-mmacosx-version-min=10.8 -stdlib=c++" pip install ~/opt/python-casacore/src/ --global-option=build_ext --global-option="-L/Users/dijkema/opt/casacore//lib" --global-option="-I/Users/dijkema/opt/casacore/include:/Users/dijkema/anaconda/envs/itaql/include/python2.7" --global-option="-R/Users/dijkema/opt/casacore/lib"

Add backwards compatability

we've moved the python code to the 'casacore' namespace. Code that still uses the pyrap namespace should work without requiring modification.

error in casacore/tables/tablehelper.py +57

/usr/lib/python2.7/dist-packages/casacore/tables/tablehelper.pyc in _check_index(key)
     55         return key.__index__()
     56     except:
---> 57         raise TypeError(name + " indices must be integer (or None in a slice");
     58 
     59 # Check a key or slice given to index a tablerow or tablecolumn object.

NameError: global name 'name' is not defined

Autocompile doc on readthedocs

Add some more tests to have at least 70% coverage

Compilation fails with casacore 2.1.0

src/quantamath.cc:72:35: error: member reference base type 'void' is not a structure or union
        unitdesc.append(i->second).getFullName();
        ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~

Tests Failing

Tests fail when tested with master branch of casacore.

Full trace:

======================================================================
ERROR: test_image (test_image.TestImage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/code/python-casacore/tests/test_image.py", line 14, in test_image
    im = image("", shape=[4, 3])
  File "/usr/local/lib/python2.7/dist-packages/python_casacore-2.1.0-py2.7-linux-x86_64.egg/casacore/images/image.py", line 178, in __init__
    maskname, tileshape, 0)
RuntimeError: IPosition::operator=(const IPosition&  - this and other differ in length

----------------------------------------------------------------------