gplepage / gvar Goto Github PK
View Code? Open in Web Editor NEWGaussian random variables.
License: Other
gvar's People
Contributors
Stargazers
Watchers
Forkers
daodaoliang matthew-wingate dchackett christwell jxi24 gattocrucco evanberkowitz ckoerber saintbenjamin apn-puckygvar's Issues
gvar does not compile with Python 3.10.1
When running sudo -H pip3 install gvar
I get:
Collecting gvar
Using cached gvar-11.9.4.tar.gz (1.9 MB)
Requirement already satisfied: numpy>=1.16 in /usr/lib/python3.10/site-packages (from gvar) (1.21.4)
Requirement already satisfied: scipy in /usr/lib/python3.10/site-packages (from gvar) (1.7.3)
Building wheels for collected packages: gvar
Building wheel for gvar (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-u82266z_/gvar_390d8f6e6d7440fba36a19ca093b9404/setup.py'"'"'; __file__='"'"'/tmp/pip-install-u82266z_/gvar_390d8f6e6d7440fba36a19ca093b9404/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-bvg1r94j
cwd: /tmp/pip-install-u82266z_/gvar_390d8f6e6d7440fba36a19ca093b9404/
Complete output (207 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.10
creating build/lib.linux-x86_64-3.10/gvar
copying src/gvar/root.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/powerseries.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/pade.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/ode.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/linalg.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/cspline.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/__init__.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/../gvar.pxd -> build/lib.linux-x86_64-3.10/gvar/..
copying src/gvar/_svec_smat.pxd -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/_gvarcore.pxd -> build/lib.linux-x86_64-3.10/gvar
running build_ext
building 'gvar._gvarcore' extension
creating build/temp.linux-x86_64-3.10
creating build/temp.linux-x86_64-3.10/src
creating build/temp.linux-x86_64-3.10/src/gvar
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fPIC -I/usr/lib/python3.10/site-packages/numpy/core/include -I/usr/include/python3.10 -c src/gvar/_gvarcore.c -o build/temp.linux-x86_64-3.10/src/gvar/_gvarcore.o
In file included from /usr/lib/python3.10/site-packages/numpy/core/include/numpy/ndarraytypes.h:1969,
from /usr/lib/python3.10/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /usr/lib/python3.10/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from src/gvar/_gvarcore.c:598:
/usr/lib/python3.10/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with " \
| ^~~~~~~
src/gvar/_gvarcore.c: In function ‘__pyx_tp_dealloc_array’:
src/gvar/_gvarcore.c:38164:5: error: lvalue required as increment operand
38164 | ++Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c:38166:5: error: lvalue required as decrement operand
38166 | --Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c: In function ‘__pyx_tp_dealloc_memoryview’:
src/gvar/_gvarcore.c:38459:5: error: lvalue required as increment operand
38459 | ++Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c:38461:5: error: lvalue required as decrement operand
38461 | --Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c: In function ‘__pyx_tp_dealloc__memoryviewslice’:
src/gvar/_gvarcore.c:38701:5: error: lvalue required as increment operand
38701 | ++Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c:38703:5: error: lvalue required as decrement operand
38703 | --Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c: In function ‘__Pyx_ParseOptionalKeywords’:
src/gvar/_gvarcore.c:42051:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42051:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42051:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42051:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42051:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42051:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c: In function ‘__Pyx_decode_c_string’:
src/gvar/_gvarcore.c:45764:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
45764 | return PyUnicode_FromUnicode(NULL, 0);
| ^~~~~~
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here
551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
| ^~~~~~~~~~~~~~~~~~~~~
At top level:
src/gvar/_gvarcore.c:34319:20: warning: ‘__pyx_mdef___pyx_memoryviewslice_3__setstate_cython__’ defined but not used [-Wunused-variable]
34319 | static PyMethodDef __pyx_mdef___pyx_memoryviewslice_3__setstate_cython__ = {"__setstate_cython__", (PyCFunction)__pyx_pw___pyx_memoryviewslice_3__setstate_cython__, METH_O, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:34264:20: warning: ‘__pyx_mdef___pyx_memoryviewslice_1__reduce_cython__’ defined but not used [-Wunused-variable]
34264 | static PyMethodDef __pyx_mdef___pyx_memoryviewslice_1__reduce_cython__ = {"__reduce_cython__", (PyCFunction)__pyx_pw___pyx_memoryviewslice_1__reduce_cython__, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31449:20: warning: ‘__pyx_mdef___pyx_memoryview_3__setstate_cython__’ defined but not used [-Wunused-variable]
31449 | static PyMethodDef __pyx_mdef___pyx_memoryview_3__setstate_cython__ = {"__setstate_cython__", (PyCFunction)__pyx_pw___pyx_memoryview_3__setstate_cython__, METH_O, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31394:20: warning: ‘__pyx_mdef___pyx_memoryview_1__reduce_cython__’ defined but not used [-Wunused-variable]
31394 | static PyMethodDef __pyx_mdef___pyx_memoryview_1__reduce_cython__ = {"__reduce_cython__", (PyCFunction)__pyx_pw___pyx_memoryview_1__reduce_cython__, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31303:20: warning: ‘__pyx_mdef_15View_dot_MemoryView_10memoryview_23copy_fortran’ defined but not used [-Wunused-variable]
31303 | static PyMethodDef __pyx_mdef_15View_dot_MemoryView_10memoryview_23copy_fortran = {"copy_fortran", (PyCFunction)__pyx_memoryview_copy_fortran, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31211:20: warning: ‘__pyx_mdef_15View_dot_MemoryView_10memoryview_21copy’ defined but not used [-Wunused-variable]
31211 | static PyMethodDef __pyx_mdef_15View_dot_MemoryView_10memoryview_21copy = {"copy", (PyCFunction)__pyx_memoryview_copy, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31139:20: warning: ‘__pyx_mdef_15View_dot_MemoryView_10memoryview_19is_f_contig’ defined but not used [-Wunused-variable]
31139 | static PyMethodDef __pyx_mdef_15View_dot_MemoryView_10memoryview_19is_f_contig = {"is_f_contig", (PyCFunction)__pyx_memoryview_is_f_contig, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31067:20: warning: ‘__pyx_mdef_15View_dot_MemoryView_10memoryview_17is_c_contig’ defined but not used [-Wunused-variable]
31067 | static PyMethodDef __pyx_mdef_15View_dot_MemoryView_10memoryview_17is_c_contig = {"is_c_contig", (PyCFunction)__pyx_memoryview_is_c_contig, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:27269:20: warning: ‘__pyx_mdef___pyx_MemviewEnum_3__setstate_cython__’ defined but not used [-Wunused-variable]
27269 | static PyMethodDef __pyx_mdef___pyx_MemviewEnum_3__setstate_cython__ = {"__setstate_cython__", (PyCFunction)__pyx_pw___pyx_MemviewEnum_3__setstate_cython__, METH_O, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:27036:20: warning: ‘__pyx_mdef___pyx_MemviewEnum_1__reduce_cython__’ defined but not used [-Wunused-variable]
27036 | static PyMethodDef __pyx_mdef___pyx_MemviewEnum_1__reduce_cython__ = {"__reduce_cython__", (PyCFunction)__pyx_pw___pyx_MemviewEnum_1__reduce_cython__, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:26667:20: warning: ‘__pyx_mdef___pyx_array_3__setstate_cython__’ defined but not used [-Wunused-variable]
26667 | static PyMethodDef __pyx_mdef___pyx_array_3__setstate_cython__ = {"__setstate_cython__", (PyCFunction)__pyx_pw___pyx_array_3__setstate_cython__, METH_O, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:26612:20: warning: ‘__pyx_mdef___pyx_array_1__reduce_cython__’ defined but not used [-Wunused-variable]
26612 | static PyMethodDef __pyx_mdef___pyx_array_1__reduce_cython__ = {"__reduce_cython__", (PyCFunction)__pyx_pw___pyx_array_1__reduce_cython__, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1
----------------------------------------
ERROR: Failed building wheel for gvar
Running setup.py clean for gvar
Failed to build gvar
Installing collected packages: gvar
Running setup.py install for gvar ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-u82266z_/gvar_390d8f6e6d7440fba36a19ca093b9404/setup.py'"'"'; __file__='"'"'/tmp/pip-install-u82266z_/gvar_390d8f6e6d7440fba36a19ca093b9404/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-2fgwewqc/install-record.txt --single-version-externally-managed --compile --install-headers /usr/include/python3.10/gvar
cwd: /tmp/pip-install-u82266z_/gvar_390d8f6e6d7440fba36a19ca093b9404/
Complete output (207 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.10
creating build/lib.linux-x86_64-3.10/gvar
copying src/gvar/root.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/powerseries.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/pade.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/ode.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/linalg.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/cspline.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/__init__.py -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/../gvar.pxd -> build/lib.linux-x86_64-3.10/gvar/..
copying src/gvar/_svec_smat.pxd -> build/lib.linux-x86_64-3.10/gvar
copying src/gvar/_gvarcore.pxd -> build/lib.linux-x86_64-3.10/gvar
running build_ext
building 'gvar._gvarcore' extension
creating build/temp.linux-x86_64-3.10
creating build/temp.linux-x86_64-3.10/src
creating build/temp.linux-x86_64-3.10/src/gvar
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fPIC -I/usr/lib/python3.10/site-packages/numpy/core/include -I/usr/include/python3.10 -c src/gvar/_gvarcore.c -o build/temp.linux-x86_64-3.10/src/gvar/_gvarcore.o
In file included from /usr/lib/python3.10/site-packages/numpy/core/include/numpy/ndarraytypes.h:1969,
from /usr/lib/python3.10/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /usr/lib/python3.10/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from src/gvar/_gvarcore.c:598:
/usr/lib/python3.10/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with " \
| ^~~~~~~
src/gvar/_gvarcore.c: In function ‘__pyx_tp_dealloc_array’:
src/gvar/_gvarcore.c:38164:5: error: lvalue required as increment operand
38164 | ++Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c:38166:5: error: lvalue required as decrement operand
38166 | --Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c: In function ‘__pyx_tp_dealloc_memoryview’:
src/gvar/_gvarcore.c:38459:5: error: lvalue required as increment operand
38459 | ++Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c:38461:5: error: lvalue required as decrement operand
38461 | --Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c: In function ‘__pyx_tp_dealloc__memoryviewslice’:
src/gvar/_gvarcore.c:38701:5: error: lvalue required as increment operand
38701 | ++Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c:38703:5: error: lvalue required as decrement operand
38703 | --Py_REFCNT(o);
| ^~
src/gvar/_gvarcore.c: In function ‘__Pyx_ParseOptionalKeywords’:
src/gvar/_gvarcore.c:42051:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42051:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42051:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42051:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42051:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42051:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42051 | (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
| ^~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:42067:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
42067 | (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
| ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c: In function ‘__Pyx_decode_c_string’:
src/gvar/_gvarcore.c:45764:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
45764 | return PyUnicode_FromUnicode(NULL, 0);
| ^~~~~~
In file included from /usr/include/python3.10/unicodeobject.h:1046,
from /usr/include/python3.10/Python.h:83,
from src/gvar/_gvarcore.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here
551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
| ^~~~~~~~~~~~~~~~~~~~~
At top level:
src/gvar/_gvarcore.c:34319:20: warning: ‘__pyx_mdef___pyx_memoryviewslice_3__setstate_cython__’ defined but not used [-Wunused-variable]
34319 | static PyMethodDef __pyx_mdef___pyx_memoryviewslice_3__setstate_cython__ = {"__setstate_cython__", (PyCFunction)__pyx_pw___pyx_memoryviewslice_3__setstate_cython__, METH_O, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:34264:20: warning: ‘__pyx_mdef___pyx_memoryviewslice_1__reduce_cython__’ defined but not used [-Wunused-variable]
34264 | static PyMethodDef __pyx_mdef___pyx_memoryviewslice_1__reduce_cython__ = {"__reduce_cython__", (PyCFunction)__pyx_pw___pyx_memoryviewslice_1__reduce_cython__, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31449:20: warning: ‘__pyx_mdef___pyx_memoryview_3__setstate_cython__’ defined but not used [-Wunused-variable]
31449 | static PyMethodDef __pyx_mdef___pyx_memoryview_3__setstate_cython__ = {"__setstate_cython__", (PyCFunction)__pyx_pw___pyx_memoryview_3__setstate_cython__, METH_O, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31394:20: warning: ‘__pyx_mdef___pyx_memoryview_1__reduce_cython__’ defined but not used [-Wunused-variable]
31394 | static PyMethodDef __pyx_mdef___pyx_memoryview_1__reduce_cython__ = {"__reduce_cython__", (PyCFunction)__pyx_pw___pyx_memoryview_1__reduce_cython__, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31303:20: warning: ‘__pyx_mdef_15View_dot_MemoryView_10memoryview_23copy_fortran’ defined but not used [-Wunused-variable]
31303 | static PyMethodDef __pyx_mdef_15View_dot_MemoryView_10memoryview_23copy_fortran = {"copy_fortran", (PyCFunction)__pyx_memoryview_copy_fortran, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31211:20: warning: ‘__pyx_mdef_15View_dot_MemoryView_10memoryview_21copy’ defined but not used [-Wunused-variable]
31211 | static PyMethodDef __pyx_mdef_15View_dot_MemoryView_10memoryview_21copy = {"copy", (PyCFunction)__pyx_memoryview_copy, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31139:20: warning: ‘__pyx_mdef_15View_dot_MemoryView_10memoryview_19is_f_contig’ defined but not used [-Wunused-variable]
31139 | static PyMethodDef __pyx_mdef_15View_dot_MemoryView_10memoryview_19is_f_contig = {"is_f_contig", (PyCFunction)__pyx_memoryview_is_f_contig, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:31067:20: warning: ‘__pyx_mdef_15View_dot_MemoryView_10memoryview_17is_c_contig’ defined but not used [-Wunused-variable]
31067 | static PyMethodDef __pyx_mdef_15View_dot_MemoryView_10memoryview_17is_c_contig = {"is_c_contig", (PyCFunction)__pyx_memoryview_is_c_contig, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:27269:20: warning: ‘__pyx_mdef___pyx_MemviewEnum_3__setstate_cython__’ defined but not used [-Wunused-variable]
27269 | static PyMethodDef __pyx_mdef___pyx_MemviewEnum_3__setstate_cython__ = {"__setstate_cython__", (PyCFunction)__pyx_pw___pyx_MemviewEnum_3__setstate_cython__, METH_O, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:27036:20: warning: ‘__pyx_mdef___pyx_MemviewEnum_1__reduce_cython__’ defined but not used [-Wunused-variable]
27036 | static PyMethodDef __pyx_mdef___pyx_MemviewEnum_1__reduce_cython__ = {"__reduce_cython__", (PyCFunction)__pyx_pw___pyx_MemviewEnum_1__reduce_cython__, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:26667:20: warning: ‘__pyx_mdef___pyx_array_3__setstate_cython__’ defined but not used [-Wunused-variable]
26667 | static PyMethodDef __pyx_mdef___pyx_array_3__setstate_cython__ = {"__setstate_cython__", (PyCFunction)__pyx_pw___pyx_array_3__setstate_cython__, METH_O, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/gvar/_gvarcore.c:26612:20: warning: ‘__pyx_mdef___pyx_array_1__reduce_cython__’ defined but not used [-Wunused-variable]
26612 | static PyMethodDef __pyx_mdef___pyx_array_1__reduce_cython__ = {"__reduce_cython__", (PyCFunction)__pyx_pw___pyx_array_1__reduce_cython__, METH_NOARGS, 0};
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-u82266z_/gvar_390d8f6e6d7440fba36a19ca093b9404/setup.py'"'"'; __file__='"'"'/tmp/pip-install-u82266z_/gvar_390d8f6e6d7440fba36a19ca093b9404/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-2fgwewqc/install-record.txt --single-version-externally-managed --compile --install-headers /usr/include/python3.10/gvar Check the logs for full command output.
Enhancement: prettier `BufferDict` formatting
When I look interactively at BufferDict objects in the shell, they are a bit illegible if they contain long arrays. So I wrote these improved versions of __repr__ and __str__ that print each item on a different line if any element has a multiline representation:
import gvar
import numpy as np
def newrepr(self):
out = self.__class__.__name__ + '(['
listrepr = [(repr(k), repr(v)) for k, v in self.items()]
newlinemode = any('\n' in rv for _, rv in listrepr)
for rk, rv in listrepr:
if not newlinemode:
out += '({}, {}), '.format(rk, rv)
elif '\n' in rv:
out += '\n ({},\n{}),'.format(rk, rv)
else:
out += '\n ({}, {}),'.format(rk, rv)
if out.endswith(', '):
out = out[:-2]
elif newlinemode:
out += '\n'
out += '])'
return out
def newstr(self):
out = '{'
listrepr = [(repr(k), repr(v)) for k, v in self.items()]
newlinemode = any('\n' in rv for _, rv in listrepr)
for rk, rv in listrepr:
if not newlinemode:
out += '{}: {}, '.format(rk, rv)
elif '\n' in rv:
out += '\n {}:\n{},'.format(rk, rv)
else:
out += '\n {}: {},'.format(rk, rv)
if out.endswith(', '):
out = out[:-2]
elif newlinemode:
out += '\n'
out += '}'
return out
ds = [
gvar.BufferDict(),
gvar.BufferDict(a=1, b=2, c=3),
gvar.BufferDict(a=1, b=[2,3], c=np.ones(40)),
]
print('------- OLD PRINT -------')
for d in ds:
print()
print(repr(d))
print()
print(str(d))
print('\n------- NEW PRINT -------')
for d in ds:
print()
print(newrepr(d))
print()
print(newstr(d))------- OLD PRINT -------
BufferDict([])
{
BufferDict([('a', 1), ('b', 2), ('c', 3)])
{'a': 1,'b': 2,'c': 3}
BufferDict([('a', 1.0), ('b', array([2., 3.])), ('c', array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1.]))])
{'a': 1.0,'b': array([2., 3.]),'c': array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1.])}
------- NEW PRINT -------
BufferDict([])
{}
BufferDict([('a', 1), ('b', 2), ('c', 3)])
{'a': 1, 'b': 2, 'c': 3}
BufferDict([
('a', 1.0),
('b', array([2., 3.])),
('c',
array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1.])),
])
{
'a': 1.0,
'b': array([2., 3.]),
'c':
array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1.]),
}
Could you include them in gvar? Maybe they would break some test that compares textual output, but the current ones are quite inconvenient (and bugged for empty BufferDicts).
Bug: inconsistent shape in `evalcov` and `evalcorr` for scalar arguments
With scalar arguments, evalcov produces a 1x1 matrix instead of a 0d-array or a scalar. This is unexpected in particular when I keep all my parameters in a dictionary of scalars.
import gvar
print(gvar.evalcov(gvar.gvar(0, 1)), '--> should be 1. or array(1.)')
print(gvar.evalcov(dict(x=gvar.gvar(0, 1))), "--> should be {('x', 'x'): 1.}")[[1.]] --> should be 1. or array(1.)
{('x', 'x'): array([[1.]])} --> should be {('x', 'x'): 1.}
Unable to build wheel using python3.5, missing numpy dependency
To reproduce:
virtualenv --python=python3.5 .venv
.venv/bin/pip wheel gvar
Note: installation via pip install gvar works, it's the wheel building the fails.
Output:
Collecting gvar
Using cached https://files.pythonhosted.org/packages/16/00/d8790d2a944e52b793c26299dd1ed3f5c7940aba58693dc75b2aa44b1d1f/gvar-9.0.3.tar.gz
Collecting numpy>=1.7 (from gvar)
File was already downloaded /home/velezj/sandbox/vegas_bugreport/numpy-1.16.0-cp35-cp35m-manylinux1_x86_64.whl
Skipping numpy, due to already being wheel.
Building wheels for collected packages: gvar
Running setup.py bdist_wheel for gvar ... error
Complete output from command /home/velezj/sandbox/vegas_bugreport/.venv/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/tmp/pip-wheel-6q168552/gvar/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-zbx13b8d:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.5
creating build/lib.linux-x86_64-3.5/gvar
copying src/gvar/linalg.py -> build/lib.linux-x86_64-3.5/gvar
copying src/gvar/cspline.py -> build/lib.linux-x86_64-3.5/gvar
copying src/gvar/ode.py -> build/lib.linux-x86_64-3.5/gvar
copying src/gvar/root.py -> build/lib.linux-x86_64-3.5/gvar
copying src/gvar/__init__.py -> build/lib.linux-x86_64-3.5/gvar
copying src/gvar/powerseries.py -> build/lib.linux-x86_64-3.5/gvar
copying src/gvar/../gvar.pxd -> build/lib.linux-x86_64-3.5/gvar/..
copying src/gvar/_svec_smat.pxd -> build/lib.linux-x86_64-3.5/gvar
copying src/gvar/_gvarcore.pxd -> build/lib.linux-x86_64-3.5/gvar
running build_ext
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-wheel-6q168552/gvar/setup.py", line 113, in <module>
'Topic :: Scientific/Engineering'
File "/usr/local/lib/python3.5/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/lib/python3.5/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/home/velezj/sandbox/vegas_bugreport/.venv/lib/python3.5/site-packages/wheel/bdist_wheel.py", line 188, in run
self.run_command('build')
File "/usr/local/lib/python3.5/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.5/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/local/lib/python3.5/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.5/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/tmp/pip-wheel-6q168552/gvar/setup.py", line 31, in build_extensions
import numpy
ImportError: No module named 'numpy'
----------------------------------------
Failed building wheel for gvar
Running setup.py clean for gvar
Failed to build gvar
ERROR: Failed to build one or more wheels
var(None) segfaults
To reproduce failure:
import gvar as gv
gv.var(None)
# also: gv.var([None])
This segfaults and kills the kernel (tested in both macOS and Ubuntu). Issue present with gvar 11.8, numpy 1.19.4, Python 3.7.6, Cython 0.29.21; seems to be new in gvar 11.8 vs gvar 11.7.
Bug: empty arrays error in `var`, `sdev` and `evalcorr`
import gvar
gvar.mean([])
gvar.evalcov([])
gvar.evalcov_blocks([])
# all okprint(repr(gvar.var([])))array(None, dtype=object)
which causes
gvar.sdev([])File _utilities.pyx:212, in gvar._utilities.sdev()
TypeError: unsupported operand type(s) for ** or pow(): 'NoneType' and 'float'
gvar.evalcorr([])File _utilities.pyx:456, in gvar._utilities.evalcorr()
IndexError: index 0 is out of bounds for axis 1 with size 0
It is a bit weird to use these functions on empty arrays, however it happened to me while emptying an array as a quick check to debug a fit.
Feature request: clear history of `GVarFactory` objects
I'm writing a script that repeats the same fit hundredths of times. Each time it generates about 1000 correlated primary gvars, so the memory usage is substantial. Moreover, I'd like to be able to repeatedly run the script in the same shell, as I usually do in development.
I thought about using switch_gvar at the beginning of the loop, but I discovered it keeps a full history of previous GVarFactorys, so the memory grows even if I dispose of all the gvars.
Could you add an option to not keep a full history? Something like switch_gvar(keep_history=False), or a new function clear_gvar(). Currently, I do gvar._GVAR_LIST.clear().
Example script showing the memory usage:
import tracemalloc
import gvar
import numpy as np
n = 1000
r = 10
a = np.random.randn(n, n)
cov = a @ a.T
mean = np.random.randn(n)
class TraceMalloc:
def __enter__(self):
tracemalloc.start()
def __exit__(self, *_):
tracemalloc.stop()
tracemalloc.clear_traces()
with TraceMalloc():
initial_memory, _ = tracemalloc.get_traced_memory()
for _ in range(r):
gvar.switch_gvar()
gvar.gvar(mean, cov)
current_memory, _ = tracemalloc.get_traced_memory()
memory = current_memory - initial_memory
print(f'memory = {memory / 1e6:.0f} MB')
with TraceMalloc():
initial_memory, _ = tracemalloc.get_traced_memory()
print('\nWITH CLEARING:')
for _ in range(r):
gvar.switch_gvar()
gvar._GVAR_LIST.clear()
gvar.gvar(mean, cov)
current_memory, _ = tracemalloc.get_traced_memory()
memory = current_memory - initial_memory
print(f'memory = {memory / 1e6:.0f} MB')memory = 16 MB
memory = 32 MB
memory = 48 MB
memory = 64 MB
memory = 80 MB
memory = 96 MB
memory = 113 MB
memory = 129 MB
memory = 145 MB
memory = 161 MB
WITH CLEARING:
memory = 16 MB
memory = 16 MB
memory = 16 MB
memory = 16 MB
memory = 16 MB
memory = 16 MB
memory = 16 MB
memory = 16 MB
memory = 16 MB
memory = 16 MB
gvar version: 11.11.10
np.array of gvars is not np.einsummable
np.einsum is a convenient way to do contractions of tensor of data.
While writing this example cross-correlation fitter I tried to use einsum rather than the quadruply-nested for loop in fcn. The issue can be shown with a simpler example
einsum-issue.py:
#!/usr/bin/env python
import numpy as np
import gvar as gv
x = 2
n = np.array([[i+j for i in range(x)] for j in range(x)])
g = np.array([[gv.gvar(f'{i+j}({(i+j)/2})') for i in range(x)] for j in range(x)])
print(n)
print(g)
nn = np.einsum('ij,jk->ik',n,n)
print(nn)
ng = np.einsum('ij,jk->ik',n,g)
gn = np.einsum('ij,jk->ik',g,n)
gg = np.einsum('ij,jk->ik',g,g)which outputs
[[0 1]
[1 2]]
[[0(0) 1.00(50)]
[1.00(50) 2.0(1.0)]]
[[1 2]
[2 5]]
Traceback (most recent call last):
File "./einsum-issue.py", line 18, in <module>
ng = np.einsum('ij,jk->ik',n,g)
File "<__array_function__ internals>", line 5, in einsum
File "/path/to/python3.8/site-packages/numpy/core/einsumfunc.py", line 1361, in einsum
return c_einsum(*operands, **kwargs)
TypeError: invalid data type for einsum
where we see the dtype=float np.array gave the expected matrix-matrix multiplication. All the einsums which try to act on g fail with the same TypeError.
Is it possible to use np.einsum on combinations of arrays of floats and gvars? What would need to be done to unlock this possibility?
`gvar.dataset.avg_data()` didn't give the unbiased estimator of the variance of average
In the function gvar.dateset.avg_data(), the uncertainty of the estimator of the mean
$$ \sigma=\sqrt{\frac{1}{n^2}\sum_{i=1}^n(x_i-\bar{x})^2} $$
The coefficients in the calculating the whole correlation matrix are also
However, the unbiased standard variance of the average
$$\sigma=\sqrt{\frac{1}{n(n-1)}\sum_{i=1}^n(x_i-\bar{x})^2}$$
This disagreement caused me a little confusion.
Missing signature in docstrings for cython-compiled functions
Example:
import gvar
help(gvar.evalcov_blocks)Output:
evalcov_blocks(...)
Evaluate covariance matrix for elements of ``g``.
Evaluates the covariance matrices for |GVar|\s stored in
array or dictionary of arrays/|GVar|\s ``g``. The covariance matrix is
[...]
If I remember correctly, in numpy this is solved by copy-pasting the signature in the docstring for all compiled functions. May I do this and make a pull request? It should be quick.
Moving dataset.pyx to dataset.py for better autocompletion in editors or IDEs.
It seems that nothing is defined as a c function or c class in dataset.pyx, so I thought we can make it a normal python file.
I renamed dataset.pyx to dataset.py, commented line 53 and 55, remove int type for binsize in line 563, chang the latter is to == in line 621, and commented line 67 in setup.py, then gvar.dataset seems to work fine.
There might be some extra things I didn't check, and more changes should be applied to the file to make it more pythonic.
Inconsistent treatment of distribution names with parentheses
The following shows inconsistent behavior:
import gvar
# define a distribution with a parenthesis in the name
gvar.BufferDict.add_distribution('a(', gvar.square)
b = gvar.BufferDict({'a((x)': 2})
# this thinks '(x' is the key
print(b.extension_pattern.match('a((x)').groups()) # -> 'a', '(x'
# these think 'x' is the key
print(b['x']) # -> 4
print(b.has_dictkey('x')) # -> True
print(b.has_dictkey('(x')) # -> False
# these find no key
print(list(b.all_keys())) # -> 'a((x)'
print(list(b.extension_keys())) # -> emptyI think the easiest way to fix this would be to forbid parentheses in BufferDict.add_distribution. I'm addressing this problem because I generate transformation names automatically based on user input, so I sanitize by removing parentheses.
gvar version: 11.11.12.
Why does gvar use SVD instead of diagonalization?
In general gvar uses SVD instead of diagonalization, for example in gvar.raniter. But for a symmetric matrix it is equivalent to diagonalization, apart from the fact that singular values are positive so the left/right singular vectors differ by a sign when the corresponding eigenvalue is negative.
This means that, when doing the cut, with the SVD negative eigenvalues become more negative, while with diagonalization negative eigenvalues become positive. Intuitively I would prefer the second since the covariance matrix should be positive, however I'm not sure because maybe the first option introduces less bias in the results.
In any case, it would be more efficient to call a diagonalization routine:
from scipy import linalg
import numpy as np
a = np.random.randn(300, 300)
cov = a.T @ a
%timeit linalg.svd(cov)
%timeit linalg.eigh(cov)Output:
147 ms ± 4.47 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
70.5 ms ± 2.63 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
`gvar.sample` does not have an svdcut by default in v13.0
I noticed some of my scripts using gvar.sample are failing due to nan samples, with gvar.sample complaining that the matrix is not pos def.
Minimal example:
import gvar
import numpy as np
print(gvar.__version__)
gvar.sample(gvar.gvar(np.zeros(2), np.ones((2, 2))))13.0
/Users/giacomo/Documents/Scuola/lsqfitgp/Repository/lsqfitgp/pyenv/lib/python3.11/site-packages/gvar/__init__.py:955: UserWarning: Matrix not positive definite, lambda_min/lambda_max = 0; need svdcut?
gflat, i_wgts, i_invwgts = svd(gflat, svdcut=svdcut, noise=noise, wgts=True)
/Users/giacomo/Documents/Scuola/lsqfitgp/Repository/lsqfitgp/pyenv/lib/python3.11/site-packages/gvar/__init__.py:955: RuntimeWarning: divide by zero encountered in log
gflat, i_wgts, i_invwgts = svd(gflat, svdcut=svdcut, noise=noise, wgts=True)
/Users/giacomo/Documents/Scuola/lsqfitgp/Repository/lsqfitgp/pyenv/lib/python3.11/site-packages/gvar/__init__.py:955: RuntimeWarning: divide by zero encountered in scalar divide
gflat, i_wgts, i_invwgts = svd(gflat, svdcut=svdcut, noise=noise, wgts=True)
The docstring states:
svdcut (float): If nonzero, singularities in the correlation
matrix are regulated using :func:`gvar.regulate`
with an SVD cutoff ``svdcut``. Default is ``svdcut=1e-12``.
Which indeed was the default behavior before v13.0.
Did you intend to drop that default? I like gvar.sample to do something robust by default.
Bug: unexpected automatic dtype determination of `gvar.BufferDict`
On creation, if a data type is not specified, gvar.BufferDict upcasts to int64 or float64:
import gvar
import numpy as np
print(gvar.BufferDict(a=np.ones(1, bool)).dtype)
print(gvar.BufferDict(a=np.ones(1, 'int32')).dtype)
print(gvar.BufferDict(a=np.ones(1, 'uint64')).dtype)
print(gvar.BufferDict(a=np.ones(1, 'float32')).dtype)int64
int64
float64
float64
This is unexpected, in particular when I pass explicitly typed numpy arrays as input. Something like this might make sense as a policy if the dtype of a BufferDict was immutable, to have a very simple rule for its automatic determination, however it is mutable:
bd = gvar.BufferDict(a=1)
print(bd.dtype)
bd['b'] = 2.3
print(bd.dtype)int64
float64
I think that the automatic data type determination should work as in numpy.
version
gvar 11.11.1
Improve evalcov for dense matrices
Even after the 11.4 update, evalcov_blocks is still the bottleneck of my scripts. I've seen the new implementation uses evalcov, and spends around 50 % of the time in it for the dense case:
import numpy as np
import gvar
n = 100
a, b = np.random.randn(2, n, n)
cov = a.T @ a
x = b @ gvar.gvar(np.zeros(n), cov)
%timeit gvar.evalcov_blocks(x)
%timeit gvar.evalcov(x)Output:
34.4 ms ± 1.49 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
17.9 ms ± 509 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
I figured that for the dense case first converting everything to dense matrices and then multiplying would give an improvement, so I made this test:
from scipy import sparse
b_sparse = sparse.csr_matrix(b)
cov_sparse = sparse.csr_matrix(cov)
def evalcov_dense_mockup():
b_dense = b_sparse.toarray() # emulate converting gvars coeffs to a matrix
cov_dense = cov_sparse.toarray()
b @ cov @ b.T
%timeit evalcov_dense_mockup()
%timeit gvar.evalcov(x)Output:
961 µs ± 45.4 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
17.9 ms ± 362 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
I've read the evalcov code and sketched a new algorithm. If you allow, I'll try to implement this and make a pull request.
-
Build the mask as usual reading the indices from
dattributes, while also
counting the total number of elements. -
Get the count of primary gvars involved by summing the mask.
-
Count the number of nonzero entries in
covwith the mask. Probably
requires a newsmatmethod. -
Using the counts and the number of primary gvars, determine if the dense
algorithm would be convenient; if it isn't, proceed as usual. Up to here the only overhead is counting nonzeros in cov. -
Extract a dense submatrix from
covusing the mask. Probably requires a
newsmatmethod, and probably it is convenient to first convert the
mask to a mapping, i.e. replace the 1s with the cumsum of the 1s, they then
point to the destination index with an offset of +1. -
Write
delements into a dense matrix, using the mapping as above.
Probably requires a newsvecmethod, which could be used by thesmat
method above. -
Perform the matrix multiplication.
-
Optionally, as a further optimization, copy only a triangular part of
covhalving diagonal elements, useBLAS_xtrmmfor doing the matrix
multiplication, and finally take twice the symmetrical part of the result.
TypeError: float() argument must be a string or a number, not 'gvar._svec_smat.smat'
Hi,
we have a pickle file that was written with gvar==11.10 and can no longer be read with gvar==11.11.10.
Reverting back to the older version works.
The structure of the pickle file is dict[str, dict[str, gvar]], where when printing the dict after reading the gvar values are displayed as, e.g., 0.01224(37).
The error displayed when trying to read the pickle dump with the current gvar version is:
Traceback (most recent call last):
File "src/gvar/_gvarcore.pyx", line 1066, in gvar._gvarcore.GVarFactory._call3
TypeError: float() argument must be a string or a number, not 'gvar._svec_smat.smat'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./load_old.py", line 12, in <module>
my_dict = pickle.load(fp)
File "src/gvar/_gvarcore.pyx", line 77, in gvar._gvarcore.GVar.__setstate__
File "src/gvar/_gvarcore.pyx", line 981, in gvar._gvarcore.GVarFactory.__call__
File "src/gvar/_gvarcore.pyx", line 1068, in gvar._gvarcore.GVarFactory._call3
TypeError: Value not a number.
where the python file reads:
#!/usr/bin/env python3
import pickle
import gvar as gv
for dict_file_name in [
'file1.p',
'file2.p'
]:
with open(dict_file_name, 'rb') as fp:
my_dict = pickle.load(fp)
print(my_dict)
incompatible pointers
I am not sure about the implications of this QA warning for your package.
gvar_function does not work with non-scalars
import gvar
import numpy as np
gvar.gvar_function(gvar.gvar(np.arange(2), np.ones(2)),np.arange(2),np.zeros(2))Error:
TypeError: only size-1 arrays can be converted to Python scalars
Bug: readonly arrays not allowed as covariance matrix for new gvars
The factory does not allow me to make new gvars from a read-only covariance matrix when I use the fast=True option:
import gvar
import numpy as np
mean = np.zeros(1)
cov = np.ones((1, 1))
cov.flags['WRITEABLE'] = False
g = gvar.gvar(mean, cov, fast=True)Output:
File _gvarcore.pyx:938, in gvar._gvarcore.GVarFactory.__call__()
File _svec_smat.pyx:354, in gvar._svec_smat.smat.append_diag_m()
File stringsource:658, in View.MemoryView.memoryview_cwrapper()
File stringsource:349, in View.MemoryView.memoryview.__cinit__()
ValueError: buffer source array is read-onlyThis is weird because I do not expect gvar.gvar to write to my matrix, and probably indeed it's not writing anything. I guess it could be a Cython typing complaint.
Enhancement: consistent number of standard deviation digits when formatting gvars
Typically when I print a number with an uncertainty I want to fix the number of digits of the error shown. gvar does not allow me to do this, the only option is changing the number of digits of the mean with gvar.fmt. Example:
import gvar
print(gvar.gvar(1, 0.01))
print(gvar.gvar(1, 0.1))
print(gvar.gvar(1, 1))
print(gvar.gvar(1, 10))
print(gvar.gvar(1, 100))
print(gvar.gvar(1, 1000))1.000(10)
1.00(10)
1.0(1.0)
1(10)
1(100)
1(1000)
Up to 1(10) the number of error digits is fixed to 2, but then it increases. This should not happen. I wrote the following alternative formatting function:
import math
import re
import gvar
def exponent(x):
return int(math.floor(math.log10(abs(x))))
def int_mantissa(x, n, e):
return round(x * 10 ** (n - 1 - e))
def naive_ndigits(x, n):
log10x = math.log10(abs(x))
n_int = int(math.floor(n))
n_frac = n - n_int
log10x_frac = log10x - math.floor(log10x)
return n_int + (log10x_frac < n_frac)
def ndigits(x, n):
ndig = naive_ndigits(x, n)
xexp = exponent(x)
rounded_x = int_mantissa(x, ndig, xexp) * 10 ** xexp
if rounded_x > x:
rounded_ndig = naive_ndigits(rounded_x, n)
if rounded_ndig > ndig:
x = rounded_x
ndig = rounded_ndig
return x, ndig
def mantissa(x, n, e):
m = int_mantissa(x, n, e)
s = str(abs(int(m)))
assert len(s) == n or len(s) == n + 1 or (m == 0 and n < 0)
if n >= 1 and len(s) == n + 1:
e = e + 1
s = s[:-1]
return s, e
def insert_dot(s, n, e, addzeros=True):
e = e + len(s) - n
n = len(s)
if e >= n - 1:
# no dot at end of mantissa
pass
elif e >= 0:
s = s[:1 + e] + '.' + s[1 + e:]
elif e <= -1 and addzeros:
s = '0' * -e + s
s = s[:1] + '.' + s[1:]
return s
def tostring(x):
return '0' if x == 0 else f'{x:#.6g}'
def uformat(mu, s, errdig, sep):
if errdig < 1:
raise ValueError('errdig < 1')
if not math.isfinite(mu) or not math.isfinite(s) or s == 0:
if sep is None:
return f'{tostring(mu)}({tostring(s)})'
else:
return f'{tostring(mu)}{sep}{tostring(s)}'
s, sndig = ndigits(s, errdig)
sexp = exponent(s)
muexp = exponent(mu) if mu != 0 else sexp - sndig - 1
smant, sexp = mantissa(s, sndig, sexp)
mundig = sndig + muexp - sexp
mumant, muexp = mantissa(mu, mundig, muexp)
musign = '-' if mu < 0 else ''
if mundig >= sndig:
use_exp = muexp >= mundig or muexp < -1
base_exp = muexp
else:
use_exp = sexp >= sndig or sexp < -1
base_exp = sexp
if use_exp:
mumant = insert_dot(mumant, mundig, muexp - base_exp)
smant = insert_dot(smant, sndig, sexp - base_exp, sep is not None)
else:
mumant = insert_dot(mumant, mundig, muexp)
smant = insert_dot(smant, sndig, sexp, sep is not None)
if sep is None:
r = mumant + '(' + smant + ')'
else:
r = mumant + sep + smant
r = musign + r
if use_exp:
if sep is not None:
r = '(' + r + ')'
return r + 'e' + f'{base_exp:+}'
else:
return rIt takes as options the number of digits of the error and the separator. The number of digits of the error can be a noninteger value, which specifies that the number of digits varies depending on the mantissa. If the separator is None, the short notation with parentheses is used.
To avoid breaking backward compatibility, the formatting function can be set up to be used only when the user passes a non-empty formatting specification:
class NewGVar(gvar._gvarcore.GVar):
def __init__(self, g):
self.g = g
def __format__(self, spec):
self = self.g
if not spec:
return str(self)
pat = r'(\d*\.?\d*)(p|s|u)'
m = re.fullmatch(pat, spec)
if not m:
raise ValueError(f'format specification {spec!r} not understood, format is r"{pat}"')
if m.group(1):
ndig = float(m.group(1))
else:
ndig = 1.5
mode = m.group(2)
sep = dict(p=None, s=' +/- ', u=' ± ')[mode]
return uformat(gvar.mean(self), gvar.sdev(self), ndig, sep)
def gv(*args):
return NewGVar(gvar.gvar(*args))Examples:
print(f'{gv(1, 100)}') # 1(100) (default gvar formatter)
print(f'{gv(1, 100):p}') # 0.0(1.0)e+2
print(f'{gv(1, 3):p}') # 1.0(3.0)
print(f'{gv(1, 4):p}') # 1(4) (the default number of digits is 1.5, which switches at 10^0.5 ~ 3.2)
print(f'{gv(1, 4):2u}') # 1.0 ± 4.0
print(f'{gv(1, 4):2.8s}') # 1.00 +/- 4.00Error in `BufferDict.del_distribution`
The following does not work with gvar 11.9.5:
import gvar
gvar.BufferDict.add_distribution('cippa', gvar.exp)
gvar.BufferDict.del_distribution('cippa')Output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "_bufferdict.pyx", line 539, in gvar._bufferdict.BufferDict.del_distribution
AttributeError: 'cython_function_or_method' object has no attribute 'invfcn'
Wrong inputs in `gvar.BufferDict.uniform` can break permanently `BufferDict.get`
import gvar
gvar.BufferDict.uniform(0, 1, 2) # pass an integer instead of a label
b = gvar.BufferDict() # an unrelated empty bufferdict
b.get('cippa', 3.14) # try to get a missing key with backup value -> error instead of 3.14Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/_collections_abc.py", line 819, in get
return self[key]
File "_bufferdict.pyx", line 321, in gvar._bufferdict.BufferDict.__getitem__
TypeError: unsupported operand type(s) for +: 'int' and 'str'
This is annoying when I make a mistake calling gvar.BufferDict.uniform and then have to restart the ipython session.
Bug: unexpected attribute access in `BufferDict.del_distribution`
Deleting a distribution produces an AttributeError if BufferDict.uniform has not been previously called, but not always, I'm not sure this bug is fully reproducible. Anyway:
import gvar
gvar.BufferDict.del_distribution('log')Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "_bufferdict.pyx", line 539, in gvar._bufferdict.BufferDict.del_distribution
AttributeError: 'cython_function_or_method' object has no attribute 'invfcn'Despite the error, the distribution has been deleted:
gvar.BufferDict.del_distribution('log')Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "_bufferdict.pyx", line 538, in gvar._bufferdict.BufferDict.del_distribution
KeyError: 'log'Using uniform solves the problem:
gvar.BufferDict.uniform('U', 0, 1)
gvar.BufferDict.del_distribution('sqrt') # -> all fineLooking at the code of del_distribution, it would seem legal:
@staticmethod
def del_distribution(name):
""" Delete |BufferDict| distribution ``name``.
Raises a ``ValueError`` if ``name`` is not the name of
an existing distribution.
"""
if name in BufferDict.invfcn:
del BufferDict.invfcn[name]
else:
raise ValueError('{} is not a distribution'.format(name))
if hasattr(BufferDict.uniform, 'invfcn') and name in BufferDict.uniform.invfcn:
del BufferDict.uniform.invfcn[name]
BufferDict._ver_g += 1My guess is that, either in the C transpiling or in the compilation proper, the compiler does not respect the non-evaluation of the second operand in the and and accesses BufferDict.uniform.invfcn even though the hasattr test failed.
ZeroDivisionError in gv.sdev for non-gvar arrays w/ length > 50
gv.sdev throws a ZeroDivisionError when applied to arrays or lists of pure numbers (i.e. not gvars) longer than 50 elements. The values don't seem to matter.
To reproduce:
gv.sdev(np.arange(50)) # == np.zeros(50)
gv.sdev(np.arange(51)) # ZeroDivisionError
gv.sdev([0]*51) # ZeroDivisionError
gv.sdev(np.random.rand(51)) # ZeroDivisionError
"Failed building wheel for gvar" when installing gvar without numpy (in cache)
Hello @gplepage,
First of all, I'd like to thank you for providing and maintaining gvar and lsqfit. Both are a tremendous help in our analysis projects.
The issue I am finding is an edge case and does not prohibit the installation of gvar; however, I believe a fix may speed up installing gvar from scratch (by (re-)using wheels).
The issue
When creating a new virtual environment or installing gvar from scratch, I encounter a
ModuleNotFoundError: No module named 'numpy'
from
setup.py", line 32, in build_extensions
which causes the building of the wheel to fail. However, the install still goes through on most of the systems/versions I have tried (only a few scenarios stop at this step). As far as I understand it, regarding #2 and #4, you have worked out a way to make it work by delaying the numpy import and installing it on the fly.
The steps to reproduce this error is to ensure numpy and gvar are not installed (i.e., in my case, a new virtual environment) and running
pip install --no-cache-dir gvaron
pip --version 21.3.1
python --version 3.8.10
Possible solution
As far as I understand it, this issue is caused when numpy is not present at the build time of gvar. Unfortunately, even if you specify numpy in setup.py install_requires, it is not installed before the building starts.
Fortunately, with PEP 517, setup tools, which manages the install in setup.py, now allows to specify dependencies needed at build time. This is done by adding a pyproject.toml to the root of the directory. In the case of gvar, I believe the following content solves the issue I have described above
[build-system]
requires = [
"setuptools >= 40.9.0",
"wheel",
"numpy",
"scipy",
]
build-backend = "setuptools.build_meta"
Generally, to allow for different building tools, it seems to be the trend to go away from "dynamic" setup scripts and make them static. I.e., replacing setup.pys with setup.cfgs.
I have created a local fork and set up different GitHub actions which try to install gvar from scratch on different operating systems and using different distributions.
The ones with the pyproject.toml (see the "install module" tab) succeed in building the wheel while the ones without the pyproject.toml fall back to the legacy install, encounter the missing numpy Error and proceed without building the wheel.
I believe adding this .toml file may also simplify the current setup.py file, but I did not look much further into it.
If this is something you are interested in, I am happy to help out and provide more information or create a PR.
Best regards,
Christopher
PS:
- The GitHub actions could also be configured to run the unit tests for PRs or pushes automatically
- I am also happy to look into whether it is possible to convert the
setup.pyinto asetup.cfg - This might also apply to
lsqfit
Confusing behaviour of `BufferDict.add_distribution`
Consider the following
import gvar
gvar.BufferDict.add_distribution('cippa', gvar.exp)
a = gvar.BufferDict({'cippa(x)': 1})
print(a['x'])
gvar.BufferDict.add_distribution('cippa', gvar.log)
print(a['x'])
a.buf # has a side effect!
print(a['x'])Output:
2.718281828459045
2.718281828459045
0.0
Two problems:
gvar.add_distributionlets me silently redefine the distribution without an error, unlikegvar.BufferDict.uniform. This implicitly changes the values stored in existingBufferDicts.- However, the
BufferDictinstance "notices" the change only after I access itsbufattribute. What's happening?
gvar.pxd is installed in toplevel
I've been trying to add gvar (and vegas) to gentoo, but this error keeps poping up:
* The following unexpected files/directories were found top-level
* in the site-packages directory:
*
* /usr/lib/python3.11/site-packages/gvar.pxd
*
* This is most likely a bug in the build system. More information
* can be found in the Python Guide:
* https://projects.gentoo.org/python/guide/qawarn.html#stray-top-level-files-in-site-packages
* ERROR: sci-libs/gvar-12.1::hep-forge failed (install phase):
I think it is related to
Line 26 in e577685
Could you maybe reconsider the package structure/build system there?
Import fails with numpy 1.16.2 and python 3.7.2 for gvar >= 11.11.3
I'm running gvar on HPC ressources with an older version of numpy (1.16.2).
Since gvar 11.11.3 it cannot be imported anymore with numpy 1.16.2:
Traceback (most recent call last):
File "post.py", line 25, in <module>
import gvar as gv
File "[...]/lib/python3.7/site-packages/gvar/__init__.py", line 123, in <module>
from ._gvarcore import *
File "src/gvar/_gvarcore.pyx", line 1, in init gvar._gvarcore
ImportError: numpy.core.multiarray failed to import (auto-generated because you didn't call 'numpy.import_array()' after cimporting numpy; use '<void>numpy._import_array' to disable if you are certain you don't need it).I have no idea, what the problem is and if there is an easy fix.
A solution would be to require a newer numpy version for gvar.
installation
The installation of gvar requires installation of numpy and cython before hand, although both packages have been specified in the install_requires of setup.py. Is that caused by the fact that setup.py import numpy and cython? Could you do something like
`
import pip
try:
import Cython
import numpy
except ImportError:
pip.main(['install', 'cython', 'numpy'])
`
at the top of the setup.py file to remove the requirment on the user to install those manually.
Zero division in evalcov
I'm dumping a dictionary with gvar.dump. The dictionary contains many gvars and lsqfit.nonlinear_fits. I have the latest version of gvar and lsqfit installed. The script fails with this traceback:
_utilities.pyx in gvar._utilities.dump()
_utilities.pyx in gvar._utilities.gdumps()
_utilities.pyx in gvar._utilities.gdump()
_utilities.pyx in gvar._utilities._gdump()
_utilities.pyx in gvar._utilities.evalcov_blocks()
_utilities.pyx in gvar._utilities.evalcov()
ZeroDivisionError: float division
I looked at the code of evalcov, the only division I see is at line 987 is_dense = cov_zeros / ni / ni < 0.5, although I would not expect ni to be zero. I've not used gvar.switch_gvar() or done other messes.
I can share the code if needed but it's long. I'm sorry I can not provide a minimal working example right now.
Install gvar on the code interpreter
Hi, did anyone tried to use gvar package in ChatGPT's code interpreter? How to make it? I found the answer to use external package in ChatGPT's code interpreter related to wheel file, but it seems no wheel file on PyPI for gvar...
Memory leak
Aloha,
I was tracking a memory leak in a code and narrowed down the source of the leak to gvar using the following test loop:
#!/usr/bin/env python
import gvar
import numpy as np
for i in range(4000000):
spectrum = np.arange(100)
spectrum = gvar.gvar(spectrum, np.ones(100))
If you could find a fix, it would greatly help!
4.9.0-5-amd64 #1 SMP Debian 4.9.65-3+deb9u2 (2018-01-04) x86_64 GNU/Linux
Thanks,
Laurie
gvar.cov does not check the gvars are compatible
import gvar
x = gvar.gvar_factory()(0, 1)
y = gvar.gvar_factory()(0, 1)
print(gvar.cov(x, y))Result: 1.0. It should be either 0 or raise an error.
`gvar.var` fails on non-gvar arrays longer than some threshold
gvar.var works for non-gvar numbers, returning 0. However it misteriously fails on an array long enough:
import numpy as np
import gvar
print(gvar.__version__)
for n in range(100):
try:
gvar.var(np.ones(n))
except Exception as exc:
print('failed at n =', n)
raise exc11.10.1
failed at n = 51
Traceback (most recent call last):
File "<stdin>", line 6, in <module>
File "<stdin>", line 3, in <module>
File "_utilities.pyx", line 900, in gvar._utilities.var
ZeroDivisionError: float division
Feature request: new primary gvars correlated with preexisting primary gvars
I wanted to implement Bayesian optimization using gvar. In Bayesian optimization you cycle between using the fit to decide where to get the next datapoint, and redoing the fit with the new datapoint.
The fit is a Gaussian process inference which I want to do with gvar, which implies creating correlated primary gvars representing the Gaussian process prior. This means that every time I add a point, it must be correlated with previous points. However, I don't know in advance all the points, so I can't compute the covariance matrix before creating the first set of gvars. Thus I would need an interface to create new gvars correlated to preexisting primary gvars.
I see that in the implementation of smat there is an incremental block index associated to each new set of correlated primary gvars. Adding new gvars correlated with old ones would be a problem because either the monotonicity of the block index or the independence of extraneous gvars created between the old and new ones would be lost. Anyway, I think the latter case would not be a serious problem in practice. What to do depends on the assumptions on the block index you used in implementing evalcov_blocks.
I'd propose an interface like this:
x = gvar.gvar(xmean, xcov)
y = gvar.gvar(ymean, ycov, x, xycov)
# xycov is a len(x)*len(y) matrix (or equivalent tensor or bufferdict)
# raises an exception if x contains non-primary gvarsAlternative:
x = gvar.gvar(xmean, xcov)
y = gvar.gvar(ymean, ycov)
gvar.addcov(x, y, xycov)
# raises an exception if x or y contain non-primary gvarsI prefer the first interface because in the second it would be possible to compute stuff with y and then change the results post-facto by introducing other correlations, while in the first the correlations can be added only at the creation of y, maintaining coherence.
This kind of interface is very user-friendly but requires a bit of effort to implement to keep into account that the gvars in x can appear in any order from any block. A simpler internal-only thing, that would be fine for my needs, could be a function to add off-diagonal covariance blocks aligned with already defined blocks:
x = gvar.gvar(xmean, xcov)
y = gvar.gvar(ymean, ycov)
def blockid(g):
assert g.is_primary()
return g.internaldata[2].blockid(g.internaldata[1].indices()[0])
gvar._addcovblock(blockid(x[0]), blockid(y[0]), xycov)
# I guess _addcovblock would set the block id of y to the one of xCan you do something like this?
If you don't have time, I can try to code something myself and make a pull request (always following your choices on the details of course).
Feature request: full iteration of `BufferDict`
I often find myself printing a BufferDict and all the transformed versions of its variables:
import gvar
d = gvar.BufferDict({'log(x)': 0})
for k in d:
print(k, d[k])
m = d.extension_pattern.match(k)
if m and d.has_distribution(m.group(1)):
q = m.group(2)
print(q, d[q])log(x) 0
x 1.0
So I thought it would be nice to add a method to BufferDict that returns an iterator going through all the keys:
def allkeys(d):
for k in d:
yield k
m = d.extension_pattern.match(k)
if m and d.has_distribution(m.group(1)):
yield m.group(2)
gvar.BufferDict.allkeys = allkeys
for k in d.allkeys():
print(k, d[k])log(x) 0
x 1.0
*.c files shipped in PyPI is not compatible with Python 3.11
For the newest Python release 3.11, pip install gvar will raise No such file or directory error because some headers are in other folders now.
Maybe there should be *.c files for both versions, or automatically enable Cython to build if 3.11 is detected.
Inconsistent behaviour with asymmetric covariance matrices
import gvar
x = gvar.gvar([0, 0], [[1, 0.1], [0.9, 1]])
print(gvar.evalcov(x))
print(gvar.evalcov_blocks(x))
print(gvar.cov(x[0], x[1]))
print(gvar.cov(x[1], x[0]))Output:
[[1. 0.9]
[0.9 1. ]]
[(array([0, 1]), array([[1. , 0.9],
[0.9, 1. ]]))]
0.9
0.1
So: gvar.gvar stores the full covariance matrix, but gvar.evalcov and gvar.evalcov_blocks only read the lower triangular part. The original matrix can be recovered with gvar.cov.
I think that one or more of the following should be implemented:
-
Check that the convariance matrix is symmetrical in
gvar.gvar. -
gvar.gvarcomputes the symmetrical part of the matrix. -
Read only the lower/upper triangular part in
gvar.gvar, controlled by a parameterlower: boollike inscipy.linalg. Would also apply to dictionaries. -
gvar.evalcovandgvar.evalcov_blocksread all the matrix. -
gvar.covreads only the lower triangular part.
Personally I prefer option (4) because it seems the simplest to implement. Also, since in general gvar and lsqfit use SVD instead of diagonalization or cholesky, the full matrix is always used, so this would be consistent with the rest of the library.
Bug: scalar multiplication
I have not been able to track down exactly what caused this issue to start happening, but after installing gvar on a new machine I found that a very simple line of old code was now broken:
>>> import gvar as gv
>>> gv.gvar("1(2)")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "src/gvar/_gvarcore.pyx", line 982, in gvar._gvarcore.GVarFactory.__call__
File "src/gvar/_gvarcore.pyx", line 1017, in gvar._gvarcore.GVarFactory._call1
TypeError: unsupported operand type(s) for *: 'float' and 'gvar._gvarcore.GVar'
>>>
This seems to be a manifestation of a simpler issue, which is that scalar multiplication does not work depending on the order of operations:
>>> a = 3.0
>>> b = gv.gvar(1,1)
>>> b*a
3.0(3.0)
>>> a*b
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for *: 'float' and 'gvar._gvarcore.GVar'
>>>
I am using Python 3.11.4 and gvar 11.11.11, although I tried creating a Python 3.9 environment and bumped into the same problem so I don't think it's 3.11 specific. I'm on MacOS Ventura with M2 CPU.
I think this can be fixed (without breaking anything else) by defining an __rmul__ procedure in _gvarcore.pyx. Here is a patch that appears to fix the issues noted above for me, although I am not certain whether any edge cases need to be handled:
def __rmul__(xx,yy):
return xx.__mul__(yy)
Bug: error in `gvar.raniter` with negative `svdcut`
In the following code I define very correlated gvars and try to take samples from them.
import gvar
import numpy as np
x = np.linspace(0, 1, 1000)
cov = np.exp(-(x[None, :] - x[:, None]) ** 2)
y = gvar.gvar(np.zeros(len(x)), cov)
gvar.sample(y) # no problem
gvar.svd(y, svdcut=-1e-12) # no problem
gvar.sample(y, svdcut=-1e-12) # failsOutput:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "_utilities.pyx", line 2319, in gvar._utilities.sample
File "_utilities.pyx", line 2374, in raniter
IndexError: index 9 is out of bounds for axis 0 with size 9
I need the negative cut because the positive one introduces a bit of noise in my samples.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.