mattloper / chumpy

chumpy

Autodifferentiation tool for Python.

Installation

Install the fork:

pip install chumpy

Import it:

import chumpy as ch

Overview

Chumpy is a Python-based framework designed to handle the auto-differentiation problem, which is to evaluate an expression and its derivatives with respect to its inputs, by use of the chain rule.

Chumpy is intended to make construction and local minimization of objectives easier.

Specifically, it provides:

• Easy problem construction by using Numpy’s application interface
• Easy access to derivatives via auto differentiation
• Easy local optimization methods (12 of them: most of which use the derivatives)

Usage

Chumpy comes with its own demos, which can be seen by typing the following:

import chumpy
chumpy.demo() # prints out a list of possible demos

License

This project is licensed under the MIT License.

chumpy's Issues

can't assign a Ch objects as a subset of another

I am trying to assign a 3x3 array as a subset of a 3x3x9 array, but it gives me
Exception: Can't assign a Ch objects as a subset of another.
The following code is troublesome:
k = ch.arange(81).reshape(3, 3, 9)
tmp = ch.eye(3)
k[:, :, 1] = tmp
The complete error message is the following:
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/chumpy/ch.py", line 468, in setitem
raise Exception("Can't assign a Ch objects as a subset of another."
Exception: Can't assign a Ch objects as a subset of another.

Forward or Backward mode autodiff?

Is chumpy using Forward mode or Reverse mode automatic differentiation? A quick browse seems like forward mode...

Problem about calling newton-cg optimization method

Hello @mattloper ,
Thank you very much for this wonderful tool! I am trying to use chumpy in one of my project, and I met one problem when calling the ch.minimize function with method newton-cg. I dive into the source code of chumpy and found that the problem is caused by function ns_jacfunc in chumpy/optimization.py. To be more concrete, a function 'hstack' is called in this function, but actually it is not defined. I know both numpy and chumpy have hstack, but it seems you didn't import them in this py file. I have also tried to import them, but it didn't help (some other problems occur after it, so I guess you don't want to use np.hstack or ch.hstack here). Do you know how to fix this bug?

Best,
Yongchi

Build fails with setuptools 50.0.0

Using pip 20.2.2 and setuptools 50.0.0 (released yesterday, August 30th)

Downgrading setuptools to 49.6.0 enabled me to workaround the issue.
Output:

Collecting chumpy
  Downloading chumpy-0.70.tar.gz (50 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/work/env/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-n2ix487y/chumpy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-n2ix487y/chumpy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-gprw26td
         cwd: /tmp/pip-install-n2ix487y/chumpy/
    Complete output (15 lines):
    /home/work/env/lib/python3.6/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
      warnings.warn("Setuptools is replacing distutils.")
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-n2ix487y/chumpy/setup.py", line 56, in <module>
        'Operating System :: POSIX :: Linux'
      File "/home/work/env/lib/python3.6/site-packages/setuptools/_distutils/core.py", line 134, in setup
        ok = dist.parse_command_line()
      File "/home/work/env/lib/python3.6/site-packages/setuptools/_distutils/dist.py", line 484, in parse_command_line
        args = self._parse_command_opts(parser, args)
      File "/home/work/env/lib/python3.6/site-packages/setuptools/dist.py", line 903, in _parse_command_opts
        nargs = _Distribution._parse_command_opts(self, parser, args)
      File "/home/work/env/lib/python3.6/site-packages/setuptools/_distutils/dist.py", line 548, in _parse_command_opts
        "command class %s must subclass Command" % cmd_class)
    distutils.errors.DistutilsClassError: command class <class 'setuptools.command.egg_info.egg_info'> must subclass Command
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Possible error while computing dr_wrt() with 2D arrays

Hi,

I tried following commands (snapshot attached) but I don't know why this error is coming (pdb breakpoint is encountered somehow).
I have shown 2 versions in my snapshot , one sequence for 1 D array (where .dr_wrt works) and one sequence with 2 D array where it is failing. Can you please check it out.
Thanks :)

Does this project has the plan to upgrade to python 3.x?

Hi @mattloper , this awesome project is currently python 2.x based. Do you have the plan to support python 3.x？Thanks!

Optimizing for GPU

Hello, first of all thank you for the useful library.

I see that computations are running on the CPU. In my work this causes a speed bottleneck and I would like to run these computations, or at least some parts on GPU. I was examining the code and my code uses minimize_dogleg function from ch.optimization. Lines with state.updateJ is where most of the is being spent.

I was trying to convert some operations to GPU using Numba library but I don't have much success so far. Does anyone tried this before, or if you can show some directions that would be appreciated.

The nondeterministic computation order of Jacobian

The computation order of Jacobian in Chumpy is nondeterministic. This in cases leads to different optimization results from run to run, since the numerical approximation errors are dependent on the computation order. Specifically, the function dr_wrt() in class Ch has the following code:

for k in set(self.dterms).intersection(propnames.union(set(self.__dict__.keys()))):
    ...

The set is an unordered container in python, and what's even worse for science computation is that its access order is nondeterministic. A quick fix to this could be adding a call to sorted():

for k in sorted(set(self.dterms).intersection(propnames.union(set(self.__dict__.keys())))):
    ...

New PyPI release without numpy imports

The current version on PyPI does not work with numpy>=1.24.0, since this is already fixed in master could you publish a new release? Thank you 🙂

Confusion about beta_multiplier function in the dogleg implementation.

Dear @mattloper , thank you for your work on the chumpy and the dogleg implementation. I have some questions about function beta_multiplier in optimization_interal.py in line#268. According to my understanding, \beta should satisfy "||d_sd + \beta*(d_gn-d_sd)||_2 == delta", but \beta is not calculated by solving the equation. Can you give me some hints about your calculation of \beta? Thank you.

chumpy `setup.py` improperly imports pip causing installation failure when using PEP 517

Issue Description

The chumpy package's setup.py file currently attempts to import pip during the installation process, as shown in the following lines:

try:
    from pip._internal import main as pip_main
except ImportError:
    from pip import main as pip_main

There are a couple of problems with this approach:

1. Violation of PEP 517: When using the --use-pep517 flag with pip (which will become the default behavior in the future), or when using other PEP 517-compliant build tools, the installation process fails with a ModuleNotFoundError: No module named 'pip' error. This is because pip is not guaranteed to be available during the installation process when PEP 517 is used.
2. Assumption of pip availability: Even without using PEP 517, the assumption that pip is installed and available during the package installation process is incorrect. Some package managers or environments may not include pip by default, leading to the same ModuleNotFoundError.

Steps to reproduce

1. Create a new virtual environment and install chumpy using pip with the --use-pep517 flag:

python -m venv myenv
source myenv/bin/activate
pip install chumpy --use-pep517

2. The installation will fail with the following error:

Traceback (most recent call last):
  File "<string>", line 9, in <module>
ModuleNotFoundError: No module named 'pip'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  ...
  File "<string>", line 11, in <module>
ModuleNotFoundError: No module named 'pip'

pip install chumpy------> ERROR mesage

ModuleNotFoundError: No module named 'pip'
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

can not find 0.69 version

when i run pip install chumpy==0.69, there turns out
WARNING: Discarding https://files.pythonhosted.org/packages/87/81/d7a94ad0ff556b9dd1bf27b84c0b255d9ba22ad5952f20088dafacd9292e/chumpy-0.69.tar.gz#sha256=801bfdd53ffcdf386620b29129dc8d7f75f397010f31554c17a30d5a7b58c4df (from https://pypi.org/simple/chumpy/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. ERROR: Could not find a version that satisfies the requirement chumpy==0.69 (from versions: 0.51, 0.52, 0.53, 0.54, 0.55, 0.56, 0.57, 0.58, 0.59, 0.60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.66, 0.67.1, 0.67.2, 0.67.3, 0.67.4, 0.67.5, 0.67.6, 0.67.7, 0.68, 0.69, 0.70) ERROR: No matching distribution found for chumpy==0.69

'import chumpy' fails: ImportError: cannot import name 'bool' from 'numpy'

>>> import chumpy
/usr/local/lib/python3.9/site-packages/chumpy/__init__.py:11: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.
  from numpy import bool, int, float, complex, object, unicode, str, nan, inf
/usr/local/lib/python3.9/site-packages/chumpy/__init__.py:11: FutureWarning: In the future `np.object` will be defined as the corresponding NumPy scalar.
  from numpy import bool, int, float, complex, object, unicode, str, nan, inf
/usr/local/lib/python3.9/site-packages/chumpy/__init__.py:11: FutureWarning: In the future `np.str` will be defined as the corresponding NumPy scalar.
  from numpy import bool, int, float, complex, object, unicode, str, nan, inf
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/chumpy/__init__.py", line 11, in <module>
    from numpy import bool, int, float, complex, object, unicode, str, nan, inf
ImportError: cannot import name 'bool' from 'numpy' (/usr/local/lib/python3.9/site-packages/numpy/__init__.py)
>>>

Version: 0.70
py39-numpy-1.25.0
Python-3.9
FreeBSD 14.0

Meet a error during installation, I use "python setup.py" to install chumpy, is it right?

lpc@lpc:~/Downloads/chumpy-master$ python setup.py
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'install_requires'
warnings.warn(msg)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help

error: no commands supplied

Python 3.10 install error

Is there anyway to get around the Python 3.10 error when installing?

Installation broken for environment that do not have scipy

The installation should install all the necessary packages that are listed in the requirements. However, the setup.py has this line now:

version=importlib.import_module('chumpy').__version__

which makes an import of the file chumpy/__init__.py, which in turn executes:

from ch import *

For the installation to work, all the packages required by Chumpy should then be installed prior to the installation of chumpy, which is, I believe, a problem.

ch.linalg.svd not working in stacked mode

From numpy doc:

If a has more than two dimensions, then broadcasting rules apply [...]. This means that SVD is working in “stacked” mode: it iterates over all indices of the first a.ndim - 2 dimensions and for each combination SVD is applied to the last two indices.

https://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.svd.html

When calculating the derivatives for SVD chumpy assumes 2D arrays:

line 196 of linalg.py:

result = np.einsum('ik,jk->kij', u, v)

line 214ff of linalg.py

mxsz = max(shp[0], shp[1])
D = np.zeros(mxsz)
D[:_D.size] = _D

The only workaround I see right now is looping over the dimensions outside of SVD (slow).

show_tree does not work even after DEBUG = True

Hello, The package seems works very well. With OpenDR and SMPLify etc. Still very difficult to use because cannot check the inside of objects. Problem come because the 'show_tree' doesnot work even after ch.DEBUG = True set successfully.

What does `r` in `compute_r` refer to?

Method compute_r in primitives serve as forward calculation. But I'm really curious about what letter r stands for?

I'd appreciate it if you could answer this trivial question.
Thank you~

Cannot try demo 'show_tree'

I have some problem trying to try demo 'show_tree', The error message is like this? Have any idea why?
I use Python 2.7.12
Thanks a lot.

- - - - - - - - - - - <CODE> - - - - - - - - - - - -

import chumpy as ch

[x1, x2, x3] = ch.array(10), ch.array(20), ch.array(30)
for i in range(3): x2 = x1 + x2 + x3

x2.dr_wrt(x1) # pull cache 
x2.dr_wrt(x3) # pull cache
x1.label='x1' # for clarity in show_tree()
x2.label='x2' # for clarity in show_tree()
x3.label='x3' # for clarity in show_tree()
x2.show_tree(cachelim=1e-4) # in MB

- - - - - - - - - - - </CODE> - - - - - - - - - - - -

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/chumpy/__init__.py", line 118, in demo
    exec('global np\n' + demos[which], globals(), locals())
  File "<string>", line 13, in <module>
  File "/usr/local/lib/python2.7/dist-packages/chumpy/ch.py", line 1009, in show_tree
    assert DEBUG, "Please use dr tree visualization functions in debug mode"
AssertionError: Please use dr tree visualization functions in debug mode

AttributeError: module 'inspect' has no attribute 'getargspec'. Did you mean: 'getargs'?

This error occured when trying to load a pkl file.
I am using an Anacona environment with Python 3.11 and chumpy 0.70 installed- I'm guessing the current chumpy version isn't compatible with python 3.11 yet?

Traceback (most recent call last):
File "c:_MyDrive\Detectron\code\SMPL-densepose.py", line 8, in
data = pickle.load(f, encoding="latin1")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\anaconda3\envs\densepose\Lib\site-packages\chumpy_init_.py", line 1, in
from .ch import *
File "C:\Users\user\anaconda3\envs\densepose\Lib\site-packages\chumpy\ch.py", line 1319, in
from . import linalg
File "C:\Users\user\anaconda3\envs\densepose\Lib\site-packages\chumpy\linalg.py", line 178, in
class SvdD(Ch):
File "C:\Users\user\anaconda3\envs\densepose\Lib\site-packages\chumpy\linalg.py", line 181, in SvdD
@depends_on('x')
^^^^^^^^^^^^^^^
File "C:\Users\user\anaconda3\envs\densepose\Lib\site-packages\chumpy\ch.py", line 1203, in _depends_on
want_out = 'out' in inspect.getargspec(func).args
^^^^^^^^^^^^^^^^^^
AttributeError: module 'inspect' has no attribute 'getargspec'. Did you mean: 'getargs'?

Optimization demo looks broken

Running ch.demo('optimization') and halted.

1.42e+02 | e1: 2.25e+01 | e2: 5.06e+01 | e3: 6.89e+01 | 
/Library/Python/2.7/site-packages/scipy/sparse/linalg/dsolve/linsolve.py:145: MatrixRankWarning: Matrix is exactly singular
  warn("Matrix is exactly singular", MatrixRankWarning)

Ctrl-c gives

/Users/chenyangliu/Desktop/BL-dev/chumpy/ch.pyc in __setattr__(self, name, value, itr)
    342             if name == 'x':
    343                 self._dirty_vars.add(name)
--> 344                 self.clear_cache(itr)
    345             #else:
    346             #    import warnings

/Users/chenyangliu/Desktop/BL-dev/chumpy/ch.pyc in clear_cache(self, itr)
    404 
    405                 for parent, parent_dict in next._parents.items():
--> 406                     object.__setattr__(parent, '_dirty_vars', parent._dirty_vars.union(parent_dict['varnames']))
    407                     parent._invalidate_cacheprop_names(parent_dict['varnames'])
    408                     todo.append(parent)

import Chumpy error

Python 3.11.8 (main, Feb 26 2024, 21:39:34) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import chumpy
Traceback (most recent call last):
File "", line 1, in
File "/home/preston/miniconda3/envs/manotorch/lib/python3.11/site-packages/chumpy/init.py", line 1, in
from .ch import *
File "/home/preston/miniconda3/envs/manotorch/lib/python3.11/site-packages/chumpy/ch.py", line 1319, in
from . import linalg
File "/home/preston/miniconda3/envs/manotorch/lib/python3.11/site-packages/chumpy/linalg.py", line 178, in
class SvdD(Ch):
File "/home/preston/miniconda3/envs/manotorch/lib/python3.11/site-packages/chumpy/linalg.py", line 181, in SvdD
@depends_on('x')
^^^^^^^^^^^^^^^
File "/home/preston/miniconda3/envs/manotorch/lib/python3.11/site-packages/chumpy/ch.py", line 1203, in _depends_on
want_out = 'out' in inspect.getargspec(func).args
^^^^^^^^^^^^^^^^^^
AttributeError: module 'inspect' has no attribute 'getargspec'. Did you mean: 'getargs'?

Setup.py fails for pip>=20.1

Environment
Conda virtual environment
pip version: 20.1
Python version: 3.7.6
OS: Mac Catalina 10.15.4

Steps to Reproduce
Install anaconda 3.7.6
pip install chumpy

Output

  Using cached chumpy-0.69.tar.gz (50 kB)
    ERROR: Command errored out with exit status 1:
     command: /Users/sandrews/.virtualenvs/py37cv4/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-install-klqvusum/chumpy/setup.py'"'"'; __file__='"'"'/private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-install-klqvusum/chumpy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-pip-egg-info-9uh6xoy2
         cwd: /private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-install-klqvusum/chumpy/
    Complete output (7 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-install-klqvusum/chumpy/setup.py", line 15, in <module>
        install_requires = [str(ir.req) for ir in install_reqs]
      File "/private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-install-klqvusum/chumpy/setup.py", line 15, in <listcomp>
        install_requires = [str(ir.req) for ir in install_reqs]
    AttributeError: 'ParsedRequirement' object has no attribute 'req'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.```

The explicit call of gc.collect()

Could you please explain the reasoning behind explicitly calling the collect() method of the garbage collector in ch.py, line 802?

Have you encountered the issue of the memory blowing up during the optimization process?

I am asking this question because I am noticing a big difference in the time taken to optimize a function (especially when the memory is filled with additional objects managed by the collector), and I've tracked the bottleneck to the gc.collect() call.

Installation broken with pip 10

I had updated to pip 10.0.1 (to get rid of pip's update nag message), and then while trying to install a package that required chumpy its setup.py failed:

Collecting chumpy (from -r requirements.txt (line 5))
  Downloading https://files.pythonhosted.org/packages/61/61/dc4e134b39674ecdd12a68139d2b82c4f9514133e023c070c973afced53a/chumpy-0.67.5.tar.gz (47kB)
    100% |████████████████████████████████| 51kB 3.9MB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-ERtEwy/chumpy/setup.py", line 8, in <module>
        from pip.req import parse_requirements
    ImportError: No module named req

[This stackoverflow answer(https://stackoverflow.com/a/49867265/69326) has a suggested fix.

Error in ptyhon3

SyntaxError: Missing parentheses in call to 'print'. Did you mean print('Please indicate which demo you want, as follows:')?

Please help me!! how to assemble a new array with chumpy object?

for example:
a = ch.array(5)

Rx = ch.array([[1, 0, 0],
[0, ch.cos(a), -ch.sin(a)],
[0, ch.sin(a), ch.cos(a)]])

but when I change a[0] ,the value in Rx will not change . It seems that my way of array assembly will break the forward-mode
So how can I assemble a new array and still hold the forward mode.

Set specific stopping condition.

Hi~
While using the method, one may have several terms to optimize. For example:

objs = {}
objs['j3d'] = obj_j3d(5e4)
objs['prior'] = pprior(0.8e2)

and I also know that the method has paras to set stopping conditions:

e_N are stopping conditions:
    e_1 is gradient magnatude threshold
    e_2 is step size magnatude threshold
    e_3 is improvement threshold (as a ratio; 0.1 means it must improve by 10%% at each step)

Here I want to learn how can we set a specific term's stopping condition?
For example, we just want to ser a improvement threshold for the specific term objs['j3d'] = obj_j3d(5e4).
Thanks in advance!

Best,
Frank