Aperiodic error when computing xo.orbits.KerplerianOrbit about exoplanet

[dfm]

I figured this out. It happens because PyMC3 changes Theano's behavior. You can get your code to run by adding the following at the top of the notebook:

import theano
theano.config.compute_test_value = "raise"

In that case, all of the test values are propagated properly.

(note: that there are other issues, like negative uncertainties, that make the results nonsense, but this was the issue causing the crash 😄)

[dfm]

The problem here is that you can't use eval with something within a PyMC3 model. Instead you need to use xo.eval_in_model. The fact that it doesn't always fail will have something to do with the fact that you're using variable outside the scope of this function (this function doesn't execute because planet doesn't exist).

[dfm]

Can you try to make a minimal example that reproduces this error message without everything else that's going on. This error always happens when you have some dependence on a pymc3 object in your call so something is happening with the objects outside the scope being overwritten. There's too much going on in your colab to really track it down so please try to make a smaller example and we'll figure it out.

[exowanderer]

I need to make two apologies with respect to my previous post.

1. I do not call this function inside the with pm.Model() as model. I said the opposite above; probably because I got trivially confused with the flow of my own code from how many times I edited it. Therefore the .eval is only called outside the PyMC3 usage.

   At the same time, the orbit instance is passed into the PyMC3 model + xo.optimize + pm.sample usage. Maybe that's connected.

   [Double] at the same time, I changed the instance name model to lightcurve in my colab notebook; after resetting the environment, I have not yet incurred the error again.

   I'll report back if I incur the error anew.

2. You found a typo in my example above because I was converting from the format I use (i.e. with an object to store the planetary properties: planet) to an easier to interpret format. I fixed that typo above. The code should run from copy/paste into an ipython terminal / notebook; i.e. it works for me.

[exowanderer]

Well, it happened again. I was running the same colab notebook very successfully for ~10 iterations; then I reset the environment, "just to check", and incurred the same error as before.

I changed absolutely nothing between the successful trials and unsuccessful trials; I only selected Runtime > Reset and Run All

The traceback makes it look as though the error is occuring when I create the KeplerianOrbit instance; but actually it occurs during the pm.Model creation phase; exactly during

light_curves = xo.LimbDarkLightCurve(u).get_light_curve(orbit=orbit, r=r, t=times)

On line 31 from Cell 3 in the same colab notebook as above.

---

Full Traceback

AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/theano/gof/op.py in __call__(self, *inputs, **kwargs)
    624                 try:
--> 625                     storage_map[ins] = [self._get_test_value(ins)]
    626                     compute_map[ins] = [True]

7 frames
/usr/local/lib/python3.6/dist-packages/theano/gof/op.py in _get_test_value(cls, v)
    580         detailed_err_msg = utils.get_variable_trace_string(v)
--> 581         raise AttributeError('%s has no test value %s' % (v, detailed_err_msg))
    582 

AttributeError: Elemwise{mul,no_inplace}.0 has no test value  
Backtrace when that variable is created:

  File "/usr/local/lib/python3.6/dist-packages/ipykernel/ipkernel.py", line 196, in do_execute
    res = shell.run_cell(code, store_history=store_history, silent=silent)
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/zmqshell.py", line 533, in run_cell
    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2718, in run_cell
    interactivity=interactivity, compiler=compiler, result=result)
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2822, in run_ast_nodes
    if self.run_code(code, result):
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2882, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-8-866db5884fd4>", line 2, in <module>
    times, synthetic_eclipse, orbit = build_synthetic_model(size=n_pts)
  File "<ipython-input-4-951c6ce8da63>", line 38, in build_synthetic_model
    omega=1.5707963267948966  # == pi/2
  File "/usr/local/lib/python3.6/dist-packages/exoplanet/orbits/keplerian.py", line 207, in __init__
    self.cos_incl = self.dcosidb * self.b


During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-10-13347aa48c42> in <module>()
      4                                               draws=draws,
      5                                               target_accept=target_accept,
----> 6                                               run_mcmc=run_mcmc)

<ipython-input-3-8f62b59eca01> in run_pymc3_with_gp(times, data, dataerr, orbit, log_Q, tune, draws, target_accept, u, run_mcmc)
     31             return pm.math.sum(light_curves, axis=-1) + mean
     32 
---> 33         light_curves = lc_model(times)
     34 
     35         # The likelihood function assuming known Gaussian uncertainty

<ipython-input-3-8f62b59eca01> in lc_model(times)
     25             # Compute the model light curve using starry
     26             light_curves = xo.LimbDarkLightCurve(u).get_light_curve(
---> 27                 orbit=orbit, r=r, t=times
     28             )
     29 

/usr/local/lib/python3.6/dist-packages/exoplanet/light_curves.py in get_light_curve(self, orbit, r, t, texp, oversample, order, use_in_transit)
    103                 tt.zeros_like(r), t.ndim
    104             ) + tt.shape_padright(tt.zeros_like(t), r.ndim)
--> 105             inds = orbit.in_transit(t, r=r, texp=texp)
    106             t = t[inds]
    107 

/usr/local/lib/python3.6/dist-packages/exoplanet/orbits/keplerian.py in in_transit(self, t, r, texp)
    605                 self.cos_omega,
    606                 self.sin_omega,
--> 607                 self.cos_incl + z,
    608                 self.sin_incl + z,
    609                 R + r,

/usr/local/lib/python3.6/dist-packages/theano/tensor/var.py in __add__(self, other)
    126     def __add__(self, other):
    127         try:
--> 128             return theano.tensor.basic.add(self, other)
    129         # We should catch the minimum number of exception here.
    130         # Otherwise this will convert error when Theano flags

/usr/local/lib/python3.6/dist-packages/theano/gof/op.py in __call__(self, *inputs, **kwargs)
    637                         raise ValueError(
    638                             'Cannot compute test value: input %i (%s) of Op %s missing default value. %s' %
--> 639                             (i, ins, node, detailed_err_msg))
    640                     elif config.compute_test_value == 'ignore':
    641                         # silently skip test

ValueError: Cannot compute test value: input 0 (Elemwise{mul,no_inplace}.0) of Op Elemwise{add,no_inplace}(Elemwise{mul,no_inplace}.0, InplaceDimShuffle{x}.0) missing default value.  
Backtrace when that variable is created:

  File "/usr/local/lib/python3.6/dist-packages/ipykernel/ipkernel.py", line 196, in do_execute
    res = shell.run_cell(code, store_history=store_history, silent=silent)
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/zmqshell.py", line 533, in run_cell
    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2718, in run_cell
    interactivity=interactivity, compiler=compiler, result=result)
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2822, in run_ast_nodes
    if self.run_code(code, result):
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2882, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-8-866db5884fd4>", line 2, in <module>
    times, synthetic_eclipse, orbit = build_synthetic_model(size=n_pts)
  File "<ipython-input-4-951c6ce8da63>", line 38, in build_synthetic_model
    omega=1.5707963267948966  # == pi/2
  File "/usr/local/lib/python3.6/dist-packages/exoplanet/orbits/keplerian.py", line 207, in __init__
    self.cos_incl = self.dcosidb * self.b

[exowanderer]

GoodNews/BadNews:

I made two simplified colabs

1. Problem Colab Notebook
2. Solution Colab Notebook

The problem colab reproduces the error repeatedly. I think that every time I hit Restart and Run all, I incurred the same error seen above.

Important factors

1. The error occurs during either the creation of the model (.get_light_curve) or the assignment of that output to a deterministic variable (pm.Deterministic('light_curve', light_curve)).

2. the problem colab creates the orbit = KeplerianOrbit(...) at the same time as the synthetic_model (to generate the data). It then passes that orbit to the pm.Model() wrapper function.

3. the solution colab creates one KeplerianOrbit(...) for the synthetic_model and one KeplerianOrbit(...) for the pm.Model() sampled light curve.

I think that my 2nd (or 3rd?) hunch was close to home -- and very related to your intuition. If we use the exact same orbit to generate the synthetic model, via light_curve = light_cruve.eval(), as the one used in the pm.Model() sampled light curve, then this error occurs.

In all (?) of your tutorials, you create the orbit on the fly with the pm.Model() sampled light curves. I did not, because I wanted to retrieve 'known' values for the physical parameters and kernel hyperparameters, in the hope that I can intuitively understand the relative scale of the auto-correlated noise in my real data (by verify it in the synthetic data). I wanted to keep everything the same.

Solution: I need to use two, independent KeplerianOrbit instances for my analysis. One to create the synthetic transit light curve; and another to model that light curve with PyMC3.

If you agree with my assessment, then we can close this issue. If you think that I've missed the puck, I'll keep digging.

[dfm]

Let's keep this open - I haven't been able to figure out why that error gets thrown even though it shouldn't (it seems to have something to do with how pymc3 changes the theano settings) so I'd like to figure it out!