pcubillos / mc3 Goto Github PK

The following line:
sys.path.append("./examples/example01/")

Should be:
sys.path.append("./examples")

On topic, search for other inconsistencies.

It would be very helpful to have indparams accept a dictionary so we can easily change input values in a function without inputting every possible argument in an ordered list.
something like indparams = {'x':x, 'a':False, 'Temp':100}

I am trying to use the implemented wavelet algorithm to estimate the red noise in exoplanet light curves. So I give inputs for gamma, sigma_w and sigma_r, but I always get a negative calculated chi**2 (although it seems the mcmc runs normally), any idea ?

Thank you !

These functions seem more standard (and probably are flexible enough, or maybe more) than the current code.

Hello,

I got the error

File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/mc3/utils/log.py", line 45, in init
self.file = open(self.logname, "aw")

ValueError: must have exactly one of create/read/write/append mode

while trying to resume a previous run. Solved it just by replacing
self.file = open(self.logname, "aw")
with
self.file = open(self.logname, "a")
line 45 in log.py

I thought you might be interested. Thank you for this great mcmc package btw

François

Hello,
I am trying to generate pure 1/f component as described in your paper: random numbers to produce wavelet coefficients respecting the variances given by equations (10) and (11). I quite struggle to actually get it right. The piece of code I have is so far (which does not quite reproduce your figure 2) is:

import dwt as dwt
def create_rednoise(n, sigma_r, sigma_w):
"""
create correlated noise component
"""
M = np.int(np.ceil(np.log(n) / np.log(2)))
if 2**M != n:
print "size is not a power of 2"

    gamma = 1
    gg = 1/(2*np.log(2))

    wv = np.zeros((n))

    sm2 = sigma_r**2 * 2**(-gamma) * gg + sigma_w**2
    wv[0] = np.random.normal(0, np.sqrt(sm2))

    k = 1
    for i in np.arange(1,M+1):
            sm2 = sigma_r**2 * 2**(np.int(-gamma*i)) + sigma_w**2
            for m in np.arange(0,2**(i-1)):
                    wv[k] = np.random.normal(0, np.sqrt(sm2))
                    k += 1

    redcomp = dwt.daubechies4(wv, -1)
    return redcomp

rednoise = create_rednoise(1024, 1.6e-3/3, 1.6e-3).

Thanks for your help !
Hugo.

You can get the size of a label with this:
text_size = yl.get_window_extent().height / fig.get_window_extent().height

which you can compare to the size of its axes:
axes_size = ax.get_position().height

The idea is to automate the size of captions when there are few/many subplots, to avoid having overlapping/tiny captions.

Throw a warning and stop the execution (before the MCMC) if the initial guess for a free parameter lies out of bounds.

Remove the hacky 'return' statement from line ~535:

• Make sure that meanp and uncertp work properly.
• Make sure the plotting functions work.
• Comment out the savemodel option (enabling this is part of a different issue)

All of this depends mostly on correctly passing the Z array to the functions.

Implement the algorithm from: ter Braak & Vrugt (2008)
Differential Evolution Markov Chain with snooker updater and fewer chains

Also, take a look at Nate's code.

Hi,

I've just downloaded the code and I have a problem running it. It doesn't find the module '_binarray'. What is the problem?

Thanks,
Peter

import mc3
Traceback (most recent call last):
File "", line 1, in
File "/Users/klagyi/astro/pythoncodes/mc3/mc3/init.py", line 4, in
from .sampler_driver import *
File "/Users/klagyi/astro/pythoncodes/mc3/mc3/sampler_driver.py", line 21, in
from .fit_driver import fit
File "/Users/klagyi/astro/pythoncodes/mc3/mc3/fit_driver.py", line 9, in
from . import stats as ms
File "/Users/klagyi/astro/pythoncodes/mc3/mc3/stats/init.py", line 5, in
from .stats import *
File "/Users/klagyi/astro/pythoncodes/mc3/mc3/stats/stats.py", line 26, in
import _binarray as ba
ModuleNotFoundError: No module named '_binarray'

Currently the Z array is a 2D array [Nsamples, Nfree]. It need to be reshaped into a 3D array (separating the different chains). See the GR documentation to find out the required input shape.

A simple change. Just to be consistent with the updated documentation.

Hi,

I apologize if this happens to be a trivial issue or due to some lack of understanding on my part.

Nevertheless, an issue that I have found is that if you create a Fortran based library/module using f2py (which I have tested on python) and use it in defining 'func', MC3 doesn't work.

For instance, I created a Fortran based module for multiplication - say 'y=fort_mult(a,b)' (that gives y=a*b as its output). I then used it with the example you have in your documentation where you define a func quad where you calculate 'y=p[0] + p[1]*x + p[2]*x2', i.e. I replaced this defintion with 'y=fort_mult(p[0],1) + fort_mult(p[1],x) + fort_mult(p[2], x2)'. But once I do this, MC3 stops working. Ofcourse, my actual Fortran code is much more involved but I also tested the code with the example I just explained.

Hope you can resolve this!

Thanks!

Some of the characters in the documentation PDF can't be copied into Python. Likely this is just an issue with the tex file being compiled, but it seems that the hyphen (-) and single quote (') get messed up somehow.

Also some spacing issues with the sys.path.append() commands. The PDF has some spaces in the path that shouldn't be there.

Hi,

I wrote a now-closed issue not so long ago about the red noise fitting using the wavelet algorithm. Let's say I fit a light curve using that method by fixing gamma=1, it will return the sigma_w and sigma_r parameters.

Would it be possible to get back the corresponding correlated and white noise values at each input data points (i.e. the time series) ? How could one do that ?

Thank you !
Best,
Hugo.

If we keep it in memory we can enable the savemodel argument.
Shared memory is the best option I can think off (certainly better than sending data through pipes).

Is it possible to synchronize the DEMC chains iteration (wait on each iteration for all chains to be done) without using Pipe calls?
I tried with locks and such for a day, but I failed miserably ...

I am trying to fit a complex function with different observational streams. Additionally, likelihoods might not be Gaussian, and priors might be different to uniform or Gaussian only.

I have code to calculate the log-likehood (and Log Posterior). Is there any way to use mc3 for this? Or is the best way to just extract the sampler I want to use and wrap it for this application?

how can I use demc method in windows？I can not intsall mc3 in windows！

Test what happens if we implement DEMC but without synchronizing at the end of each iteration. This deviates from the original algorithm of ter Braak (2006). But still, check if the results looks ok. This should be much faster than the current version of DEMC (the pipe's communications are a bottleneck).

Hello, after using pip to install mc3 I get the error "Failed building wheel for mc3". Any ideas on how to fix this?

It would be nice to have an option to run until convergence is reached. Something like "run N iterations, then check for convergence every i iterations (with a flag for quitting on convergence), until P iterations (if specified) is reached." That way we wouldn't have to do many runs to narrow down on the required iterations.

See the implementation in the master branch, export to the multiproc branch.

Hi Patricio, how are you. I have developed a new package, PyIRoGlass, for fitting baselines to FTIR spectra with MC3. I have created a corresponding Google Colab for running this code on the cloud. The Google Colab version appears to have stopped working as I am now getting this error:

ValueError: Cannot populate an initial sample set of parameters, try updating the parameters initial guess to avoid sampling beyond the parameter boundaries or where the model returns non-finite values.

Do you know what changes in 3.1.1 could be returning this new error? Nothing in my code has changed with the initial set of parameters. The local version of the code also remains functional, as I am running 3.0.13 and have not updated to 3.1.1 yet. I have used the 'trf' option in the mc3.sample function and all the least squares best-fitting parameters are further within my range of initials. Here is what is returned from least squares:

Least-squares best-fitting parameters:
[1.5850219, -0.0199404429, 0.711353268, -0.229082749, 
-0.00998274823, 1428.13013, 30.0187476, 0.108494754,
 1517.87931, 34.8462181, 0.110622216, 0.673443334,
 0.0359489381, 0.0949624095, 0.000125732721, 1.25690699]

compared against what I provide as initials:

params = np.array([1.25, 2.00, 0.25, 0.01, 0.01, 1430, 25.0, 0.0100, 1510, 25.0, 0.0100, 0.10, 0.02, 0.01, 5e-4, 0.70])
pmin = np.array([0.00, -5.00, -1.00, -0.75, -0.75, 1415, 22.5, 0.0000, 1500, 22.5, 0.0000, 0.00, -0.50, -0.50, -5e-2, -1.00])
pmax = np.array([5.00, 8.00, 1.00, 0.75, 0.75, 1445, 40.0, 3.0000, 1535, 40.0, 3.0000, 3.00, 0.50, 0.50, 5e-2, 3.00])
pstep = np.array([0.30, 0.50, 0.20, 0.20, 0.20, 3.25, 2.25, 0.0005, 6.0, 2.25, 0.0005, 0.25, 0.75, 0.75, 0.002, 0.20])

Any insight you have into this would be immensely helpful!

How about this?, Stop an MCMC if:

• GR < a threshold more strict than the 1% convergence (0.1%?)
• number of current samples > user-set minimum number of samples (half?, 20% of nsamples?)

Make this happen as early as possible in the code.

I'm a newbie to mc3. When I try to install it, I get

.....
creating build\temp.win-amd64-3.8
creating build\temp.win-amd64-3.8\Release
creating build\temp.win-amd64-3.8\Release\src_c
"C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.30.30705\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users...\AppData\Local\Temp\pip-build-env-pzxyxxq1\overlay\Lib\site-packages\numpy\core\include -Isrc_c/include/ -Ic:\users...\python38\include -Ic:\users\fenwick\appdata\local\programs\python\python38\Include -IC:\Users...\AppData\Local\Temp\pip-build-env-pzxyxxq1\overlay\Lib\site-packages\numpy\core\include -Isrc_c/include/ -Ic:\users...\appdata\local\programs\python\python38\include -Ic:\users...\appdata\local\programs\python\python38\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.30.30705\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.30.30705\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt" /Tcsrc_c/_binarray.c /Fobuild\temp.win-amd64-3.8\Release\src_c/_binarray.obj -ffast-math
cl : Command line warning D9002 : ignoring unknown option '-ffast-math'
_binarray.c
src_c/include/stats.h(98): error C2036: 'void *': unknown size
src_c/include/stats.h(99): error C2036: 'void *': unknown size
src_c/include/stats.h(102): error C2036: 'void *': unknown size
src_c/include/stats.h(103): error C2036: 'void *': unknown size
src_c/include/stats.h(105): error C2036: 'void *': unknown size
src_c/_binarray.c(57): error C2036: 'void *': unknown size
src_c/_binarray.c(59): error C2036: 'void *': unknown size
src_c/_binarray.c(60): error C2036: 'void *': unknown size
src_c/_binarray.c(70): error C2036: 'void *': unknown size
src_c/_binarray.c(71): error C2036: 'void *': unknown size
src_c/_binarray.c(73): error C2036: 'void *': unknown size
src_c/_binarray.c(74): error C2036: 'void *': unknown size
src_c/_binarray.c(76): error C2036: 'void *': unknown size
src_c/_binarray.c(77): error C2036: 'void *': unknown size
error: command 'C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.30.30705\bin\HostX86\x64\cl.exe' failed with exit code 2

ERROR: Failed building wheel for mc3
Failed to build mc3
ERROR: Could not build wheels for mc3, which is required to install pyproject.toml-based projects

Any thoughts on how I can resolve this?

Thanks!

Find out if and how can this be done.

Hi Patricio,

I am a PhD student of Monika Lendl and I work my work I use Monikas photometric routine CONAN that uses MC3. When I started using it in my Mac (previously in UBUNTU) I keep having the following error:

    RuntimeError: 
    An attempt has been made to start a new process before the
    current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

The error originates fro the multiprocessing package that MC3 uses. Do you know how I can solve this issue?
Thank you very much in advance!

Best,
Angelica

Hi Patricio,

Thanks for the updated version, very nice !
I have switched to Python 3.6 and encounter a segmentation fault when using the dwt_daub4 function in stats.py .
For instance:

import numpy as np
import mc3 as mc3

res = np.random.randn(64)*100e-6
dwt1 = mc3.stats.dwt_daub4(res,False)

Which returns, when running in debug with gdb (i.e. gdb --args python3 script.py):

Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
daub4 (self=<optimized out>, args=<optimized out>) at src_c/_dwt.c:181
181	src_c/_dwt.c: No such file or directory

I need dwt_daub4 to compute the time series of my correlated noise, after estimating sigma_r and sigma_w.

Thanks a lot !

Use OOP to avoid the following warning when plotting data in multiple subplots (using plt.axes to move back and forth between panels):

MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance. In a future version, a new instance will always be created and returned. Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance. warnings.warn(message, mplDeprecation, stacklevel=1)

Hi Patricio,
It seems there is a problem installing MC3 on Windows via pip or Conda. It gives an error about needing a C++ compiler. It might be helpful to have a note for Windows users in the getting started section about how to resolve this.

The error is this:

  building 'mc3.lib.timeavg' extension
  error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/

This can be easily rectified by installing the latest version from this page and then retrying the pip or Conda install. I found this by looking at answers here.

Thanks for making such a great package,
Henry

See the numaccept variable as an example.

Hi again Patricio,

I just updated to new version of mc3, very nice thanks !
I think there is a typo in the stats.dwt_chisq function. It should be:

iprior = (priorlow > 0) & (priorup > 0)
    dprior = (params - priors)[iprior]
    return dwt.chisq(params, model, data, dprior, priorlow**[iprior]**, priorup**[iprior]**)

(the [iprior] have been forgotten if I am correct). Otherwise the chisq returned is 'nan'.

Best,
Hugo.

Hi,
I see that you have not tested this in Windows, however would it be possible for you to offer a .whl file for this package?