Unable to load previous run during batch stage? about dynesty

[segasai]

It seems you are using the wrong restore method "dnst.NestedSampler.restore"
instead of dynamic sampler's restore

[ajw278]

Apologies, silly mistake. Thanks.

[segasai]

actually I'm not sure which restore you use makes a difference, so there is possibly still a bug there...

[ajw278]

Actually, yes, it seems it doesn't help changing to DynamicNestedSampler.restore, however it is unclear if I already 'corrupted' the initial file by trying to load it with NestedSampler. If this cannot be the case, then it seems there is a bug..

[segasai]

I don't think the file will be corrupted this way.

Do you have a way to reproduce this issue ?

Did you also actively interrupt this run ? or did it stop due to say maxcall or maxiter limit ?

[ajw278]

In that case I would think it is a bug.

Reproducing is complicated, because my run takes a long time to complete the initial stages. The code is also quite large and dependent on data files, I can provide my code, data and the save file from the previous run privately?

It ended due to the max number of batches being executed. I wanted to try extending the number of batches (from 10-> 20).

[ajw278]

PS. If you have a test problem with the dynamic run, you could try restarting it and extending the number of batches to see if it does the same thing.

[segasai]

Okay, that's the problem.

the resume can only be used to resume an interrupted run, not to continue a run with different parameters.

I can now even see where the error is coming from. The batch sampling was finished, the batch_sampelr was deleted.
There is nothing to resume to.

[segasai]

It is certainly fixable, but the whole save/restore is very tricky because of the number of possible states of the system.
So I think there is only a bug that the more helpful error message is needed.

[ajw278]

I thought it would be something like this - thanks for explaining!

[segasai]

Basically if you want to continue running the sampler (adding more batches) after a successful run, I think you can still
.restore and do add_batch().
I have also committed a change 1005fd1 that will issue a warning and will exit if you try to resume the nested run that is finished,

[ajw278]

Thanks!