RAR interval argument vs log-every argument about pinnacle

[i207M]

Sorry for the late response. It's been a hectic month for me. First, thanks for your interest in using our benchmark and digging into the implementation details to help.

Main issue

Your diagnosis is right. The reason is that, RAR is implemented by calling train multiple times, so multiple on_train_end as well. This can be fixed by modifying the callbacks so they don't produce outputs when there's no history. But I feel like there will be another issue that, once reached log_every epochs, each train call (for example every 10 epochs) will produce some outputs, which may be too much. So I suggest also checking log_every in on_train_end and if the last call is shorter than log_every, skip this run. What's your opinion?

Other issues

Agree with things about argparse and docs.

The summary issue is related to DeepXDE which defaults to tensorflow. This will fix it

os.environ["DDEBACKEND"] = "pytorch"

I will check the RAR implementation issue later soon. Yes, there is a mismatch between the algorithm and the implementation. The points should be sampled from the real domain instead of X_train. We will work out a fix for this.

Sorry we are unable to provide the exact scripts to generate table results, but you can find some useful functions in scr/utils/plot.py and scr/utils/summary.py.

Thank you once again and PRs are more than welcome.

[svdcvt]

Hi! Thanks for your reply.
In the end, I just wrote a callback for resampling. I also added other algorithms for adaptive sampling. I might do a pull-request at some point later when it is ready and proved to work (currently writing the paper). But you can check my fork for that.

[i207M]

That's great. Please do leave an arxiv link if available. Good luck with your paper!