Cross Validation score for MaximumLikelihoodMSMs and BayesianMSMs about deeptime

[clonker]

Cheers, you are right, I have never added an example regarding that! My bad! For the time being, you can check this notebook: https://github.com/markovmodel/pyemma-workshop/blob/master/notebooks/02-io-features-hands-on.ipynb

The relevant bit is this:

from deeptime.decomposition import TICA, vamp_score_cv

fig, axes = plt.subplots(1, 3, figsize=(12, 3), sharey=True)
labels = ['backbone\ntorsions', 'heavy Atom\ndistances']
tica_estimator = TICA(lagtime=lags[0], dim=dim)

for ax, lag in zip(axes.flat, lags):
    tica_estimator.lagtime = lag
    torsions_scores = vamp_score_cv(tica_estimator, trajs=bbtorsions, blocksplit=False, n=3)
    scores = [torsions_scores.mean()]
    errors = [torsions_scores.std()]
    distances_scores = vamp_score_cv(tica_estimator, trajs=heavy_atom_distances, blocksplit=False, n=3)
    scores += [distances_scores.mean()]
    errors += [distances_scores.std()]
    ax.bar(labels, scores, yerr=errors, color=['C0', 'C1', 'C2'])
    ax.set_title(r'lag time $\tau$={}ps'.format(lag))

axes[0].set_ylabel('VAMP2 score')
fig.tight_layout()

You can provide an estimated MSM and/or bayesian MSM as well.

Reference: https://deeptime-ml.github.io/latest/api/generated/deeptime.decomposition.vamp_score_cv.html

[prateekbansal97]

Hello!

Thanks for the reply. I was able to implement the suggestion.