alchemistry / alchemical-best-practices

Best practice document for alchemical free energy calculations going to livecoms journal

Best Practices for Alchemical Free Energy Simulations

Goals

• Checklist and background for new practitioners of relative and absolute alchemical free energy calculations: What should you pay attention to in setting up and running calculations in common codes and why.
• Provide guidance to authors as to what should be reported about their protocols in a Methods section, either conforming to standard or reporting where they did not conform and why. Useful to reviewers as well.

Scope

Preparation (focused on aspects unique to alchemical free energies), execution, and analysis of:

• Transfer free energies (hydration free energies, partition coefficients, etc.)
• Binding free energies

List of Authors

(As ordered on the paper)

• Antonia S. J. S. Mey
• Bryce Allen
• Hannah E. Bruce Macdonald
• John D. Chodera
• David F. Hahn
• Maximilian Kuhn
• Julien Michel
• David L. Mobley
• Levi N. Naden
• Samarjeet Prasad
• Andrea Rizzi
• Jenke Scheen
• Michael R. Shirts
• Gary Tresadern
• Huafeng Xu

List of Contributors

(non-author list of people who contributed to document)

• Julia Rice -- Attended initial discussion meeting at NIST in Aug. 2017

Paper Writing as Code Development

This paper is being developed as a living document, open to changes from the community. You can read more about the concept of writing a paper in the same way one would write software code in the essay "Paper writing as code development". If you have comments or suggestions, we welcome them! Please submit them as issues to this GitHub repository so they can be recorded and given credit for the contribution. Specific changes can be proposed via pull requests.

Online Resources

Original brainstorming document: (https://docs.google.com/document/d/1h7GIyOJykBWz-GPKtYZK-Pv7cbLbX8XTM2xLUpSppoE)

List of Released Versions

revisions for v 1.0

alchemical-best-practices's Issues

Section 6 Simulation prerequisites

What are the typical steps to setup a simulation either for absolute or free calculations

8.4 Estimators for free energies

TI, MBAR, BAR, other estimators?

[Sec:1 and Sec:2] Review and check flow

Section 1 and 2 are in a late draft stage, but happy for changes should they make improvements to the paper. Can @everyone check references, grammar, and general flow.

[Figure X] Sampling schemes figure [help]

I am trying to converge on some of the figures, before finally sending out a paper draft.

I have created the following figure to illustrate different sampling schemes for alchemical free energies. Happy to hear some thoughts on it for improvements.

sampling_schemes.pdf

Figure - relative free energy thermodynamic cycle

I don't really want to volunteer people for figures, I am just putting it out here that they will need to be created or adapted from existing ones.

Section 7.+ Stopping conditions section for simulations

What are typical simulation stopping conditions?
Data saving and frequency, what is essential for analysis?

Figure (s) - typical plots for analysis

Example plots for a typical analysis.

Section 7.1.0 What is a dual and single topology

I think it would be beneficial to add a bit on describing how single dual and mixed topologies work. Exact location in the manuscript to be determined.

Authorship proposal

I would propose that Toni get first authorship (or last, whichever is more useful for her at this point) and the rest of us be ordered alphabetically. I think there is a difference in quantity there.

We can hold off such a decision until it's ready to submit, though.

[Figure 1] What types of simulations can be run using alchemical simulations

This will be the first figure in the paper and it should illustrate what different types of free energies can be estimated using alchemical free energy protocols and therefore give a broad overview where the method can be useful.

I have drafted a very crude outline of the figure and @JenkeScheen has kindly agreed to flash it out a little. The basic idea is a cartoon representation of the type of simulations one can do with alchemical free energy calculation on the left and then the type of free energy difference computed on the right from the simulations.

Currently I have these types of simulations:

• binding free energies (absolute and relative)
• hydration free energies (or logP logD)
• mutations
• membrane insertions
  What else should be on this list?

The rough idea of what things should look like is here:
Fig1.pdf

Definitely not in any kind of final state though and thanks @JenkeScheen for prettifying it.
Again any kind of input would be much appreciated here.

Virtual meetup

Hi all,
apologies for Spamming. Trying to get the polling feature to work.
it would be great to get some people together for a virtual meetup. I would propose the following two dates and if at least 3+ people can join we can go ahead with a virtual discussion and share the outcome with the rest.

I'll pick the option with most people available, currently by default I can make all of the suggested times:

Wednesday, January 30th options:

Monday, February 4th options:

8.3 Typical ways of examining output ways and useful plot

Examining output data for common problems with discussions of what exactly to plot or look at; examples of typical curves for dV/dlambda and free energy versus lambda, etc.

[sec:8] Missing good analysis practice

This is more of a general questions. We discuss how to analyse the data and check if the simulations make sense quite thoroughly. I feel there should also be a small subsection on best practices of how to show the results. I was thinking something along these lines here:
http://practicalcheminformatics.blogspot.com/2019/02/some-thoughts-on-evaluating-predictive.html

And maybe other topics such as, how to compare to experimental results, what should I look at DG , DDG, what kind of errors, what kind of plots make sense. Should this be included, or would you say this kind of discussion is out of scope?

[Sec:8] overall review required

This section has a lot of nice bits, but also still needs a bit of work. Can you again check consistency and references for this section in general? I'll add a couple issues for subsections as well.

[Sec:4 Can I afford the calculation?]

What are peoples thoughs on some example calculations regarding affordability? While reading the manuscript I definitely agree that it would be super useful to calculate affordability, but I am not really given the tools on how to do it, or where to look for this information. (Maybe this is fairly obvious, but certainly in terms of synthesis I have no idea myself. )

test

Hi all,
it would be great to get some people together for a virtual meetup. I would propose the following two dates and if at least 3+ people can join we can go ahead with a virtual discussion and share the outcome with the rest.

I'll pick the option with most people available, currently by default I can make all of the suggested times:

Wednesday, January 30th options:
/poll "11:30am EST" "3:00 pm EST" "4:00 pm EST"

Monday, February 4th options:
/poll "11:00am EST" "12:00 pm EST" "1:00 pm EST"

formalize subsection vs subsection*?

We should decide if we want numbering or not on subsections. I kind of like having numbers, but the most important part is to be consistent.

all figures and their captions

It would be good to check the following for all figures:

1. Are figures legible?
2. Do the figures make sense?
3. Are figure captions sufficiently descriptive?
4. Are figures mentioned in the text appropriately?

Question on documents

There are two documents: 'manuscript.tex' and paper/manuscript.tex. It looks like the one in the main directory is more complete, but doesn't compile because of the lack of the style sheet. Do I have permission to copy 'manuscript.tex' over to 'paper/manuscript.tex' and delete the redundant file?

Figure 9 -- data and legibility

Figure 9 currently only has a pdf and not the underlying data and plotting script available. The labels are pretty small and difficult to read. @brycestx I believe you made the figure, can you help out?

compiling document fails with missing figures

Hi all,

As it stands, trying to build a pdf of the document doesn't work well because the figures are referenced as if you're building from the top level directory and saying
pdflatex paper/manuscript

, but livecoms.cls, the bib file, and the bst file are in paper, as if you were building inside the paper directory.

I worked around this by copying the cls and bst files into the top level directory, but going forward you probably want to fix this. However, the refs don't render in the document, at the moment.

raw data for figure 7 -- convergence of DG from SAMPL

I have been going through figures again. It would be quite useful to get the data for all of the figures, so that if in doubt it would be possible to redo some of the plots. I may have had this data already, but can't find it now. Who can help?

8.6 Cycle closure errors

A subsection on discussing how to deal with cycle closures.

[Figure 2] Illustrating lambda scheme figure [help]

As one of the figures I wanted a clear conceptual figure that explains how the lambda switching works, but using a concrete example to make it easier to understand. I also wanted to make a connection to thinking of it in terms of probability distributions to then make the concept of overlap easier to understand later on.

Below is my first pass at this figure. I would appreciate any comments, or suggestions for improvements.

Fig2.pdf

Pick relative or absolute, weak binders, and restraint bias

Might be early for rounds of comments, but I wanted to save these somewhere. I'm not a practitioner of relative free energy calculations so apologies if some of this doesn't make sense, but here are a couple of points that might need to be clarified in the paper.

• Section 5/Checklist: Since the paper is targeted towards new practitioners, it might be useful to give a broad overview that guides choosing the right type of calculation by highlighting the main differences between relative vs absolute calculations with pros and cons. For example:
  • The quantity of interest (i.e. DeltaG vs DeltaDeltaG).
  • Typical convergence performance on small transformations and transformations that affect the ligand scaffold and/or binding mode.
• Section 7: With weak binders, do relative free energy calculations require a restraint too? If not, how should people handle them? Also, at least for absolute calculations, should we mention the dependency of the final free energy on the restraint parameters?
• Section 8: Should we mention that restraints can introduce a bias in the estimated free energy? We've observed this with harmonic restraints. The correction is usually negligible with flat-bottom restraints when they have "steep" walls. I'm not sure about orientational restraints like Boresch.
• Section 10: YANK includes setup and analysis tools, although they mostly work only for YANK calculations so I'm not sure we want to include them separately.

Write introduction

[Sec:3] Theory Needs expansion

This section still needs a little expansion. I am happy to work on it, and have an idea of hopefully a relatively easily digestible approach to it, but if people get there before also happy with that. I'll loosely assign myself in any case.

Section 10: Available software summary

This section should include a selection of alchemical free energy simulations software commercial and free, setup tools for these and analysis tools, with links to installation instructions and publications where available.

Reference "basic simulation training" rather than giving a list here?

We have a prior article in LiveCoMS on "basic simulation training" that intends to provide an overview of the basic fundamentals people should know before even starting, along with giving a list of books/materials people may want to read. I wonder if section 2 ought to simply refer to that rather than trying to provide its own, separate list of texts people should read? It seems redundant to do so.

See eg https://github.com/MobleyLab/basic_simulation_training

@ppxasjsm thoughts?

[sec:8:estimators] references

@mrshirts I think I figured out most of the references in this section, but a couple eluded me. Could you please add them and just generally look over the section?

Reference management?

Just checking how are the references being stored and managed?
We created https://www.zotero.org/groups/2207075/best_practices_in_free_energy_calculations, but I'm not sure how/if we are using it I note that Toni is a group member, and added a bunch of them, but none recently, and there are only 5 members (David Mobley, myself, Toni, Julien, Bryce Allen).

Toni, what's the current process for managing references among the group?

Figure - absolute free energy thermodynamic cycle

Section 5 What can be expected from alchemical simulations

@davidlmobley I think it would be great if Huafeng could write part of this section. I will email him separately, but just putting it here so that I remember.

[sec:The importance of the alchemical pathway]

This section needs work. It is rather general, does not explain concepts well, or give good recommendations that are easy to follow.
@brycestx , @andrrizzi, @Lnaden, @jchodera, and @huafeng, can you take a look at this? I would quite like a bit more detail on softcore potentials, what do they look like, how do you actually use the lambda to switch between two states. As far as I am aware there are no good recent resouces for this kind of information and I feel a best practices paper would be a good place for this. The section is also currently lacking citation and I would be happy to have some additional ones, or better suggestions for the ones that currently exist.

[Sec4:, sec:5, sec:6] tidy up

Section 5 looks at some aspects specific to drug discovery which is nice. It would be good if all of these sections could be reviewed a little and made more concise where possible and improved in structure. I would suggest for people who wrote these sections to do this.

Furthermore, it would be great if @davidlmobley and @jmichel80 could double check the references I have put in, in these sections and maybe help with pointing out any additional ones needed. This would be massively appreciated. Thanks.

[Fig: perturbation network] Review

I added an additional figure to the paper today showing two types of perturbation networks with @JenkeScheen help. At the moment it is quite plane looking. Should chemical structures or cycle closures or the like be added for ease of discussion in the analysis section?

Figure.pdf

8.5 Uncertain estimation

What are good strategies to report uncertainties on DG and DDG?

Next steps

Hi all,

apologies for the long silence, but this is an attempt to revive this manuscript and get it to a publishable state for the next issue of livecoms. I will identify tasks in the next days and would be happy to assign them to willing volunteers (Including myself). Are people ok with this idea? @andrrizzi, @jchodera, @davidlmobley, @Lnaden, @mrshirts, @brycestx

Also would everyone be interested in a virtual chat in the next week or two or so? I'd be happy to arrange this. However, I am also fine to just work with github (or emails if this is preferred), but it seemed easier to get everyone who contributed to this so far tagged. Any thoughts?

If you want to invite anyone else to work on this please go ahead.

Best,
-Toni

Review section on perturbation maps and cycle closure

Section 7.1 and 8.3 both can use some improvements. Help with this would be appreciated

How best to add new text?

Toni, what's the best way to add text over the next 10 days? Should we do it on Overleaf, or through PR? Is either mechanism OK?

Reference inclusion in the manuscript

Adding appropriate references.

4.1 and 4.2 subsection on de-correlation and subsampling

Subsection on: Equilibration detection, decorrelation and subsampling

New templates?

New templates were just released a couple of weeks ago: I can go ahead and install them if that's OK.

[Sec:4 subsection] reproducibility:

There is a subsection in sec:4 on reproducibility that hasn't been filled in. Can @davidlmobley or @jmichel80 or both of you have a look at it? That would be fantastic. It shouldn't be more than a paragraph that is needed.

Some book keeping

A few things that will make life easier I suppose.

• Bibilography
  @jchodera / @andrrizzi could you share your bib file for the bibliography entries in the introduction?

-Conor Parks
He emailed early on saying that he will not be contributing. I will remove him from the author list and mention him in the acknowledgements instead, since he was part of the original meeting.

-Sections:
@ppxasjsm will complete section 5 and 6 what can be expected from alchemical fee energy simulations and simulation prerequisites
@mrshirts said he will contribute on the analysis techniques
@brycestx, @andrrizzi have some to do's assigned in the tex.

If in your section you use abbreviations please add them also to the abbreviations section.

@samarjeet do you intend to contribute to this? If so what bits will you be working on? Restraints?

Are we taking advantage of previous reviews?

We should try to make sure we have covered the important points from previous reviews (or reject the points because they are beyond scope or wrong), and also referred to them for certain information that is beyond the scope of what we want to study.

Particular papers:
"Best Practices in Free Energy Calculations for Drug Design": https://doi.org/10.1007/978-1-61779-465-0_26
"An Introduction to Best Practices in Free Energy Calculations". https://doi.org/10.1007/978-1-62703-017-5_11
"Good Practices in Free-Energy Calculations": http://dx.doi.org/10.1021/jp102971x
"Practical Aspects of Free-Energy Calculations: A Review": https://doi.org/10.1021/ct500161f
"Guidelines for the analysis of free energy calculations", https://link.springer.com/article/10.1007/s10822-015-9840-9

Probably others . . . .

[fig9_convergence]

There is This lovely convergence figure, which I think currently isn't referenced in the text, nor sits in a reasonable place in the manuscript. @andrrizzi could you please find a home for it?

write abstract

[sec7:] what simulation protocol should I use

I haven't spend that much time making changes to this section. At the moment bits of it feel not very precise and lack clear recommendations.

• I would suggest a summary recommendations subsection at the end.
• Maybe adding an illustration on restraints might be helpful, or a discussion in terms of what software supports these kind of restraints easy.
• check the notation, is it consistent with the theory section?

Fig8/Fig9 sampling convergence and forward and backward

Do we want both of these figures:
free_energy_trajectories.pdf
Forward_Reverse_Fig.pdf
If so, I need the data for the second one. Or is first figure replacing the second one?