mjschnie / ffx Goto Github PK

As of now, RotamerOptimization is a very large, very complicated class with insufficient encapsulation, with significant chunks written by at least five authors. Right now, it's either working or close to working, and we're currently using it for a project. However, even small feature additions have proven to induce bugs that take days to track down.

As such, I propose a temporary moratorium on adding features to RotamerOptimization.java, at least in the master branch. This, however, does not mean we cannot think about improvements to RotamerOptimization, possibly in a development branch of the repository.

Definite to-dos

• Refactor RotamerOptimization, breaking off chunks of the logic into smaller classes.
• Add meta-info to restart files (such as residue selection, algorithm, box/window parameters, cutoffs, etc).
• Remove the .partial files from restarting box optimizations; instead include all information in restart file(s).

Proposed to-dos

• Distance matrix class.
• External energy region class(es).
• Re-implement DEE purely as a mathematical operation on double and boolean arrays, with no reference to Residue, Rotamer, etc.
• Avoid creation of work unit maps.
• Think about scaling to more than a handful of worker processes (e.g. distribute storage of energies, instead of each worker having its own complete array/map of energies).
• Replace all of the energy/distance arrays with a Map-based implementation. For example, map "i,ri,j,rj" to an object containing its energy, its distance, whether it's eliminated, whether it's been nulled due to distance, and whether the energy has been calculated. May become more efficient if Project Valhalla succeeds in creating value types. With some testing, the Map-based implementation of 3-body energies has been shown to have noticeable performance impacts. Thus, we're generally sticking with arrays, some of them condensed using neighbor lists.

Attempting to call potential.energy(x) at the end of the Minimizer script results in failure, and prints out an error PDB file that looks reasonable... just scaled down and compressed. This suggests some issue with scaling/de-scaling in the Minimize class.

First: this would help immensely in applying our superpositionThreshold in rotamer optimization to self energies.

Second: it would be helpful to have a method, which uses the pre-existing van der Waals neighbor list, to find all the Atoms close to another Atom, plus the distances, and maybe their index in the atom array used by vdW. The return value of getNeighborLists is an int[][][], which is only really useful when you know the index of all the Atoms you are interested in.

While xyzIndex is sometimes a useable shortcut, the MultiResidue machinery interferes with this.

Our current testing suite is very comprehensive and thorough in its logging, close to 4 MB worth of comprehensive. The Travis service simply kills any tests that run over 4 MB, causing a "test failure" when it is simply a lot of logging.

This is occurring to my fork; the ThermodynamicsTest it adds is the straw that breaks the camel's back and brings logging over 4 MB in size.

I'd suggest we find where our logs are particularly verbose and either remove these logging messages, set them to the FINE level, condense the output, or add flags to do the above.

Adding flags is a bit inelegant, but due to how we've been using the standard logging levels, we may have to choose between that or defining a custom logging level, just beneath INFO, and changing some of the problematic logging messages to use that level.

Places which are particularly clogging our test output:

1. Prefaces to messages take up a lot of space. For example, this is 79 bytes:
   Jul 25, 2018 12:10:10 PM ffx.xray.SigmaAMinimize optimizationUpdate
   INFO:
2. Printouts of rotamer optimization energies and eliminations. Could definitely be condensed to a CSV-like format.
3. Listing softcored atoms. Could be done as a range (e.g. Atoms 1-4821 softcored).
4. Printout of the OSRW histogram. Possible condensation to CSV-like format.
5. Rotamer optimization Monte Carlo. Could log every N steps with current E, accepts/rejects.
6. Minimization steps.

As of commit 223d469 , we are now using PicoCLI for our scripting engine instead of the old-style Groovy Commons CLI.

PicoCLI supports argument globbing, but this interferes with some of our multi-character short options.

As such, many of our short options now require two dashes, as such:
Old: -la1 5-18
New: --la1 5-18

In the short run, this piece of regular expression should help edit FFX commands saved in files:

sed -r -i'.bak' -e '/^[:space:]*[^ ]*ffxc\ /s/\ (-[^D][^ ]+\ )/\ -\1/g' theFile

In the long run, we may start changing some of our short options to be just a single character, so as to support option globbing.

I tried this command:

ffxc xray.scaleBulk --mtz lys_tricl_4_refmac12.pdb lys_tricl_4_refmac12.mtz

If I leave out the "--mtz" flag, it works fine; but with that flag I get this error:

Writing MTZ HKL file: "lys_tricl_4_refmac12_ffx.mtz"

Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
at ffx.xray.parsers.MTZWriter.write(MTZWriter.java:301)

Googling the error message didn't help (at least for me!). I'm on MacOS with jdk1.8.0_171, built from today's github using maven 3.2.5. If I use ffxc from the binary build of 05-25-2018, it works fine, so this is low priority; I just like to feel that I can potentially modify the code to do new things.

Hi Mike: I'm finally(!) getting around to trying to learn about ffx, but had trouble getting started.

I'm on MacOS, and JAVA_HOME is /Library/Java/JavaVirtualMachines/jdk-10.0.1.jdk/Contents/Home.

I used mvn to install ffx from github, and (separately) downloaded the ffx-1.0.0-beta.bin.tar file. Both gave the problem in the subject header, when trying the run the first example script. Uninstalling jdk10 and going back to jdk1.8.0_171.jdk solved the problem. But I'm not sure that is the right thing to be doing, since there might be a better solution.

If you go to the Oracle site (e.g. by following the link on the ffx website), it puts JDK 10 as the "latest release" and the one they offer first. Other java-clueless people (like me) might be helped if indeed saying "java version 1.8 or later" is not in fact always the best thing to do.

....thanks....dave

Constant-pressure simulations do not seem to be functioning with the RESPA integrator. They behave stably for some time, but before the end of a 250 psec nptMD trajectory, the box size expands drastically culminating in a NaN gradient. Input files are attached.

Fortunately, most of our NPT simulations at present are either small-scale (where RESPA is inadvisable anyways), or just short nptMD simulations to equilibrate the box size prior to production NVT simulations.

nptRespaInput.tar.gz