openforcefield / openff-bespokefit

Automated tools for the generation of bespoke SMIRNOFF format parameters for individual molecules.

Status
Latest Release
Communication
Foundation
Installation

The Open Force Field toolkit

The Open Force Field Toolkit, built by the Open Force Field Initiative, is a Python toolkit for the development and application of modern molecular mechanics force fields based on direct chemical perception and rigorous statistical parameterization methods.

The toolkit currently covers two main areas we have committed to stably maintain throughout their lifetimes:

• Tools for using SMIRKS Native Open Force Field (SMIRNOFF) force fields
• Tools for direct chemical environment perception and manipulation

Note: Prior to version 0.9.0, this toolkit and its associated repository were named openforcefield and used different import paths. For details on this change and migration instructions, see the release notes of version 0.9.0.

Documentation

Documentation for the Open Force Field Toolkit is hosted at readthedocs. Example notebooks are available in the examples/ directory and also hosted on the Open Force Field website.

How to cite

Please cite the OpenFF Toolkit using the Zenodo record of the latest release or the version that was used. The BibTeX reference of the latest release can be found here.

Installation

The Open Force Field Toolkit (openff-toolkit) is a Python toolkit, and supports Python 3.9 through 3.11.

Installing via Mamba/Conda

Detailed installation instructions can be found here.

Force Fields

Two major force field development efforts have been undertaken by the Open Force Field Initiative, with results hosted in separate repositories.

• The Open Force Fields repository, which features the Parsley and Sage force field lines. These are the Open Force Field Initiative's efforts toward building new force fields. The initial parameters are taken from smirnoff99Frosst, but software and data produced by the Initiative's efforts have been used to refit parameter values and add new SMIRKS-based parameters.
• The smirnoff99Frosst repository, which is descended from AMBER's parm99 force field as well as Merck-Frosst's parm@frosst. This line of force fields does not aim to alter parameter values, but is instead a test of accurately converting an atom type-based force field to the SMIRNOFF format.

Force fields from both of these packages are available in their respective GitHub repositories and also as conda packages. Tables detailing the individual file names/versions within these force field lines are in the README of each repository. By default, installing the Open Force Field toolkit using conda or the single-file toolkit installers will also install these conda packages. A plugin architecture is provided for other force field developers to produce python/conda packages that can be imported by the Open Force Field Toolkit as well.

Toolkit features

The SMIRKS Native Open Force Field (SMIRNOFF) format

This repository provides tools for using the SMIRKS Native Open Force Field (SMIRNOFF) specification, which currently supports an XML representation for force field definition files.

By convention, files containing XML representations of SMIRNOFF force fields carry .offxml extensions.

Example SMIRNOFF .offxml force field definitions can be found in openff/toolkit/data/test_forcefields/. These force fields are for testing only, and we neither record versions of these files, nor do we guarantee their correctness or completeness.

Working with SMIRNOFF parameter sets

SMIRNOFF force fields can be parsed by the ForceField class, which offers methods including create_openmm_system for exporting to OpenMM and create_interchange for exporting to other formats (GROMACS, Amber, LAMMPS) via Interchange.

# Load a molecule into the OpenFF Molecule object
from openff.toolkit import Molecule
from openff.toolkit.utils import get_data_file_path
sdf_file_path = get_data_file_path('molecules/ethanol.sdf')
molecule = Molecule.from_file(sdf_file_path)

# Create an OpenFF Topology object from the molecule
from openff.toolkit import Topology
topology = Topology.from_molecules(molecule)

# Load the latest OpenFF force field release: version 2.1.0, codename "Sage"
from openff.toolkit import ForceField
forcefield = ForceField('openff-2.1.0.offxml')

# Create an OpenMM system representing the molecule with SMIRNOFF-applied parameters
openmm_system = forcefield.create_openmm_system(topology)

# Create an Interchange object for representations in other formats
interchange = forcefield.create_interchange(topology)

Detailed examples of using SMIRNOFF with the toolkit can be found in the documentation.

Frequently asked questions (FAQ)

See FAQ.md for answers to a variety of common problems, such as:

• Why do I need to provide molecules corresponding to the components of my system, or a Topology with bond orders?
• Can I use an Amber, CHARMM, or GROMACS topology/coordinate file as a starting point for applying a SMIRNOFF force field?
• What if I am starting from a PDB file?

Contributors

For a partial list of contributors, see the GitHub Contributors page. Others whose work constitutes significant contributions but did not make it into the git history include Shuzhe Wang.

openff-bespokefit's Issues

No module named 'click'

When trying to run bespoke on the command line I get the following error:

Traceback (most recent call last):
  File "/Users/daniellebergaziin/anaconda/bin/openff-bespoke", line 33, in <module>
    sys.exit(load_entry_point('openff-bespokefit', 'console_scripts', 'openff-bespoke')())
  File "/Users/daniellebergaziin/anaconda/bin/openff-bespoke", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/Users/daniellebergaziin/anaconda/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 166, in load
    module = import_module(match.group('module'))
  File "/Users/daniellebergaziin/anaconda/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/daniellebergaziin/bespoke_setup/openff-bespokefit/openff/bespokefit/cli/__init__.py", line 1, in <module>
    from openff.bespokefit.cli.cli import cli
  File "/Users/daniellebergaziin/bespoke_setup/openff-bespokefit/openff/bespokefit/cli/cli.py", line 1, in <module>
    import click
ModuleNotFoundError: No module named 'click'

I'm able to run bespoke in a jupyter notebook, and import click in the terminal. Anyone know how to fix this?

Install xTB using MKL backend

Would it be better to update the doc to install MKL as the backend for xTB so that parallelisation could work better?
related grimme-lab/xtb#401

Can I have some help running a test molecule

Hi, I'm trying to run the default workflow on a test molecule but It failed. I wonder if I could have some help? Thank you.
The test molecule sdf is generated by openeye and is

  -OEChem-04042209103D
 xtb: 6.4.1 (conda-forge)
 32 33  0     0  0  0  0  0  0999 V2000
   -1.8741   21.5782  -27.4162 C   0  0  0  0  0  0  0  0  0  0  0  0
   -1.3296   22.4662  -28.3280 C   0  0  0  0  0  0  0  0  0  0  0  0
   -2.1178   22.9451  -29.3623 C   0  0  0  0  0  0  0  0  0  0  0  0
   -3.4444   22.5460  -29.5022 C   0  0  0  0  0  0  0  0  0  0  0  0
   -3.9674   21.6514  -28.5716 C   0  0  0  0  0  0  0  0  0  0  0  0
   -3.1914   21.1698  -27.5282 C   0  0  0  0  0  0  0  0  0  0  0  0
   -5.6139   21.1665  -28.7107 Cl  0  0  0  0  0  0  0  0  0  0  0  0
   -4.2810   23.0060  -30.6713 C   0  0  0  0  0  0  0  0  0  0  0  0
   -4.6417   22.2517  -31.5407 O   0  0  0  0  0  0  0  0  0  0  0  0
   -4.5595   24.3384  -30.6025 N   0  0  0  0  0  0  0  0  0  0  0  0
   -5.2872   25.1170  -31.4989 C   0  0  0  0  0  0  0  0  0  0  0  0
   -5.8992   24.6061  -32.6433 C   0  0  0  0  0  0  0  0  0  0  0  0
   -6.5957   25.4895  -33.4480 C   0  0  0  0  0  0  0  0  0  0  0  0
   -6.7178   26.7817  -33.1998 N   0  0  0  0  0  0  0  0  0  0  0  0
   -6.1311   27.2639  -32.1070 C   0  0  0  0  0  0  0  0  0  0  0  0
   -5.4044   26.4764  -31.2228 C   0  0  0  0  0  0  0  0  0  0  0  0
   -6.3207   28.6307  -31.9389 N   0  0  0  0  0  0  0  0  0  0  0  0
   -5.8773   29.4459  -30.9349 C   0  0  0  0  0  0  0  0  0  0  0  0
   -5.2120   29.0687  -29.9987 O   0  0  0  0  0  0  0  0  0  0  0  0
   -6.3155   30.8813  -31.1250 C   0  0  0  0  0  0  0  0  0  0  0  0
   -1.4386   24.0448  -30.5124 Cl  0  0  0  0  0  0  0  0  0  0  0  0
   -7.4016   30.9398  -31.1493 H   0  0  0  0  0  0  0  0  0  0  0  0
   -1.2641   21.2005  -26.6104 H   0  0  0  0  0  0  0  0  0  0  0  0
   -0.3038   22.7865  -28.2443 H   0  0  0  0  0  0  0  0  0  0  0  0
   -3.6205   20.4817  -26.8182 H   0  0  0  0  0  0  0  0  0  0  0  0
   -4.1740   24.8510  -29.8194 H   0  0  0  0  0  0  0  0  0  0  0  0
   -5.8296   23.5618  -32.8900 H   0  0  0  0  0  0  0  0  0  0  0  0
   -7.0891   25.1453  -34.3467 H   0  0  0  0  0  0  0  0  0  0  0  0
   -4.9483   26.9175  -30.3525 H   0  0  0  0  0  0  0  0  0  0  0  0
   -6.8727   29.0355  -32.6844 H   0  0  0  0  0  0  0  0  0  0  0  0
   -5.9228   31.2719  -32.0613 H   0  0  0  0  0  0  0  0  0  0  0  0
   -5.9421   31.4820  -30.3010 H   0  0  0  0  0  0  0  0  0  0  0  0
  1  2  2  0  0  0  0
  2  3  1  0  0  0  0
  3  4  2  0  0  0  0
  4  5  1  0  0  0  0
  1  6  1  0  0  0  0
  5  6  2  0  0  0  0
  5  7  1  0  0  0  0
  4  8  1  0  0  0  0
  8  9  2  0  0  0  0
  8 10  1  0  0  0  0
 10 11  1  0  0  0  0
 11 12  2  0  0  0  0
 12 13  1  0  0  0  0
 13 14  2  0  0  0  0
 14 15  1  0  0  0  0
 11 16  1  0  0  0  0
 15 16  2  0  0  0  0
 15 17  1  0  0  0  0
 17 18  1  0  0  0  0
 18 19  2  0  0  0  0
 18 20  1  0  0  0  0
  3 21  1  0  0  0  0
 20 22  1  0  0  0  0
  1 23  1  0  0  0  0
  2 24  1  0  0  0  0
  6 25  1  0  0  0  0
 10 26  1  0  0  0  0
 12 27  1  0  0  0  0
 13 28  1  0  0  0  0
 16 29  1  0  0  0  0
 17 30  1  0  0  0  0
 20 31  1  0  0  0  0
 20 32  1  0  0  0  0
M  END
$$$$

I created the env with mamba create -n bespokefit -c conda-forge -c openeye -c psi4 openeye-toolkits openff-bespokefit xtb-python torchani psi4
and generate the topology with openff-bespoke executor run --file "lig.sdf" --workflow "default" --output "acetaminophen.json" --output-force-field "acetaminophen.offxml" --n-qc-compute-workers 2 --qc-compute-n-cores 8

But I got the error of

────────────────────────────────────────────────────────────────────── OpenFF Bespoke ──────────────────────────────────────────────────────────────────────

[✓] bespoke executor launched

1. preparing the bespoke workflow                                                                                                                           
                                                                                                                                                            
[✓] 1 molecules found
[✓] fitting schemas generated
                                                                                                                                                            
2. submitting the workflow                                                                                                                                  
                                                                                                                                                            
[✓] the following workflows were submitted
┏━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━┓
┃ ID ┃ SMILES                                ┃ NAME ┃ FILE    ┃
┡━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━┩
│ 1  │ CC(=O)Nc1cc(ccn1)NC(=O)c2c(cccc2Cl)Cl │      │ lig.sdf │
└────┴───────────────────────────────────────┴──────┴─────────┘
                                                                                                                                                            
3. running the fitting pipeline                                                                                                                             
                                                                                                                                                            
[✓] fragmentation successful
[x] qc-generation failed
                                                                                                                                                            
 [{"type": "ValueError", "message": "TorsionDrive error at 0:\ngeomeTRIC run_json error:\nTraceback (most recent call last):\n  File                        
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/run_json.py\", line 225, in geometric_run_json\n                         
 geometric.optimize.Optimize(coords, M, IC, engine, None, params)\n  File                                                                                   
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1331, in Optimize\n    return                        
 optimizer.optimizeGeometry()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1293, in        
 optimizeGeometry\n    self.calcEnergyForce()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 
 1002, in calcEnergyForce\n    spcalc = self.engine.calc(self.X, self.dirname)\n  File                                                                      
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 873, in calc\n    return self.calc_new(coords,         
 dirname)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 865, in calc_new\n    raise           
 QCEngineAPIEngineError(\"QCEngineAPI computation did not execute correctly. Message: \" +                                                                  
 ret[\"error\"][\"error_message\"])\ngeometric.errors.QCEngineAPIEngineError: QCEngineAPI computation did not execute correctly. Message: QCEngine Execution
 Error:\nTraceback (most recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 114, in
 compute_wrapper\n    yield metadata\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 91, in     
 compute\n    output_data = executor.compute(input_data, config)\n  File                                                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 121, in compute\n    pversion =                  
 parse_version(self.get_version())\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 91, in 
 get_version\n    self.version_cache = safe_version(exc[\"stdout\"].split()[-1])\nIndexError: list index out of range\n\n", "traceback": "Traceback (most   
 recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/celery/app/trace.py\", line 451, in trace_task\n    R = 
 retval = fun(*args, **kwargs)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/celery/app/trace.py\", line 734, in          
 __protected_call__\n    return self.run(*args, **kwargs)\n  File                                                                                           
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/openff/bespokefit/executor/services/qcgenerator/worker.py\", line 132, in          
 compute_torsion_drive\n    return_value = qcengine.compute_procedure(\n  File                                                                              
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 149, in compute_procedure\n    return                  
 handle_output_metadata(output_data, metadata, raise_error=raise_error, return_dict=return_dict)\n  File                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 177, in handle_output_metadata\n    raise                 
 ValueError(output_fusion[\"error\"][\"error_message\"])\nValueError: TorsionDrive error at 0:\ngeomeTRIC run_json error:\nTraceback (most recent call      
 last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/run_json.py\", line 225, in geometric_run_json\n          
 geometric.optimize.Optimize(coords, M, IC, engine, None, params)\n  File                                                                                   
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1331, in Optimize\n    return                        
 optimizer.optimizeGeometry()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1293, in        
 optimizeGeometry\n    self.calcEnergyForce()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 
 1002, in calcEnergyForce\n    spcalc = self.engine.calc(self.X, self.dirname)\n  File                                                                      
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 873, in calc\n    return self.calc_new(coords,         
 dirname)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 865, in calc_new\n    raise           
 QCEngineAPIEngineError(\"QCEngineAPI computation did not execute correctly. Message: \" +                                                                  
 ret[\"error\"][\"error_message\"])\ngeometric.errors.QCEngineAPIEngineError: QCEngineAPI computation did not execute correctly. Message: QCEngine Execution
 Error:\nTraceback (most recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 114, in
 compute_wrapper\n    yield metadata\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 91, in     
 compute\n    output_data = executor.compute(input_data, config)\n  File                                                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 121, in compute\n    pversion =                  
 parse_version(self.get_version())\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 91, in 
 get_version\n    self.version_cache = safe_version(exc[\"stdout\"].split()[-1])\nIndexError: list index out of range\n\n\n"}, {"type": "ValueError",       
 "message": "TorsionDrive error at 90:\ngeomeTRIC run_json error:\nTraceback (most recent call last):\n  File                                               
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/run_json.py\", line 225, in geometric_run_json\n                         
 geometric.optimize.Optimize(coords, M, IC, engine, None, params)\n  File                                                                                   
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1331, in Optimize\n    return                        
 optimizer.optimizeGeometry()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1293, in        
 optimizeGeometry\n    self.calcEnergyForce()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 
 1002, in calcEnergyForce\n    spcalc = self.engine.calc(self.X, self.dirname)\n  File                                                                      
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 873, in calc\n    return self.calc_new(coords,         
 dirname)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 865, in calc_new\n    raise           
 QCEngineAPIEngineError(\"QCEngineAPI computation did not execute correctly. Message: \" +                                                                  
 ret[\"error\"][\"error_message\"])\ngeometric.errors.QCEngineAPIEngineError: QCEngineAPI computation did not execute correctly. Message: QCEngine Execution
 Error:\nTraceback (most recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 114, in
 compute_wrapper\n    yield metadata\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 91, in     
 compute\n    output_data = executor.compute(input_data, config)\n  File                                                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 121, in compute\n    pversion =                  
 parse_version(self.get_version())\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 91, in 
 get_version\n    self.version_cache = safe_version(exc[\"stdout\"].split()[-1])\nIndexError: list index out of range\n\n", "traceback": "Traceback (most   
 recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/celery/app/trace.py\", line 451, in trace_task\n    R = 
 retval = fun(*args, **kwargs)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/celery/app/trace.py\", line 734, in          
 __protected_call__\n    return self.run(*args, **kwargs)\n  File                                                                                           
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/openff/bespokefit/executor/services/qcgenerator/worker.py\", line 132, in          
 compute_torsion_drive\n    return_value = qcengine.compute_procedure(\n  File                                                                              
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 149, in compute_procedure\n    return                  
 handle_output_metadata(output_data, metadata, raise_error=raise_error, return_dict=return_dict)\n  File                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 177, in handle_output_metadata\n    raise                 
 ValueError(output_fusion[\"error\"][\"error_message\"])\nValueError: TorsionDrive error at 90:\ngeomeTRIC run_json error:\nTraceback (most recent call     
 last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/run_json.py\", line 225, in geometric_run_json\n          
 geometric.optimize.Optimize(coords, M, IC, engine, None, params)\n  File                                                                                   
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1331, in Optimize\n    return                        
 optimizer.optimizeGeometry()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1293, in        
 optimizeGeometry\n    self.calcEnergyForce()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 
 1002, in calcEnergyForce\n    spcalc = self.engine.calc(self.X, self.dirname)\n  File                                                                      
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 873, in calc\n    return self.calc_new(coords,         
 dirname)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 865, in calc_new\n    raise           
 QCEngineAPIEngineError(\"QCEngineAPI computation did not execute correctly. Message: \" +                                                                  
 ret[\"error\"][\"error_message\"])\ngeometric.errors.QCEngineAPIEngineError: QCEngineAPI computation did not execute correctly. Message: QCEngine Execution
 Error:\nTraceback (most recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 114, in
 compute_wrapper\n    yield metadata\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 91, in     
 compute\n    output_data = executor.compute(input_data, config)\n  File                                                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 121, in compute\n    pversion =                  
 parse_version(self.get_version())\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 91, in 
 get_version\n    self.version_cache = safe_version(exc[\"stdout\"].split()[-1])\nIndexError: list index out of range\n\n\n"}, {"type": "ValueError",       
 "message": "TorsionDrive error at 0:\ngeomeTRIC run_json error:\nTraceback (most recent call last):\n  File                                                
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/run_json.py\", line 225, in geometric_run_json\n                         
 geometric.optimize.Optimize(coords, M, IC, engine, None, params)\n  File                                                                                   
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1331, in Optimize\n    return                        
 optimizer.optimizeGeometry()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1293, in        
 optimizeGeometry\n    self.calcEnergyForce()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 
 1002, in calcEnergyForce\n    spcalc = self.engine.calc(self.X, self.dirname)\n  File                                                                      
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 873, in calc\n    return self.calc_new(coords,         
 dirname)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 865, in calc_new\n    raise           
 QCEngineAPIEngineError(\"QCEngineAPI computation did not execute correctly. Message: \" +                                                                  
 ret[\"error\"][\"error_message\"])\ngeometric.errors.QCEngineAPIEngineError: QCEngineAPI computation did not execute correctly. Message: QCEngine Execution
 Error:\nTraceback (most recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 114, in
 compute_wrapper\n    yield metadata\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 91, in     
 compute\n    output_data = executor.compute(input_data, config)\n  File                                                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 121, in compute\n    pversion =                  
 parse_version(self.get_version())\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 91, in 
 get_version\n    self.version_cache = safe_version(exc[\"stdout\"].split()[-1])\nIndexError: list index out of range\n\n", "traceback": "Traceback (most   
 recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/celery/app/trace.py\", line 451, in trace_task\n    R = 
 retval = fun(*args, **kwargs)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/celery/app/trace.py\", line 734, in          
 __protected_call__\n    return self.run(*args, **kwargs)\n  File                                                                                           
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/openff/bespokefit/executor/services/qcgenerator/worker.py\", line 132, in          
 compute_torsion_drive\n    return_value = qcengine.compute_procedure(\n  File                                                                              
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 149, in compute_procedure\n    return                  
 handle_output_metadata(output_data, metadata, raise_error=raise_error, return_dict=return_dict)\n  File                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 177, in handle_output_metadata\n    raise                 
 ValueError(output_fusion[\"error\"][\"error_message\"])\nValueError: TorsionDrive error at 0:\ngeomeTRIC run_json error:\nTraceback (most recent call      
 last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/run_json.py\", line 225, in geometric_run_json\n          
 geometric.optimize.Optimize(coords, M, IC, engine, None, params)\n  File                                                                                   
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1331, in Optimize\n    return                        
 optimizer.optimizeGeometry()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1293, in        
 optimizeGeometry\n    self.calcEnergyForce()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 
 1002, in calcEnergyForce\n    spcalc = self.engine.calc(self.X, self.dirname)\n  File                                                                      
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 873, in calc\n    return self.calc_new(coords,         
 dirname)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 865, in calc_new\n    raise           
 QCEngineAPIEngineError(\"QCEngineAPI computation did not execute correctly. Message: \" +                                                                  
 ret[\"error\"][\"error_message\"])\ngeometric.errors.QCEngineAPIEngineError: QCEngineAPI computation did not execute correctly. Message: QCEngine Execution
 Error:\nTraceback (most recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 114, in
 compute_wrapper\n    yield metadata\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 91, in     
 compute\n    output_data = executor.compute(input_data, config)\n  File                                                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 121, in compute\n    pversion =                  
 parse_version(self.get_version())\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 91, in 
 get_version\n    self.version_cache = safe_version(exc[\"stdout\"].split()[-1])\nIndexError: list index out of range\n\n\n"}, {"type": "ValueError",       
 "message": "TorsionDrive error at 0:\ngeomeTRIC run_json error:\nTraceback (most recent call last):\n  File                                                
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/run_json.py\", line 225, in geometric_run_json\n                         
 geometric.optimize.Optimize(coords, M, IC, engine, None, params)\n  File                                                                                   
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1331, in Optimize\n    return                        
 optimizer.optimizeGeometry()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1293, in        
 optimizeGeometry\n    self.calcEnergyForce()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 
 1002, in calcEnergyForce\n    spcalc = self.engine.calc(self.X, self.dirname)\n  File                                                                      
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 873, in calc\n    return self.calc_new(coords,         
 dirname)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 865, in calc_new\n    raise           
 QCEngineAPIEngineError(\"QCEngineAPI computation did not execute correctly. Message: \" +                                                                  
 ret[\"error\"][\"error_message\"])\ngeometric.errors.QCEngineAPIEngineError: QCEngineAPI computation did not execute correctly. Message: QCEngine Execution
 Error:\nTraceback (most recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 114, in
 compute_wrapper\n    yield metadata\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 91, in     
 compute\n    output_data = executor.compute(input_data, config)\n  File                                                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 121, in compute\n    pversion =                  
 parse_version(self.get_version())\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 91, in 
 get_version\n    self.version_cache = safe_version(exc[\"stdout\"].split()[-1])\nIndexError: list index out of range\n\n", "traceback": "Traceback (most   
 recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/celery/app/trace.py\", line 451, in trace_task\n    R = 
 retval = fun(*args, **kwargs)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/celery/app/trace.py\", line 734, in          
 __protected_call__\n    return self.run(*args, **kwargs)\n  File                                                                                           
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/openff/bespokefit/executor/services/qcgenerator/worker.py\", line 132, in          
 compute_torsion_drive\n    return_value = qcengine.compute_procedure(\n  File                                                                              
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 149, in compute_procedure\n    return                  
 handle_output_metadata(output_data, metadata, raise_error=raise_error, return_dict=return_dict)\n  File                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 177, in handle_output_metadata\n    raise                 
 ValueError(output_fusion[\"error\"][\"error_message\"])\nValueError: TorsionDrive error at 0:\ngeomeTRIC run_json error:\nTraceback (most recent call      
 last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/run_json.py\", line 225, in geometric_run_json\n          
 geometric.optimize.Optimize(coords, M, IC, engine, None, params)\n  File                                                                                   
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1331, in Optimize\n    return                        
 optimizer.optimizeGeometry()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1293, in        
 optimizeGeometry\n    self.calcEnergyForce()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 
 1002, in calcEnergyForce\n    spcalc = self.engine.calc(self.X, self.dirname)\n  File                                                                      
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 873, in calc\n    return self.calc_new(coords,         
 dirname)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 865, in calc_new\n    raise           
 QCEngineAPIEngineError(\"QCEngineAPI computation did not execute correctly. Message: \" +                                                                  
 ret[\"error\"][\"error_message\"])\ngeometric.errors.QCEngineAPIEngineError: QCEngineAPI computation did not execute correctly. Message: QCEngine Execution
 Error:\nTraceback (most recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 114, in
 compute_wrapper\n    yield metadata\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 91, in     
 compute\n    output_data = executor.compute(input_data, config)\n  File                                                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 121, in compute\n    pversion =                  
 parse_version(self.get_version())\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 91, in 
 get_version\n    self.version_cache = safe_version(exc[\"stdout\"].split()[-1])\nIndexError: list index out of range\n\n\n"}, {"type": "ValueError",       
 "message": "TorsionDrive error at 0:\ngeomeTRIC run_json error:\nTraceback (most recent call last):\n  File                                                
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/run_json.py\", line 225, in geometric_run_json\n                         
 geometric.optimize.Optimize(coords, M, IC, engine, None, params)\n  File                                                                                   
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1331, in Optimize\n    return                        
 optimizer.optimizeGeometry()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1293, in        
 optimizeGeometry\n    self.calcEnergyForce()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 
 1002, in calcEnergyForce\n    spcalc = self.engine.calc(self.X, self.dirname)\n  File                                                                      
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 873, in calc\n    return self.calc_new(coords,         
 dirname)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 865, in calc_new\n    raise           
 QCEngineAPIEngineError(\"QCEngineAPI computation did not execute correctly. Message: \" +                                                                  
 ret[\"error\"][\"error_message\"])\ngeometric.errors.QCEngineAPIEngineError: QCEngineAPI computation did not execute correctly. Message: QCEngine Execution
 Error:\nTraceback (most recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 114, in
 compute_wrapper\n    yield metadata\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 91, in     
 compute\n    output_data = executor.compute(input_data, config)\n  File                                                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 121, in compute\n    pversion =                  
 parse_version(self.get_version())\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 91, in 
 get_version\n    self.version_cache = safe_version(exc[\"stdout\"].split()[-1])\nIndexError: list index out of range\n\n", "traceback": "Traceback (most   
 recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/celery/app/trace.py\", line 451, in trace_task\n    R = 
 retval = fun(*args, **kwargs)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/celery/app/trace.py\", line 734, in          
 __protected_call__\n    return self.run(*args, **kwargs)\n  File                                                                                           
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/openff/bespokefit/executor/services/qcgenerator/worker.py\", line 132, in          
 compute_torsion_drive\n    return_value = qcengine.compute_procedure(\n  File                                                                              
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 149, in compute_procedure\n    return                  
 handle_output_metadata(output_data, metadata, raise_error=raise_error, return_dict=return_dict)\n  File                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 177, in handle_output_metadata\n    raise                 
 ValueError(output_fusion[\"error\"][\"error_message\"])\nValueError: TorsionDrive error at 0:\ngeomeTRIC run_json error:\nTraceback (most recent call      
 last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/run_json.py\", line 225, in geometric_run_json\n          
 geometric.optimize.Optimize(coords, M, IC, engine, None, params)\n  File                                                                                   
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1331, in Optimize\n    return                        
 optimizer.optimizeGeometry()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 1293, in        
 optimizeGeometry\n    self.calcEnergyForce()\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/optimize.py\", line 
 1002, in calcEnergyForce\n    spcalc = self.engine.calc(self.X, self.dirname)\n  File                                                                      
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 873, in calc\n    return self.calc_new(coords,         
 dirname)\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/geometric/engine.py\", line 865, in calc_new\n    raise           
 QCEngineAPIEngineError(\"QCEngineAPI computation did not execute correctly. Message: \" +                                                                  
 ret[\"error\"][\"error_message\"])\ngeometric.errors.QCEngineAPIEngineError: QCEngineAPI computation did not execute correctly. Message: QCEngine Execution
 Error:\nTraceback (most recent call last):\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/util.py\", line 114, in
 compute_wrapper\n    yield metadata\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/compute.py\", line 91, in     
 compute\n    output_data = executor.compute(input_data, config)\n  File                                                                                    
 \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 121, in compute\n    pversion =                  
 parse_version(self.get_version())\n  File \"/home/ec2-user/miniconda3/envs/bespokefit/lib/python3.9/site-packages/qcengine/programs/psi4.py\", line 91, in 
 get_version\n    self.version_cache = safe_version(exc[\"stdout\"].split()[-1])\nIndexError: list index out of range\n\n\n"}]                              
                                                                                                                                                            
outputs have been saved to acetaminophen.json                                                                                                               
                                                                                                                                                            

worker: Warm shutdown (MainProcess)

worker: Warm shutdown (MainProcess)

worker: Warm shutdown (MainProcess)

Test issue

Parallelize torsiondrives

@SimonBoothroyd made a great start exposing QC worker settings including memory and number of cores for each job #120. It would be good if we could also parallelize the torsiondrives so each worker could run multiple optimisations simultaneously. This might mean we need to rework the qcengine procedure slightly to achieve this possibly by subclassing the current one and registering a new parallel implementation under a different alias. We may also want to introduce a new setting like N_QC_PARALLEL_TASKS which determines the maximum number of tasks to run in parallel and how the maximum resources should be divied.

Run the script in the blog

Dear openff developers, recently I saw your nice blog post: https://openforcefield.org/community/news/science-updates/bespokefit-update-2021-10-20/ and would like to try it.

When running the code from the blog, I found that there was no openff-fragmenter and openff-toolkit after installing bespoke-fit following the instruction from the README.md:

conda create --name bespokefit -c conda-forge -c omnia -c omnia/label/rc -c openeye openff-qcsubmit chemper fragmenter==0.0.7 openeye-toolkits forcebalance openforcefield==0.8.0 pydantic==1.7.2
python setup.py develop

Then I tried to installed them together and failed:

conda create --name bes -c conda-forge
 -c omnia -c omnia/label/rc -c openeye openff-qcsubmit chemper fragmenter==0.0.7 openeye-toolkit
s forcebalance openforcefield==0.8.0 pydantic==1.7.2 openff-toolkit openff-fragmenter

There were lots of package conflict, here is part of them:

Package libgcc-ng conflicts for:
pydantic==1.7.2 -> libgcc-ng[version='>=7.3.0|>=7.5.0|>=9.3.0']
pydantic==1.7.2 -> python[version='>=3.7,<3.8.0a0'] -> libgcc-ng[version='>=4.9|>=9.4.0|>=7.2.0']

Package pluggy conflicts for:
openforcefield==0.8.0 -> pytest -> pluggy[version='>=0.12,<1.0|>=0.12,<1.0.0a1|>=0.12,<2|>=0.9,!=0.10,<1.0|>=0.9|>=0.7|>=0.5,<0.7']
openeye-toolkits -> pytest -> pluggy[version='>=0.12,<1.0|>=0.12,<1.0.0a1|>=0.12,<2|>=0.9,!=0.10,<1.0|>=0.9|>=0.7|>=0.5,<0.7']
openff-qcsubmit -> pytest -> pluggy[version='>=0.12,<1.0|>=0.12,<1.0.0a1|>=0.12,<2|>=0.9,!=0.10,<1.0|>=0.9|>=0.7|>=0.5,<0.7']

Package lxml conflicts for:
openff-qcsubmit -> openmmforcefields -> lxml
forcebalance -> lxml

Package ambertools conflicts for:
openforcefield==0.8.0 -> ambertools[version='>=20']
openff-qcsubmit -> openforcefield[version='>=0.8.0'] -> ambertools[version='>=18.0|>=20|>=20.0']
openff-toolkit -> ambertools[version='>=20']
openff-fragmenter -> ambertools

Package python conflicts for:
pydantic==1.7.2 -> python[version='>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0']
pydantic==1.7.2 -> python_abi=3.7[build=*_cp37m] -> python[version='3.6.*|3.7.*|>=3.7|>=3.6,<3.7|3.8.*|3.9.*']The following specifications were found to be incompatible with your system:

  - feature:/linux-64::__glibc==2.17=0
  - feature:|@/linux-64::__glibc==2.17=0
  - forcebalance -> libgcc-ng[version='>=9.3.0'] -> __glibc[version='>=2.17']
  - openeye-toolkits -> libstdcxx-ng -> __glibc[version='>=2.17']
  - openff-toolkit -> openmm -> __glibc[version='>=2.17']
  - openforcefield==0.8.0 -> openmm -> __glibc[version='>=2.17']

Any idea what I should do next? thanks!

Hyperparameter error

When I specify hyperparameters with this line:

        parameters=target_parameters,
        parameter_hyperparameters=[
            AngleHyperparameters(k=100, length=20),
            ImproperTorsionHyperparameters(k=5.0),
        ],

I am getting this error:

Traceback (most recent call last):
  File "/Users/jessica/Downloads/fit-example/2.create-fb-inputs.py", line 103, in <module>
    main()
  File "/Users/jessica/Downloads/fit-example/2.create-fb-inputs.py", line 80, in main
    AngleHyperparameters(k=100, length=20),
  File "pydantic/main.py", line 406, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 2 validation errors for AngleHyperparameters
k
  extra fields not permitted (type=value_error.extra)
length
  extra fields not permitted (type=value_error.extra)

@mattwthompson @JoshHorton

Installation on MacOS

Hi, I was trying to do a conda install of the openff-bespokefit on Mac with conda.
I tried to create a new env and install the openff-bespokefit with
conda create -n bespokefit -c conda-forge openff-bespokefit.
However, it seems that it cannot solve the environment.

Install the psi4

Hi, I was wondering how should I install the psi4 with openff-bespokefit?
I did this via

mamba create -n bespokefit -c conda-forge openff-bespokefit python=3.9
conda install -c conda-forge -c defaults -c psi4 psi4

I used mamba in the first part as it would otherwise be too slow.
But running psi4 will give

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/bespokefit/lib//python3.9/site-packages/psi4/__init__.py", line 55, in <module>
    from . import core
ImportError: /home/ec2-user/miniconda3/envs/bespokefit/lib//python3.9/site-packages/psi4/core.cpython-39-x86_64-linux-gnu.so: undefined symbol: __svml_exp4_mask_e9

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/bespokefit/bin/psi4", line 183, in <module>
    import psi4  # isort:skip
  File "/home/ec2-user/miniconda3/envs/bespokefit/lib//python3.9/site-packages/psi4/__init__.py", line 60, in <module>
    raise ImportError("{0}".format(err))
ImportError: /home/ec2-user/miniconda3/envs/bespokefit/lib//python3.9/site-packages/psi4/core.cpython-39-x86_64-linux-gnu.so: undefined symbol: __svml_exp4_mask_e9

I wonder if I can have some help on this, please? Thank you.
This is a regular linux x86 machine.

Default arguments to `BespokeExecutor` cause it to hang

The default arguments to BespokeExecutor's constructor method cause it to hang.

def __init__(
        self,
        n_fragmenter_workers: int = 0,
        n_qc_compute_workers: int = 0,
        n_optimizer_workers: int = 0,
        directory: Optional[str] = "bespoke-executor",
        launch_redis_if_unavailable: bool = True,
    ):

It seems to be that celery is taking 0 very literally, not spawning any workers, and then getting stuck in an infinite loop when no work is done. I think the default behaviour should be to automatically choose the number of workers based on the number of cores available and the parallelism of the underlying task, or else for the default to be 1 worker for each.

QC generation stage fails when no fragments are generated

The following command fails when using a fresh install from main

openff-bespoke executor run --input BrCO.sdf --spec-file schema.json --n-qc-compute-workers 4

BrCO.sdf and schema.json (and the run.py file that makes them but crashes in the process )can be found here:
inputs.tar.gz

Env created by appending xtb-python to the test env, then using conda env create -f devtools/conda-envs/test-env.yaml. Full conda list is below.

This setup failed to run on @bergazin's macbook, my macbook, and a linux docker container on @bergazin's macbook. I also tried

• replacing --spec-file with --spec default
• removing --n-qc-compute-workers
• running with AmberTools+RDKit instead of OE
• running from c087e0e instead
• running from dffb29e instead
• Downgrading qcelemental and qcengine

Output:

(test) jeffreywagner@JW-MBP$ openff-bespoke executor run --input BrCO.sdf --spec-file schema.json --n-qc-compute-workers 4

─────────────────────────────────────────────────────────── OpenFF Bespoke ───────────────────────────────────────────────────────────

[✓] bespoke executor launched

1. preparing the bespoke workflow                                                                                                     
                                                                                                                                      
[✓] 1 molecule was found
[✓] fitting schema generated
                                                                                                                                      
2. submitting the workflow                                                                                                            
                                                                                                                                      
[✓] workflow submitted: id=1
                                                                                                                                      
3. running the fitting pipeline                                                                                                       
                                                                                                                                      
[✓] fragmentation successful
[x] qc-generation failed
                                                                                                                                      
 "Internal Server Error"                                                                                                              

worker: Warm shutdown (MainProcess)

worker: Warm shutdown (MainProcess)

worker: Warm shutdown (MainProcess)

The file bespoke_executor/gateway.log contains the following error:

(test) jeffreywagner@JW-MBP$ cat bespoke-executor/gateway.log 
[task id=b'coordinator:optimization:1'] transitioned from waiting -> running
[task id=b'coordinator:optimization:1'] fragmentation transitioned from waiting -> running
[task id=b'coordinator:optimization:1'] fragmentation transitioned from running -> success
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/fastapi/applications.py", line 199, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/starlette/middleware/cors.py", line 78, in __call__
    await self.app(scope, receive, send)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/starlette/routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/starlette/routing.py", line 241, in handle
    await self.app(scope, receive, send)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/starlette/routing.py", line 52, in app
    response = await func(request)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/fastapi/routing.py", line 219, in app
    raw_response = await run_endpoint_function(
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/fastapi/routing.py", line 154, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/starlette/concurrency.py", line 40, in run_in_threadpool
    return await loop.run_in_executor(None, func, *args)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/openff_bespokefit-27rc4904+28.g739f680.dirty-py3.9.egg/openff/bespokefit/executor/services/qcgenerator/app.py", line 70, in get_qc_results
    contents=[_retrieve_qc_result(qc_calc_id, results) for qc_calc_id in ids],
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/openff_bespokefit-27rc4904+28.g739f680.dirty-py3.9.egg/openff/bespokefit/executor/services/qcgenerator/app.py", line 70, in <listcomp>
    contents=[_retrieve_qc_result(qc_calc_id, results) for qc_calc_id in ids],
  File "/Users/jeffreywagner/miniconda3/envs/test/lib/python3.9/site-packages/openff_bespokefit-27rc4904+28.g739f680.dirty-py3.9.egg/openff/bespokefit/executor/services/qcgenerator/app.py", line 46, in _retrieve_qc_result
    "type": qc_calc_type.decode(),
AttributeError: 'NoneType' object has no attribute 'decode'
[task id=b'coordinator:optimization:1'] qc-generation transitioned from waiting -> errored
[task id=b'coordinator:optimization:1'] transitioned from running -> errored

Env info:

(test) jeffreywagner@JW-MBP$ conda list 
# packages in environment at /Users/jeffreywagner/miniconda3/envs/test:
#
# Name                    Version                   Build  Channel
abseil-cpp                20210324.2           he49afe7_0    conda-forge
amberlite                 16.0                     pypi_0    pypi
ambertools                21.11            py39hf80593e_0    conda-forge
amberutils                21.0                     pypi_0    pypi
amqp                      5.0.6              pyhd8ed1ab_0    conda-forge
anyio                     3.3.4            py39h6e9494a_0    conda-forge
appnope                   0.1.2            py39h6e9494a_2    conda-forge
argcomplete               1.12.3             pyhd8ed1ab_2    conda-forge
argon2-cffi               21.1.0           py39h89e85a6_2    conda-forge
arpack                    3.7.0                hefb7bc6_2    conda-forge
arrow-cpp                 6.0.0           py39h1bf2758_0_cpu    conda-forge
asgiref                   3.4.1              pyhd8ed1ab_0    conda-forge
astunparse                1.6.3              pyhd8ed1ab_0    conda-forge
async_generator           1.10                       py_0    conda-forge
attrs                     21.2.0             pyhd8ed1ab_0    conda-forge
aws-c-auth                0.6.4                hd0defc5_3    conda-forge
aws-c-cal                 0.5.12               h88d0821_2    conda-forge
aws-c-common              0.6.11               h0d85af4_0    conda-forge
aws-c-compression         0.2.14               h4dc997f_2    conda-forge
aws-c-event-stream        0.2.7               h0d10642_24    conda-forge
aws-c-http                0.6.6                h3059b7b_1    conda-forge
aws-c-io                  0.10.9               h42eeb19_3    conda-forge
aws-c-mqtt                0.7.8                h7ee32f5_2    conda-forge
aws-c-s3                  0.1.27               h14340f6_1    conda-forge
aws-checksums             0.1.12               h4dc997f_1    conda-forge
aws-crt-cpp               0.17.1               hf6c2175_3    conda-forge
aws-sdk-cpp               1.9.120              hb246b62_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
basis_set_exchange        0.8.13                     py_0    conda-forge
billiard                  3.6.4.0          py39h89e85a6_1    conda-forge
bleach                    4.1.0              pyhd8ed1ab_0    conda-forge
blosc                     1.21.0               he49afe7_0    conda-forge
boost                     1.74.0           py39ha1f3e3e_4    conda-forge
boost-cpp                 1.74.0               hf3dc895_5    conda-forge
brotlipy                  0.7.0           py39h89e85a6_1003    conda-forge
bzip2                     1.0.8                h0d85af4_4    conda-forge
c-ares                    1.18.1               h0d85af4_0    conda-forge
ca-certificates           2021.10.8            h033912b_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                4.2.4              pyhd8ed1ab_0    conda-forge
cairo                     1.16.0            he01c77b_1009    conda-forge
celery                    5.1.2              pyhd8ed1ab_0    conda-forge
certifi                   2021.10.8        py39h6e9494a_1    conda-forge
cffi                      1.15.0           py39he338e87_0    conda-forge
chardet                   4.0.0            py39h6e9494a_2    conda-forge
charset-normalizer        2.0.0              pyhd8ed1ab_0    conda-forge
chemper                   1.0.0              pyh9f0ad1d_0    conda-forge
click                     7.1.2              pyh9f0ad1d_0    conda-forge
click-didyoumean          0.0.3              pyh8c360ce_0    conda-forge
click-option-group        0.5.3              pyhd8ed1ab_0    conda-forge
click-plugins             1.1.1                      py_0    conda-forge
click-repl                0.2.0              pyhd8ed1ab_0    conda-forge
codecov                   2.1.11             pyhd3deb0d_0    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
commonmark                0.9.1                      py_0    conda-forge
coverage                  6.1.1            py39h89e85a6_1    conda-forge
cryptography              35.0.0           py39h209aa08_2    conda-forge
curl                      7.79.1               hf45b732_1    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cython                    0.29.24          py39h9fcab8e_1    conda-forge
debugpy                   1.5.1            py39h9fcab8e_0    conda-forge
decorator                 5.1.0              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
entrypoints               0.3             py39hde42818_1002    conda-forge
fastapi                   0.67.0             pyhd8ed1ab_0    conda-forge
fftw                      3.3.10          nompi_h4024a70_101    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.13.1            h10f422b_1005    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
forcebalance              1.9.2            py39ha1f3e3e_0    conda-forge
freetype                  2.10.4               h4cff582_1    conda-forge
future                    0.18.2           py39h6e9494a_4    conda-forge
geometric                 0.9.7.2                    py_0    conda-forge
gettext                   0.19.8.1          hd1a6beb_1008    conda-forge
gflags                    2.2.2             hb1e8313_1004    conda-forge
glog                      0.5.0                h25b26a9_0    conda-forge
greenlet                  1.1.2            py39h9fcab8e_1    conda-forge
grpc-cpp                  1.40.0               h8fdf118_2    conda-forge
h11                       0.12.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0            py39h6e9494a_0    conda-forge
h5py                      3.4.0           nompi_py39hbc6cb89_101    conda-forge
hdf4                      4.2.15               hefd3b78_3    conda-forge
hdf5                      1.12.1          nompi_h2f0ef1a_101    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
httpcore                  0.13.7             pyhd8ed1ab_0    conda-forge
httpx                     0.20.0           py39h6e9494a_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
icu                       69.1                 he49afe7_0    conda-forge
idna                      3.1                pyhd3deb0d_0    conda-forge
importlib-metadata        4.8.2            py39h6e9494a_0    conda-forge
importlib_metadata        4.8.2                hd8ed1ab_0    conda-forge
importlib_resources       5.4.0              pyhd8ed1ab_0    conda-forge
iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
ipykernel                 6.4.2            py39h71a6800_0    conda-forge
ipython                   7.29.0           py39h71a6800_1    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.6.5              pyhd8ed1ab_0    conda-forge
jbig                      2.1               h0d85af4_2003    conda-forge
jedi                      0.18.0           py39h6e9494a_3    conda-forge
jinja2                    3.0.2              pyhd8ed1ab_0    conda-forge
jpeg                      9d                   hbcb3906_0    conda-forge
jsonschema                4.2.1              pyhd8ed1ab_0    conda-forge
jupyter_client            7.0.6              pyhd8ed1ab_0    conda-forge
jupyter_core              4.9.1            py39h6e9494a_0    conda-forge
jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge
jupyterlab_widgets        1.0.2              pyhd8ed1ab_0    conda-forge
khronos-opencl-icd-loader 2021.06.30           h0d85af4_1    conda-forge
kiwisolver                1.3.2            py39hf018cea_1    conda-forge
kombu                     5.2.1            py39h6e9494a_0    conda-forge
krb5                      1.19.2               hcfbf3a7_3    conda-forge
lcms2                     2.12                 h577c468_0    conda-forge
lerc                      3.0                  he49afe7_0    conda-forge
libblas                   3.9.0           12_osx64_openblas    conda-forge
libbrotlicommon           1.0.9                h0d85af4_6    conda-forge
libbrotlidec              1.0.9                h0d85af4_6    conda-forge
libbrotlienc              1.0.9                h0d85af4_6    conda-forge
libcblas                  3.9.0           12_osx64_openblas    conda-forge
libcurl                   7.79.1               hf45b732_1    conda-forge
libcxx                    12.0.1               habf9029_0    conda-forge
libdeflate                1.8                  h0d85af4_0    conda-forge
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libev                     4.33                 haf1e3a3_1    conda-forge
libevent                  2.1.10               h815e4d9_4    conda-forge
libffi                    3.4.2                he49afe7_4    conda-forge
libgfortran               5.0.0           9_3_0_h6c81a4c_23    conda-forge
libgfortran5              9.3.0               h6c81a4c_23    conda-forge
libglib                   2.70.0               hf1fb8c0_1    conda-forge
libiconv                  1.16                 haf1e3a3_0    conda-forge
liblapack                 3.9.0           12_osx64_openblas    conda-forge
libnetcdf                 4.8.1           nompi_h6609ca0_101    conda-forge
libnghttp2                1.43.0               h6f36284_1    conda-forge
libopenblas               0.3.18          openmp_h3351f45_0    conda-forge
libpng                    1.6.37               h7cec526_2    conda-forge
libprotobuf               3.18.1               hcf210ce_0    conda-forge
libsodium                 1.0.18               hbcb3906_1    conda-forge
libssh2                   1.10.0               h52ee1ee_2    conda-forge
libthrift                 0.15.0               hab56fdc_1    conda-forge
libtiff                   4.3.0                hd146c10_2    conda-forge
libutf8proc               2.6.1                h35c211d_0    conda-forge
libwebp-base              1.2.1                h0d85af4_0    conda-forge
libxcb                    1.13              h35c211d_1003    conda-forge
libxml2                   2.9.12               h7e28ab6_1    conda-forge
libxslt                   1.1.33               h1acebb3_3    conda-forge
libzip                    1.8.0                h8b0c345_1    conda-forge
libzlib                   1.2.11            h9173be1_1013    conda-forge
llvm-openmp               12.0.1               hda6cdc1_1    conda-forge
lxml                      4.6.4            py39hf41e7f8_0    conda-forge
lz4-c                     1.9.3                he49afe7_1    conda-forge
markupsafe                2.0.1            py39h89e85a6_1    conda-forge
matplotlib-base           3.4.3            py39hb07454d_1    conda-forge
matplotlib-inline         0.1.3              pyhd8ed1ab_0    conda-forge
mdtraj                    1.9.6            py39h996af62_1    conda-forge
mistune                   0.8.4           py39h89e85a6_1005    conda-forge
mmpbsa-py                 16.0                     pypi_0    pypi
mock                      4.0.3            py39h6e9494a_2    conda-forge
more-itertools            8.10.0             pyhd8ed1ab_0    conda-forge
msgpack-python            1.0.2            py39hf018cea_2    conda-forge
nbclient                  0.5.5              pyhd8ed1ab_0    conda-forge
nbconvert                 6.2.0            py39h6e9494a_0    conda-forge
nbformat                  5.1.3              pyhd8ed1ab_0    conda-forge
ncurses                   6.2                  h2e338ed_4    conda-forge
nest-asyncio              1.5.1              pyhd8ed1ab_0    conda-forge
netcdf-fortran            4.5.3           nompi_h9ed14b0_106    conda-forge
networkx                  2.6.3              pyhd8ed1ab_1    conda-forge
nglview                   3.0.3              pyh8a188c0_0    conda-forge
notebook                  6.4.5              pyha770c72_0    conda-forge
numexpr                   2.7.3            py39h4d6be9b_1    conda-forge
numpy                     1.21.4           py39h7eed0ac_0    conda-forge
ocl_icd_wrapper_apple     1.0.0                hbcb3906_0    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openff-bespokefit         27rc4904+28.g739f680.dirty          pypi_0    pypi
openff-forcefields        2.0.0              pyh6c4a22f_0    conda-forge
openff-fragmenter-base    0.1.2              pyhd8ed1ab_0    conda-forge
openff-qcsubmit           0.2.4              pyhd8ed1ab_0    conda-forge
openff-toolkit-base       0.10.1             pyhd8ed1ab_0    conda-forge
openff-utilities          0.1.1              pyh6c4a22f_0    conda-forge
openjpeg                  2.4.0                h6e7aa92_1    conda-forge
openmm                    7.6.0           py39h8d72adf_0_khronos    conda-forge
openssl                   1.1.1l               h0d85af4_0    conda-forge
orc                       1.7.0                hf163092_0    conda-forge
packaging                 21.2               pyhd8ed1ab_1    conda-forge
packmol                   20.010               h508aa58_0    conda-forge
packmol-memgen            1.2.1rc0                 pypi_0    pypi
pandas                    1.3.4            py39h4d6be9b_1    conda-forge
pandoc                    2.16.1               h0d85af4_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parmed                    3.4.3            py39h9fcab8e_1    conda-forge
parquet-cpp               1.5.1                         1    conda-forge
parso                     0.8.2              pyhd8ed1ab_0    conda-forge
pcre                      8.45                 he49afe7_0    conda-forge
pdb4amber                 20.1                     pypi_0    pypi
perl                      5.32.1          1_h0d85af4_perl5    conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
pickleshare               0.7.5           py39hde42818_1002    conda-forge
pillow                    8.3.2            py39he9bb72f_0    conda-forge
pint                      0.18               pyhd8ed1ab_0    conda-forge
pip                       21.3.1             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               hbcb3906_0    conda-forge
plotly                    5.3.1              pyhd8ed1ab_0    conda-forge
pluggy                    1.0.0            py39h6e9494a_2    conda-forge
prometheus_client         0.12.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.22             pyha770c72_0    conda-forge
prompt_toolkit            3.0.22               hd8ed1ab_0    conda-forge
psutil                    5.8.0            py39h89e85a6_2    conda-forge
pthread-stubs             0.4               hc929b4f_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
py                        1.11.0             pyh6c4a22f_0    conda-forge
py-cpuinfo                8.0.0              pyhd8ed1ab_0    conda-forge
pyarrow                   6.0.0           py39h4d6536f_0_cpu    conda-forge
pycairo                   1.20.1           py39hbe14034_1    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydantic                  1.8.2            py39h89e85a6_2    conda-forge
pygments                  2.10.0             pyhd8ed1ab_0    conda-forge
pymbar                    3.0.5            py39h329c335_2    conda-forge
pyopenssl                 21.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyhd8ed1ab_1    conda-forge
pyrsistent                0.18.0           py39h89e85a6_0    conda-forge
pysocks                   1.7.1            py39h6e9494a_4    conda-forge
pytables                  3.6.1            py39hfd850c7_4    conda-forge
pytest                    6.2.5            py39h6e9494a_1    conda-forge
pytest-asyncio            0.16.0             pyhd8ed1ab_0    conda-forge
pytest-celery             0.0.0a1            pyh9f0ad1d_1    conda-forge
pytest-cov                3.0.0              pyhd8ed1ab_0    conda-forge
python                    3.9.7           h1248fe1_3_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.9                      2_cp39    conda-forge
pytraj                    2.0.6                    pypi_0    pypi
pytz                      2021.3             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0              py39h89e85a6_3    conda-forge
pyzmq                     22.3.0           py39h7fec2f1_1    conda-forge
qcelemental               0.23.0             pyhd8ed1ab_0    conda-forge
qcengine                  0.20.1             pyhd8ed1ab_0    conda-forge
qcportal                  0.15.6             pyhd8ed1ab_0    conda-forge
rdkit                     2021.09.2        py39h88273a1_0    conda-forge
re2                       2021.09.01           he49afe7_0    conda-forge
readline                  8.1                  h05e3726_0    conda-forge
redis-py                  3.5.3              pyh9f0ad1d_0    conda-forge
redis-server              6.2.6                h05f672f_0    conda-forge
reportlab                 3.5.68           py39hf37cc50_0    conda-forge
requests                  2.26.0             pyhd8ed1ab_0    conda-forge
requests-mock             1.9.3              pyhd8ed1ab_0    conda-forge
rfc3986                   1.5.0              pyhd8ed1ab_0    conda-forge
rich                      10.13.0          py39h6e9494a_0    conda-forge
sander                    16.0                     pypi_0    pypi
scipy                     1.7.2            py39h056f1c0_0    conda-forge
send2trash                1.8.0              pyhd8ed1ab_0    conda-forge
setuptools                58.5.3           py39h6e9494a_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
smirnoff99frosst          1.1.0              pyh44b312d_0    conda-forge
snappy                    1.1.8                hb1e8313_3    conda-forge
sniffio                   1.2.0            py39h6e9494a_2    conda-forge
sqlalchemy                1.4.26           py39h89e85a6_1    conda-forge
sqlite                    3.36.0               h23a322b_2    conda-forge
starlette                 0.14.2             pyhd8ed1ab_0    conda-forge
tenacity                  8.0.1              pyhd8ed1ab_0    conda-forge
terminado                 0.12.1           py39h6e9494a_1    conda-forge
testpath                  0.5.0              pyhd8ed1ab_0    conda-forge
tk                        8.6.11               h5dbffcc_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     1.2.2              pyhd8ed1ab_0    conda-forge
tornado                   6.1              py39h89e85a6_2    conda-forge
torsiondrive              1.1.0              pyhd8ed1ab_0    conda-forge
tqdm                      4.62.3             pyhd8ed1ab_0    conda-forge
traitlets                 5.1.1              pyhd8ed1ab_0    conda-forge
typing-extensions         3.10.0.2             hd8ed1ab_0    conda-forge
typing_extensions         3.10.0.2           pyha770c72_0    conda-forge
tzdata                    2021e                he74cb21_0    conda-forge
urllib3                   1.26.7             pyhd8ed1ab_0    conda-forge
uvicorn                   0.15.0           py39h6e9494a_2    conda-forge
vine                      5.0.0              pyhd8ed1ab_1    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.37.0             pyhd8ed1ab_1    conda-forge
widgetsnbextension        3.5.2            py39h6e9494a_0    conda-forge
xmltodict                 0.12.0                     py_0    conda-forge
xorg-kbproto              1.0.7             h35c211d_1002    conda-forge
xorg-libice               1.0.10               h0d85af4_0    conda-forge
xorg-libsm                1.2.3             h0d85af4_1000    conda-forge
xorg-libx11               1.7.2                h0d85af4_0    conda-forge
xorg-libxau               1.0.9                h35c211d_0    conda-forge
xorg-libxdmcp             1.1.3                h35c211d_0    conda-forge
xorg-libxext              1.3.4                h0d85af4_1    conda-forge
xorg-libxt                1.2.1                h0d85af4_2    conda-forge
xorg-xextproto            7.3.0             h35c211d_1002    conda-forge
xorg-xproto               7.0.31            h35c211d_1007    conda-forge
xtb                       6.4.1                h67747af_1    conda-forge
xtb-python                20.2             py39h89e85a6_3    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
yaml                      0.2.5                haf1e3a3_0    conda-forge
zeromq                    4.3.4                he49afe7_1    conda-forge
zipp                      3.6.0              pyhd8ed1ab_0    conda-forge
zlib                      1.2.11            h9173be1_1013    conda-forge
zstd                      1.5.0                h582d3a0_0    conda-forge

Make the generating bespoke QC data more verbose

For the case of using DFT for openff-bespokefit the step of enerating bespoke QC data is quite slow and I think this could be more verbose.
I would imagine that this step is doing dihedral scan, so the program could echo that it is now doing dihedral 1 and 5 dihedrals left to be done.

Update to use openff-toolkit

Currently, the package is very hard to use due to the dependencies and a few other packages need to change first before we can update bespokefit.

TODO

• Update bespoke-fit to use openff-toolkit
• Wait for a new QCEngine release that uses openff-toolkit
• Wait for a new forcebalance release that uses openff-toolkit
• Update qcsubmit to use openff-toolkit

Executor run CLI not respecting `--qc-compute-n-cores`

Running the example in the docs using bespokefit version 0.1.0 installed via conda-forge sets up 2 qc workers but gives them both all available cores rather than the 8 specified via the CLI.

Support local/modified forcefields

Currently, we only support any of the installed forcefields openff forcefields as optimization starting points but we should expand this to local files if users want to run optimization on an already modified forcefield. This could be done by keeping the string version of the forcefield in the fitting schema.

Command to reproduce the tweet

Congratulations, it is nice to see the exciting new result in https://twitter.com/ColeGroupNCL/status/1526871422798266371?s=20&t=dDinH7MFq9w5r6nQTjYs7g.
I wonder if I could have the command to reproduce this force field, I guess it is openff-bespoke executor run --file "lig.sdf" --workflow "default" --output "lig.json" --output-force-field "lig.offxml" --default-qc-spec psi4 B3LYP-D3BJ DZVP?

Allow gateway port to be configured in CLI

The default gateway port is 8000, which is a commonly used port for HTTP servers. I just got this error trying to run BespokeFit:

Traceback (most recent call last):
  File "/home/joshmitchell/conda/envs/bespokefitdevdocsupdate/bin/openff-bespoke", line 33, in 
<module>
    sys.exit(load_entry_point('openff-bespokefit', 'console_scripts', 'openff-bespoke')())
  File 
"/home/joshmitchell/conda/envs/bespokefitdevdocsupdate/lib/python3.9/site-packages/click/core.py", 
line 829, in __call__
    return self.main(*args, **kwargs)
  File 
"/home/joshmitchell/conda/envs/bespokefitdevdocsupdate/lib/python3.9/site-packages/click/core.py", 
line 782, in main
    rv = self.invoke(ctx)
  File 
"/home/joshmitchell/conda/envs/bespokefitdevdocsupdate/lib/python3.9/site-packages/click/core.py", 
line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File 
"/home/joshmitchell/conda/envs/bespokefitdevdocsupdate/lib/python3.9/site-packages/click/core.py", 
line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File 
"/home/joshmitchell/conda/envs/bespokefitdevdocsupdate/lib/python3.9/site-packages/click/core.py", 
line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File 
"/home/joshmitchell/conda/envs/bespokefitdevdocsupdate/lib/python3.9/site-packages/click/core.py", 
line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/joshmitchell/Documents/openff/bespokefit/openff/bespokefit/cli/executor/run.py", line 
53, in _run_cli
    with BespokeExecutor(
  File "/home/joshmitchell/Documents/openff/bespokefit/openff/bespokefit/executor/executor.py", line
399, in __enter__
    self._start(asynchronous=True)
  File "/home/joshmitchell/Documents/openff/bespokefit/openff/bespokefit/executor/executor.py", line
346, in _start
    wait_for_gateway()
  File 
"/home/joshmitchell/Documents/openff/bespokefit/openff/bespokefit/executor/services/gateway.py", 
line 114, in wait_for_gateway
    raise RuntimeError("The gateway could not be reached.")
RuntimeError: The gateway could not be reached.

Turns out this was because the gateway port was already in use. The other software I was using allows the port to be configured so this wasn't a huge issue, but we should add a --gateway-port and possible a --redis-port CLI flag to allow this to be changed.

Gaff support

Dear openff developers, is it possible to use gaff as the initial force field? I tried to set it by the following code:

from openff.bespokefit.workflows import BespokeWorkflowFactory

workflow = BespokeWorkflowFactory(
    fragmentation_engine=None,
    optimizer=fb,
    parameter_hyperparameters=[],
    target_templates=[],
    default_qc_specs=[]
)
workflow.initial_force_field = 'data/gaff-2.1.xml'

The gaff-2.1.xml was downloaded from here , here is the error message:

OSError: Source data/gaff-2.1.xml could not be read. If this is a file, ensure that the path is correct.
If the file is present, ensure it is in a known SMIRNOFF encoding.
Valid formats are: ['XML']
Parsing failed with the following error:
not well-formed (invalid token): line 1, column 4

`BespokeWorkflowFactory.default_qc_specs` may lead to silent irreproducibility

Hi! Just had the thought that if I specify BespokeWorkflowFactory.default_qc_specs with two different QC engines, say one for a specific subset of molecules and then a second for a more general set, and then hand that factory schema off to a colleague who only has the latter installed, it might produce different results silently. This seems to go against the reproducibility focus of a lot of OpenFF's tooling. It might be better to error by default if a QC spec can't be satisfied by the executing machine.

General cache location

When running via the CLI users have to provide a working directory else the job is run in a temporary location and all files are removed after meaning that QC calculations and parameters can not be cached. It might be best to have a central location where the redis database is always stored, similar to setting up a qcfractal instance, by default and the directory cmd just overwrites this for the particular run.

We could then extend the CLI with some management commands which can clean out the cache if users want and restart/ remove calculations from the queue if they have errored.

`BespokeExecutor` context manager `__exit__` method could block

Hi - Just to explain all the issues I've been raising, I'm just raising them as I come across API poitns I think could be simpler.

If I understand correctly, the BespokeExecutor context manager quits immediately when the context manager ends its scope. I think this might be a frustrating footgun for users, and it might be better to have the __exit__() method block until all tasks complete. Users who want to run code asynchronously could still use the start() and stop() methods.

Examples

To help demonstrate all of the features of bespokefit we need some example notebooks/tutorials (look at evaluator/toolkit) for the inspiration they should be quick to run and give a general overview of the use of the project.

About the way soft torsion are identified

I'm interested in how bespokefit identifies the soft torsions to be scanned or exclude the hard torsions to not be scanned.
I wonder if there is any documentation for that?
If there is no documentation for that, I wonder if i could be pointed towards the part of the code that does.
Thank you.

Parallelisation with --n-qc-compute-workers

I'm running the openff-bespokefit with 12 works on a 96 core machine with the command
openff-bespoke executor run --file "lig.sdf" --workflow "default" --output "lig.json" --output-force-field "lig.offxml" --n-qc-compute-workers 12 --qc-compute-n-cores 8 --default-qc-spec psi4 B3LYP-D3BJ DZVP
I noticed that only 5 workers are being used (5 psi4 processes were being used). I wonder how is the parallelisation being set up, so I could better allocate the resources? Thank you.

Program hangs

Dear openff developers:
I came across some molecules which made the program hang at generating bespoke QC data for a long time (one day) and it consumes lots of memory (about 15G). But I did not see any backend QC program is running ( in this case I use torchani with GPU, but I did not see any GPU utilization). When I pause the program manually in debug mode, it stopped at the optimizeGeometry function.

 def optimizeGeometry(self):
        """
        High-level optimization loop.
        This allows calcEnergyForce() to be separated from the rest of the codes
        """
        self.calcEnergyForce()
        self.prepareFirstStep()
        while self.state not in [OPT_STATE.CONVERGED, OPT_STATE.FAILED]:
            self.step()
            if self.state == OPT_STATE.NEEDS_EVALUATION:
                self.calcEnergyForce()
                self.evaluateStep()
        if self.state == OPT_STATE.FAILED:
            raise GeomOptNotConvergedError("Optimizer.optimizeGeometry() failed to converge.")
        return self.progress

Here is one of the molecules:
mol25_minimised.zip

I used mostly the same code as the blog post, except xtb was replaced by torchani. Here are the fragments it generated:

Conda package

We want a complete conda package for users to perform the fitting due to how clashes with psi4 and the toolkit the QC package will be left optional.

• Make sure the package can run all features apart from the QC calculations
• do not include psi4
• Have the run function make sure the QCpackage is installed
• include torchani?

About the type of QM calculations that will be performed

I wonder if I could have some clarification with regard to the type of QM calculations that will be performed?
In the https://openff-bespokefit.readthedocs.io/en/latest/users/theory.html#optimization
It is said that

torsion profile: the torsion profile computed by performing a torsion scan using the current force field parameters against the reference QC torsion profile.

vibrational frequency: vibrational frequencies computed from MM hessian data against those computed from reference QC hessian calculations.

optimized geometry: internal coordinates of a conformer of the molecule minimized using the current values of the force field against the same conformer minimized using QC methods.

So I would imagine that QM hessian will be computed.
QM geometry optimisation will be performed as part of the TorsionDrive process.

I wonder if this understanding is correct? Thank you.

Change docs to recommend doing conda create

Since bespokefit pulls down a ton of other dependencies, defaulting to recommend conda install may make it hard for many users to get set up. So maybe we recommend the default path to be conda create.

Executor._error_cycle_task fails due to BespokeOptimizationSchema having no "status" attribute

The method fails here:
https://github.com/openforcefield/bespoke-fit/blob/7f02c8363e91e05bc404309ce58a678ad4eacc21/openff/bespokefit/bespoke/executor.py#L350

However, in the code above, task.status is only assigned under specific conditions:
https://github.com/openforcefield/bespoke-fit/blob/7f02c8363e91e05bc404309ce58a678ad4eacc21/openff/bespokefit/bespoke/executor.py#L311-L329

I'm still trying to figure bespokefit and the new API out but I'm guessing you may want to give BespokeOptimizationSchema a placeholder status attribute (=Status.Undefined) or throw in a if hasattr(task, "status")?

Edit: error trace here:

Process Process-1:
Traceback (most recent call last):
  File "/Users/lily/anaconda3/envs/bespokefit/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/Users/lily/anaconda3/envs/bespokefit/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/lily/pydev/bespoke-fit/openff/bespokefit/bespoke/executor.py", line 381, in error_cycle
    self._error_cycle_task(task=task, client=client)
  File "/Users/lily/pydev/bespoke-fit/openff/bespokefit/bespoke/executor.py", line 350, in _error_cycle_task
    elif task.status == Status.CollectionError:
AttributeError: 'BespokeOptimizationSchema' object has no attribute 'status'

Consider renaming `export_factory()` method to `to_file()`

BespokeWorkflowFactory has two methods that are inverses of each other: export_factory() and from_file(). export_factory() is unusual in that it is particular to the object being exported, and the fact that these are inverses of each other is not obvious. While the API is still floating, I'd suggest renaming export_factory() to to_file() and making to_file and from_file the conventional names for equivalent methods across the library.

Using the ANI to compute the single point energy and use xTB for geometry optimisation

In #152 (comment), it is mentioned that
Of course as we use QCEngine to do our QC reference generation you can use any supported package, however with ANI we often hit geometry optimisation convergence issues in testing but you are welcome to try yourself just add the following to your run command --default-qc-spec torchani ani2x none

I wonder if it is possible to use xTB for geometry optimisation and ANI for energy calculation? So as to skip the geometry optimisation convergence issue with ANI.

Default force field is 1.3.0

BespokeWorkflowFactory's default initial force field is Parsley 1.3.0. It should probably be updated to 1.3.1, or more likely Sage 2.0.0

EDIT: It might also be nice to allow initial_force_field to be set to an OpenFF Toolkit ForceField object. I imagine this would make it easier for people to optimize their pet force fields.

Use base classes rather than unions in type hints

In BespokeWorkflowFactory, and possibly elsewhere, several field's types are given as unions of classes that share a base class. For example:

target_templates: List[Union[TorsionProfileTargetSchema, AbInitioTargetSchema, VibrationTargetSchema, OptGeoTargetSchema]]
fragmentation_engine: Optional[Union[WBOFragmenter, PfizerFragmenter]]

And several others. It might be better to replace these unions with the base class, both to make it clear to the user what they need to do to extend functionality themselves, and to make adding new options easier in the future.

Default install doesn't bring in rdkit/ambertools

In testing today, we found that the openff-bespokefit conda package doesn't bring in ambertools/rdkit, leading to an opaque error if the user doesn't have openeye loaded/licensed, since it needs to do WBO-based fragmentation. So we might consider having it bring in RDK/AT by listing openff-toolkit as a dependency (instead of openff-toolkit-base).

Alternatively, we could update the install instructions to tell people they MUST install EITHER (rdkit+ambertools) OR (openeye) separately. @jthorton points out that AmberTools makes the psi4 environment resolution even harder, so users with OpenEye could save some install time by leaving AmberTools out.

Repeated calculations for symmetry equivalent fragments

When passing in a molecule that has symmetry and symmetry equivalent fragments bespokefit is unable to deduplicate these tasks resulting in repeated calculations. We also generate redundant SMIRKS patterns for each symmetry equivalent fragment meaning that only the last pattern in the hierarchy will actually match and be optimized. This is also an issue in the optimization stage as we take the gradient of parameters that are not actually present in the system, this is wasteful and avoiding this would speed up the optimization stage. An example of this is Dimethyl hydrogen phosphate which runs two torsiondrives with default fragmentation settings when only one is needed.

I think that by simply removing symmetry equivalent fragments from the fragmentation result we should avoid this issue as the parameters derived for one fragment will still transfer to the rest.

Theory section notes

The following is a list of theory section notes that should be fleshed out into a full docs section. I think the main parts of the workflow are well described in the docs already so I have mainly focused on the smirks generation and parameter cache, I have included some notes on figures that might help explain some of the ideas and will add them once we agree on which ones are needed.

Do we think we should describe the features used for each cache so users can tell when bespokefit will/won't reuse results?

tagging @SimonBoothroyd @j-wags @Yoshanuikabundi

Fragmentation

• Molecules are fragmented to reduce the computational cost of generating the reference QC data.
• Currently, only torsiondrives take advantage of fragmentation.
• Bespokefit can use fragmentation engines provided by openff-fragmenter directly and by default uses the WBO fragmentation method.
• In short, it builds the minimum fragment around each non-terminal rotatable bond which is electronically decoupled from the rest of the molecule by ensuring the WBO in the bond is the same in the fragment and the parent molecule. Link back to the theory in the fragmenter docs.

SMIRKS Generation

• After fragmentation we can produce the bespoke SMIRKS patterns here we focus on the torsion smirks.
• We will have one fragment per rotatable bond (note some fragments may be the same if the torsions are coupled.)
• Maybe a picture of the parent molecule with rotatable bonds highlighted and arrows from the bond to the resulting fragment. Best to pick a molecule with some repeated fragments (the TYK2 series is a good example).
• Then we use chemper to automatically generate new bespoke smirks patterns for each symmetry unique group of torsions that pass through the targeted central bond.
• First, we get the atom symmetry groups using rdkit or openeye and then assign each torsion a symmetry code in the form of a tuple of atom symmetry groups corresponding to each atom in the torsion.
• Torsions are then grouped by this torsion symmetry code and a bespoke smirks pattern is made for each symmetry unique group of torsions.
• This helps minimise the number of parameters we introduce which helps simplify the optimisation problem while still having enough degrees of freedom to achieve good accuracy.
• Image of a fragment with the symmetry groups labelled next to another image with some symmetrical torsions highlighted.
• Chemper is then used to generate a pattern that targets all symmetry equivalent torsions per torsion symmetry group in both the parent and fragment using the ClusterGraph class.
• Layers is set to all to ensure the maximum common substructure between the parent and fragment is encoded into the resulting bespoke smirks pattern.
• This ensures the pattern is transferable between the fragment and parent with no further modifications but also ensures it is specific enough to make sure the pattern can only be applied to other molecules which share the electronically decoupled substructure as determined by fragmenter.
• This is important as the parameter will account for non-local effects which can not be encoded using a standard atom type approach where by parameters are stored under the combination of the atom types that make up the torsion.
• Image of a bespoke SMIRKS pattern and with the corresponding parent and fragment molecule. The molecules should have a substructure in the SMIRKS highlighted.
• In the case of a congeneric series like the TYK2 this means that the common core need only be parameterised once, the bespoke parameters are then cached and can be reused.
• Grid image of the TYK2 series with the matched atoms highlighted from the above SMIRKS pattern.
• By default, we take a conservative approach when generating bespoke torsion parameters and expand the k terms of our reference parameter to include all periodicities from 1 to 4 (can be changed).
• In many cases, this is not needed and leads to redundant parameters which are held very close to the initial guess of 0 by the L1 regulariser in forcebalance.

QC Generation

• for each molecule/fragment (only in the case of torsiondrives) and fitting target combination a QC reference data generation task is created.
• To avoid unnecessary torsiondrive calculations this stage will only generate tasks for fragments that can not reuse any cached bespoke parameters.
• Tasks include optimisations, hessians and 1D torsiondrives. Each task is then submitted to the QC generation queue and consumed by celery workers which run the job via QCEngine thus any qcengine supported program can be used to calculate the reference data.
• Once a task has successfully completed we also cache the result for later use which can save significant time when refitting a torsion with different optimisation settings to the same reference data.
• For high throughput bespoke parameter deviation we recommend using the fantastic QCFracal distributed computing and database for quantum chemistry.
• Users can easily set up a local server and submit large datasets of torsiondrives using QCSubmit which can then be used as reference data by bespokefit via the cache update feature- need a section on how to pull down qcarchive data and cache locally.

Optimisation

• We use the same fitting techniques to optimise the bespoke parameters as those used to make the mainline openff force fields.
• In short torsion parameters are optimised by minimising the RMSE difference between the QM and MM potential energy surfaces.
• Due to limitations in the functional form of the MM force field we perform a restrained relaxation from the QM reference minimums before computing the potential energy.
• This helps relax high energy steric clashes while the restraints keep us close to the QM minimum.
• By default we use quite large priors on the torsion k values so we are not hindered by our initial guess parameters which are taken from the reference force field or 0 if the term was not already present.
• Once optimisation has been completed successfully the bespoke parameters are cached so that they can be reused on any other molecule which shares this substructure provided the optimisation settings remain the same.

The DZVP basis set in psi4

In the doc (https://openff-bespokefit.readthedocs.io/en/latest/users/theory.html?highlight=DZVP#qc-generation), it is said that
The default fitting workflow currently generates all reference data at the B3LYP-D3BJ/DZVP level of theory. This balances computational efficiency against accurately reproducing conformations generated using higher levels of theories. This is also the level of theory used in training the main OpenFF force fields, and so ensures that bespoke and general parameters can be mixed with minimal compatibility concerns.

However, I think the only QM engine that is supported by openff-bespokefit is psi4 and psi4 doesn't have the DZVP basis set https://psicode.org/psi4manual/1.5.0/basissets_tables.html?

Failed to generate SMIRKS patterns that match both the parent and torsion fragments

Hello,

Bespoke is failing for a molecule at the optimization stage and outputs the following:

[✓] qc-generation successful
[x] optimization failed

 "Failed to generate SMIRKS patterns that match both the parent and torsion fragments: "
 
worker: Warm shutdown (MainProcess)
worker: Warm shutdown (MainProcess)
worker: Warm shutdown (MainProcess)

This file contains the bespoke-executor folder, the sdf and workflow.json for running bespokefit, and the script that made the sdf/json files here: OA-fragment-3.zip

Is there any way to have bespoke work for this molecule?

SnowFlake Reliability

The Snowflake qcfractal instance can be unreliable sometimes so we have added support to export QCSubmit datasets that can be run in a persistent QCArchive instance. We should also work on making the SnowFlake more reliable and configurable to support quick QC/ML scans.

• add controls so the user can chose how many workers to spin up and how much memory and cores they each get.
• Add tests for the snowflake server to catch common issues
• Make sure only one instance of the server is used at any time, as some stages use multiprocessing this could be copying the database.

COOH issue

I'm trying to setup bespoke for a new user and in testing I'm running into a COOH conformer issue at the fragmentation stage.
This happened previously and a workaround was created here but for some reason I can't get it to work this time around.

These are my setup steps:

[1] git clone https://github.com/openforcefield/openff-bespokefit.git and move into openff-bespokefit/
[2] Add xtb-python to devtools/conda-envs/test-env.yaml
[3] create project env via conda env update -n bespoke --file devtools/conda-envs/test-env.yaml
[4] python setup.py develop in openff-bespokefit/
[5] pip install git+git://github.com/openforcefield/bespoke-fit.git@931bb66bb3700208616e41bfde1a80d90a37d74f in openff-bespokefit/
[6] Openeye fragmenter issue workaround which no longer seems to working(?):

• git clone https://github.com/openforcefield/openff-fragmenter.git
• git checkout continue-if-elf-fails in openff-fragmenter/
• execute python setup.py develop in openff-fragmenter/

The notebook I'm using:
COOH_error.ipynb.zip
COOH_error.html.zip

I'm not sure what's going wrong here

Consider adding a `--smiles` option to the CLI

Over in Toolkit land, we often document preferring a SMILES string to a coordinate file, as it makes it harder to make mistakes where the molecular graph is misinterpreted. Consistent with that, it would be good if we could add a --smiles option to the CLI that can be used in place of --input to provide the target molecule.

shut down a redis server

So I was trying to run something but I noticed that I did something wrong. So I use control+C to terminate the run. Then if I try to start a new run, I would get

[ERROR] a redis server is already running at host=localhost and port=6363, continuing to run would likely causeunintended consequences.
⠋ launching the bespoke executor

I wonder how do I terminate the redis server? Thank you.

Using resulting forcefield

After a user submits an optimisation using openff-bespoke executor run they will get a json output file which is a serialised CoordinatorGetResponse object which does contain the final refit force field but is awkward to get to and involves something like

from openff.qcsubmit.serializers import deserialize
from openff.bespokefit.schema.results import BespokeOptimizationResults

data = deserialize("output.json")
result = BespokeOptimizationResults.parse_obj(data["results"])
force_field = result.refit_force_field

Maybe we should add a CLI entry point to extract the force field quickly from a result and write out the offxml file, something like
openff-bespoke result extract <output.json> new_ff.offxml
It might also then make sense to have a command which can combine multiple outputs to a single force field, like in the case of parametrising a congeneric series for binding free energy calculations which require a single input force field this could be
openff-bespoke result combine <list of results>. new_ff.offxml.

Create single-file installers with Psi4 bundled

After #161, we now have a turnkey for creating single-file installers with the latest version of bespokefit bundled alongside AmberTools, RDKit, and XTB, on both macOS and Linux.

We should also create installers with Psi4 bundled. These will be available on Linux only because of long-standing incompatibilities between AmberTools and Psi4 on macOS, but this should not be an issue given the 0% market share of macOS in production-scale hardware.

As a reminder, OpenEye Toolkits will not be included in any installers we produce as we currently do not have permission to repackage them in this way. This, however, does not prevent a conda install openeye-toolkits -c openeye after unpacking the installer for those that can run conda install.

Export to other format

Hi, I wonder if it is possible to export the FF into other format? Such as the gromacs topology.

I usually use parmed to convert the topology but in this case, it won't work.

from simtk.openmm.app import *
from simtk.openmm import *
from simtk.unit import *
from sys import stdout
import parmed

    # Load PDB
    mol = parmed.load_file('lig.sdf') # This is the input
    mol[0].save('lig.pdb')
    pdb = PDBFile('lig.pdb')
    # Create system
    ff = ForceField('lig.offxml')
    system = ff.createSystem(pdb.topology,
                             nonbondedMethod=PME,
                             nonbondedCutoff=12.0 * angstroms,
                             switchDistance=10.0 * angstroms,
                             constraints=HBonds, flexibleConstraints=True)
    structure = parmed.openmm.load_topology(pdb.topology, system=system,
                                            xyz=pdb.positions)
    structure.save('lig.top', format='gromacs', overwrite=True)

Gives

Warning: importing 'simtk.openmm' is deprecated.  Import 'openmm' instead.
Traceback (most recent call last):
  File "/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3361, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-dfaf84a31ef0>", line 1, in <cell line: 1>
    runfile('/Users/zwu/Documents/protein-ligand-benchmark/2020-02-07_tyk2/bespokefit/convert.py', wdir='/Users/zwu/Documents/protein-ligand-benchmark/2020-02-07_tyk2/bespokefit')
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 198, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/zwu/Documents/protein-ligand-benchmark/2020-02-07_tyk2/bespokefit/convert.py", line 18, in <module>
    system = ff.createSystem(pdb.topology,
  File "/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.9/site-packages/openmm/app/forcefield.py", line 1212, in createSystem
    templateForResidue = self._matchAllResiduesToTemplates(data, topology, residueTemplates, ignoreExternalBonds)
  File "/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.9/site-packages/openmm/app/forcefield.py", line 1427, in _matchAllResiduesToTemplates
    raise ValueError('No template found for residue %d (%s).  %s' % (res.index+1, res.name, _findMatchErrors(self, res)))
ValueError: No template found for residue 1 (UNL).  This might mean your input topology is missing some atoms or bonds, or possibly that you are using the wrong force field.

This seems to be an openMM issue so I wonder if I can have some of your help. Thanks.

Add a CLI

It would be good to get a basic CLI working to help first-time users get familiar with the project, there are a few key functions that would be useful to start with.

CLI Features

• Build a fitting schema for a list of molecules
• Build QCSubmit datasets for a given fitting schema
• Combine QC results into a fitting schema
• Check the status of a schema to see if it is ready to fit
• Run a fitting schema
• Extract a final forcefield from a complete fitting schema

Find OE_liscence from ~/.OpenEye/oe_license.txt

It is quite common that one doesn't explicitly set the OE_LICENSE variable but puts it in the position as is mentioned in https://docs.eyesopen.com/applications/common/license.html

So it would be good to check if the oe_license.txt is in

~USERNAME/.OpenEye
/Users/USERNAME/Library/OpenEye
~USERNAME
C:\Users\USERNAME\AppData\Local\OpenEye

Before saying that cannot find it.

"IndexError: list index out of range" when no fragments are generated by fragmentation_engine

Building a schema with molecules like ethanol failed at

from openff.bespokefit.workflows import bespoke
workflow = bespoke.BespokeWorkflowFactory(
    initial_force_field="openff_unconstrained-1.3.0.offxml", 
)

from openff.toolkit.topology import Molecule 
eth_mol = Molecule.from_smiles("CCO")
schema = workflow.optimization_schema_from_molecule(molecule=eth_mol)

with the following error:

bespoke-fit/openff/bespokefit/workflows/bespoke.py in optimization_schema_from_molecule(self, molecule, index)
394         # the parent molecule is put in canonical order by fragmenter to use the new parent
395         attributes = MoleculeAttributes.from_openff_molecule(
--> 396             molecule=fragment_data[0].parent_molecule
397         )
398         molecule_schema = MoleculeSchema(
IndexError: list index out of range

I'm guessing perhaps adding a condition to check for fragments after calling the fragmentation_engine and if none generated to directly proceed with the parent molecule could fix this.

Pin celery version in c-f recipe.

installing bespokefit and psi4 via mamba create -n bespoke psi4 openff-bespoke -c psi4 -c defaults solves to a celery version which is too old. We need to pin to at least 5.2.1 in the recipe.

Support optimisation of specific dihedrals

Currently, an input molecule is fragmented on all rotatable torsions, however it would be good to offer a way for users to select certain rotatable bonds and only refit those. Maybe we could follow qcsubmit and allow users to tag a bond and store the data in the molecules.properties dictionary something like

# tag the rotatable bonds to be refit
mol.properties["dihedrals"] = [(0, 1), (1, 2)]