hackingmaterials / atomate

atomate is a powerful software for computational materials science and contains pre-built workflows.

atomate is a software for computational materials science that contains pre-built workflows to compute and analyze the properties of materials.

• Website (including documentation): https://hackingmaterials.github.io/atomate/
• Help/Support: https://discuss.matsci.org/c/atomate
• Source: https://github.com/hackingmaterials/atomate

If you find atomate useful, please encourage its development by citing the following paper in your research output:

Mathew, K., Montoya, J. H., Faghaninia, A., Dwarakanath, S., Aykol,
M., Tang, H., Chu, I., Smidt, T., Bocklund, B., Horton, M., Dagdelen,
J., Wood, B., Liu, Z.-K., Neaton, J., Ong, S. P., Persson, K., Jain,
A., Atomate: A high-level interface to generate, execute, and analyze
computational materials science workflows. Comput. Mater. Sci. 139,
140-152 (2017).

atomate's Issues

Calc_loc for specifying previous OptimizeFW for a StaticFW?

I'm cleaning up some old MatMethods workflows to push to atomate and I noticed that the current atomate.vasp.fireworks.StaticFW no longer has keywords for calc_loc.

Within a single workflow, I run OptimizeFW for two structures in parallel followed by a StaticFW for each structure (these structures are then used together later to create interpolated structures). I use calc_loc to specify which of the OptimizeFW runs the StaticFW should use for copy_vasp_outputs.

It's my understanding that calc_loc = True will grab the location of the most recently completed calculation (rather than the parent) so I cannot use it for this situation. Is this the case? If so, what's the best work around? Creating a new class that inherits from StaticFW that has this parameter?

Serialized VaspInputSets don't contain user_incar_settings added after instantiation

It is sometimes the case that I would like to use one customized input set as a base and change the parameters (e.g. ISIF) programmatically later down the line.
The most straightforward way I can see to do this is to update the user_incar_settings dict after instantiating the input set.
In code, what I want to do is:

def relax_743(structure, my_input_set):
    vis1 = copy(my_input_set)
    vis1.user_incar_settings.update({'ISIF': 7})
    fw1 = OptimizeFW(structure, vasp_input_set=vis1)

    vis2 = copy(my_input_set)
    vis2.user_incar_settings.update({'ISIF': 4})
    fw2 = OptimizeFW(structure, vasp_input_set=vis2, parents=fw1)

    vis3 = copy(my_input_set)
    vis3.user_incar_settings.update({'ISIF': 3})
    fw3 = OptimizeFW(structure, vasp_input_set=vis3, parents=fw2)
    
    fws = [fw1, fw2, fw3]
    return Workflow(fws)

When doing this, serializing the input sets doesn't work:

from pymatgen import Structure
from pymatgen.io.vasp.sets import MPRelaxSet

### Setting during initialization

vis = MPRelaxSet(Structure.from_file('POSCAR'), user_incar_settings={'ISIF': 7})

print(vis.as_dict()['user_incar_settings'])
# prints:
# {'ISIF': 7}


### Setting after initialization

vis = MPRelaxSet(Structure.from_file('POSCAR'))

print(vis.user_incar_settings)
# prints (as expected, we didn't set any user incar settings)
# {}

vis.user_incar_settings.update({'ISIF': 7})
print(vis.user_incar_settings)
# prints (as expected, we just set this)
# {'ISIF': 7}

print(vis.as_dict()['user_incar_settings'])
# KeyError! The INCAR settings are not serialized

@shyuep is this the intended behavior of the VaspInputSets?

If it is the intended behavior, is there a better way to do this? In atomate, we pass everything by instance, but maybe we should instantiate inside the Firework.__init__ method (for the subclass of Firework) and instead pass the class MPRelaxSet rather than an instance of MPRelaxSet as arguments to the Fireworks?

Running workflows docs - example should be more intuitive

@bocklund

The goal of the Running Workflows docs is to demonstrate atomate feeling "magical" - i.e., things that are normally difficult become super easy. i.e., the steps should be short and intuitive. This is our primary example to demonstrate why someone spent so much time installing the software. After that, the rest should be easy.

The example doesn't really demonstrate that.

1. If I were a new user, setting up the eos workflow wouldn't seem like something I could do myself. That YAML file is super complicated containing all sorts of foreign terms like "TransmuterFW" that a new user will have no clue about. And it's also really long and tedious - not exactly short and tidy. I have no idea where to start in terms of looking.
2. If I were a new user, analyzing the eos workflow wouldn't seem like something I could do myself. It seems to require intimate knowledge of what's being put in the database - field names, etc. - and even after that requires quite a bit of extra coding on my part. Feels more like manual labor that requires a lot of prior knowledge than magical. And I don't even get the fit of the equation of state at the end, despite the workflow name! So it's pretty much a letdown.

A better example would be:

1. Add a band structure workflow with one line. Either of the below

atwf add Si.cif -p wf_bandstructure
OR
atwf add Si.cif -l vasp -s bandstructure.yaml
OR
atwf add -m mp-149 -p wf_bandstructure

You can take the opportunity to point out all the other preset workflows that are available as Python functions (option 1) or YAML (option 2). e.g., if you just want to optimize, use the optimize_only.yaml file. Plus the YAML file is super clean for those. Also point out that they have access to 60,000 crystal structures if they use the -m option. A newcomer won't think the code is about having huge YAML specifications that are for expert coders.

2. Run using:
   qlaunch -r rapidfire

They already met this command in the install tutorial. If they have the ability to run locally (not everyone does, e.g. NERSC officially doesn't allow this even if you can technically do it), then you can point them to rlaunch as well as a vasp_cmd that is more basic (see my other ticket about this)

3. Analyze and plot using pymatgen. I'd suggest something like:

• Use VaspCalcDb to connect to MongoDB
• Use the get_band_structure() and get_dos() methods of the above to get band structure and dos objects
• print out a few attributes of the above objects like band gap or fermi energy
• Use BSDosPlotter() to produce a pretty plot

All 4 of the above steps can probably be done in 10 lines of code or less - with each line being very short and clear about what it's doing (almost like English).

If there are more complex analysis that require digging into the database, you can demonstrate those too. But it shouldn't be the first thing the user is exposed to and the user shouldn't get the feeling that is what's needed to extract useful information from atomate workflows.

For the current EOS example, I'd suggest:

• encouraging a preset workflow instead of the YAML format, which misses the all-important fitting portion (and one of the great benefits of using atomate)
• having the analysis be little more than querying the database for the result or inspecting the JSON file in the result directory. You can also give an example of manual analysis for the brave (retain the current analysis) but again one should not get the impression that this is what is expected.

Rename mmwf utility

Summary

The mmwf script should probably be renamed to reflect MatMethods -> atomate like the other recent changes

LepsFW no longer works due to deprecated decorator

@matk86 can you fix soon? All LepsFW will fail until fixed.

Run this

from atomate.vasp.fireworks import LepsFW
from pymatgen import Lattice, Structure

coords = [[0, 0, 0], [0.75, 0.5, 0.75]]
lattice = Lattice([[3.8401979337, 0.00, 0.00],
                        [1.9200989668, 3.3257101909, 0.00],
                        [0.00, -2.2171384943, 3.1355090603]])
struct = Structure(lattice, ["Si"] * 2, coords)

lfw = LepsFW(struct)

print(lfw)

You'll hit this error

/Users/ajain/PycharmProjects/scratch_work/scratch/scratch3.py:10: DeprecationWarning: LepsFW is deprecated
This firework will be removed soon. Use DFPTFW and/or RamanFW fireworks.
  lfw = LepsFW(struct)
Traceback (most recent call last):
  File "/Users/ajain/PycharmProjects/scratch_work/scratch/scratch3.py", line 10, in <module>
    lfw = LepsFW(struct)
  File "/Users/ajain/Documents/code_matgen/monty/monty/dev.py", line 54, in wrapped
    return old(*args, **kwargs)
  File "/Users/ajain/Documents/code_matgen/atomate/atomate/vasp/fireworks/core.py", line 244, in __init__
    super(LepsFW, self).__init__(t, parents=parents, name="{}-{}".format(
TypeError: super() argument 1 must be type, not function

Fireworks from Firetasks in atwf

• One feature of atomate is being able to synthesize and analyze results automatically e.g. FitEquationOfStateTask
• Another is the use of atwf to run and share repeatable workflows without writing code

It appears that these two things do not, at least not obviously. @shyuep can you comment on this? I know you expressed interest in the paper on discussing this and I am working on documentation to include atwf.

Any suggestions on a good way to implement this? Two I can think of are:

1. Wrap the tasks as Fireworks in the code. We end up doing this programmatically in several workflows for different analysis tasks anyway, so maybe each analysis task should have its own Firework.

2. Extend atwf to handle custom Fireworks instead of looking up the Firework by module/class.

How to reference calc_dir of non parent run?

Suppose I have a workflow in which the job dependency DAG and the copying of previous output DAG are not the same. Here’s an example workflow:

fireworks:

• fw: classA # 0th firework
• fw: classB # 1st firework
  params:
  parents: 0
• fw: classC # 2nd firework
  params:
  parents: 1
  grab_output_from: 0

The 2nd firework will not run until the 1st is complete, but I would like the 2nd firework to run a continuation job from the output files of the 0th firework. What is the best way for the 2nd firework to get the calculation directory of the 0th firework? Can this be specified in the YAML file?

Thanks!

POTCAR modifications ignored in elastic workflow

System

• Version: master
• Python version: 3.5.3
• OS version: fedora 25

Summary

• Modifying the vasp_input_set as recommended here is ignored after the workflow is generated.
• PAW_PBE Sm_3 07Sep2000 is used by default, I would like to use PAW_PBE Sm 21Nov2014

Example code

from pymatgen.io.vasp.inputs import Poscar
from pymatgen.io.vasp.sets import MPStaticSet
from atomate.vasp.workflows.base.elastic import get_wf_elastic_constant
from fireworks import LaunchPad

lp = LaunchPad.auto_load()
poscar = Poscar.from_file("POSCAR")
structure = poscar.structure
vis_input = MPStaticSet(structure)
vis_input.config_dict['POTCAR']['Sm'] = 'Sm'   # Changed here from Sm_3
wf = get_wf_elastic_constant(structure, vasp_input_set=vis_input)
lp.add_wf(wf)

Error message

• N/A

Suggested solution (if any)

• It seems to be passed correctly all the way to TransmuterFW. I'm not sure what causes it to be modified later.

• If no solution is easily available, is there an equivalent to add_modify_incar (but add_modify_potcar or similar) that can work as a post-fix?

Gibbs workflow tutorial pymongo error

System

• Version: "master" (dev)
• Python version: 2.7.13
• OS version: SuSE Linux Enterprise Server (SLES) (Cori compute node)

Summary

• Gibbs workflow tutorial's GibbsAnalysisToDb throws pymongo error during Elastic Tensor Fitting Firework

Example code

The main script from the Gibbs workflow tutorial:

#!/usr/bin/env python
import numpy as np
from pymatgen import MPRester
from fireworks import LaunchPad
from atomate.vasp.workflows.presets.core import wf_gibbs_free_energy

with MPRester() as mpr:
    struct = mpr.get_structure_by_material_id('mp-149')

deformations = [(np.eye(3)*((1+x)**(1.0/3.0))).tolist() for x in np.linspace(-0.1, 0.1, 7)]
c = {"T_MIN": 10, "T_STEP": 10, "T_MAX": 2000,
     "POISSON": 0.20, "ANHARMONIC_CONTRIBUTION": True,
     "DEFORMATIONS": deformations}

workflow = wf_gibbs_free_energy(struct, c)

launchpad = LaunchPad.auto_load()
launchpad.add_wf(workflow)

Followed by qlaunch singleshot

Error message

Traceback (most recent call last):

  File \"/global/homes/a/ardunn/.conda/envs/atomate_dev_env/lib/python2.7/site-packages/fireworks/core/rocket.py\", line 253, in run
    m_action = t.run_task(my_spec)
  File \"/global/u2/a/ardunn/tutorials/atomate_dev/codes/atomate/atomate/vasp/firetasks/parse_outputs.py\", line 547, in run_task
    coll.insert_one(gibbs_dict)
  File \"/global/homes/a/ardunn/.conda/envs/atomate_dev_env/lib/python2.7/site-packages/pymongo/collection.py\", line 670, in insert_one
    bypass_doc_val=bypass_document_validation),
  File \"/global/homes/a/ardunn/.conda/envs/atomate_dev_env/lib/python2.7/site-packages/pymongo/collection.py\", line 575, in _insert
    check_keys, manipulate, write_concern, op_id, bypass_doc_val)
  File \"/global/homes/a/ardunn/.conda/envs/atomate_dev_env/lib/python2.7/site-packages/pymongo/collection.py\", line 556, in _insert_one
    check_keys=check_keys)
  File \"/global/homes/a/ardunn/.conda/envs/atomate_dev_env/lib/python2.7/site-packages/pymongo/pool.py\", line 482, in command
    self._raise_connection_failure(error)
  File \"/global/homes/a/ardunn/.conda/envs/atomate_dev_env/lib/python2.7/site-packages/pymongo/pool.py\", line 610, in _raise_connection_failure
    raise error
InvalidDocument: Cannot encode object: array([-10.88950285])

Suggested solution (if any)

• Seems to be a problem trying to store numpy arrays w/ pymongo
• Probably can be fixed with a simple .tolist()

unit tests give error "'ElasticTensor' object has no attribute 'kg_average'"

System

• Version: master

Summary

• Running the unit tests locally gives an error in (atomate.vasp.workflows.tests.test_elastic_workflow.TestElasticWorkflow)
• Issue 1: the test fails
• Issue 2: the test does not fail on CircleCI - why? Is it due to older pymatgen on CircleCI?

Example code

Run the unit tests

Error message

Traceback (most recent call last):
  File "/Users/ajain/Documents/code_matgen/atomate/atomate/vasp/workflows/tests/test_elastic_workflow.py", line 155, in test_wf
    self._check_run(d, mode="elastic analysis")
  File "/Users/ajain/Documents/code_matgen/atomate/atomate/vasp/workflows/tests/test_elastic_workflow.py", line 126, in _check_run
    c_ij = np.array(d['elastic_tensor'])
TypeError: 'NoneType' object has no attribute '__getitem__'

Also

Traceback (most recent call last):
  File "/Users/ajain/Documents/code_matgen/fireworks/fireworks/core/rocket.py", line 229, in run
    m_action = t.run_task(my_spec)
  File "/Users/ajain/Documents/code_matgen/atomate/atomate/vasp/firetasks/parse_outputs.py", line 242, in run_task
    kg_average = result.kg_average
AttributeError: 'ElasticTensor' object has no attribute 'kg_average'

Suggested solution (if any)

• In March 15 @montoyjh seemed to remove the kg_average parameter from the ElasticTensor object. This should be updated...

issues in drone parsing

@matk86 I am looking over the results of the MatMethods drone parsing a run with both "relax1" and "relax2" jobs and see several issues:

1. calcs_reversed.0.completed_at is earlier than calcs_reversed.1.completed_at. Remember that the zero index is supposed to be the later job. Do you know why this is? Other properties might be OK and in the right order. e.g. I checked the final energies and the zero index indeed gives the correct final energy of relax2. But the "completed_at" looks wrong, and maybe other things are too? Can you check this?
2. the "run_stats" (root key) has sub-keys "standard" and "relax2". This should be "relax1" (not standard) and "relax2". Mainly, I feel the sub-keys in "run_stats" should match the labels in "calcs_reversed.x.task.name" if the latter are present.
3. In the root "output" key, there is an "outcar" subkey. I would delete this outcar subkey; it is redundant with calcs_reversed.0.output.outcar. Besides, it looks like it is using the wrong outcar (the calcs_reversed.1.output.outcar)!!!

You should be able to reproduce the above errors by running your drone on:
/MatMethods/matmethods/vasp/tests/reference_files/Si_structure_optimization_relax2/outputs

However, in case you are not seeing the above problems, let me know. The actual problems I saw were in a directory at NERSC (/global/project/projectdirs/m2439/matmethods_test/scratch/block_2016-04-15-21-06-54-570920/launcher_2016-04-15-21-06-54-727961) and I will need to give you access.

vasp firework does not check env for incar_update

System

• Version: master
• Python version: python 3.6
• OS version: CentOS release 6.8

Summary

the <env.incar_update> does not affect incar in wf_structure_optimization

Example code

1. my_fworker.yaml

name: 'SJTU_Pi'
category: ''
query: '{}'
env:
    db_file: '/lustre/home/umjzhh-1/keliu/envs/atomate/config/db.json'
    vasp_cmd: 'srun -v -n 16 vasp'
    scratch_dir: '/lustre/home/umjzhh-1/keliu/envs/atomate/scratch'
    incar_update:
        NCORE: 4

2. submit script

from atomate.vasp.workflows.presets.core import wf_structure_optimization
from pymatgen.matproj.rest import MPRester
from fireworks.core.launchpad import LaunchPad


a = MPRester()
struct = a.get_structure_by_material_id('mp-153')
struct.make_supercell([4, 4, 3])
wf = wf_structure_optimization(struct)
lp = LaunchPad.auto_load()
lp.add_wf(wf)

Error message

there is not NCORE in my INCAR file, and the following message is my .out file, and it didn't show any firetask related to modify incar, unless adding incar_update to vis.

Atomate environment of KeLiu
2017-02-14 10:32:49,002 INFO Hostname/IP lookup (this will take a few seconds)
2017-02-14 10:32:49,014 INFO Launching Rocket
2017-02-14 10:33:02,410 DEBUG FW with id: 3 is unique!
2017-02-14 10:33:02,415 DEBUG Created/updated Launch with launch_id: 5
2017-02-14 10:33:02,576 DEBUG RUNNING FW with id: 3
2017-02-14 10:33:02,576 INFO RUNNING fw_id: 3 in directory: /lustre/home/acct-umjzhh/umjzhh-1/launcher/block_2016-10-12-18-29-00-023399/launcher_2017-02-14-02-32-36-803633
2017-02-14 10:33:02,623 INFO Task started: FileWriteTask.
2017-02-14 10:33:02,628 INFO Task completed: FileWriteTask
2017-02-14 10:33:02,629 INFO Task started: {{atomate.vasp.firetasks.write_inputs.WriteVaspFromIOSet}}.
2017-02-14 10:33:02,957 INFO Task completed: {{atomate.vasp.firetasks.write_inputs.WriteVaspFromIOSet}}
2017-02-14 10:33:02,957 INFO Task started: {{atomate.vasp.firetasks.run_calc.RunVaspCustodian}}.

Suggested solution (if any)

• <If you have a suggestion on how to solve the issue, you may write it
  here.>

Files (if any)

<If input files are needed to reproduce the error, please provide
either links (e.g., Dropbox or GDrive) for large files or simply paste
the file below for small files.>

<contents of file 1>

Default procedures for optimizations

I think the whole MP workflow can use a few tune-ups. For structure relaxations, instead of the current dumb "double-relaxation" that custodian does, I suggest the following recipe.

1. Start with looser kpt grid, e.g., 0.5 of normal grid.
2. Second run, do the actual grid we want + EDIFFG = -0.05. This last criteria is not used in MPVaspInputSet right now, but is actually critical to get good structures.
3. Check CONTCAR volume - POSCAR volume. If volume change > 2%, repeat step 2.

I don't have time to do this myself, and I am a bit confused about where each of the many RunVasp do in the actual grand scheme of things. So I suggest someone else take the lead. Actually, the full_opt_run in Custodian already does this without the EDIFFG part. I will push something shortly which will enable this.

DOS and BS insertion to `mmdb.insert`

The removal of DOS and BS from the task doc and its insertion to GridFS is currently implemented in VaspToDb.run_task (see vasp.firetasks.parse_outputs). Would it make sense to move this part of the code into mmdb.insert to enable re-use when manually running drones (e.g. with pymatgen.hive)?

fix CircleCI tests so that VASP integration tests are also run

It seems the CircleCI integration tests skip most of the tests. Probably this is because running VASP tests (even the simulated VASP runs) requires MongoDB (think this is already set up and so shouldn't be a problem) and also VASP POTCAR files with associated set up of PMG_VASP_PSP_DIR (probably the issue).

atwf and "files" serialization of workflows doesn't make it easy to set vasp command

@bocklund @shyuep - this relates to your part of atomate

Summary

• The documentation for atomate suggests that users create their first workflow using the command: atwf add -l vasp -s optimize_only.yaml -m mp-149
• Unfortunately, this command does not honor the "vasp_cmd" set up in my_fworker.yaml and just runs "vasp" as the executable (making the set up of my_fworker.yaml pointless)
• I don't see any easy way for the user to put in their vasp_cmd. They can edit optimize_only.yaml to add a common_params but that is clunky - they need to locate the file and edit it. I actually think it less of a pain to just use Python in that case.
• This has led to user failures in trying to follow the atomate tutorials

Example code

See this thread:
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/atomate/xVghmE3YOco

Suggested solution (if any)

Possible solutions include:

• Replace the docs command with the one suggested in the Google Groups thread (atwf add -l vasp -p wf_bandstructure -m mp-149). This will at least stop initial users from seeing errors right away but then the tutorial is more about preset function based workflows than file-based workflows
• Edit all the file-based workflows in the library so that the common_params has vasp_cmd set to ">>vasp_cmd<<". Kind of a pain and everyone that needs to do this for all file-based workflows
• Allow atwf to set common_params during the command line command itself. Something similar to (``atwf add -l vasp -s optimize_only.yaml -m mp-149 --common_params vasp_cmd=">>vasp_cmd<<"). Again a little clunky and non-intuitive, but it would work

There are probably other solutions but those are 3 possibilities

Symmetry reduced elasticity workflow

Summary

• Elastic workflow should have an option for reduction of fireworks according to symmetry

I'm working on this and have a solution implemented, but am in the process of benchmarking and constructing a unit-test. Just wanted to put into the issue tracker so other folks know it's in the works.

why is "magmom" a required parameter for WriteVaspSOCFromPrev?

@matk86 - see title

It looks like the from_prev_run() method is perfectly happy to not need a magmom parameter, so why require it in order to use WriteVaspSOCFromPrev?

ToDbTask test fails

@shyamd

Not sure why it might have passed before (perhaps it wasn't run properly), but the current test for ToDbTask fails. I put a SkipTest for now just because it's not crucial. Check out:

atomate/common/firetasks/tests/test_parse_outputs.py:57

in the latest master branch. The issue seems to be whether ToDBTask takes in a string classname or an actual Drone object. Note that the latter may not serialize/deserialize well in FireWorks

Proposal for storing volumetric data

Posting here for feedback/discussion.

Summary

It would be useful to be able to:

• Make it easy (via a mongo query) to see what volumetric data is available for a given task doc

• Optionally, store the volumetric data to make it easy to retrieve via a mongo query

The benefits of this are mainly:

• Easier aggregation and building (particularly for bader and other topological analysis, also for processing of locpots)

• We're looking into charge density visualization in the future, and need to be able to easily retrieve chgcars to be able to do this

Approaches

Two ideas:

1. In the Atomate Drone, in each calcs_reversed, store a list of available volumetric data files. In insert_task, we then optionally look up the files requested by the user and insert these into gridfs, e.g. insert_task(store_volumetric=('chgcar', 'locpot')).

or

2. In each calcs_reversed, we store the full volumetric data file as a string when generating the taskdoc, e.g. calcs_reversed[0]['volumetric_data']['chgcar'] = chgcar_string. However, in insert_task, we choose whether to keep this string and store it via insertion into gridfs, or whether to just discard the string from the taskdoc and not store it at all.

Discussing with @shyamd, we were perhaps erring towards option 2, but I thought it'd be useful to open this to feedback from others. The problem with 2 is that it involves storing a lot of data in the taskdoc, even if temporarily, but this is basically what we do with bandstructure and dos objects so it seemed logically cleaner.

I have a draft implementation for option 1 available (master...mkhorton:chgcar, changes to VaspDrone have been tested, changes to insert_task have not) to give an idea for what this might look like -- I think the changes to VaspDrone seem reasonable?

AdsorptionTests fail unit tests

@montoyjh (?)

The adsorption workflow fails the unit tests. Unfortunately, CircleCI is not properly configured to run VASP simulation tests (e.g., POTCARs required and such) which is why this was not caught earlier but that is a separate issue.

I am pretty sure this is a simple fix(?). The EDIFFG for MVLSlabSet looks to have changed to -0.02. So either the MVLSlab set default or the test needs to be changed. At least that's what a casual look at the code seems like.

======================================================================
FAIL: test_wf (atomate.vasp.workflows.tests.test_adsorbate_workflow.TestAdsorptionWorkflow)

Traceback (most recent call last):
File "/Users/ajain/Documents/code_matgen/atomate/atomate/vasp/workflows/tests/test_adsorbate_workflow.py", line 68, in test_wf
assert all([vis.incar['EDIFFG']==-0.01 for vis in ads_vis])
AssertionError:
-------------------- >> begin captured logging << --------------------
launchpad: DEBUG: RESTARTED fw_id, launch_id to (1, 1)
launchpad: INFO: Performing db tune-up
launchpad: DEBUG: Updating indices...
launchpad: INFO: LaunchPad was RESET.
pymatgen.core.surface: DEBUG: (1, 1, 1) has 1 slabs...
pymatgen.core.surface: DEBUG: (1, 1, 0) has 1 slabs...
pymatgen.core.surface: DEBUG: (1, 0, 0) has 1 slabs...
launchpad: DEBUG: RESTARTED fw_id, launch_id to (1, 1)
launchpad: INFO: Performing db tune-up
launchpad: DEBUG: Updating indices...
launchpad: INFO: LaunchPad was RESET.
--------------------- >> end captured logging << ---------------------

Issue with add_tags and vasp_input_set

Howdy,
When I run the following:

from pymatgen.core.structure import Structure
from matmethods.vasp.fireworks.core import OptimizeFW
from fireworks.core.firework import Workflow
from pymatgen.io.vasp.sets import MPStaticSet
from pymatgen.matproj.rest import MPRester
from matmethods.vasp.vasp_powerups import add_tags

a = MPRester()
Si = a.query({'material_id':'mp-149'},['structure']).pop()['structure']

tags = ['test_1','test_2']

wf = [OptimizeFW(Si,name="test",vasp_cmd='>>vasp_cmd<<',db_file='>>db_file<<')]
wf = Workflow(wf)
add_tags(wf,tags)

I get this error:

Traceback (most recent call last):
  File "debug_tags.py", line 18, in <module>
    add_tags(wf,tags)
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/matmethods-tess/matmethods/vasp/vasp_powerups.py", line 316, in add_tags
    return Workflow.from_dict(wf_dict)
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/fireworks-tess/fireworks/core/firework.py", line 1120, in from_dict
    return Workflow([Firework.from_dict(f) for f in m_dict['fws']],
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/fireworks-tess/fireworks/utilities/fw_serializers.py", line 147, in _decorator
    new_args[0] = {k: _recursive_load(v) for k, v in args[0].items()}
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/fireworks-tess/fireworks/utilities/fw_serializers.py", line 147, in <dictcomp>
    new_args[0] = {k: _recursive_load(v) for k, v in args[0].items()}
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/fireworks-tess/fireworks/utilities/fw_serializers.py", line 108, in _recursive_load
    return {k: _recursive_load(v) for k, v in obj.items()}
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/fireworks-tess/fireworks/utilities/fw_serializers.py", line 108, in <dictcomp>
    return {k: _recursive_load(v) for k, v in obj.items()}
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/fireworks-tess/fireworks/utilities/fw_serializers.py", line 111, in _recursive_load
    return [_recursive_load(v) for v in obj]
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/fireworks-tess/fireworks/utilities/fw_serializers.py", line 103, in _recursive_load
    return load_object(obj)
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/fireworks-tess/fireworks/utilities/fw_serializers.py", line 309, in load_object
    return cls_.from_dict(obj_dict)
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/fireworks-tess/fireworks/utilities/fw_serializers.py", line 147, in _decorator
    new_args[0] = {k: _recursive_load(v) for k, v in args[0].items()}
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/fireworks-tess/fireworks/utilities/fw_serializers.py", line 147, in <dictcomp>
    new_args[0] = {k: _recursive_load(v) for k, v in args[0].items()}
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/fireworks-tess/fireworks/utilities/fw_serializers.py", line 106, in _recursive_load
    return json.loads(json.dumps(obj), cls=MontyDecoder)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 352, in loads
    return cls(encoding=encoding, **kw).decode(s)
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/lib/python2.7/site-packages/monty/json.py", line 187, in decode
    return self.process_decoded(d)
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/lib/python2.7/site-packages/monty/json.py", line 177, in process_decoded
    return cls_.from_dict(data)
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/lib/python2.7/site-packages/monty/json.py", line 73, in from_dict
    return cls(**kwargs)
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/pymatgen-tess/pymatgen/io/vasp/sets.py", line 421, in __init__
    structure, MPRelaxSet.CONFIG, **kwargs)
  File "/Users/tesssmidt/Dropbox/test_pymatgen/venv/src/pymatgen-tess/pymatgen/io/vasp/sets.py", line 239, in __init__
    structure = structure.get_sorted_structure()
AttributeError: 'dict' object has no attribute 'get_sorted_structure'

Can you reproduce this error?

Thanks!
Tess

boltons

Someone needs to fix the conda yaml for atomate. Boltons was introduced as a dependency, but not added to the conda yaml. Right now, atomate cannot be built.

In any case, I am not sure why the need for boltons. AS far as I can see, the only use is boltons.fileutils.copytree. Based on the doc, this is an exact replica of shutil.copytree with the exception that it does not raise an Exception if part of the tree exists. This sounds to be like a rather trivial thing to work around instead of introducing a new dependency.

Needed docs: builders

A short doc on builders would be useful, it's one of the core features of atomate

issue with wf_elastic_constant

Elastic constant workflow for ['mp-5094', 'mp-30452']

• Version: latest
• Python version: 2.7
• OS version: Edison

Summary

• raised error when trying to add elastic constant workflow for these two materials (I tried them separately, both have problems)

Example code

from matmethods.vasp.workflows.presets.core import *
list = ['mp-5094', 'mp-30452']  # Co2Ge3Se3, DyNiBi
for mpid in list:
    structure = mpr.get_structure_by_material_id(mpid)
    wf = wf_elastic_constant(structure)

Error message

Traceback (most recent call last):
  File "run_matmethods.py", line 25, in <module>
    wf2 = wf_elastic_constant(structure)
  File "/global/project/projectdirs/m2439/alireza/MatMethods/codes/MatMethods/matmethods/vasp/workflows/presets/core.py", line 149, in wf_elastic_constant
    vasp_cmd = c.get("VASP_CMD", VASP_CMD)
AttributeError: 'NoneType' object has no attribute 'get'

Please note that I ran the above script for Si and GaAs w/o problems.

as_dict() and from_dict() additions to MatMethods input sets

@matk86

Are explicit versions of these methods really necessary? They are kind of cumbersome to maintain and I figure that superclass DictVaspInputSet could just take care of dict-ifying everything. Let me know if it's not the case.

update installation docs for pip

@bocklund can you take a stab at this?

See conversation here:
4f99e5f#commitcomment-23798480

spec_path is wrong when atwf script gets copied instead of run via a wrapper

System

• Version: 0.7.0
• Python version: 2.7.14, 3.6.4
• OS version: CentOS 6.9

Summary

• The atwf script uses os.path.abspath(__file__) to define module_dir and build spec_path.

• Usually (e.g. when atomate is installed directly via setup.py), a wrapper is created (by setuptools) which uses pkg_resources.run_script to run the atwf script from the atomate/scripts folder, and the spec_path is correct.

• However, in some cases (here, when atomate is installed with pip in a virtualenv), the atwf script gets copied verbatin to the venv/bin folder, and spec_path is therefore wrong.

Suggested solution

• Not sure if this is an issue with atomate, pip, setuptools or virtualenv, but maybe building module_dir from os.path.abspath(atomate.__file__) instead of os.path.abspath(__file__) would be more robust (?)

atomate/vasp/firetasks/parse_outputs.py imports and uses non-existing IndependentStrain from pymatgen

System

• Version: master
• Python version: 2.7.12
• OS version: Linux

Summary

atomate/vasp/firetasks/parse_outputs.py imports and uses IndependentStrain from pymatgen, but pymatgen doesn't seem to have any such class. Looks like there are also some usages in pymatgen. @montoyjh

I have pymatgen 2017.7.4 installed.

Error from VaspToDdTask -- not able to read '>>db_file<<' credentials -- no JSON

I recently pulled from upstream and am getting errors when using VaspToDbTask for code that worked with previous versions. I am using the '>>db_file<<' variable for the db_file parameter and this variable has been converted to the appropriate path in the fw_spec for all FireWorks affected. I am uncertain how to fix this error and would appreciate any advice. Thanks!

Error message

"Traceback (most recent call last):\n  File \"/global/home/users/tsmidt/htenv/src/fireworks-tess/fireworks/core/rocket.py\", line 211, in run\n    m_action = t.run_task(my_spec)\n  File \"/global/home/users/tsmidt/htenv/src/matmethods-tess/matmethods/vasp/firetasks/parse_outputs.py\", line 90, in run_task\n    mmdb = MMDb.from_db_file(db_file, admin=True)\n  File \"/global/home/users/tsmidt/htenv/src/matmethods-tess/matmethods/vasp/database.py\", line 165, in from_db_file\n    creds = loadfn(db_file)\n  File \"/global/home/users/tsmidt/htenv/lib/python2.7/site-packages/monty/serialization.py\", line 75, in loadfn\n    return json.load(fp, *args, **kwargs)\n  File \"/global/software/sl-6.x86_64/modules/langs/python/2.7.8/lib/python2.7/json/__init__.py\", line 290, in load\n    **kw)\n  File \"/global/software/sl-6.x86_64/modules/langs/python/2.7.8/lib/python2.7/json/__init__.py\", line 351, in loads\n    return cls(encoding=encoding, **kw).decode(s)\n  File \"/global/home/users/tsmidt/htenv/lib/python2.7/site-packages/monty/json.py\", line 186, in decode\n    d = json.JSONDecoder.decode(self, *args, **kwargs)\n  File \"/global/software/sl-6.x86_64/modules/langs/python/2.7.8/lib/python2.7/json/decoder.py\", line 366, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File \"/global/software/sl-6.x86_64/modules/langs/python/2.7.8/lib/python2.7/json/decoder.py\", line 384, in raw_decode\n    raise ValueError(\"No JSON object could be decoded\")\nValueError: No JSON object could be decoded\n"

TestWriteVasp unit test fails on system w/VASP_PSP_DIR set

@montoyjh

Running test_write_vasp.py on my local system fails. This is because a few of these tests try to modify the POTCAR to Si_alt. That POTCAR only exists in the atomate tests (is not a real VASP POTCAR). If the user doesn't have a VASP_PSP_DIR set on their system, it's fine since AtomateTest.setUp() will set the testing dir with Si_alt POTCAR as the VASP_PSP_DIR. But if the user does have a legitimate VASP_PSP_DIR set up, then the test fails

You should modify the test so that it either doesn't require a nonexistent POTCAR or so that the testing VASP_PSP_DIR is used for those specific tests regardless of whether the user has set up a VASP_PSP_DIR or not.

TransmuterFW serialization issue

System

• Version: master
• Python version: 2.7
• OS version: GNU/Linux

Summary

• Serialization of transmuter FW

The transmuter FW fails to serialize because the parameter "transformations" is a list of classes, rather than instances of classes (which are then constructed into instances with the transformation parameters). In the FireWorks recursive_dict procedure, the class (which has an "as_dict" attribute) fails to execute the as_dict method since it's not an instance.

Example code

from matmethods.vasp.fireworks.core import TransmuterFW
from pymatgen.transformations.standard_transformations import \
    DeformStructureTransformation
from pymatgen.util.testing import PymatgenTest

structure=PymatgenTest.get_structure("Si")
fw = TransmuterFW(name="elastic deformation",
                  structure=structure,
                  transformations=[DeformStructureTransformation],
                  transformation_params=[
                      {"deformation": [[1, 0, 0.01],
                                       [0, 1, 0],
                                       [0, 0, 1]]}])

Error message

File "/global/u1/m/montoyjh/jhm_test/codes/fireworks/fireworks/utilities/fw_serializers.py", line 70, in recursive_dict
    return recursive_dict(obj.as_dict(), preserve_unicode)
TypeError: unbound method as_dict() must be called with DeformStructureTransformation instance as first argument (got nothing instead)

Elastic task fails with KeyError: "calc_locs" in CopyVaspOutputs

System

• Version: "master"
• Python version: 3.6.1
• OS version: Fedora 25

Summary

• Elastic task started via presets/core/wf_elastic_constant fails
• Seems to only occur if "optimize_stucture" is set to False

Example code

from fireworks import LaunchPad, FWorker
from fireworks.core.rocket_launcher import rapidfire
#from atomate.vasp.workflows.presets.core import wf_elastic_constant_minimal
from atomate.vasp.workflows.presets.core import wf_elastic_constant
from atomate.vasp.powerups import add_modify_incar
from pymatgen.io.vasp.inputs import Poscar
import os

db_dir = os.path.dirname("/data/icurtis/atomate/config/")
lp = LaunchPad.from_file(os.path.join(db_dir, "my_launchpad.yaml"))
poscar = Poscar.from_file("POSCAR_orig")
structure = poscar.structure
c_new = {"optimize_structure": False}
#c_new.update(c or {})
wf = wf_elastic_constant(structure, c_new)
wf = add_modify_incar(wf, modify_incar_params={"incar_update":{"ENCUT": 1000,
                                                               "ALGO": "Normal",
                                                               "KPAR": 4,
                                                               "NCORE": 4}})
lp.add_wf(wf)
fworker = FWorker(env={"db_file": os.path.join(db_dir, "db.json"),
                       "vasp_cmd": "mpirun -n 32 vasp_std",
                       "scratch_dir": "/zdata/icurtis/vasp_tmp"})
rapidfire(lp, fworker=fworker)

Error message

2017-05-21 18:23:32,996 DEBUG FW with id: 55 is unique!                            
2017-05-21 18:23:33,028 INFO Created new dir /data/icurtis/Auxetic/OtherSystems/TiS/launcher_2017-05-22-01-23-32-996363
2017-05-21 18:23:33,071 INFO Launching Rocket                                     
2017-05-21 18:23:33,114 DEBUG FW with id: 55 is unique!                                               
2017-05-21 18:23:33,117 DEBUG Created/updated Launch with launch_id: 71          
2017-05-21 18:23:33,153 DEBUG RUNNING FW with id: 55                                                                                         2017-05-21 18:23:33,153 INFO RUNNING fw_id: 55 in directory: /data/icurtis/Auxetic/OtherSystems/TiS/launcher_2017-05-22-01-23-32-996363      2017-05-21 18:23:33,196 INFO Task started: FileWriteTask.
2017-05-21 18:23:33,200 INFO Task completed: FileWriteTask                                                             
2017-05-21 18:23:33,200 INFO Task started: {{atomate.vasp.firetasks.glue_tasks.CopyVaspOutputs}}.         
Traceback (most recent call last):                      
  File "/zdata/icurtis/atomate/codes/fireworks/fireworks/core/rocket.py", line 230, in run                         
    m_action = t.run_task(my_spec)                                                                                                  
  File "/zdata/icurtis/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py", line 74, in run_task                                
    calc_loc = get_calc_loc(self["calc_loc"], fw_spec["calc_locs"])
KeyError: 'calc_locs'                                                                                 
2017-05-21 18:23:33,260 INFO Rocket finished

CopyVaspOutputs and scratch_dir

I am using atomate and its default workflows to produce this sequence:
Static -> Uniform -> Boltztrap

I set a scratch_dir in the fw_worker.yaml.
What I get is that in the directory where the uniform task stores the final results,
I have a vasprun.xml that correspond to the static task and a vasprun.xml.gz that
is the result of the uniform calculation. I think that is due to use of a scratch_dir.
The problem is that the CopyVaspOutputs inside the Boltztrap firetask copies
by default the vasprun.xml (static calculation) and not the vasprun.xml.gz (unifom one)
that is what I want but in general what it should be copied.

Ferroelectric workflow tests fail

@blondegeek

Not sure exactly what is the problem, but when I run locally I get the following.

Note that CircleCI is not configured to run VASP integration tests correctly so we didn't catch this whenever it actually failed (that is literally a separate issue)

======================================================================
ERROR: test_wf (atomate.vasp.workflows.tests.test_ferroelectric_workflow.TestFerroelectricWorkflow)

Traceback (most recent call last):
File "/Users/ajain/Documents/code_matgen/atomate/atomate/vasp/workflows/tests/test_ferroelectric_workflow.py", line 124, in test_wf
self._check_run(d, "_polar_relaxation")
File "/Users/ajain/Documents/code_matgen/atomate/atomate/vasp/workflows/tests/test_ferroelectric_workflow.py", line 86, in _check_run
self.assertAlmostEqual(d["calcs_reversed"][0]["output"]["structure"]["lattice"]["c"], 4.2157, 2)
TypeError: 'NoneType' object is not subscriptable
-------------------- >> begin captured logging << --------------------
launchpad: DEBUG: RESTARTED fw_id, launch_id to (1, 1)
launchpad: INFO: Performing db tune-up
launchpad: DEBUG: Updating indices...
launchpad: INFO: LaunchPad was RESET.
launchpad: INFO: Added a workflow. id_map: {-221: 1, -219: 2, -217: 3, -215: 4, -214: 5}
launchpad: DEBUG: FW with id: 5 is unique!
rocket.launcher: INFO: Created new dir /Users/ajain/Documents/code_matgen/atomate/atomate/utils/scratch/launcher_2018-02-02-22-22-51-777535
rocket.launcher: INFO: Launching Rocket
launchpad: DEBUG: FW with id: 5 is unique!
launchpad: DEBUG: Created/updated Launch with launch_id: 1
launchpad: DEBUG: RUNNING FW with id: 5
rocket.launcher: INFO: RUNNING fw_id: 5 in directory: /Users/ajain/Documents/code_matgen/atomate/atomate/utils/scratch/launcher_2018-02-02-22-22-51-777535
rocket.launcher: INFO: Task started: {{atomate.vasp.firetasks.write_inputs.WriteVaspFromIOSet}}.
rocket.launcher: INFO: Task completed: {{atomate.vasp.firetasks.write_inputs.WriteVaspFromIOSet}}
rocket.launcher: INFO: Task started: {{atomate.vasp.firetasks.run_calc.RunVaspFake}}.
rocket.launcher: INFO: Rocket finished
launchpad: DEBUG: FW with id: 4 is unique!
rocket.launcher: INFO: Created new dir /Users/ajain/Documents/code_matgen/atomate/atomate/utils/scratch/launcher_2018-02-02-22-22-51-851444
rocket.launcher: INFO: Launching Rocket
launchpad: DEBUG: FW with id: 4 is unique!
launchpad: DEBUG: Created/updated Launch with launch_id: 2
launchpad: DEBUG: RUNNING FW with id: 4
rocket.launcher: INFO: RUNNING fw_id: 4 in directory: /Users/ajain/Documents/code_matgen/atomate/atomate/utils/scratch/launcher_2018-02-02-22-22-51-851444
rocket.launcher: INFO: Task started: {{atomate.vasp.firetasks.write_inputs.WriteVaspFromIOSet}}.
rocket.launcher: INFO: Task completed: {{atomate.vasp.firetasks.write_inputs.WriteVaspFromIOSet}}
rocket.launcher: INFO: Task started: {{atomate.vasp.firetasks.run_calc.RunVaspFake}}.
rocket.launcher: INFO: Rocket finished
launchpad: DEBUG: RESTARTED fw_id, launch_id to (1, 1)
launchpad: INFO: Performing db tune-up
launchpad: DEBUG: Updating indices...
launchpad: INFO: LaunchPad was RESET.

No gamma_vasp in default workflows

None of the default workflows populate vasp_gamma_cmd in RunVaspCustodian so that doesn't get set. I can go ahead and modify the default workflows to set that or add in a powerup. Any preference?

YAML Workflow Library not being included in PyPi package

Installing atomate from pypi doesn't include the library folder with all the yaml definitions for basic workflows. This folder should probably be included in there. I'm not sure how to change this or I'd put in a pull request myself.

Add GW workflow using new MVLGWSet.

@zbwang to implement.

additional_fields parameters to the Vasp calculations fireworks in atomate/vasp/fireworks/core.py

An optional parameter to the different fireworks in core.py called additional_fields will give the user more control over what information gets stored in their DB. By default, additional_fields={} which just stores {"task_label": name} as an additional field. This is just so users don't have to rewrite something like the OptimizeFW just so they can create their own additional_fields.

boltons.copytree() instead of custom function

@albalu - I noticed a recent PR of yours implemented a custom copytree() function. Can you see if the implementation in this library is what you want?

http://boltons.readthedocs.io/en/latest/fileutils.html

If so, you can remove the copytree() code from atomate, add a dependency to boltons (in requirements.txt and setup.py), and use the copytree implemented in boltons.

The boltons library should be pretty lightweight and there might be other functions we want to use in their later, so I don't mind the added dependency.

Drone suggestions

System

• Version: Master

Summary

• MMVaspToDbTaskDrone is written in a rather inflexible manner. For example, there is a calculation_initial and calculation key in assimilate. When happens if the run is only a single run? It is replicated?

Suggested solution

• I think basically calculations should be a dict or a list. Originally, it was a list but that made it difficult to handle. But we can always have a dict with keyword: vasprun. E.g., {"relax1": , "relax2": }.
• I would suggest revising some key names. For example, unit_cell_formula and pretty_formula suggests they are the same type, when in fact the former is a dict and the latter is a string. I suggest we rename all the dict versions as composition. E.g., unit_cell_composition and reduced_composition. Drop the word "cell" from reduced_composition since after reduction, the concept of a "cell" doesn't make sense.
• Do we want to invert the names? E.g., composition_reduced, composition_unit_cell instead of "reduced_composition" and "unit_cell_composition".
• I would drop the word "final" from "final_energy" and "final_energy_per_atom" for brevity.

write a code that contains convergence test workflow for different cases

I think it will be useful if Atomate includes the sets of convergence test workflows. Each workflow will perform different types of convergence tests, which include the general k-points and cutoff energy convergence test and the size test for slab and vacuum, and defects in different systems (slab and bulk).

get_settings error when using any builder

System

• Version: "master"
• Python version: 2.7.13, 3.6.2
• OS version: MacOS 10.13.3

Summary

• Error whenever creating a builder object
• Throws error whether using db from file or a writable collection passed as arg

Example code

from atomate.vasp.builders.fix_tasks import FixTasksBuilder
DB_FILE = "/Users/me/path/to/db.json"
fix = FixTasksBuilder.from_file(DB_FILE)

Error message

Traceback (most recent call last):
  File "/Users/ardunn/alex/lbl/projects/thermoelectrics/code/thermoelectrics_work/thermoelectrics_work/build_te_database.py", line 28, in <module>
    fix = FixTasksBuilder.from_file(DB_FILE)
  File "/Users/ardunn/alex/lbl/projects/common_env/common_env/lib/python2.7/site-packages/atomate/vasp/builders/fix_tasks.py", line 72, in from_file
    db_write = get_database(db_file, admin=True)
  File "/Users/ardunn/alex/lbl/projects/common_env/common_env/lib/python2.7/site-packages/atomate/utils/utils.py", line 338, in get_database
    d = get_settings(config_file) if settings is None else settings
NameError: global name 'get_settings' is not defined

Suggested solution (if any)

• Defineget_settings ? Not sure
  @albalu

ToDBTask() will only work with matmethods drone

@shyamd - the purpose of your suggestion to add ToDBTask() was to make the db task drone-agnostic. But this line:

t_id, task_doc = drone.assimilate_return_task_doc(calc_dir)

assumes the drone has a method called assimilate_return_task_doc. Actually, only the MatMathods drone has this method. It is not a general method of drones. This means that only the MatMethods drone can in practice be used here.

Can you figure out a solution, perhaps working with @shyuep in case changes are desired in underlying pymatgen?

Docs: `link me to debye`_

@bocklund The Creating workflows docs guide has a missing link (link me to debye)

Running workflows docs - strange guidance on vasp_cmd

@bocklund

At several points in the "Running workflows" tutorials, the docs state:

 If your VASP command is anything other than ``vasp_std``, then you'll want to change the line setting the ``VASP_CMD`` key of the configuration dictionary. *Don't use any parallelization software (ibrun, srun, etc.)* it's bad behavior and probably won't work.

I don't understand this guidance:

• using srun, mpirun, etc. should work for the vasp_cmd - we do this all the time (although we use env_chk to override the command via my_fworker.yaml - but it's the same thing in the end). So I don't understand the "bad behavior" and "probably won't work". I agree that if you try to run srun on a supercomputing login node you will likely have problems. But that is not what is typically done (see next point).
• the tutorial uses rlaunch whereas the installation sets up an infrastructure for using qlaunch. I am not sure why. Why not just show the user how to use qlaunch? That is what they will be using in production anyway. If your intention is to help the user run a quick test rather than waiting for days in the queue to get the result of the tests, I would suggest to anyway have the docs assume the user is going to run qlaunch (i.e., no restrictions against using srun type commands, use qlaunch -r rapidfire to launch, etc). But in the actual section where it tells you to run the qlaunch command, make a note that saying if you just want to test locally (on the current computer or login node) rather than wait on a queue then use rlaunch instead and make sure the vasp_cmd doesn't include any special commands like srun. But we want the default procedure to be what people are expected to do regularly because that is the pattern they will repeat, and then just make a quick exception for testing purposes.

Workfunction workflow to be moved to atomate

@richardtran415 to implement

Use git lfs

This repo is getting pretty big, and will only get bigger as we add more workflows with chgcars, etc. I suggest we move to using git lfs for faster clones and pulls. It is pretty transparent, and the only requirement is that the developers have to install git lfs first. I have already implemented lfs support with a custom image for testing.

Pymatgen has already moved to a git lfs system. The impetus is even stronger for atomate.

Debugging a Firework that defuses children

From @computron in this Google Groups discussion:

I am not sure what happened to the message, but Brandon replied:

========
I saw that as well. Seems like the only place that could come from.

is it the correct behavior for jobs to be marked as COMPLETED rather than FIZZLED if this is the case? Is the rationale that I try parsing those outputs by hand?

========

It is up to use to define what is the correct behavior. The "defuse" action happens when the VASP run completes OK, but something is still wrong. For example, let's say you hit the maximum number of ionic steps and didn't actually converge to your desired tolerance. The job didn't "fail" and the VASP outputs would look normal at first glance, but we also didn't succeed in getting what we want to proceed to the next step.

Two options are:

1. Call that particular run COMPLETED but defuse remaining runs. This (the current behavior) reflects the idea that the run did in fact complete without any errors, but that we wouldn't want to continue on with the workflow.
2. Call that particular run FIZZLED; this would alert a user that this perhaps a run that needs some kind of "fixing".

I'm happy to discuss with some of the others if you think (2) is the better way forward. Let me know.

Thanks. This actually cleared it up for me. I think option (1) you presented makes more sense than (2) because in this instance the code behaved correctly and based on some configuration determined not to continue.

What could be improved is

1. The behavior of the parsing the outputs. I was not aware that assimilating the result with a drone has some baked in behavior to defuse child Fireworks based on the results of the calculation.

Some things to consider

• Documentation that explains this behavior
• Allow users to turn off the behavior of defuse on failed analysis
• Allow users to set the parameters for defusing in their FWorker (we should definitely do this)

2. Defining the state. I haven't checked if there is any unsuccessful state output in the database. I guess if a Firework is COMPLETED, I should have suspected that there would be an entry in the tasks database.

I think there should be some information in the defused Firework and the defusing Firework on why this happened. Especially if we can make that clear on the WebGUI because that's where users will look. Presumably the machinery to do this is already in FireWorks, we should just make better use of it here.

It seems that the versions on conda are still 0.5.8/0.6.0

Summary

• linux-64 v0.6.0
• osx-64 v0.5.8
• win-64 v0.6.0

It seems that the versions on the matsci channel are too old.