pyiron / pylammpsmpi

Parallel Lammps Python interface - control a mpi4py parallel LAMMPS instance from a serial python process or a Jupyter notebook

pylammpsmpi - Parallel Lammps Python interface

With pylammpsmpi you can control a mpi4py parallel LAMMPS instance from a serial python process or a Jupyter notebook. Internally pylammpsmpi leverages the executorlib communication interface to connect the serial python process the user interacts with, with the mpi4py parallel LAMMPS instance. The advantage of separating the mpi4py parallel LAMMPS instance from the rest of the workflow is that the workflow can be written as serial python code, while still benefiting from the parallel performance of LAMMPS. Still this comes at the cost of additional data transfer, as the LAMMPS pointers cannot be transferred this way and the linked data has to be copied instead. So copying large atomistic structures can decrease the performance of the pylammpsmpi interface in comparison to writing your own fully mpi4py parallel LAMMPS workflows.

Interfaces

The pylammpsmpi module implements three different interfaces for different use cases:

• pylammpsmpi.LammpsBase: The most basic interface is the LammpsBase, it implements the same commands like the default lammps.lammps interface and returns the same datatypes. With this API compatibility to the standard interface, this interface is commonly the easiest way to accelerate a serial LAMMPS based workflow by leveraging mpi4py parallel LAMMPS instances.
• pylammpsmpi.LammpsConcurrent: Inspired by the concurrent.futures module in the standard python library the pylammpsmpi.LammpsConcurrent interface implements the same API as the pylammpsmpi.LammpsBase class but rather than holding the controlling process until the mpi4py parallel LAMMPS instance finishes the execution of a given set of commands, the pylammpsmpi.LammpsConcurrent interface returns a concurrent.futures.Future object. This enables the development of asynchronous / concurrent workflows.
• pylammpsmpi.LammpsLibrary: Finally, the pylammpsmpi.LammpsLibrary interface adds a higher level interface on top of the default lammps.lammps interface. This higher level interface provides direct access to the commands and thermodynamic properties used in the LAMMPS input files. Especially for experienced LAMMPS users who are familiar with the LAMMPS input files this interface simplifies switching from file based input to using the python interface.

The choice of interface depends on the users background, experience and the simulation protocol the user wants to implement. Still internally all three interfaces are based on the pylammpsmpi.LammpsConcurrent interface, so they use an additional thread to connect the mpi4py parallel LAMMPS instance to the serial python process or Jupyter notebook.

Documentation

• Installation
  • conda-based installation
  • pypi-based installation
• Interfaces
  • LammpsBase
  • LammpsConcurrent
  • LammpsLibrary
• Development

License

pylammpsmpi is released under the BSD license https://github.com/pyiron/pylammpsmpi/blob/main/LICENSE . It is a spin-off of the pyiron project https://github.com/pyiron/pyiron therefore if you use pylammpsmpi for calculation which result in a scientific publication, please cite:

@article{pyiron-paper,
  title = {pyiron: An integrated development environment for computational materials science},
  journal = {Computational Materials Science},
  volume = {163},
  pages = {24 - 36},
  year = {2019},
  issn = {0927-0256},
  doi = {https://doi.org/10.1016/j.commatsci.2018.07.043},
  url = {http://www.sciencedirect.com/science/article/pii/S0927025618304786},
  author = {Jan Janssen and Sudarsan Surendralal and Yury Lysogorskiy and Mira Todorova and Tilmann Hickel and Ralf Drautz and Jörg Neugebauer},
  keywords = {Modelling workflow, Integrated development environment, Complex simulation protocols},
}

pylammpsmpi's Issues

Pip package broken

The current pip package is still broken and for some reason the build process with GitHub actions is slightly different to the build process with travis, so I have to take a second look at this.

[Idea] Next generation interface

There are three things we learned so far:

• using zmq simplifies the communication between the primary process and the subprocess.
• with __getattribute__() we can overwrite the functions of a given class.
• with cloudpickle we can even pickle ctypes

So for the next version of pylammpsmpi I guess we need much less code. Just load the LAMMPS library class and then before a function is called, interfere using __getattribute__(), cloudpickle the data that is communicated transfer it to the MPI parallel subprocess and then execute the call there. Afterwards cloudpickle the response and communicate it back up. This should drastically decrease the code base for pylammpsmpi and increase the consistency with the official LAMMPS python API.

Release

I have to admit I was not expecting progress so quickly, so I guess we still need some minor fixes but then we can release a first test version on pip and conda. Now the question is if we should move the repository before we release it. My suggestion would be to move it to https://github.com/pyiron but I am also open to create a separate Github group for this project. I guess in the long term we would be happy if the changes get adopted in the official Lammps release but as this might take some time, we should look for an intermediate solution for now.

recall variables in python to lammps

Hi there,

I was wondering if it's possible to create a new variable in python and then pass it on to lammps, e.g.,
from pylammpsmpi import LammpsLibrary lmp = LammpsLibrary(cores=36) number = 5 lmp.command("variable r equal %d" % number)

Many thank,
HZ

About Queuing systems

@jan-janssen Any idea how we could possibly run this on a queuing system? As in information should still be accessible through a jupyter notebook, but the run for example should be on a queue.

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Setting up documentation

@jan-janssen We should also think about setting up some documentation. My opinion is to go with readthedocs. What do you think?

[Feature] Create custom `concurrent.futures.Executor`

To enable the asynchronous development of simulation protocols, it would be great to create a custom concurrent.futures.Executor class. This class would represent one LAMMPS instance and the user can send different commands to this class, which are then executed in the background.

`ImportError: cannot import name 'LammpsLibrary'` when running the jupyter notebook

When running the jupyter notebook, the above error is generated. Do you know if I am doing something wrong?

[Bug] LAMMPS does not support mpi4py version 4.0.0

The tests here did not fail only the ones in pyiron_lammps did. Still LAMMPS itself has a strict constraint to mpi4py < 4.

socket connection stuck

When using socket as communicator, the creation of the object seems to get stuck on
hostsocket, _ = s.accept() - Line 85 from communicate.py

Using commands directly throws error

Using lmp.run(1) instead of lmp.command('run 1') sometimes crashes the subprocess. See #26

extract_compute produces different results for mpi_cores > 1

When I use extract_compute on cluster/atom compute with mpi_cores > 1 I get strange float numbers < 1 (which is impossible considering that this compute produces only ints). The same thing happens without debug mode.

compute clusters clusters cluster/atom 3

mpi_cores = 1

mpi_cores = 3

Hi @srmnitc

Hi @srmnitc,

I saw that you stared the repository, so I wanted to give you access. The reason why I removed public access is that some functions do not work at the moment, like extract_atom. Luckily the functions we need inside pyiron, work out of the box https://github.com/pyiron/pyiron/blob/master/pyiron/lammps/interactive.py#L606 .

The general idea is to provide a single core lammps process - running in a jupyter notebook - which then controls a mpi parallel lammps process. Currently this implementation only works with openmpi - figuring this out took me quite some time pyiron/pyiron#516 .

As this scheme is potentially interesting for users of lammps beyond the pyiron community I would be interested to make this a public package. If we get the standard commands working I am pretty sure we can also get Steve Plimpton - the core developer of lammps to implement it in the core lammps code. I met Steve last year when I visited Los Alamos.

So if you are interested to work on this feel free to continue the project, the Lammps developers are always happy about contributions and being named in the Lammps documentation helps to get more people interested in your work - that is what I learned from contributing the Lammps conda forge package https://github.com/lammps/lammps/blob/master/doc/src/Install_conda.rst .

Best regards,

Jan

Release pylammpsmpi 0.0.9

It is already over a year since the last version was release and we recently added the ability to support additional command line arguments, so it makes sense to release a new version.

[documentation] New documentation for version 0.2.0

@srmnitc I updated the documentation on https://pylammpsmpi.readthedocs.io/ feel free to take a look and modify it as you see fit.

Communication with the MPI process

I just learned about the https://github.com/NCAR/mpirical package, maybe we can use it to remove our own subprocess interface for calling Lammps. Again following the idea to reuse existing code wherever possible.

Strange problem

Sorry for the cryptic title, but I seem to have a strange problem. So far, I was the example notebook from the main repo folder, and hence when the subprocess is launched the path of the executable it selects is directly from the repo, for example, like /pylammpsmpi/pylammpsmpi/mpi/lmpmpi.py. In this case, everything works.
Now, when I try with the jupyter notebook in a different folder, the executable path now is somethings like .conda/envs/p3/lib/python3.6/site-packages/pylammpsmpi-0.post0.dev84-py3.6.egg/pylammpsmpi/mpi/lmpmpi.py. This is case trying a lmp.version throws EOFError: Ran out of input. I am trying a couple of things to see what exactly the problem is, but for the moment I am stuck.

Fix travis

The tests work locally but somehow they fail on travis.

Examples not working

I tried to execute the examples again and they both get stuck at:

ke = lmp.extract_compute("ke", 1, 1)
len(ke)

Output file locations when submitting to dask cluster

This problem if specific : Consider a dask cluster created from a notebook located in directory A, when lammps simulation is carried out from a notebook in directory B, which connects to the cluster created in A. The output files - log or traj files are not generated in B, but in A. Probably the dask cluster needs some kind of output directory. Or, full paths needs to be specified for lammps files.

LammpsLibrary hangs on close

With the latest version, 0.2.13, the interactive lammps sessions work, but don't properly clean up. Running the snippet below never finishes (on the cmti cluster)

from pylammpsmpi import LammpsLibrary
import pylammpsmpi

lmp = LammpsLibrary(2)

lmp.version, pylammpsmpi.__version__

lmp.close() # <- hangs indefinitely 

I've watched the lmpmpi.py process with top, and it does disappear when close is called, but apparently that's not properly communicated back to the foreground process.

When I run this snippet on my laptop in a fresh conda environment it hangs similarly, but also prints this warning

[cmleo38:13075] mca_base_component_repository_open: unable to open mca_btl_openib: librdmacm.so.1: cannot open shared object file: No such file or directory (ignored)
[cmleo38:13074] mca_base_component_repository_open: unable to open mca_btl_openib: librdmacm.so.1: cannot open shared object file: No such file or directory (ignored)

[Bug] create_atoms can be both a function or a command

@srmnitc It seems currently the command is working but the function is not. In particular it is also not listed in https://github.com/pyiron/pylammpsmpi/blob/main/pylammpsmpi/utils/commands.py#L17

Environment issues

Could be related to #142

An environment file:

channels:
  - conda-forge
dependencies:
  - lammps =2023.03.28
  - openmpi
  - mpi4py =3.1.4
  - pylammpsmpi =0.2.3

does not resolve for me and gives the error nothing provides __cuda needed by lammps-2023.03.28-cuda110_py310_h007d4e1_openmpi_2

The following, however,

channels:
  - conda-forge
dependencies:
  - openmpi
  - mpi4py =3.1.4
  - pylammpsmpi =0.2.3

works fine, and LAMMPS is installed as a dependency through pylammpsmpi. @jan-janssen any thoughts on what could be the issue?

New version 0.2.3 is not installed

The import error with ´RaisingThread´ persists as the new version 0.2.3 is not installed by conda. When trying to install, it complains that a compatible LAMMPS version could not be found. On debugging a bi, lammps complained about libcurl version, libcurl about krb5, krb5 about pyqt where I stopped.

I am not sure what the solution is.

About PyLammps

Hi @jan-janssen , I was exploring a bit more, one question I have is if we could wrap the code around PyLammps instead of the normal lammps class. Is there any downside to it?

[Feature] Support for ASE structures

We already have this functionality inside pyiron_atomistics but I guess the ASE structure class is so widely used that it makes sense to integrate this functionality directly in pylammpsmpi.

[Idea] Add integration tests

RaisingThread not found

from pylammpsmpi import LammpsASELibrary

raises

ImportError                               Traceback (most recent call last)
Cell In[17], line 1
----> 1 from pylammpsmpi import LammpsASELibrary

File /u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pylammpsmpi/__init__.py:1
----> 1 from pylammpsmpi.wrapper.extended import LammpsLibrary
      2 from pylammpsmpi.wrapper.concurrent import LammpsConcurrent
      3 from pylammpsmpi.wrapper.base import LammpsBase

File /u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pylammpsmpi/wrapper/extended.py:5
      1 # coding: utf-8
      2 # Copyright (c) Max-Planck-Institut für Eisenforschung GmbH - Computational Materials Design (CM) Department
      3 # Distributed under the terms of "New BSD License", see the LICENSE file.
----> 5 from pylammpsmpi.wrapper.base import LammpsConcurrent
      7 __author__ = "Sarath Menon, Jan Janssen"
      8 __copyright__ = (
      9     "Copyright 2020, Max-Planck-Institut für Eisenforschung GmbH - "
     10     "Computational Materials Design (CM) Department"
     11 )

File /u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pylammpsmpi/wrapper/base.py:5
      1 # coding: utf-8
      2 # Copyright (c) Max-Planck-Institut für Eisenforschung GmbH - Computational Materials Design (CM) Department
      3 # Distributed under the terms of "New BSD License", see the LICENSE file.
----> 5 from pylammpsmpi.wrapper.concurrent import LammpsConcurrent
      8 __author__ = "Sarath Menon, Jan Janssen"
      9 __copyright__ = (
     10     "Copyright 2020, Max-Planck-Institut für Eisenforschung GmbH - "
     11     "Computational Materials Design (CM) Department"
     12 )

File /u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pylammpsmpi/wrapper/concurrent.py:9
      7 from concurrent.futures import Future
      8 from queue import Queue
----> 9 from pympipool import RaisingThread, cancel_items_in_queue, interface_bootup
     12 __author__ = "Sarath Menon, Jan Janssen"
     13 __copyright__ = (
     14     "Copyright 2020, Max-Planck-Institut für Eisenforschung GmbH - "
     15     "Computational Materials Design (CM) Department"
     16 )

ImportError: cannot import name 'RaisingThread' from 'pympipool' (/u/system/SLES12/soft/pyiron/dev/anaconda3/lib/python3.10/site-packages/pympipool/__init__.py)

Switch communication to zmq

This enables mpich support in addition to openmpi support. I have done a similar transition for pympipool in pyiron/executorlib#20

for mpi_cores > 1 extract_variable does not return full array

Script output:

mpi_cores = 1: len - 131144
mpi_cores = 4: len - 32448

Script itself:

#!/bin/python3

from pylammpsmpi import LammpsLibrary
import numpy as np

lmp1 = LammpsLibrary(cores=1)
lmp2 = LammpsLibrary(cores=4)


def run_cmds(lmp):
    lmp.command("units metal")
    lmp.command("dimension 3")
    lmp.command("boundary p p m")
    lmp.command("atom_style atomic")
    lmp.command("variable width equal 13")
    lmp.command("lattice diamond 5.43 orient x 1 0 0 orient y 0 1 0 orient z 0 0 1")
    lmp.command("region box block -${width} ${width} -${width} ${width} -${width} ${width} units lattice")
    lmp.command("create_box 1 box")
    lmp.command("region Si block -${width} ${width} -${width} ${width} $(-v_width + 1) $(v_width - 1) units lattice")
    lmp.command("create_atoms 1 region Si")
    lmp.mass(1, 28.0855)
    lmp.command("variable ids atom id")
    lmp.run(0)
    return len(lmp.extract_variable("ids", "all", 1))


print("mpi_cores = 1: len -", run_cmds(lmp1))
print("mpi_cores = 4: len -", run_cmds(lmp2))

Python seems to automatically delete the subprocess

When I set up a command like this-

from pylammpsmpi.lammps import LammpsLibrary
lmp = LammpsLibrary(cores=4)
lmp.units("metal")
lmp.boundary("p p p")
lmp.atom_style("atomic")
lmp.lattice("fcc", 4.0)
lmp.region("box", "block", 0, 10, 0, 10, 0, 30)
lmp.create_box(1, "box")
lmp.create_atoms(1, "box")

While executing you can see that there are 4 python threads started, but once it is done, the subprocess seems to end. A bit later any reference, such as lmp.version would throw a BrokenPipe error.

I will look into it a bit more.

Waiting until a command is executed

If a command is run, for example lmp.command("minimize 1.e-14 1.e-14 1000000 1000000"), and then use lmp.get_thermo("vol"), the second command has to be blocked until the minimize is over. This seems to be not happening. Any idea on how we can achieve this @jan-janssen ?

Reporting pylammpsmpi in publications/thesis etc

Hi @jan-janssen, I have a student who will finish a project which used pylammpsmpi completely for the simulations. Although its not important for the project, I was thinking how in general should one mention pylammpsmpi in publications/thesis etc.

[Feature] Communication of lammps errors

@jan-janssen If lammps throws an error due to a wrong command or so, it is not communicated back to the main process. Is there a good way to do this?

[Feature] leverage triclinic cell support in LAMMPS

As implemented in lammps/lammps#3984

pylammpsmpi not running on 2 cores or more

Hey there, I'm trying to get pylammpsmpi up and running on the cluster of my institution. The cluster uses slurm as a resource manager. I installed pylammpsmpi 0.0.9 via pip install. When I try to run the job test script (see below) on one core, it works fine. When I try to run it on two cores or more, it doesn't work. It doesn't seem to be an issue of mpi4py since the test script suggested on the lammps website (https://docs.lammps.org/Python_launch.html) works just fine. I also used different versions of openmpi, but it didn't change anything. Do you have any suggestions for further troubleshooting?

The test script looks like this:

from pylammpsmpi import LammpsLibrary

lmp = LammpsLibrary(cores=2, mode='local')
print(f"LAMMPS version: {lmp.version}")
lmp.file("crn.in")
lmp.command("fix 2 all npt temp 300.0 300.0 100.0 iso 100.0 100.0 100.0")
lmp.command("run 1")
print("test2")

And the output like this:

Traceback (most recent call last):
  File "/work/smjbweis/Programs/may22/lib/python3.9/site-packages/pylammpsmpi/utils/../mpi/lmpmpi.py", line 466, in <module>
    input_dict = pickle.load(sys.stdin.buffer)
EOFError: Ran out of input
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpiexec detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[64109,1],1]
  Exit code:    1
--------------------------------------------------------------------------
LAMMPS version: 20210929
Traceback (most recent call last):
  File "/work/smjbweis/hrmc_files/hrmc_eg_zif4_TCN210622/./test.py", line 5, in <module>
    lmp.file("crn.in")
  File "/work/smjbweis/Programs/may22/lib/python3.9/site-packages/pylammpsmpi/lammps_wrapper.py", line 73, in func_wrapper
    fut = func(*args, **kwargs)
  File "/work/smjbweis/Programs/may22/lib/python3.9/site-packages/pylammpsmpi/utils/lammps.py", line 119, in file
    _ = self._receive()
  File "/work/smjbweis/Programs/may22/lib/python3.9/site-packages/pylammpsmpi/utils/lammps.py", line 83, in _receive
    output = pickle.load(self._process.stdout)
EOFError: Ran out of input
Exception ignored in: <function LammpsBase.__del__ at 0x2b65ff7b5550>
Traceback (most recent call last):
  File "/work/smjbweis/Programs/may22/lib/python3.9/site-packages/pylammpsmpi/utils/lammps.py", line 677, in __del__
  File "/work/smjbweis/Programs/may22/lib/python3.9/site-packages/pylammpsmpi/utils/lammps.py", line 667, in close
  File "/work/smjbweis/Programs/may22/lib/python3.9/site-packages/pylammpsmpi/utils/lammps.py", line 68, in _send
BrokenPipeError: [Errno 32] Broken pipe

License

We should briefly discuss the license. My suggestion would be BSD 3-Clause License simply because that is what we also use for pyiron.

[Code quality] It would be great to increase the test coverage

I started a couple of pull requests to achieve this:

• #197
• #195

So far I was able to increase the coverage from 53% to 79%.

@srmnitc Do you have some more examples which can be converted to tests to increase the coverage?

Kill error

On mybinder I get the following error:

Exception ignored in: <function LammpsLibrary.__del__ at 0x7efd0c77a290>
Traceback (most recent call last):
  File "/home/jovyan/pylammpsmpi/lammps.py", line 585, in __del__
  File "/home/jovyan/pylammpsmpi/lammps.py", line 579, in close
  File "/srv/conda/envs/notebook/lib/python3.7/subprocess.py", line 1790, in kill
AttributeError: 'NoneType' object has no attribute 'SIGKILL'

[Bug] Test core dependency

Like we discussed during the pyiron meeting, it would be interesting to see what happens when DASK only gets a single core and then still calls Lammps via the MPI subprocess with more than one core.