ornl / papyrus

Papyrus: Parallel Aggregate Persistent Storage

Papyrus

Papyrus is a programming system that provides features for scalable, aggregate, persistent memory in an extreme-scale system for typical HPC usage scenarios. Papyrus provides a portable and scalable programming interface to access and manage parallel data structures on the distributed NVM storage.

Requirements

• C++11 compiler
• MPI library supporting MPI_THREAD_MULTIPLE
• CMake (>=3.8)

Installation

You can download the Papyrus source code from GitHub:

$ git clone https://github.com/ORNL/papyrus.git
$ cd papyrus

You can build Papyrus with CMake and Make:

$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=<install_dir>
  or if you want to run tests after build
$ cmake .. -DCMAKE_INSTALL_PREFIX=<install_dir> \
           -DMPIEXEC=<mpiexec_path> -DMPIEXEC_NUMPROC_FLAG=<np_flag>
$ make install

Running tests

For the Cray MPI Library, an environment variable MPICH_MAX_THREAD_SAFETY has to be set to multiple:

$ export MPICH_MAX_THREAD_SAFETY=multiple

The project's test suite can be run by executing:

$ make test (or use 'ctest -V' for verbose test output)

Repository contents

• The public interface is in include/papyrus/*.h.
• The Key-Value Store is in kv/.

References

To cite Papyrus, please use the following papers:

• Jungwon Kim, Seyong Lee, and Jeffrey S. Vetter. "PapyrusKV: A High-Performance Parallel Key-Value Store for Distributed NVM Architectures". SC 2017. DOI: 10.1145/3126908.3126943
• Jungwon Kim, Kittisak Sajjapongse, Seyong Lee, and Jeffrey S. Vetter. "Design and Implementation of Papyrus: Parallel Aggregate Persistent Storage". IPDPS 2017, DOI: 10.1109/IPDPS.2017.72
• Jungwon Kim and Jeffrey S. Vetter. "Implementing Efficient Data Compression and Encryption in a Persistent Key-Value Store for HPC". IJHPCA 2019. DOI: 10.1117/1094342019847264