Check the role of MPI in MPI+X about llnl-work-notes

[ouankou]

In the AMR library based on Charm, MPI or any other load balancing techniques are independent of the computation inside a cell.
Each cell has its own local data for computing and they have the same amount of data. The refinement or coarsening is to increase or reduce the number of cells rather than changing the data amount inside a cell.
Load balancing is used to distribute the cells to different nodes for better parallelism, but doesn't affect the computing inside a cell directly.

Our goal is using metadirective to speed up the computing in all cases. For AMR in Charm, if the cell size is very large in an application, which indicates it has more data, the computing should be offloaded to GPU. It's irrelevant to how many cells there are in the application. Load balancing is responsible for that subject.

[ouankou]

Added to Overleaf.
MPI is used for load balancing for the outer parallelism, such as to distribute part fo AMR mesh to a compute node.
After that, within an MPI rank, the computing could be performed sequentially or in parallel by OpenMP, OpenACC, or CUDA. This is the inner-parallelism.

Metadirective is used for the inner-parallelism and doesn't affect the load balancing.