ggciag / mandyoc

MANDYOC is a finite element code written on top of the PETSc library to simulate thermo-chemical convection of the Earth's mantle

Mandyoc

MANtle DYnamics simulatOr Code

About

Mandyoc is a finite element code written on top of the PETSc library to simulate thermo-chemical convection of the Earth's mantle. Different linear and non-linear rheologies can be adopted, appropriately simulating the strain and stress pattern in the Earth's crust and mantle, both in extensional or collisional tectonics.

Documentation

On the Documentation page, you will be able to find relevant information on how the code works, how it was implemented, how to install it, the description of the parameters and input files, some applications and some useful examples.

Need any help?

• Most discussion happens on Github. Feel free to open an issue or comment on any open issue or pull request.
• You can also send an email to [email protected] or [email protected]

Contributing

Code of conduct

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

Contributing Guidelines

Contributions to Mandyoc are welcome. If you have an issue, a bug, a code contribution or a documentation contribution, thanks for helping to improve Mandyoc! Check out our Contributing guide.

License

This is free software, you can redistribute it and/or modify it under the terms of the BSD 3-clause License. A copy of this license is provided in LICENSE.

mandyoc's Issues

Missing DOIs

Review citation DOIs for JOSS paper.

openjournals/joss-reviews#3551

fix deprecations in tests

There is some annotations in tests that must be fixed soon.

JOSS REVIEW: Paper

I think the paper needs a bit of work.

• Statement of Need There is a lot of codes doing similar things, please state clearly why a new code is needed and what are the new functionalities.
• State of the field is needed. The authors do not describe how this software compares to other commonly-used packages. Consider adding a benchmark for thermochemical convection in parallel... Cite other code/packages.
• There is a Mathematics section that briefly describe Stokes and then the authors list a range of additional functionalities that are not directly related to the title of the paper (Why do you need surface processes, plasticity etc in a thermochemical code?) Please consider rewriting and reorganizing that part.
• I am not sure the chosen example is the best to show here. It needs a benchmark model (van Keken would actually be better suited). You can add a figure of a rifting model to show that the code capabilities go beyond thermochemical simulations.
• The list of references needs to be extended. Some key papers missing there! You should at least cite similar packages that have been published on JOSS.

Setup and update docker image

Some time ago I tested a docker image for mandyoc.

I will update it manually for now, but we should automate it as well.

Add Crameri benchmark to docs

Add a new section for the Crameri benchmark to documentation benchmarks chapter.

Makefile - broken variable name for the installation

@aguspesce I merged #55, but missed this commit: 3b1cd11

This has broken the install path to ~/.local/bin (according to docs).

It also broke compilation to the new bin folder.

Typo in tests/run_tests.sh

There is a typo in line break: /n instead of \n: tests/run_tests.sh#L18.

To fix: just change the forward-slash to backslash.

JOSS REVIEW: Tutorial

The submission needs a basic tutorial to take the user through the setup of a simple problem.
Consider Stokes Sinker, Cooling examples... etc.

It looks like the documentation of the input parameters file is incomplete.

• Surface processes is missing among other things.

You give an example but some parameters are commented out...

I think basic tutorials to illustrate the different functionalities of the code would be enough.

Improve examples based on JOSS review

To-do list to improve the examples based on the JOSS review:

• In run.sh use environmental variables.
  $ git grep /Users examples/continental_rift/run.sh:/Users/victorsacek/Documents/petsc/arch-label-debug/bin/mpirun -n 8 \ examples/continental_rift/run.sh:/Users/victorsacek/Documents/gits/md2d/md2d_aux/mandyoc \ examples/old_version/Crameri2012_Case2/run.sh:nohup /Users/victorsacek/Documents/petsc/arch-label-debug/bin/mpirun -n 4 ./mandyoc -denok 1.0e-15 -rtol 1.0e-7 -particles_per_ele 1000 -theta_FSSA 0.5 -sub_division_time_step 1.0 -visc_harmonic_mean 0 -particles_perturb_factor 0 -visc_const_per_element 1 -pressure_in_rheol 1 <FD.in >FD.out & examples/old_version/vanKeken1997/run.sh:nohup /Users/victorsacek/Documents/petsc/arch-label-debug/bin/mpirun -n 4 ./mandyoc -denok 1.0e-15 -particles_per_ele 1000 -theta_FSSA 0.5 -sub_division_time_step 1.0 -visc_harmonic_mean 0 -particles_perturb_factor 0 -visc_const_per_element 1 -pressure_in_rheol 1 <FD.in >FD.out & examples/old_version/vanKeken1997_nondim/run-mg.sh:nohup /Users/victorsacek/Documents/petsc/arch-label-debug/bin/mpirun -n 4 ./mandyoc -denok 1.0e-15 -particles_per_ele 1000 -theta_FSSA 0.5 -sub_division_time_step 1.0 -visc_harmonic_mean 0 -particles_perturb_factor 0 -visc_const_per_element 1 -direct_solver 0 -pressure_in_rheol 1 -veloc_ksp_monitor_true_residual -veloc_ksp_type fgmres -veloc_pc_type mg -veloc_pc_mg_galerkin -veloc_pc_mg_levels 3 -veloc_mg_levels_ksp_max_it 8 <FD.in >FD.out & examples/old_version/vanKeken1997_nondim/run.sh:nohup /Users/victorsacek/Documents/petsc/arch-label-debug/bin/mpirun -n 4 ./mandyoc -denok 1.0e-15 -particles_per_ele 1000 -theta_FSSA 0.5 -sub_division_time_step 1.0 -visc_harmonic_mean 0 -particles_perturb_factor 0 -visc_const_per_element 1 -direct_solver 0 -pressure_in_rheol 1 <FD.in >FD.out & examples/vanKeken1997_case1a/run.sh:/Users/kugelblitz/opt/petsc/arch-0-fast/bin/mpirun -n 2 /Users/kugelblitz/Desktop/mandyoc-misc/mandyoc/mandyoc

• Remove old_version folder.

• Maybe we can add a README file for each example, explaining the flag, how to run the example, and what each file is in the folder.

openjournals/joss-reviews#3551

JOSS REVIEW: Functionalities

As per JOSS REVIEW Guidelines I need to check the functionalities that are claimed in the paper.

• Surface Processes
• Material properties
• Newtonian Rheologies
• Non-Linear Rheologies
• Plasticity
• Variable Boundary Conditions (This is one of the main claim)

JOSS REVIEW: Model progression is hard to track

I am running the examples. All seems to go as expected. However, I find it really hard to track progress of the model.
Can you add a step number, time evolution etc?

Improve if conditionals and set a default case to evaluate viscosity in calc_visc

Currently, in calc_visc.cpp there is a series of ifs conditionals to match a rheology and then to evaluate the viscosity.

This need to be improved. A first improvement could be using a switch-case pattern with a default case.

Typo "/n" in Makefile

There is still some wrong "\n" typed as "/n" in Makefile.

Create test for the code

We need to create some test.

JOSS REVIEW: Add Basic Scaling examples

Please add some scaling tests to show how the code behaves on multiple processors.
Can be Weak and/or Strong scaling.

Fix Links

Hello team (@victorsacek, @rafaelmds),
It is necessary to fix the links in all the files.

Don't worry, I can do it

Minor points on the installation instructions

• gfortran is stated as a dependency for PETSc. Most Fortran compilers (not just that one) will work (--with-fc=myfortrancompiler) and it is also possible to configure PETSc without Fortran by using --with-fc=0. The wrinkle in that case is that if you were using --download-fblaslapack, you need to change it to --download-f2cblaslapack)

• Typo: "extrictly needded" --> "strictly needed"

• This sentence has formatting problems: "Finally, add a symlinks of mpirun to ~/.local/bin" (and a typo: "symlinks" --> "symlink")

Originally posted by @psanan in #46 (comment)

JOSS REVIEW: Benchmarks comparison.

• Please add versions of the codes used for comparison in the Benchmarks... Use the most recent versions when possible...

JOSS REVIEW: Tests

For now your submission does not have automated tests.
Please consider adding some basic tests. Maybe run tutorials and/or examples.

Compilation warnings due to conditionally defined values

In some machines, the compiler warns about -Wmaybe-uninitialized variables.

This is caused by some variables whose values are conditionally defined, although a no-matching scenario is never to be expected.

So, this need some improvement to define a default value and some improvement in the series of if conditionals.

Some variables presenting this issue:

• dh_v_mod in Calc_dt_calor (main.cpp)
• temper_aux in Thermal_init_3d (DM1_3d.cpp)
• p_prox in Swarm_add_remove_2d (DMSwarm_move.cpp)
• p_prox_total and chosen in Swarm_add_remove_3d (DMSwarm_move.cpp)

Add description to Crameri/continental_rift examples

We need to a add a simple description file to Crameri and continental_rift examples to explain what the simulation is about and some explanation of the files in the folder.

Improve paper based on JOSS review

This is a to-do list for the paper, according to the JOSS reviewer:

• Explain its positioning relative to other software in this space. You don't have to be comprehensive, but please communicate enough about why a researcher would choose this over a representative set of well-known packages.

• The equations can be formatted with \begin{align} or \begin{gather} rather than a sequence of display maths.

• Use other solver in the examples:
  " I see the acknowledgement mentions multigrid, but the van Keken benchmark is configured to use MUMPS. A note on solvers would be useful (even if the intent is that expert users tune using PETSc options)".

• Mention that in a 3D version will be developed:
  "The paper also doesn't mention whether it supports 3D problems and at what scale it has been used. The code isn't entirely helpful in that "future 3d version" appears in comments, while some functions say they are 3d."

openjournals/joss-reviews#3551

Random number generation

@DMSwarm_3d.cpp and @DMSwarm_2d.cpp:

rand_r(&seed_strain) > RAND_MAX

Update build-docs-web workflow to trigger on docs changes only

Currently, build-docs-web action triggers on every commit for every PR.

So, I think we can adjust the action to trigger only for changes in docs/ folder.

For reference: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#example-using-positive-and-negative-patterns-1

Crameri example does not have and "run" script

A example script to run the Crameri example is missing.
We need to just add a simple example (similar to continental rift script run.sh).

Mandyoc is not beeing build in Dockerfile

The build step of manyoc in Dockerfile is missing the make all call.

So, the fix is simple: just add this call to the RUN command before make install.

Wrong check of number of layers from -seed command and interfaces file

The current code don't check if the -seed command was used, hence, it uses the auxiliary large, default number of layers used to allocate space to parse the -seed command to compare to the number of layers from interfaces.txt file.

To fix this we need to check if the -seed command was used.

Docs - fix expression for viscous component

The expression for viscous component in https://ggciag.github.io/mandyoc/files/implementation.html#viscous-deformation is wrong.

The root of the second invariant of the deviatoric strain rate tensor must be to the power (1-n)/n.

Add note about MPIEXEC (Makefile)

Add a note to mandyoc-installation to alert the user that runtime MPIEXEC might be adjusted.

JOSS REVIEW: clarification of support for nonlinear rheology

I think that some further clarification in the docs is necessary on how exactly strain-rate-dependent nonlinear rheology is supported, since this was the only part of my "Functionality" check that gave me trouble. The rheology section of the docs describes how the viscosity $\eta$ may depend on the strain rate $\dot \epsilon$, and the continental_rift example uses this with various (non-$1$) values of $n$ in different layers. This means that equation 1.4 is now a nonlinear function of $u$, and thus equation 2.3 is using an "old" value of $u$ to compute $\eta$. This seems to be confirmed based on looking at the source code - $\eta$ is computed at each particle location based on the strain rate there, and then then viscosity is considered to be independent of velocity during the next Stokes solve. If I'm correct in my characterization above, this should at least be noted more clearly in the documentation, since I believe that some other packages would perform a nonlinear solve to solve these nonlinear Stokes equations, for example by repeatedly fixing the viscosity based on the last value of $u$ and then re-solving the linear Stokes equations within each time step. For comparison, looking in the ASPECT manual, I see that they support both "single Stokes" and "iterated Stokes" options for their solve. I think you're doing the former, but potential users might wonder if you support the latter.

Fix enviramental variable in src folder

src/compile.sh:make all PETSC_DIR=/Users/victorsacek/Documents/petsc PETSC_ARCH=arch-label-debug
src/compile_opt.sh:make all PETSC_DIR=/Users/victorsacek/Documents/petsc PETSC_ARCH=arch-label-optimized

openjournals/joss-reviews#3551

Output the command used to execute mandyoc in the log

Most information about the input setup is logged when the program starts.
Additionally, we should also log the command line used to launch the simulation.

This is just a log to help ensure the simulation setup is correct.

Segmentation Violation related to particles swarm

In some cases, when running on multiple cores, a condition can occur where some array allocated to support particles operations (addition or removal of particles) is causing a SEGV.

JOSS REVIEW: Examples

• Please consider merging README markdown files and PYTHON scripts into a Jupyter notebook.
  They are easier to read, especially on GitHub. You can also take the user through the step of installing the required Python dependencies...

• Please don't expect user to have PETSc installed in their HOME directory as default...

• Jupyter notebooks would also help to visualize the outcome of the models / Benchmarks. As it is now, users must run the model themselves to see the result and they don't have a mean of comparison. That is also where you should compare with other codes (I2VES, UNDERWORLD, STAGYY)

JOSS REVIEW: Installation Instructions

Hi All,

I am starting my review for JOSS.

First of all. Thanks for submitting the code. The submission is in great shape. Hopefully it won't take too long...
Sorry I'm starting a bit late. It's the beginning of the academic year here in Australia.

A few remarks regarding the installation instructions.

• Please specify the latest version of PETSc that you support (PETSc has a tendency to introduce breaking changes...)
  I would pin the version for the JOSS submission.
• You are relying on PETSc configure to get the external package which is the right thing to do. Maybe you should tell the user that they will have to update a few environment variables (PETSC_DIR, PATH (for mpicc, mpirun))
• Maybe list build requirements and running requirements?
• Are all PETSc external packages required or some of them are optional?
• I suppose the code will eventually be used on HPC. Might be good to add some information on the requirements (COMPILERS, BLAS (openblas?, MKL?), parallel HDF5?, MPICH vs OPENMPI?, CMAKE version (although I suppose this one is for building SUPERLU_DIST)
• Why is gfortran optional?
• A Docker container would be greatly appreciated...

⚠️ In the Mandyoc Makefile, I would not default to $HOME/petsc you should specifically ask for the PETSC_DIR to be set by the user. That will save you lots of trouble...

sticky_blanket_air in surface processes

We need to evaluate the necessity of sticky_blanket_air in models with surface processes.

Also, we need to check the values on the "boundaries" of subdomains in parallels computations.

Abort if periodic_boundary is active in 3D mode (not supported yet)

There is a warning message about periodic_boundary not being supported in 3D mode.
But, we should abort the execution at this point, if it is the case.

Revert and fix MPIEXEC in Makefile

As suggested by @jedbrown, its better to not change MPIEXEC in Makefile (otherwise the system might not be able to use system MPI.

openjournals/joss-reviews#3551

Fix warnings during the Mandyoc build

During the Mandyoc build, there are some warnings -Wunused-but-set-variable. Maybe we can fix.

openjournals/joss-reviews#3551

fix gravity values

In some cases, the gravity acceleration value is hard coded.
We should use an already defined variable instead.

Complite the authors file

@victorsacek, @rafaelmds and Jamison.
It is necessary to complete the AUTHORS file

Improve benchmark section in documentation

It is a good idea to include in the benchmark section how to run the models that we used to create the benchmark and where they are.

So, my suggestions are:

• Explain where the models are in the documentation.
• Create a README file, which explains how to run and create the models in each models folder.

Docs for 3D version

We need to prepare and add the documentation for the 3D version to the docs.

Wrong parse of "-seed" and "-strain_seed" command values

The number of parsed values for the commands -seed and -strain_seed is equal to the number of interfaces.
Hence, if specified a number of values equal to the number of layers, the last value is ignored.

To fix this issue, it is just needed to adjust the size of the arrays (used to store the strain values) to the size of layers (number of interfaces + 1).