Coder Social home page Coder Social logo

extremeflow / neko Goto Github PK

View Code? Open in Web Editor NEW
161.0 8.0 29.0 596.3 MB

/ᐠ. 。.ᐟ\ᵐᵉᵒʷˎˊ˗

Home Page: https://neko.cfd/

License: Other

Makefile 0.78% Fortran 78.25% M4 1.48% Shell 0.13% C 6.62% Cuda 3.08% C++ 9.33% Python 0.33%
fortran cfd hpc

neko's Introduction

Neko

CI develop DOI

About

Neko is a portable framework for high-order spectral element flow simulations. Written in modern Fortran, Neko adopts an object-oriented approach, allowing multi-tier abstractions of the solver stack and facilitating various hardware backends ranging from general-purpose processors, CUDA and HIP enabled accelerators to SX-Aurora vector processors. Neko has its roots in the spectral element code Nek5000 from UChicago/ANL, from where many of the namings, code structure and numerical methods are adopted.

Cloning the project

git clone https://github.com/ExtremeFLOW/neko

Documentation

Documentation for Neko is available at https://neko.cfd and most things related to the code, cases, and different features are described there. The documentation is always improving, in large part due to our active users and if something is missing or hard to understand, don't be afraid to start a discussion or create a Pull request. It is a great way to help us improve and also to start getting involved in the project.

Building the project

To build the project you will need: A Fortran compiler supporting the Fortran-08 standard, a working MPI installation, JSON-Fortran, and BLAS/lapack. Optional dependencies are gslib and ParMETIS. We use autotools to build the project. These instructions should work in general, but as the project is quickly developing, things might change. While we assume MPI and BLAS are installed, if JSON-Fortran is not already available it can be cloned, installed, and the correct paths set with the following commands (Skip this step if you already have an installation of JSON-Fortran).

export JSON_INSTALL=/path/to/json-fortran_install # Where you want to install json-fortran
git clone --depth 1 https://github.com/ExtremeFLOW/json-fortran/
cmake -S json-fortran -B json-fortran/build -DCMAKE_INSTALL_PREFIX=${JSON_INSTALL} -DUSE_GNU_INSTALL_CONVENTION=ON ..
cmake --build json-fortran/build --parallel
cmake --install json-fortran/build
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${JSON_INSTALL}/lib/ #On some systems lib should be replaced with lib64
export PKG_CONFIG_PATH=${PKG_CONFIG_PATH}:${JSON_INSTALL}/lib/pkgconfig 

A basic CPU version of Neko can then be installed according to the following

cd neko
./regen.sh
./configure --prefix=/path/to/neko_install # Where you want to install neko
make install

More detailed installation instructions and all the different options (such as how to install Neko for GPUs) can be found in the documentation available at https://neko.cfd.

Running examples

After the project has been built

cd examples/tgv
/path/to/neko_install/bin/makeneko tgv.f90
mpirun -np 4 ./neko tgv.case

If there is not a .f90 (user) file in the example, the standard executable neko can also be used, for example:

cd examples/hemi
mpirun -np 4 /path/to/neko_install/bin/neko hemi.case

only uses built-in functions and does not need a compiled user file. Whether you will need a user file or not depends on what functionality you want and this is also documented in the documentation.

Publications using Neko

  • Jansson, N., 2021. Spectral Element Simulations on the NEC SX-Aurora TSUBASA. In proc. HPCAsia 2021.
  • Karp, M., Podobas, A., Kenter, T., Jansson, N., Plessl, C., Schlatter, P. and Markidis, S., 2022. A high-fidelity flow solver for unstructured meshes on field-programmable gate arrays: Design, evaluation, and future challenges. In proc. HPCAsia 2022.
  • Karp, M., Jansson, N., Podobas, A., Schlatter, P., and Markidis, S., 2022. Reducing Communication in the Conjugate Gradient Method: A Case Study on High-Order Finite Elements. In proc. PASC 2022.
  • Karp, M., Massaro, D., Jansson, N., Hart, A., Wahlgren, J., Schlatter, P., and Markidis, S., 2023. Large-Scale Direct Numerical Simulations of Turbulence Using GPUs and Modern Fortran. The International Journal of High Performance Computing Applications, 37, 5.
  • Jansson, N., Karp, M., Perez, A., Mukha, T., Ju, Y., Liu, J., Páll, S., Laure, E., Weinkauf, T., Schumacher, J., Schlatter, P., Markidis, S., 2023. Exploring the Ultimate Regime of Turbulent Rayleigh–Bénard Convection Through Unprecedented Spectral-Element Simulations. SC '23: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis.
  • Jansson, N., Karp, M., Podobas, A., Markidis, S. and Schlatter, P., 2024. Neko: A modern, portable, and scalable framework for high-fidelity computational fluid dynamics. Computer & Fluids, 275.

Acknowledgments

The development of Neko was supported by the European Commission Horizon 2020 project grant EPiGRAM-HS: Exascale Programming Models for Heterogeneous Systems (grant reference 801039), the Swedish Research Council project grant Efficient Algorithms for Exascale Computational Fluid Dynamics (grant reference 2019-04723) and the SeRC Exascale Simulation Software Initiative (SESSI). The Neko logo was designed by Robert Hansen Jagrelius.

Sponsored by Zulip, an open-source modern team chat app designed to keep both live and asynchronous conversations organized.

neko's People

Contributors

adampep avatar adperezm avatar al42and avatar jacwah avatar jonvin73 avatar maansandersson avatar martinkarp avatar muenschm avatar njansson avatar pohm01 avatar pschlatt1 avatar pszi1ard avatar shiyu-sandy-du avatar steven-chien avatar timfelle avatar timofeymukha avatar vbaconnet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

neko's Issues

Wrappers fails for ParMETIS 5.x

Recent versions of ParMETIS mixes 32 and 64bit idx types for integer and floats. The current wrapper codes assumes both are 32bit, throwing warnings on e.g. Cray's ParMETIS (with int32, float64).

parmetis_wrapper.c:15:2: error: "ParMETIS built with 64bit support"
#error "ParMETIS built with 64bit support"
 ^
parmetis_wrapper.c:33:48: warning: incompatible pointer types passing 'float *' to parameter of type 'real_t *' (aka 'double *') [-Wincompatible-pointer-types]
  rcode = ParMETIS_V3_PartGeom(vtxdist, ndims, xyz, part, &comm);
                                               ^~~
/opt/cray/pe/tpsl/20.03.2/CRAYCLANG/9.0/haswell/include/parmetis.h:78:52: note: passing argument to parameter 'xyz' here
             idx_t *vtxdist, idx_t *ndims, real_t *xyz, idx_t *part, MPI_Comm *comm);
                                                   ^
parmetis_wrapper.c:60:44: warning: incompatible pointer types passing 'float *' to parameter of type 'real_t *' (aka 'double *') [-Wincompatible-pointer-types]
                                   numflag, ncon, ncommonnodes, nparts,tpwgts,
                                                                       ^~~~~~
/opt/cray/pe/tpsl/20.03.2/CRAYCLANG/9.0/haswell/include/parmetis.h:99:15: note: passing argument to parameter 'tpwgts' here
             real_t *tpwgts, real_t *ubvec, idx_t *options, idx_t *edgecut, idx_t *part, 
                     ^
parmetis_wrapper.c:61:8: warning: incompatible pointer types passing 'float *' to parameter of type 'real_t *' (aka 'double *') [-Wincompatible-pointer-types]
                                   ubvec, options, edgecut, part, &comm);
                                   ^~~~~
/opt/cray/pe/tpsl/20.03.2/CRAYCLANG/9.0/haswell/include/parmetis.h:99:31: note: passing argument to parameter 'ubvec' here
             real_t *tpwgts, real_t *ubvec, idx_t *options, idx_t *edgecut, idx_t *part, 
                                     ^

Either the wrappers or Fortran interfaces needs to handle this before

Abstract BDF type

In order to be able to use different BDF schemes a new BDF type is necessary with subroutines such as 'bdf%apply'

NEKO fails with AOCC/2.3.0

Fails at start (any of the bench programs or examples) with the following trace,

#0  0x00007ffff6261aca in f90_get_object_size_i8 () from /local/spack/opt/spack/linux-centos8-zen/gcc-8.3.1/aocc-2.3.0-vyf7kpa53rdwnpwlxhoeeomnmgbqc7mv/lib/libflang.so
#1  0x00007ffff6261c88 in sourced_alloc_and_assign () from /local/spack/opt/spack/linux-centos8-zen/gcc-8.3.1/aocc-2.3.0-vyf7kpa53rdwnpwlxhoeeomnmgbqc7mv/lib/libflang.so
#2  0x00007ffff626355f in f90_poly_asn_i8 () from /local/spack/opt/spack/linux-centos8-zen/gcc-8.3.1/aocc-2.3.0-vyf7kpa53rdwnpwlxhoeeomnmgbqc7mv/lib/libflang.so
#3  0x00000000002617f3 in htable::htable_init (this=..., size=<optimized out>, key=..., data=...) at htable.f90:200
#4  0x000000000026459b in htable::htable_i4_init (this=..., size=64, data=...) at htable.f90:602
#5  0x000000000026633d in uset::uset_i4_init (this=..., n=<optimized out>) at uset.f90:146
#6  0x0000000000216f5b in distdata::distdata_init (distdata=...) at distdata.f90:28
#7  0x000000000026ca62 in mesh::mesh_init_common (m=...) at mesh.f90:211
#8  0x000000000026bb82 in mesh::mesh_init_nelv (m=..., gdim=3, nelv=<optimized out>) at mesh.f90:129
#9  0x0000000000284d8a in nmsh_file::nmsh_file_read (this=..., data=...) at nmsh_file.f90:71
#10 0x0000000000286d4f in file::file_read (this=..., data=...) at file.f90:91
#11 0x0000000000290a35 in case::case_init (c=..., case_file=...) at case.f90:108
#12 0x00000000002b37a1 in neko::neko_init (c=...) at neko.f90:97
#13 0x00000000002b37e7 in turboneko () at driver.f90:5
#14 0x00000000002b3836 in main ()

Seems to be related to the key mold in htable

(This is a duplicate of the Jira issue: NEKO-159 )

Reduce indirect module clutter

Reduce indirect dependency and indirectly defined types e.g.

use apa, only : apa_t

and aim for the entire module to be private.

Pipelined BiCGSTAB

Implement and evaluate stability and performance vs both BiCGSTAB and GMRES.

Refs.

10.1016/j.parco.2017.04.005

rea2bin fails

Fails after curved element support

ndim = 3, nelements =   2042
 *** WARNING: No NEKTON map file found ***
forrtl: severe (151): allocatable array is already allocated
Image              PC                Routine            Line        Source             

LES Filtering

Some kind of filter to enable LES simulations, most likely similar to what is in Nek5000.

BiCGSTAB

Implement and assess performance of BiCGSTAB vs GMRES.

Device memory abstraction layer

Module using the OpenMP runtime for allocation and data transfer calls.

Will allow for transfers outside of OpenMP data regions

Device association fails with SIGFPE when using GCC

Hashing a C pointer throws a SIGFPE when using GCC 10.2.0 from device_associate call (works fine with CCE 11.0.2)

    _  __  ____  __ __  ____ 
   / |/ / / __/ / //_/ / __ \
  /    / / _/  / ,<   / /_/ /
 /_/|_/ /___/ /_/|_|  \____/ 
 
 (version: 0.1.1)
 (build: 2021-06-02 on x86_64-pc-linux-gnu using gnu)
 

Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.

Backtrace for this error:
#0  0x7fa071cbc78f in ???
#1  0x406799 in __htable_MOD_htable_cptr_hash
	at /scratch/njansson/neko_cuda/src/htable.f90:1375
#2  0x40a5e2 in htable_set
	at /scratch/njansson/neko_cuda/src/htable.f90:276
#3  0x40b0cf in __htable_MOD_htable_cptr_set
	at /scratch/njansson/neko_cuda/src/htable.f90:1350
#4  0x4034d8 in __device_MOD_device_associate
	at /scratch/njansson/neko_cuda/src/device.F90:124
#5  0x402ddf in axbench
	at /scratch/njansson/gpu_test/driver.f90:18
#6  0x402c0c in main
	at /scratch/njansson/gpu_test/driver.f90:2

Pipelined CG

Implement and investigate performance of pipelined CG solvers.

Refs.
10.1016/j.parco.2013.06.001
10.1109/SC.2016.17

Out of memory when compiling hsmg with CCE/10.0.3

For -O3 and higher CCE can't compile hsmg with the following error,

ftn-7951 crayftn: LIMIT HSMG_SOLVE, File = pc_hsmg.f90, Line = 279 
  Insufficient memory is available for compiler to continue.

Internal compiler error Cray Fortran/11.0.2

Latest Cray Fortran compiler goes belly up when compiling Neko, throwing an interesting ICE

type notype
   Error message      ::  type
   Error detected     ::  File 'pdgcs/llvm-type.c', line 2073
   Optimizer built    ::  2021-01-19 (production)

   File               ::  hex.f90
   Function           ::  tuple_i4_assign_vector$tuple__clone_5766_1618238902_1
   at or near line    ::  194

   File path          ::  /afs/pdc.kth.se/home/n/njansson/Private/src/neko/src/hex.f90
   Compiler hash      ::  bb0c222f226193f58f46bc6c5e47a320418ac947
   Target             ::  haswell

Creating internal compiler error backtrace (please wait):
Unable to launch gdb in batch mode.
gdb either does not exist or is already attached.
Trying alternative backtrace mechanism...

[0x000000010b82c3] linux_backtrace ??:?
[0x000000010b9192] pdgcs_internal_error(char const*, char const*, int) ??:?
[0x000000012c5860] llvm_cg::gen_llvm_type(TYPE, unsigned int) ??:?
[0x0000000131c5c0] llvm_cg::gen_llvm_lin_info_addr(EXT_LIN_INFO const*, EXP_INFO) ??:?
[0x00000001312a13] llvm_cg::gen_llvm_obj_expr(EXP_INFO, bool) ??:?
[0x000000012ebfc5] llvm_cg::gen_llvm_expr(EXP_INFO, bool) ??:?
[0x0000000134a597] llvm_cg::gen_llvm_binary_expr(EXP_INFO) ??:?
[0x000000012ebff9] llvm_cg::gen_llvm_expr(EXP_INFO, bool) ??:?
[0x0000000133cf28] llvm_cg::gen_llvm_unary_expr(EXP_INFO) ??:?
[0x000000012ec039] llvm_cg::gen_llvm_expr(EXP_INFO, bool) ??:?
[0x0000000134a597] llvm_cg::gen_llvm_binary_expr(EXP_INFO) ??:?
[0x000000012ebff9] llvm_cg::gen_llvm_expr(EXP_INFO, bool) ??:?
[0x000000012bad8a] llvm_cg::gen_llvm_stmt(int) ??:?
[0x0000000118bcb9] llvm_cg::llvm_function_with_body() llvm-pdgcs.c:?
[0x0000000118e33c] internal_llvm_function(int, int, int, int, llvm_cg::FunctionTranslateMode) llvm-pdgcs.c:?
[0x00000001190f84] llvm_function(int) ??:?
[0x000000006e89f5] PDGCS_do_proc ??:?
[0x00000000649e44] cvrt_proc_to_pdg m_cvrt.c:?
[0x0000000064a826] m_cvrt_to_pdg ??:?
[0x0000000067b976] process_scp m_i_control.c:?
[0x0000000067d0ee] m_start_ipa ??:?
[0x00000000580659] main ??:?
[0x007f4683056349] ?? ??:0
[0x000000006059c0] _start /usr/src/packages/BUILD/glibc-2.11.3/csu/../sysdeps/x86_64/elf/start.S:113

Note:  This is a non-debug compiler.  Technical support should
       continue problem isolation using a compiler built for
       debugging.

ftn-7991 crayftn: INTERNAL TUPLE_I4_ASSIGN_VECTOR$TUPLE__CLONE_5766_1618238902_1, File = hex.f90, Line = 194 
  INTERNAL COMPILER ERROR:  "type" (pdgcs/llvm-type.c, line 2073, version bb0c222f226193f58f46bc6c5e47a320418ac947)
ftn-2116 crayftn: INTERNAL  
  "/opt/cray/pe/cce/11.0.2/cce/x86_64/bin/optcg" was terminated due to receipt of signal 013:  Segmentation fault (core dumped).
`

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.