fschuch / xcompact3d_toolbox

A set of tools for pre and postprocessing prepared for the high-order Navier-Stokes solver XCompact3d

About

I've been working in Computational Fluid Dynamics research for about ten years, applying and also developing tools that are often powered by High-Performance Computing (HPC). I bring with me the experience of solving complex problems employing computational tools, besides processing, visualizing, and communicating the data produced by these solutions.

My main interests are Software Development and related topics (Python and Fortran), High-performance computing, parallel computing, numerical methods, data science, data visualization, and technical writing.

Highlights

• Open-source contributions:

  • XCompact3d (Member of the development Team): A Fortran-MPI based, finite difference high-performance code for solving the incompressible Navier-Stokes equation and as many as you need scalar transport equations;
  • XCompact3d-toolbox (Core developer): A Python package with many tools for pre and postprocessing prepared for the high-order Navier-Stokes solver XCompact3d.

• Reasearch results:

  • Plunging Criterion: This repository contains the source code and dataset from our work "Plunging condition for particle-laden flows over sloping bottoms: three-dimensional turbulence-resolving simulations" (Computers & Geosciences, 2021);
  • The Plunging Flow by 3D-LES: The Plunging of Hyperpycnal Plumes on Tilted Bed by Three-Dimensional Large-Eddy Simulations (EPTT, 2020).

• Tutorials (all in Portuguese):

  • CFDPython (Translated to Portuguese): A sequence of Jupyter notebooks featuring the "12 Steps to Navier-Stokes";
  • Xarray - Python Brazil (Translated to Portuguese): A 3h course presented at Python Brazil 2020 and 2021;
  • Numerical Methods Applied to Heat Transfer (Author);
  • Numerical Methods in Python (Author).

xcompact3d_toolbox's Issues

Bug at Ahmed body when using double precision

There is a bug at the Ahmed body when using double precision.

How to reproduce the error:

import matplotlib.pyplot as plt
import numpy as np
import xarray as xr
import xcompact3d_toolbox as x3d
import xcompact3d_toolbox.sandbox

x3d.param["mytype"] = np.float64

prm = x3d.Parameters(
    filename="input_ahmed_body.i3d",
    # BasicParam
    itype=10,  # Sandbox is 11 in this branch, change this afterward
    p_row=0,
    p_col=0,
    nx=641,
    ny=129,
    nz=129,
    xlx=20.0,
    yly=4.0,
    zlz=4.0,
    nclx1=2,
    nclxn=2,
    ncly1=2,
    nclyn=1,
    nclz1=1,
    nclzn=1,
    iin=1,
    re=300.0,
    init_noise=0.0125,
    inflow_noise=0.0125,
    dt=0.0025,
    ifirst=1,
    ilast=60000,
    ilesmod=1,
    iibm=1,
    # ibmstuff
    nraf=10,
    # NumOptions
    nu0nu=32.0,
    cnu=0.44,
    # InOutParam
    irestart=0,
    icheckpoint=30000,
    ioutput=500,
    iprocessing=200,
    # LESModel
    jles=4,
)

epsi = x3d.sandbox.init_epsi(prm, dask=True)
for key in epsi.keys():
    epsi[key] = epsi[key].geo.ahmed_body(x=0.25 * prm.xlx, wheels=False)

A side view:

epsi["epsi"].sel(x=slice(4, 10)).isel(z=prm.nz // 2).plot(x="x")

Now a cut near the front:

epsi["epsi"].sel(x=5.1, method="nearest").plot(x="z")

The horizontal cylinders are not represented precisely at the central plane. This does not happen with single precision.

Review n coordinate when phi is represented by more than one digit

xcompact3d_toolbox/xcompact3d_toolbox/io.py

Line 755 in b1c1acd

.assign_coords(n=[int(var[-1]) for var in scalar_variables])

‘-1’ should be replaced by ‘-number_of_digits’

param.mytype is not updating

import xcompact3d_toolbox as x3d

x3d.mytype = np.float32

Even though print(x3d.mytype) shows np.float32, the other methods, like I/O x3d.read_all(), are still showing np.float64.

`_observe_resolution` is not working well

Looks like _observe_resolution is triggering itself when x3d.Parameters.read() is used, crashing the kernel.

Steps to solve:

• Code an unittest for all methods, validations and observations at x3d.Parameters (good for the long run);
• Test solutions and fix the problem;
• Remove warning at tutorial/parameters.ipynb;
• Remove warning at class Parameters docstring.

The files describing the geometry are incompatible when running on Linux

A compatibility problem was reported between the files written at gene_epsi_3D (genepsi.py from xcompact3d-toolbox) and read_geomcomplex (genepsi3d.f90 from Xcompact3d) when running on Linux:

===========================================================
 Reading the geometry!
At line 1055 of file src/genepsi3d.f90 (unit = 67, file = 'data/geometry/obj-x.csv')
Fortran runtime error: Bad value during floating point read

Error termination. Backtrace:
At line 1055 of file src/genepsi3d.f90 (unit = 67, file = 'data/geometry/obj-x.csv')
Fortran runtime error: Bad value during floating point read

Error termination. Backtrace:
At line 1055 of file src/genepsi3d.f90 (unit = 67, file = 'data/geometry/obj-x.csv')
Fortran runtime error: Bad value during floating point read

Error termination. Backtrace:
At line 1055 of file src/genepsi3d.f90 (unit = 67, file = 'data/geometry/obj-x.csv')
Fortran runtime error: Bad value during floating point read

Error termination. Backtrace:
At line 1055 of file src/genepsi3d.f90 (unit = 67, file = 'data/geometry/obj-x.csv')
Fortran runtime error: Bad value during integer read

Error termination. Backtrace:
At line 1055 of file src/genepsi3d.f90 (unit = 67, file = 'data/geometry/obj-x.csv')
Fortran runtime error: Bad value during floating point read

`fix_bug` at genepsi.py is not working properly

The function fix_bug at genepsi.py is not working properly.

It is not a big issue, because it should not be called at all for most cases. However, it must be fixed in a future release.

• The code is breaking because the arguments are not what it is expecting, fix it;
• Find a benchmark case and compare it versus the original Fortran code;
• Code unit tests to keep it working in the long run.

Add support for previsous parameters format

It is desirable to add support for the previous parameters file format .prm.

The challenge is that parameters.f90 is very sensible about the line position of each parameter, and incompact3d.prm does not necessarily include a piece of precise information about the parameter on each line.

The proposed solution, for compatibility, is to use the first comment after each value to describe the variable name that is going to be loaded into xcompact3d_toolbox.Parameters, and a second comment (optional) to explain the variable. Lists are also supported. See a compatible example:

#
# INCOMPACT 3D parameters - Lock-exchange configuration
#
97     #nx
97     #ny
24     #nz
1       #numscalar
2       #p_row
2       #p_col
#
# INCOMPACT 3D Flow parameters
#
2.        #xlx       # Lx (Adimensional size in x-direction)
1.        #yly       # Ly (Adimensional size in y-direction)
0.25    #zlz       # Lz (Adimensional size in z-direction)
4000.  #re        # Reynolds number
# Particle 1
1.     #ri(1)   # Richardson part number
1.     #sc(1)   # Schmidt number
0.0039 #uset(1) # (part settling speed)
1.     #cp(1)   # Concentration (part)
0.27   #cv(1)   # Concentration for variable properties
#
0.0002 #dt        # Time step
#
# INCOMPACT3D Flow configuration
#
1      #ifirst         # First iteration
50001  #ilast      # Last iteration
2      #nscheme  # Temporal scheme (1:AB2, 2:AB3, 3: RK3, 4:KN+AB3 (-DIMPLICIT flag required))
#velocity
1      #nclx1      #Boundary condition in x=0  (0: Periodic, 1:Free-slip, 2: Dirichlet)
1      #nclxn      #Boundary condition in x=Lx (0: Periodic, 1:Free-slip, 2: Dirichlet)
2      #ncly1      #Boundary condition in y=0  (0: Periodic, 1:Free-slip, 2: Dirichlet)
1      #nclyn      #Boundary condition in y=Ly (0: Periodic, 1:Free-slip, 2: Dirichlet)
0      #nclz1      #Boundary condition in z=0  (0: Periodic, 1:Free-slip, 2: Dirichlet)
0      #nclzn      #Boundary condition in z=Lz (0: Periodic, 1:Free-slip, 2: Dirichlet)
#scalar
1      #nclxS1     #Boundary condition in x=0  (0: Periodic, 1:No-flux, 2: Dirichlet)
1      #nclxSn     #Boundary condition in x=Lx (0: Periodic, 1:No-flux, 2: Dirichlet)
2      #nclyS1     #Boundary condition in y=0  (0: Periodic, 1:No-flux, 2: Dirichlet)
1      #nclySn     #Boundary condition in y=Ly (0: Periodic, 1:No-flux, 2: Dirichlet)
0      #nclzS1     #Boundary condition in z=0  (0: Periodic, 1:No-flux, 2: Dirichlet)
0      #nclzSn     #Boundary condition in z=Lz (0: Periodic, 1:No-flux, 2: Dirichlet)
#
# INCOMPACT 3D File parameters
#
0          # ilit               # Read initial flow field? (0:no, 1:yes)
25000  # isave           # Frequency for writing backup file (sauve.dat)
500      # imodulo      # Frequency for visualization for VISU_INSTA
500      # iprocessing # Frequency for statistics computation
10        # itest
#
# NUMERICAL DISSIPATION
#
1      # jLES        #LES Model (0:DNS, 1:ISVV, 2:SMAG, 3:WALE, 4: DSMAG)
32.    # npi2

Two tips:

• Include at the first comment the name that xcompact3d_toolbox expects, and they are not necessarily the same used by incompact3d (see API reference for a complete view);
• Append at the end of the .prm file any extra parameters that you need to use in the Python application. They will not break incompact3d, since it uses a limited number of lines.

`.i3d` parser, from file to Python dictionary and vice versa

• .i3d to Dict;
• Dict to .i3d;
• Init dict with default values.

def read_i3d_to_dict(filepath):
    '''
    This function reads an .i3d file from Xcompact3d and converts it into a Python dictionary
    
    Args:
        filepath: Path to the file
    
    Returns:
        A dictionary containing the flow parameters
    '''
    
    f = open(filepath)

    prm = {}

    for line in f:
        
        line = line.partition('!')[0].replace(' ', '')  #Remove comments
        line = " ".join(line.split())  #Remove spaces

        if line == '':  #Cycle if line is empty
            continue

        #Beginning of a new group
        if line[0] == '&':
            key = line[1:]
            dic = {}
            continue

        #End of the group
        if line.lower() == '/end':
            prm[key] = dic
            continue
        
        #Get variable's name and value
        param = line.partition('=')[0]
        value = line.partition('=')[-1]
        
        try:
            #Converting from string according to datatype
            if value[0] == "'" and value[-1] == "'":  #String
                value = value[1:-1]
            elif value.lower() == '.false.':  #Bool
                value = False
            elif value.lower() == '.true.':  #Bool
                value = True
            elif "." in value:  #Float
                value = float(value)
            else:  #Int
                value = int(value)
        except:
            print(f"Can't convert {value}")
            continue

        if "(" in param and ")" == param[-1]:  #List
            param = param.split('(')[0]
            if param not in dic:
                dic[param] = []
            dic[param].append(value)
        else:  #Not a list
            dic[param] = value

    f.close()

    return prm

Installation Errorin Ubuntu: HDR5 version already higher than the required version still throwing error

I am trying to install the library using pip install xcompact3d-toolbox on ubuntu. I have the latest python version 3.10.6. I am not able to install the library. Can you check what is the problem ?

Error thrown in ubuntu. have HDF5 installed in my ubuntu. The version of hdf5 is 1.10.7. Even though the installed version higher than the required version, it is throwing this error.

  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [9 lines of output]
      /tmp/pip-install-t2d9p96x/netcdf4_cc59d687b3284bc59f45ac26b7c46ba9/setup.py:136: DeprecationWarning: The SafeConfigParser class has been renamed to ConfigParser in Python 3.2. This alias will be removed in Python 3.12. Use ConfigParser directly instead.
        config = configparser.SafeConfigParser()
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-t2d9p96x/netcdf4_cc59d687b3284bc59f45ac26b7c46ba9/setup.py", line 224, in <module>
          raise ValueError('HDF5 version >= 1.8.0 is required')
      ValueError: HDF5 version >= 1.8.0 is required
      reading from setup.cfg...
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Some namespaces are not optional

The last release included a representation of the class Parameters on screen. In order to keep it short, some namespaces were removed when not necessary, but in fact, some of them are always expected by Xcompact3d.