Fluctuation KE objective for cavity & step flows about hydrogym

[jcallaham]

Hi @ReHoss - are you running with a controller or just the natural flow? I can't remember for sure, but I thought I had tested it without control and it worked okay, but that was a while ago. If you can share exactly what your parameters and setup is, I can try to reproduce it and see if I can help. I haven't tried RL on the cavity yet (and the objective function does still need a bit of work).

This whole project is pre-release and is still pretty actively under development, so feel free to try stuff out but just be warned that everything may not work out of the box. We're hoping to have a first release in the next couple of months, and doing things like making sure all the flow configs run on all the meshes will definitely be a part of that.

By the way, thanks for the feedback on the installation process (#64) - I'll have to take a closer look at that and get back to you.

[jcallaham]

Got it, okay I have a guess at what's happening. You're starting from a zero initial condition, and I'm guessing if you look at the visualization around the time it fails you'll see that's when the first vortex on the trailing edge hits the outlet. We're currently just using zero-pressure Dirichlet boundary conditions on the outlet, which probably isn't ideal but seems to work well enough for the time being. But it can be a bit touchy with the initial transient.

What I usually do for initializing the transient flow is to start from the steady-state base flow and then perturb it so that the solution doesn't do anything too different from the natural post-transient behavior. You can do this in a principled way (like with the least-stable eigenvectors), but this takes a bit of work. I plan to add an example of that soon, but it's not ready quite yet. An easier alternative is to just add random noise, and then of course there will be some component in the eigenvector direction - this example is working for me on the coarse mesh.

If you're still having trouble, go ahead and open a new issue so I can close this out when the cavity objective is fixed, since I think this is a separate thing. Also just a word of warning - this coarse solution is definitely not converged with respect to mesh size, so don't put too much trust in anything except the default meshes.

[jcallaham]

Good question... maybe let's move that kind of thing to the Discussions section? Then issues can track active development, bugs, etc. But also, keep in mind we're still very much in "beta" right now, so no guarantees on anything yet!

[SamAhnert]

I also like 3 since it seems like a quick way to get an approximate idea of the where the fluctuation KE is at without a lot of overhead. So long as the trends of the "fluctuating measurement" are reliable (in that they correspond strongly with changes in fluctuation KE), I'd imagine it would be fine since we're only using that quantity as a reward to the agent.

For the sake of justification though, would it be necessary to show how the "fluctuating measurement" corresponds with fluctuating KE and quantify that somehow? Is that possible? Or is it generally a given that these two correlate strongly?

(Sorry if these questions seem obvious, this is a bit of a new territory to me)

[ReHoss]

Hi @jcallaham , I tried to integrate the Cavity with the coarse mesh, it looks good at the beginning but then I get a divergence error at around t = 2.0 . Regarding the medium mesh, I directly obtain something like numerical explosion?

What could we do to solve this ?

I'd like to implement RL on the coarse mesh, a bit like you suggest in the introductory Jupyter Notebook for the Cylinder flow.

Thank you for your consideration,

[ReHoss]

Hi @jcallaham ,

By running the following script with mpiexec:

import hydrogym.firedrake as hgym
import psutil

name_checkpoint = "checkpoint-coarse-cavity-jcallaham-test.h5"

Tf = 1000
DT = 1e-4
n_steps = int(Tf // DT)


def log_postprocess(flow):
    (val,) = flow.get_observations()
    mem_usage = psutil.virtual_memory().percent
    return val, mem_usage


def main():
    callbacks = [
        hgym.io.LogCallback(
            postprocess=log_postprocess,
            nvals=2,
            interval=10,
            print_fmt="t: {0:0.2f},\t\t value: {1:0.3f},\t\t RAM: {2:0.1f}%",
            filename=None,
        ),
        hgym.io.CheckpointCallback(interval=100, filename=name_checkpoint)
    ]

    flow = hgym.Cavity(
        mesh="coarse",
        restart=None,
        callbacks=callbacks)

    hgym.integrate(flow, t_span=(0, Tf), dt=DT, callbacks=callbacks)


if __name__ == "__main__":
    main()

At t = 3.09 I get the following error:

t: 3.09, value: 6.801, RAM: 88.0%
Traceback (most recent call last):
File "./examples/cylinder/pd-control-cavity-jcallaham.py", line 38, in
main()
File "./examples/cylinder/pd-control-cavity-jcallaham.py", line 34, in main
hgym.integrate(flow, t_span=(0, Tf), dt=DT, callbacks=callbacks)
File "/home/hosseinkhan/Documents/work/phd/project_hydrogym/firedrake/lib/python3.8/site-packages/hydrogym/firedrake/solver.py", line 456, in integrate
return solver.solve(t_span, callbacks=callbacks, controller=controller)
File "/home/hosseinkhan/Documents/work/phd/project_hydrogym/firedrake/lib/python3.8/site-packages/hydrogym/core.py", line 264, in solve
flow = self.step(iter, control=u)
File "/home/hosseinkhan/Documents/work/phd/project_hydrogym/firedrake/lib/python3.8/site-packages/hydrogym/firedrake/solver.py", line 183, in step
self.predictor.solve()
File "PETSc/Log.pyx", line 115, in petsc4py.PETSc.Log.EventDecorator.decorator.wrapped_func
File "PETSc/Log.pyx", line 116, in petsc4py.PETSc.Log.EventDecorator.decorator.wrapped_func
File "/home/hosseinkhan/Documents/work/phd/project_hydrogym/firedrake/src/firedrake/firedrake/adjoint/variational_solver.py", line 90, in wrapper
out = solve(self, **kwargs)
File "/home/hosseinkhan/Documents/work/phd/project_hydrogym/firedrake/src/firedrake/firedrake/variational_solver.py", line 282, in solve
solving_utils.check_snes_convergence(self.snes)
File "/home/hosseinkhan/Documents/work/phd/project_hydrogym/firedrake/src/firedrake/firedrake/solving_utils.py", line 138, in check_snes_convergence
raise ConvergenceError(r"""Nonlinear solve failed to converge after %d nonlinear iterations.
firedrake.exceptions.ConvergenceError: Nonlinear solve failed to converge after 0 nonlinear iterations.
Reason:
DIVERGED_LINEAR_SOLVE
application called MPI_Abort(PYOP2_COMM_WORLD, 1) - process 0

Here is the list of packages installed on my environment:

aiosignal==1.3.1
anyio==3.6.2
appdirs==1.4.4
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
arrow==1.2.3
astroid==2.12.13
asttokens==2.2.1
attrs==22.2.0
backcall==0.2.0
beautifulsoup4==4.11.1
bleach==5.0.1
cached-property==1.5.2
cachetools==5.2.0
certifi==2022.12.7
cffi==1.15.1
cgen==2020.1
charset-normalizer==2.1.1
click==8.1.3
cloudpickle==2.2.0
cma==3.2.2
codepy==2019.1
-e git+https://github.com/coneoproject/COFFEE.git@70c1e66a4e4e39d3bf75274505a16901af110751#egg=COFFEE
colorama==0.4.6
comm==0.1.2
contourpy==1.0.6
control==0.9.3.post2
coverage==7.0.1
cycler==0.11.0
Cython==0.29.32
dataclasses==0.6
debugpy==1.6.4
decorator==4.4.2
defusedxml==0.7.1
dill==0.3.6
distlib==0.3.6
dmsuite==0.1.1
-e git+https://github.com/dolfin-adjoint/pyadjoint.git@4742834d89730c32bae71423f699569ee78027a4#egg=dolfin_adjoint
entrypoints==0.4
evotorch==0.3.0
exceptiongroup==1.1.0
execnet==1.9.0
executing==1.2.0
fastjsonschema==2.16.2
-e git+https://github.com/firedrakeproject/fiat.git@43bc840302475f3001aebb4a821e2727f927fd89#egg=fenics_fiat
fenics-ufl @ git+https://github.com/firedrakeproject/ufl.git@3c951541ad1f9633427e72898d584dbcfae82293
filelock==3.9.0
-e git+https://github.com/FInAT/FInAT.git@3f28113b5c31ed34c7a129a0a958e3b82a7c6bba#egg=FInAT
-e git+https://github.com/firedrakeproject/firedrake.git@660d46b5f380ed23ec3074efdf9301033fc1e3fa#egg=firedrake
flake8==6.0.0
fonttools==4.38.0
fqdn==1.5.1
frozenlist==1.3.3
functorch==1.13.0
genpy==2022.1
gmsh==4.11.1
grpcio==1.51.1
gym==0.26.2
gym-notices==0.0.8
h5py @ file:///home/hosseinkhan/Documents/work/phd/project_hydrogym/firedrake/src/h5py
hydrogym==0.1.2.1
idna==3.4
importlib-metadata==6.0.0
importlib-resources==5.10.2
iniconfig==1.1.1
ipykernel==6.19.4
ipython==8.7.0
ipython-genutils==0.2.0
ipywidgets==8.0.4
islpy==2022.2.1
isoduration==20.11.0
isort==5.11.4
jedi==0.18.2
Jinja2==3.1.2
jsonpointer==2.3
jsonschema==4.17.3
jupyter==1.0.0
jupyter-console==6.4.4
jupyter-events==0.5.0
jupyter_client==7.4.8
jupyter_core==5.1.1
jupyter_server==2.0.6
jupyter_server_terminals==0.4.3
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.5
kiwisolver==1.4.4
lazy-object-proxy==1.7.1
-e git+https://github.com/firedrakeproject/loopy.git@3988272b385fd770a3427b539e8b27e367f9db33#egg=loopy
Mako==1.2.4
MarkupSafe==2.1.1
matplotlib==3.6.2
matplotlib-inline==0.1.6
mccabe==0.7.0
mistune==2.0.4
modred==2.1.0
mpi4py==3.1.4
mpmath==1.2.1
msgpack==1.0.4
nbclassic==0.4.8
nbclient==0.7.2
nbconvert==7.2.7
nbformat==5.7.1
nbval==0.9.6
nest-asyncio==1.5.6
networkx==2.8.8
notebook==6.5.2
notebook_shim==0.2.2
numpy==1.24.1
nvidia-cublas-cu11==11.10.3.66
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-runtime-cu11==11.7.99
nvidia-cudnn-cu11==8.5.0.96
packaging==22.0
pandas==1.5.2
pandocfilters==1.5.0
parso==0.8.3
petsc4py @ file:///home/hosseinkhan/Documents/work/phd/project_hydrogym/firedrake/src/petsc/src/binding/petsc4py
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.3.0
pkg_resources==0.0.0
pkgconfig==1.5.5
pkgutil_resolve_name==1.3.10
platformdirs==2.6.2
pluggy==1.0.0
progress==1.6
prometheus-client==0.15.0
prompt-toolkit==3.0.36
protobuf==4.21.12
psutil==5.9.4
ptyprocess==0.7.0
PuLP==2.7.0
pure-eval==0.2.2
pybind11==2.9.2
pycodestyle==2.10.0
pycparser==2.21
pyflakes==3.0.1
Pygments==2.13.0
pylint==2.15.9
pymbolic==2022.2
-e git+https://github.com/OP2/PyOP2.git@7aca5e500b939ad35d937c5d1ffba7a27e40ff62#egg=PyOP2
pyparsing==3.0.9
pyrsistent==0.19.3
pytest==7.2.0
pytest-xdist==3.1.0
python-dateutil==2.8.2
python-json-logger==2.0.4
pytools==2022.1.14
pytz==2022.7
PyYAML==6.0
pyzmq==24.0.1
qtconsole==5.4.0
QtPy==2.3.0
ray==2.2.0
requests==2.28.1
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
scipy==1.9.3
seaborn==0.12.2
Send2Trash==1.8.0
six==1.16.0
sniffio==1.3.0
soupsieve==2.3.2.post1
stack-data==0.6.2
symengine==0.9.2
sympy==1.11.1
terminado==0.17.1
tinycss2==1.2.1
tomli==2.0.1
tomlkit==0.11.6
torch==1.13.0
tornado==6.2
traitlets==5.8.0
-e git+https://github.com/firedrakeproject/tsfc.git@ceebdd5044f2b6549f256b45cb85718283a15e9a#egg=tsfc
typing_extensions==4.4.0
uri-template==1.2.0
urllib3==1.26.13
virtualenv==20.17.1
vtk==9.2.3
wcwidth==0.2.5
webcolors==1.12
webencodings==0.5.1
websocket-client==1.4.2
widgetsnbextension==4.0.5
wrapt==1.14.1
zipp==3.11.0

By the way, what are the technical difficulties to implement the feedback reward from your 3rd suggestion ?

Thank you for your consideration:)!

[ReHoss]

Thanks a lot @jcallaham

In fact I am interested on the time delay property of the Cavity flow and this coarse mesh should be okay for the beginning even if it does not represent all the features of this flow.

I have basic knowledge in CFD and I'll have probably a lot of question about the implementation of the flows and such kinds of issues etc (e.g. actuators and sensors settings; reward signal). the channel recommended to discuss about it are Github issues then ?

Best,

[jcallaham]

On the objective function for cavity and step flows, I think I'm settling on modifying the flows slightly to additionally store a base flow qB that can be read in from a restart file, for instance computed along with a transient simulation. This will add a bit of overhead to these flows and is an extra thing users will have to keep track of, but if we have example scripts hopefully it won't be too burdensome. That's the only solution I can think of that won't break easily with changing meshes or Reynolds numbers.

Working on branch jc/cavity-mean-ke, and I still need to add the implementation+examples for the backwards-facing step before PR.