jviquerat / beacon

A benchmark library for DRL-based flow control

beacon

This repository proposes benchmark cases for DRL-based flow control. The considered cases are voluntarily limited in CPU requirements, so they can be used for prototyping on local machines while still presenting realistic flow control aspects. If you use this library in the context of your research work, please consider citing:

Beacon, a lightweight deep reinforcement learning benchmark library for flow control, J. Viquerat, P. Meliga, P. Jeken, E. Hachem, 2024 (pre-print and paper)

Pull requests with new cases are welcome !

Environment	Description	Illustration
shkadov-v0	The agent actuates several jets to damp the instabilities of a falling fluid film (original approach was from Belus et al. in this paper.) States are the mass flow rates of the fluid upstream of each jet. One episode represents 400 actions, and the training is made on 200000 transitions for 10 jets.
rayleigh-v0	The agent learns to disable the convection in a Rayleigh-Benard cell by locally modifying the temperature of different segments on the bottom plate (original approach was from Beintema et al. in this paper.) States are the temperature and velocities collected at probe points in the domain. One episode represents 100 actions, and the training is made on 600000 transitions.
mixing-v0	The agent learns to optimize the mixing of a passive scalar quantity in a cavity by selecting the lid velocities. States are the scalar concentration and velocities collected at probe points in the domain. One episode represents 100 actions, and the training is made on 100000 transitions.
lorenz-v0	The agent learns to lock the Lorenz attractor in the x<0 quadrant by acting on a forcing term in the ODE system (original approach was from Beintema et al. in this paper.) States are the (x,y,z) variables and their time derivatives. One episode represents 500 actions, and the training is made on 2000000 transitions.
burgers-v0	The agent controls a forcing term corresponding to an actuator located in the middle of the 1D domain. The goal is to suppress the noise generated at the inlet. States are the velocities sampled upstream of the actuator. One episode represents 200 actions, and the training is made on 500000 transitions.
sloshing-v0	The agent controls the acceleration of a tank containing a fluid in order to damp the sloshing movement initiated during an excitation pahse. State vector is a downsampled mass flow rate vector. One episode represents 200 actions, and the training is made on 200000 transitions.
vortex-v0	The agent controls the amplitude and phase of a forcing term acting on a model representing a cylinder attached to the wall with a damper-spring, immersed in a flow at Reynolds 50. One episode represents 800 actions, and the training is made on 1000000 transitions.

beacon's Issues

kuramoto–sivashinsky

add control of kuramoto-sivashinsky case

https://royalsocietypublishing.org/doi/epdf/10.1098/rspa.2019.0351
https://arxiv.org/pdf/2104.05437.pdf

format run.py for lorenz

granular

add a granular case

close figures

check that figures get closed during rendering

general things to check before finalizing

• check that fvm solvers remain stable with lower residuals OK
• check that outputs and renderings are well formatted and stored in all envs OK
• "png" folder should become "render" folder + open folders at first rendering step (see lorenz) OK
• remove all sh and json files and store them separately
• check all default parameters OK
• nb of # in comments OK
• remove init parameter in constructors when it is not necessary OK
• check copy of previous actions OK

shkadov fix

• fix shkadov case and doc given recent modifications
• re-test with td3/sac using random initialization

swimming

simplified swimming model : https://arxiv.org/pdf/2009.14280.pdf

shkadov obs

shkadov obs are the values of q but obs labels are with h in the init routine of the env

rayleigh-benard

add the rayleigh-benard cell case

lorenz

add lorenz case

shkadov scheme

correct doc for shkadov : fd -> fv

fix rayleigh

big mistake in rayleigh case, loop on segments uses same index as loop on actions.
Fix and re-run

burgers

re-run burgers with 4000 transitions per batch and 400.000 total transitions

check history of observations in rayleigh + mixing

periodic channel

check related case here : https://arxiv.org/pdf/2308.13029.pdf

karman

add a von karman cylinder case

burgers

overall coherence

it would be good to revise the entire structure of an environment and make all environments very similar.
To be done once the sloshing environment works, and to report modifications to the shkadov one.

rendering rayleigh

rework rayleigh rendering with subplots

sloshing boostrap

sloshing results were obtained without bootstrap -> modify and re-run

numba loops

try to use loops on shkadov instead of array indexing to check speedup

rayleigh observations

correct rayleigh observations in documentation

actions

output actions in single file using file-append in dump

shorten burgers

the nb of transitions can be shortened to 250k or 300k