nickhnelsen / fourier-neural-mappings

An extension of Fourier Neural Operator to finite-dimensional input and/or output spaces.

Fourier Neural Mappings

Fourier Neural Mappings (FNMs) generalize Fourier Neural Operators (FNOs) by allowing the input space and/or the output space to be finite-dimensional (instead of both being purely infinite-dimensional function spaces as with FNO). This is especially relevant for surrogate modeling tasks in uncertainty quantification, inverse problems, and design optimization, where a finite number of parameters or quantities of interest (QoIs) characterize the inputs and/or outputs.

In particular, FNMs are able to accommodate

• nonlinear functions (V2V): Fourier Neural Networks mapping vectors to vectors (going through a latent function space in between);
• nonlinear functionals (F2V): Fourier Neural Functionals mapping functions to vectors (a.k.a. nonlinear encoders);
• nonlinear decoders (V2F): Fourier Neural Decoders mapping vectors to functions; and of course
• nonlinear operators (F2F): Fourier Neural Operators mapping functions to functions.

In fourier-neural-mappings, the network layers in all four types of mappings above are efficiently implemented (via FFT) in Fourier space in a function-space consistent way. The code defaults to running on GPU, if one is available.

Installation

The command

conda env create -f Project.yml

creates an environment called fno. PyTorch will be installed in this step.

Activate the environment with

conda activate fno

and deactivate with

conda deactivate

The advection-diffusion example requires additional packages to generate and process the data; please refer to the README instructions within that directory for more details.

Data

The data may be downloaded at , which contains three *.zip files:

1. advection_diffusion: train and test sets for KLE dimension 2, 20, 1000.
2. airfoil: deformation map (X,Y) coordinates, pressure field, and control nodes.
3. homogenization: V2V, F2V, V2F, and F2F formats.

The data are stored as PyTorch *.pt files, *.npy arrays, or pickle *.pkl files.

Huang, D. Z., Nelsen, N. H., & Trautner, M. (2024). An operator learning perspective on parameter-to-observable maps [Data set]. CaltechDATA. https://doi.org/10.22002/r5ga1-55d06. Feb. 12, 2024.

References

The main reference that explains the Fourier Neural Mappings framework is the paper ``An operator learning perspective on parameter-to-observable maps'' by Daniel Zhengyu Huang, Nicholas H. Nelsen, and Margaret Trautner. Other relevant references include:

• Fourier Neural Operator for Parametric Partial Differential Equations
• Fourier Neural Operator with Learned Deformations for PDEs on General Geometries
• Learning Homogenization for Elliptic Operators

Citing

If you use fourier-neural-mappings in an academic paper, please cite the main reference ``An operator learning perspective on parameter-to-observable maps'' as follows:

@article{huang2024fnm,
  title={An operator learning perspective on parameter-to-observable maps},
  author={Huang, Daniel Zhengyu and Nelsen, Nicholas H and Trautner, Margaret},
  journal={arXiv preprint arXiv:2402.06031},
  year={2024}
}

Contribute

You are welcome to submit an issue for any questions related to fourier-neural-mappings or to contribute to the code by submitting pull requests.

Acknowledgements

The FNO implementation in fourier-neural-mappings is adapted from the original implementation by Nikola Kovachki and Zongyi Li. The data generation code for the advection-diffusion example was provided by Zachary Morrow. The matplotlib formatting used to produce figures is adapted from the PyApprox package by John Jakeman.

fourier-neural-mappings's Issues

Name change to plural "s'"

Change repo name and classes to "Functionals" instead of "Functional"?

na

na

vec-to-vec model

Add argument to choose whether to use 1D or 2D spatial layers in the latent function space.

Performance of vec_to_*.py models

https://github.com/nickhnelsen/FourierNeuralMappings/blob/95cd6260f1b89b087bbdadfc70abf468257c3c4d/models/vec_to_vec2d.py#L87

^^ The nonlinear processing at the beginning of all the architectures in vec_to_*.py files may not be necessary and hurt accuracy. We should try removing this nonlinearity to see if that helps.

Renaming

FNO2d
FNO2dto3d
FNF2d

Cleanup repo before making public: models directory, shared.py for shared layers and fucntions, etc.

Lose information in FNF?

Does the truncation of information in Fourier Neural Functionals cause issues? It takes a function and integrates it against another function with k_max Fourier modes (so only the first k_maxd Fourier modes are used). But unlike FNO layers, there is no notion of skip connection to retain the truncated modes i.e. use all the information in the input.
An alternative could be to take function $u$, map to the right number of output channels using $MLP(u(x))$, then take the mean of the resulting function to map it to a vector. Then concatenate this with an FNF layer. MLP being nonlinear is crucial here, else only the zero-th Fourier mode would be used.

So we would have
torch.cat(Lu, \int MLP(u(x)) dx),
where L is the FNF map.

Test scripts

After clean PR #7 , scripts have not been tested so there are perhaps some errors in the function import paths. Need to check all.

Renaming

FNF (Fourier neural functional) could also be named Fourier Neural Encoder (FNE) for consistency with FND. Then Fourier Neural Network (FNN) function could be renamed Fourier Neural Function (FNF).

Can decide later.

Possible padding discretization bug?

7abbf99#r128768027

rename repo

fourier-neural-mappings

change readme to reflect this

also add conda yml environment requirement following the readme template here

acknowledgements in repo (original fno code, pyapprox, etc)