nathanaelbosch / fenrir.jl

Physics-Enhanced Regression for Initial Value Problems

This package is deprecated and will not be maintained anymore, as the functionality of Fenrir.jl is now implemented in ProbNumDiffEq.jl. Just use ProbNumDiffEq.jl directy.

Fenrir: Physics-Enhanced Regression for IVPs

This package exports a single function, fenrir_nll:

    fenrir_nll(prob::ODEProblem, data::NamedTuple{(:t, :u)}, observation_noise_var::Real,
        diffusion_var::Union{Real,Vector{<:Real}};
        adaptive=false, dt=false,  proj=I, order=3::Int, tstops=[])

Compute the "Fenrir" approximate negative log-likelihood (NLL) of the data.

To see the full docstring, check out the documentation.

Minimal example: Parameter likelihood on FitzHugh-Nagumo

using ProbNumDiffEq, Plots, LinearAlgebra, Fenrir

# Define problem:
function f(du, u, p, t)
    a, b, c = p
    du[1] = c*(u[1] - u[1]^3/3 + u[2])
    du[2] = -(1/c)*(u[1] -  a - b*u[2])
end
u0 = [-1.0, 1.0]
tspan = (0.0, 20.0)
p = (0.2, 0.2, 3.0)
prob = ODEProblem(f, u0, tspan, p)

# Generate data:
true_sol = solve(prob, EK1())
times = 1:0.1:20
odedata = [u + 0.1*randn(size(u)) for u in true_sol(times).u.μ]
scatter(times, ProbNumDiffEq.stack(odedata), markersize=2, markerstrokewidth=0.1,
        color=1, label=["Data" ""])

# With the wrong parameters:
pwrong = (0.1, 0.1, 2.0)
solwrong = solve(remake(prob, p=pwrong), EK1(smooth=false), dense=false);
plot!(solwrong, color=2, label=["Wrong solution" ""])

# Fenrir:
data = (t=times, u=odedata);
σ² = 1e-1
κ² = 1e30
nll, ts, states = fenrir_nll(remake(prob, p=pwrong), data, σ², κ²)

means = ProbNumDiffEq.stack([x.μ for x in states]);
stddevs = ProbNumDiffEq.stack([sqrt.(diag(x.Σ)) for x in states]);

plot!(ts, means, ribbon=2stddevs,
      marker=:o, markersize=1, markerstrokewidth=0.1,
      color=3, fillalpha=0.1, label=["Fenrir interpolation" ""])

println("Negative log-likelihood: $nll")

Prints: Negative log-likelihood: 5849.3096741464615

You can use this NLL it as any other NLL: Optimize it to compute maximum-likelihood estimates or MAPs, or plug it into MCMC to sample from the posterior. In our paper we compute MLEs by pairing Fenrir with Optimization.jl and ForwardDiff.jl. Check out the documentation for more details on how to do this.

Reference

This method has been developed in the paper "Fenrir: Physics-Enhanced Regression for Initial Value Problems" by Tronarp et al, published at ICML 2022 (link).

@InProceedings{pmlr-v162-tronarp22a,
  title = 	 {Fenrir: Physics-Enhanced Regression for Initial Value Problems},
  author =       {Tronarp, Filip and Bosch, Nathanael and Hennig, Philipp},
  booktitle = 	 {Proceedings of the 39th International Conference on Machine Learning},
  pages = 	 {21776--21794},
  year = 	 {2022},
  editor = 	 {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
  volume = 	 {162},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {17--23 Jul},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v162/tronarp22a/tronarp22a.pdf},
  url = 	 {https://proceedings.mlr.press/v162/tronarp22a.html}
}

fenrir.jl's Issues

Add some way to specify other `ProbNumDiffEq.jl` solvers

Currently the solver is hard-coded in fenrir_nll here:

Fenrir.jl/src/likelihood.jl

Line 51 in 85202f4

alg = EK1(order=order, diffusionmodel=diffmodel, smooth=true)

This was mostly the case as we only used this one solver in the paper, and I wanted to reduce the complexity for using the method. But in times of a fast EK0, probabilistic exponential integrators, and priors, it would probably be best to let the user supply any ProbNumDiffEq.jl solver they want.

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

`compute_nll_and_update!` (partly) assumes noiseless measurements

The main issue is this:

Fenrir.jl/src/likelihood.jl

Line 168 in c5f4e2d

ProbNumDiffEq.update!(xout, x, msmnt, H, K1, K2, C_DxD, C_dxd, C_dxd)

ProbNumDiffEq.update! is a Joseph form update, and thus assumes that the measurement noise covariance is zero. That's not (necessarily) the case in Fenrir.jl! So, another function should be implemented (in Fenrir.jl or in ProbNumDiffEq.jl) and used instead.

Printing the estimated values of parameters

Excellent work! Fantastic integration with SciML and DifferentialEquations.jl. . It was a long time coming but it is here.

Where do you find the estimated parameters in the output? You should automatically print them to compare with the true p = (0.2, 0.2, 3.0) in the README example perhaps with a standard deviation to show how close the estimated parameters are from the true p.

Also, Fenrir should be applied to the DAE example in ProbNumDiffEq