rafaqz / modelparameters.jl Goto Github PK

No more default Bulma interface.

I still need to isolate this in a MWE, but I wanted to open the issue for preliminary discussion first.

The time it takes to compile the first solve call for my model seems to skyrocket when using Params + ForwardDiff types. This doesn't seem to be a problem with ModelParameters per-se but rather an issue with inflating type complexity by using autodiff types in all of the model structs containing parameters (I think).

It could be that this is specific to my case, however; maybe it has something to do with the way I wrote my code.

@rafaqz Have you noticed very long compile times with ForwardDiff?

This is a minor issue, but I think it would make more sense to use IntervalSets.Interval for the bounds field of Param (this goes for FieldMetadata as well). As far as I can tell, there is nothing that stops you from doing this at the moment, but since the type of bounds is listed as Tuple in FieldMetadata, I assume there is some code somewhere that assumes it will be a tuple of numbers..?

Anyway, IntervalSets provides nice syntax for closed intervals 0..1, membership checks with ∈, automatically handles casting between number types (e.g. declaring the interval with Int but using it with Float64 values), and of course properly distinguishes between open and closed intervals.

I don't really see any reason not to use it as the standard type for bounds.

current Project.toml

 ..
version = "0.3.3"
 ..
[compat]
 ..
Setfield = "0.6, 0.7"
 ..

suggested edit

 ..
version = "0.3.4"
 ..
[compat]
 ..
Setfield = "0.6, 0.7,0.8"
 ..

(edit, reregister, lather, rinse, ..

Math operations with Param/AbstractNumber do not work with Unitful quantities.

Param(1.0)*1.0u"m"
ERROR: MethodError: *(::Param{Float64, NamedTuple{(:val,), Tuple{Float64}}}, ::Quantity{Float64, 𝐋, Unitful.FreeUnits{(m,), 𝐋, nothing}}) is ambiguous. Candidates:
  *(y::Number, x::Unitful.AbstractQuantity) in Unitful at /home/bgroenke/.julia/packages/Unitful/SUQzL/src/quantities.jl:32
  *(a::AbstractNumbers.AbstractNumber, b::Number) in AbstractNumbers at /home/bgroenke/.julia/packages/AbstractNumbers/jJpxf/src/overloads.jl:586
Possible fix, define
  *(::AbstractNumbers.AbstractNumber, ::Unitful.AbstractQuantity)
Stacktrace:
 [1] top-level scope
   @ REPL[13]:1

I suppose this is really more of an issue with AbstractNumbers, but what is the quickest workaround here?

I have a use case where I would like to be able to efficiently "reparameterize" certain parameters as arbitrary functions of the current state. Suppose for example we have a parameter p and we would like to investigate what happens when p increases linearly with time. Then we reparameterize p as:

p = \beta*t + \alpha

where beta and alpha are the new parameters for optimizing/sampling/whatever.

This can be accomplished in ModelParameters.jl (or more generally), I think, as a transformation between parameter vectors.

However, this requires that the object be reconstructed on each step (each f evaluation in a diffeq model). So update needs to be efficient.

Currently, the immutable update implementation looks like this:

update(x, values::AbstractVector) = update(x, Tuple(values))
function update(x, values)
    newparams = map(params(x), values) do param, value
        Param(NamedTuple{keys(param)}((value, Base.tail(parent(param))...)))
    end
    Flatten.reconstruct(x, newparams, SELECT, IGNORE)
end

This causes two problems:

First, an ambiguous dispatch error when calling update(m::StaticModel, vals::Vector) (although perhaps this is ok since technically m is a table, not a vector..?).

Second, the conversion to a Tuple allocates, slowing things down considerably.

It seems like update works fine on a Vector type:

using Parameters
using ModelParameters
using Flatten
using BenchmarkTools
function _update(x, values)
    newparams = map(ModelParameters.params(x), values) do param, value
        Param(NamedTuple{keys(param)}((value, Base.tail(ModelParameters.parent(param))...)))
    end
    Flatten.reconstruct(x, newparams, ModelParameters.SELECT, ModelParameters.IGNORE)
end
@with_kw struct TestComponent{T}
    p::T = Param(2.0)
end
@with_kw struct TestModel{C,R}
    c1::C = TestComponent()
    c2::C = TestComponent()
    p::R = Param(1.0)
end
m = StaticModel(TestModel())
p = collect(m[:val]).*2
obj = parent(m)
@btime $_update($obj, $p);
# 39.235 ns (1 allocation: 112 bytes)
@btime Flatten.reconstruct(parent($m), flatten(parent($m)));
# 1.641 ns (0 allocations: 0 bytes)
@btime ModelParameters.params(parent($m));
# 1.599 ns (0 allocations: 0 bytes)

... so I'm not sure what the conversion to Tuple is for?

The only minor issue is that one sneaky allocation. I'm not sure what's causing that.

It's also worth noting that StaticModel's constructor also allocates:

@btime $_update($obj, $p) |> StaticModel;
# 3.401 μs (59 allocations: 2.06 KiB)

but I suppose this is expected. It also doesn't matter since I don't think there is any need to rebuild StaticModel inside of the step function. We just need to rebuild the parent object.

Is there any reason why we can't change the implementation of update to allow Vectors? And maybe find that one allocation?

Just thought I'd share this with you. I just tried

Base.@kwdef struct Submodel1{A,B}
    α::A = Param(0.8, bounds=(0.2, 0.9))
    β::B = Param(0.5, bounds=(0.7, 0.4))
end

Base.@kwdef struct Submodel2{Γ}
    γ::Γ = Param(1e-3, bounds=(1e-4, 1e-2))
end

Base.@kwdef struct SubModel3{Λ,X}
    λ::Λ = Param(0.8, bounds=(0.2, 0.9))
    x::X = Submodel2()
end

model = Model((Submodel1(), SubModel3()))

and got the error:

Failed to show value:

UndefVarError: fields not defined

    rows(::ModelParameters.Model)@tables.jl:22
    table_data(::ModelParameters.Model, ::IOContext{IOBuffer})@PlutoRunner.jl:1168
    show_richest(::IOContext{IOBuffer}, ::Any)@PlutoRunner.jl:829
    var"#sprint_withreturned#35"(::IOContext{Base.DevNull}, ::Int64, ::typeof(Main.PlutoRunner.sprint_withreturned), ::Function, ::ModelParameters.Model)@PlutoRunner.jl:779
    format_output_default(::Any, ::Any)@PlutoRunner.jl:676
    var"#format_output#23"(::IOContext{Base.DevNull}, ::typeof(Main.PlutoRunner.format_output), ::Any)@PlutoRunner.jl:693
    formatted_result_of(::Base.UUID, ::Bool, ::Nothing, ::Module)@PlutoRunner.jl:609
    top-level scope@none:1

So there might be an issue with an incompatibility with how Pluto.jl displays things, because

model = Model((Submodel1(), SubModel3())); # suppress output

works!

Just found this package, it looks great! Seems to address a lot of the issues I have been struggling with for the past year w.r.t to parameters and optimization.

I have plans down the road to play around with more complex functions (e.g. neural networks) in my model that may require large vectors or matrices as parameters. How would that fit into the ModelParameters framework?

It seems that, at the moment, Param allows val to be a vector or matrix (though it's not clear that this makes sense since it is supposed to act like a number?), but this gets a bit awkward when collecting the parameter values collect(model[:val]) since it ends up being of type Any if you have a mix of scalar and vector/matrix parameters.

I can imagine that it might be possible to do write a function more clever than collect that takes this into account and flattens the matrix/vector parameters. But I suppose this would introduce some difficulties when reconstructing.

Do you have any ideas at the moment as to how this could be handled?

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

It's kind of weird that AbstractModel implements a Table interface but the Array interface (size, length, getindex, setindex, etc) behaves a like a vector of Params.

It would be really useful for AbstractModel to live up to its aspirations to be a real Table and allow itself to be indexed as such.

This has the potential to address #21 and #34 since it would allow for things like:

m = Model(obj)
m[.!m[:isconstant], :val] .= values

i.e. permitting both row and column indexing.

I will take a crack at it and submit a PR since this would solve a problem I am facing at the moment.

update!(ruleset,df) is fantastic.

Maybe some facilities to update df would be nice (maybe already implemented but I'm not aware of).
Using DataFrame.jl, I do:

df[(df.component .== MyRule) .& (df.fieldname .== :myParam), :val] .= NewValue

It's not disastrous, but the following currently happens on Julia 1.9:

WARNING: Method definition constructorof(Type{var"#s1"} where var"#s1"<:(StaticArraysCore.SArray{Tuple{S}, T, 1, S} where T where S)) in module StaticArraysExt at C:\Users\XXX\.julia\packages\ConstructionBase\VB0co\ext\StaticArraysExt.jl:12 overwritten in module ConstructionBaseExtras at C:\Users\XXX\.julia\packages\ConstructionBaseExtras\cy4E1\src\staticarrays.jl:8.
  ** incremental compilation may be fatally broken for this module **

and a few other warnings like that; all caused by the same underlying issue as far as I can tell.

It's caused by having loaded StaticArrays.jl, which triggers the corresponding extension in ConstructionBase.jl, then ModelParameters.jl loads ConstructionBaseExtras.jl, which overwrites the method already defined by the extension in ConstructionBase.jl.

I can give it a try to fix this, but if someone else is already on it, then I'll just wait of course. I would probably have to rely on Requires.jl to make things compatible for Julia versions <1.9, though?

@rafaqz, would that be fine with you?

Basically the same as the interact interface, but Makie

Impressive package!
Tracking parameters is a problem in complex models. Your solution is elegant and does not require manual tracking.
Fantastic!

I would like to use it for statistics models.
Do you think it is possible for the real Param be <:Real? It would allow combining with Distributions.jl

MWE

using Distributions
using ModelParameters

μ = Param(0.8, bounds=(0.2, 0.9))
σ = Param(0.8, bounds=(1.2, 1.9))
d = Normal(μ, σ) # fails because Normal{T<:Real}

μ isa Real # false

Hi,
I was wondering if you be so kind to upload an example where a table is used to update the model parameters.
Also, not sure from the documentation: is possible to just update a few parameters? because let's say the table is incomplete or I just need to update some of them.

Not sure if recent changes break ModelParameters.jl or Flatten.jl but in any case specifying the Param type for a field raises an error, whereas it was fine previously:

Base.@kwdef mutable struct Example
    d::Param = Param(200.0, bounds=(10.0, 550.0))
end

# Model(Example())
# ERROR: ArgumentError: type does not have a definite number of fields

# Appears to work without specifying Param type
Base.@kwdef mutable struct Example2
    d = Param(200.0, bounds=(10.0, 550.0))
end

# Model(Example2())
# Model with parent object of type: 

# Example2

# Parameters:
# ┌───────────┬───────────┬───────┬───────────────┐
# │ component │ fieldname │   val │        bounds │
# ├───────────┼───────────┼───────┼───────────────┤
# │  Example2 │         d │ 200.0 │ (10.0, 550.0) │
# └───────────┴───────────┴───────┴───────────────┘

Another possible useful thing will be able to do the following (by name)

m = Model(m2, m1, m3)
update!(m, table) # let's say we update parameters
# now I want to work with just one component, then I will need.
m1 = getComponent(m, :component, :m1) # or something similar

not high priority, but I see that this will might be the case for more complicated use cases.

Currently, the :component field is set to the type name in all cases.

It would useful in some cases to allow the user to customize the :component field for user-defined types; i.e. there should be a method:

component(::Type{T}) where T = T.name.wrapper

for which dispatches may be provided to override the component name assigned to type T.

Using the same parameter in multiple places isn't currently possible. It would be good to find a solution to this.

Packing one or more Param() into an Array that also contains some other number and then accessing elements in the array turns the Param()s into Float (or whatever the parent type is). The following code snippet reproduces the behavior:

a = [Param(1.0), 1.0]
typeof(a) #Vector{Number}
a[1] #1.0
typeof(a[1]) #Float64

This is quite unintuitive to me. Is it a bug? When would this be desired?

Is there any way to update the value for a single parameter?

Something like:

x = Param(0, bounds=(0, 100))

x.val = 50

@briochemc @ConnectedSystems ModelParameters.jl and InteractModels.jl are pretty much ready to use now.

I have built them into GrowthMaps.jl and DynamicGrids.jl/DynamicGridsInteract.jl for display of params in the REPL, parameter flattening and for auto generating control interfaces. So it's in a working state. But it would be great to get some early feedback from you both, as you indicated wanting to use this once it's ready.

It has some behaviours I have discussed previously:

• @briochemc units are handled and merged with values as for FieldMetadata.jl - there is a withunits method that will apply the units to the val, range, bounds etc fields. They just don't serialize/deserialze properly to csv yet because of that Unitful.jl issue.
• @ConnectedSystems you should be able to load/save parameters, bounds etc to CSV.

There is a demo of the InteractModel in the tests, just using NamedTuple but that could be any object:
https://github.com/rafaqz/ModelParameters.jl/blob/master/InteractModels/test/runtests.jl

And there are some docs, not sure if they are adequate yet.