tanhevg / gpabc.jl Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
I believe it's currently not possible to simulate trajectories from the model in parallel using multiple cores. It would be nice if this could be supported. In the meantime how hard would it be to fork the repo and add this function myself?
It seems like multi-processing is quite easy to do in Julia, basically something like this:
results = pmap(my_function, collect(1:N))
Error is like this, and I can't build GpABC. Do you guys have any suggestions?
PS: Julia version is v1.4.2; XX in C:\Users\XX\.julia
represents my user name and it's chinese.
(@v1.4) pkg> add GpABC
Cloning default registries into `C:\Users\XX\.julia`
Cloning registry from "https://github.com/JuliaRegistries/General.git"
Added registry `General` to `C:\Users\XX\.julia\registries\General`
Resolving package versions...
Installed Missings ───────────────────── v0.4.5
Installed Compat ─────────────────────── v3.27.0
Installed RecursiveArrayTools ────────── v2.11.3
Installed NLsolve ────────────────────── v4.5.1
Installed FiniteDiff ─────────────────── v2.8.0
Installed MacroTools ─────────────────── v0.5.6
Installed ArnoldiMethod ──────────────── v0.1.0
Installed GpABC ──────────────────────── v0.1.0
Installed OpenSpecFun_jll ────────────── v0.5.3+4
Installed FFTW_jll ───────────────────── v3.3.9+7
Installed LazyArtifacts ──────────────── v1.3.0
Installed NLSolversBase ──────────────── v7.5.0
Installed Inflate ────────────────────── v0.1.2
Installed AbstractFFTs ───────────────── v1.0.1
Installed FixedPointNumbers ──────────── v0.8.4
Installed FastClosures ───────────────── v0.3.2
Installed Reexport ───────────────────── v1.0.0
Installed PlotUtils ──────────────────── v1.0.10
Installed TableTraits ────────────────── v1.0.1
Installed Preferences ────────────────── v1.2.1
Installed CompilerSupportLibraries_jll ─ v0.3.4+0
Installed StatsBase ──────────────────── v0.33.5
Installed OrderedCollections ─────────── v1.4.0
Installed LightGraphs ────────────────── v1.3.5
Installed SIMDPirates ────────────────── v0.8.26
Installed StaticArrays ───────────────── v0.12.5
Installed CommonSubexpressions ───────── v0.3.0
Installed Rmath_jll ──────────────────── v0.2.2+2
Installed VectorizationBase ──────────── v0.12.33
Installed KernelDensity ──────────────── v0.5.1
Installed AxisAlgorithms ─────────────── v1.0.0
Installed Rmath ──────────────────────── v0.6.1
Installed ExponentialUtilities ───────── v1.8.2
Installed MuladdMacro ────────────────── v0.2.2
Installed VertexSafeGraphs ───────────── v0.1.2
Installed NaNMath ────────────────────── v0.3.5
Installed IteratorInterfaceExtensions ── v1.0.0
Installed FunctionWrappers ───────────── v1.1.2
Installed DocStringExtensions ────────── v0.8.4
Installed OffsetArrays ───────────────── v1.6.2
Installed NonlinearSolve ─────────────── v0.3.8
Installed LabelledArrays ─────────────── v1.6.0
Installed ZygoteRules ────────────────── v0.2.1
Installed Optim ──────────────────────── v0.22.0
Installed CpuId ──────────────────────── v0.2.2
Installed DataValueInterfaces ────────── v1.0.0
Installed OrdinaryDiffEq ─────────────── v5.52.4
Installed Adapt ──────────────────────── v3.3.0
Installed Colors ─────────────────────── v0.12.7
Installed IterativeSolvers ───────────── v0.9.0
Installed WoodburyMatrices ───────────── v0.5.3
Installed ArrayInterface ─────────────── v2.14.17
Installed SparseDiffTools ────────────── v1.13.2
Installed JLLWrappers ────────────────── v1.3.0
Installed RecipesBase ────────────────── v1.1.1
Installed Interpolations ─────────────── v0.12.10
Installed Ratios ─────────────────────── v0.4.0
Installed CommonSolve ────────────────── v0.2.0
Installed Distributions ──────────────── v0.23.8
Installed LoopVectorization ──────────── v0.8.26
Installed Requires ───────────────────── v1.1.3
Installed Artifacts ──────────────────── v1.3.0
Installed MKL_jll ────────────────────── v2021.1.1+1
Installed PositiveFactorizations ─────── v0.2.4
Installed Distances ──────────────────── v0.9.2
Installed Tables ─────────────────────── v1.4.2
Installed SciMLBase ──────────────────── v1.12.1
Installed ChainRulesCore ─────────────── v0.9.37
Installed SortingAlgorithms ──────────── v0.3.1
Installed SimpleTraits ───────────────── v0.9.3
Installed TreeViews ──────────────────── v0.3.0
Installed UnPack ─────────────────────── v1.0.2
Installed DiffResults ────────────────── v1.0.3
Installed DiffRules ──────────────────── v1.0.2
Installed RecursiveFactorization ─────── v0.1.8
Installed DiffEqBase ─────────────────── v6.59.0
Installed QuadGK ─────────────────────── v2.4.1
Installed Setfield ───────────────────── v0.7.0
Installed Calculus ───────────────────── v0.5.1
Installed DiffEqDiffTools ────────────── v1.7.0
Installed StatsFuns ──────────────────── v0.9.7
Installed FillArrays ─────────────────── v0.8.14
Installed Parameters ─────────────────── v0.12.2
Installed TOML ───────────────────────── v1.0.3
Installed ForwardDiff ────────────────── v0.10.18
Installed DataAPI ────────────────────── v1.6.0
Installed IntelOpenMP_jll ────────────── v2018.0.3+2
Installed FFTW ───────────────────────── v1.3.2
Installed ColorTypes ─────────────────── v0.10.12
Installed LineSearches ───────────────── v7.1.1
Installed SLEEFPirates ───────────────── v0.5.5
Installed DataStructures ─────────────── v0.18.9
Installed ConstructionBase ───────────── v1.1.0
Installed PDMats ─────────────────────── v0.10.1
Installed SpecialFunctions ───────────── v0.10.3
Installed ColorSchemes ───────────────── v3.12.0
Updating `C:\Users\XX\.julia\environments\v1.4\Project.toml`
[e850a1a4] + GpABC v0.1.0
Updating `C:\Users\XX\.julia\environments\v1.4\Manifest.toml`
[621f4979] + AbstractFFTs v1.0.1
[79e6a3ab] + Adapt v3.3.0
[ec485272] + ArnoldiMethod v0.1.0
[4fba245c] + ArrayInterface v2.14.17
[56f22d72] + Artifacts v1.3.0
[13072b0f] + AxisAlgorithms v1.0.0
[49dc2e85] + Calculus v0.5.1
[d360d2e6] + ChainRulesCore v0.9.37
[35d6a980] + ColorSchemes v3.12.0
[3da002f7] + ColorTypes v0.10.12
[5ae59095] + Colors v0.12.7
[38540f10] + CommonSolve v0.2.0
[bbf7d656] + CommonSubexpressions v0.3.0
[34da2185] + Compat v3.27.0
[e66e0078] + CompilerSupportLibraries_jll v0.3.4+0
[187b0558] + ConstructionBase v1.1.0
[adafc99b] + CpuId v0.2.2
[9a962f9c] + DataAPI v1.6.0
[864edb3b] + DataStructures v0.18.9
[e2d170a0] + DataValueInterfaces v1.0.0
[2b5f629d] + DiffEqBase v6.59.0
[01453d9d] + DiffEqDiffTools v1.7.0
[163ba53b] + DiffResults v1.0.3
[b552c78f] + DiffRules v1.0.2
[b4f34e82] + Distances v0.9.2
[31c24e10] + Distributions v0.23.8
[ffbed154] + DocStringExtensions v0.8.4
[d4d017d3] + ExponentialUtilities v1.8.2
[7a1cc6ca] + FFTW v1.3.2
[f5851436] + FFTW_jll v3.3.9+7
[9aa1b823] + FastClosures v0.3.2
[1a297f60] + FillArrays v0.8.14
[6a86dc24] + FiniteDiff v2.8.0
[53c48c17] + FixedPointNumbers v0.8.4
[f6369f11] + ForwardDiff v0.10.18
[069b7b12] + FunctionWrappers v1.1.2
[e850a1a4] + GpABC v0.1.0
[d25df0c9] + Inflate v0.1.2
[1d5cc7b8] + IntelOpenMP_jll v2018.0.3+2
[a98d9a8b] + Interpolations v0.12.10
[42fd0dbc] + IterativeSolvers v0.9.0
[82899510] + IteratorInterfaceExtensions v1.0.0
[692b3bcd] + JLLWrappers v1.3.0
[5ab0869b] + KernelDensity v0.5.1
[2ee39098] + LabelledArrays v1.6.0
[4af54fe1] + LazyArtifacts v1.3.0
[093fc24a] + LightGraphs v1.3.5
[d3d80556] + LineSearches v7.1.1
[bdcacae8] + LoopVectorization v0.8.26
[856f044c] + MKL_jll v2021.1.1+1
[1914dd2f] + MacroTools v0.5.6
[e1d29d7a] + Missings v0.4.5
[46d2c3a1] + MuladdMacro v0.2.2
[d41bc354] + NLSolversBase v7.5.0
[2774e3e8] + NLsolve v4.5.1
[77ba4419] + NaNMath v0.3.5
[8913a72c] + NonlinearSolve v0.3.8
[6fe1bfb0] + OffsetArrays v1.6.2
[efe28fd5] + OpenSpecFun_jll v0.5.3+4
[429524aa] + Optim v0.22.0
[bac558e1] + OrderedCollections v1.4.0
[1dea7af3] + OrdinaryDiffEq v5.52.4
[90014a1f] + PDMats v0.10.1
[d96e819e] + Parameters v0.12.2
[995b91a9] + PlotUtils v1.0.10
[85a6dd25] + PositiveFactorizations v0.2.4
[21216c6a] + Preferences v1.2.1
[1fd47b50] + QuadGK v2.4.1
[c84ed2f1] + Ratios v0.4.0
[3cdcf5f2] + RecipesBase v1.1.1
[731186ca] + RecursiveArrayTools v2.11.3
[f2c3362d] + RecursiveFactorization v0.1.8
[189a3867] + Reexport v1.0.0
[ae029012] + Requires v1.1.3
[79098fc4] + Rmath v0.6.1
[f50d1b31] + Rmath_jll v0.2.2+2
[21efa798] + SIMDPirates v0.8.26
[476501e8] + SLEEFPirates v0.5.5
[0bca4576] + SciMLBase v1.12.1
[efcf1570] + Setfield v0.7.0
[699a6c99] + SimpleTraits v0.9.3
[a2af1166] + SortingAlgorithms v0.3.1
[47a9eef4] + SparseDiffTools v1.13.2
[276daf66] + SpecialFunctions v0.10.3
[90137ffa] + StaticArrays v0.12.5
[2913bbd2] + StatsBase v0.33.5
[4c63d2b9] + StatsFuns v0.9.7
[fa267f1f] + TOML v1.0.3
[3783bdb8] + TableTraits v1.0.1
[bd369af6] + Tables v1.4.2
[a2a6695c] + TreeViews v0.3.0
[3a884ed6] + UnPack v1.0.2
[3d5dd08c] + VectorizationBase v0.12.33
[19fa3120] + VertexSafeGraphs v0.1.2
[efce3f68] + WoodburyMatrices v0.5.3
[700de1a5] + ZygoteRules v0.2.1
[2a0f44e3] + Base64
[ade2ca70] + Dates
[8bb1440f] + DelimitedFiles
[8ba89e20] + Distributed
[9fa8497b] + Future
[b77e0a4c] + InteractiveUtils
[76f85450] + LibGit2
[8f399da3] + Libdl
[37e2e46d] + LinearAlgebra
[56ddb016] + Logging
[d6f4376e] + Markdown
[a63ad114] + Mmap
[44cfe95a] + Pkg
[de0858da] + Printf
[3fa0cd96] + REPL
[9a3f8284] + Random
[ea8e919c] + SHA
[9e88b42a] + Serialization
[1a1011a3] + SharedArrays
[6462fe0b] + Sockets
[2f01184e] + SparseArrays
[10745b16] + Statistics
[4607b0f0] + SuiteSparse
[8dfed614] + Test
[cf7118a7] + UUIDs
[4ec0a83e] + Unicode
Building FFTW ────────→ `C:\Users\XX\.julia\packages\FFTW\G3lSO\deps\build.log`
Building SLEEFPirates → `C:\Users\XX\.julia\packages\SLEEFPirates\jGsib\deps\build.log`
┌ **Error: Error building `SLEEFPirates`:
│ ERROR: LoadError: LoadError: could not load library "C:\Users\XX\AppData\Local\Programs\Julia\Julia-1.4.2\bin\LLVM.dll"
│ The specified module could not be found.
│ Stacktrace:
│ [1] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109
[inlined] (repeats 2 times)
│ [2] top-level scope at C:\Users\XX\.julia\packages\VectorizationBase\so9eG\src\cpu_info_x86_llvm.jl:5
│ [3] include(::String) at C:\Users\XX\.julia\packages\VectorizationBase\so9eG\src\VectorizationBase.jl:1
│ [4] top-level scope at C:\Users\XX\.julia\packages\VectorizationBase\so9eG\src\VectorizationBase.jl:175
│ [5] top-level scope at none:2
│ [6] eval at .\boot.jl:331 [inlined]
│ in expression starting at C:\Users\XX\.julia\packages\VectorizationBase\so9eG\src\cpu_info_x86_llvm.jl:5
│ in expression starting at C:\Users\XX\.julia\packages\VectorizationBase\so9eG\src\VectorizationBase.jl:173
│ ERROR: LoadError: Failed to precompile VectorizationBase [3d5dd08c-fd9d-11e8-17fa-ed2836048c2f] to C:\Users\XX\.julia\compiled\v1.4\VectorizationBase\Dto5m_icrZj.ji.
│ Stacktrace:
│ [1] top-level scope at none:5
│ in expression starting at C:\Users\XX\.julia\packages\SLEEFPirates\jGsib\deps\build.jl:1
└ @ Pkg.Operations D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Operations.jl:899**
(@v1.4) pkg>
M = [result.population[1]';result.population[2]';result.population[3]']
theta1 = result.population[1]'
theta2 = result.population[2]'
theta3 = result.population[3]'
fig2, axes2 = subplots(3,3)
axes2[1][:hist](M[:,1]',50)
axes2[1][:set_ylabel]("ϴ1")
axes2[2][:scatter](theta1[:,1],theta1[:,2],c="green",alpha=0.5,s=4)
axes2[2][:scatter](theta2[:,1],theta2[:,2],c="blue",alpha=0.5,s=4)
axes2[2][:scatter](theta3[:,1],theta3[:,2],c="red",alpha=0.5,s=4)
axes2[2][:set_ylabel]("ϴ2")
axes2[2][:grid]("on")
axes2[1][:spines]["top"][:set_color]("none")
axes2[1][:spines]["right"][:set_color]("none")
axes2[2][:spines]["top"][:set_color]("none")
axes2[2][:spines]["right"][:set_color]("none")
axes2[3][:spines]["top"][:set_color]("none")
axes2[3][:spines]["right"][:set_color]("none")
axes2[5][:spines]["top"][:set_color]("none")
axes2[5][:spines]["right"][:set_color]("none")
axes2[6][:spines]["top"][:set_color]("none")
axes2[6][:spines]["right"][:set_color]("none")
axes2[9][:spines]["top"][:set_color]("none")
axes2[9][:spines]["right"][:set_color]("none")
axes2[1][:set_xticklabels]("")
axes2[2][:set_xticklabels]("")
axes2[5][:set_xticklabels]("")
axes2[6][:set_yticklabels]("")
axes2[5][:hist](M[:,2]',50)
axes2[3][:scatter](theta1[:,1],theta1[:,3],c="green",alpha=0.5,s=4)
axes2[3][:scatter](theta2[:,1],theta2[:,3],c="blue",alpha=0.5,s=4)
axes2[3][:scatter](theta3[:,1],theta3[:,3],c="red",alpha=0.5,s=4)
axes2[3][:set_xlabel]("ϴ1")
axes2[3][:set_ylabel]("ϴ3")
axes2[3][:grid]("on")
axes2[4][:axis]("off")
axes2[7][:axis]("off")
axes2[8][:axis]("off")
axes2[6][:scatter](theta1[:,2],theta1[:,3],c="green",alpha=0.5,s=4)
axes2[6][:scatter](theta2[:,2],theta2[:,3],c="blue",alpha=0.5,s=4)
axes2[6][:scatter](theta3[:,2],theta3[:,3],c="red",alpha=0.5,s=4)
axes2[6][:set_xlim]([0.,2.])
axes2[6][:set_ylim]([10.,20.])
axes2[6][:set_xlabel]("ϴ2")
axes2[6][:grid]("on")
axes2[9][:hist](M[:,3]',50)
axes2[9][:set_xlabel]("ϴ3")
Installation with "add GpABC" is failing. In particular the dependency OrdinaryDiffEq isn't being installed properly. When running "using GpABC", I get the following error message:
Julia version: v"1.7.1"
`[ Info: Precompiling GpABC [e850a1a4-d859-11e8-3d54-a195e6d045d3]
WARNING: Method definition setindex(AbstractArray{T, 2} where T, Any, Int64, Int64) in module ArrayInterface at /home/username/.julia/packages/ArrayInterface/R0AhD/src/ArrayInterface.jl:199 overwritten in module ArrayInterfaceCore at /home/username/.julia/packages/ArrayInterfaceCore/edbd7/src/ArrayInterfaceCore.jl:115.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition setindex(AbstractArray{T, 1} where T, Any, Int64) in module ArrayInterface at /home/username/.julia/packages/ArrayInterface/R0AhD/src/ArrayInterface.jl:194 overwritten in module ArrayInterfaceCore at /home/username/.julia/packages/ArrayInterfaceCore/edbd7/src/ArrayInterfaceCore.jl:110.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition setindex(AbstractArray{T, N} where N where T, Any, Any...) in module ArrayInterface at /home/username/.julia/packages/ArrayInterface/R0AhD/src/ArrayInterface.jl:188 overwritten in module ArrayInterfaceCore at /home/username/.julia/packages/ArrayInterfaceCore/edbd7/src/ArrayInterfaceCore.jl:104.
** incremental compilation may be fatally broken for this module **
ERROR: LoadError: UndefVarError: allowscomplex not defined
Stacktrace:
[1] check_prob_alg_pairing(prob::SciMLBase.ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, SciMLBase.NullParameters, SciMLBase.ODEFunction{true, OrdinaryDiffEq.var"#lorenz#526", LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, alg::OrdinaryDiffEq.BS3{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False})
@ DiffEqBase ~/.julia/packages/DiffEqBase/fI1KN/src/solve.jl:933
[2] #solve_up#30
@ ~/.julia/packages/DiffEqBase/fI1KN/src/solve.jl:685 [inlined]
[3] solve_up
@ ~/.julia/packages/DiffEqBase/fI1KN/src/solve.jl:675 [inlined]
[4] #solve#29
@ ~/.julia/packages/DiffEqBase/fI1KN/src/solve.jl:670 [inlined]
[5] solve(prob::SciMLBase.ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, SciMLBase.NullParameters, SciMLBase.ODEFunction{true, OrdinaryDiffEq.var"#lorenz#526", LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, args::OrdinaryDiffEq.BS3{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False})
@ DiffEqBase ~/.julia/packages/DiffEqBase/fI1KN/src/solve.jl:665
[6] top-level scope
@ ~/.julia/packages/OrdinaryDiffEq/1Xl3n/src/OrdinaryDiffEq.jl:195
[7] include
@ ./Base.jl:418 [inlined]
[8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
@ Base ./loading.jl:1318
[9] top-level scope
@ none:1
[10] eval
@ ./boot.jl:373 [inlined]
[11] eval(x::Expr)
@ Base.MainInclude ./client.jl:453
[12] top-level scope
@ none:1
in expression starting at /home/username/.julia/packages/OrdinaryDiffEq/1Xl3n/src/OrdinaryDiffEq.jl:1
ERROR: LoadError: Failed to precompile OrdinaryDiffEq [1dea7af3-3e70-54e6-95c3-0bf5283fa5ed] to /home/username/.julia/compiled/v1.7/OrdinaryDiffEq/jl_YKTe66.
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
@ Base ./loading.jl:1466
[3] compilecache(pkg::Base.PkgId, path::String)
@ Base ./loading.jl:1410
[4] _require(pkg::Base.PkgId)
@ Base ./loading.jl:1120
[5] require(uuidkey::Base.PkgId)
@ Base ./loading.jl:1013
[6] require(into::Module, mod::Symbol)
@ Base ./loading.jl:997
[7] include
@ ./Base.jl:418 [inlined]
[8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
@ Base ./loading.jl:1318
[9] top-level scope
@ none:1
[10] eval
@ ./boot.jl:373 [inlined]
[11] eval(x::Expr)
@ Base.MainInclude ./client.jl:453
[12] top-level scope
@ none:1
in expression starting at /home/username/.julia/packages/GpABC/o0EN1/src/GpABC.jl:1
ERROR: Failed to precompile GpABC [e850a1a4-d859-11e8-3d54-a195e6d045d3] to /home/username/.julia/compiled/v1.7/GpABC/jl_OY2Vwy.
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
@ Base ./loading.jl:1466
[3] compilecache(pkg::Base.PkgId, path::String)
@ Base ./loading.jl:1410
[4] _require(pkg::Base.PkgId)
@ Base ./loading.jl:1120
[5] require(uuidkey::Base.PkgId)
@ Base ./loading.jl:1013
[6] require(into::Module, mod::Symbol)
@ Base ./loading.jl:997
`
Hi
I am trying to install GpABC. Unfortunately, there is a version compatibility issue with KernelDensity.jl (GpABC requires 0.5.0-0.5.1), but I have a requirement for 0.6.3 from other packages. What's the best approach to solve this? Thanks
Illustrated in Figure 9 in this review article by Lintusaari et al means we can use rejected particles in the posterior with some simple post processing.
The basic approach is to fit a model (linear regression) that predicts the summary statistic from the parameters and use it to project rejected particles into the accepted region.
Rather than setting the threshold beforehand it is possible to evaluate distances (via simulation or emulation) using a fixed number of particles.
The threshold is then chosen to be a quantile of the empirical distribution of the distances across all these particles.
E.g. if we simulate N=10^6 particles we could select a threshold that would accept 1% of them.
ODEs and SDEs
Simulated ABC-SMC
Emulated ABC-SMC with repetitive training
Consider the following use case:
Currently, since the keep_all
summary statistic reshapes everything into 1D array, we'd have to reshape everything back into a matrix in the distance function, before taking the trace norm. The limitation of all summary statistics having to produce a 1D array has to be removed.
Hi! I have run a SimulatedABCSMC inference using a given threshold schedule, for instance
threshold_schedule = [3.0, 2.0, 1.0, 0.5, 0.2]
Now, after checking my posterior distributions, I believe I could get better results by continuing the inference process with smaller thresholds, say
threshold_schedule = [0.1, 0.05]
Is there a way to continue the inference from the latest posterior distributions using only the two new thresholds [0.1, 0.05], without restarting the whole process from the beginning using [3.0, 2.0, 1.0, 0.5, 0.2, 0.1, 0.05] ?
It would save me a lot of time.
Thanks for your help!
Best,
Simon
I tried to run the first In[1]--In[4] of the example.
If I run only In[1] and In[2] and then try to run simulator_function() it leads to an
ERROR: MethodError: Cannot
convertan object of type ODESolution{Float64, 2, Vector{Vector{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Vector{Float64}}}, ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Float64}, ODEFunction{true, var"#ODE_3GeneReg#7", LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, RK4, OrdinaryDiffEq.InterpolationData{ODEFunction{true, var"#ODE_3GeneReg#7", LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Vector{Float64}}, Vector{Float64}, Vector{Vector{Vector{Float64}}}, OrdinaryDiffEq.RK4Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}}}, DiffEqBase.DEStats} to an object of type Float64 Closest candidates are: convert(::Type{T}, ::Static.StaticFloat64{N}) where {N, T<:AbstractFloat} at ~/.julia/packages/Static/pkxBE/src/float.jl:26 convert(::Type{T}, ::LLVM.GenericValue, ::LLVM.LLVMType) where T<:AbstractFloat at ~/.julia/packages/LLVM/YSJ2s/src/execution.jl:39 convert(::Type{T}, ::LLVM.ConstantFP) where T<:AbstractFloat at ~/.julia/packages/LLVM/YSJ2s/src/core/value/constant.jl:111 ...
I made the simulator_function() run only, when I changed the function GeneReg() to return Obs only (without the type).
Then I tried to run In[3] and In[4]. Nevertheless, the SimulatedABCRejection() fails.
Here is the exact code I used:
`
using GpABC
using OrdinaryDiffEq
using Distances
using Distributions
using Plots
using StatsBase
using Printf
pyplot()
true_params = [2.0, 1.0, 15.0, 1.0, 1.0, 1.0, 100.0, 1.0, 1.0, 1.0] # nominal parameter values
priors = [Uniform(0.2, 5.), Uniform(0.2, 5.), Uniform(10., 20.),
Uniform(0.2, 2.), Uniform(0.2, 2.), Uniform(0.2, 2.),
Uniform(75., 125.), Uniform(0.2, 2.), Uniform(0.2, 2.),
Uniform(0.2, 2.)]
param_indices = [1, 3, 9] #indices of the parameters we want to estimate
priors = priors[param_indices]
Tspan = (0.0, 10.0)
x0 = [3.0, 2.0, 1.0]
solver = RK4()
saveat = 0.1
GeneReg = function(params::AbstractArray{Float64,1},
Tspan::Tuple{Float64,Float64}, x0::AbstractArray{Float64,1},
solver::OrdinaryDiffEq.OrdinaryDiffEqAlgorithm, saveat::Float64)
if size(params,1) != 10
throw(ArgumentError("GeneReg needs 10 parameters, $(size(params,1)) were provided"))
end
function ODE_3GeneReg(dx, x, par, t)
dx[1] = par[1]/(1+par[7]*x[3]) - par[4]*x[1]
dx[2] = par[2]*par[8]*x[1]/(1+par[8]*x[1]) - par[5]*x[2]
dx[3] = par[3]*par[9]*x[1]*par[10]*x[2]./(1+par[9]*x[1])./(1+par[10]*x[2]) - par[6]*x[3]
end
prob = ODEProblem(ODE_3GeneReg, x0 ,Tspan, params)
Obs = solve(prob, solver, saveat=saveat)
return Obs
######## #instead of: return Array{Float64, 2}(Obs)
end
function simulator_function(var_params)
params = copy(true_params)
params[param_indices] .= var_params
GeneReg(params, Tspan, x0, solver, saveat)
end
simulator_function([2.0, 15.0, 1.0])
reference_data = simulator_function(true_params[param_indices])
plot(reference_data', xlabel="t", ylabel="C(t)", linewidth=2, labels=["u1(t)" "u2(t)" "u3(t)"])
n_particles = 2000
threshold = 1.0
sim_result = SimulatedABCRejection(reference_data, simulator_function, priors, threshold, n_particles;
max_iter=convert(Int, 2e6),
write_progress=false)
plot(sim_result)
`
From the SimulatedABCRejection() I get an
ERROR: DimensionMismatch("parent has 101 elements, which is incompatible with size (303,)") Stacktrace: [1] _throw_dmrs(n::Int64, str::String, dims::Tuple{Int64}) @ Base ./reshapedarray.jl:181 [2] _reshape @ ./reshapedarray.jl:176 [inlined] [3] reshape @ ./reshapedarray.jl:112 [inlined] [4] reshape @ ./reshapedarray.jl:116 [inlined] [5] keep_all_summary_statistic(data::ODESolution{Float64, 2, Vector{Vector{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Vector{Float64}}}, ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Float64}, ODEFunction{true, var"#ODE_3GeneReg#7", LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, RK4, OrdinaryDiffEq.InterpolationData{ODEFunction{true, var"#ODE_3GeneReg#7", LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Vector{Float64}}, Vector{Float64}, Vector{Vector{Vector{Float64}}}, OrdinaryDiffEq.RK4Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}}}, DiffEqBase.DEStats}) @ GpABC ~/.julia/packages/GpABC/o0EN1/src/abc/summary_stats.jl:107 [6] SimulatedABCRejection(reference_data::ODESolution{Float64, 2, Vector{Vector{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Vector{Float64}}}, ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Float64}, ODEFunction{true, var"#ODE_3GeneReg#7", LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, RK4, OrdinaryDiffEq.InterpolationData{ODEFunction{true, var"#ODE_3GeneReg#7", LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Vector{Float64}}, Vector{Float64}, Vector{Vector{Vector{Float64}}}, OrdinaryDiffEq.RK4Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}}}, DiffEqBase.DEStats}, simulator_function::Function, priors::Vector{Uniform{Float64}}, threshold::Float64, n_particles::Int64; summary_statistic::String, distance_function::Euclidean, max_iter::Int64, kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:write_progress,), Tuple{Bool}}}) @ GpABC ~/.julia/packages/GpABC/o0EN1/src/abc/simulation.jl:54 [7] top-level scope @ ~/GIT/GIT_Schistoxpkg.jl_1.2.15/src/ABCexample.jl:76
Maybe smt has been changed in julia or even in the SimulatedABCRejection() since the last example update (June 2020)?
I would like to use GpABC for my project, however runtimes of my simulator are long enough that I would need to run simulations in parallel on an HPC cluster. If I understand the code and the documentation correctly, at the moment simulations are run strictly sequentially, is that correct?
I would be happy to try and give an implementation a go myself. If I understand the code correctly, the easiest way to do it would probably be to allow for simulator functions that take batches of parameter combinations (and let the simulator take care of parallelisation). For that it seems it would be sufficient to modify simulate_distance and everything upwards from that, or is there anything else to change?
Hi there,
There is some compatibility drift on this package, which now means it can't be installed with Flux. Which is a bit of a problem!
Hi,
I just found the GpABC Julia package, and I'd be interested to use it for my research. I am trying to understand the example in the repo, but I cannot figure out how you 1) define the summary statistics and 2) the distance function? Can you give a MWE for these steps? I want to start with rejection ABC and ABC SMC without the emulation initially. Thanks!
Hi! I'm one of the people involved in developing the packages such as AbstractGPs.jl around https://github.com/JuliaGaussianProcesses/, and we were wondering if you'd be interested in working together on how GpABC.jl might build on the AbstractGPs API, instead of having to maintain your own GP implementation? As mentioned on #6, this should immediately let you use sparse GPs, a much wider set of kernels, etc. Let us know if you'd like to discuss this!
From my understanding of ABC, it should result in obtaining posteriors with the distance as close to the reference data as possible. However, when using SimulatedABCRejection(), I got many posteriors with distances far from the reference data if the distance is negative. I am afraid, there is an absolute value missing, so e.g. for a threshold of 5.0 it should only accept simulations with abs(distance) <= 5.0, but currently, all simulations with distance <= 5.0 are accepted (which includes all negative distances regardless of how far they are from zero).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.