Provide insights about latency (TTFX) for Julia packages
Home Page: https://timholy.github.io/SnoopCompile.jl/dev/
License: Other
SnoopCompile observes the Julia compiler, causing it to record the
functions and argument types it's compiling. From these lists of methods,
you can generate lists of precompile directives that may reduce the latency between
loading packages and using them to do "real work."
See the documentation:
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot]")
When I run Pkg.test("somepackage") it doesn't give me any precompile sentence for that somepackage.
This makes it hard to use runtest.jl for generating a precompile file (a file that is already available for most of the packages).
We can include runtests.jl but because some packages have test dependencies it is hard to add those dependencies to the current environment
In my tests, I defined a couple of types to test my package, however, SnoopCompile generates precompile sentences for these.
We should exclude those from appearing in the precompilition files.
aminya/AcuteML.jl@3e9210a
Instead of fixing the existing code, I have written up a script to do the same job. Also, my parse_call function is much simpler than the original one.
I get the following error in the SnoopCompile bot, that I don't see in normal use of the package or CI tests
m = #_#28(detectThresh, overlapThresh, yolo::ObjectDetector.YOLO.yolo, img::DenseArray) in ObjectDetector.YOLO at /home/runner/work/ObjectDetector.jl/ObjectDetector.jl/src/yolo/yolo.jl:542
m.name = Symbol("#_#28")
ERROR: LoadError: UndefVarError: _ not defined
Here's the run:
https://github.com/r3tex/ObjectDetector.jl/pull/32/checks?check_run_id=486306736#step:6:64
It happens on the execution of the last line of the snoopCompile.jl file
https://github.com/r3tex/ObjectDetector.jl/blob/3423c11b130f2d6a4248af23daf952f8917d6416/deps/SnoopCompile/snoopCompile.jl
Could it be related to the way the function it's calling is defined?
https://github.com/r3tex/ObjectDetector.jl/blob/3423c11b130f2d6a4248af23daf952f8917d6416/src/yolo/yolo.jl#L541
JuliaLang/julia#35729 will make it possible to determine the causes of invalidation. It would be nice to organize these reports in a way that makes them more digestible to the user. It should be a fairly simple parsing problem, along with managing the state of the debug flag.
Helping users interpret the cause & cure might be more challenging; the kind of analysis in timholy/Revise.jl#456 (comment) requires knowledge of many different aspect of Julia. Not sure what besides documentation one can do here.
@vtjnash, what do you think should be done with the -- isequal(Method, Method) lines? I'm inclined to drop them if they don't trigger a non--- invalidation, but perhaps I don't appreciate their full significance.
Can we use something like this?
@static if Sys.iswindows()
include("windows/MatLang_precompiles.jl")
_precompile_()
elseif Sys.islinux()
include("linux/MatLang_precompiles.jl")
_precompile_()
endThis is valid, and probably mostly already has implementation support, it's just rejected by our syntax checker.
┌ Warning: failed parse of line:
│ line = "Tuple{typeof(ColorTypes.to_top), Type{C}} where C<:(ColorTypes.Colorant{T, N} where N where T)"
└ @ SnoopCompile ~/.julia/packages/SnoopCompile/JDKqS/src/SnoopCompile.jl:122
julia> Meta.parse("Tuple{typeof(ColorTypes.to_top), Type{C}} where C<:(ColorTypes.Colorant{T, N} where N where T)") |> Meta.show_sexpr
(:where, (:curly, :Tuple, (:call, :typeof, (:., :ColorTypes, (:quote, #QuoteNode
:to_top
))), (:curly, :Type, :C)), (:<:, :C, (:where, (:where, (:curly, (:., :ColorTypes, (:quote, #QuoteNode
:Colorant
)), :T, :N), :N), :T)))SnoopCompile in the Plots.jl repository operates only on one branch which causes changes on different branches to reset the PRs to other branches. Is there some configuration, such that there is 1:1 correspondence between precompile update branches and open PR branches?
We registered the inner packages, but we did not find time to check if the master will work when the inner packages are registered. We had a hack script that solved this issue, but by removing it now we have a broken master that is tagged as 1.6.
I think we should have registered the inner packages first, and then registered the top-level package separately.
Also, I am getting the error in #104, which we were not getting it before the split.
I'm not sure how to simplify this into a minimal example.
Here are two examples in LoopVectorization and VectorizedRNG.jl using SnoopCompile v1.0.2. The former has an unrecognized keyword argument function, and the latter a generated function.
using Pkg
Pkg.add(PackageSpec(url="https://github.com/chriselrod/VectorizationBase.jl"))
Pkg.add(PackageSpec(url="https://github.com/chriselrod/SIMDPirates.jl"))
Pkg.add(PackageSpec(url="https://github.com/chriselrod/SLEEFPirates.jl"))
Pkg.add(PackageSpec(url="https://github.com/chriselrod/LoopVectorization.jl", rev="graph"))
Pkg.add(PackageSpec(url="https://github.com/chriselrod/VectorizedRNG.jl"))If you have ideas on how to come up with a simpler example, please let me know.
On Julia 1.3:
julia> using VectorizedRNG
[ Info: Precompiling VectorizedRNG [33b4df10-0173-11e9-2a0c-851a7edac40e]
ERROR: LoadError: UndefVarError: ##s41#17 not defined
Stacktrace:
[1] _precompile_() at /home/chriselrod/.julia/dev/VectorizedRNG/src/precompile.jl:4
[2] top-level scope at /home/chriselrod/.julia/dev/VectorizedRNG/src/VectorizedRNG.jl:29
[3] include at ./boot.jl:328 [inlined]
[4] include_relative(::Module, ::String) at ./loading.jl:1105
[5] include(::Module, ::String) at ./Base.jl:31
[6] top-level scope at none:2
[7] eval at ./boot.jl:330 [inlined]
[8] eval(::Expr) at ./client.jl:425
[9] top-level scope at ./none:3
in expression starting at /home/chriselrod/.julia/dev/VectorizedRNG/src/VectorizedRNG.jl:29
ERROR: Failed to precompile VectorizedRNG [33b4df10-0173-11e9-2a0c-851a7edac40e] to /home/chriselrod/.julia/compiled/v1.3/VectorizedRNG/wqLDZ_R5WJb.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
[3] _require(::Base.PkgId) at ./loading.jl:1024
[4] require(::Base.PkgId) at ./loading.jl:922
[5] require(::Module, ::Symbol) at ./loading.jl:917
julia> using LoopVectorization
[ Info: Precompiling LoopVectorization [bdcacae8-1622-11e9-2a5c-532679323890]
ERROR: LoadError: UndefVarError: #_vectorloads!##kw not defined
Stacktrace:
[1] _precompile_() at /home/chriselrod/.julia/dev/LoopVectorization/src/precompile.jl:5
[2] top-level scope at /home/chriselrod/.julia/dev/LoopVectorization/src/LoopVectorization.jl:699
[3] include at ./boot.jl:328 [inlined]
[4] include_relative(::Module, ::String) at ./loading.jl:1105
[5] include(::Module, ::String) at ./Base.jl:31
[6] top-level scope at none:2
[7] eval at ./boot.jl:330 [inlined]
[8] eval(::Expr) at ./client.jl:425
[9] top-level scope at ./none:3
in expression starting at /home/chriselrod/.julia/dev/LoopVectorization/src/LoopVectorization.jl:699
ERROR: Failed to precompile LoopVectorization [bdcacae8-1622-11e9-2a5c-532679323890] to /home/chriselrod/.julia/compiled/v1.3/LoopVectorization/4TogI_R5WJb.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
[3] _require(::Base.PkgId) at ./loading.jl:1024
[4] require(::Base.PkgId) at ./loading.jl:922
[5] require(::Module, ::Symbol) at ./loading.jl:917
julia> versioninfo()
Julia Version 1.3.1-pre.0
Commit b42f4ab37f* (2019-11-26 17:58 UTC)
Platform Info:
OS: Linux (x86_64-generic-linux)
CPU: Intel(R) Core(TM) i9-9940X CPU @ 3.30GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-8.0.0 (ORCJIT, skylake)
julia> using Pkg; Pkg.status("VectorizedRNG")
Status `~/.julia/environments/v1.3/Project.toml`
[33b4df10] VectorizedRNG v0.1.0 [`~/.julia/dev/VectorizedRNG`]
julia> Pkg.status("LoopVectorization")
Status `~/.julia/environments/v1.3/Project.toml`
[bdcacae8] LoopVectorization v0.1.0 [`~/.julia/dev/LoopVectorization`]On master:
julia> using VectorizedRNG
julia> using LoopVectorization
julia> versioninfo()
Julia Version 1.4.0-DEV.534
Commit e2cd1ff1e3* (2019-11-30 17:29 UTC)
Platform Info:
OS: Linux (x86_64-generic-linux)
CPU: Intel(R) Core(TM) i9-9940X CPU @ 3.30GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-8.0.1 (ORCJIT, skylake)
julia> using Pkg; Pkg.status("VectorizedRNG")
Status `~/.julia/environments/v1.4/Project.toml`
[33b4df10] VectorizedRNG v0.1.0 [`~/.julia/dev/VectorizedRNG`]
julia> Pkg.status("LoopVectorization")
Status `~/.julia/environments/v1.4/Project.toml`
[bdcacae8] LoopVectorization v0.1.0 [`~/.julia/dev/LoopVectorization`]
julia> dump(VectorizedRNG.var"##s41#17")
VectorizedRNG.var"##s41#17" <: Function
julia> dump(LoopVectorization.var"#_vectorloads!##kw")
LoopVectorization.var"#_vectorloads!##kw" <: FunctionEDIT:
Julia master on a different computer, different commit (535 vs 545):
julia> using VectorizedRNG
[ Info: Precompiling VectorizedRNG [33b4df10-0173-11e9-2a0c-851a7edac40e]
ERROR: LoadError: UndefVarError: ##s40#188 not defined
Stacktrace:
[1] _precompile_()
[2] top-level scope at /home/username/.julia/dev/SIMDPirates/src/SIMDPirates.jl:71
[3] include(::Module, ::String) at ./Base.jl:377
[4] top-level scope at none:2
[5] eval [inlined]
[6] eval(::Expr) at ./client.jl:449
[7] top-level scope
in expression starting at /home/username/.julia/dev/SIMDPirates/src/SIMDPirates.jl:71
ERROR: LoadError: Failed to precompile SIMDPirates [21efa798-c60a-11e8-04d3-e1a92915a26a] to /home/username/.julia/compiled/v1.4/SIMDPirates/jEO2E_tYj51.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1263
[3] _require(::Base.PkgId) at ./loading.jl:1022
[4] require(::Base.PkgId) at ./loading.jl:920
[5] require(::Module, ::Symbol) at ./loading.jl:915
[6] include(::Module, ::String) at ./Base.jl:377
[7] top-level scope at none:2
[8] eval [inlined]
[9] eval(::Expr) at ./client.jl:449
[10] top-level scope
in expression starting at /home/username/.julia/dev/VectorizedRNG/src/VectorizedRNG.jl:3
ERROR: Failed to precompile VectorizedRNG [33b4df10-0173-11e9-2a0c-851a7edac40e] to /home/username/.julia/compiled/v1.4/VectorizedRNG/wqLDZ_tYj51.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String)
[3] _require(::Base.PkgId)
[4] require(::Base.PkgId) (repeats 2 times)
julia> using LoopVectorization
[ Info: Precompiling LoopVectorization [bdcacae8-1622-11e9-2a5c-532679323890]
ERROR: LoadError: UndefVarError: ##s40#188 not defined
Stacktrace:
[1] _precompile_()
[2] top-level scope at /home/username/.julia/dev/SIMDPirates/src/SIMDPirates.jl:71
[3] include(::Module, ::String) at ./Base.jl:377
[4] top-level scope at none:2
[5] eval [inlined]
[6] eval(::Expr) at ./client.jl:449
[7] top-level scope
in expression starting at /home/username/.julia/dev/SIMDPirates/src/SIMDPirates.jl:71
ERROR: LoadError: Failed to precompile SIMDPirates [21efa798-c60a-11e8-04d3-e1a92915a26a] to /home/username/.julia/compiled/v1.4/SIMDPirates/jEO2E_tYj51.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1263
[3] _require(::Base.PkgId) at ./loading.jl:1022
[4] require(::Base.PkgId) at ./loading.jl:920
[5] require(::Module, ::Symbol) at ./loading.jl:915
[6] include(::Module, ::String) at ./Base.jl:377
[7] top-level scope at none:2
[8] eval [inlined]
[9] eval(::Expr) at ./client.jl:449
[10] top-level scope
in expression starting at /home/username/.julia/dev/LoopVectorization/src/LoopVectorization.jl:3
ERROR: Failed to precompile LoopVectorization [bdcacae8-1622-11e9-2a5c-532679323890] to /home/username/.julia/compiled/v1.4/LoopVectorization/4TogI_tYj51.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String)
[3] _require(::Base.PkgId)
[4] require(::Base.PkgId) (repeats 2 times)
julia> versioninfo()
Julia Version 1.4.0-DEV.535
Commit 9babbf5* (2019-11-30 20:29 UTC)
Platform Info:
OS: Linux (x86_64-redhat-linux)
CPU: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-9.0.0 (ORCJIT, haswell)
Environment:
JULIA_NUM_THREADS = 24The error is in a precompile statement in the shared dependency SIMDPirates.jl
I'm going to remove all kw and #s functions from the precompile statements in these libraries and push.
It would be nice if there be a keyword argument for parcel to pass some extra precompile sentences manually.
It should check if the given sentences are not already detected by the @snoopi (are not duplicate), and then merge them with the return.
When the user gives
version = [v"1.4.1",v"1.2.0"]The following code is considered:
@static if VERSION < v"1.2.0"
# nothing - `snoopi_bot` isn't supported for `VERSION < v"1.2"` yet.
elseif VERSION <= v"1.2.0"
include("$precompiles_rootpath_rel//1.2.0/precompile_TestPackage0.jl")
_precompile_()
elseif VERSION <= v"1.4.1"
include("$precompiles_rootpath_rel//1.4.1/precompile_TestPackage0.jl")
_precompile_()
else
end
The assumption is that for lower versions the same precompilation signatures should be used.
Should we make this more limited and assume that the signatures are only valid for minor version changes? For example, if the user gives v"1.4.1", the following should be considered instead?
elseif v"1.4.0" <= VERSION <= v"1.4.1"
What should be the assumption?
Pkg.jl only downloads the package directory, not the whole git repo. This means that you won't have a license or readme locally after pkg> add SnoppCompileCore:
λ tree SnoopCompileCore
SnoopCompileCore
└── uT7Ty
├── Project.toml
└── src
├── snoopc.jl
├── SnoopCompileCore.jl
├── snoopi.jl
└── snoopr.jl
It's possible that this can be fixed on the Pkg (or package server) side, but for now it'd probably be best to have symlinks/copies in each package dir.
(On julia v1.2) I'm trying snoopcompile out for the first time. When snooping while running the tests of my package, the generated precompile file contains a lot of code with getfield(Main, Symbol("##xx#xx")), like this
precompile(Tuple{typeof(MonteCarloMeasurements.gradient), getfield(Main, Symbol("##28#95")), MonteCarloMeasurements.Particles{Float64, 500}})which causes
julia> using MonteCarloMeasurements
[ Info: Recompiling stale cache file /local/home/fredrikb/.julia/compiled/v1.2/MonteCarloMeasurements/tHYBD.ji for MonteCarloMeasurements [0987c9cc-fe09-11e8-30f0-b96dd679fdca]
ERROR: Error while loading expression starting at /local/home/fredrikb/.julia/dev/MonteCarloMeasurements/src/MonteCarloMeasurements.jl:89
caused by [exception 1]
UndefVarError: ##28#95 not defined
Stacktrace:
[1] _precompile_() at /local/home/fredrikb/.julia/dev/MonteCarloMeasurements/src/precompile_MonteCarloMeasurements.jl:11
I guess that the line in runtests.jl that lead to this is somewhere here
Am I doing something wrong?
For reference, the code used to generate the precompile file is below
using SnoopCompile
SnoopCompile.@snoopc "/tmp/robust_opt_compiles.log" begin
using MonteCarloMeasurements, Pkg
include(joinpath(dirname(dirname(pathof(MonteCarloMeasurements))), "test", "runtests.jl"))
end
data = SnoopCompile.read("/tmp/robust_opt_compiles.log")
pc = SnoopCompile.parcel(reverse!(data[2]))
SnoopCompile.write("/tmp/precompile", pc)I am trying to measure the compile times of different functions in our package (to be released soon).
The total compile time is around 70s, but the sum of the first column of SnoopCompile's output is 4.3s. This is assuming that the output is in nanoseconds since uv_hrtime() is used underneath. Could you explain this? Does it take the number of times a function is compiled into account?
In https://gist.github.com/ericphanson/959210a4c44900f2f379a805a0aa3d7c, I tried out generating precompilation statements for the COSMO optimization solver, used through Convex.jl, which relies on the MathOptInterface (MOI) intermediary layer. I found that just adding precompilation statements for COSMO (not Convex or MOI) takes off 15-25 seconds from the first solve (see the third entry in output.txt and output2.txt in that gist). Which is great! Adding precompile statements for Convex and MOI takes off another 4-8 seconds.
However, these statements add ~25 seconds to the precompilation time, and ~3 seconds to the using time (after precompilation). The precompilation time increase is too bad, but I did expect an increase. But the using time seems strange, because you'd hope nothing would need to be done if it's already precompiled. Is that to be expected?
Edit: the precompilation statements are available on the precompile branch of my fork https://github.com/ericphanson/COSMO.jl.
I love the inference snooping bot, but I think we might need to drop duplicates in the case of locally-generated functions. Example:
using SnoopCompile
inf_results = @snoopi begin
function eval_local_function(i)
@eval generated() = $i
return Base.invokelatest(generated) end
eval_local_function(1)
eval_local_function(2)
eval_local_function(3)
end
display(inf_results)results in:
(3.790855407714844e-5, MethodInstance for generated())
(4.291534423828125e-5, MethodInstance for generated())
(7.510185241699219e-5, MethodInstance for generated())
(0.0003750324249267578, MethodInstance for (::var"#eval_local_function#3")(::Int64))
Using this in a @snoopiBot context will result in a bunch of identical precompile calls being made. BinaryBuilder uses these kinds of auto-generated functions in places, and results in a precompilation script that looks like:
...
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
precompile(Tuple{typeof(strip),IOStream,Linux})
...
It makes sense for @snoopi to show them, but I think the precompilation script bot should probably de-duplicate these values.
I am trying to use SnoopCompile on DifferentialEquations and I am getting the following error. I am pretty much following exactly the readme:
julia> using DifferentialEquations
julia> data = SnoopCompile.read("/tmp/diffeq_compiles.csv")
ERROR: unexpected character 'e' after quoted field at row 198 column 2
in dlm_parse(::Array{UInt8,1}, ::UInt8, ::UInt8, ::UInt8, ::UInt8, ::Bool, ::Bool, ::Bool, ::Int64, ::Bool, ::Base.DataFmt.DLMOffsets) at ./datafmt.jl:553
in dlm_parse(::String, ::Char, ::Char, ::Char, ::Char, ::Bool, ::Bool, ::Bool, ::Int64, ::Bool, ::Base.DataFmt.DLMOffsets) at ./datafmt.jl:399
in readdlm_string(::String, ::Char, ::Type{T}, ::Char, ::Bool, ::Dict{Symbol,Union{Char,Integer,Tuple{Integer,Integer}}}) at ./datafmt.jl:268
in #readdlm_auto#11(::Array{Any,1}, ::Function, ::String, ::Char, ::Type{T}, ::Char, ::Bool) at ./datafmt.jl:57
in #readdlm#7(::Array{Any,1}, ::Function, ::String, ::Char, ::Char) at ./datafmt.jl:39
in #readdlm#6(::Array{Any,1}, ::Function, ::String, ::Char) at ./datafmt.jl:37
in read(::String) at /home/crackauc/.julia/v0.5/SnoopCompile/src/SnoopCompile.jl:130
in eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:64
in macro expansion at ./REPL.jl:95 [inlined]
in (::Base.REPL.##3#4{Base.REPL.REPLBackend})() at ./event.jl:46
For reference:
Julia Version 0.5.0-rc0+99
Commit c10667f* (2016-08-01 07:45 UTC)
Platform Info:
System: Linux (x86_64-redhat-linux)
CPU: Intel(R) Xeon(R) CPU E5-2667 v4 @ 3.20GHz
WORD_SIZE: 64
BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Haswell)
LAPACK: libopenblasp.so.0
LIBM: libopenlibm
LLVM: libLLVM-3.7.1 (ORCJIT, broadwell)Sorry if this is not the right place to post usage questions, but I'm really excited to try out the new snoopi macro (because inference is what I'm having problems with). However, when I try to snoopi on a call that takes 1+seconds to infer, I get the following trace:
(4.100799560546875e-5, MethodInstance for eltype(::Type{Array{Expr,2}}))
...
(0.0013608932495117188, MethodInstance for methods(::Any, ::Any))
(1.1331579685211182, MethodInstance for *(::StridedView{Float64,2,Array{Float64,1},typeof(identity)}, ::StridedView{Float64,2,Array{Float64,1},typeof(identity)}))
The call I'm curious about is the multiplication, but also in how long all methods called by multiplication take. The problem is that the sum of all timings given by snoopi does not equal the time it takes to infer the multiplication call. There seems to be function calls missing, which perhaps got inlined, but even with --inline=no there is no change.
to be clear, I ran:
using SnoopCompile,Stridedc
a = StridedView(rand(5,5))
vs = @snoopi a*a
I was trying to run this example, but countchildren is not defined.
Related to a couple of comments:
#49 (comment)
#49 (comment)
#49 (comment)
I am assuming that the users of this package are following the rules I set and specified in the documentation, and so the code works for these situations.
Handling all the other possibilities does not have much value.
For example:
To make the whole code independent of the current working directory and where packages files are placed a couple of things should be done.
For example to get the path of the package we should do:
packagePath = Base.read(`cmd /c julia -e 'import MatLang; print(pathof(MatLang))'`, String)packagePath = joinpath(pwd(),"src","$packageName.jl")or for precompile_$packageName.jl, some script can search in the package folder.
or if the user decides to have multiple precompile inclusion by mistake, such as:
# include("precompile.jl")
# _precompile_()
# some other stuff
include("precompile.jl")
_precompile_()The code will not work.
julia> Pkg.test("SnoopCompile")
INFO: Computing test dependencies for SnoopCompile...
INFO: No packages to install, update or remove
INFO: Testing SnoopCompile
WARNING: Base.UTF8String is deprecated, use String instead.
likely near /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl:7
WARNING: Base.UTF8String is deprecated, use String instead.
likely near /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl:7
WARNING: Base.UTF8String is deprecated, use String instead.
likely near /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl:7
WARNING: Base.UTF8String is deprecated, use String instead.
likely near /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl:7
WARNING: Base.UTF8String is deprecated, use String instead.
likely near /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl:7
WARNING: Base.UTF8String is deprecated, use String instead.
likely near /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl:7
WARNING: Base.UTF8String is deprecated, use String instead.
likely near /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl:7
WARNING: Base.UTF8String is deprecated, use String instead.
likely near /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl:7
WARNING: Base.UTF8String is deprecated, use String instead.
likely near /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl:7
in #parse_call#5 at /Users/solver/.julia/v0.5/SnoopCompile/src/SnoopCompile.jl
WARNING: symbol is deprecated, use Symbol instead.
in depwarn(::String, ::Symbol) at ./deprecated.jl:64
in symbol(::SubString{String}, ::Vararg{SubString{String},N}) at ./deprecated.jl:30
in #parse_call#5(::Dict{Any,Any}, ::Array{String,1}, ::Function, ::String) at /Users/solver/.julia/v0.5/SnoopCompile/src/SnoopCompile.jl:192
in parse_call(::String) at /Users/solver/.julia/v0.5/SnoopCompile/src/SnoopCompile.jl:137
in include_from_node1(::String) at ./loading.jl:426
in process_options(::Base.JLOptions) at ./client.jl:262
in _start() at ./client.jl:318
while loading /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl, in expression starting on line 7
Launching new julia process to run commands...
INFO: Recompiling stale cache file /Users/solver/.julia/lib/v0.5/FixedPointNumbers.ji for module FixedPointNumbers.
INFO: Recompiling stale cache file /Users/solver/.julia/lib/v0.5/ColorTypes.ji for module ColorTypes.
done.
ERROR: LoadError: LoadError: unexpected character 'e' after quoted field at row 109 column 2
in dlm_parse(::Array{UInt8,1}, ::UInt8, ::UInt8, ::UInt8, ::UInt8, ::Bool, ::Bool, ::Bool, ::Int64, ::Bool, ::Base.DataFmt.DLMOffsets) at ./datafmt.jl:618
in dlm_parse(::String, ::Char, ::Char, ::Char, ::Char, ::Bool, ::Bool, ::Bool, ::Int64, ::Bool, ::Base.DataFmt.DLMOffsets) at ./datafmt.jl:464
in readdlm_string(::String, ::Char, ::Type{T}, ::Char, ::Bool, ::Dict{Symbol,Union{Char,Integer,Tuple{Integer,Integer}}}) at ./datafmt.jl:333
in #readdlm_auto#11(::Array{Any,1}, ::Function, ::String, ::Char, ::Type{T}, ::Char, ::Bool) at ./datafmt.jl:122
in #readdlm#7(::Array{Any,1}, ::Function, ::String, ::Char, ::Char) at ./datafmt.jl:73
in #readdlm#6(::Array{Any,1}, ::Function, ::String, ::Char) at ./datafmt.jl:65
in read(::String) at /Users/solver/.julia/v0.5/SnoopCompile/src/SnoopCompile.jl:130
in include_from_node1(::String) at ./loading.jl:426 (repeats 2 times)
in process_options(::Base.JLOptions) at ./client.jl:262
in _start() at ./client.jl:318
while loading /Users/solver/.julia/v0.5/SnoopCompile/test/colortypes.jl, in expression starting on line 17
while loading /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl, in expression starting on line 21
=======================================================================[ ERROR: SnoopCompile ]=======================================================================
failed process: Process(`/Users/solver/Projects/julia5/usr/bin/julia -Cnative -J/Users/solver/Projects/julia5/usr/lib/julia/sys.dylib --compile=yes --depwarn=yes --check-bounds=yes --code-coverage=none --color=yes --compilecache=yes /Users/solver/.julia/v0.5/SnoopCompile/test/runtests.jl`, ProcessExited(1)) [1]
=====================================================================================================================================================================
INFO: No packages to install, update or remove
ERROR: SnoopCompile had test errors
in #test#61(::Bool, ::Function, ::Array{AbstractString,1}) at ./pkg/entry.jl:740
in (::Base.Pkg.Entry.#kw##test)(::Array{Any,1}, ::Base.Pkg.Entry.#test, ::Array{AbstractString,1}) at ./<missing>:0
in (::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}})() at ./pkg/dir.jl:31
in cd(::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}}, ::String) at ./file.jl:59
in #cd#1(::Array{Any,1}, ::Function, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./pkg/dir.jl:31
in (::Base.Pkg.Dir.#kw##cd)(::Array{Any,1}, ::Base.Pkg.Dir.#cd, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./<missing>:0
in #test#3(::Bool, ::Function, ::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258
in test(::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258snoopi generates some sentence here that gives this error:
#kw##something
https://travis-ci.com/juliamatlab/MatLang/jobs/262514753
@var_str:
https://travis-ci.com/juliamatlab/MatLang/jobs/262514750#L212
Removing a couple of sentences doesn't help juliamatlab/MatLang@be7ecfb:
https://travis-ci.com/juliamatlab/MatLang/jobs/262791428
0.5 signatures contain a lot of calls with # in them, which SnoopCompile ignores.
I've written this snippet of code for inclusion in PackageCompiler.jl, but it feels like it should belong somewhere here.
let M = Module() # Prevent this from putting anything into the Main namespace
# get a local reference to any module that may be directly referenced in the `precompile.jl` file
for m in Base.loaded_modules_array()
Core.isdefined(M, nameof(m)) || Core.eval(M, Expr(:const, Expr(:(=), nameof(m), m)))
end
for n in names(Main)
if Core.isdefined(Main, n) && isconst(Main, n)
m = getfield(Main, n)
m isa Module && (Core.isdefined(M, nameof(m)) || Core.eval(M, Expr(:const, Expr(:(=), nameof(m), m))))
end
end
Base.include(M, \"precompile.jl\")")
end # letJust for fun, I ran this on Cxx, but SnoopCompile failed with the following error:
julia> pc, discards = SnoopCompile.parcel(data[end:-1:1,2])
ERROR: c is Cxx.specialize_template(Cxx.ClangCompiler, Cxx.CppPtr{Cxx.CxxQualType{Cxx.CppBaseType{:clang::ClassTemplateDecl}, (false, false, false)}, (false, false, false)}, DataType), fargs is SubString{ASCIIString}["Cxx.specialize_template","Cxx.ClangCompiler, Cxx.CppPtr{Cxx.CxxQualType{Cxx.CppBaseType{:clang::ClassTemplateDecl}, ","false, false, false)}, ","false, false, false)}, DataType)"]
in error at ./error.jl:22
in parse_call at /Users/kfischer/.julia/v0.5/SnoopCompile/src/SnoopCompile.jl:145
in parcel at /Users/kfischer/.julia/v0.5/SnoopCompile/src/SnoopCompile.jl:196
We should mention how to correctly use SnoopCompile for those who want to use the inner Core and Analysis packages directly, but in my opinion, we should state that the easiest and safest way to use SnoopCompile features is through SnoopCompileBot (offline or online) because it loads the packages in the correct order, uses an external process that will be always clean, and adds some extra features like benchmarking, automatic precompile generation and handling.
I will need to address my last comment in #77 to make SnoopCompileBot a safe interface to all of the snooping operations including invalidations.
Copied from #98
Some packages are using .github/workflows/SnoopCompile.yml, and inside that, they add SnoopCompile and use it. Is there a way that CompatHelper allows us to find these packages and give them some notification that SnoopCompile is updated and they might need to update some of their code?
Deprecation warnings are just shown passed in GitHub actions and the users may not notice anything.
One solution might be using CompatHelper:
JuliaRegistries/CompatHelper.jl#218
For Julia 1, Base.BroadCast.restart_copyto_nonleaf! is not defined which results in error:
https://travis-ci.com/aminya/Atom.jl/jobs/270937636#L418
Parcel doesn't seem to filter this out for old Julia
In the docs it is highlighted that the versions specified in SnoopCompile.yml have to match the versions in BotConfig. In the .yml file I can use the versions
matrix:
version: # NOTE: the versions below should match those in your botconfig
- '1'
- '1.3'
- 'nightly'
where (I think) '1' corresponds to '1.4.2' and '1.3' to '1.3.1'.
However:
julia> BotConfig("Plots", version = ["1", "1.3", "nightly"]).version
3-element Array{VersionNumber,1}:
v"1.0.0"
v"1.3.0"
v"1.6.0-DEV"Does this mean I have to set the versions to 1.3.1 and 1.4.2 in both (or only BotConfig respectively) and update the versions manually with new julia releases, or is there another way to do this?
regarding addtestdep there is no way to add the tests deps to the current environment. So I wrote my own way. If there is some way in Pkg itself, I would be happy to remove it.
I asked this a couple of times in the Slack group. https://julialang.slack.com/archives/C67EFTEF3/p1577556453102300
There is an issue for it without any solution
JuliaLang/Pkg.jl#1008
#66 introduces following error around deprecation:
ERROR: LoadError: LoadError: UndefVarError: warning not defined
Stacktrace:
[1] @snoopiBot(::LineNumberNode, ::Module, ::Vararg{Any,N} where N) at /home/runner/.julia/packages/SnoopCompile/AmOIN/src/bot.jl:35
[2] include(::String) at ./client.jl:439
[3] top-level scope at none:1
in expression starting at /home/runner/work/Atom.jl/Atom.jl/deps/SnoopCompile/snoopCompile.jl:3
in expression starting at /home/runner/work/Atom.jl/Atom.jl/deps/SnoopCompile/snoopCompile.jl:3
##[error]Process completed with exit code 1.
ref: https://github.com/JunoLab/Atom.jl/pull/252/checks?check_run_id=429043709
You may want to use depwarn instead: (e.g.: JuliaLang/julia#34646)
Opening an issue to track, appears related to the parsing of the generated csv files:
WARNING: failed parse of line: Tuple{getfield(SnoopTestTemp, Symbol("getfield(Main.SnoopTestTemp, Symbol("##2#4")){Int32}")), Array{Float32, 1}}
Test Failed at /home/alex/.julia/v0.7/SnoopCompile/test/runtests.jl:34
Expression: keep
ERROR: LoadError: There was an error during testing
in expression starting at /home/alex/.julia/v0.7/SnoopCompile/test/runtests.jl:34
despite Plots.jl's recent 2x reduction in first-time-to-plot using SnoopCompile, for Gadfly i can only achieve a 25% speedup. i presume this is because "there are some significant constraints that sometimes prevent Julia from saving even the inference results". would be nice to know ways Gadfly's source code could be refactored instead. is it worth itemizing in the SnoopCompile docs the specific cases for which julia can't save the inference results? or are things too much in flux now with the ongoing efforts to improve the compiler??
I am trying to use SnoopCompileBot via Github Actions for Plots.jl and get the following error:
Run julia --project -e 'include("deps/SnoopCompile/snoop_bot.jl")'
[ Info: /home/runner/work/Plots.jl/Plots.jl/src/precompile_includer.jl file will be created/overwritten
[ Info: SnoopCompile will try to write "include("precompile_includer.jl")" before end of the module in /home/runner/work/Plots.jl/Plots.jl/src/Plots.jl. Assume that the last `end` is the end of a module.
ERROR: LoadError: ArgumentError: Package SnoopCompileBot not found in current path:
- Run `import Pkg; Pkg.add("SnoopCompileBot")` to install the SnoopCompileBot package.
Stacktrace:
[1] require(::Module, ::Symbol) at ./loading.jl:892
[2] top-level scope at /home/runner/.julia/packages/SnoopCompileBot/K9naN/src/snoop_bot.jl:112
[3] eval at ./boot.jl:331 [inlined]
[4] snoop_bot(::BotConfig, ::String, ::Module; snoop_mode::Symbol) at /home/runner/.julia/packages/SnoopCompileBot/K9naN/src/snoop_bot.jl:182
[5] snoop_bot at /home/runner/.julia/packages/SnoopCompileBot/K9naN/src/snoop_bot.jl:177 [inlined] (repeats 2 times)
[6] top-level scope at /home/runner/work/Plots.jl/Plots.jl/deps/SnoopCompile/snoop_bot.jl:3
[7] include(::String) at ./client.jl:439
[8] top-level scope at none:1
in expression starting at /home/runner/work/Plots.jl/Plots.jl/deps/SnoopCompile/snoop_bot.jl:3
##[error]Process completed with exit code 1.I think I followed the instructions in the docs but I am probably doing something stupid. Here's the test PR on my fork: daschw/Plots.jl#2. I would really appreciate some hints to what I am doing wrong. Thanks!
Is there any other timing measure that shows the effect of using precompiles (in a dynamic manner without sysimage) other than what we have in @snoopi and snoopi_bot?
Is @timev a good measure?
We allow passing flags to julia_cmd, but it would be nice to be able to pass the entire command instead. This would be helpful for SnoopCompile.jl, where we want to re-write some of the flags from julia_cmd (specifically, --sysimg and --target) and to change the cwd (might need to wait for JuliaLang/julia#24353 though)
Automatic repository updating using Github actions for uploading the files generated by snoopCompile.jl script
Similar to https://github.com/julia-actions/julia-uploadcodecov or https://github.com/julia-actions/julia-docdeploy
I was trying to run a basic example from the documentation: https://timholy.github.io/SnoopCompile.jl/stable/snoopr/#A-first-example-1 and it doesn't work:
julia> using SnoopCompileCore
julia> invalidations = @snoopr f(::Float64) = 2;
ERROR: LoadError: UndefVarError: @snoopr not defined
in expression starting at REPL[2]:1I'm using version 2.1.1 of SnoopCompileCore.
julia> versioninfo()
Julia Version 0.4.0-rc3
Commit 483d548* (2015-09-27 20:34 UTC)
Platform Info:
System: Darwin (x86_64-apple-darwin13.4.0)
CPU: Intel(R) Core(TM) i5-2500S CPU @ 2.70GHz
WORD_SIZE: 64
BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
LAPACK: libopenblas
LIBM: libopenlibm
LLVM: libLLVM-3.3
julia> using SnoopCompile
julia> SnoopCompile.@snoop "/tmp/images_compiles.csv" begin
1+2
end
Turning on compiler logging
error: src/codegen.cpp: No such file or directory
make: *** No targets specified and no makefile found. Stop.
ERROR: failed process: Process(`sh snoop.sh ''`, ProcessExited(2)) [2]I created a PR here:
aminya#2
We are trying to understand how to build a user image in Julia and playing with SnoopCompile/PackageCompiler. There are a couple of things that would be nice to clarify.
cc: @SimonDanisch
SnoopCompile.jl/src/parcel_snoopi.jl
Line 197 in 6ad1d8c
I thought I'd give the new invalidation tooling a go on Flux, but hit this error below.
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.6.0-DEV.425 (2020-07-10)
_/ |\__'_|_|_|\__'_| | Commit 8320fcc7d9 (0 days old master)
|__/ |
(@v1.6) pkg> st
Status `~/.julia/environments/v1.6/Project.toml`
[587475ba] Flux v0.11.0
[7869d1d1] IRTools v0.4.0 `~/.julia/dev/IRTools`
[aa65fe97] SnoopCompile v1.7.1
[9ea4277c] SnoopCompileAnalysis v1.7.1
[e2b509da] SnoopCompileCore v1.7.1
julia> using SnoopCompileCore
julia> invalidations = @snoopr using Flux
6076-element Vector{Any}:
MethodInstance for #_print#9(::Int64, ::Bool, ::Bool, ::Pkg.API.var"#4#6", ::typeof(Pkg.TOML._print), ::IOStream, ::AbstractDict, ::Vector{String})
0
MethodInstance for (::Pkg.TOML.var"#_print##kw")(::NamedTuple{(:indent, :first_block, :sorted, :by),Tuple{Int64,Bool,Bool,Pkg.API.var"#4#6"}}, ::typeof(Pkg.TOML._print), ::IOStream, ::AbstractDict, ::Vector{String})
1
MethodInstance for (::Pkg.TOML.var"#_print##kw")(::NamedTuple{(:indent, :sorted, :by),Tuple{Int64,Bool,Pkg.API.var"#4#6"}}, ::typeof(Pkg.TOML._print), ::IOStream, ::AbstractDict, ::Vector{String})
1
Tuple{Base.var"#sort!##kw",NamedTuple{(:by,),Tuple{Pkg.API.var"#4#6"}},typeof(sort!),Any}
MethodInstance for #_print#9(::Int64, ::Bool, ::Bool, ::typeof(identity), ::typeof(Pkg.TOML._print), ::IOStream, ::AbstractDict, ::Vector{String})
0
MethodInstance for (::Pkg.TOML.var"#_print##kw")(::NamedTuple{(:indent, :first_block, :sorted, :by),Tuple{Int64,Bool,Bool,typeof(identity)}}, ::typeof(Pkg.TOML._print), ::IOStream, ::AbstractDict, ::Vector{String})
⋮
"insert_backedges"
MethodInstance for Base.IteratorEltype(::AbstractArray)
1
MethodInstance for unique(::AbstractArray)
2
MethodInstance for Base.IteratorEltype(::Type{var"#s429"} where var"#s429"<:AbstractArray{T,N} where N where T)
"jl_method_table_insert"
Base.IteratorEltype(::Type{var"#s34"} where var"#s34"<:AbstractTrees.TreeIterator) in AbstractTrees at /Users/ian/.julia/packages/AbstractTrees/VQ0nX/src/iteration.jl:5
"jl_method_table_insert"
julia> using SnoopCompileAnalysis
julia> trees = invalidation_trees(invalidations);
insert_backedges for MethodInstance for _progress(::String, ::Float64, ::Expr, ::Union{Expr, Symbol}, ::Union{Expr, Symbol}, ::Union{typeof(Juno._comprehension), typeof(Juno._for)}, ::Any, ::Any, ::Expr)
insert_backedges for MethodInstance for _progress(::Expr, ::Float64, ::Expr, ::Union{Expr, Symbol}, ::Union{Expr, Symbol}, ::Union{typeof(Juno._comprehension), typeof(Juno._for)}, ::Any, ::Any, ::Expr)
insert_backedges for MethodInstance for (::StaticArrays.var"#65#71")(::Int64)
insert_backedges for MethodInstance for (::ForwardDiff.var"#6#10"{Dict{Symbol,Expr}})(::Any)
insert_backedges for MethodInstance for rand(::Type{ColorTypes.AGray32}, ::Tuple{Int64,Int64})
insert_backedges for MethodInstance for find_maximum_chroma(::ColorTypes.LCHuv, ::Float64, ::Float64)
insert_backedges for MethodInstance for find_maximum_chroma(::ColorTypes.LCHuv, ::Int64, ::Float64)
insert_backedges for MethodInstance for find_maximum_chroma(::ColorTypes.LCHuv, ::Float64, ::Int64)
insert_backedges for MethodInstance for find_maximum_chroma(::ColorTypes.LCHuv, ::Int64, ::Int64)
insert_backedges for MethodInstance for #MSC#13(::Bool, ::typeof(Colors.MSC), ::Any, ::Any)
insert_backedges for MethodInstance for _parse_colorant(::String)
insert_backedges for MethodInstance for evaluate_cost_tile(::LoopVectorization.LoopSet, ::Vector{Symbol}, ::LoopVectorization.UnrollSymbols)
insert_backedges for MethodInstance for prefetchisagoodidea(::LoopVectorization.LoopSet, ::LoopVectorization.Operation, ::LoopVectorization.UnrollArgs{Int64})
ERROR: unexpected item Core.TypeMapEntry(nothing, Tuple{typeof(show),IOBuffer,Type}, Tuple{typeof(show),IOBuffer,Any}, svec(Tuple{typeof(show),IOBuffer,Core.TypeofBottom}), 0x000000000000269a, 0x0000000000006f0e, MethodInstance for show(::IOBuffer, ::Type), false, false, false) at 5814
Stacktrace:
[1] error(::String, ::Core.TypeMapEntry, ::String, ::Int64)
@ Base ./error.jl:42
[2] invalidation_trees(list::Vector{Any})
@ SnoopCompileAnalysis ~/.julia/packages/SnoopCompileAnalysis/vI2vl/src/invalidations.jl:323
[3] top-level scope
@ REPL[4]:1
To edit a specific method, type the corresponding number into the REPL and press Ctrl+Q
Every time I run the snoopc it seems that a new file is generated (which causes duplicate PR in #37). Is it possible to sort the result before writing it to the file?
Due to AbstractString - maybe should just support 0.4 only anyway?
I've been meaning to post about this issue for a while, because its behavior has been baffling me for some time.
I was finally gearing up to post about it today, but thanks to some great sleuthing work together with @Sacha0, we managed to understand the situation! :) I'm not sure about a fix yet, but I'm glad to be able to understand and explain everything that's going on.
First, here is the bizarre behavior I've observed, which maybe others have seen before, too:
Adding @time to a @snoopi call changes the results!
Consider this example, where I create a module with some functions, where I've forced an inference barrier to let us see inference at work. It includes a forced runtime dispatch, so we see three separate results from @snoopi - one for each of the top-level calls to inference. This result is repeatable; as long as you redefine the module and resnoop, you again get [g,i,i2].
The interesting behavior is that when we then @time the call to @snoopi, we are missing the top-level inference call! We instead only get the calls for the functions called via dynamic dispatch.
julia> module M
i(x) = x+5
i2(x) = x+2
h(a::Array) = i2(a[1]::Integer) + i(a[1]::Integer) + 2
g(y::Integer, x) = h(Any[y]) + Int(x)
end;
julia> timings = SnoopCompileCore.@snoopi let
M.g(2,3.0)
end; # Run once to compile all the downstream functions (like + and getindex) to make the subsequent logs clearer.
julia> module M
i(x) = x+5
i2(x) = x+2
h(a::Array) = i2(a[1]::Integer) + i(a[1]::Integer) + 2
g(y::Integer, x) = h(Any[y]) + Int(x)
end;
WARNING: replacing module M.
julia> timings = SnoopCompileCore.@snoopi let
M.g(2,3.0)
end
3-element Array{Tuple{Float64,Core.MethodInstance},1}:
(0.0002079010009765625, MethodInstance for i(::Int64))
(0.000225067138671875, MethodInstance for i2(::Int64))
(0.0015878677368164062, MethodInstance for g(::Int64, ::Float64))
julia> module M
i(x) = x+5
i2(x) = x+2
h(a::Array) = i2(a[1]::Integer) + i(a[1]::Integer) + 2
g(y::Integer, x) = h(Any[y]) + Int(x)
end;
WARNING: replacing module M.
julia> @time timings = SnoopCompileCore.@snoopi let
M.g(2,3.0)
end
0.004031 seconds (1.45 k allocations: 86.355 KiB)
2-element Array{Tuple{Float64,Core.MethodInstance},1}:
(0.00015401840209960938, MethodInstance for i(::Int64))
(0.00024318695068359375, MethodInstance for i2(::Int64))I found this result quite surprising! But don't worry, if you don't already see the problem, it'll be clear in the end.
Next, here is another very surprising behavior I've seen with the @time macro, which baffled us for a few hours:
julia> elapsedtime1 = Base.time_ns(); @time db = Delve.API.create_database(); elapsedtime = Base.time_ns() - elapsedtime1;
29.711536 seconds (34.38 M allocations: 1.745 GiB, 2.00% gc time)
julia> elapsedtime / 1e9
98.999127862create_database() is some internal function in our codebase that takes quite a long time to compile.
But somehow, the time reported by time is 70 seconds less than the time I measured! It's weird, right!?
We finally found the answer, and it's the same answer for both cases. What is happening here is that the @time macro causes its entire body to be compiled before executing it. And this includes the body of the code that we want to infer with @snoopi!
So basically, by the time we actually get to the part of the code that enables snooping within julia, we've already finished inferring the top-level call we wanted to measure!
We can of course fix this with any kind of inference barrier inside the body we're measuring. e.g. an @eval will do it:
julia> module M
i(x) = x+5
i2(x) = x+2
h(a::Array) = i2(a[1]::Integer) + i(a[1]::Integer) + 2
g(y::Integer, x) = h(Any[y]) + Int(x)
end;
WARNING: replacing module M.
julia> @time timings = SnoopCompileCore.@snoopi let
@eval M.g(2,3.0) # thanks to this @eval, we now delay inferring `g()` until _after_ we've enabled snooping.
end
0.017935 seconds (11.62 k allocations: 569.847 KiB)
3-element Array{Tuple{Float64,Core.MethodInstance},1}:
(0.0002300739288330078, MethodInstance for i(::Int64))
(0.0002770423889160156, MethodInstance for i2(::Int64))
(0.0019459724426269531, MethodInstance for g(::Int64, ::Float64))And so in my example above, where we've always thought our create_database() function takes 30 seconds to compile the first time, it ACTUALLY takes 100 seconds! I was just only starting my timer after all the top-level inference was completed! And since it's fairly type-stable code, we were apparently able to get most of the way through it.
Hooray! :) It all makes sense now!
But I'm not sure what should be done about it.
It strikes me that this may surprise other users, since users of @snoopi really are intending to measure inference times of the body of the macro. So it seems to me that we should change the macro to go out of our way to prevent it from being inferred before we enable snooping.
I'm not sure if we should go as far as adding an @eval, since that does actually change the meaning of the code, but is there any other clever way to add an inference barrier that we can use?
:) Thanks!
I'm a bit concerned about SnoopCompile's growing size. The main issue is that SnoopCompile has tools to spy on the system, but as it adds more of its own methods it introduces ever-more perturbations of that system.
Here's an interesting example: start a fresh REPL (with --startup-file=no) and cut-and-paste a "private" version of @snoopr, and then use it to snoop on using SnoopCompile:
julia> macro mysnoopr(expr)
quote
local list = ccall(:jl_debug_method_invalidation, Any, (Cint,), 1)
Expr(:tryfinally,
$(esc(expr)),
ccall(:jl_debug_method_invalidation, Any, (Cint,), 0)
)
list
end
end
@mysnoopr (macro with 1 method)
julia> invs = @mysnoopr using SnoopCompile
3212-element Array{Any,1}:
MethodInstance for isempty(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}})
"invalidate_mt_cache"
MethodInstance for ==(::Symbol, ::Int64)
"invalidate_mt_cache"
MethodInstance for ==(::Char, ::String)
"invalidate_mt_cache"
MethodInstance for ==(::Module, ::Module)
"invalidate_mt_cache"
MethodInstance for ==(::Expr, ::Int64)
"invalidate_mt_cache"
MethodInstance for ==(::TextDisplay, ::REPL.REPLDisplay{REPL.LineEditREPL})
"invalidate_mt_cache"
MethodInstance for ==(::Distributed.WorkerState, ::Distributed.WorkerState)
"invalidate_mt_cache"
MethodInstance for ==(::Array{String,1}, ::Bool)
"invalidate_mt_cache"
MethodInstance for join(::Base.Iterators.Take{Base.Iterators.Repeated{DataType}}, ::String)
"invalidate_mt_cache"
MethodInstance for join(::Tuple{Symbol}, ::String)
"invalidate_mt_cache"
MethodInstance for join(::Tuple{Symbol,Symbol}, ::String)
"invalidate_mt_cache"
MethodInstance for join(::Array{String,1}, ::String, ::String)
"invalidate_mt_cache"
⋮
MethodInstance for #_print#9(::Int64, ::Bool, ::Bool, ::Any, ::typeof(Pkg.TOML._print), ::Base.GenericIOBuffer{Array{UInt8,1}}, ::AbstractDict, ::Array{String,1})
0
MethodInstance for (::Pkg.TOML.var"#_print##kw")(::NamedTuple{(:indent, :first_block, :sorted, :by),_A} where _A<:Tuple, ::typeof(Pkg.TOML._print), ::Base.GenericIOBuffer{Array{UInt8,1}}, ::AbstractDict, ::Array{String,1})
1
MethodInstance for (::Pkg.TOML.var"#_print##kw")(::NamedTuple{(:indent, :sorted, :by),_A} where _A<:Tuple, ::typeof(Pkg.TOML._print), ::Base.GenericIOBuffer{Array{UInt8,1}}, ::AbstractDict, ::Array{String,1})
1
Tuple{Base.var"#sort!##kw",NamedTuple{(:by,),_A} where _A<:Tuple,typeof(sort!),Any}
MethodInstance for #_print#9(::Int64, ::Bool, ::Bool, ::Any, ::typeof(Pkg.TOML._print), ::IOStream, ::AbstractDict, ::Array{String,1})
0
MethodInstance for (::Pkg.TOML.var"#_print##kw")(::NamedTuple{(:indent, :first_block, :sorted, :by),_A} where _A<:Tuple, ::typeof(Pkg.TOML._print), ::IOStream, ::AbstractDict, ::Array{String,1})
1
MethodInstance for (::Pkg.TOML.var"#_print##kw")(::NamedTuple{(:indent, :sorted, :by),_A} where _A<:Tuple, ::typeof(Pkg.TOML._print), ::IOStream, ::AbstractDict, ::Array{String,1})
1
Tuple{Base.var"#sort!##kw",NamedTuple{(:by,),_A} where _A<:Tuple,typeof(sort!),Any}
(::Base.var"#sort!##kw")(::Any, ::typeof(sort!), d::OrderedCollections.OrderedDict) in OrderedCollections at /home/tim/.julia/packages/OrderedCollections/P6ntV/src/dict_sorting.jl:4
"jl_method_table_insert"
MethodInstance for fzero(::Base.Broadcast.Broadcasted{LinearAlgebra.StructuredMatrixStyle{T},Axes,F,Args} where Args<:Tuple where F where Axes where T)
1
MethodInstance for fzero(::Base.Broadcast.Broadcasted{LinearAlgebra.StructuredMatrixStyle{T},Nothing,typeof(-),_B} where _B<:Tuple where T)
1
MethodInstance for any(::typeof(ismissing), ::Any)
"jl_method_table_insert"
any(f, node::SnoopCompile.InstanceNode) in SnoopCompile at /home/tim/.julia/dev/SnoopCompile/src/invalidations.jl:49
"jl_method_table_insert"
julia> length(unique(filter(x->isa(x, Core.MethodInstance), invs)))
1097So, loading SnoopCompile itself invalidates more than 1000 MethodInstances. This is quite bad for a tool that is used to detect invalidations. By calling invalidation_trees(invs), you can see that the overwhelming source of these invalidations is FilePaths.jl, with a small handful contributed by OrderedCollections. But as usual, the fix might be in Julia rather than FilePaths---without looking at FilePaths more carefully, this should not be interpreted as saying that FilePaths is necessarily doing something wrong.
One might worry about some of the other tools, too. SnoopCompile does not have a huge __init__ function, so this won't force much inference/compilation, and what is done is really essential for @snoopi to be able to function. But there's a bit of top-level work in botutils.jl (it will compile methods for Dict{String,String}, if they aren't already), deprecations.jl (the @eval statements), invalidations.jl (surrounding which for dummyinstance), parcel_snoopi.jl (stuff around lookup_kwbody_ex and default_exclusions), and the regexs in src/SnoopCompile.jl itself. (There are a couple of others but I am pretty sure they aren't a problem or are strictly necessary for even the most minimal functionality.)
I am not quite sure about the best way to address this. One option would be to split out the absolute minimal "core" functionality into one or more packages of their own, and re-export it from SnoopCompile. People who care about this issue could use the micro-package(s) directly. The ability to put multiple packages into a single repo might be nice for this kind of thing.
I solved the issue of precompiling the Pkg sentences by using Base.precompile.
However, running the snoopi_bot with the exhaustive feature enabled on Zygote generates some strange errors that: Base not defined.
This happens when if I copy-paste the sentence in REPL (without Meta.parse) it works fine. For example:
# Works:
Base.precompile(Tuple{Type{NamedTuple{(:bc,),T} where T<:Tuple},Tuple{NamedTuple{(:f, :args, :axes),Tuple{Nothing,Tuple{Nothing,Tuple{Nothing,Float64,Float64,Float64,Float64,Float64},Nothing},Nothing}}}})
# Errors:
Core.eval(Core, Meta.parse("Base.precompile(Tuple{Type{NamedTuple{(:bc,),T} where T<:Tuple},Tuple{NamedTuple{(:f, :args, :axes),Tuple{Nothing,Tuple{Nothing,Tuple{Nothing,Float64,Float64,Float64,Float64,Float64},Nothing},Nothing}}}})") )┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{Type{NamedTuple{(:bc,),T} where T<:Tuple},Tuple{NamedTuple{(:f, :args, :axes),Tuple{Nothing,Tuple{Nothing,Tuple{Nothing,Float64,Float64,Float64,Float64,Float64},Nothing},Nothing}}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{Type{NamedTuple{(:f, :iter),T} where T<:Tuple},Tuple{NamedTuple{(:t,),Tuple{Tuple{Int64,Int64,Nothing}}},Array{Nothing,1}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{Type{NamedTuple{(:condition, :args),T} where T<:Tuple},Tuple{Nothing,Array{Nothing,1}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{Type{NamedTuple{(:x, :y),T} where T<:Tuple},Tuple{Int64,Float64}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{Type{NamedTuple{(:condition, :args),T} where T<:Tuple},Tuple{Nothing,Array{Float64,1}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{typeof(Core.Compiler.eltype),Type{Array{Int64,0}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{Type{Array{Tuple{Function,Array{T,2} where T,Array{T,2} where T,Any,Int64,Array{T,2} where T},1}},UndefInitializer,Int64})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{typeof(Core.Compiler.eltype),Type{Array{Int64,2}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{typeof(Core.Compiler.eltype),Type{Array{Float64,2}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{typeof(Core.Compiler.eltype),Type{Array{Array{Float64,2},1}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{typeof(Core.Compiler.eltype),Type{Array{Array{Float64,1},1}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{Type{NamedTuple{(:bc,),T} where T<:Tuple},Tuple{NamedTuple{(:f, :args, :axes),Tuple{Nothing,Tuple{Tuple{Array{Float64,2},Array{Float64,2}}},Nothing}}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{Type{NamedTuple{(:f, :args, :axes),T} where T<:Tuple},Tuple{Nothing,Tuple{Tuple{Array{Float64,2},Array{Float64,2}}},Nothing}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{Type{NamedTuple{(:diag,),T} where T<:Tuple},Tuple{Array{Float64,1}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{Type{NamedTuple{(:factors,),T} where T<:Tuple},Tuple{Array{Float64,2}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
┌ Warning: Faulty precompile sentence: Base.precompile(Tuple{Type{NamedTuple{(:expr,),T} where T<:Tuple},Tuple{Tuple{Symbol,Symbol,Symbol}}})
│ exception = UndefVarError: Base not defined
└ @ Core precompile_Core.jl:1
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.