julialinearalgebra / blasbenchmarkscpu.jl Goto Github PK

View Code? Open in Web Editor NEW

5.0 5.0 2.0 450 KB

Benchmark BLAS Libraries

License: MIT License

Julia 100.00%

blasbenchmarkscpu.jl's People

Contributors

Stargazers

Watchers

Forkers

ranocha vandenman

blasbenchmarkscpu.jl's Issues

Fails to Precompile in Julia 1.6.2

CheapThreads.jl is the source of the issue.

The same error messages both in Mac and Linux Ubuntu

Click here for the full Pkg stacktrace

pkg> add BLASBenchmarksCPU
   Resolving package versions...
    Updating `/tmp/jl_SE9jam/Project.toml`
  [5fdc822c] + BLASBenchmarksCPU v0.3.3
    Updating `/tmp/jl_SE9jam/Manifest.toml`
  [621f4979] + AbstractFFTs v1.0.1
  [79e6a3ab] + Adapt v3.3.1
  [4fba245c] + ArrayInterface v3.1.17
  [13072b0f] + AxisAlgorithms v1.0.0
  [5fdc822c] + BLASBenchmarksCPU v0.3.3
  [6e4b80f9] + BenchmarkTools v0.7.0
  [159f3aea] + Cairo v1.0.5
  [324d7699] + CategoricalArrays v0.9.7
  [d360d2e6] + ChainRulesCore v0.10.11
  [b630d9fa] + CheapThreads v0.2.4
  [3da002f7] + ColorTypes v0.11.0
  [5ae59095] + Colors v0.12.8
  [34da2185] + Compat v3.31.0
  [a81c6b42] + Compose v0.9.2
  [d38c429a] + Contour v0.5.7
  [7ad07ef1] + CoupledFields v0.2.0
  [a8cc5b0e] + Crayons v4.0.4
  [9a962f9c] + DataAPI v1.7.0
  [a93c6f00] + DataFrames v0.22.7
  [864edb3b] + DataStructures v0.18.9
  [e2d170a0] + DataValueInterfaces v1.0.0
  [b552c78f] + DiffRules v1.2.1
  [b4f34e82] + Distances v0.10.3
  [31c24e10] + Distributions v0.24.18
  [ffbed154] + DocStringExtensions v0.8.5
  [7a1cc6ca] + FFTW v1.4.3
  [1a297f60] + FillArrays v0.11.9
  [53c48c17] + FixedPointNumbers v0.8.4
  [186bb1d3] + Fontconfig v0.4.0
  [59287772] + Formatting v0.4.2
  [c91e804a] + Gadfly v1.3.3
  [bffe22d1] + Gaius v0.6.5
  [a2bd30eb] + Graphics v1.1.0
  [42e2da0e] + Grisu v1.0.2
  [a1b4810d] + Hexagons v0.2.0
  [0e44f5e4] + Hwloc v2.0.0
  [615f187c] + IfElse v0.1.0
  [9b13fd28] + IndirectArrays v0.5.1
  [a98d9a8b] + Interpolations v0.13.3
  [41ab1584] + InvertedIndices v1.0.0
  [c8e1da08] + IterTools v1.3.0
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [692b3bcd] + JLLWrappers v1.3.0
  [682c06a0] + JSON v0.21.1
  [e5e0dc1b] + Juno v0.8.4
  [5ab0869b] + KernelDensity v0.6.3
  [4345ca2d] + Loess v0.5.3
  [2ab3a3ac] + LogExpFunctions v0.2.5
  [bdcacae8] + LoopVectorization v0.12.18
  [1914dd2f] + MacroTools v0.5.6
  [d125e4d3] + ManualMemory v0.1.4
  [442fdcdd] + Measures v0.3.1
  [e89f7d12] + Media v0.5.0
  [e1d29d7a] + Missings v0.4.5
  [77ba4419] + NaNMath v0.3.5
  [6fd5a793] + Octavian v0.2.15
  [6fe1bfb0] + OffsetArrays v1.10.2
  [bac558e1] + OrderedCollections v1.4.1
  [90014a1f] + PDMats v0.11.1
  [69de0a69] + Parsers v1.1.0
  [2dfb63ee] + PooledArrays v1.2.1
  [21216c6a] + Preferences v1.2.2
  [08abe8d2] + PrettyTables v0.11.1
  [92933f4c] + ProgressMeter v1.7.1
  [1fd47b50] + QuadGK v2.4.1
  [c84ed2f1] + Ratios v0.4.0
  [189a3867] + Reexport v1.1.0
  [ae029012] + Requires v1.1.3
  [79098fc4] + Rmath v0.7.0
  [476501e8] + SLEEFPirates v0.6.20
  [992d4aef] + Showoff v1.0.3
  [a2af1166] + SortingAlgorithms v0.3.1
  [276daf66] + SpecialFunctions v1.5.1
  [aedffcd0] + Static v0.2.5
  [90137ffa] + StaticArrays v1.2.6
  [82ae8749] + StatsAPI v1.0.0
  [2913bbd2] + StatsBase v0.33.8
  [4c63d2b9] + StatsFuns v0.9.8
  [7792a7ef] + StrideArraysCore v0.1.15
  [09ab397b] + StructArrays v0.5.1
  [856f2bd8] + StructTypes v1.7.2
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.4.4
  [8290d209] + ThreadingUtilities v0.4.5
  [bc48ee85] + Tullio v0.2.14
  [3a884ed6] + UnPack v1.0.2
  [c4a57d5a] + UnsafeArrays v1.0.2
  [3d5dd08c] + VectorizationBase v0.19.39
  [efce3f68] + WoodburyMatrices v0.5.3
  [6e34b625] + Bzip2_jll v1.0.8+0
  [83423d85] + Cairo_jll v1.16.1+0
  [2e619515] + Expat_jll v2.2.10+0
  [f5851436] + FFTW_jll v3.3.9+8
  [a3f928ae] + Fontconfig_jll v2.13.93+0
  [d7e528f0] + FreeType2_jll v2.10.4+0
  [559328eb] + FriBidi_jll v1.0.10+0
  [78b55507] + Gettext_jll v0.21.0+0
  [7746bdde] + Glib_jll v2.68.1+0
  [3b182d85] + Graphite2_jll v1.3.14+0
  [2e76f6c2] + HarfBuzz_jll v2.8.1+0
  [e33a78d0] + Hwloc_jll v2.5.0+0
  [1d5cc7b8] + IntelOpenMP_jll v2018.0.3+2
  [dd4b983a] + LZO_jll v2.10.1+0
  [e9f186c6] + Libffi_jll v3.2.2+0
  [d4300ac3] + Libgcrypt_jll v1.8.7+0
  [7add5ba3] + Libgpg_error_jll v1.42.0+0
  [94ce4f54] + Libiconv_jll v1.16.1+1
  [4b2f31a3] + Libmount_jll v2.35.0+0
  [38a345b3] + Libuuid_jll v2.36.0+0
  [856f044c] + MKL_jll v2021.1.1+1
  [efe28fd5] + OpenSpecFun_jll v0.5.5+0
  [2f80f16e] + PCRE_jll v8.44.0+0
  [36c8627f] + Pango_jll v1.47.0+0
  [30392449] + Pixman_jll v0.40.1+0
  [f50d1b31] + Rmath_jll v0.3.0+0
  [02c8fc9c] + XML2_jll v2.9.12+0
  [aed1982a] + XSLT_jll v1.1.34+0
  [4f6342f7] + Xorg_libX11_jll v1.6.9+4
  [0c0b7dd1] + Xorg_libXau_jll v1.0.9+4
  [a3789734] + Xorg_libXdmcp_jll v1.1.3+4
  [1082639a] + Xorg_libXext_jll v1.3.4+4
  [ea2f1a96] + Xorg_libXrender_jll v0.9.10+4
  [14d82f49] + Xorg_libpthread_stubs_jll v0.1.0+3
  [c7cfdc94] + Xorg_libxcb_jll v1.13.0+3
  [c5fb5394] + Xorg_xtrans_jll v1.4.0+3
  [6136c539] + blis_jll v0.8.0+1
  [b53b4c65] + libpng_jll v1.6.38+0
  [0dad84c5] + ArgTools
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [f43a241f] + Downloads
  [9fa8497b] + Future
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [9abbd945] + Profile
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [4607b0f0] + SuiteSparse
  [fa267f1f] + TOML
  [a4e569a6] + Tar
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll
  [deac9b47] + LibCURL_jll
  [29816b5a] + LibSSH2_jll
  [c8ffd9c3] + MbedTLS_jll
  [14a3606d] + MozillaCACerts_jll
  [4536629a] + OpenBLAS_jll
  [83775a58] + Zlib_jll
  [8e850ede] + nghttp2_jll
  [3f19e933] + p7zip_jll
Precompiling project...
  ✓ DiffRules
  ✓ VectorizationBase
  ✓ Distributions
  ✓ SLEEFPirates
  ✓ StrideArraysCore
  ✗ CheapThreads
  ✗ LoopVectorization
  ✗ Gaius
  ✗ Octavian
  ✗ BLASBenchmarksCPU
  11 dependencies successfully precompiled in 29 seconds (116 already precompiled)
  5 dependencies precompiled but different versions are currently loaded. Restart julia to access the new versions
  5 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages

pkg> precompile
Precompiling project...
  ✓ Distributions
  ✓ SLEEFPirates
  ✓ StrideArraysCore
  ✗ CheapThreads
  ✗ LoopVectorization
  ✗ Gaius
  ✗ Octavian
  ✗ BLASBenchmarksCPU
  9 dependencies successfully precompiled in 27 seconds (118 already precompiled)
  3 dependencies precompiled but different versions are currently loaded. Restart julia to access the new versions

ERROR: The following 1 direct dependency failed to precompile:

BLASBenchmarksCPU [5fdc822c-4560-4d20-af7e-e5ee461714d5]

Failed to precompile BLASBenchmarksCPU [5fdc822c-4560-4d20-af7e-e5ee461714d5] to /home/storopoli/.julia/compiled/v1.6/BLASBenchmarksCPU/jl_FWoPsy.
ERROR: LoadError: UndefVarError: dereference not defined
Stacktrace:
 [1] include
   @ ./Base.jl:386 [inlined]
 [2] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
   @ Base ./loading.jl:1235
 [3] top-level scope
   @ none:1
 [4] eval
   @ ./boot.jl:360 [inlined]
 [5] eval(x::Expr)
   @ Base.MainInclude ./client.jl:446
 [6] top-level scope
   @ none:1
in expression starting at /home/storopoli/.julia/packages/CheapThreads/dAYTB/src/CheapThreads.jl:1
ERROR: LoadError: Failed to precompile CheapThreads [b630d9fa-e28e-4980-896d-83ce5e2106b2] to /home/storopoli/.julia/compiled/v1.6/CheapThreads/jl_pr133y.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IOContext{Base.PipeEndpoint}, internal_stdout::IOContext{IOStream}, ignore_loaded_modules::Bool)
    @ Base ./loading.jl:1385
  [3] compilecache(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1329
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1043
  [5] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:936
  [6] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:923
  [7] include
    @ ./Base.jl:386 [inlined]
  [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1235
  [9] top-level scope
    @ none:1
 [10] eval
    @ ./boot.jl:360 [inlined]
 [11] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [12] top-level scope
    @ none:1
in expression starting at /home/storopoli/.julia/packages/LoopVectorization/WDPSy/src/LoopVectorization.jl:1
ERROR: LoadError: Failed to precompile LoopVectorization [bdcacae8-1622-11e9-2a5c-532679323890] to /home/storopoli/.julia/compiled/v1.6/LoopVectorization/jl_zOTShb.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IOContext{Base.PipeEndpoint}, internal_stdout::IOContext{IOStream}, ignore_loaded_modules::Bool)
    @ Base ./loading.jl:1385
  [3] compilecache(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1329
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1043
  [5] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:936
  [6] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:923
  [7] include
    @ ./Base.jl:386 [inlined]
  [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1235
  [9] top-level scope
    @ none:1
 [10] eval
    @ ./boot.jl:360 [inlined]
 [11] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [12] top-level scope
    @ none:1
in expression starting at /home/storopoli/.julia/packages/Octavian/yOhDw/src/Octavian.jl:1
ERROR: LoadError: Failed to precompile Octavian [6fd5a793-0b7e-452c-907f-f8bfe9c57db4] to /home/storopoli/.julia/compiled/v1.6/Octavian/jl_Xw3hW5.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IOContext{Base.PipeEndpoint}, internal_stdout::IOContext{IOStream}, ignore_loaded_modules::Bool)
    @ Base ./loading.jl:1385
  [3] compilecache(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1329
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1043
  [5] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:936
  [6] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:923
  [7] include
    @ ./Base.jl:386 [inlined]
  [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1235
  [9] top-level scope
    @ none:1
 [10] eval
    @ ./boot.jl:360 [inlined]
 [11] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [12] top-level scope
    @ none:1
in expression starting at /home/storopoli/.julia/packages/BLASBenchmarksCPU/gm1om/src/BLASBenchmarksCPU.jl:1

Replace `@assert`s

@asserts can be disabled at certain optimization levels.

Actually, there is a PR open to Base Julia that will disable @asserts by default.

So, instead of using @assert, we should do something else.

Can't run benchmarks with integer matrices?

dilum:/Users/dilum/Downloads/BLASBenchmarks.jl$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.0-DEV.321 (2021-01-17)
 _/ |\__'_|_|_|\__'_|  |  Commit 26a721b28a (3 days old master)
|__/                   |

julia> using BLASBenchmarks

julia> libs = [:Gaius, :Octavian, :OpenBLAS]
3-element Vector{Symbol}:
 :Gaius
 :Octavian
 :OpenBLAS

julia> sizes = [10, 20, 30]
3-element Vector{Int64}:
 10
 20
 30

julia> benchmark_result = runbench(Int64; libs = libs, sizes = sizes)
ERROR: ArgumentError: Sampler for this object is not defined
Stacktrace:
 [1] Random.Sampler(#unused#::Type{VectorizedRNG.Xoshift{2}}, sp::Random.SamplerType{Int64}, #unused#::Val{1})
   @ Random ~/dev/repos/JuliaLang/julia/usr/share/julia/stdlib/v1.7/Random/src/Random.jl:145
 [2] Random.Sampler(rng::VectorizedRNG.Xoshift{2}, x::Random.SamplerType{Int64}, r::Val{1})
   @ Random ~/dev/repos/JuliaLang/julia/usr/share/julia/stdlib/v1.7/Random/src/Random.jl:139
 [3] rand(rng::VectorizedRNG.Xoshift{2}, X::Random.SamplerType{Int64})
   @ Random ~/dev/repos/JuliaLang/julia/usr/share/julia/stdlib/v1.7/Random/src/Random.jl:253
 [4] rand!
   @ ~/dev/repos/JuliaLang/julia/usr/share/julia/stdlib/v1.7/Random/src/Random.jl:271 [inlined]
 [5] rand!
   @ ~/dev/repos/JuliaLang/julia/usr/share/julia/stdlib/v1.7/Random/src/Random.jl:267 [inlined]
 [6] runbench(::Type{Int64}; libs::Vector{Symbol}, sizes::Vector{Int64}, threaded::Bool, A_transform::Function, B_transform::Function, sleep_time::Float64)
   @ BLASBenchmarks ~/Downloads/BLASBenchmarks.jl/src/runbenchmark.jl:147
 [7] top-level scope
   @ REPL[4]:1

Docs: add table with amount of memory required at different matrix sizes

E.g. here's what I'm envisioning:

Matrix Size	Required memory
10 by 10	123 MB
100 by 100	456 MB
1k by 1k	789 MB
10k by 10k	123 GB
50k by 50k	456 GB
73k by 73k	789 GB
100k by 100k	123 TB

I just made those numbers up - obviously they are incorrect. But it would be nice to have a table like that, with the actual numbers.

That way, when people want to run these benchmarks on their own computer, they can just:

Look up how much RAM their computer has
Cross-reference with this table to figure out the biggest matrix size they can use

Also, if someone wants to run the benchmarks on a cluster, they can ask their scheduler (e.g. SLURM) for the necessary amount of memory.

Customize the colors used in the plot?

Is there a way to provide a list of colors for the plot? By default, it picks some greens and blues that look pretty similar.

Rename to BLASBenchmarksCPU.jl and register in the General registry?

Once StrideArrays.jl is registered, what do you think about registering this package as well? That way, the process for a user to run benchmarks becomes really really easy:

using Pkg
Pkg.add("BLASBenchmarksCPU")
using BLASBenchmarksCPU
result = runbench()
plot(result)

Then, we can advertise this on Discourse, Slack, etc. and we can try to get a bunch of different people to run the benchmarks, which will give us a better idea of how Octavian performs on multiple different machines.

Before registering though, I was thinking about renaming the package. I was trying to figure out whether GPU BLAS benchmarking would go into this package or a different package. Looking at some examples (e.g. https://github.com/JuliaGPU/GemmKernels.jl/tree/master/benchmarks/wmma), it looks like the workflow for benchmarking on CUDA devices is different enough from the CPU workflow (e.g. you have to use the NVIDIA profiler). So I'm thinking it makes the most sense to have the GPU benchmarking in a different package.

I was thinking that we could rename this package to BLASBenchmarksCPU.jl or something like that.

And then I'm planning on creating a separate package BLASBenchmarksGPU.jl or maybe BLASBenchmarksCUDA.jl.

How does that sound?

MethodError: no method matching mkl_set_num_threads(::Static.StaticInt)

~~Since this commit in JuliaSIMD/CPUSummary.jl it looks like BLASBenchmarksCPU.jl is broken?~~

Edit: the bug is probably caused by a change in Static.jl where it no longer subtype base types to avoid invalidations (see SciML/Static.jl#73 and the README of Static.jl)

Basically,

CPUSummary.num_cores() returns a Static.StaticInt.
Static.StaticInt is not a subtype of Integer.
mkl_set_num_threads(N::Integer) and all other XXX_set_num_threads(N::Integer) throw a MethodError.

Possible fix:

change the argument to

-function mkl_set_num_threads(N::Integer)
+function mkl_set_num_threads(N::Union{Integer, Static.StaticInt})

likewise for all other XXX_set_num_threads.

I'd be happy to open a PR if this is a reasonable solution!

WARN Missing type for channel "color", using "nominal" instead.

I get this when running plot. Everything still seems to plot fine.

CI is failing on master

`benchmark_result_df` is broken

#32 broke the public function benchmark_result_df(::BenchmarkResult) because the BenchmarkResult struct no longer has a df field.

So we need a new public API that allows users to extract the data from a BenchmarkResult so that they can pass it to the plotting library of their choosing.

Also, we should make sure that for this new public API, we add some tests for it. I forgot to add tests for benchmark_result_df, which is why we didn't notice that we broke it 😂

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Bind benchmark libraries to fixed, standard colors in plots

Easier to compare from run to run, or between different folks' benchmarks, if MKL is always red, OpenBLAS is always black, etc.

A way to disable MKL?

Hey i am running on an ARM system where MKL doesn't exist as such. BLASBenchmarksCPU automatically (init) running mkl related code is not elegant. What would be a good way to handle this?

Support Accelerate

Example from @staticfloat :
https://gist.github.com/staticfloat/164a795e6eb9ae4ec4cfaecd66458aa4

Docs: mention the `sleep_time` kwarg

I imagine that for users that run BLASBenchmarksCPU.runbench on their laptops, it won't be uncommon to see people running into thermal throttling.

Does setting a positive value for sleep_time help address that? If so, we should add a section or page to the docs that explains this.

Precompile failed for Gaius in Julia 1.8+

After trying to download BLASbenchmarkCPU, I eventually get this error:

julia> using BLASbenchmarkCPU
 │ Package BLASBenchmarksCPU not found, but a package named BLASBenchmarksCPU is available from a registry.
 │ Install package?
 │   (@v1.8) pkg> add BLASBenchmarksCPU
 └ (y/n/o) [y]: y

  ✗ BLASBenchmarksCPU
  147 dependencies successfully precompiled in 86 seconds. 15 already precompiled.
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package
[ Info: Precompiling BLASBenchmarksCPU [5fdc822c-4560-4d20-af7e-e5ee461714d5]
ERROR: LoadError: InitError: MethodError: no method matching _print_num_threads_warning(::Static.StaticInt{160}, ::Int64)
Closest candidates are:
  _print_num_threads_warning(::Integer, ::Integer) at ~/.julia/packages/Gaius/xYgac/src/init.jl:12
Stacktrace:
  [1] _print_num_threads_warning()
    @ Gaius ~/.julia/packages/Gaius/xYgac/src/init.jl:9
  [2] __init__()
    @ Gaius ~/.julia/packages/Gaius/xYgac/src/init.jl:2
  [3] _include_from_serialized(pkg::Base.PkgId, path::String, depmods::Vector{Any})
    @ Base ./loading.jl:831
  [4] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt64)
    @ Base ./loading.jl:1039
  [5] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1315
  [6] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1200
  [7] macro expansion
    @ ./loading.jl:1180 [inlined]
  [8] macro expansion
    @ ./lock.jl:223 [inlined]
  [9] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1144
 [10] include
    @ ./Base.jl:419 [inlined]
 [11] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1554
 [12] top-level scope
    @ stdin:1
during initialization of module Gaius
in expression starting at /u/bmgroth/.julia/packages/BLASBenchmarksCPU/VxQHL/src/BLASBenchmarksCPU.jl:1
in expression starting at stdin:1
ERROR: Failed to precompile BLASBenchmarksCPU [5fdc822c-4560-4d20-af7e-e5ee461714d5] to /u/bmgroth/.julia/compiled/v1.8/BLASBenchmarksCPU/jl_7Lr3Al.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:1707
 [3] compilecache
   @ ./loading.jl:1651 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1337
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [6] macro expansion
   @ ./loading.jl:1180 [inlined]
 [7] macro expansion
   @ ./lock.jl:223 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144

Trying to download Gaius also fails:

julia> using Gaius
 │ Package Gaius not found, but a package named Gaius is available from a registry.
 │ Install package?
 │   (@v1.8) pkg> add Gaius
 └ (y/n/o) [y]: y
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/.julia/environments/v1.8/Project.toml`
  [bffe22d1] + Gaius v0.6.7
  No Changes to `~/.julia/environments/v1.8/Manifest.toml`
ERROR: InitError: MethodError: no method matching _print_num_threads_warning(::Static.StaticInt{160}, ::Int64)
Closest candidates are:
  _print_num_threads_warning(::Integer, ::Integer) at ~/.julia/packages/Gaius/xYgac/src/init.jl:12
Stacktrace:
 [1] _print_num_threads_warning()
   @ Gaius ~/.julia/packages/Gaius/xYgac/src/init.jl:9
 [2] __init__()
   @ Gaius ~/.julia/packages/Gaius/xYgac/src/init.jl:2
 [3] _include_from_serialized(pkg::Base.PkgId, path::String, depmods::Vector{Any})
   @ Base ./loading.jl:831
 [4] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt64)
   @ Base ./loading.jl:1039
 [5] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1315
 [6] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [7] macro expansion
   @ ./loading.jl:1180 [inlined]
 [8] macro expansion
   @ ./lock.jl:223 [inlined]
 [9] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
during initialization of module Gaius

Someone else reported this issue at their Github page: MasonProtter/Gaius.jl#91

Unfortunately, it looks as though the Gaius project has been abandoned by the author and likely will not be updated according to the Gaius README:

Note: Gaius is not actively maintained and I do not anticipate doing further work on it. However, you may find it useful as a relatively simple playground for learning about the implementation of linear algebra routines.

Benchmarks to include in `BLASBenchmarks` docs and `Octavian` docs?

I think size ranges of 1:100 and logspace(100, 10_000, N) are good, where N is the number of sampled sizes.

I can add benchmarks for Skylake-X, Haswell (probably not all the way up to 10_000, perhaps stopping at just 1_000), and Tiger Lake.

Having example results would be nice for BLASBenchmarks, while Octavian should showcase its performance.