joel-dahne / yggdrasil

Collection of builder repositories for BinaryBuilder.jl

Yggdrasil

Build Status:

This repository contains recipes for building binaries for Julia packages using BinaryBuilder.jl.

Contributing

To contribute a new recipe, you can either

• use BinaryBuilder.run_wizard(), which will automatically open a pull request to this repository after a successfull build for all requested platforms
• Copy another build recipe using it as a template, and then open a manual pull request to this repository

Yggdrasil builds the tarballs using master version of BinaryBuilder.jl, which requires Julia 1.3.0 or later versions. Note that this BinaryBuilder.jl version has some differences compared to v0.1.4 and the builders generated are slightly different. You are welcome to contribute builders written for BinaryBuilder.jl v0.1.4, but they will likely need minor adjustements.

Buildkite CI is used to test that the builders can successfully produce the tarballs.

If you prefer to test your manual buildscript before opening the pull request, we suggest installing BinaryBuilder.jl on Julia 1.3 or any following release and running julia --color=yes build_tarballs.jl --verbose --debug locally. On MacOS, you will need to have docker installed for this to work.

Updating the version of an existing builder

To trigger the build of a new version of the upstream package simply open a pull request to update the builder as necessary. This usually boils down to only updating the version number and the source (e.g., URL and hash for an archive, or the revision for a git repository), but in some cases more changes may be needed.

Using the generated tarballs

JLL packages

The last versions of BinaryBuilder make use of the Artifacts system shipping in Julia 1.3. This means that BinaryBuilder no longer generates build.jl files that are placed into your Julia package's deps/ folder, but instead generates whole Julia packages (known colloquially as "jll" packages) that are placed within the JuliaBinaryWrappers organization. Merged pull requests to Yggdrasil result in new versions of these wrapper packages being generated, uploaded and registered, allowing your client Julia code to simply invoke using LibFoo_jll to get ahold of your binaries with no need for a Pkg.build() step. (This will, of course, only be the case for Julia 1.3+).

We encourage Julia developers to use JLL packages for their libraries. Read the documention of BinaryBuilder to learn how to use them.

Here are a few examples of pull requests of Julia packages switching to using JLL package to provide the prebuilt binaries to the users:

• Cairo.jl
• FFTW.jl
• GSL.jl
• Gtk.jl
• Rsvg.jl

You can read more about the Artifacts system and how it is important for reproducibility in this post on Julia's blog "Pkg + BinaryBuilder -- The Next Generation".

BinaryProvider.jl

We hope we convinced you about why it is important to switch to JLL packages. However, if you really need to support Julia v1.2 or previous versions, you should keep using BinaryProvider.jl. As explained in the previous section, a build.jl file is not generated after the build anymore as that is not used for JLL packages, instead you would need to generate it yourself: In the top-directory of Yggdrasil there is a script to do this: generate_buildjl.jl. It takes three positional arguments:

• the path to the build_tarballs.jl script
• the owner/name of the repository where the tarballs have been uploaded. If omitted, this defaults to JuliaBinaryWrappers/BuilderName_jll.jl
• the tag name where the tarballs have been uploaded. If omitted, this defaults to the latest version of the JLL package in the General registry. If there are no versions of the package in the registry, the script will fail.

For example, to get the build.jl file for the latest version of Zlib you can run the following command:

julia --color=yes generate_buildjl.jl Z/Zlib/build_tarballs.jl

If instead you want to get the build_tarballs.jl file for the tag named Zlib-v1.2.11+6 you have to run the command

julia --color=yes generate_buildjl.jl Z/Zlib/build_tarballs.jl JuliaBinaryWrappers/Zlib_jll.jl Zlib-v1.2.11+6

Note: you have to manually add prefix as the first argument to all Product constructors in the generated build.jl files. This is necessary because the syntax between BinaryBuilder v0.2+ and BinaryProvider has diverged.

Remember that you will also need the build.jl files for all direct and indirect dependencies.

Here are a few examples of packages using this system to install their libraries:

• FFMPEG.jl;
• MySQL.jl uses JLL packages for Julia v1.3+ and build.jl scripts with BinaryProvider.jl for previous releases.