openttd / compilefarm

Deprecated by GitHub Actions - Compile Farm related files to build OpenTTD for all kind of platforms

OpenTTD-CF

A collection of Dockerfiles which are prepared to compile OpenTTD in all kind of flavours.

The images created from these Dockerfiles are used by the Compile Farm of OpenTTD to produce all the official releases since 1.9.0. This includes binaries like nightlies, experimentals, release-candidates, forks, etc.

Requirements

Install the latest Docker CE. At least 17.05 is needed.

Building

The images are created and published on Docker Hub, and can be found under:

https://hub.docker.com/r/openttd/compile-farm/ https://hub.docker.com/r/openttd/compile-farm-ci/

They can simply be pulled like:

docker pull openttd/compile-farm:linux-debian-stretch-amd64-gcc
docker pull openttd/compile-farm-ci:linux-amd64-gcc

If you want to build them yourself, things get a bit more complicated. To avoid code duplication, best is to look in azure-pipelines-linux.yml for the full set of what is being created.

Automated building

This repository uses Azure Pipelines to create all the required images. When a Pull Request is made, it will validate that all Linux containers still build. If a Pull Request is merged in master, it will automatically publish the new Linux containers.

Every week master is rebuild, both the Linux containers and the Windows dependencies.

• azure-pipelines-linux.yml defines how the Linux Containers are build.
• azure-pipelines-windows.yml defines how the Windows Dependencies are build.

Running (releases)

To compile your source via a Docker, simply create a new directory, say 'build', and put in there a folder called 'source' with the source you would like to compile. The Docker will put some temporary files in the 'build' folder, but mostly it will work inside the 'source' folder.

Now run from your build folder:

docker run --rm --user=`id -u`:`id -g` -v $(realpath $(pwd)):/workdir openttd/compile-farm:<your flavor>

This will produce the resulting binaries in the bundles directory of your source directory. Depending on the target, this can be a zip, deb, ..

Running (CI)

The dockers marked 'ci' are meant for code validation and tests (like regression-check). These don't produce any binaries, but only validate that the new code is correct. It follows the above chapter exactly, with a few key differences:

• The image name is not 'openttd/compile-farm', but 'openttd/compile-farm-ci'.
• When running the docker, it doesn't produce any output in bundles folder.

Listing current images

To list all the images you currently have on your system, simply run:

docker images openttd/compile-farm --format "{{.Tag}}"

Or for all the CI images:

docker images openttd/compile-farm-ci --format "{{.Tag}}"

Targets

In this repository you see different folders, each for their own target.

A short walkthrough:

• ci-commit-checker: scripts to validate commit / commit messages.
• ci-linux-*: targets that only validate sources on errors.
• release-linux-deb-gcc: targets that produce .deb files (Debian and Ubuntu).
• release-linux-generic-gcc: targets that produce tarballs (for any Linux OS, build with GCC).

compilefarm's Issues

Commit checker should verify generated files are updated correctly

There are several generated files checked in to the main repository out of convenience:

• Makefiles
• Visual Studio project files
• Scripting system headers (src/script/(ai,game,template)/)

These files should always be kept updated when a commit modifies the source files they are generated from. The commit checker should verify that re-generating these files do not produce a diff, and fail if there is a difference.

Other generated files checked in are baseset definition files, which are built from a template and translations. Since these are closely tied to translations they should likely be handled by Eints, out of scope of this ticket.

Lastly there is openttd.grf, which requires additional tooling to generate, and might have binary differences depending on the exact versions of the tools used. It should possibly be maintained by a separate job, out of scope of this ticket.

Add Linux Generic binaries

There is an interest to build generic linux binaries (both for nightlies as for releases). Currently we don't have any. We used to have those.

Few things to keep in mind:

• The older the libc version, the better for compatability
• Compiling most libraries static would be most ideal (especially things like ICU)

When looking at old releases, < 2% uses these generic linux binaries, to give an idea of the userbase that is interested.

For releases we do produce Debian/Ubuntu binaries. This issue is NOT about those. This is about linux binaries which will run on "most" linux systems out-of-the-box.

mingw64 support

We should have a docker that validates if mingw64 compiles still work.

If we have a docker, we can attach it to the CI, to ensure we don't break mingw by accident.

release-docs doesn't create NoAI / NoGS docs

7c0d24d#diff-0925013e17fd6908cd3214d19b78915eR41

We used to also produce docs for NoAI and NoGS. Please make this a thing again.

Commit checker always expect origin/master, even for non master PRs

For backports we open PRs against origin/release/XXX but commit checker assumes origin/master, and fails.

Linux error messages contain lots of docker "errors"

As best I can tell, this "log preview" just displays the first N lines of stderr

It would be nice if the docker errors when building the image were able to be suppressed in some way, as they only serve to hide the actual compile error message people are interested in