sociomantic-tsunami / cachalot Goto Github PK

Remove trusty support from all projects. Trusty is already more than 3 years old and the latest LTS (Xenial) has already been out there for more than a year, so it should be enough to maintain the latest LTS only now.

Probably Boost, as it is very flexible.

We want to start using uncrustify to automatically format/verify the code, so we need it installed in the docker images.

There is some weird error when building the fpm ruby gem:

$ gem install --version=1.10.2 fpm
Successfully installed stud-0.0.23
Successfully installed dotenv-2.7.4
Successfully installed insist-1.0.0
Successfully installed mustache-0.99.8
Successfully installed clamp-1.0.1
Successfully installed cabin-0.9.0
Successfully installed pleaserun-0.0.30
Successfully installed io-like-0.3.0
Successfully installed ruby-xz-0.2.3
Building native extensions.  This could take a while...
ERROR:  Error installing fpm:
	ERROR: Failed to build gem native extension.
    current directory: /var/lib/gems/2.3.0/gems/childprocess-2.0.0/ext
/usr/bin/ruby2.3 mkrf_conf.rb
current directory: /var/lib/gems/2.3.0/gems/childprocess-2.0.0/ext
/usr/bin/ruby2.3 -rubygems /usr/share/rubygems-integration/all/gems/rake-10.5.0/bin/rake RUBYARCHDIR=/var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/childprocess-2.0.0 RUBYLIBDIR=/var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/childprocess-2.0.0
/usr/bin/ruby2.3: No such file or directory -- /usr/share/rubygems-integration/all/gems/rake-10.5.0/bin/rake (LoadError)
rake failed, exit code 1
Gem files will remain installed in /var/lib/gems/2.3.0/gems/childprocess-2.0.0 for inspection.
Results logged to /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/childprocess-2.0.0/gem_make.out
The command '/bin/sh -c cd /docker-tmp && ./develbase && rm -fr /docker-tmp' returned a non-zero code: 1

sociomantic-tsunami/ocean#102 (comment)

Core issue is that eventually we will need to test at least ocean with multiple compiler version (for example, "current dmd1" + "current dmd-transitional" + "upcoming dmd-transitional" + "vanilla upstream") while using pre-built/cached toolchain docker images as much as possible.

Additionally, it would make travis report (https://travis-ci.org/sociomantic-tsunami/ocean/builds/229419888) much readable if matrix was to show actual compiler versions as opposed to abstract dmd1/dmd-transitional.

The approach I am envisioning right now would be to considerably change docker image hierarchy and the way D compiler is installed inside. Right now we have single dlang image with both compilers and ubuntu release tags. Instead I would introduce base-xenial and base-precise images and generate dlang-xenial:v1.080.0 and dlang-precise:v2.070.2.s10 from it with latest tag matching our current apt-provided versions. What is more important, internally I'd provide single dmd link to whatever compiler is used not matter the version, simplifying further scripting.

Cachalot uses git 2.13.x for its images:

https://github.com/sociomantic-tsunami/cachalot/blob/v3.x.x/base.Dockerfile#L9
https://github.com/sociomantic-tsunami/cachalot/blob/v2.x.x/base.Dockerfile#L9

and it's using git's ppa:

But since release of git v2.14.x, ppa doesn't host v2.13.x packages anymore, so the build is broken.
We should probably move to the more legacy git versions.

Even when unnecessary (as this repo will only have "Major and Patch releases" because we consider any changes to images, that are not really a "bug fix", risky enough to not move users automatically to them), it's weird to have a different branch scheme than we normally use.

So v1 will be renamed v1.x.x and v2, v2.x.x. There will be no v1.0.x branch for the reasons above, and all releases will be vX.0.Z.

Travis scripts are duplicated in several projects, like ocean and swarm. They should live in this central repo that should be used as a submodule.

Currently dmd is on 2.070.2 which is at this point over one year old,
We should update to the latest release which ocean can use. (2.077) at this point.
@leandro-lucarella-sociomantic, @mihails-strasuns-sociomantic

Comes from sociomantic-tsunami/dhtnode#136 (comment) - current base image is a bit too "fat" for deployment purpose as it install bunch of tools used for development. The real base should simply add sociomantic bintray repos and GPG key and stop at that.

When building images, sometimes skipping the cache is needed to update an image when something external changed instead of something in the image definition. We should at least provide some option to use --no-cache when needed.

After all projects deploy tags to bintray apt repositories, we should include those repositories and get the binaries from there instead of building, installing the same default version as before.

The plan for the final v2.0.0 is:

• Merge sociomantic-tsunami/ebtree#9 and reate tag v6.0.socio7-rc2 to verify the package uploading really works

• Spread this to all the other repos that should produce packages:

  • dlang/dmd#6904
  • sociomantic-tsunami/tangort#13
  • sociomantic-tsunami/dmd-transitional#2
  • sociomantic-tsunami/d1to2fix#19

• Add the current used versions also to the bintray repo

  • dmd1 v1.080.0
  • dmd-transitional 2.070.2.s11
  • libtangort-dmd-dev v1.7.0
  • libebtree6{-dbg,-dev} v6.0.socio6
  • d1to2fix v0.9.0

• Use the apt repos in cachalot instead of building stuff (#15)

• Do a release candidate

• Close this issue. Release Cachalot v2.0.0 final

Like dump-init:
https://github.com/Yelp/dumb-init

As we want to make sure that :

• application are getting PID1
• bash can be used from command line.

We need to do matrix builds for D1/D2 and more than one D version in general and for different ubuntu distros. For this we might need to use a similar mechanism than build-img to produce different dockerfiles from a main one to avoid duplication, but keeping a minimal dockerfile just with a

FROM sociomantictsunami/dlang:v1-xxx
COPY ci/ /ci-tmp
RUN /ci-tmp/run

For each flavour could also be an option (although there is still quite some boilerplate there).

It would be much more convenient if we get the images built and pushed to the docker hub when a tag is created.

For this, we should probably make tags like v1.1 but then push to the docker hub only v1 (or maybe both v1.1 and v1, but I'm not sure if we want people to stick to a patchlevel version, probably not).

Coming from #32, I observed some warnings about empty line continuations, but it seems impossible to mix line continuations and comments. This is just a warning for now, so it's not urgent to fix (although it says it will be an error in the future), but I hope to get a way to do this properly. I asked in this issue that introduced this feature, the only relevant issue I found about this topic.

moby/moby#3898 (comment)

While xenial-built packages should work on bionic, it would be convenient to be able to generate explicitly bionic (18.04) packages for some of our pre-built libraries (e.g. libmxnet). This will need bionic images (at least for sociomantictsunami/base, as I imagine dlang will be problematic for the time being).

Even when Ubuntu Trusty is supported until 2019, it is already more than 3 years old and not widely used, so we might kill trusty support soon to save some DockerHub space and some Travis CI resources.

Specially for dmd packages, if there is an apt full-upgrade or the package is specified somehow in apt install, is very likely that a new version is installed, as the repo used contains all old and new versions.

Ideally the dmd version should be changed only if a version is specified explicitly to apt. We could try either a prio for the repo that is lower than 500, or a negative prio, but using the former might completely remove the package (we need to test).

See apt priorities for more details.

The new repo https://bintray.com/sociomantic-tsunami/ebtree will be populated after sociomantic-tsunami/ebtree#20 is merged and released.

latest tag is pretty useless as there's no way to know the distribution name from it. We should tag <dist>-latest instead (or even just <dist>) may be enough.

Since the SKS Network is under attack, we should use the new experimental SKS server that has some measures to prevent spamming attacks.