sociomantic-tsunami / cachalot Goto Github PK
View Code? Open in Web Editor NEWSir Cachalot: Public docker images and travis scripts to simplify CI
License: Boost Software License 1.0
Sir Cachalot: Public docker images and travis scripts to simplify CI
License: Boost Software License 1.0
Cachalot uses git 2.13.x for its images:
https://github.com/sociomantic-tsunami/cachalot/blob/v3.x.x/base.Dockerfile#L9
https://github.com/sociomantic-tsunami/cachalot/blob/v2.x.x/base.Dockerfile#L9
and it's using git's ppa:
Line 40 in 7d2bfdb
But since release of git v2.14.x, ppa doesn't host v2.13.x packages anymore, so the build is broken.
We should probably move to the more legacy git versions.
The new repo https://bintray.com/sociomantic-tsunami/ebtree will be populated after sociomantic-tsunami/ebtree#20 is merged and released.
While xenial-built packages should work on bionic, it would be convenient to be able to generate explicitly bionic (18.04) packages for some of our pre-built libraries (e.g. libmxnet
). This will need bionic images (at least for sociomantictsunami/base
, as I imagine dlang
will be problematic for the time being).
There is some weird error when building the fpm
ruby gem:
$ gem install --version=1.10.2 fpm
Successfully installed stud-0.0.23
Successfully installed dotenv-2.7.4
Successfully installed insist-1.0.0
Successfully installed mustache-0.99.8
Successfully installed clamp-1.0.1
Successfully installed cabin-0.9.0
Successfully installed pleaserun-0.0.30
Successfully installed io-like-0.3.0
Successfully installed ruby-xz-0.2.3
Building native extensions. This could take a while...
ERROR: Error installing fpm:
ERROR: Failed to build gem native extension.
current directory: /var/lib/gems/2.3.0/gems/childprocess-2.0.0/ext
/usr/bin/ruby2.3 mkrf_conf.rb
current directory: /var/lib/gems/2.3.0/gems/childprocess-2.0.0/ext
/usr/bin/ruby2.3 -rubygems /usr/share/rubygems-integration/all/gems/rake-10.5.0/bin/rake RUBYARCHDIR=/var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/childprocess-2.0.0 RUBYLIBDIR=/var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/childprocess-2.0.0
/usr/bin/ruby2.3: No such file or directory -- /usr/share/rubygems-integration/all/gems/rake-10.5.0/bin/rake (LoadError)
rake failed, exit code 1
Gem files will remain installed in /var/lib/gems/2.3.0/gems/childprocess-2.0.0 for inspection.
Results logged to /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/childprocess-2.0.0/gem_make.out
The command '/bin/sh -c cd /docker-tmp && ./develbase && rm -fr /docker-tmp' returned a non-zero code: 1
Even when unnecessary (as this repo will only have "Major and Patch releases" because we consider any changes to images, that are not really a "bug fix", risky enough to not move users automatically to them), it's weird to have a different branch scheme than we normally use.
So v1 will be renamed v1.x.x and v2, v2.x.x. There will be no v1.0.x branch for the reasons above, and all releases will be vX.0.Z.
Currently dmd is on 2.070.2
which is at this point over one year old,
We should update to the latest release which ocean can use. (2.077
) at this point.
@leandro-lucarella-sociomantic, @mihails-strasuns-sociomantic
Even when Ubuntu Trusty is supported until 2019, it is already more than 3 years old and not widely used, so we might kill trusty support soon to save some DockerHub space and some Travis CI resources.
sociomantic-tsunami/ocean#102 (comment)
Core issue is that eventually we will need to test at least ocean with multiple compiler version (for example, "current dmd1" + "current dmd-transitional" + "upcoming dmd-transitional" + "vanilla upstream") while using pre-built/cached toolchain docker images as much as possible.
Additionally, it would make travis report (https://travis-ci.org/sociomantic-tsunami/ocean/builds/229419888) much readable if matrix was to show actual compiler versions as opposed to abstract dmd1
/dmd-transitional
.
The approach I am envisioning right now would be to considerably change docker image hierarchy and the way D compiler is installed inside. Right now we have single dlang
image with both compilers and ubuntu release tags. Instead I would introduce base-xenial
and base-precise
images and generate dlang-xenial:v1.080.0
and dlang-precise:v2.070.2.s10
from it with latest
tag matching our current apt-provided versions. What is more important, internally I'd provide single dmd
link to whatever compiler is used not matter the version, simplifying further scripting.
We need to do matrix builds for D1/D2 and more than one D version in general and for different ubuntu distros. For this we might need to use a similar mechanism than build-img
to produce different dockerfiles from a main one to avoid duplication, but keeping a minimal dockerfile just with a
FROM sociomantictsunami/dlang:v1-xxx
COPY ci/ /ci-tmp
RUN /ci-tmp/run
For each flavour could also be an option (although there is still quite some boilerplate there).
Like dump-init:
https://github.com/Yelp/dumb-init
As we want to make sure that :
After all projects deploy tags to bintray apt repositories, we should include those repositories and get the binaries from there instead of building, installing the same default version as before.
The plan for the final v2.0.0 is:
Merge sociomantic-tsunami/ebtree#9 and reate tag v6.0.socio7-rc2
to verify the package uploading really works
Spread this to all the other repos that should produce packages:
Add the current used versions also to the bintray repo
Use the apt repos in cachalot instead of building stuff (#15)
Do a release candidate
Close this issue. Release Cachalot v2.0.0 final
It would be much more convenient if we get the images built and pushed to the docker hub when a tag is created.
For this, we should probably make tags like v1.1
but then push to the docker hub only v1
(or maybe both v1.1
and v1
, but I'm not sure if we want people to stick to a patchlevel version, probably not).
We want to start using uncrustify
to automatically format/verify the code, so we need it installed in the docker images.
Probably Boost, as it is very flexible.
Specially for dmd packages, if there is an apt full-upgrade
or the package is specified somehow in apt install
, is very likely that a new version is installed, as the repo used contains all old and new versions.
Ideally the dmd version should be changed only if a version is specified explicitly to apt
. We could try either a prio for the repo that is lower than 500, or a negative prio, but using the former might completely remove the package (we need to test).
See apt priorities for more details.
Coming from #32, I observed some warnings about empty line continuations, but it seems impossible to mix line continuations and comments. This is just a warning for now, so it's not urgent to fix (although it says it will be an error in the future), but I hope to get a way to do this properly. I asked in this issue that introduced this feature, the only relevant issue I found about this topic.
Travis scripts are duplicated in several projects, like ocean and swarm. They should live in this central repo that should be used as a submodule.
Comes from sociomantic-tsunami/dhtnode#136 (comment) - current base
image is a bit too "fat" for deployment purpose as it install bunch of tools used for development. The real base should simply add sociomantic bintray repos and GPG key and stop at that.
Since the SKS Network is under attack, we should use the new experimental SKS server that has some measures to prevent spamming attacks.
latest
tag is pretty useless as there's no way to know the distribution name from it. We should tag <dist>-latest
instead (or even just <dist>
) may be enough.
Remove trusty support from all projects. Trusty is already more than 3 years old and the latest LTS (Xenial) has already been out there for more than a year, so it should be enough to maintain the latest LTS only now.
When building images, sometimes skipping the cache is needed to update an image when something external changed instead of something in the image definition. We should at least provide some option to use --no-cache
when needed.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.