Design thoughts for build images #3
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # Design | ||
|
|
||
| This document covers the design thoughts behind build images for speeding up Travis CI and a proposal for how to implement things going forward. | ||
|
|
||
| ## Background | ||
|
|
||
| Travis CI currently uses standard Ubuntu containers to build curl. There are lots and lots of entries in the curl build matrix and currently a single job takes a long time to complete. | ||
|
|
||
| Part of this is due to downloading and compiling dependencies (e.g. brotli, wolfssl). There's also elapsed time taken in downloading and installing packages for compilation. | ||
|
|
||
| Using Docker containers, we can create a build image whereby the dependencies are already downloaded and installed, and compiled where necessary. Theoretically it also means developers can have the same build environment as the CI jobs, which is good for reproducibility. | ||
|
|
||
| ## Design thoughts | ||
|
|
||
| - The build images should be based on Ubuntu Trusty, as Travis CI currently use Ubuntu Trusty as their base image. | ||
|
There was a problem hiding this comment. I think it is reasonable that the first step is to keep things similar (eg. using Ubuntu Trusty) and increment from there. There was a problem hiding this comment. Yep. I'm kinda inclined to use Xenial given it's the majority image of the build jobs now. @bagder; no idea if you want to weigh in here about which distribution you'd prefer to build on by default? There was a problem hiding this comment. I don't have any strong opinion either way. For our travis CI builds we strive to switch to as recent versions as possible when we can. There was a problem hiding this comment. Ok. With this approach we should theoretically be able to run the newest Ubuntu version if desired. For now, let's do Xenial, but with an eye to easily replacing this in future. |
||
| - An alternative could be Centos - any Linux distribution is likely fine. It might be that we choose Trusty to start with then, then move onto Centos for testing different distributions. | ||
| - Another alternative is Xenial as many of the builds explicitly request the Xenial distribution. | ||
| - Build images will be strictly versioned using semver for reproducibility. The "latest" tag will not be used. | ||
|
There was a problem hiding this comment. we have debated 'latest' tag at my company - at best it is something to be considered for production/consumption ... here is a useful link calling it out as an antipattern https://vsupalov.com/docker-latest-tag/ There was a problem hiding this comment. Given this is all basically for the CI of curl, I don't believe that we should support :latest for these images. There's no upside. 95% of the time people won't even be using these images (though they can if they want!) |
||
|
|
||
| ## Current set of builds | ||
|
|
||
| The current set of Linux curl CI builds (from .travis.yml) are as follows: | ||
|
|
||
| - gcc-8 (with-gssapi, with-libssh2) CHECKSRC=1 | ||
| - gcc-8 (disable-http, disable-smtp, disable-imap) | ||
| - gcc-8 (enable-ares) | ||
| - gcc-8 (dist:xenial, disable-verbose, no-variadic-macros) NOTESTS=1 | ||
| - gcc-8 (with-brotli) | ||
| - gcc-8 (dist:xenial, with-boringssl) T=novalgrind | ||
| - gcc-8 (dist:xenial, with-wolfssl, without-ssl) | ||
| - gcc-8 (dist:xenial, with-mesalink, without-ssl) | ||
| - clang-7 (dist:xenial) | ||
| - clang-7 (dist:xenial, enable-alt-svc) | ||
| - clang-7 (dist:xenial, with-mbedtls, without-tls) | ||
| - clang-7 (dist:xenial, with-gnutls, without-tls) | ||
| - clang-7 (dist:xenial, disable-threaded-resolver) T=debug | ||
| - clang-7 (dist:xenial, with-nss, without-ssl) T=debug NOTESTS=1 | ||
| - gcc-8 () T=iconv | ||
| - gcc-8 (dist:xenial) T=cmake | ||
| - clang-7 (dist:xenial) T=cmake | ||
| - gcc-8 (dist:xenial) T=coverage | ||
| - gcc-8 (dist:xenial) T=distcheck | ||
| - clang-7 (dist:xenial) T=fuzzer | ||
| - clang-7 (dist:xenial) T=tidy | ||
| - clang-7 (dist:xenial) T=scan-build | ||
| - clang-7 (dist:xenial, many sanitization options) T=debug | ||
|
|
||
| There are several OSX builds which I don't intend to run through Docker. | ||
|
|
||
| It probably makes sense to have an individual build image for each of these different options, caching the dependencies of each. Docker builds are "cheap". | ||
|
|
||
| It might make sense to have several stages of build: | ||
|
|
||
|
|
||
| - Build gcc-8, clang-7 images on top of the base image (Trusty/Xenial/whatever) | ||
| - Build all these images on top of these bases. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, the docker image being a declarative definition checked into SCM means we can manage/track changes in environment with absolute precision - for me this is docker main use case (eg. in dev) ... production usage is completely different side of the coin.