Design thoughts for build images #3
Conversation
|
|
||
| Part of this is due to downloading and compiling dependencies (e.g. brotli, wolfssl). There's also elapsed time taken in downloading and installing packages for compilation. | ||
|
|
||
| Using Docker containers, we can create a build image whereby the dependencies are already downloaded and installed, and compiled where necessary. Theoretically it also means developers can have the same build environment as the CI jobs, which is good for reproducibility. |
There was a problem hiding this comment.
yes, the docker image being a declarative definition checked into SCM means we can manage/track changes in environment with absolute precision - for me this is docker main use case (eg. in dev) ... production usage is completely different side of the coin.
|
|
||
| ## Design thoughts | ||
|
|
||
| - The build images should be based on Ubuntu Trusty, as Travis CI currently use Ubuntu Trusty as their base image. |
There was a problem hiding this comment.
I think it is reasonable that the first step is to keep things similar (eg. using Ubuntu Trusty) and increment from there.
There was a problem hiding this comment.
Yep. I'm kinda inclined to use Xenial given it's the majority image of the build jobs now. @bagder; no idea if you want to weigh in here about which distribution you'd prefer to build on by default?
There was a problem hiding this comment.
I don't have any strong opinion either way. For our travis CI builds we strive to switch to as recent versions as possible when we can.
There was a problem hiding this comment.
Ok. With this approach we should theoretically be able to run the newest Ubuntu version if desired. For now, let's do Xenial, but with an eye to easily replacing this in future.
| - The build images should be based on Ubuntu Trusty, as Travis CI currently use Ubuntu Trusty as their base image. | ||
| - An alternative could be Centos - any Linux distribution is likely fine. It might be that we choose Trusty to start with then, then move onto Centos for testing different distributions. | ||
| - Another alternative is Xenial as many of the builds explicitly request the Xenial distribution. | ||
| - Build images will be strictly versioned using semver for reproducibility. The "latest" tag will not be used. |
There was a problem hiding this comment.
we have debated 'latest' tag at my company - at best it is something to be considered for production/consumption ... here is a useful link calling it out as an antipattern https://vsupalov.com/docker-latest-tag/
There was a problem hiding this comment.
Given this is all basically for the CI of curl, I don't believe that we should support :latest for these images. There's no upside. 95% of the time people won't even be using these images (though they can if they want!)
|
|
||
| It probably makes sense to have an individual build image for each of these different options, caching the dependencies of each. Docker builds are "cheap". | ||
|
|
||
| It might make sense to have several stages of build: |
|
generally looks good to me ...
|
|
I think for the moment teasing apart the testing from the rest of curl would be a tricky prospect. Probably not best to address that right now. WRT build configuration options, we're hopefully just replicating what the Travis builds already do (that's the initial aim anyway). Once we get parity, it's worth working out in what direction the next steps are (if any) |
|
I've merged this design now, but can still chat here. So at work we basically have a repository per docker image, and each repository is set up and versioned separately. So in this model we'd have for example The benefits of this approach are:
The downsides:
In the github model, individual repos would probably necessitate creating another organization ( The alternative is just having a coarse VERSION at the top level of this repository, then regenerating all images and tagging them with that version. Quick and dirty and some images may not even see any changes at all. It will be simpler for the curl CI build system to operate using a single VERSION number across the board. @xquery - thoughts on any of the above? I think I wanted the first of these, but have talked myself around to the coarse VERSION number, simply because updating a single number in the curl CI builds is very appealing. |
|
I prefer a single repo with every image using a single coarse level version. As these images are not for wide distribution we could consider embedding timestamp into version number which removes any ambiguity tracing back commit to this repo. |
|
Sounds good. I will continue on my current plan then, and hope to get something sensible in place shortly for testing. |
@xquery: would you be able to take a look at this proposal? Given as you're currently also doing some work in the area of Docker.