-
Notifications
You must be signed in to change notification settings - Fork 335
Fix parallel builds and use them in gh workflow #797
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
7156224
to
eb63ef9
Compare
While we do not have proper benchmarks from the Github Actions, we can see that adding
The The few seconds here and there are not significant, but reducing |
FWIW, on the faster runners (autoreconf about 20 seconds; the slower runners take about 25 seconds for autoreconf) with I am not advocating for or against adding |
d26577c
to
eb63ef9
Compare
change looks good to me. |
Require gettext >= 0.19.1 (from 2014-06-10), which both works properly when building on many cores, and is also the first gettext release whose po/Makefile.in.in allows disabling rebuilding of the *.po files on normal (non dist target) builds. The new po/Makefile.in.in rules fix "make distcheck" consistently failing to remove ../../../po/libgphoto2-6.pot when running "make -j$(nproc) distcheck" on a 12 core system: rm: cannot remove '../../../po/libgphoto2-6.pot': Permission denied The "make dist" (and "make distcheck") targets will still update the po files by default, but that could also be disabled in po/Makevars if we wanted to separate updating the *.po files for translation from "make dist" or "make distcheck" at any time in the future. As gettext-0.19.1 was released almost 8 years ago, this is the newest build tool release requirement for bulding libgphoto2. If you ever need to build a 2022 or later libgphoto2 from git on a system with pre 0.19.1 gettext, you can can always prepare a tarball running "make dist" on a system with gettext 0.19.1 or later, and build that tarball on the older machine.
Use parallel make for builds. We use the GNU coreutils' "nproc" utility to determine the number of cores available for us to use. We then tell make to launch up to that number of processes (-j$(nproc)) in parallel, but only if the system load is below $(nproc) (-l$(nproc)). That should make reasonably good use of however many cores github makes available to us, while not overloading the build server or the build server VM.
The old gettext 0.18.3
po/Makefile.in.in
has problems when used with parallel make (make -jN
with N>1). Runningmake -j$(nproc) -l$(nproc) distcheck
on a 12 core system fails reproducibly.gettext 0.19.1 has apparently fixed those problems with parallel builds, as running
make -j$(nproc) -l$(nproc) distcheck
on the same 12 core system reproducibly succeeds with gettext 0.19.1'spo/Makefile.in.in
fileWhen parallel builds work, we can now also make use of them during the CI check github Action workflow, so we change all
make
commands there tomake -j$(nproc) -l$(nproc)
.This is a PR to trigger and test the CI builds, and to allow discussion about requiring gettext 0.19.1 which is a release not even 8 years old, making it the youngest build tool release required to build libgphoto2.