-
Notifications
You must be signed in to change notification settings - Fork 0
Add pkg-precompile.jl
script
#1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
1d7e140
7f727ab
17ef5c8
a5da306
b13f82d
fa3d5cf
1bad93b
69170e0
c13bacb
8565988
ec3b96a
c0f5aa3
5ed93cf
89cd162
577e4ee
f86c140
666f777
ad9c40f
2d239f9
04ad700
bed42af
3ef5839
06b2b0e
aa472d8
03a4cea
df0cfd2
164922c
a1d667c
bbf07a9
2564169
8a0e022
48c7004
b0e8ceb
1b290af
3a0bdc0
6d4025c
b43042a
a098c3e
0fd7156
45bc4af
9d3102b
6eb62f9
0b0b85e
a27a5ce
959f1a1
b162ce8
c9f5879
fc3a9e6
a5a7cf5
ae48682
2c02318
868dbeb
bd4ba4e
7a025a2
55054ee
475f846
ed0fd56
0518e35
9e9b265
f2684a5
98964ab
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
--- | ||
name: CI | ||
on: | ||
pull_request: | ||
paths: | ||
- "pkg-precompile.jl" | ||
- "test/**" | ||
- ".github/workflows/CI.yaml" | ||
push: | ||
branches: | ||
- main | ||
tags: ["*"] | ||
|
||
jobs: | ||
version: | ||
name: Resolve Julia Versions | ||
# These permissions are needed to: | ||
# - Checkout the Git repository (`contents: read`) | ||
permissions: | ||
contents: read | ||
runs-on: ubuntu-latest | ||
outputs: | ||
json: ${{ steps.julia-version.outputs.resolved-json }} | ||
steps: | ||
- uses: actions/checkout@v4 # Needed for "min" to access the Project.toml | ||
- uses: julia-actions/[email protected] | ||
id: julia-version | ||
with: | ||
versions: | | ||
- min # Oldest supported version | ||
- lts # Long-Term Stable | ||
- 1.10.0 # Earliest 1.10 release | ||
- 1.10 # Latest 1.10 release | ||
- 1.11.0 # Earliest 1.11 release | ||
- 1.11 # Latest 1.11 release | ||
- 1.12-nightly | ||
project: test | ||
if-missing: error | ||
|
||
test: | ||
name: Julia ${{ matrix.version }} - ${{ matrix.os }} | ||
needs: version | ||
# These permissions are needed to: | ||
# - Delete old caches: https://github.com/julia-actions/cache#usage | ||
permissions: | ||
actions: write | ||
contents: read | ||
runs-on: ${{ matrix.os }} | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
version: ${{ fromJSON(needs.version.outputs.json) }} | ||
os: | ||
- ubuntu-24.04 | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- uses: julia-actions/setup-julia@v2 | ||
with: | ||
version: ${{ matrix.version }} | ||
- uses: julia-actions/cache@v1 | ||
- name: Instantiate | ||
shell: julia --color=yes --project=test {0} | ||
run: | | ||
using Pkg | ||
Pkg.instantiate() | ||
- name: Test | ||
run: julia --color=yes --project=test test/runtests.jl |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
test/Manifest.toml | ||
test/*/Manifest.toml |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,56 @@ | ||
# julia-container-scripts | ||
# julia-container-scripts | ||
|
||
Provides useful scripts for creating Julia container images. | ||
|
||
## `pkg-precompile.jl` | ||
|
||
The `pkg-precompile.jl` script supports creating Julia precompilation cache (`.ji`) files in a [build container cache mount](https://docs.docker.com/reference/dockerfile/#run---mounttypecache). By utilizing the build cache mount we can reuse precompilation files between builds which signficantly improves reduces Docker build times. A complete `Dockerfile` example can be seen below: | ||
|
||
```Dockerfile | ||
ARG JULIA_VERSION=1.11.4 | ||
FROM julia:${JULIA_VERSION}-bookworm AS julia-base | ||
|
||
# Disable automatic package precompilation. We'll control when packages are precompiled. | ||
ENV JULIA_PKG_PRECOMPILE_AUTO="0" | ||
|
||
# Add registries required for instantiation | ||
RUN julia --color=yes -e 'using Pkg; Pkg.Registry.add("General")' | ||
|
||
# Limit Docker layer invalidation by only copying the Project.toml/Manifest.toml files. | ||
ENV JULIA_PROJECT="/project" | ||
COPY Project.toml *Manifest.toml ${JULIA_PROJECT}/ | ||
|
||
# TODO: Delete this optional statement if your Project.toml does not include the field | ||
# "name" or you don't care about supporting the Julia versions listed below. | ||
# | ||
# Julia 1.10.0 - 1.10.6 and 1.11.0 require this source file to be present when | ||
# instantiating a named Julia project. | ||
Comment on lines
+26
to
+27
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 1.10.8-1.10 require this for precompilation too. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I determined the versions which required this from using the test suite. We are testing for this in with the |
||
RUN curl -fsSLO https://raw.githubusercontent.com/beacon-biosignals/julia-container-scripts/refs/tags/v1/gen-pkg-src.jl && \ | ||
chmod +x gen-pkg-src.jl && \ | ||
./gen-pkg-src.jl && \ | ||
rm gen-pkg-src.jl | ||
|
||
# Instantiate the Julia project environment and avoid precompiling. Ensure we perform a | ||
# registry update here as changes to the Project.toml/Manifest.toml do not invalidate the | ||
# Docker layer which added the registry. | ||
RUN julia --color=yes -e 'using Pkg; Pkg.Registry.update(); Pkg.instantiate(); Pkg.build()' | ||
|
||
# TODO: Delete this optional statement if you don't care about Julia 1.10 support or combine | ||
# this statement with instantiate above to avoid bloating image size. | ||
# | ||
# Use a fixed modification time for all files in "packages" to avoid unnecessary precompile | ||
# cache invalidation on Julia 1.10. | ||
RUN julia -e 'VERSION < v"1.11" || exit(1)' && \ | ||
find "$(julia -e 'println(DEPOT_PATH[1])')/packages" -exec touch -m -t 197001010000 {} \; | ||
|
||
# Precompile project dependencies using a Docker cache mount which persists between builds. | ||
RUN --mount=type=cache,id=julia-depot,sharing=shared,target=/mnt/julia-depot \ | ||
curl -fsSLO https://raw.githubusercontent.com/beacon-biosignals/julia-container-scripts/refs/tags/v1/pkg-precompile.jl && | ||
chmod +x pkg-precompile.jl && \ | ||
./pkg-precompile.jl "/mnt/julia-depot" && \ | ||
rm pkg-precompile.jl | ||
|
||
# Copy files necessary to load package and perform the first initialization. | ||
COPY src ${JULIA_PROJECT}/src | ||
RUN julia -e 'using Pkg; name = Pkg.Types.EnvCache().project.name; Pkg.precompile(name; timing=true); Base.require(Main, Symbol(name))' | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
#!/usr/bin/env -S julia --color=yes | ||
|
||
using Pkg | ||
|
||
env = Pkg.Types.EnvCache() | ||
project_file = env.project_file | ||
project_name = env.project.name | ||
|
||
if project_name !== nothing | ||
entry_file = joinpath(dirname(project_file), "src", "$(project_name).jl") | ||
|
||
if !isfile(entry_file) | ||
mkdir(dirname(entry_file)) | ||
|
||
# Define an empty module to avoid warnings such as: | ||
# ``` | ||
# WARNING: --output requested, but no modules defined during run | ||
# ``` | ||
open(entry_file, "w") do io | ||
println(io, "module $(project_name)") | ||
println(io, "end") | ||
end | ||
end | ||
else | ||
@warn "Julia project is unnamed and does not require an entry source file" | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe
julia-container-build-scripts
orjulia-docker-build-scripts
is a better name?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is clear enough as far as I'm concerned.