Skip to content

Commit 3f9ff72

Browse files
VampireVampire
committed
Use Kotlin DSL for GHA workflow files
1 parent 3a53ca8 commit 3f9ff72

21 files changed

+965
-196
lines changed

‎.github/workflows/README.adoc

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
== The YAML workflow files vs. the `*.main.kts` files
2+
3+
The YAML workflow files are generated from the `*.main.kts` files.
4+
5+
These use the https://github.com/typesafegithub/github-workflows-kt[github-workflows-kt]
6+
Kotlin DSL library to conveniently and type-safely write GitHub Action workflow files.
7+
8+
As there is no official built-in support in GitHub Actions yet until
9+
https://github.com/orgs/community/discussions/15904 is considered, the YAML files
10+
need to be generated manually.
11+
12+
There is a safeguard check in all the generated files that this is not forgotten.
13+
Running a workflow where the according `*.main.kts` produces a different output will
14+
fail the execution. Additionally, the workflow that runs for pull requests checks
15+
the consistency of all the YAML files as not all are run for pull requests.
16+
17+
18+
19+
== Ways to generate the YAML workflow files
20+
21+
There are multiple ways to generate the YAML files and all of them are fine,
22+
but be aware of the last one of the caveats below if you are not using the Gradle method:
23+
24+
* If you are in a `sh` derivate like e.g. `bash` and Kotlin is installed and
25+
available in the `PATH`, you can just call the `*.main.kts` script like any
26+
other shell script:
27+
+
28+
[source,bash]
29+
----
30+
$ ./ci.main.kts
31+
----
32+
33+
* If Kotlin is installed somewhere you can call it with the `*.main.kts` script
34+
as argument:
35+
+
36+
[source,bash]
37+
----
38+
$ path/to/kotlin ci.main.kts
39+
----
40+
41+
* From the IDE you can create a run configuration that executes the `*.main.kts` script.
42+
43+
* There is a Gradle task `preprocessWorkflows` that generates all YAML files from the
44+
according `*.main.kts` files. Additionally, there is also one task per workflow to
45+
only generate that one:
46+
+
47+
[source,bash]
48+
----
49+
$ ./gradlew preprocessCiWorkflow
50+
$ ./gradlew preprocessWorkflows
51+
----
52+
53+
54+
55+
== Caveats
56+
57+
There are currently three known caveats with the approach we follow.
58+
59+
=== https://youtrack.jetbrains.com/issue/KTIJ-16532
60+
61+
If you navigate to a file in the dependencies, only a decompiled file is opened,
62+
even though the source JAR would be available. Also the quick documentation is missing.
63+
64+
This can easily by mitigated by attaching the library to the normal project
65+
dependencies while having the need to navigate the source files or while editing them,
66+
which makes them properly viewable and documentation displayable in the editor.
67+
68+
=== https://youtrack.jetbrains.com/issue/KTIJ-14580
69+
70+
We use `@file:Import` to reduce code duplication by having common code in a common file.
71+
Unfortunately, this triggers a Kotlin IntelliJ plugin bug where the imported file cannot
72+
be loaded properly and so the things supplied by it like dependencies or common functions
73+
are not available. This makes most of the workflow `*.main.kts` files red as hell in the
74+
IDE currently.
75+
76+
To reduce risk for eye-cancer while reading the `*.main.kts` scripts or to be able to
77+
sanely edit them, temporarily add the `@file:DependsOn` from the imported file to the
78+
importing file and wait a second, then remove the line again once you are done.
79+
80+
=== https://youtrack.jetbrains.com/issue/KT-42101
81+
82+
We use `@file:Import` to reduce code duplication by having common code in a common file.
83+
Unfortunately, this triggers a Kotlin bug where the compilation cache becomes confused
84+
if the imported file is changed without the importing file being changed too.
85+
86+
If only the imported file is changed, it could happen that an old version is used,
87+
or it could also happen that classes added by a `@file:DependsOn` in the imported file
88+
are not available to the importing file. So if there was a change in the imported file,
89+
you either need to also change the importing file, or to properly execute the script,
90+
you need to delete the stale entry from the compilation cache which can be found here:
91+
92+
- On Windows the default location is at `%LOCALAPPDATA%\main.kts.compiled.cache\`.
93+
- On Linux the default location is at `$XDG_CACHE_HOME/main.kts.compiled.cache/` or `~/.cache/main.kts.compiled.cache/`.
94+
- On macOS the default location is at `~/Library/Caches/main.kts.compiled.cache/`.
95+
96+
Alternatively, you can also delete the whole cache directory.
97+
98+
Another option is to disable the compilation cache for the execution by setting the
99+
environment variable `KOTLIN_MAIN_KTS_COMPILED_SCRIPTS_CACHE_DIR` or the system property
100+
`kotlin.main.kts.compiled.scripts.cache.dir` to an empty value, depending on the run
101+
method you chose. The Gradle tasks already do that, so when using the Gradle tasks you
102+
do not have this problem and it just works.

‎.github/workflows/ci-K2120.main.kts

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
#!/usr/bin/env kotlin
2+
3+
@file:Import("common.main.kts")
4+
5+
import io.github.typesafegithub.workflows.domain.triggers.WorkflowDispatch
6+
import io.github.typesafegithub.workflows.dsl.workflow
7+
8+
workflow(
9+
name = "CI-K2.1.20",
10+
on = listOf(WorkflowDispatch()),
11+
sourceFile = __FILE__
12+
) {
13+
buildJob(
14+
id = "build",
15+
name = "Build"
16+
) {
17+
run(
18+
name = "Build",
19+
command = "./gradlew build -Pkotlin.version=2.1.20-Beta1"
20+
)
21+
}
22+
23+
buildJob(
24+
id = "build-examples",
25+
name = "Build Examples"
26+
) {
27+
run(
28+
name = "Build Examples",
29+
command = """
30+
./gradlew publishToMavenLocal -Pkotlin.version=2.1.20-Beta1
31+
cd examples/
32+
./gradlew build -Pkotlin.version=2.1.0-RC2 -Pseskar.version=3.50.1
33+
""".trimIndent()
34+
)
35+
}
36+
}

‎.github/workflows/ci-K2120.yaml

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# This file was generated using Kotlin DSL (.github/workflows/ci-K2120.main.kts).
2+
# If you want to modify the workflow, please change the Kotlin file and regenerate this YAML file.
3+
# Generated with https://github.com/typesafegithub/github-workflows-kt
4+
5+
name: 'CI-K2.1.20'
6+
on:
7+
workflow_dispatch: {}
8+
jobs:
9+
check_yaml_consistency:
10+
name: 'Check YAML consistency'
11+
runs-on: 'ubuntu-latest'
12+
steps:
13+
- id: 'step-0'
14+
name: 'Check out'
15+
uses: 'actions/checkout@v4'
16+
- id: 'step-1'
17+
name: 'Execute script'
18+
run: 'rm ''.github/workflows/ci-K2120.yaml'' && ''.github/workflows/ci-K2120.main.kts'''
19+
- id: 'step-2'
20+
name: 'Consistency check'
21+
run: 'git diff --exit-code ''.github/workflows/ci-K2120.yaml'''
22+
build:
23+
name: 'Build'
24+
runs-on: 'ubuntu-latest'
25+
needs:
26+
- 'check_yaml_consistency'
27+
steps:
28+
- id: 'step-0'
29+
name: 'Checkout the repo'
30+
uses: 'actions/checkout@v4'
31+
- id: 'step-1'
32+
name: 'Validate Gradle Wrapper'
33+
uses: 'gradle/actions/wrapper-validation@v4'
34+
- id: 'step-2'
35+
name: 'Set up JDK 21'
36+
uses: 'actions/setup-java@v4'
37+
with:
38+
java-version: '21'
39+
distribution: 'zulu'
40+
- id: 'step-3'
41+
name: 'Setup Gradle'
42+
uses: 'gradle/actions/setup-gradle@v4'
43+
- id: 'step-4'
44+
name: 'Build'
45+
run: './gradlew build -Pkotlin.version=2.1.20-Beta1'
46+
build-examples:
47+
name: 'Build Examples'
48+
runs-on: 'ubuntu-latest'
49+
needs:
50+
- 'check_yaml_consistency'
51+
steps:
52+
- id: 'step-0'
53+
name: 'Checkout the repo'
54+
uses: 'actions/checkout@v4'
55+
- id: 'step-1'
56+
name: 'Validate Gradle Wrapper'
57+
uses: 'gradle/actions/wrapper-validation@v4'
58+
- id: 'step-2'
59+
name: 'Set up JDK 21'
60+
uses: 'actions/setup-java@v4'
61+
with:
62+
java-version: '21'
63+
distribution: 'zulu'
64+
- id: 'step-3'
65+
name: 'Setup Gradle'
66+
uses: 'gradle/actions/setup-gradle@v4'
67+
- id: 'step-4'
68+
name: 'Build Examples'
69+
run: |-
70+
./gradlew publishToMavenLocal -Pkotlin.version=2.1.20-Beta1
71+
cd examples/
72+
./gradlew build -Pkotlin.version=2.1.0-RC2 -Pseskar.version=3.50.1

‎.github/workflows/ci-K2120.yml

Lines changed: 0 additions & 45 deletions
This file was deleted.

‎.github/workflows/ci.main.kts

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
#!/usr/bin/env kotlin
2+
3+
@file:Import("common.main.kts")
4+
5+
import io.github.typesafegithub.workflows.actions.actions.Checkout
6+
import io.github.typesafegithub.workflows.domain.RunnerType.UbuntuLatest
7+
import io.github.typesafegithub.workflows.domain.triggers.PullRequest
8+
import io.github.typesafegithub.workflows.domain.triggers.Push
9+
import io.github.typesafegithub.workflows.dsl.workflow
10+
11+
workflow(
12+
name = "CI",
13+
on = listOf(
14+
Push(
15+
branches = listOf("master")
16+
),
17+
PullRequest()
18+
),
19+
sourceFile = __FILE__
20+
) {
21+
job(
22+
id = "check_all_workflow_yaml_consistency",
23+
name = "Check all Workflow YAML Consistency",
24+
runsOn = UbuntuLatest
25+
) {
26+
uses(
27+
name = "Checkout the repo",
28+
action = Checkout()
29+
)
30+
run(
31+
name = "Regenerate all Workflow YAMLs",
32+
command = """find .github/workflows -mindepth 1 -maxdepth 1 -name '*.main.kts' -exec {} \;"""
33+
)
34+
run(
35+
name = "Check for Modifications",
36+
command = """
37+
git add --intent-to-add .
38+
git diff --exit-code
39+
""".trimIndent()
40+
)
41+
}
42+
43+
buildJob(
44+
id = "build",
45+
name = "Build"
46+
) {
47+
run(
48+
name = "Build",
49+
command = "./gradlew build preprocessWorkflows"
50+
)
51+
}
52+
53+
buildJob(
54+
id = "build-examples",
55+
name = "Build Examples"
56+
) {
57+
run(
58+
name = "Build Examples",
59+
command = """
60+
./gradlew publishToMavenLocal
61+
cd examples/
62+
./gradlew build
63+
""".trimIndent()
64+
)
65+
}
66+
67+
buildJob(
68+
id = "test",
69+
name = "Test"
70+
) {
71+
run(
72+
name = "Tests and benchmarks",
73+
command = "./gradlew allTests -Ptest=true"
74+
)
75+
}
76+
}

0 commit comments

Comments
 (0)