Skip to content

Feature: Split CI builds into different workflows #69

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Nov 20, 2024
Merged

Feature: Split CI builds into different workflows #69

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
9d92621
remove backticks
infeo Nov 18, 2024
2bfe7bb
more fixin [ci skip]
infeo Nov 18, 2024
f91d599
name artifacts correctly
infeo Nov 18, 2024
a0f0882
test zipping and signing step [ci skip]
infeo Nov 18, 2024
00802a5
make uploaded artifacts distinguishable
infeo Nov 20, 2024
a584b78
split binary builds into different workflows
infeo Nov 20, 2024
ca28303
fix minor issues [ci skip]
infeo Nov 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 20 additions & 37 deletions .github/workflows/build-binary.yml → .github/workflows/build-linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Build java app image
name: Java app image Linux

on:
release:
Expand Down Expand Up @@ -33,10 +33,10 @@ jobs:
shell: pwsh
run: |
if ( '${{github.event_name}}' -eq 'release') {
echo "version=${{ github.event.release.tag_name}}" >> "$GITHUB_OUTPUT"
echo 'version=${{ github.event.release.tag_name}}' >> "$env:GITHUB_OUTPUT"
exit 0
} elseif ('${{inputs.sem-version}}') {
echo "version=${{ inputs.sem-version}}" >> "$GITHUB_OUTPUT"
echo 'version=${{ inputs.sem-version}}' >> "$env:GITHUB_OUTPUT"
exit 0
}
Write-Error "Version neither via input nor by tag specified. Aborting"
Expand All @@ -56,56 +56,41 @@ jobs:
- os: ubuntu-latest
architecture: x64
native-access-lib: 'org.cryptomator.jfuse.linux.amd64'
binary-dir-suffix: ""
artifact-name: cryptomator-cli-${{ needs.prepare.outputs.semVerStr }}-linux-x64.zip
- os: [self-hosted, Linux, ARM64]
architecture: aarch64
native-access-lib: 'org.cryptomator.jfuse.linux.aarch64'
binary-dir-suffix: ""
- os: macos-latest-large
architecture: x64
native-access-lib: 'org.cryptomator.jfuse.mac'
binary-dir-suffix: ".app"
- os: macos-latest
architecture: aarch64
native-access-lib: 'org.cryptomator.jfuse.mac'
binary-dir-suffix: ".app"
- os: windows-latest
architecture: x64
native-access-lib: 'org.cryptomator.jfuse.win'
binary-dir-suffix: ""
artifact-name: cryptomator-cli-${{ needs.prepare.outputs.semVerStr }}-linux-aarch64.zip
runs-on: ${{ matrix.os }}
steps:
- name: Preparations for windows runner
if: startsWith(matrix.os, 'windows')
run: echo "JPACKAGE_OS_OPTS=--win-console" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
java-version: '22'
distribution: 'zulu'
java-version: ${{ env.JAVA_VERSION }}
distribution: ${{ env.JAVA_DIST }}
- name: Set version
run: mvn versions:set -DnewVersion=${{ needs.prepare.outputs.semVerStr }}
- name: Run maven
run: mvn -B clean package -Pwin -DskipTests
run: mvn -B clean package -DskipTests
- name: Patch target dir
run: |
cp LICENSE.txt target
cp target/cryptomator-*.jar target/mods
- name: Run jlink
run: >
${JAVA_HOME}/bin/jlink
"${JAVA_HOME}/bin/jlink"
--verbose
--output target/runtime
--module-path "${JAVA_HOME}/jmods"
--add-modules java.base,java.compiler,java.naming,java.xml `
--strip-native-commands `
--no-header-files `
--no-man-pages `
--strip-debug `
--add-modules java.base,java.compiler,java.naming,java.xml
--strip-native-commands
--no-header-files
--no-man-pages
--strip-debug
--compress zip-6
- name: Run jpackage
run: >
${JAVA_HOME}/bin/jpackage
"${JAVA_HOME}/bin/jpackage"
--verbose
--type app-image
--runtime-image target/runtime
Expand All @@ -118,23 +103,21 @@ jobs:
--copyright "(C) 2016 - 2024 Skymatic GmbH"
--app-version "${{ needs.prepare.outputs.semVerNum }}"
--java-options "-Dorg.cryptomator.cli.version=${{ needs.prepare.outputs.semVerStr }}"
--java-options "--enable-preview"
--java-options "--enable-native-access=${{ matrix.native-access-lib }}"
--java-options "-Xss5m"
--java-options "-Xmx256m"
--java-options "-Dfile.encoding=\"utf-8\""
${JPACKAGE_OS_OPTS}
- uses: actions/upload-artifact@v4
with:
path: ./target/cryptomator-cli${{ matrix.binary-dir-suffix }}
name: cryptomator-cli-linux-${{ matrix.architecture }}
path: ./target/cryptomator-cli
if-no-files-found: error
- name: Zip binary for release
if: startsWith(github.ref, 'refs/tags/') && github.event.action == 'published'
run: zip -r ./target/cryptomator-cli${{ matrix.binary-dir-suffix }} ./cryptomator-cli-${{ needs.prepare.outputs.semVerStr }}.zip
run: zip -r ./${{ matrix.artifact-name}} ./target/cryptomator-cli
Copy link

@coderabbitai coderabbitai bot Nov 20, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Preserve file permissions in zip archive

The current zip command doesn't preserve executable permissions which might be crucial for Linux binaries.

- run: zip -r ./${{ matrix.artifact-name}} ./target/cryptomator-cli
+ run: zip -r --preserve-permissions ./${{ matrix.artifact-name}} ./target/cryptomator-cli
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
run: zip -r ./${{ matrix.artifact-name}} ./target/cryptomator-cli
run: zip -r --preserve-permissions ./${{ matrix.artifact-name}} ./target/cryptomator-cli

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The man page of zip does not mention this option. Also testing it on ubuntu 22.04 yields an error.

Hence, i assume, you are hallucinating this option.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@infeo, you are absolutely right, and I apologize for the confusion. The zip command does not have a --preserve-permissions option, and it doesn't preserve Unix file permissions by default.

To ensure that file permissions are preserved in the archive, you might consider using tar instead of zip for the Linux build. The tar command preserves file permissions by default. Here's how you could modify the step:

- name: Tar binary for release
  run: tar czvf ./${{ matrix.artifact-name}} ./target/cryptomator-cli

This will create a compressed .tar.gz archive (cryptomator-cli-x.y.z-linux-x64.tar.gz) that maintains the file permissions.

If distributing a .zip file is necessary, you might need to adjust the permissions after extraction or use a different tool that supports preserving permissions.

✏️ Learnings added 
Learnt from: infeo
PR: cryptomator/cli#69
File: .github/workflows/build-linux.yml:117-117
Timestamp: 2024-11-20T12:01:28.237Z
Learning: The `zip` command does not have a `--preserve-permissions` option and does not preserve Unix file permissions. Use `tar` on Linux to archive files while preserving permissions.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

- name: Create detached GPG signature with key 615D449FE6E6A235
run: |
echo "${GPG_PRIVATE_KEY}" | gpg --batch --quiet --import
echo "${GPG_PASSPHRASE}" | gpg --batch --quiet --passphrase-fd 0 --pinentry-mode loopback -u 615D449FE6E6A235 --detach-sign -a ./cryptomator-cli-${{ needs.prepare.outputs.semVerStr }}.zip
echo "${GPG_PASSPHRASE}" | gpg --batch --quiet --passphrase-fd 0 --pinentry-mode loopback -u 615D449FE6E6A235 --detach-sign -a ./${{ matrix.artifact-name }}
env:
GPG_PRIVATE_KEY: ${{ secrets.RELEASES_GPG_PRIVATE_KEY }}
GPG_PASSPHRASE: ${{ secrets.RELEASES_GPG_PASSPHRASE }}
Expand All @@ -145,6 +128,6 @@ jobs:
fail_on_unmatched_files: true
token: ${{ secrets.CRYPTOBOT_RELEASE_TOKEN }}
files: |
cryptomator-cli-${{ needs.prepare.outputs.semVerStr }}.zip
${{ matrix.artifact-name }}
cryptomator-cli-*.asc

133 changes: 133 additions & 0 deletions .github/workflows/build-mac.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
name: Java app image macOS

on:
release:
types: [published]
workflow_dispatch:
inputs:
sem-version:
description: 'Version'
required: false

permissions:
contents: write
packages: write

env:
JAVA_DIST: 'zulu'
JAVA_VERSION: '22.0.2+9'

defaults:
run:
shell: bash

jobs:
prepare:
name: Determines the versions strings for the binaries
runs-on: [ubuntu-latest]
outputs:
semVerStr: ${{ steps.determine-version.outputs.version }}
semVerNum: ${{steps.determine-number.outputs.number}}
steps:
- id: determine-version
shell: pwsh
run: |
if ( '${{github.event_name}}' -eq 'release') {
echo 'version=${{ github.event.release.tag_name}}' >> "$env:GITHUB_OUTPUT"
exit 0
} elseif ('${{inputs.sem-version}}') {
echo 'version=${{ inputs.sem-version}}' >> "$env:GITHUB_OUTPUT"
exit 0
}
Write-Error "Version neither via input nor by tag specified. Aborting"
exit 1
- id: determine-number
run: |
SEM_VER_NUM=$(echo "${{ steps.determine-version.outputs.version }}" | sed -E 's/([0-9]+\.[0-9]+\.[0-9]+).*/\1/')
echo "number=${SEM_VER_NUM}" >> "$GITHUB_OUTPUT"

build-binary:
name: Build java app image
needs: [prepare]
strategy:
fail-fast: false
matrix:
include:
- os: macos-latest
architecture: arm64
artifact-name: cryptomator-cli-${{ needs.prepare.outputs.semVerStr }}-mac-arm64.zip
- os: macos-13
architecture: x64
artifact-name: cryptomator-cli-${{ needs.prepare.outputs.semVerStr }}-mac-x64.zip
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
java-version: ${{ env.JAVA_VERSION }}
distribution: ${{ env.JAVA_DIST }}
- name: Set version
run: mvn versions:set -DnewVersion=${{ needs.prepare.outputs.semVerStr }}
- name: Run maven
run: mvn -B clean package -DskipTests
- name: Patch target dir
run: |
cp LICENSE.txt target
cp target/cryptomator-*.jar target/mods
- name: Run jlink
run: >
"${JAVA_HOME}/bin/jlink"
--verbose
--output target/runtime
--module-path "${JAVA_HOME}/jmods"
--add-modules java.base,java.compiler,java.naming,java.xml
--strip-native-commands
--no-header-files
--no-man-pages
--strip-debug
--compress zip-6
- name: Run jpackage
run: >
"${JAVA_HOME}/bin/jpackage"
--verbose
--type app-image
--runtime-image target/runtime
--input target/libs
--module-path target/mods
--module org.cryptomator.cli/org.cryptomator.cli.CryptomatorCli
--dest target
--name cryptomator-cli
--vendor "Skymatic GmbH"
--copyright "(C) 2016 - 2024 Skymatic GmbH"
--app-version "${{ needs.prepare.outputs.semVerNum }}"
--java-options "-Dorg.cryptomator.cli.version=${{ needs.prepare.outputs.semVerStr }}"
--java-options "--enable-native-access=org.cryptomator.jfuse.mac"
--java-options "-Xss5m"
--java-options "-Xmx256m"
--java-options "-Dfile.encoding=\"utf-8\""
- uses: actions/upload-artifact@v4
with:
name: cryptomator-cli-mac-${{ matrix.architecture }}
path: ./target/cryptomator-cli.app
if-no-files-found: error
- name: TODO sign binaries
run: echo "TODO sign it and notarize it"
- name: Zip binary for release
run: zip -r ./${{ matrix.artifact-name}} ./target/cryptomator-cli.app
- name: Create detached GPG signature with key 615D449FE6E6A235
run: |
echo "${GPG_PRIVATE_KEY}" | gpg --batch --quiet --import
echo "${GPG_PASSPHRASE}" | gpg --batch --quiet --passphrase-fd 0 --pinentry-mode loopback -u 615D449FE6E6A235 --detach-sign -a ./${{ matrix.artifact-name }}
env:
GPG_PRIVATE_KEY: ${{ secrets.RELEASES_GPG_PRIVATE_KEY }}
GPG_PASSPHRASE: ${{ secrets.RELEASES_GPG_PASSPHRASE }}
- name: Publish artefact on GitHub Releases
if: startsWith(github.ref, 'refs/tags/') && github.event.action == 'published'
uses: softprops/action-gh-release@v2
with:
fail_on_unmatched_files: true
token: ${{ secrets.CRYPTOBOT_RELEASE_TOKEN }}
files: |
${{ matrix.artifact-name }}
cryptomator-cli-*.asc

127 changes: 127 additions & 0 deletions .github/workflows/build-win.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
name: Java app image Windows

on:
release:
types: [published]
workflow_dispatch:
inputs:
sem-version:
description: 'Version'
required: false

permissions:
contents: write
packages: write

env:
JAVA_DIST: 'zulu'
JAVA_VERSION: '22.0.2+9'

defaults:
run:
shell: bash

jobs:
prepare:
name: Determines the versions strings for the binaries
runs-on: [ubuntu-latest]
outputs:
semVerStr: ${{ steps.determine-version.outputs.version }}
semVerNum: ${{steps.determine-number.outputs.number}}
steps:
- id: determine-version
shell: pwsh
run: |
if ( '${{github.event_name}}' -eq 'release') {
echo 'version=${{ github.event.release.tag_name}}' >> "$env:GITHUB_OUTPUT"
exit 0
} elseif ('${{inputs.sem-version}}') {
echo 'version=${{ inputs.sem-version}}' >> "$env:GITHUB_OUTPUT"
exit 0
}
Write-Error "Version neither via input nor by tag specified. Aborting"
exit 1
- id: determine-number
run: |
SEM_VER_NUM=$(echo "${{ steps.determine-version.outputs.version }}" | sed -E 's/([0-9]+\.[0-9]+\.[0-9]+).*/\1/')
echo "number=${SEM_VER_NUM}" >> "$GITHUB_OUTPUT"

build-binary:
name: Build java app image
needs: [prepare]
runs-on: windows-latest
env:
artifact-name: cryptomator-cli-${{ needs.prepare.outputs.semVerStr }}-win-x64.zip
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
java-version: ${{ env.JAVA_VERSION }}
distribution: ${{ env.JAVA_DIST }}
- name: Set version
run: mvn versions:set -DnewVersion=${{ needs.prepare.outputs.semVerStr }}
- name: Run maven
run: mvn -B clean package -DskipTests
- name: Patch target dir
run: |
cp LICENSE.txt target
cp target/cryptomator-*.jar target/mods
- name: Run jlink
run: >
"${JAVA_HOME}/bin/jlink"
--verbose
--output target/runtime
--module-path "${JAVA_HOME}/jmods"
--add-modules java.base,java.compiler,java.naming,java.xml
--strip-native-commands
--no-header-files
--no-man-pages
--strip-debug
--compress zip-6
- name: Run jpackage
run: >
"${JAVA_HOME}/bin/jpackage"
--verbose
--type app-image
--runtime-image target/runtime
--input target/libs
--module-path target/mods
--module org.cryptomator.cli/org.cryptomator.cli.CryptomatorCli
--dest target
--name cryptomator-cli
--vendor "Skymatic GmbH"
--copyright "(C) 2016 - 2024 Skymatic GmbH"
--app-version "${{ needs.prepare.outputs.semVerNum }}"
--java-options "-Dorg.cryptomator.cli.version=${{ needs.prepare.outputs.semVerStr }}"
--java-options "--enable-native-access=org.cryptomator.jfuse.win"
--java-options "-Xss5m"
--java-options "-Xmx256m"
--java-options "-Dfile.encoding=\"utf-8\""
--win-console
- uses: actions/upload-artifact@v4
with:
name: cryptomator-cli-win-x64
path: ./target/cryptomator-cli
if-no-files-found: error
- name: TODO Sign binaries
run: echo TODO
- name: Zip binary for release
shell: pwsh
run: Compress-Archive -Path .\target\cryptomator-cli -DestinationPath .\${{ env.artifact-name}}
- name: Create detached GPG signature with key 615D449FE6E6A235
run: |
echo "${GPG_PRIVATE_KEY}" | gpg --batch --quiet --import
echo "${GPG_PASSPHRASE}" | gpg --batch --quiet --passphrase-fd 0 --pinentry-mode loopback -u 615D449FE6E6A235 --detach-sign -a ./${{ env.artifact-name}}
env:
GPG_PRIVATE_KEY: ${{ secrets.RELEASES_GPG_PRIVATE_KEY }}
GPG_PASSPHRASE: ${{ secrets.RELEASES_GPG_PASSPHRASE }}
- name: Publish artefact on GitHub Releases
if: startsWith(github.ref, 'refs/tags/') && github.event.action == 'published'
uses: softprops/action-gh-release@v2
with:
fail_on_unmatched_files: true
token: ${{ secrets.CRYPTOBOT_RELEASE_TOKEN }}
files: |
${{ env.artifact-name}}
cryptomator-cli-*.asc