Remove usage of Buildkite.

CONTRIBUTING.md

@@ -13,6 +13,7 @@
   * [Code quality](#code-quality)
     * [Internal tool](#internal-tool)
     * [ktlint](#ktlint)
+    * [knit](#knit)
     * [lint](#lint)
   * [Unit tests](#unit-tests)
   * [Tests](#tests)
@@ -126,6 +127,23 @@ Note that you can run
 
 For ktlint to fix some detected errors for you (you still have to check and commit the fix of course)
 
+#### knit
+
+[knit](https://github.com/Kotlin/kotlinx-knit) is a tool which checks markdown files on the project. Also it generates/updates the table of content (toc) of the markdown files.
+
+So everytime the toc should be updated, just run
+<pre>
+./gradlew knit
+</pre>
+
+and commit the changes.
+
+The CI will check that markdown files are up to date by running
+
+<pre>
+./gradlew knitCheck
+</pre>
+
 #### lint
 
 <pre>

README.md

@@ -1,4 +1,4 @@
-[![Buildkite](https://badge.buildkite.com/ad0065c1b70f557cd3b1d3d68f9c2154010f83c4d6f71706a9.svg?branch=develop)](https://buildkite.com/matrix-dot-org/element-android/builds?branch=develop)
+[![Latest build](https://github.com/vector-im/element-android/actions/workflows/build.yml/badge.svg?query=branch%3Adevelop)](https://github.com/vector-im/element-android/actions/workflows/build.yml?query=branch%3Adevelop)
 [![Weblate](https://translate.element.io/widgets/element-android/-/svg-badge.svg)](https://translate.element.io/engage/element-android/?utm_source=widget)
 [![Element Android Matrix room #element-android:matrix.org](https://img.shields.io/matrix/element-android:matrix.org.svg?label=%23element-android:matrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#element-android:matrix.org)
 [![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=vector-im_element-android&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=vector-im_element-android)
@@ -14,7 +14,7 @@ It is a total rewrite of [Riot-Android](https://github.com/vector-im/riot-androi
 [<img src="resources/img/google-play-badge.png" alt="Get it on Google Play" height="60">](https://play.google.com/store/apps/details?id=im.vector.app)
 [<img src="resources/img/f-droid-badge.png" alt="Get it on F-Droid" height="60">](https://f-droid.org/app/im.vector.app)
 
-Nightly build: [![Buildkite](https://badge.buildkite.com/ad0065c1b70f557cd3b1d3d68f9c2154010f83c4d6f71706a9.svg?branch=develop)](https://buildkite.com/matrix-dot-org/element-android/builds?branch=develop) Nightly test status: [![allScreensTest](https://github.com/vector-im/element-android/actions/workflows/nightly.yml/badge.svg)](https://github.com/vector-im/element-android/actions/workflows/nightly.yml)
+Build of develop branch: [![GitHub Action](https://github.com/vector-im/element-android/actions/workflows/build.yml/badge.svg?query=branch%3Adevelop)](https://github.com/vector-im/element-android/actions/workflows/build.yml?query=branch%3Adevelop) Nightly test status: [![allScreensTest](https://github.com/vector-im/element-android/actions/workflows/nightly.yml/badge.svg)](https://github.com/vector-im/element-android/actions/workflows/nightly.yml)
 
 
 # New Android SDK
@@ -40,7 +40,7 @@ If you would like to receive releases more quickly (bearing in mind that they ma
 
 1. [Sign up to receive beta releases](https://play.google.com/apps/testing/im.vector.app) via the Google Play Store.
 2. Install a [release APK](https://github.com/vector-im/element-android/releases) directly - download the relevant .apk file and allow installing from untrusted sources in your device settings.  Note: these releases are the Google Play version, which depend on some Google services.  If you prefer to avoid that, try the latest dev builds, and choose the F-Droid version.
-3. If you're really brave, install the [very latest dev build](https://buildkite.com/matrix-dot-org/element-android/builds/latest?branch=develop&state=passed) - click on *Assemble (GPlay or FDroid) Debug version* then on *Artifacts*.
+3. If you're really brave, install the [very latest dev build](https://github.com/vector-im/element-android/actions/workflows/build.yml?query=branch%3Adevelop) - pick a build, then click on `Summary` to download the APKs from there: `vector-Fdroid-debug` and `vector-Gplay-debug` contains the APK for the desired store. Each file contains 5 APKs. 4 APKs for every supported specific architecture of device. In doubt you can install the `universal` APK.
 
 ## Contributing
 

changelog.d/7583.misc

@@ -0,0 +1 @@
+Remove usage of Buildkite.

docs/installing_from_ci.md

@@ -0,0 +1,52 @@
+## Installing from CI
+
+<!--- TOC -->
+
+  * [Installing from Buildkite](#installing-from-buildkite)
+  * [Installing from GitHub](#installing-from-github)
+    * [Create a GitHub token](#create-a-github-token)
+  * [Provide artifact URL](#provide-artifact-url)
+  * [Next steps](#next-steps)
+  * [Future improvement](#future-improvement)
+
+<!--- END -->
+
+Installing APK build by the CI is possible
+
+### Installing from Buildkite
+
+The script `./tools/install/installFromBuildkite.sh` can be used, but Builkite will be removed soon. See next section.
+
+### Installing from GitHub
+
+To install an APK built by a GitHub action, run the script `./tools/install/installFromGitHub.sh`. You will need to pass a GitHub token to do so.
+
+#### Create a GitHub token
+
+You can create a GitHub token going to your Github account, at this page: [https://github.com/settings/tokens](https://github.com/settings/tokens).
+
+You need to create a token (classic) with the scope `repo/public_repo`. So just check the corresponding checkbox.
+Validity can be long since the scope of this token is limited. You will still be able to delete the token and generate a new one.
+Click on Generate token and save the token locally.
+
+### Provide artifact URL
+
+The script will ask for an artifact URL. You can get this artifact URL by following these steps:
+
+- open the pull request
+- in the check at the bottom, click on `APK Build / Build debug APKs`
+- click on `Summary`
+- scroll to the bottom of the page
+- copy the link `vector-Fdroid-debug` if you want the F-Droid variant or `vector-Gplay-debug` if you want the Gplay variant.
+
+The copied link can be provided to the script.
+
+### Next steps
+
+The script will download the artifact, unzip it and install the correct version (regarding arch) on your device.
+
+Files will be added to the folder `./tmp/DebugApks`. Feel free to cleanup this folder from time to time, the script will not delete files.
+
+### Future improvement
+
+The script could ask the user for a Pull Request number and Gplay/Fdroid choice like it was done with Buildkite script. Using GitHub API may be possible to do that.

tools/gradle/doctor.gradle

@@ -1,6 +1,6 @@
 // Default configuration copied from https://runningcode.github.io/gradle-doctor/configuration/
 
-def isCiBuild = System.env.BUILDKITE == "true" || System.env.GITHUB_ACTIONS == "true"
+def isCiBuild = System.env.GITHUB_ACTIONS == "true"
 println "Is CI build: $isCiBuild"
 
 doctor {

tools/install/installFromGitHub.sh

@@ -0,0 +1,95 @@
+#!/usr/bin/env bash
+
+# Exit on any error
+set -e
+
+if [[ "$#" -ne 1 ]]; then
+  echo "Usage: $0 GitHub_Token" >&2
+  echo "Read more about this script in the doc ./docs/installing_from_ci.md"
+  exit 1
+fi
+
+gitHubToken=$1
+
+# Path where the app is cloned (it's where this project has been cloned)
+appPath=$(dirname $(dirname $(dirname $0)))
+# Path where the APK will be downloaded from CI (it's a dir)
+baseImportPath="${appPath}/tmp/DebugApks"
+
+# Select device
+serialNumber=$(${appPath}/tools/install/androidSelectDevice.sh)
+
+# Detect device architecture
+arch=$(adb -s ${serialNumber} shell getprop ro.product.cpu.abi)
+
+echo
+echo "Will install the application on device ${serialNumber} with arch ${arch}"
+
+# Artifact URL
+echo
+read -p "Artifact url (ex: https://github.com/vector-im/element-android/suites/9293388174/artifacts/435942121)? " artifactUrl
+
+## Example of default value for Gplay
+#artifactUrl=${artifactUrl:-https://github.com/vector-im/element-android/suites/9293388174/artifacts/435942121}
+## Example of default value for FDroid
+# artifactUrl=${artifactUrl:-https://github.com/vector-im/element-android/suites/9293388174/artifacts/435942119}
+
+artifactId=$(echo ${artifactUrl} | rev | cut -d'/' -f1 | rev)
+
+# Download files
+targetPath=${baseImportPath}/${artifactId}
+
+filename="artifact.zip"
+
+fullFilePath="${targetPath}/${filename}"
+
+# Check if file already exists
+if test -f "$fullFilePath"; then
+    read -p "$fullFilePath already exists. Override (yes/no) default to no ? " download
+    download=${download:-no}
+else
+    download="yes"
+fi
+
+# Ignore error from now
+set +e
+
+if [ ${download} == "yes" ]; then
+    echo "Downloading ${filename} to ${targetPath}..."
+    python3 ${appPath}/tools/release/download_github_artifacts.py \
+        --token ${gitHubToken} \
+        --artifactUrl ${artifactUrl} \
+        --directory ${targetPath} \
+        --filename ${filename} \
+        --ignoreErrors
+fi
+
+echo "Unzipping ${filename}..."
+unzip $fullFilePath -d ${targetPath}
+
+## gplay or fdroid
+if test -d "${targetPath}/gplay"; then
+  variant="gplay"
+elif test -d "${targetPath}/fdroid"; then
+  variant="fdroid"
+else
+  echo "No variant found"
+  exit 1
+fi
+
+fullApkPath="${targetPath}/${variant}/debug/vector-${variant}-${arch}-debug.apk"
+
+echo "Installing ${fullApkPath} to device ${serialNumber}..."
+adb -s ${serialNumber} install -r ${fullApkPath}
+
+# Check error and propose to uninstall and retry installing
+if [[ "$?" -ne 0 ]]; then
+    read -p "Error, do you want to uninstall the application then retry (yes/no) default to no ? " retry
+    retry=${retry:-no}
+    if [ ${retry} == "yes" ]; then
+        echo "Uninstalling..."
+        adb -s ${serialNumber} uninstall im.vector.app.debug
+        echo "Installing again..."
+        adb -s ${serialNumber} install -r ${fullApkPath}
+    fi
+fi

tools/release/download_github_artifacts.py

@@ -0,0 +1,154 @@
+#!/usr/bin/env python3
+#
+# Copyright 2022 New Vector Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import argparse
+import hashlib
+import json
+import os
+# Run `pip3 install requests` if not installed yet
+import requests
+
+# This script downloads artifacts from GitHub.
+# Ref: https://docs.github.com/en/rest/actions/artifacts#get-an-artifact
+
+error = False
+
+### Arguments
+
+parser = argparse.ArgumentParser(description='Download artifacts from GitHub.')
+parser.add_argument('-t',
+                    '--token',
+                    required=True,
+                    help='The GitHub token with read access.')
+parser.add_argument('-a',
+                    '--artifactUrl',
+                    required=True,
+                    help='the artifact_url from GitHub.')
+parser.add_argument('-f',
+                    '--filename',
+                    help='the filename, if not provided, will use the artifact name.')
+parser.add_argument('-i',
+                    '--ignoreErrors',
+                    help='Ignore errors that can be ignored. Build state and number of artifacts.',
+                    action="store_true")
+parser.add_argument('-d',
+                    '--directory',
+                    default="",
+                    help='the target directory, where files will be downloaded. If not provided the build number will be used to create a directory.')
+parser.add_argument('-v',
+                    '--verbose',
+                    help="increase output verbosity.",
+                    action="store_true")
+parser.add_argument('-s',
+                    '--simulate',
+                    help="simulate action, do not create folder or download any file.",
+                    action="store_true")
+
+args = parser.parse_args()
+
+if args.verbose:
+    print("Argument:")
+    print(args)
+
+# Split the artifact URL to get information
+# Ex: https://github.com/vector-im/element-android/suites/9293388174/artifacts/435942121
+artifactUrl = args.artifactUrl
+if not artifactUrl.startswith('https://github.com/'):
+    print("❌ Invalid parameter --artifactUrl %s. Must start with 'https://github.com/'" % artifactUrl)
+    exit(1)
+if "/artifacts/" not in artifactUrl:
+    print("❌ Invalid parameter --artifactUrl %s. Must contain '/artifacts/'" % artifactUrl)
+    exit(1)
+artifactItems = artifactUrl.split("/")
+if len(artifactItems) != 9:
+    print("❌ Invalid parameter --artifactUrl %s. Please check the format." % (artifactUrl))
+    exit(1)
+
+gitHubRepoOwner = artifactItems[3]
+gitHubRepo = artifactItems[4]
+artifactId = artifactItems[8]
+
+if args.verbose:
+    print("gitHubRepoOwner: %s, gitHubRepo: %s, artifactId: %s" % (gitHubRepoOwner, gitHubRepo, artifactId))
+
+headers = {
+   'Authorization': "Bearer %s" % args.token,
+   'Accept': 'application/vnd.github+json'
+}
+base_url = "https://api.github.com/repos/%s/%s/actions/artifacts/%s" % (gitHubRepoOwner, gitHubRepo, artifactId)
+
+### Fetch build state
+
+print("Getting artifacts data of project '%s/%s' artifactId '%s'..." % (gitHubRepoOwner, gitHubRepo, artifactId))
+
+if args.verbose:
+    print("Url: %s" % base_url)
+
+r = requests.get(base_url, headers=headers)
+data = json.loads(r.content.decode())
+
+if args.verbose:
+    print("Json data:")
+    print(data)
+
+if args.verbose:
+    print("Create subfolder %s to download artifacts..." % artifactId)
+
+if args.directory == "":
+    targetDir = artifactId
+else:
+    targetDir = args.directory
+
+if not args.simulate:
+    os.makedirs(targetDir, exist_ok=True)
+
+url = data.get("archive_download_url")
+if args.filename is not None:
+    filename = args.filename
+else:
+    filename = data.get("name") + ".zip"
+
+## Print some info about the artifact origin
+commitLink = "https://github.com/%s/%s/commit/%s" % (gitHubRepoOwner, gitHubRepo, data.get("workflow_run").get("head_sha"))
+print("Preparing to download artifact `%s`, built from branch: `%s` (commit %s)" % (data.get("name"), data.get("workflow_run").get("head_branch"), commitLink))
+
+if args.verbose:
+    print()
+    print("Artifact url: %s" % url)
+
+target = targetDir + "/" + filename
+sizeInBytes = data.get("size_in_bytes")
+print("Downloading %s to '%s' (file size is %s bytes, this may take a while)..." % (filename, targetDir, sizeInBytes))
+if not args.simulate:
+    # open file to write in binary mode
+    with open(target, "wb") as file:
+        # get request
+        response = requests.get(url, headers=headers)
+        # write to file
+        file.write(response.content)
+    print("Verifying file size...")
+    # get the file size
+    size = os.path.getsize(target)
+    if sizeInBytes != size:
+        # error = True
+        print("Warning, file size mismatch: expecting %s and get %s. This is just a warning for now..." % (sizeInBytes, size))
+
+if error:
+    print("❌ Error(s) occurred, please check the log")
+    exit(1)
+else:
+    print("Done!")

tools/release/releaseScript.sh

@@ -214,7 +214,7 @@ else
 fi
 
 printf "\n================================================================================\n"
-read -p "Wait for Buildkite https://buildkite.com/matrix-dot-org/element-android/builds?branch=main to build the 'main' branch. Press enter when it's done."
+read -p "Wait for the GitHub action https://github.com/vector-im/element-android/actions/workflows/build.yml?query=branch%3Amain to build the 'main' branch. Press enter when it's done."
 
 printf "\n================================================================================\n"
 printf "Running the release script...\n"