Skip to content

Commit 4a910e2

Browse files
authored
♻️ Refactor Packaging and CI (#366)
## Description This PR brings a major refactoring for the overall Python packaging as well as the CI employed in the project. ### Packaging We are (finally) moving away from `setuptools` and towards modern `scikit-build-core` (https://scikit-build-core.readthedocs.io/) This brings many advantages such as - truly editable installs that also include the C++ part; this implies incredibly fast rebuilds that can be further improved by installing ccache. - completely eliminates the `setup.py` (and the `MANIFEST.in`) file and all custom build code that was required previously. As part of this change, we adopt the established best practice of a src-layout for the Python package. This implies that the Python package no longer lives in the main package directory, but is moved to `src/mqt/...`. This avoids several pitfalls when working with Python packages and makes the overall handling way easier. This also means there is no direct build-time dependency on cmake and ninja anymore as this is automatically handled by scikit-build-core on demand. As part of this rewrite, several `nox` sessions are also updated ### CI MQT Core has introduced reusable workflows in - munich-quantum-toolkit/core#396 These have been further refined in - munich-quantum-toolkit/core#407 Overall, this allows to - remove almost all workflow files of this repository - eliminate a lot of maintenance required in this repository to keep up with the latest best practices - rely on dependabot for workflow updates once MQT Core has settled enough so that it follows proper versioning and we can use tags for the workflows here instead of `@main` while at the same time: - having automatic change detection and only triggering jobs that are really necessary; with a single pass check that can be used for branch protection - setting up ccache for all supported workflow jobs which greatly improves compilation times - setting up mold as a better linker under linux - splitting jobs into as many subjobs as possible and feasible to distribute work - automatically creating wheels for all supported platforms (including exotic platforms that need emulation) and Python versions using PyPI trusted publishing - enabling automatic retries for flaky jobs such as MSVC builds or coverage uploads - enabling automatic build parallelization via using Ninja wherever suitable - running an automatic workflow using the minimal versiosn for all Python dependencies to guarantee compatibility As a nice side effect, the new CI has revealed two bugs in the heuristic mapper. One where the single-qubit gates could not be placed properly and one where the output permutation inference accessed arrays out of bounds. Both issues have subsequently been fixed. ## Checklist: <!--- This checklist serves as a reminder of a couple of things that ensure your pull request will be merged swiftly. --> - [x] The pull request only contains commits that are related to it. - [x] I have added appropriate tests and documentation. - [x] I have made sure that all CI jobs on GitHub pass. - [x] The pull request introduces no new warnings and follows the project's style guidelines.
2 parents fbc1e03 + c9744a6 commit 4a910e2

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+1005
-1063
lines changed

.cirrus.yml

+117
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
.clone_script: &clone |
2+
if [ -z "$CIRRUS_PR" ]; then
3+
git clone --recursive --branch=$CIRRUS_BRANCH https://x-access-token:${CIRRUS_REPO_CLONE_TOKEN}@github.com/${CIRRUS_REPO_FULL_NAME}.git $CIRRUS_WORKING_DIR
4+
git reset --hard $CIRRUS_CHANGE_IN_REPO
5+
else
6+
git clone https://x-access-token:${CIRRUS_REPO_CLONE_TOKEN}@github.com/${CIRRUS_REPO_FULL_NAME}.git $CIRRUS_WORKING_DIR
7+
git fetch origin pull/$CIRRUS_PR/head:pull/$CIRRUS_PR
8+
git checkout $CIRRUS_BASE_BRANCH
9+
git -c user.email="[email protected]" merge --no-commit pull/$CIRRUS_PR
10+
git submodule update --init --recursive
11+
fi
12+
13+
.statistics_script: &statistics |
14+
ccache -s -v
15+
echo $(python -m pip cache dir)
16+
echo $(python -m pip cache list)
17+
18+
macos_arm64_test_task:
19+
name: 🐍 Test / 🍎 arm64
20+
alias: macos_arm64_test
21+
clone_script: *clone
22+
macos_instance:
23+
image: ghcr.io/cirruslabs/macos-monterey-xcode:latest
24+
ccache_cache:
25+
folder: .ccache
26+
populate_script:
27+
- mkdir -p .ccache
28+
fingerprint_key: ccache-macosx_arm64
29+
pip_cache:
30+
folder: /Users/admin/Library/Caches/pip
31+
prepare_env_script: |
32+
brew install [email protected] ccache pipx ninja nox z3
33+
ln -s $(which python3.10) python
34+
export PATH=/opt/homebrew/opt/[email protected]/libexec/bin:$PATH
35+
export PATH=/opt/homebrew/opt/ccache/libexec:$PATH
36+
export PATH=/opt/homebrew/opt/pipx/libexec:$PATH
37+
export PATH=/opt/homebrew/opt/ninja/libexec:$PATH
38+
export PATH=/opt/homebrew/opt/nox/libexec:$PATH
39+
echo "PATH=$PATH" >> $CIRRUS_ENV
40+
echo "CCACHE_DIR=$PWD/.ccache" >> $CIRRUS_ENV
41+
echo "Z3_ROOT=/opt/homebrew/opt/z3" >> $CIRRUS_ENV
42+
run_nox_script:
43+
- nox -s tests-3.10 --verbose
44+
statistics_script: *statistics
45+
46+
build_and_store_wheels: &BUILD_AND_STORE_WHEELS
47+
install_cibuildwheel_script:
48+
- python -m pip install cibuildwheel~=2.15.0
49+
run_cibuildwheel_script:
50+
- cibuildwheel
51+
wheels_artifacts:
52+
path: "wheelhouse/*"
53+
54+
linux_aarch64_wheels_task:
55+
only_if: "$CIRRUS_RELEASE != ''"
56+
name: 🐍 Packaging / 🎡 🐧 arm64
57+
alias: linux_aarch64_wheels
58+
clone_script: *clone
59+
compute_engine_instance:
60+
image_project: cirrus-images
61+
image: family/docker-builder-arm64
62+
architecture: arm64
63+
platform: linux
64+
cpu: 8
65+
env:
66+
CIBW_BEFORE_ALL_LINUX: /opt/python/cp311-cp311/bin/pip install z3-solver==4.11.2
67+
setup_pyhton_script:
68+
- apt-get install -y python3-venv python-is-python3
69+
<<: *BUILD_AND_STORE_WHEELS
70+
71+
macos_arm64_wheels_task:
72+
only_if: "$CIRRUS_RELEASE != ''"
73+
name: 🐍 Packaging / 🎡 🍎 arm64
74+
alias: macos_arm64_wheels
75+
clone_script: *clone
76+
macos_instance:
77+
image: ghcr.io/cirruslabs/macos-monterey-xcode:latest
78+
ccache_cache:
79+
folder: .ccache
80+
populate_script:
81+
- mkdir -p .ccache
82+
fingerprint_key: ccache-macosx_arm64-wheels
83+
pip_cache:
84+
folder: /Users/admin/Library/Caches/pip
85+
prepare_env_script: |
86+
brew install [email protected] ccache pipx ninja nox z3
87+
ln -s $(which python3.10) python
88+
export PATH=/opt/homebrew/opt/[email protected]/libexec/bin:$PATH
89+
export PATH=/opt/homebrew/opt/ccache/libexec:$PATH
90+
export PATH=/opt/homebrew/opt/pipx/libexec:$PATH
91+
export PATH=/opt/homebrew/opt/ninja/libexec:$PATH
92+
export PATH=/opt/homebrew/opt/nox/libexec:$PATH
93+
echo "PATH=$PATH" >> $CIRRUS_ENV
94+
echo "CCACHE_DIR=$PWD/.ccache" >> $CIRRUS_ENV
95+
echo "Z3_ROOT=/opt/homebrew/opt/z3" >> $CIRRUS_ENV
96+
<<: *BUILD_AND_STORE_WHEELS
97+
statistics_script: *statistics
98+
99+
publish_task:
100+
name: 🚀 Deploy to PyPI
101+
container: { image: "python:3.10-bullseye" }
102+
depends_on:
103+
- linux_aarch64_wheels
104+
- macos_arm64_wheels
105+
only_if: "$CIRRUS_RELEASE != ''"
106+
env:
107+
TWINE_REPOSITORY: pypi
108+
TWINE_USERNAME: __token__
109+
TWINE_PASSWORD: "ENCRYPTED\
110+
[068b092a2dca55f5b1bc74cff487cf53b9cacb3b77bcbc74\
111+
057a446c826e4265e3b46f20081ab5fe44ac5aa4bbd74ee4]"
112+
install_script: pip install twine
113+
publish_script:
114+
- curl -L https://api.cirrus-ci.com/v1/artifact/build/$CIRRUS_BUILD_ID/wheels.zip -o wheels.zip
115+
- unzip wheels.zip
116+
- python -m twine check wheelhouse/*
117+
- python -m twine upload wheelhouse/*

.github/ISSUE_TEMPLATE/bug-report.yml

+11-30
Original file line numberDiff line numberDiff line change
@@ -1,47 +1,28 @@
11
name: 🐛 Bug report
22
description: Something is not working correctly.
33
title: "🐛 <title>"
4-
labels: ["bug"]
5-
assignees:
6-
- burgholzer
74
body:
85
- type: markdown
96
attributes:
107
value: >-
118
**Thank you for wanting to report a bug for this project!**
129
1310
14-
Verify first that your issue is not [already reported on GitHub](https://github.com/cda-tum/qmap/search?q=is%3Aissue&type=issues).
11+
Verify first that your issue is not [already reported on GitHub](https://github.com/cda-tum/mqt-qmap/search?q=is%3Aissue&type=issues).
1512
16-
If you are having general questions, please consider [starting a discussion](https://github.com/cda-tum/qmap/discussions).
17-
- type: markdown
18-
attributes:
19-
value: >-
20-
**Environment**
21-
- type: input
22-
attributes:
23-
label: mqt.qmap version
24-
placeholder: For example, mqt.qmap===1.11.0
25-
validations:
26-
required: true
27-
- type: input
13+
If you have general questions, please consider [starting a discussion](https://github.com/cda-tum/mqt-qmap/discussions).
14+
- type: textarea
2815
attributes:
29-
label: OS
30-
placeholder: For example, Ubuntu 22.04, macOS Big Sur, Windows etc.
16+
label: Environment information
17+
description: >-
18+
Please provide information about your environment. For example, OS, C++ compiler, mqt.core version etc.
19+
placeholder: |
20+
- OS:
21+
- C++ compiler:
22+
- mqt.core version:
23+
- Additional environment information:
3124
validations:
3225
required: true
33-
- type: input
34-
attributes:
35-
label: Python version
36-
placeholder: For example, Python 3.10
37-
- type: input
38-
attributes:
39-
label: C++ compiler
40-
placeholder: For example, gcc-10
41-
- type: textarea
42-
attributes:
43-
label: Additional environment information
44-
description: Feel free to add more information about your environment here.
4526
- type: textarea
4627
attributes:
4728
label: Description

.github/ISSUE_TEMPLATE/feature-request.yml

-3
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,6 @@
11
name: ✨ Feature request
22
description: Suggest an idea
33
title: "✨ <title>"
4-
labels: ["enhancement"]
5-
assignees:
6-
- burgholzer
74
body:
85
- type: markdown
96
attributes:

.github/codecov.yml

+11-6
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,7 @@
11
ignore:
22
- "extern/**/*"
3-
- "apps/*"
3+
- "**/python"
44
- "test/**/*"
5-
- "mqt/**/*.cpp"
65

76
coverage:
87
range: 60..90
@@ -13,10 +12,12 @@ coverage:
1312

1413
flag_management:
1514
default_rules:
15+
carryforward: true
1616
statuses:
1717
- type: project
1818
target: auto
1919
threshold: 0.5%
20+
removed_code_behavior: adjust_base
2021
- type: patch
2122
target: 90%
2223
threshold: 1%
@@ -25,12 +26,15 @@ flag_management:
2526
paths:
2627
- "include"
2728
- "src"
29+
after_n_builds: 1
2830
- name: python
2931
paths:
30-
- "mqt/**/*.py"
32+
- "src/mqt/**/*.py"
33+
after_n_builds: 3
3134
statuses:
3235
- type: project
3336
threshold: 0.5%
37+
removed_code_behavior: adjust_base
3438
- type: patch
3539
target: 95%
3640
threshold: 1%
@@ -41,6 +45,7 @@ parsers:
4145
conditional: no
4246
loop: no
4347

44-
codecov:
45-
notify:
46-
after_n_builds: 4
48+
comment:
49+
layout: "reach, diff, flags, files"
50+
require_changes: true
51+
show_carryforward_flags: true

.github/codeql-config.yml

-4
This file was deleted.

.github/dependabot.yml

+13-2
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,21 @@ version: 2
22
updates:
33
- package-ecosystem: "gitsubmodule"
44
directory: "/"
5+
groups:
6+
submodules:
7+
patterns:
8+
- "*"
59
schedule:
6-
interval: "weekly"
7-
day: "friday"
10+
interval: "monthly"
811
time: "06:00"
912
timezone: "Europe/Vienna"
1013

1114
- package-ecosystem: "github-actions"
1215
directory: "/"
16+
groups:
17+
github-actions:
18+
patterns:
19+
- "*"
1320
schedule:
1421
interval: "weekly"
1522
day: "friday"
@@ -18,6 +25,10 @@ updates:
1825

1926
- package-ecosystem: "pip"
2027
directory: "/"
28+
groups:
29+
python-dependencies:
30+
patterns:
31+
- "*"
2132
schedule:
2233
interval: "weekly"
2334
day: "friday"

.github/matchers/pylint.json

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
{
2+
"problemMatcher": [
3+
{
4+
"severity": "warning",
5+
"pattern": [
6+
{
7+
"regexp": "^([^:]+):(\\d+):(\\d+): ([A-DF-Z]\\d+): \\033\\[[\\d;]+m([^\\033]+).*$",
8+
"file": 1,
9+
"line": 2,
10+
"column": 3,
11+
"code": 4,
12+
"message": 5
13+
}
14+
],
15+
"owner": "pylint-warning"
16+
},
17+
{
18+
"severity": "error",
19+
"pattern": [
20+
{
21+
"regexp": "^([^:]+):(\\d+):(\\d+): (E\\d+): \\033\\[[\\d;]+m([^\\033]+).*$",
22+
"file": 1,
23+
"line": 2,
24+
"column": 3,
25+
"code": 4,
26+
"message": 5
27+
}
28+
],
29+
"owner": "pylint-error"
30+
}
31+
]
32+
}

.github/workflows/cd.yml

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: CD
2+
on:
3+
release:
4+
types: [published]
5+
workflow_dispatch:
6+
7+
jobs:
8+
python-packaging:
9+
name: 🐍 Packaging
10+
uses: cda-tum/mqt-core/.github/workflows/reusable-python-packaging.yml@main
11+
with:
12+
setup-z3: true
13+
14+
deploy:
15+
if: github.event_name == 'release' && github.event.action == 'published'
16+
name: 🚀 Deploy to PyPI
17+
runs-on: ubuntu-latest
18+
environment:
19+
name: pypi
20+
url: https://pypi.org/p/mqt.qmap
21+
permissions:
22+
id-token: write
23+
needs: [python-packaging]
24+
steps:
25+
- uses: actions/download-artifact@v3
26+
with:
27+
name: artifact
28+
path: dist
29+
- uses: pypa/gh-action-pypi-publish@release/v1

0 commit comments

Comments
 (0)