Skip to content

Commit 612b1f2

Browse files
authored
🎉🐍 Initial mqt.core Python Package Configuration (#359)
## Description This PR set up all the infrastructure for developing a modern `mqt.core` Python package. Instead of relying on `pybind11` and `setuptools` it uses [scikit-build-core](https://github.com/scikit-build/scikit-build-core) and [nanobind](https://github.com/wjakob/nanobind) to create a C++ extension module and package everything as a complete Python package. This is a substantial improvement over all previous efforts that relied on setuptools. For example, the new setup allows for fast, editable installs where the Python package is automatically rebuilt on C++ source code changes. Furthermore, `nanobind` is directly included as a build time dependency and need not be included as a git submodule. Finally, all the nasty configuration from the `setup.py` file could be removed. In fact, there no longer is a `setup.py` file at all. As for `nanobind` over `pybind11`: (from https://github.com/wjakob/nanobind) > nanobind is a small binding library that exposes C++ types in Python and vice versa. It is reminiscent of [Boost.Python](https://www.boost.org/doc/libs/1_64_0/libs/python/doc/html) and [pybind11](https://github.com/pybind/pybind11) and uses near-identical syntax. In contrast to these existing tools, nanobind is more efficient: bindings compile in a shorter amount of time, produce smaller binaries, and have better runtime performance. > > More concretely, [benchmarks](https://nanobind.readthedocs.io/en/latest/benchmark.html) show up to ~4× faster compile time, ~5× smaller binaries, and ~10× lower runtime overheads compared to pybind11. nanobind also outperforms Cython in important metrics (3-12× binary size reduction, 1.6-4× compilation time reduction, similar runtime performance). Another nice feature available after this PR is the addition of CI check gates, that is, a check that makes sure that all checks it depends on have succeeded. This significantly simplifies branch protection rules as there are only very few checks that actually need protection and the rules need not be updated whenever, e.g., a new matrix entry gets added to the CI config. > **Note** > The Python package itself is extremely limited at the moment and does not expose any useful functionality. That will be added in a subsequent PR. ## Checklist: <!--- This checklist serves as a reminder of a couple of things that ensure your pull request will be merged swiftly. --> - [x] The pull request only contains commits that are related to it. - [x] I have added appropriate tests and documentation. - [x] I have made sure that all CI jobs on GitHub pass. - [x] The pull request introduces no new warnings and follows the project's style guidelines.
2 parents 0761787 + 0c95918 commit 612b1f2

26 files changed

+1052
-79
lines changed

.github/SECURITY.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -7,13 +7,13 @@ Security updates are applied only to the most recent releases.
77
## Reporting a Vulnerability
88

99
To report vulnerabilities, you can privately report a potential security issue
10-
via the Github security vulnerabilities feature. This can be done here:
10+
via the GitHub security vulnerabilities feature. This can be done here:
1111

1212
https://github.com/cda-tum/mqt-core/security/advisories
1313

1414
Please do **not** open a public issue about a potential security vulnerability.
1515

16-
You can find more details on the security vulnerability feature in the Github
16+
You can find more details on the security vulnerability feature in the GitHub
1717
documentation here:
1818

1919
https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing/privately-reporting-a-security-vulnerability

.github/codecov.yml

+6
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
ignore:
22
- "extern/**/*"
3+
- "include/python/**/*"
4+
- "src/python/**/*"
35
- "test/**/*"
46

57
coverage:
@@ -18,3 +20,7 @@ parsers:
1820
branch_detection:
1921
conditional: no
2022
loop: no
23+
24+
codecov:
25+
notify:
26+
after_n_builds: 3

.github/matchers/pylint.json

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
{
2+
"problemMatcher": [
3+
{
4+
"severity": "warning",
5+
"pattern": [
6+
{
7+
"regexp": "^([^:]+):(\\d+):(\\d+): ([A-DF-Z]\\d+): \\033\\[[\\d;]+m([^\\033]+).*$",
8+
"file": 1,
9+
"line": 2,
10+
"column": 3,
11+
"code": 4,
12+
"message": 5
13+
}
14+
],
15+
"owner": "pylint-warning"
16+
},
17+
{
18+
"severity": "error",
19+
"pattern": [
20+
{
21+
"regexp": "^([^:]+):(\\d+):(\\d+): (E\\d+): \\033\\[[\\d;]+m([^\\033]+).*$",
22+
"file": 1,
23+
"line": 2,
24+
"column": 3,
25+
"code": 4,
26+
"message": 5
27+
}
28+
],
29+
"owner": "pylint-error"
30+
}
31+
]
32+
}

.github/workflows/ci.yml

+32-14
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
1-
name: CI
1+
name: 🇨‌➕➕ • CI
22

33
on:
44
push:
55
branches: [main]
66
pull_request:
7+
merge_group:
78
workflow_dispatch:
89

910
concurrency:
@@ -17,46 +18,50 @@ defaults:
1718
env:
1819
CMAKE_BUILD_PARALLEL_LEVEL: 3
1920
CTEST_PARALLEL_LEVEL: 3
21+
FORCE_COLOR: 3
2022

2123
jobs:
2224
cpp-tests:
23-
name: Tests ${{ matrix.config.os }}
24-
runs-on: ${{ matrix.config.os }}
25+
name: Tests ${{ matrix.runs-on }}
26+
runs-on: ${{ matrix.runs-on }}
2527
strategy:
2628
fail-fast: false
2729
matrix:
28-
config:
29-
- { os: ubuntu-latest, toolchain: "" }
30-
- { os: macos-latest, toolchain: "" }
31-
- { os: windows-latest, toolchain: "-T ClangCl" }
30+
runs-on: [ubuntu-latest, macos-latest, windows-latest]
3231
steps:
3332
- uses: actions/checkout@v3
3433
with:
3534
submodules: recursive
3635
- name: Setup ccache
37-
uses: hendrikmuhs/ccache-action@v1.2
36+
uses: Chocobo1/setup-ccache-action@v1
3837
with:
39-
key: "${{matrix.config.os}}"
38+
prepend_symlinks_to_path: false
39+
windows_compile_environment: msvc
40+
override_cache_key: c++-tests-${{ matrix.runs-on }}
4041
- name: Set up mold as linker (Linux only)
4142
uses: rui314/setup-mold@v1
43+
- name: Install Ninja
44+
if: runner.os != 'Windows'
45+
run: pipx install ninja
4246
- name: Configure CMake
43-
run: cmake -S . -B build -DCMAKE_BUILD_TYPE=Release ${{ matrix.config.toolchain }} -DENABLE_IPO=ON
47+
run: cmake -G Ninja -S . -B build -DCMAKE_BUILD_TYPE=Release
4448
- name: Build
4549
run: cmake --build build --config Release
4650
- name: Test
4751
run: ctest -C Release --output-on-failure --test-dir build --repeat until-pass:3 --timeout 300
4852

4953
coverage:
50-
name: C++ Coverage
54+
name: Coverage • ubuntu-latest
5155
runs-on: ubuntu-latest
5256
steps:
5357
- uses: actions/checkout@v3
5458
with:
5559
submodules: recursive
5660
- name: Setup ccache
57-
uses: hendrikmuhs/ccache-action@v1.2
61+
uses: Chocobo1/setup-ccache-action@v1
5862
with:
59-
key: "coverage"
63+
prepend_symlinks_to_path: false
64+
override_cache_key: c++-coverage
6065
- name: Set up mold as linker
6166
uses: rui314/setup-mold@v1
6267
- name: Configure CMake
@@ -66,9 +71,22 @@ jobs:
6671
- name: Test
6772
run: ctest -C Debug --output-on-failure --test-dir build --repeat until-pass:3 --timeout 300
6873
- name: Upload coverage to Codecov
69-
uses: codecov/codecov-action@v3.1.4
74+
uses: codecov/codecov-action@v3
7075
with:
7176
fail_ci_if_error: true
7277
gcov: true
7378
gcov_ignore: "extern/**/*"
7479
token: ${{ secrets.CODECOV_TOKEN }}
80+
81+
cpp-ci-check:
82+
if: always()
83+
name: 🚦 Check
84+
needs: [cpp-tests, coverage]
85+
runs-on: ubuntu-latest
86+
timeout-minutes: 1
87+
steps:
88+
- name: Decide whether the needed jobs succeeded or failed
89+
uses: re-actors/alls-green@release/v1
90+
with:
91+
allowed-skips: deploy
92+
jobs: ${{ toJSON(needs) }}

.github/workflows/codeql-analysis.yml

+51-6
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,35 @@
1-
name: "CodeQL"
1+
name: 📝 • CodeQL
22

33
on:
44
push:
5+
branches: [main, protected]
6+
paths-ignore:
7+
- "docs/**"
8+
- "**/*.md"
9+
- "**/*.rst"
10+
- "**/*.toml"
11+
- "**/*.txt"
12+
- "**/*.yml"
13+
- "**/*.yaml"
14+
- ".github/**"
15+
- ".clang-format"
16+
- ".clang-tidy"
17+
- ".gitignore"
518
pull_request:
19+
branches: [main]
20+
paths-ignore:
21+
- "docs/**"
22+
- "**/*.md"
23+
- "**/*.rst"
24+
- "**/*.toml"
25+
- "**/*.txt"
26+
- "**/*.yml"
27+
- "**/*.yaml"
28+
- ".github/**"
29+
- ".clang-format"
30+
- ".clang-tidy"
31+
- ".gitignore"
32+
merge_group:
633
schedule:
734
- cron: "15 21 * * 6"
835

@@ -12,18 +39,19 @@ concurrency:
1239

1340
env:
1441
CMAKE_BUILD_PARALLEL_LEVEL: 3
42+
FORCE_COLOR: 3
1543

1644
jobs:
1745
analyze:
18-
name: Analyze ${{ matrix.language }}
46+
name: ${{ matrix.language }}
1947
runs-on: ubuntu-latest
2048
permissions:
2149
security-events: write
2250

2351
strategy:
2452
fail-fast: false
2553
matrix:
26-
language: ["cpp"]
54+
language: ["cpp", "python"]
2755

2856
steps:
2957
- name: Checkout repository
@@ -33,23 +61,40 @@ jobs:
3361
fetch-depth: 0
3462

3563
- name: Setup ccache
36-
uses: hendrikmuhs/ccache-action@v1.2
64+
uses: Chocobo1/setup-ccache-action@v1
3765
with:
38-
key: "CodeQL-${{ matrix.language }}"
66+
prepend_symlinks_to_path: false
67+
override_cache_key: codeql-${{ matrix.language }}
3968

4069
- name: Set up mold as linker
4170
uses: rui314/setup-mold@v1
4271

72+
- if: matrix.language == 'python'
73+
name: Set up Python
74+
uses: actions/setup-python@v4
75+
with:
76+
python-version: "3.11"
77+
cache: "pip"
78+
79+
- if: matrix.language == 'python'
80+
name: Install dependencies
81+
run: |
82+
python -m pip install --upgrade pip wheel
83+
pip install nanobind scikit-build-core[pyproject] setuptools_scm
84+
pip install --no-build-isolation -ve .
85+
echo "CODEQL_PYTHON=$(which python)" >> $GITHUB_ENV
86+
4387
# Initializes the CodeQL tools for scanning.
4488
- name: Initialize CodeQL
4589
uses: github/codeql-action/init@v2
4690
with:
4791
languages: ${{ matrix.language }}
4892
config-file: .github/codeql-config.yml
93+
setup-python-dependencies: false
4994

5095
- if: matrix.language == 'cpp'
5196
name: Configure CMake
52-
run: cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
97+
run: cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug
5398

5499
- if: matrix.language == 'cpp'
55100
name: Build

.github/workflows/cpp-linter.yml

+19-3
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,25 @@
1-
name: cpp-linter
1+
name: 🇨‌➕➕ • Lint
22

33
on:
4+
merge_group:
45
pull_request:
6+
paths:
7+
- "**/*.cpp"
8+
- "**/*.hpp"
9+
- ".github/workflows/cpp-linter.yml"
10+
- ".clang-tidy"
11+
- "!include/python/**"
12+
- "!src/python/**"
513
push:
614
branches:
715
- main
16+
paths:
17+
- "**/*.cpp"
18+
- "**/*.hpp"
19+
- ".github/workflows/cpp-linter.yml"
20+
- ".clang-tidy"
21+
- "!include/python/**"
22+
- "!src/python/**"
823
workflow_dispatch:
924

1025
concurrency:
@@ -17,6 +32,7 @@ env:
1732
jobs:
1833
cpp-linter:
1934
runs-on: ubuntu-latest
35+
name: Run
2036
steps:
2137
- uses: actions/checkout@v3
2238
with:
@@ -49,8 +65,8 @@ jobs:
4965
with:
5066
style: ""
5167
tidy-checks: ""
52-
version: 16
53-
ignore: build
68+
version: ${{ env.clang-version }}
69+
ignore: "build|include/python|src/python"
5470
thread-comments: true
5571
step-summary: true
5672
database: "build"

.github/workflows/emulated-wheels.yml

+53
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
name: 🐍 • Emulated Wheels
2+
3+
on:
4+
release:
5+
types: [published]
6+
workflow_dispatch:
7+
8+
concurrency:
9+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
10+
cancel-in-progress: true
11+
12+
jobs:
13+
build_wheels_emulation:
14+
name: ${{ matrix.arch }} • ${{ matrix.python }}
15+
runs-on: ubuntu-latest
16+
strategy:
17+
fail-fast: false
18+
matrix:
19+
arch: ["s390x", "ppc64le"]
20+
python: ["cp38", "cp39", "cp310", "cp311", "cp312"]
21+
steps:
22+
- uses: actions/checkout@v3
23+
with:
24+
fetch-depth: 0
25+
submodules: recursive
26+
- name: Set up QEMU
27+
uses: docker/setup-qemu-action@v2
28+
- name: Build wheels
29+
uses: pypa/[email protected]
30+
env:
31+
CIBW_ARCHS_LINUX: ${{ matrix.arch }}
32+
CIBW_BUILD: ${{ matrix.python }}-*
33+
CIBW_TEST_SKIP: "cp*"
34+
- uses: actions/upload-artifact@v3
35+
with:
36+
path: ./wheelhouse/*.whl
37+
# TODO: Uncomment this in a subsequent PR after one release has been made
38+
# deploy:
39+
# if: github.event_name == 'release' && github.event.action == 'published'
40+
# name: 🚀 Deploy to PyPI
41+
# runs-on: ubuntu-latest
42+
# environment:
43+
# name: pypi
44+
# url: https://pypi.org/p/mqt.core
45+
# permissions:
46+
# id-token: write
47+
# needs: [build_wheels_emulation]
48+
# steps:
49+
# - uses: actions/download-artifact@v3
50+
# with:
51+
# name: artifact
52+
# path: dist
53+
# - uses: pypa/gh-action-pypi-publish@release/v1

0 commit comments

Comments
 (0)