Skip to content

Update the building wheels procedure to use cibuildwheels tool #425

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 146 commits into from
Apr 23, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
146 commits
Select commit Hold shift + click to select a range
9d5bac5
Add: first iteration of buildwheels.yml
Griger5 Mar 27, 2025
256b66a
Disable tests+pypi
Griger5 Mar 27, 2025
4a2c00c
Fix: fix whitespace
Griger5 Mar 27, 2025
e92f5f3
Add: submodule checkout
Griger5 Mar 27, 2025
31cef8f
Disable all other actions
Griger5 Mar 27, 2025
6f35c85
Disable all other actions
Griger5 Mar 27, 2025
789ab65
Add: Fortran compiler installation on windows platforms
Griger5 Mar 28, 2025
ebf51ad
Update: windows-latest build
Griger5 Mar 28, 2025
e8b5730
fix: windows build wheels
Griger5 Mar 28, 2025
5c49fd0
fix
Griger5 Mar 28, 2025
2b65787
fix
Griger5 Mar 29, 2025
c95be38
debug windows-latest only
Griger5 Mar 29, 2025
096642c
fix
Griger5 Mar 29, 2025
0ffda02
fortran fix
Griger5 Mar 29, 2025
55f406b
fortran fix
Griger5 Mar 29, 2025
1f0b205
fortran fix
Griger5 Mar 29, 2025
6226fa0
fortran fix
Griger5 Mar 29, 2025
e10403b
enable ubuntu-latest only
Griger5 Mar 30, 2025
f693b94
fix
Griger5 Mar 31, 2025
956ec81
fortran
Griger5 Apr 1, 2025
a2b577c
fortran
Griger5 Apr 1, 2025
beff4fb
fortran
Griger5 Apr 1, 2025
13a5bfc
sanity fix
Griger5 Apr 1, 2025
b4d892b
fortran fix
Griger5 Apr 3, 2025
192e62a
fortran fix
Griger5 Apr 3, 2025
bf80865
fortran fix
Griger5 Apr 3, 2025
9e92904
fortran fix
Griger5 Apr 3, 2025
2343dc1
fortran fix
Griger5 Apr 4, 2025
73d1f47
fortran fix
Griger5 Apr 4, 2025
ccb245a
fortran fix
Griger5 Apr 4, 2025
02080a0
fortran fix
Griger5 Apr 4, 2025
bb6934a
fortran fix
Griger5 Apr 4, 2025
cb3ded5
fortran fix
Griger5 Apr 4, 2025
f69b0df
skip 32 bit builds
Griger5 Apr 4, 2025
8ba4dbf
Delete python versions matrix, general clean up
Griger5 Apr 5, 2025
f04ce48
Skip PyPy builds
Griger5 Apr 5, 2025
af89287
Add: build wheels for all supported os
Griger5 Apr 5, 2025
9c55d8e
Fix: download gcc for macos builds
Griger5 Apr 5, 2025
8293c92
Fix: macos builds
Griger5 Apr 5, 2025
005f22a
Fix: macos builds
Griger5 Apr 5, 2025
25ddec5
Check all builds
Griger5 Apr 5, 2025
8e10d95
Skip musllinux builds
Griger5 Apr 5, 2025
4b48720
change artifact naming
Griger5 Apr 5, 2025
a742450
change artifact naming
Griger5 Apr 5, 2025
ae2b0f5
sanity check: change artifact naming
Griger5 Apr 5, 2025
15510c1
move windows' gfortran and ninja installation
Griger5 Apr 5, 2025
42795bb
move windows' gfortran and ninja installation
Griger5 Apr 5, 2025
64730ff
Attempt to add testing
Griger5 Apr 5, 2025
414ceae
Attempt to add testing
Griger5 Apr 5, 2025
54ac76a
Attempt to add testing
Griger5 Apr 5, 2025
0b18142
Attempt to fix macos testing
Griger5 Apr 5, 2025
eb2f7c9
Enable all other actions
Griger5 Apr 6, 2025
0cecacf
add: zenodo_json and debug_build
Griger5 Apr 6, 2025
89e552e
change the command for testing
Griger5 Apr 6, 2025
06494c2
fix path in the command for testing
Griger5 Apr 6, 2025
07aba87
add pytest-order as an requirement before testing
Griger5 Apr 6, 2025
8f70219
add: dist_check and dist_upload
Griger5 Apr 6, 2025
467838f
disable dist_check
Griger5 Apr 6, 2025
30f5401
check CIBW_BEFORE_ALL behaviour
Griger5 Apr 10, 2025
a2e1622
attempt to test the notebooks
Griger5 Apr 10, 2025
9d9d834
attempt to test the notebooks
Griger5 Apr 10, 2025
25da54e
fix typo
Griger5 Apr 10, 2025
e146f00
fix typo
Griger5 Apr 10, 2025
8e9ee14
attempt to test the notebooks
Griger5 Apr 10, 2025
0ef6803
attempt to add dist_check
Griger5 Apr 10, 2025
f64c3bc
attempt to add dist_check
Griger5 Apr 10, 2025
fa93aba
fix notebooks
Griger5 Apr 10, 2025
05e8ab3
fix notebooks
Griger5 Apr 10, 2025
ec795d1
fix notebooks
Griger5 Apr 10, 2025
e337fe9
try out dist_check
Griger5 Apr 10, 2025
d36466b
fix notebooks
Griger5 Apr 11, 2025
906bb19
fix notebooks
Griger5 Apr 11, 2025
880ca2a
fix typo
Griger5 Apr 11, 2025
c17beb2
attempt to fix notebook run
Griger5 Apr 12, 2025
6b3a88a
attempt to fix notebook run
Griger5 Apr 12, 2025
20c9c29
attempt to fix notebook run
Griger5 Apr 12, 2025
7087c3e
check with ls
Griger5 Apr 12, 2025
a8953fe
check with ls
Griger5 Apr 12, 2025
74f51fe
attempt to fix notebook run
Griger5 Apr 12, 2025
7d366ec
fix typo
Griger5 Apr 12, 2025
f5dd319
attempt to fix notebook run
Griger5 Apr 12, 2025
d129b47
attempt to fix notebook run
Griger5 Apr 12, 2025
f77e21e
attempt to fix notebook run
Griger5 Apr 12, 2025
4c3277c
attempt to fix notebook run
Griger5 Apr 12, 2025
8e93cf2
dumb-fix: added fixes were in the wrong place
Griger5 Apr 13, 2025
275c398
attempt to fix notebook run
Griger5 Apr 13, 2025
a5ecbfc
attempt to fix notebook run
Griger5 Apr 13, 2025
23f030c
attempt to fix notebook run
Griger5 Apr 13, 2025
62362ad
attempt to fix notebook run
Griger5 Apr 13, 2025
aaa5fa2
attempt to fix notebook run
Griger5 Apr 13, 2025
d2a871f
sanity check
Griger5 Apr 13, 2025
9f24221
sanity check
Griger5 Apr 13, 2025
684d9f7
replace direct wheel name
Griger5 Apr 13, 2025
4cf7bad
delete ls
Griger5 Apr 13, 2025
2878a80
every os now runs the notebooks once, test_todos_annotated are run once
Griger5 Apr 14, 2025
74a85da
disable separate run_notebooks job, add name to running todos
Griger5 Apr 14, 2025
fae0300
run all os again
Griger5 Apr 14, 2025
b273895
delete commented code, try escaping the square brackets
Griger5 Apr 14, 2025
929d356
attempt to fix windows build
Griger5 Apr 14, 2025
489a972
fix: add missing if
Griger5 Apr 14, 2025
e06b08f
try to fix the yml
Griger5 Apr 14, 2025
7cab715
try to fix the yml syntax
Griger5 Apr 14, 2025
1698890
try to fix the yml syntax
Griger5 Apr 14, 2025
13a93e7
use bash on windows
Griger5 Apr 14, 2025
03b7241
delete old .yml file, add some comments
Griger5 Apr 14, 2025
7c53127
rename the CI run
Griger5 Apr 15, 2025
2a8bef6
running example notebooks is now a separate job
Griger5 Apr 15, 2025
ff4a39b
fix indent
Griger5 Apr 15, 2025
5208b04
move checking annotated todos into a separate job
Griger5 Apr 15, 2025
e8ba61f
add macos-13 arm builds
Griger5 Apr 15, 2025
3c6452c
change spaces to _ in job name
Griger5 Apr 15, 2025
640722e
add missing 'needs' parameter for the example notebooks jon
Griger5 Apr 15, 2025
484aaee
fix missing dependencies
Griger5 Apr 15, 2025
a7eebb4
change uppercase to lower so it matches the rest of the file + check …
Griger5 Apr 15, 2025
7f9275b
sanity check: change macos-13-arm64 to macos-13-arm
Griger5 Apr 15, 2025
64eb4d8
add wheel repair for windows
Griger5 Apr 15, 2025
8c8672a
build arm64 wheels for macos on an Intel runner
Griger5 Apr 15, 2025
afa3730
delete macos-13-arm from os matrix
Griger5 Apr 15, 2025
f7c139e
build macos wheels for universal2
Griger5 Apr 16, 2025
2ac27d3
add flags for fortran on macos universal2 builds
Griger5 Apr 16, 2025
2fc520a
try to fix the flag
Griger5 Apr 16, 2025
ee701ef
try to fix the flag syntax
Griger5 Apr 16, 2025
70db19e
attempt to fix the flag syntax
Griger5 Apr 16, 2025
7b7caa7
add a comment + sanity check
Griger5 Apr 16, 2025
a5efb4b
attempt to fix universal2 builds
Griger5 Apr 16, 2025
8e2364d
try to add the cmake fortran flag again
Griger5 Apr 16, 2025
512a419
disable other oses for testing
Griger5 Apr 16, 2025
b844b8c
fix yaml syntax
Griger5 Apr 16, 2025
f8f770c
move setup-python
Griger5 Apr 17, 2025
4e407fa
attempt to fix the fortran flags syntax again
Griger5 Apr 17, 2025
92864fd
replace cmake_args with cmake_cache_args
Griger5 Apr 17, 2025
40181b7
try to enable cross-compilation for universal2 builds
Griger5 Apr 18, 2025
b300b1a
use full path in the FC variable
Griger5 Apr 18, 2025
0c4e84d
install lld on macos
Griger5 Apr 18, 2025
21557fd
add missing &&
Griger5 Apr 18, 2025
847df5c
delete sudo from the brew install command
Griger5 Apr 18, 2025
a961ea4
reinstall gcc before build
Griger5 Apr 18, 2025
992d052
add gfortran path to PATH
Griger5 Apr 18, 2025
cf26327
delete &&
Griger5 Apr 18, 2025
b5337c3
delete ARCHFLAGS
Griger5 Apr 18, 2025
98b256f
check gcc version
Griger5 Apr 18, 2025
321f1a6
try to uninstall gcc
Griger5 Apr 18, 2025
cc378d5
add missing &&
Griger5 Apr 18, 2025
7b39bc4
revert changes, get rid of universal2 builds for now
Griger5 Apr 23, 2025
7746ff7
enable all os for builds
Griger5 Apr 23, 2025
a5ac0a2
Merge branch 'main' into update-build-cibuildwheel
Griger5 Apr 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
168 changes: 168 additions & 0 deletions .github/workflows/buildwheels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
name: build_wheels+tests

defaults:
run:
shell: bash

on:
pull_request:
branches: [ main ]

jobs:
debug_build_ok:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
fetch-depth: 0 # https://github.com/pypa/setuptools_scm/issues/480
- run: DEBUG=1 VERBOSE=1 pip install --verbose -e .[tests]
- run: pytest -v -s -We -p no:unraisableexception tests

zenodo_json:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: notiz-dev/github-action-json-property@release
with:
path: '.zenodo.json'
prop_path: 'creators'

build_wheels:
needs: [debug_build_ok, zenodo_json]
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, ubuntu-24.04-arm, macos-13, macos-latest, windows-latest]

steps:
- uses: actions/checkout@v4
with:
submodules: recursive
fetch-depth: 0

# download C and Fortran compiler on windows
- if: matrix.os == 'windows-latest'
uses: msys2/setup-msys2@v2
with:
msystem: MINGW64
update: false
install: >-
mingw-w64-x86_64-gcc-fortran
mingw-w64-x86_64-ninja
m4

- uses: actions/setup-python@v5

- name: Install cibuildwheel
run: python -m pip install cibuildwheel==2.23.2

- name: Build and test wheels
env:
# skip 32-bit, PyPy, and musllinux builds
CIBW_SKIP: "*-win32 *-manylinux_i686 pp* *musllinux*"
CIBW_BEFORE_BUILD_WINDOWS: pip install delvewheel
CIBW_ENVIRONMENT_WINDOWS: CMAKE_ARGS="-DCMAKE_MAKE_PROGRAM=D:/a/_temp/msys64/mingw64/bin/ninja.exe" CMAKE_PROGRAM_PATH="D:/a/_temp/msys64/usr/bin" CMAKE_GENERATOR="Ninja" TEMP="D:/a/_temp/"
CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: delvewheel repair -w {dest_dir} {wheel}
CIBW_BEFORE_BUILD_MACOS: brew reinstall gcc
CIBW_ENVIRONMENT_MACOS: MACOSX_DEPLOYMENT_TARGET=`sw_vers -productVersion` SYSTEM_VERSION_COMPAT=0
CIBW_TEST_REQUIRES: pytest pytest-order
CIBW_TEST_COMMAND: pytest -v -s -We -p no:unraisableexception {package}/tests
run: python -m cibuildwheel --output-dir dist

- uses: actions/upload-artifact@v4
with:
name: dist-${{matrix.os}}-${{matrix.manylinux}}
path: dist

run_example_notebooks:
needs: [build_wheels]
runs-on: ${{ matrix.os }}
name: Run example notebooks on ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, ubuntu-24.04-arm, macos-13, macos-latest, windows-latest]

steps:
- uses: actions/checkout@v4
with:
submodules: recursive
fetch-depth: 0

- uses: actions/[email protected]
with:
python-version: "3.13"

- uses: actions/download-artifact@v4
with:
name: dist-${{matrix.os}}-${{matrix.manylinux}}
path: dist

- run: |
temp=`find dist/ -name "*cp313*.whl"`
python -m pip install $temp[examples]
ex -sc 'g/^PyPartMC/d' -cx .binder/requirements.txt
python -m pip install --force-reinstall --no-deps $PIP_INSTALL_OPTS -r .binder/requirements.txt
python -m pip install $PIP_INSTALL_OPTS -r gitmodules/devops_tests/requirements.txt
GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} python -m pytest --durations=10 -v -s -We -p no:unraisableexception gitmodules/devops_tests/test_run_notebooks.py

check_annotated_todos:
runs-on: ubuntu-latest
name: Check if todos are annotated

steps:
- uses: actions/checkout@v4
with:
submodules: recursive
fetch-depth: 0

- run: |
python -m pip install pytest
python -m pip install $PIP_INSTALL_OPTS -r gitmodules/devops_tests/requirements.txt
GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} python -m pytest --durations=10 -v -s -We -p no:unraisableexception gitmodules/devops_tests/test_todos_annotated.py

dist_check:
runs-on: ${{ matrix.os }}
needs: [build_wheels]
strategy:
matrix:
os: [ubuntu-latest, ubuntu-24.04-arm]
steps:
- uses: actions/[email protected]
with:
python-version: "3.11"
- run: pip install twine auditwheel
- uses: actions/download-artifact@v4
with:
name: dist-${{matrix.os}}-${{matrix.manylinux}}
path: dist
- run: twine check --strict dist/*
- run: for i in dist/*-manylinux*.whl; do auditwheel show $i; done;

dist_upload:
runs-on: ubuntu-latest
needs: [build_wheels]
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v4
with:
pattern: dist-*
merge-multiple: true
path: dist
- if: github.event_name == 'push' && github.ref == 'refs/heads/main'
uses: pypa/gh-action-pypi-publish@release/v1.12
with:
repository_url: https://test.pypi.org/legacy/
attestations: false
- run: |
echo "github.event_name:" ${{ github.event_name }}
echo "github.event.action:" ${{ github.event.action }}
echo "github.event.prerelease:" ${{ github.event.prerelease }}
echo "env.GITHUB_REF:" ${{ env.GITHUB_REF }}
echo "env.GITHUB_REF:" ${{ env.GITHUB_REF_NAME }}
- if: github.event_name == 'release' && github.event.prerelease == false
uses: pypa/gh-action-pypi-publish@release/v1.12
with:
attestations: false
Loading
Loading