Feat/build multiple wheels

[Oisin-M]

Adds the ability to also build wheels for packages. This is needed e.g. for Rust-Python packages.

This PR:

• maintains the previous functionality by default, but makes it explicit now to only python -m build --sdist instead of a generic python -m build. The checks on versions etc. are still done here
• adds optional extra task that depends on this first task with the ability to build wheels for config-specified platforms and Python versions

An example successful workflow building wheels for a rust-python package is https://github.com/ecmwf/earthkit-hydro/actions/runs/14662296289/job/41149252856, which is largely driven via the pyproject.toml. I have a version on https://github.com/ecmwf/earthkit-hydro/tree/develop that's working

[iainrussell]

Hi Oisin, this looks nice, but it does not seem to work for other packages. I made a test release of findlibs, and it does not like the empty os matrix:
https://github.com/ecmwf/findlibs/actions/runs/14665097091

Error when evaluating 'strategy' for job 'deploy_wheels'. ecmwf/reusable-workflows/.github/workflows/cd-pypi.yml@feat/build_multiple_wheels (Line: 118, Col: 13): Matrix vector 'os' does not contain any values,ecmwf/reusable-workflows/.github/workflows/cd-pypi.yml@feat/build_multiple_wheels (Line: 119, Col: 17): Matrix vector 'python' does not contain any values

[Oisin-M]

Just documenting after further investigation and discussion.

Two current issues:

1. Sometimes the pyversion and platforms default values (namely []) cause an Error when evaluating 'strategy' but not always.
2. The previous default functionality was python -m build which also may build a wheel. Changing now to --sdist will no longer build the wheel which is undesirable and we should avoid. However, actually the previous python -m build also caused issues in some cases with rejections from PyPI e.g.

Binary wheel
'earthkit_hydro-0.1.4.dev58-cp313-cp313-linux_x86_64.whl' has
an unsupported platform tag 'linux_x86_64'.

Will investigate further, but it seems like the easiest solution would be

1. Make an explicit if check to see if pyversion or platforms are empty to avoid allowing it to even get to the strategy
2. Add a boolean sdist_only with default False to switch between python -m build and python -m build --sdist. This retains the old functionality by default but makes it customisable.

[iainrussell]

Thanks @Oisin-M and @tmi - I tested it with findlibs and it's working as it used to :) I'm happy to merge this in if you're also happy for me to do it

[Oisin-M]

Agreed to separate the wheels out from the cd for pure python packages. Plan is to have separate

cd-pypi.yml
cd-pypi-binwheel.yml
cd-pypi-cppwrapperwheel.yml

[Oisin-M]

I have now split the action for binary wheels out from the original cd-pypi action.

Overview of changes are as follows

• cd-pypi.yml:
  • can now give buildargs i.e. python -m build {buildargs}
  • can now set env variables from input env_vars
• cd-pypi-binwheel.yml
  • can also set env variables from input env_vars
  • can specify platforms and python versions to build for

I've now got a way using these actions to build both purepython wheels as well as binary wheels in earthkit-hydro.

[iainrussell]

Maybe better to rename to cd-pypi-binwheel to avoid confusion

[iainrussell]

You might want to consider adding the tag/branch checks that cd-pypi has, for consistency. These checks actually saved us from an erroneous release a couple of weeks ago!