Hongru/cibuildwheel
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

chore(CI): test on GHA macOS-14 runner (#1700)

Browse Source 
* chore(CI): test on GHA macOS-14 runner
* update error message for skipped arm64 tests
* Update docs, README and examples

---------

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Joe Rickerby <joerick@mac.com>
Co-authored-by: Andrew Murray <3112309+radarhere@users.noreply.github.com>
This commit is contained in:
Matthieu Darbois
2024-02-10 09:55:38 +01:00
committed by GitHub
co-authored by Henry Schreiner Joe Rickerby Andrew Murray
parent 27d0942e1c
commit 81b0a578df
12 changed files with 46 additions and 85 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/test.yml
+1 -1
View File
@@ -38,7 +38,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, windows-latest, macos-11]
os: [ubuntu-20.04, windows-latest, macos-13, macos-14]
python_version: ['3.12']
include:
- os: ubuntu-22.04
README.md
+12 -17
View File
@@ -27,20 +27,17 @@ What does it do?
| CPython 3.6 | ✅ | N/A | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ |
| CPython 3.7 | ✅ | N/A | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ |
| CPython 3.8 | ✅ | ✅ | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ |
| CPython 3.9 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅³ | ✅ | ✅ | ✅ | ✅ |
| CPython 3.9 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
| CPython 3.10 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
| CPython 3.11 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
| CPython 3.12⁵ | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
| CPython 3.12 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
| PyPy 3.7 v7.3 | ✅ | N/A | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
| PyPy 3.8 v7.3 | ✅ | ✅⁴ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
| PyPy 3.9 v7.3 | ✅ | ✅⁴ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
| PyPy 3.10 v7.3 | ✅ | ✅⁴ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
| PyPy 3.8 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
| PyPy 3.9 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
| PyPy 3.10 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
<sup>¹ PyPy is only supported for manylinux wheels.</sup><br>
<sup>² Windows arm64 support is experimental.</sup><br>
<sup>³ Alpine 3.14 and very briefly 3.15's default python3 [was not able to load](https://github.com/pypa/cibuildwheel/issues/934) musllinux wheels. This has been fixed; please upgrade the python package if using Alpine from before the fix.</sup><br>
<sup>⁴ Cross-compilation not supported with PyPy - to build these wheels you need to run cibuildwheel on an Apple Silicon machine.</sup><br>
<sup>⁵ CPython 3.12 is built by default using Python RCs, starting with cibuildwheel 2.15.</sup><br>
- Builds manylinux, musllinux, macOS 10.9+, and Windows wheels for CPython and PyPy
- Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, GitLab CI, and Cirrus CI
@@ -56,18 +53,16 @@ Usage
| | Linux | macOS | Windows | Linux ARM | macOS ARM | Windows ARM |
|-----------------|-------|-------|---------|-----------|-----------|-------------|
| GitHub Actions | ✅ | ✅ | ✅ | ✅¹ | ✅² | ✅⁴ |
| Azure Pipelines | ✅ | ✅ | ✅ | | ✅² | ✅⁴ |
| GitHub Actions | ✅ | ✅ | ✅ | ✅¹ | ✅ | ✅² |
| Azure Pipelines | ✅ | ✅ | ✅ | | ✅ | ✅² |
| Travis CI | ✅ | | ✅ | ✅ | | |
| AppVeyor | ✅ | ✅ | ✅ | | ✅² | ✅⁴ |
| CircleCI | ✅ | ✅ | | ✅ | ✅² | |
| AppVeyor | ✅ | ✅ | ✅ | | ✅ | ✅² |
| CircleCI | ✅ | ✅ | | ✅ | ✅ | |
| Gitlab CI | ✅ | | ✅ | ✅¹ | | |
| Cirrus CI | ✅ | ✅³ | ✅ | ✅ | ✅ | |
| Cirrus CI | ✅ | ✅ | ✅ | ✅ | ✅ | |
<sup>¹ [Requires emulation](https://cibuildwheel.readthedocs.io/en/stable/faq/#emulation), distributed separately. Other services may also support Linux ARM through emulation or third-party build hosts, but these are not tested in our CI.</sup><br>
<sup>² [Uses cross-compilation](https://cibuildwheel.readthedocs.io/en/stable/faq/#universal2). It is not possible to test `arm64` and the `arm64` part of a `universal2` wheel on this CI platform.</sup><br>
<sup>³ [Uses cross-compilation](https://cibuildwheel.readthedocs.io/en/stable/faq/#universal2). Thanks to Rosetta 2 emulation, it is possible to test `x86_64` and both parts of a `universal2` wheel on this CI platform.</sup><br>
<sup>⁴ [Uses cross-compilation](https://cibuildwheel.readthedocs.io/en/stable/faq/#windows-arm64). It is not possible to test `arm64` on this CI platform.</sup>
<sup>² [Uses cross-compilation](https://cibuildwheel.readthedocs.io/en/stable/faq/#windows-arm64). It is not possible to test `arm64` on this CI platform.</sup>
<!--intro-end-->
@@ -87,7 +82,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, windows-2019, macOS-11]
os: [ubuntu-latest, windows-latest, macos-13, macos-14]
steps:
- uses: actions/checkout@v4
cibuildwheel/macos.py
+7 -8
View File
@@ -488,9 +488,8 @@ def build(options: Options, tmp_path: Path) -> None:
unwrap(
"""
While arm64 wheels can be built on x86_64, they cannot be
tested. The ability to test the arm64 wheels will be added in a
future release of cibuildwheel, once Apple Silicon CI runners
are widely available. To silence this warning, set
tested. Consider building arm64 wheels natively, if your CI
provider offers this. To silence this warning, set
`CIBW_TEST_SKIP: "*-macosx_arm64"`.
"""
)
@@ -500,11 +499,11 @@ def build(options: Options, tmp_path: Path) -> None:
unwrap(
"""
While universal2 wheels can be built on x86_64, the arm64 part
of them cannot currently be tested. The ability to test the
arm64 part of a universal2 wheel will be added in a future
release of cibuildwheel, once Apple Silicon CI runners are
widely available. To silence this warning, set
`CIBW_TEST_SKIP: "*-macosx_universal2:arm64"`.
of the wheel cannot be tested on x86_64. Consider building
universal2 wheels on an arm64 runner, if your CI provider offers
this. Notably, an arm64 runner can also test the x86_64 part of
the wheel, through Rosetta emulation. To silence this warning,
set `CIBW_TEST_SKIP: "*-macosx_universal2:arm64"`.
"""
)
)
docs/extra.css
+1 -1
View File
@@ -255,7 +255,7 @@ h1, h2, h3, h4, h5, h6 {
border: none;
color: inherit;
background-color: #f0f1f1;
font-size: 100%;
font-size: 80%;
}
docs/faq.md
+11 -28
View File
@@ -20,11 +20,11 @@ Linux wheels are built in [`manylinux`/`musllinux` containers](https://github.co
### Building macOS wheels for Apple Silicon {: #apple-silicon}
`cibuildwheel` supports cross-compiling `universal2` and `arm64` wheels on `x86_64` runners.
`cibuildwheel` supports both native builds and cross-compiling between `arm64` (Apple Silicon) and `x86_64` (Intel) architectures, including the cross-compatible `universal2` format.
#### Overview of Mac architectures
With the introduction of Apple Silicon, you now have several choices for wheels for Python 3.8+:
You have several choices for wheels for Python 3.8+:
##### `x86_64`
@@ -68,43 +68,26 @@ Opinions vary on which of arch-specific or `universal2` wheels are best - some p
See [GitHub issue 1333](https://github.com/pypa/cibuildwheel/issues/1333) for more discussion.
#### How to cross-compile
#### How?
With the exception of Cirrus CI, macOS CI runners are still Intel-based, and Apple Silicon wheels are not built by default on Intel runners. However, cross-compilation can be enabled by adding extra archs to the [`CIBW_ARCHS_MACOS` option](options.md#archs) - e.g. `x86_64 arm64`. Cross-compilation is provided by the Xcode toolchain.
It's easiest to build `x86_64` wheels on `x86_64` runners, and `arm64` wheels on `arm64` runners.
!!! important
When cross-compiling on Intel, it is not possible to test `arm64` and the `arm64` part of a `universal2` wheel.
On GitHub Actions, `macos-14` runners are `arm64`, and `macos-13` runners are `x86_64`. So all you need to do is ensure both are in your build matrix.
`cibuildwheel` will raise a warning to notify you of this - these warnings be be silenced by skipping testing on these platforms: `CIBW_TEST_SKIP: "*_arm64 *_universal2:arm64"`.
#### Cross-compiling
!!! note
Your runner needs Xcode Command Line Tools 12.2 or later to build `universal2` or `arm64`.
If your CI provider doesn't offer arm64 runners yet, or you want to create `universal2`, you'll have to cross-compile. Cross-compilation can be enabled by adding extra archs to the [`CIBW_ARCHS_MACOS` option](options.md#archs) - e.g. `CIBW_ARCHS_MACOS="x86_64 universal2"`. Cross-compilation is provided by Xcode toolchain v12.2+.
Only CPython 3.8 and newer support `universal2` and `arm64` wheels.
Regarding testing,
- On an arm64 runner, it is possible to test x86_64 wheels and both parts of a universal2 wheel using Rosetta 2 emulation.
- On an x86_64 runner, arm64 code can be compiled but it can't be tested. `cibuildwheel` will raise a warning to notify you of this - these warnings can be silenced by skipping testing on these platforms: `CIBW_TEST_SKIP: "*_arm64 *_universal2:arm64"`.
!!! note
If your project uses **Poetry** as a build backend, cross-compiling on macOS [does not currently work](https://github.com/python-poetry/poetry/issues/7107). In some cases arm64 wheels can be built but their tags will be incorrect, with the platform tag showing `x86_64` instead of `arm64`.
As a workaround, the tag can be fixed before running delocate to repair the wheel. The [`wheel tags`](https://wheel.readthedocs.io/en/stable/reference/wheel_tags.html) command is ideal for this. See [this workflow](https://gist.github.com/anderssonjohan/49f07e33fc5cb2420515a8ac76dc0c95#file-build-pendulum-wheels-yml-L39-L53) for an example usage of `wheel tags`.
Hopefully, cross-compilation is a temporary situation. Once we have widely available Apple Silicon CI runners, we can build `arm64` wheels natively. Until then, cross-compiling `arm64` or `universal2` wheels requires opt-in by setting `CIBW_ARCHS_MACOS` on Intel runners.
Here's an example GitHub Actions workflow with a job that builds for Apple Silicon:
> .github/workflows/build_macos.yml
```yml
{% include "../examples/github-apple-silicon.yml" %}
```
Here's an example Cirrus CI workflow with a job that builds for macOS Intel through Rosetta 2 emulation and for Apple Silicon natively:
> .cirrus.yml
```yml
{% include "../examples/cirrus-ci-intel-mac.yml" %}
```
### Building Linux wheels for non-native archs using emulation {: #emulation}
cibuildwheel supports building non-native architectures on Linux, via
docs/options.md
+2 -2
View File
@@ -343,8 +343,8 @@ See the [cibuildwheel 1 documentation](https://cibuildwheel.readthedocs.io/en/1.
A list of architectures to build.
On macOS, this option can be used to cross-compile between `x86_64`,
`universal2` and `arm64` for Apple Silicon support.
On macOS, this option can be used to [cross-compile](faq.md#cross-compiling)
between `x86_64`, `universal2` and `arm64`.
On Linux, this option can be used to build non-native architectures under
emulation. See [this guide](faq.md#emulation) for more information.
docs/setup.md
+4 -2
View File
@@ -178,7 +178,8 @@ To build Linux, Mac, and Windows wheels using GitHub Actions, create a `.github/
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, windows-2019, macos-11]
# macos-13 is an intel runner, macos-14 is apple silicon
os: [ubuntu-20.04, windows-2019, macos-13, macos-14]
steps:
- uses: actions/checkout@v4
@@ -211,7 +212,8 @@ To build Linux, Mac, and Windows wheels using GitHub Actions, create a `.github/
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, windows-2019, macos-11]
# macos-13 is an intel runner, macos-14 is apple silicon
os: [ubuntu-20.04, windows-2019, macos-13, macos-14]
steps:
- uses: actions/checkout@v4
examples/github-apple-silicon.yml
-20
View File
@@ -1,20 +0,0 @@
name: Build
on: [push, pull_request]
jobs:
build_wheels_macos:
name: Build wheels on macos-11
runs-on: macos-11
steps:
- uses: actions/checkout@v4
- name: Build wheels
uses: pypa/cibuildwheel@v2.16.5
env:
CIBW_ARCHS_MACOS: x86_64 arm64
- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-macos
path: ./wheelhouse/*.whl
examples/github-deploy.yml
+2 -1
View File
@@ -16,7 +16,8 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-22.04, windows-2022, macos-11]
# macos-13 is an intel runner, macos-14 is apple silicon
os: [ubuntu-22.04, windows-2022, macos-13, macos-14]
steps:
- uses: actions/checkout@v4
examples/github-minimal.yml
+2 -1
View File
@@ -8,7 +8,8 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, windows-2019, macos-11]
# macos-13 is an intel runner, macos-14 is apple silicon
os: [ubuntu-20.04, windows-2019, macos-13, macos-14]
steps:
- uses: actions/checkout@v4
examples/github-with-qemu.yml
+1 -1
View File
@@ -8,7 +8,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, windows-2019, macos-11]
os: [ubuntu-20.04, windows-2019, macos-13, macos-14]
steps:
- uses: actions/checkout@v4
test/test_macos_archs.py
+3 -3
View File
@@ -83,7 +83,7 @@ def test_cross_compiled_test(tmp_path, capfd, build_universal2):
assert "running tests on arm64" not in captured.out
if build_universal2:
assert (
"While universal2 wheels can be built on x86_64, the arm64 part of them cannot currently be tested"
"While universal2 wheels can be built on x86_64, the arm64 part of the wheel cannot be tested"
in captured.err
)
else:
@@ -95,7 +95,7 @@ def test_cross_compiled_test(tmp_path, capfd, build_universal2):
assert "running tests on x86_64" in captured.out
assert "running tests on arm64" in captured.out
assert (
"While universal2 wheels can be built on x86_64, the arm64 part of them cannot currently be tested"
"While universal2 wheels can be built on x86_64, the arm64 part of the wheel cannot be tested"
not in captured.err
)
assert (
@@ -162,7 +162,7 @@ def test_universal2_testing_on_x86_64(tmp_path, capfd, skip_arm64_test):
assert "running tests on x86_64" in captured.out
assert "running tests on arm64" not in captured.out
warning_message = "While universal2 wheels can be built on x86_64, the arm64 part of them cannot currently be tested"
warning_message = "While universal2 wheels can be built on x86_64, the arm64 part of the wheel cannot be tested"
if skip_arm64_test:
assert warning_message not in captured.err
else: