Hongru/cibuildwheel
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'main' into gha-images-latest

Browse Source
This commit is contained in:
Matthieu Darbois
2024-02-10 09:58:58 +01:00
committed by GitHub
parent 6c93c131c4 81b0a578df
commit 75d6d9d55b
33 changed files with 245 additions and 176 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/changelog.md
+13
View File
@@ -4,6 +4,19 @@ title: Changelog
# Changelog
### v2.16.5
_30 January 2024_
- 🐛 Fix an incompatibility with the GitHub Action and new GitHub Runner images for Windows that bundle Powershell 7.3+ (#1741)
- 🛠 Preliminary support for new `macos-14` arm64 runners (#1743)
### v2.16.4
_28 January 2024_
- 🛠 Update manylinux pins to upgrade from a problematic PyPy version. (#1737)
### v2.16.3
_26 January 2024_
docs/data/projects.schema.json
+37
View File
@@ -0,0 +1,37 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://github.com/pypa/cibuildwheel/blob/main/cibuildwheel/docs/data/projects.schema.json",
"type": "array",
"description": "The projects file for cibuildwheel",
"items": {
"type": "object",
"additionalProperties": false,
"properties": {
"name": { "type": "string" },
"gh": { "type": "string", "pattern": "[^/]*/[^/]*" },
"stars": { "$ref": "#/items/properties/gh" },
"pypi": { "type": "string" },
"notes": { "type": "string" },
"ci": {
"type": "array",
"items": {
"enum": [
"github",
"travisci",
"appveyor",
"circleci",
"gitlab",
"cirrusci",
"azurepipelines"
]
}
},
"ci_config": { "type": "string" },
"os": {
"type": "array",
"items": { "enum": ["windows", "apple", "linux"] }
}
},
"required": ["name", "gh", "ci", "os"]
}
}
docs/data/projects.yml
+9 -2
View File
@@ -603,13 +603,13 @@
gh: tensorchord/envd
ci: [github]
os: [apple, linux, windows]
note: A machine learning development environment build tool
notes: A machine learning development environment build tool
- name: mosec
gh: mosecorg/mosec
ci: [github]
os: [linux, apple]
note: A machine learning model serving framework powered by Rust
notes: A machine learning model serving framework powered by Rust
- name: ril
gh: Cryptex-github/ril-py
@@ -638,3 +638,10 @@
os: [windows, linux, apple]
pypi: werpy
notes: An ultra-fast python package using optimized dynamic programming to compute the Word Error Rate (WER).
- name: keyvi
gh: KeyviDev/keyvi
ci: [github]
os: [linux, apple]
pypi: keyvi
notes: FST based key value index highly optimized for size and lookup performance, utilizes ccache action for improved runtime
docs/extra.css
+1 -1
View File
@@ -255,7 +255,7 @@ h1, h2, h3, h4, h5, h6 {
border: none;
color: inherit;
background-color: #f0f1f1;
font-size: 100%;
font-size: 80%;
}
docs/faq.md
+14 -31
View File
@@ -20,11 +20,11 @@ Linux wheels are built in [`manylinux`/`musllinux` containers](https://github.co
### Building macOS wheels for Apple Silicon {: #apple-silicon}
`cibuildwheel` supports cross-compiling `universal2` and `arm64` wheels on `x86_64` runners.
`cibuildwheel` supports both native builds and cross-compiling between `arm64` (Apple Silicon) and `x86_64` (Intel) architectures, including the cross-compatible `universal2` format.
#### Overview of Mac architectures
With the introduction of Apple Silicon, you now have several choices for wheels for Python 3.8+:
You have several choices for wheels for Python 3.8+:
##### `x86_64`
@@ -68,43 +68,26 @@ Opinions vary on which of arch-specific or `universal2` wheels are best - some p
See [GitHub issue 1333](https://github.com/pypa/cibuildwheel/issues/1333) for more discussion.
#### How to cross-compile
#### How?
With the exception of Cirrus CI, macOS CI runners are still Intel-based, and Apple Silicon wheels are not built by default on Intel runners. However, cross-compilation can be enabled by adding extra archs to the [`CIBW_ARCHS_MACOS` option](options.md#archs) - e.g. `x86_64 arm64`. Cross-compilation is provided by the Xcode toolchain.
It's easiest to build `x86_64` wheels on `x86_64` runners, and `arm64` wheels on `arm64` runners.
!!! important
When cross-compiling on Intel, it is not possible to test `arm64` and the `arm64` part of a `universal2` wheel.
On GitHub Actions, `macos-14` runners are `arm64`, and `macos-13` runners are `x86_64`. So all you need to do is ensure both are in your build matrix.
`cibuildwheel` will raise a warning to notify you of this - these warnings be be silenced by skipping testing on these platforms: `CIBW_TEST_SKIP: "*_arm64 *_universal2:arm64"`.
#### Cross-compiling
!!! note
Your runner needs Xcode Command Line Tools 12.2 or later to build `universal2` or `arm64`.
If your CI provider doesn't offer arm64 runners yet, or you want to create `universal2`, you'll have to cross-compile. Cross-compilation can be enabled by adding extra archs to the [`CIBW_ARCHS_MACOS` option](options.md#archs) - e.g. `CIBW_ARCHS_MACOS="x86_64 universal2"`. Cross-compilation is provided by Xcode toolchain v12.2+.
Only CPython 3.8 and newer support `universal2` and `arm64` wheels.
Regarding testing,
- On an arm64 runner, it is possible to test x86_64 wheels and both parts of a universal2 wheel using Rosetta 2 emulation.
- On an x86_64 runner, arm64 code can be compiled but it can't be tested. `cibuildwheel` will raise a warning to notify you of this - these warnings can be silenced by skipping testing on these platforms: `CIBW_TEST_SKIP: "*_arm64 *_universal2:arm64"`.
!!! note
If your project uses **Poetry** as a build backend, cross-compiling on macOS [does not currently work](https://github.com/python-poetry/poetry/issues/7107). In some cases arm64 wheels can be built but their tags will be incorrect, with the platform tag showing `x86_64` instead of `arm64`.
As a workaround, the tag can be fixed before running delocate to repair the wheel. The [`wheel tags`](https://wheel.readthedocs.io/en/stable/reference/wheel_tags.html) command is ideal for this. See [this workflow](https://gist.github.com/anderssonjohan/49f07e33fc5cb2420515a8ac76dc0c95#file-build-pendulum-wheels-yml-L39-L53) for an example usage of `wheel tags`.
Hopefully, cross-compilation is a temporary situation. Once we have widely available Apple Silicon CI runners, we can build `arm64` wheels natively. Until then, cross-compiling `arm64` or `universal2` wheels requires opt-in by setting `CIBW_ARCHS_MACOS` on Intel runners.
Here's an example GitHub Actions workflow with a job that builds for Apple Silicon:
> .github/workflows/build_macos.yml
```yml
{% include "../examples/github-apple-silicon.yml" %}
```
Here's an example Cirrus CI workflow with a job that builds for macOS Intel through Rosetta 2 emulation and for Apple Silicon natively:
> .cirrus.yml
```yml
{% include "../examples/cirrus-ci-intel-mac.yml" %}
```
### Building Linux wheels for non-native archs using emulation {: #emulation}
cibuildwheel supports building non-native architectures on Linux, via
@@ -159,7 +142,7 @@ There are two suggested methods for keeping cibuildwheel up to date that instead
If you use GitHub Actions for builds, you can use cibuildwheel as an action:
```yaml
uses: pypa/cibuildwheel@v2.16.3
uses: pypa/cibuildwheel@v2.16.5
```
This is a composite step that just runs cibuildwheel using pipx. You can set command-line options as `with:` parameters, and use `env:` as normal.
@@ -181,7 +164,7 @@ The second option, and the only one that supports other CI systems, is using a `
```bash
# requirements-cibw.txt
cibuildwheel==2.16.3
cibuildwheel==2.16.5
```
Then your install step would have `python -m pip install -r requirements-cibw.txt` in it. Your `.github/dependabot.yml` file could look like this:
@@ -324,7 +307,7 @@ Solutions to this vary, but the simplest is to use pipx:
# most runners have pipx preinstalled, but in case you don't
python3 -m pip install pipx
pipx run cibuildwheel==2.16.3 --output-dir wheelhouse
pipx run cibuildwheel==2.16.5 --output-dir wheelhouse
pipx run twine upload wheelhouse/*.whl
```
docs/options.md
+2 -2
View File
@@ -343,8 +343,8 @@ See the [cibuildwheel 1 documentation](https://cibuildwheel.readthedocs.io/en/1.
A list of architectures to build.
On macOS, this option can be used to cross-compile between `x86_64`,
`universal2` and `arm64` for Apple Silicon support.
On macOS, this option can be used to [cross-compile](faq.md#cross-compiling)
between `x86_64`, `universal2` and `arm64`.
On Linux, this option can be used to build non-native architectures under
emulation. See [this guide](faq.md#emulation) for more information.
docs/setup.md
+6 -4
View File
@@ -178,13 +178,14 @@ To build Linux, Mac, and Windows wheels using GitHub Actions, create a `.github/
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-12]
# macos-13 is an intel runner, macos-14 is apple silicon
os: [ubuntu-latest, windows-latest, macos-13, macos-14]
steps:
- uses: actions/checkout@v4
- name: Build wheels
run: pipx run cibuildwheel==2.16.3
run: pipx run cibuildwheel==2.16.5
- uses: actions/upload-artifact@v4
with:
@@ -211,7 +212,8 @@ To build Linux, Mac, and Windows wheels using GitHub Actions, create a `.github/
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-12]
# macos-13 is an intel runner, macos-14 is apple silicon
os: [ubuntu-latest, windows-latest, macos-13, macos-14]
steps:
- uses: actions/checkout@v4
@@ -220,7 +222,7 @@ To build Linux, Mac, and Windows wheels using GitHub Actions, create a `.github/
- uses: actions/setup-python@v3
- name: Install cibuildwheel
run: python -m pip install cibuildwheel==2.16.3
run: python -m pip install cibuildwheel==2.16.5
- name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse
docs/working-examples.md
+6 -6
View File
@@ -86,15 +86,15 @@ title: Working examples
| [iminuit][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Jupyter-friendly Python interface for C++ MINUIT2 |
| [Tokenizer][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Fast and customizable text tokenization library with BPE and SentencePiece support |
| [PyGLM][] | ![github icon][] | ![apple icon][] ![linux icon][] ![windows icon][] | Fast OpenGL Mathematics (GLM) for Python |
| [TgCrypto][] | ![travisci icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Includes a Windows Travis build. |
| [pillow-heif][] | ![github icon][] ![cirrusci icon][] | ![apple icon][] ![linux icon][] ![windows icon][] | Bindings to libheif library with third party dependencies. Fully automated CI for tests and publishing including Apple Silicon builds. |
| [iDynTree][] | ![github icon][] | ![linux icon][] | Uses manylinux_2_24 |
| [TgCrypto][] | ![travisci icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Includes a Windows Travis build. |
| [bx-python][] | ![travisci icon][] | ![apple icon][] ![linux icon][] | A library that includes Cython extensions. |
| [iDynTree][] | ![github icon][] | ![linux icon][] | Uses manylinux_2_24 |
| [boost-histogram][] | ![github icon][] ![travisci icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Supports full range of wheels, including PyPy and alternate archs. |
| [Python-WebRTC][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | a Python extension that provides bindings to WebRTC M92 |
| [pybase64][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Fast Base64 encoding/decoding in Python |
| [fathon][] | ![travisci icon][] | ![apple icon][] ![linux icon][] | python package for DFA (Detrended Fluctuation Analysis) and related algorithms |
| [power-grid-model][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Python/C++ library for distribution power system analysis |
| [fathon][] | ![travisci icon][] | ![apple icon][] ![linux icon][] | python package for DFA (Detrended Fluctuation Analysis) and related algorithms |
| [Arbor][] | ![github icon][] | ![apple icon][] ![linux icon][] | Arbor is a multi-compartment neuron simulation library; compatible with next-generation accelerators; best-practices applied to research software; focused on community-driven development. Includes a [small script](https://github.com/arbor-sim/arbor/blob/master/scripts/patchwheel.py) patching `rpath` in bundled libraries. |
| [Imagecodecs (fork)][] | ![azurepipelines icon][] | ![apple icon][] ![linux icon][] | Over 20 external dependencies in compiled libraries, custom docker image, `libomp`, `openblas` and `install_name_tool` for macOS. |
| [pybind11 scikit_build_example][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | An example combining scikit-build and pybind11 |
@@ -193,15 +193,15 @@ title: Working examples
[iminuit]: https://github.com/scikit-hep/iminuit
[Tokenizer]: https://github.com/OpenNMT/Tokenizer
[PyGLM]: https://github.com/Zuzu-Typ/PyGLM
[TgCrypto]: https://github.com/pyrogram/tgcrypto
[pillow-heif]: https://github.com/bigcat88/pillow_heif
[iDynTree]: https://github.com/robotology/idyntree
[TgCrypto]: https://github.com/pyrogram/tgcrypto
[bx-python]: https://github.com/bxlab/bx-python
[iDynTree]: https://github.com/robotology/idyntree
[boost-histogram]: https://github.com/scikit-hep/boost-histogram
[Python-WebRTC]: https://github.com/MarshalX/python-webrtc
[pybase64]: https://github.com/mayeut/pybase64
[fathon]: https://github.com/stfbnc/fathon
[power-grid-model]: https://github.com/alliander-opensource/power-grid-model
[fathon]: https://github.com/stfbnc/fathon
[Arbor]: https://github.com/arbor-sim/arbor
[Imagecodecs (fork)]: https://github.com/czaki/imagecodecs_build
[pybind11 scikit_build_example]: https://github.com/pybind/scikit_build_example