309 lines
20 KiB
Markdown
309 lines
20 KiB
Markdown
cibuildwheel
|
||
============
|
||
|
||
[](https://pypi.python.org/pypi/cibuildwheel)
|
||
[](https://cibuildwheel.pypa.io/en/stable/?badge=stable)
|
||
[](https://github.com/pypa/cibuildwheel/actions)
|
||
[](https://travis-ci.com/pypa/cibuildwheel)
|
||
[](https://ci.appveyor.com/project/joerick/cibuildwheel/branch/main)
|
||
[](https://circleci.com/gh/pypa/cibuildwheel)
|
||
[](https://dev.azure.com/joerick0429/cibuildwheel/_build/latest?definitionId=4&branchName=main)
|
||
|
||
|
||
[Documentation](https://cibuildwheel.pypa.io)
|
||
|
||
<!--intro-start-->
|
||
|
||
Python wheels are great. Building them across **Mac, Linux, Windows**, on **multiple versions of Python**, is not.
|
||
|
||
`cibuildwheel` is here to help. `cibuildwheel` runs on your CI server - currently it supports GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, and GitLab CI - and it builds and tests your wheels across all of your platforms.
|
||
|
||
|
||
What does it do?
|
||
----------------
|
||
|
||
| | macOS Intel | macOS Apple Silicon | Windows 64bit | Windows 32bit | Windows Arm64 | manylinux<br/>musllinux x86_64 | manylinux<br/>musllinux i686 | manylinux<br/>musllinux aarch64 | manylinux<br/>musllinux ppc64le | manylinux<br/>musllinux s390x |
|
||
|----------------|----|-----|-----|-----|-----|----|-----|----|-----|-----|
|
||
| CPython 3.6 | ✅ | N/A | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| CPython 3.7 | ✅ | N/A | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| CPython 3.8 | ✅ | ✅ | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| CPython 3.9 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| CPython 3.10 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| CPython 3.11 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| CPython 3.12 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| PyPy 3.7 v7.3 | ✅ | N/A | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
|
||
| PyPy 3.8 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
|
||
| PyPy 3.9 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
|
||
| PyPy 3.10 v7.3 | ✅ | ✅ | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
|
||
|
||
<sup>¹ PyPy is only supported for manylinux wheels.</sup><br>
|
||
<sup>² Windows arm64 support is experimental.</sup><br>
|
||
|
||
- Builds manylinux, musllinux, macOS 10.9+, and Windows wheels for CPython and PyPy
|
||
- Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, GitLab CI, and Cirrus CI
|
||
- Bundles shared library dependencies on Linux and macOS through [auditwheel](https://github.com/pypa/auditwheel) and [delocate](https://github.com/matthew-brett/delocate)
|
||
- Runs your library's tests against the wheel-installed version of your library
|
||
|
||
See the [cibuildwheel 1 documentation](https://cibuildwheel.pypa.io/en/1.x/) if you need to build unsupported versions of Python, such as Python 2.
|
||
|
||
Usage
|
||
-----
|
||
|
||
`cibuildwheel` runs inside a CI service. Supported platforms depend on which service you're using:
|
||
|
||
| | Linux | macOS | Windows | Linux ARM | macOS ARM | Windows ARM |
|
||
|-----------------|-------|-------|---------|-----------|-----------|-------------|
|
||
| GitHub Actions | ✅ | ✅ | ✅ | ✅¹ | ✅ | ✅² |
|
||
| Azure Pipelines | ✅ | ✅ | ✅ | | ✅ | ✅² |
|
||
| Travis CI | ✅ | | ✅ | ✅ | | |
|
||
| AppVeyor | ✅ | ✅ | ✅ | | ✅ | ✅² |
|
||
| CircleCI | ✅ | ✅ | | ✅ | ✅ | |
|
||
| Gitlab CI | ✅ | | ✅ | ✅¹ | | |
|
||
| Cirrus CI | ✅ | ✅ | ✅ | ✅ | ✅ | |
|
||
|
||
<sup>¹ [Requires emulation](https://cibuildwheel.pypa.io/en/stable/faq/#emulation), distributed separately. Other services may also support Linux ARM through emulation or third-party build hosts, but these are not tested in our CI.</sup><br>
|
||
<sup>² [Uses cross-compilation](https://cibuildwheel.pypa.io/en/stable/faq/#windows-arm64). It is not possible to test `arm64` on this CI platform.</sup>
|
||
|
||
<!--intro-end-->
|
||
|
||
Example setup
|
||
-------------
|
||
|
||
To build manylinux, musllinux, macOS, and Windows wheels on GitHub Actions, you could use this `.github/workflows/wheels.yml`:
|
||
|
||
```yaml
|
||
name: Build
|
||
|
||
on: [push, pull_request]
|
||
|
||
jobs:
|
||
build_wheels:
|
||
name: Build wheels on ${{ matrix.os }}
|
||
runs-on: ${{ matrix.os }}
|
||
strategy:
|
||
matrix:
|
||
os: [ubuntu-latest, windows-latest, macos-13, macos-14]
|
||
|
||
steps:
|
||
- uses: actions/checkout@v4
|
||
|
||
# Used to host cibuildwheel
|
||
- uses: actions/setup-python@v5
|
||
|
||
- name: Install cibuildwheel
|
||
run: python -m pip install cibuildwheel==2.17.0
|
||
|
||
- name: Build wheels
|
||
run: python -m cibuildwheel --output-dir wheelhouse
|
||
# to supply options, put them in 'env', like:
|
||
# env:
|
||
# CIBW_SOME_OPTION: value
|
||
|
||
- uses: actions/upload-artifact@v4
|
||
with:
|
||
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
|
||
path: ./wheelhouse/*.whl
|
||
```
|
||
|
||
For more information, including PyPI deployment, and the use of other CI services or the dedicated GitHub Action, check out the [documentation](https://cibuildwheel.pypa.io) and the [examples](https://github.com/pypa/cibuildwheel/tree/main/examples).
|
||
|
||
How it works
|
||
------------
|
||
|
||
The following diagram summarises the steps that cibuildwheel takes on each platform.
|
||
|
||
|
||
|
||
<sup>Explore an interactive version of this diagram [in the docs](https://cibuildwheel.pypa.io/en/stable/#how-it-works).</sup>
|
||
|
||
Options
|
||
-------
|
||
|
||
| | Option | Description |
|
||
|---|--------|-------------|
|
||
| **Build selection** | [`CIBW_PLATFORM`](https://cibuildwheel.pypa.io/en/stable/options/#platform) | Override the auto-detected target platform |
|
||
| | [`CIBW_BUILD`](https://cibuildwheel.pypa.io/en/stable/options/#build-skip) <br> [`CIBW_SKIP`](https://cibuildwheel.pypa.io/en/stable/options/#build-skip) | Choose the Python versions to build |
|
||
| | [`CIBW_ARCHS`](https://cibuildwheel.pypa.io/en/stable/options/#archs) | Change the architectures built on your machine by default. |
|
||
| | [`CIBW_PROJECT_REQUIRES_PYTHON`](https://cibuildwheel.pypa.io/en/stable/options/#requires-python) | Manually set the Python compatibility of your project |
|
||
| | [`CIBW_PRERELEASE_PYTHONS`](https://cibuildwheel.pypa.io/en/stable/options/#prerelease-pythons) | Enable building with pre-release versions of Python if available |
|
||
| **Build customization** | [`CIBW_BUILD_FRONTEND`](https://cibuildwheel.pypa.io/en/stable/options/#build-frontend) | Set the tool to use to build, either "pip" (default for now) or "build" |
|
||
| | [`CIBW_ENVIRONMENT`](https://cibuildwheel.pypa.io/en/stable/options/#environment) | Set environment variables needed during the build |
|
||
| | [`CIBW_ENVIRONMENT_PASS_LINUX`](https://cibuildwheel.pypa.io/en/stable/options/#environment-pass) | Set environment variables on the host to pass-through to the container during the build. |
|
||
| | [`CIBW_BEFORE_ALL`](https://cibuildwheel.pypa.io/en/stable/options/#before-all) | Execute a shell command on the build system before any wheels are built. |
|
||
| | [`CIBW_BEFORE_BUILD`](https://cibuildwheel.pypa.io/en/stable/options/#before-build) | Execute a shell command preparing each wheel's build |
|
||
| | [`CIBW_REPAIR_WHEEL_COMMAND`](https://cibuildwheel.pypa.io/en/stable/options/#repair-wheel-command) | Execute a shell command to repair each built wheel |
|
||
| | [`CIBW_MANYLINUX_*_IMAGE`<br/>`CIBW_MUSLLINUX_*_IMAGE`](https://cibuildwheel.pypa.io/en/stable/options/#linux-image) | Specify alternative manylinux / musllinux Docker images |
|
||
| | [`CIBW_CONTAINER_ENGINE`](https://cibuildwheel.pypa.io/en/stable/options/#container-engine) | Specify which container engine to use when building Linux wheels |
|
||
| | [`CIBW_DEPENDENCY_VERSIONS`](https://cibuildwheel.pypa.io/en/stable/options/#dependency-versions) | Specify how cibuildwheel controls the versions of the tools it uses |
|
||
| **Testing** | [`CIBW_TEST_COMMAND`](https://cibuildwheel.pypa.io/en/stable/options/#test-command) | Execute a shell command to test each built wheel |
|
||
| | [`CIBW_BEFORE_TEST`](https://cibuildwheel.pypa.io/en/stable/options/#before-test) | Execute a shell command before testing each wheel |
|
||
| | [`CIBW_TEST_REQUIRES`](https://cibuildwheel.pypa.io/en/stable/options/#test-requires) | Install Python dependencies before running the tests |
|
||
| | [`CIBW_TEST_EXTRAS`](https://cibuildwheel.pypa.io/en/stable/options/#test-extras) | Install your wheel for testing using extras_require |
|
||
| | [`CIBW_TEST_SKIP`](https://cibuildwheel.pypa.io/en/stable/options/#test-skip) | Skip running tests on some builds |
|
||
| **Other** | [`CIBW_BUILD_VERBOSITY`](https://cibuildwheel.pypa.io/en/stable/options/#build-verbosity) | Increase/decrease the output of pip wheel |
|
||
|
||
These options can be specified in a pyproject.toml file, as well; see [configuration](https://cibuildwheel.pypa.io/en/stable/options/#configuration).
|
||
|
||
Working examples
|
||
----------------
|
||
|
||
Here are some repos that use cibuildwheel.
|
||
|
||
<!-- START bin/projects.py -->
|
||
|
||
<!-- this section is generated by bin/projects.py. Don't edit it directly, instead, edit docs/data/projects.yml -->
|
||
|
||
| Name | CI | OS | Notes |
|
||
|-----------------------------------|----|----|:------|
|
||
| [scikit-learn][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | The machine learning library. A complex but clean config using many of cibuildwheel's features to build a large project with Cython and C++ extensions. |
|
||
| [pytorch-fairseq][] | ![github icon][] | ![apple icon][] ![linux icon][] | Facebook AI Research Sequence-to-Sequence Toolkit written in Python. |
|
||
| [NumPy][] | ![github icon][] ![travisci icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | The fundamental package for scientific computing with Python. |
|
||
| [Tornado][] | ![github icon][] | ![linux icon][] ![apple icon][] ![windows icon][] | Tornado is a Python web framework and asynchronous networking library. Uses stable ABI for a small C extension. |
|
||
| [Matplotlib][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | The venerable Matplotlib, a Python library with C++ portions |
|
||
| [NCNN][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | ncnn is a high-performance neural network inference framework optimized for the mobile platform |
|
||
| [Prophet][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth. |
|
||
| [MyPy][] | ![github icon][] | ![apple icon][] ![linux icon][] ![windows icon][] | The compiled version of MyPy using MyPyC. |
|
||
| [Kivy][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Open source UI framework written in Python, running on Windows, Linux, macOS, Android and iOS |
|
||
| [duckdb][] | ![github icon][] | ![apple icon][] ![linux icon][] ![windows icon][] | DuckDB is an in-process SQL OLAP Database Management System |
|
||
|
||
[scikit-learn]: https://github.com/scikit-learn/scikit-learn
|
||
[pytorch-fairseq]: https://github.com/pytorch/fairseq
|
||
[NumPy]: https://github.com/numpy/numpy
|
||
[Tornado]: https://github.com/tornadoweb/tornado
|
||
[Matplotlib]: https://github.com/matplotlib/matplotlib
|
||
[NCNN]: https://github.com/Tencent/ncnn
|
||
[Prophet]: https://github.com/facebook/prophet
|
||
[MyPy]: https://github.com/mypyc/mypy_mypyc-wheels
|
||
[Kivy]: https://github.com/kivy/kivy
|
||
[duckdb]: https://github.com/duckdb/duckdb
|
||
|
||
[appveyor icon]: docs/data/readme_icons/appveyor.svg
|
||
[github icon]: docs/data/readme_icons/github.svg
|
||
[azurepipelines icon]: docs/data/readme_icons/azurepipelines.svg
|
||
[circleci icon]: docs/data/readme_icons/circleci.svg
|
||
[gitlab icon]: docs/data/readme_icons/gitlab.svg
|
||
[travisci icon]: docs/data/readme_icons/travisci.svg
|
||
[cirrusci icon]: docs/data/readme_icons/cirrusci.svg
|
||
[windows icon]: docs/data/readme_icons/windows.svg
|
||
[apple icon]: docs/data/readme_icons/apple.svg
|
||
[linux icon]: docs/data/readme_icons/linux.svg
|
||
|
||
<!-- END bin/projects.py -->
|
||
|
||
> ℹ️ That's just a handful, there are many more! Check out the [Working Examples](https://cibuildwheel.pypa.io/en/stable/working-examples) page in the docs.
|
||
|
||
Legal note
|
||
----------
|
||
|
||
Since `cibuildwheel` repairs the wheel with `delocate` or `auditwheel`, it might automatically bundle dynamically linked libraries from the build machine.
|
||
|
||
It helps ensure that the library can run without any dependencies outside of the pip toolchain.
|
||
|
||
This is similar to static linking, so it might have some license implications. Check the license for any code you're pulling in to make sure that's allowed.
|
||
|
||
Changelog
|
||
=========
|
||
|
||
<!-- START bin/update_readme_changelog.py -->
|
||
|
||
<!-- this section was generated by bin/update_readme_changelog.py -- do not edit manually -->
|
||
|
||
### v2.17.0
|
||
|
||
_11 March 2024_
|
||
|
||
- 🌟 Adds the ability to inherit configuration in TOML overrides. This makes certain configurations much simpler. If you're overriding an option like `before-build` or `environment`, and you just want to add an extra command or environment variable, you can just append (or prepend) to the previous config. See [the docs](https://cibuildwheel.pypa.io/en/stable/options/#inherit) for more information. (#1730)
|
||
- 🌟 Adds official support for native arm64 macOS GitHub runners. To use them, just specify `macos-14` as an `os` of your job in your workflow file. You can also keep `macos-13` in your build matrix to build x86_64. Check out the new [GitHub Actions example config](https://cibuildwheel.pypa.io/en/stable/setup/#github-actions).
|
||
- ✨ You no longer need to specify `--platform` to run cibuildwheel locally! Instead it will detect your platform automatically. This was a safety feature, no longer necessary. (#1727)
|
||
- 🛠 Removed setuptools and wheel pinned versions. This only affects old-style projects without a `pyproject.toml`, projects with `pyproject.toml` are already getting fresh versions of their `build-system.requires` installed into an isolated environment. (#1725)
|
||
- 🛠 Improve how the GitHub Action passes arguments (#1757)
|
||
- 🛠 Remove a system-wide install of pipx in the GitHub Action (#1745)
|
||
- 🐛 No longer will cibuildwheel override the PIP_CONSTRAINT environment variable when using the `build` frontend. Instead it will be extended. (#1675)
|
||
- 🐛 Fix a bug where building and testing both x86_86 and arm64 wheels on the same runner caused the wrong architectures in the test environment (#1750)
|
||
- 🐛 Fix a bug that prevented testing a CPython 3.8 wheel targeting macOS 11+ on x86_64 (#1768)
|
||
- 📚 Moved the docs onto the official PyPA domain - they're now available at https://cibuildwheel.pypa.io . (#1775)
|
||
- 📚 Docs and examples improvements (#1762, #1734)
|
||
|
||
|
||
### v2.16.5
|
||
|
||
_30 January 2024_
|
||
|
||
- 🐛 Fix an incompatibility with the GitHub Action and new GitHub Runner images for Windows that bundle Powershell 7.3+ (#1741)
|
||
- 🛠 Preliminary support for new `macos-14` arm64 runners (#1743)
|
||
|
||
### v2.16.4
|
||
|
||
_28 January 2024_
|
||
|
||
- 🛠 Update manylinux pins to upgrade from a problematic PyPy version. (#1737)
|
||
|
||
### v2.16.3
|
||
|
||
_26 January 2024_
|
||
|
||
- 🐛 Fix a bug when building from sdist, where relative paths to files in the package didn't work because the working directory was wrong (#1687)
|
||
- 🛠 Adds the ability to disable mounting the host filesystem in containers to `/host`, through the `disable_host_mount` suboption on [`CIBW_CONTAINER_ENGINE`](https://cibuildwheel.pypa.io/en/stable/options/#container-engine).
|
||
- 📚 A lot of docs improvements! (#1708, #1705, #1686, #1679, #1667, #1665)
|
||
|
||
### v2.16.2
|
||
|
||
_3 October 2023_
|
||
|
||
- 🛠 Updates CPython 3.12 version to 3.12.0, final release (#1635)
|
||
- ✨ Adds a debug option [`CIBW_DEBUG_KEEP_CONTAINER`](https://cibuildwheel.pypa.io/en/stable/options/#cibw_debug_keep_container) to stop cibuildwheel deleting build containers after the build finishes. (#1620)
|
||
- 📚 Adds support for `[tool.cibuildwheel]` checking by adding a schema compatible with the [validate-pyproject](https://github.com/abravalheri/validate-pyproject/) tool (#1622, #1628, #1629)
|
||
- 🐛 Fix parsing of `CIBW_CONTAINER_ENGINE` and `CIBW_BUILD_FRONTEND` options to not break arguments on `:` characters (#1621)
|
||
- 🐛 Fix the evaluation order of `CIBW_ENVIRONMENT` and `CIBW_ENVIRONMENT_PASS` so that `CIBW_ENVIRONMENT` assignments can reference environment variables passed through from the host machine. (#1617)
|
||
- 🛠 Supports manylinux images' deferred installation of interpreters through the `manylinux-interpreters` tool (#1630)
|
||
|
||
<!-- END bin/update_readme_changelog.py -->
|
||
|
||
---
|
||
|
||
That's the last few versions.
|
||
|
||
ℹ️ **Want more changelog? Head over to [the changelog page in the docs](https://cibuildwheel.pypa.io/en/stable/changelog/).**
|
||
|
||
---
|
||
|
||
Contributing
|
||
============
|
||
|
||
For more info on how to contribute to cibuildwheel, see the [docs](https://cibuildwheel.pypa.io/en/latest/contributing/).
|
||
|
||
Everyone interacting with the cibuildwheel project via codebase, issue tracker, chat rooms, or otherwise is expected to follow the [PSF Code of Conduct](https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md).
|
||
|
||
Maintainers
|
||
-----------
|
||
|
||
- Joe Rickerby [@joerick](https://github.com/joerick)
|
||
- Yannick Jadoul [@YannickJadoul](https://github.com/YannickJadoul)
|
||
- Matthieu Darbois [@mayeut](https://github.com/mayeut)
|
||
- Henry Schreiner [@henryiii](https://github.com/henryiii)
|
||
- Grzegorz Bokota [@Czaki](https://github.com/Czaki)
|
||
|
||
Credits
|
||
-------
|
||
|
||
`cibuildwheel` stands on the shoulders of giants.
|
||
|
||
- ⭐️ @matthew-brett for [multibuild](https://github.com/multi-build/multibuild) and [matthew-brett/delocate](http://github.com/matthew-brett/delocate)
|
||
- @PyPA for the manylinux Docker images [pypa/manylinux](https://github.com/pypa/manylinux)
|
||
- @ogrisel for [wheelhouse-uploader](https://github.com/ogrisel/wheelhouse-uploader) and `run_with_env.cmd`
|
||
|
||
Massive props also to-
|
||
|
||
- @zfrenchee for [help debugging many issues](https://github.com/pypa/cibuildwheel/issues/2)
|
||
- @lelit for some great bug reports and [contributions](https://github.com/pypa/cibuildwheel/pull/73)
|
||
- @mayeut for a [phenomenal PR](https://github.com/pypa/cibuildwheel/pull/71) patching Python itself for better compatibility!
|
||
- @czaki for being a super-contributor over many PRs and helping out with countless issues!
|
||
- @mattip for his help with adding PyPy support to cibuildwheel
|
||
|
||
See also
|
||
========
|
||
|
||
Another very similar tool to consider is [matthew-brett/multibuild](http://github.com/matthew-brett/multibuild). `multibuild` is a shell script toolbox for building a wheel on various platforms. It is used as a basis to build some of the big data science tools, like SciPy.
|
||
|
||
If you are building Rust wheels, you can get by without some of the tricks required to make GLIBC work via manylinux; this is especially relevant for cross-compiling, which is easy with Rust. See [maturin-action](https://github.com/messense/maturin-action) for a tool that is optimized for building Rust wheels and cross-compiling.
|