296 lines
19 KiB
Markdown
296 lines
19 KiB
Markdown
cibuildwheel
|
||
============
|
||
|
||
[](https://pypi.python.org/pypi/cibuildwheel)
|
||
[](https://cibuildwheel.readthedocs.io/en/stable/?badge=stable)
|
||
[](https://github.com/pypa/cibuildwheel/actions)
|
||
[](https://travis-ci.com/pypa/cibuildwheel)
|
||
[](https://ci.appveyor.com/project/joerick/cibuildwheel/branch/main)
|
||
[](https://circleci.com/gh/pypa/cibuildwheel)
|
||
[](https://dev.azure.com/joerick0429/cibuildwheel/_build/latest?definitionId=4&branchName=main)
|
||
|
||
|
||
[Documentation](https://cibuildwheel.readthedocs.org)
|
||
|
||
<!--intro-start-->
|
||
|
||
Python wheels are great. Building them across **Mac, Linux, Windows**, on **multiple versions of Python**, is not.
|
||
|
||
`cibuildwheel` is here to help. `cibuildwheel` runs on your CI server - currently it supports GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, and GitLab CI - and it builds and tests your wheels across all of your platforms.
|
||
|
||
|
||
What does it do?
|
||
----------------
|
||
|
||
| | macOS Intel | macOS Apple Silicon | Windows 64bit | Windows 32bit | Windows Arm64 | manylinux<br/>musllinux x86_64 | manylinux<br/>musllinux i686 | manylinux<br/>musllinux aarch64 | manylinux<br/>musllinux ppc64le | manylinux<br/>musllinux s390x |
|
||
|---------------|----|-----|-----|-----|-----|----|-----|----|-----|-----|
|
||
| CPython 3.6 | ✅ | N/A | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| CPython 3.7 | ✅ | N/A | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| CPython 3.8 | ✅ | ✅ | ✅ | ✅ | N/A | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| CPython 3.9 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅³ | ✅ | ✅ | ✅ | ✅ |
|
||
| CPython 3.10 | ✅ | ✅ | ✅ | ✅ | ✅² | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| PyPy 3.7 v7.3 | ✅ | N/A | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
|
||
| PyPy 3.8 v7.3 | ✅ | N/A | ✅ | N/A | N/A | ✅¹ | ✅¹ | ✅¹ | N/A | N/A |
|
||
|
||
<sup>¹ PyPy is only supported for manylinux wheels.</sup><br>
|
||
<sup>² Windows arm64 support is experimental.</sup><br>
|
||
<sup>³ Alpine 3.14 and very briefly 3.15's default python3 [was not able to load](https://github.com/pypa/cibuildwheel/issues/934) musllinux wheels. This has been fixed; please upgrade the python package if using Alpine from before the fix.</sup><br>
|
||
|
||
- Builds manylinux, musllinux, macOS 10.9+, and Windows wheels for CPython and PyPy
|
||
- Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, and GitLab CI
|
||
- Bundles shared library dependencies on Linux and macOS through [auditwheel](https://github.com/pypa/auditwheel) and [delocate](https://github.com/matthew-brett/delocate)
|
||
- Runs your library's tests against the wheel-installed version of your library
|
||
|
||
See the [cibuildwheel 1 documentation](https://cibuildwheel.readthedocs.io/en/1.x/) if you need to build unsupported versions of Python, such as Python 2.
|
||
|
||
Usage
|
||
-----
|
||
|
||
`cibuildwheel` runs inside a CI service. Supported platforms depend on which service you're using:
|
||
|
||
| | Linux | macOS | Windows | Linux ARM |
|
||
|-----------------|-------|-------|---------|--------------|
|
||
| GitHub Actions | ✅ | ✅ | ✅ | ✅¹ |
|
||
| Azure Pipelines | ✅ | ✅ | ✅ | |
|
||
| Travis CI | ✅ | | ✅ | ✅ |
|
||
| AppVeyor | ✅ | ✅ | ✅ | |
|
||
| CircleCI | ✅ | ✅ | | |
|
||
| Gitlab CI | ✅ | | | |
|
||
|
||
<sup>¹ [Requires emulation](https://cibuildwheel.readthedocs.io/en/stable/faq/#emulation), distributed separately. Other services may also support Linux ARM through emulation or third-party build hosts, but these are not tested in our CI.</sup><br>
|
||
|
||
`cibuildwheel` is not intended to run on your development machine. Because it uses system Python from Python.org on macOS and Windows, it will try to install packages globally - not what you expect from a build tool! Instead, isolated CI services like those mentioned above are ideal. For Linux builds, it uses manylinux docker images, so those can be done locally for testing in a pinch.
|
||
|
||
<!--intro-end-->
|
||
|
||
Example setup
|
||
-------------
|
||
|
||
To build manylinux, musllinux, macOS, and Windows wheels on GitHub Actions, you could use this `.github/workflows/wheels.yml`:
|
||
|
||
```yaml
|
||
name: Build
|
||
|
||
on: [push, pull_request]
|
||
|
||
jobs:
|
||
build_wheels:
|
||
name: Build wheels on ${{ matrix.os }}
|
||
runs-on: ${{ matrix.os }}
|
||
strategy:
|
||
matrix:
|
||
os: [ubuntu-20.04, windows-2019, macOS-10.15]
|
||
|
||
steps:
|
||
- uses: actions/checkout@v2
|
||
|
||
# Used to host cibuildwheel
|
||
- uses: actions/setup-python@v2
|
||
|
||
- name: Install cibuildwheel
|
||
run: python -m pip install cibuildwheel==2.3.0
|
||
|
||
- name: Build wheels
|
||
run: python -m cibuildwheel --output-dir wheelhouse
|
||
# to supply options, put them in 'env', like:
|
||
# env:
|
||
# CIBW_SOME_OPTION: value
|
||
|
||
- uses: actions/upload-artifact@v2
|
||
with:
|
||
path: ./wheelhouse/*.whl
|
||
```
|
||
|
||
For more information, including PyPI deployment, and the use of other CI services or the dedicated GitHub Action, check out the [documentation](https://cibuildwheel.readthedocs.org) and the [examples](https://github.com/pypa/cibuildwheel/tree/main/examples).
|
||
|
||
Options
|
||
-------
|
||
|
||
| | Option | Description |
|
||
|---|--------|-------------|
|
||
| **Build selection** | [`CIBW_PLATFORM`](https://cibuildwheel.readthedocs.io/en/stable/options/#platform) | Override the auto-detected target platform |
|
||
| | [`CIBW_BUILD`](https://cibuildwheel.readthedocs.io/en/stable/options/#build-skip) <br> [`CIBW_SKIP`](https://cibuildwheel.readthedocs.io/en/stable/options/#build-skip) | Choose the Python versions to build |
|
||
| | [`CIBW_ARCHS`](https://cibuildwheel.readthedocs.io/en/stable/options/#archs) | Change the architectures built on your machine by default. |
|
||
| | [`CIBW_PROJECT_REQUIRES_PYTHON`](https://cibuildwheel.readthedocs.io/en/stable/options/#requires-python) | Manually set the Python compatibility of your project |
|
||
| | [`CIBW_PRERELEASE_PYTHONS`](https://cibuildwheel.readthedocs.io/en/stable/options/#prerelease-pythons) | Enable building with pre-release versions of Python if available |
|
||
| **Build customization** | [`CIBW_BUILD_FRONTEND`](https://cibuildwheel.readthedocs.io/en/stable/options/#build-frontend) | Set the tool to use to build, either "pip" (default for now) or "build" |
|
||
| | [`CIBW_ENVIRONMENT`](https://cibuildwheel.readthedocs.io/en/stable/options/#environment) | Set environment variables needed during the build |
|
||
| | [`CIBW_ENVIRONMENT_PASS_LINUX`](https://cibuildwheel.readthedocs.io/en/stable/options/#environment-pass) | Set environment variables on the host to pass-through to the container during the build. |
|
||
| | [`CIBW_BEFORE_ALL`](https://cibuildwheel.readthedocs.io/en/stable/options/#before-all) | Execute a shell command on the build system before any wheels are built. |
|
||
| | [`CIBW_BEFORE_BUILD`](https://cibuildwheel.readthedocs.io/en/stable/options/#before-build) | Execute a shell command preparing each wheel's build |
|
||
| | [`CIBW_REPAIR_WHEEL_COMMAND`](https://cibuildwheel.readthedocs.io/en/stable/options/#repair-wheel-command) | Execute a shell command to repair each (non-pure Python) built wheel |
|
||
| | [`CIBW_MANYLINUX_*_IMAGE`<br/>`CIBW_MUSLLINUX_*_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#linux-image) | Specify alternative manylinux / musllinux Docker images |
|
||
| | [`CIBW_DEPENDENCY_VERSIONS`](https://cibuildwheel.readthedocs.io/en/stable/options/#dependency-versions) | Specify how cibuildwheel controls the versions of the tools it uses |
|
||
| **Testing** | [`CIBW_TEST_COMMAND`](https://cibuildwheel.readthedocs.io/en/stable/options/#test-command) | Execute a shell command to test each built wheel |
|
||
| | [`CIBW_BEFORE_TEST`](https://cibuildwheel.readthedocs.io/en/stable/options/#before-test) | Execute a shell command before testing each wheel |
|
||
| | [`CIBW_TEST_REQUIRES`](https://cibuildwheel.readthedocs.io/en/stable/options/#test-requires) | Install Python dependencies before running the tests |
|
||
| | [`CIBW_TEST_EXTRAS`](https://cibuildwheel.readthedocs.io/en/stable/options/#test-extras) | Install your wheel for testing using extras_require |
|
||
| | [`CIBW_TEST_SKIP`](https://cibuildwheel.readthedocs.io/en/stable/options/#test-skip) | Skip running tests on some builds |
|
||
| **Other** | [`CIBW_BUILD_VERBOSITY`](https://cibuildwheel.readthedocs.io/en/stable/options/#build-verbosity) | Increase/decrease the output of pip wheel |
|
||
|
||
These options can be specified in a pyproject.toml file, as well; see [configuration](https://cibuildwheel.readthedocs.io/en/stable/options/#configuration).
|
||
|
||
Working examples
|
||
----------------
|
||
|
||
Here are some repos that use cibuildwheel.
|
||
|
||
<!-- START bin/projects.py -->
|
||
|
||
<!-- this section is generated by bin/projects.py. Don't edit it directly, instead, edit docs/data/projects.yml -->
|
||
|
||
| Name | CI | OS | Notes |
|
||
|-----------------------------------|----|----|:------|
|
||
| [scikit-learn][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | The machine learning library. A complex but clean config using many of cibuildwheel's features to build a large project with Cython and C++ extensions. |
|
||
| [Tornado][] | ![travisci icon][] | ![apple icon][] ![linux icon][] | Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed. |
|
||
| [Matplotlib][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | The venerable Matplotlib, a Python library with C++ portions |
|
||
| [pytorch-fairseq][] | ![github icon][] | ![apple icon][] ![linux icon][] | Facebook AI Research Sequence-to-Sequence Toolkit written in Python. |
|
||
| [MyPy][] | ![github icon][] | ![apple icon][] ![linux icon][] ![windows icon][] | MyPyC, the compiled component of MyPy. |
|
||
| [uvloop][] | ![github icon][] | ![apple icon][] ![linux icon][] | Ultra fast asyncio event loop. |
|
||
| [pydantic][] | ![github icon][] | ![apple icon][] ![linux icon][] ![windows icon][] | Data parsing and validation using Python type hints |
|
||
| [psutil][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Cross-platform lib for process and system monitoring in Python |
|
||
| [vaex][] | ![github icon][] | ![apple icon][] ![linux icon][] ![windows icon][] | Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualize and explore big tabular data at a billion rows per second 🚀 |
|
||
| [Google Benchmark][] | ![github icon][] | ![apple icon][] ![linux icon][] ![windows icon][] | A microbenchmark support library |
|
||
|
||
[scikit-learn]: https://github.com/scikit-learn/scikit-learn
|
||
[Tornado]: https://github.com/tornadoweb/tornado
|
||
[Matplotlib]: https://github.com/matplotlib/matplotlib
|
||
[pytorch-fairseq]: https://github.com/pytorch/fairseq
|
||
[MyPy]: https://github.com/mypyc/mypy_mypyc-wheels
|
||
[uvloop]: https://github.com/MagicStack/uvloop
|
||
[pydantic]: https://github.com/samuelcolvin/pydantic
|
||
[psutil]: https://github.com/giampaolo/psutil
|
||
[vaex]: https://github.com/vaexio/vaex
|
||
[Google Benchmark]: https://github.com/google/benchmark
|
||
|
||
[appveyor icon]: docs/data/readme_icons/appveyor.svg
|
||
[github icon]: docs/data/readme_icons/github.svg
|
||
[azurepipelines icon]: docs/data/readme_icons/azurepipelines.svg
|
||
[circleci icon]: docs/data/readme_icons/circleci.svg
|
||
[gitlab icon]: docs/data/readme_icons/gitlab.svg
|
||
[travisci icon]: docs/data/readme_icons/travisci.svg
|
||
[windows icon]: docs/data/readme_icons/windows.svg
|
||
[apple icon]: docs/data/readme_icons/apple.svg
|
||
[linux icon]: docs/data/readme_icons/linux.svg
|
||
|
||
<!-- END bin/projects.py -->
|
||
|
||
> ℹ️ That's just a handful, there are many more! Check out the [Working Examples](https://cibuildwheel.readthedocs.io/en/stable/working-examples) page in the docs.
|
||
|
||
Legal note
|
||
----------
|
||
|
||
Since `cibuildwheel` repairs the wheel with `delocate` or `auditwheel`, it might automatically bundle dynamically linked libraries from the build machine.
|
||
|
||
It helps ensure that the library can run without any dependencies outside of the pip toolchain.
|
||
|
||
This is similar to static linking, so it might have some licence implications. Check the license for any code you're pulling in to make sure that's allowed.
|
||
|
||
Changelog
|
||
=========
|
||
|
||
<!-- START bin/update_readme_changelog.py -->
|
||
|
||
<!-- this section was generated by bin/update_readme_changelog.py -- do not edit manually -->
|
||
|
||
### v2.3.0
|
||
|
||
_26 November 2021_
|
||
|
||
- 📈 cibuildwheel now defaults to manylinux2014 image for linux builds, rather than manylinux2010. If you want to stick with manylinux2010, it's simple to set this using [the image options](https://cibuildwheel.readthedocs.io/en/stable/options/#linux-image). (#926)
|
||
- ✨ You can now pass environment variables from the host machine into the Docker container during a Linux build. Check out [the docs for `CIBW_ENVIRONMENT_PASS_LINUX `](https://cibuildwheel.readthedocs.io/en/latest/options/#environment-pass) for the details. (#914)
|
||
- ✨ Added support for building PyPy 3.8 wheels. (#881)
|
||
- ✨ Added support for building Windows arm64 CPython wheels on a Windows arm64 runner. We can't test this in CI yet, so for now, this is experimental. (#920)
|
||
- 📚 Improved the deployment documentation (#911)
|
||
- 🛠 Changed the escaping behaviour inside cibuildwheel's option placeholders e.g. `{project}` in `before_build` or `{dest_dir}` in `repair_wheel_command`. This allows bash syntax like `${SOME_VAR}` to passthrough without being interpreted as a placeholder by cibuildwheel. See [this section](https://cibuildwheel.readthedocs.io/en/stable/options/#placeholders) in the docs for more info. (#889)
|
||
- 🛠 Pip updated to 21.3, meaning it now defaults to in-tree builds again. If this causes an issue with your project, setting environment variable `PIP_USE_DEPRECATED=out-of-tree-build` is available as a temporary flag to restore the old behaviour. However, be aware that this flag will probably be removed soon. (#881)
|
||
- 🐛 You can now access the current Python interpreter using `python3` within a build on Windows (#917)
|
||
|
||
### v2.2.2
|
||
|
||
_26 October 2021_
|
||
|
||
- 🐛 Fix bug in the GitHub Action step causing a syntax error (#895)
|
||
|
||
### v2.2.1
|
||
|
||
_26 October 2021_
|
||
|
||
- 🛠 Added a `config-file` option on the GitHub Action to specify something other than pyproject.toml in your GitHub Workflow file. (#883)
|
||
- 🐛 Fix missing resources in sdist and released wheel on PyPI. We've also made some internal changes to our release processes to make them more reliable. (#893, #894)
|
||
|
||
### v2.2.0
|
||
|
||
_22 October 2021_
|
||
|
||
- 🌟 Added support for [musllinux](https://www.python.org/dev/peps/pep-0656/). Support for this new wheel format lets projects build wheels for Linux distributions that use [musl libc](https://musl.libc.org/), notably, [Alpine](https://alpinelinux.org/) Docker containers. (#768)
|
||
|
||
Musllinux builds are enabled by default. If you're not ready to build musllinux, add `*-musllinux_*` to your [`CIBW_SKIP`/`skip`](https://cibuildwheel.readthedocs.io/en/stable/options/#build-skip) option. Or, you might have to make some changes to your options - to simplify that process, you can use...
|
||
|
||
- 🌟 TOML option overrides! This provides much greater flexibility in configuration via pyproject.toml. (#854)
|
||
|
||
You can now set build options for any subset of your builds using a match pattern. So, for example, you can customise CPython 3.8 builds with an override on `cp38-*` or musllinux builds by selecting `*musllinux*`. Check out [the docs](https://cibuildwheel.readthedocs.io/en/latest/options/#overrides) for more info on the specifics.
|
||
|
||
- 🛠 Added support for building PyPy wheels on macOS 11 CI runners. (#875)
|
||
|
||
- 🛠 Setting an empty string for the [`CIBW_*_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) option will now fallthrough to the config file or cibuildwheel's default, rather than causing an error. This makes the option easier to use in CI build matricies. (#829)
|
||
|
||
- 🛠 Support for TOML 1.0 when reading config files, via the `tomli` package. (#876)
|
||
|
||
<sup>Note: This version is not available on PyPI due to some missing resources in the release files. Please use a later version instead.</sup>
|
||
|
||
### v2.1.3
|
||
|
||
_6 October 2021_
|
||
|
||
- 🛠 Updated CPython 3.10 to the 3.10.0 final release
|
||
|
||
<!-- END bin/update_readme_changelog.py -->
|
||
|
||
---
|
||
|
||
That's the last few versions.
|
||
|
||
ℹ️ **Want more changelog? Head over to [the changelog page in the docs](https://cibuildwheel.readthedocs.io/en/stable/changelog/).**
|
||
|
||
---
|
||
|
||
Contributing
|
||
============
|
||
|
||
For more info on how to contribute to cibuildwheel, see the [docs](https://cibuildwheel.readthedocs.io/en/latest/contributing/).
|
||
|
||
Everyone interacting with the cibuildwheel project via codebase, issue tracker, chat rooms, or otherwise is expected to follow the [PSF Code of Conduct](https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md).
|
||
|
||
Maintainers
|
||
-----------
|
||
|
||
- Joe Rickerby [@joerick](https://github.com/joerick)
|
||
- Yannick Jadoul [@YannickJadoul](https://github.com/YannickJadoul)
|
||
- Matthieu Darbois [@mayeut](https://github.com/mayeut)
|
||
- Henry Schreiner [@henryiii](https://github.com/henryiii)
|
||
|
||
Credits
|
||
-------
|
||
|
||
`cibuildwheel` stands on the shoulders of giants.
|
||
|
||
- ⭐️ @matthew-brett for [multibuild](https://github.com/multi-build/multibuild) and [matthew-brett/delocate](http://github.com/matthew-brett/delocate)
|
||
- @PyPA for the manylinux Docker images [pypa/manylinux](https://github.com/pypa/manylinux)
|
||
- @ogrisel for [wheelhouse-uploader](https://github.com/ogrisel/wheelhouse-uploader) and `run_with_env.cmd`
|
||
|
||
Massive props also to-
|
||
|
||
- @zfrenchee for [help debugging many issues](https://github.com/pypa/cibuildwheel/issues/2)
|
||
- @lelit for some great bug reports and [contributions](https://github.com/pypa/cibuildwheel/pull/73)
|
||
- @mayeut for a [phenomenal PR](https://github.com/pypa/cibuildwheel/pull/71) patching Python itself for better compatibility!
|
||
- @czaki for being a super-contributor over many PRs and helping out with countless issues!
|
||
- @mattip for his help with adding PyPy support to cibuildwheel
|
||
|
||
See also
|
||
========
|
||
|
||
If you'd like to keep wheel building separate from the package itself, check out [astrofrog/autowheel](https://github.com/astrofrog/autowheel). It builds packages using cibuildwheel from source distributions on PyPI.
|
||
|
||
Another very similar tool to consider is [matthew-brett/multibuild](http://github.com/matthew-brett/multibuild). `multibuild` is a shell script toolbox for building a wheel on various platforms. It is used as a basis to build some of the big data science tools, like SciPy.
|