Hongru/cibuildwheel
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Migrate more README over to the docs

Browse Source
This commit is contained in:
Joe Rickerby
2019-10-02 09:49:31 +01:00
parent d280f0f16a
commit c006fdab36
9 changed files with 327 additions and 360 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+26 -291
View File
@@ -1,7 +1,7 @@
cibuildwheel
============
[![PyPI](https://img.shields.io/pypi/v/cibuildwheel.svg)](https://pypi.python.org/pypi/cibuildwheel) [![Build Status](https://travis-ci.org/joerick/cibuildwheel.svg?branch=master)](https://travis-ci.org/joerick/cibuildwheel) [![Build status](https://ci.appveyor.com/api/projects/status/wbsgxshp05tt1tif/branch/master?svg=true)](https://ci.appveyor.com/project/joerick/cibuildwheel/branch/master) [![CircleCI](https://circleci.com/gh/joerick/cibuildwheel.svg?style=svg)](https://circleci.com/gh/joerick/cibuildwheel)
[Documentation](cibuildwheel.readthedocs.org) [![PyPI](https://img.shields.io/pypi/v/cibuildwheel.svg)](https://pypi.python.org/pypi/cibuildwheel) [![Build Status](https://travis-ci.org/joerick/cibuildwheel.svg?branch=master)](https://travis-ci.org/joerick/cibuildwheel) [![Build status](https://ci.appveyor.com/api/projects/status/wbsgxshp05tt1tif/branch/master?svg=true)](https://ci.appveyor.com/project/joerick/cibuildwheel/branch/master) [![CircleCI](https://circleci.com/gh/joerick/cibuildwheel.svg?style=svg)](https://circleci.com/gh/joerick/cibuildwheel)
Python wheels are great. Building them across **Mac, Linux, Windows**, on **multiple versions of Python**, is not.
@@ -40,107 +40,39 @@ Usage
`cibuildwheel` is not intended to run on your development machine. It will try to install packages globally; this is no good. Travis CI, CircleCI, and AppVeyor run their builds in isolated environments, so are ideal for this kind of script.
Example setup
-------------
### Linux builds on Docker
To build manylinux and macOS wheels on Travis CI, and upload them to PyPI whenever you tag a version, you could use this `.travis.yml`:
Linux wheels are built in the [`manylinux1` docker images](https://github.com/pypa/manylinux) to provide binary compatible wheels on Linux, according to [PEP 513](https://www.python.org/dev/peps/pep-0513/). Because of this, when building with `cibuildwheel` on Linux, a few things should be taken into account:
- Programs and libraries cannot be installed on the Travis CI Ubuntu host with `apt-get`, but can be installed inside of the Docker image using `yum` or manually. The same goes for environment variables that are potentially needed to customize the wheel building. `cibuildwheel` supports this by providing the `CIBW_ENVIRONMENT` and `CIBW_BEFORE_BUILD` options to setup the build environment inside the running Docker image. See [below](#options) for details on these options.
- The project directory is mounted in the running Docker instance as `/project`, the output directory for the wheels as `/output`. In general, this is handled transparently by `cibuildwheel`. For a more finegrained level of control however, the root of the host file system is mounted as `/host`, allowing for example to access shared files, caches, etc. on the host file system. Note that this is not available on CircleCI due to their Docker policies.
- Alternative dockers images can be specified with the `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE` options to allow for a custom, preconfigured build environment for the Linux builds. See [below](#options) for more details.
```yaml
language: python
matrix:
include:
- sudo: required
services:
- docker
env: PIP=pip
- os: osx
language: generic
env: PIP=pip2
Options
-------
```
usage: cibuildwheel [-h] [--platform {auto,linux,macos,windows}]
[--output-dir OUTPUT_DIR] [--print-build-identifiers]
[project_dir]
Build wheels for all the platforms.
positional arguments:
project_dir Path to the project that you want wheels for.
Default: the current directory.
optional arguments:
-h, --help show this help message and exit
--platform {auto,linux,macos,windows}
Platform to build for. For "linux" you need docker
running, on Mac or Linux. For "macos", you need a Mac
machine, and note that this script is going to
automatically install MacPython on your system, so
don't run on your development machine. For "windows",
you need to run in Windows, and it will build and test
for all versions of Python at C:\PythonXX[-x64].
--output-dir OUTPUT_DIR
Destination folder for the wheels.
--print-build-identifiers
Print the build identifiers matched by the current
invocation and exit.
```
### Example YML syntax
<table>
<tr><td><i>example .travis.yml environment variables</i><pre><code>env:
env:
global:
- CIBW_TEST_REQUIRES=nose
- CIBW_TEST_COMMAND="nosetests {project}/tests"
</code></pre></td>
<td><i>example appveyor.yml environment variables</i><pre><code>environment:
global:
CIBW_TEST_REQUIRES: nose
CIBW_TEST_COMMAND: "nosetests {project}\\tests"
</code></pre></td>
</tr></table>
- TWINE_USERNAME=joerick
# Note: TWINE_PASSWORD is set in Travis settings
Delivering to PyPI
------------------
After you've built your wheels, you'll probably want to deliver them to PyPI.
### Manual method
On your development machine, do the following...
```bash
# Clear out your 'dist' folder.
rm -rf dist
# Make a source distribution
python setup.py sdist
# 🏃🏻
# Go and download your wheel files from wherever you put them. Put
# them all into the 'dist' folder.
# Upload using 'twine' (you may need to 'pip install twine')
twine upload dist/*
script:
- $PIP install cibuildwheel==0.12.0
- cibuildwheel --output-dir wheelhouse
- |
if [[ $TRAVIS_TAG ]]; then
python -m pip install twine
python -m twine upload wheelhouse/*.whl
fi
```
### Semi-automatic method using wheelhouse-uploader
Obviously, manual steps are for chumps, so we can automate this a little by using [wheelhouse-uploader](https://github.com/ogrisel/wheelhouse-uploader).
> Quick note from me - using S3 as a storage didn't work due to a [bug](https://issues.apache.org/jira/browse/LIBCLOUD-792) in libcloud. Feel free to use my fork of that package that fixes the bug `pip install https://github.com/joerick/libcloud/archive/v1.5.0-s3fix.zip`
### Automatic method
If you don't need much control over the release of a package, you can set up cibuildwheel to deliver the wheels straight to PyPI. This doesn't require any cloud storage to work - you just need to bump the version and tag it.
Check out [this example repo](https://github.com/joerick/cibuildwheel-autopypi-example) for instructions on how to set this up.
It didn't work!
---------------
If your wheel didn't compile, check the list below for some debugging tips.
- A mistake in your config. To quickly test your config without doing a git push and waiting for your code to build on CI, you can run the Linux build in a Docker container. On Mac or Linux, with Docker running, try `cibuildwheel --platform linux`. You'll have to bring your config into the current environment first.
- Missing dependency. You might need to install something on the build machine. You can do this in `.travis.yml`, `appveyor.yml`, or `.circleci/config.yml`, with apt-get, brew or whatever Windows uses :P . Given how the Linux build works, we'll probably have to build something into `cibuildwheel`. Let's chat about that over in the issues!
- Windows: missing C feature. The Windows C compiler doesn't support C language features invented after 1990, so you'll have to backport your C code to C90. For me, this mostly involved putting my variable declarations at the top of the function like an animal.
- MacOS: calling cibuildwheel from a python3 script and getting a `ModuleNotFoundError`? Due to a [bug](https://bugs.python.org/issue22490) in CPython, you'll need to [unset the `__PYVENV_LAUNCHER__` variable](https://github.com/joerick/cibuildwheel/issues/133#issuecomment-478288597) before activating a venv.
Working examples
----------------
@@ -168,203 +100,6 @@ It helps ensure that the library can run without any dependencies outside of the
This is similar to static linking, so it might have some licence implications. Check the license for any code you're pulling in to make sure that's allowed.
Changelog
=========
### 0.12.0
_29 September 2019_
- ✨ Add CIBW_TEST_EXTRAS option, to allow testing using extra_require
options. For example, set `CIBW_TEST_EXTRAS=test,qt` to make the wheel
installed with `pip install <wheel_file>[test,qt]`
- 🛠 Update Python from 3.7.2 to 3.7.4 on macOS
- 🛠 Update OpenSSL patch to 1.0.2t on macOS
### 0.11.1
_28 May 2019_
- 🐛 Fix missing file in the release tarball, that was causing problems with
Windows builds (#141)
### 0.11.0
_26 May 2019_
- ✨ Add support for building on Azure pipelines! This lets you build all
Linux, Mac and Windows wheels on one service, so it promises to be the
easiest to set up! Check out the quickstart in the docs, or
[cibuildwheel-azure-example](https://github.com/joerick/cibuildwheel-azure-example)
for an example project. (#126, #132)
- 🛠 Internal change - the end-to-end test projects format was updated, so we
can more precisely assert what should be produced for each one. (#136, #137).
### 0.10.2
_10 March 2019_
- 🛠 Revert temporary fix in macOS, that was working around a bug in pip 19 (#129)
- 🛠 Update Python to 2.7.16 on macOS
- 🛠 Update OpenSSL patch to 1.0.2r on macOS
### 0.10.1
_3 February 2019_
- 🐛 Fix build stalling on macOS (that was introduced in pip 19) (#122)
- 🐛 Fix "AttributeError: 'Popen' object has no attribute 'args'" on Python 2.7 for Linux builds (#108)
- 🛠 Update Python from 3.6.7, 3.7.1 to 3.6.8, 3.7.2 on macOS
- 🛠 Update openssl patch from 1.0.2p to 1.0.2q on macOS
- 🛠 Sorting build options dict items when printing preamble (#114)
### 0.10.0
_23 September 2018_
- ✨ Add `CIBW_BUILD` option, for specifying which specific builds to perform (#101)
- ✨ Add support for building Mac and Linux on CircleCI (#91, #97)
- 🛠 Improved support for building universal wheels (#95)
- 🛠 Ensure log output is unbuffered and therefore in the correct order (#92)
- 🛠 Improved error reporting for errors that occur inside a package's setup.py (#88)
- ⚠️ Removed support for Python 3.3 on Windows.
### 0.9.4
_29 July 2018_
- 🛠 CIBW_TEST_COMMAND now runs in a shell on Mac (as well as Linux) (#81)
### 0.9.3
_10 July 2018_
- 🛠 Update to Python 3.6.6 on macOS (#82)
- ✨ Add support for building Python 3.7 wheels on Windows (#76)
- ⚠️ Deprecated support for Python 3.3 on Windows.
### 0.9.2
_1 July 2018_
- 🛠 Update Python 3.7.0rc1 to 3.7.0 on macOS (#79)
### 0.9.1
_18 June 2018_
- 🛠 Removed the need to use `{python}` and `{pip}` in `CIBW_BEFORE_BUILD` statements, by ensuring the correct version is always on the path at `python` and `pip` instead. (#60)
- 🛠 We now patch the _ssl module on Python 3.4 and 3.5 so these versions can still make SSL web requests using TLS 1.2 while building. (#71)
### 0.9.0
_18 June 2018_
- ✨ Add support for Python 3.7 (#73)
### 0.8.0
_4 May 2018_
- ⚠️ Drop support for Python 3.3 on Linux (#67)
- 🐛 Fix TLS by updating setuptools (#69)
### 0.7.1
_2 April 2017_
- 🐛 macOS: Fix Pip bugs resulting from PyPI TLS 1.2 enforcement
- 🐛 macOS: Fix brew Python3 version problems in the CI
### 0.7.0
_7 January 2018_
- ✨ You can now specify a custom docker image using the `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE` options. (#46)
- 🐛 Fixed a bug where cibuildwheel would download and build a package from PyPI(!) instead of building the package on the local machine. (#51)
### 0.6.0
_9 October 2017_
- ✨ On the Linux build, the host filesystem is now accessible via `/host` (#36)
- 🐛 Fixed a bug where setup.py scripts would run the wrong version of Python when running subprocesses on Linux (#35)
### 0.5.1
_10 September 2017_
- 🐛 Fixed a couple of bugs on Python 3.
- ✨ Added experimental support for Mac builds on [Bitrise.io](https://www.bitrise.io)
### 0.5.0
_7 September 2017_
- ✨ `CIBW_ENVIRONMENT` added. You can now set environment variables for each build, even within the Docker container on Linux. This is a big one! (#21)
- ✨ `CIBW_BEFORE_BUILD` now runs in a system shell on all platforms. You can now do things like `CIBW_BEFORE_BUILD="cmd1 && cmd2"`. (#32)
### 0.4.1
_14 August 2017_
- 🐛 Fixed a bug on Windows where subprocess' output was hidden (#23)
- 🐛 Fixed a bug on AppVeyor where logs would appear in the wrong order due to output buffering (#24, thanks @YannickJadoul!)
### 0.4.0
_23 July 2017_
- 🐛 Fixed a bug that was increasing the build time by building the wheel twice. This was a problem for large projects that have a long build time. If you're upgrading and you need the old behaviour, use `CIBW_BEFORE_BUILD={pip} install .`, or install exactly the dependencies you need in `CIBW_BEFORE_BUILD`. See #18.
### 0.3.0
_27 June 2017_
- ⚠️ Removed Python 2.6 support on Linux (#12)
### 0.2.1
_11 June 2017_
- 🛠 Changed the build process to install the package before building the wheel - this allows direct dependencies to be installed first (#9, thanks @tgarc!)
- ✨ Added Python 3 support for the main process, for systems where Python 3 is the default (#8, thanks @tgarc).
### 0.2.0
_13 April 2017_
- ✨ Added `CIBW_SKIP` option, letting users explicitly skip a build
- ✨ Added `CIBW_BEFORE_BUILD` option, letting users run a shell command before the build starts
### 0.1.3
_31 March 2017_
- 🌟 First public release!
Contributing
============
Wheel-building is pretty complex. I expect users to find many edge-cases - please help the rest of the community out by documenting these, adding features to support them, and reporting bugs.
I plan to be pretty liberal in accepting pull requests, as long as they align with the design goals below.
`cibuildwheel` is indie open source. I'm not paid to work on this.
Design Goals
------------
- `cibuildwheel` should wrap the complexity of wheel building.
- The user interface to `cibuildwheel` is the build script (e.g. `.travis.yml`). Feature additions should not increase the complexity of this script.
- Options should be environment variables (these lend themselves better to YML config files). They should be prefixed with `CIBW_`.
- Options should be generalise to all platforms. If platform-specific options are required, they should be namespaced e.g. `CIBW_TEST_COMMAND_MACOS`
Other notes:
- The platforms are very similar, until they're not. I'd rather have straight-forward code than totally DRY code, so let's keep airy platfrom abstractions to a minimum.
- I might want to break the options into a shared config file one day, so that config is more easily shared. That has motivated some of the design decisions.
Maintainers
-----------
docs/changelog.md
+174
View File
@@ -0,0 +1,174 @@
Changelog
=========
### 0.12.0
_29 September 2019_
- ✨ Add CIBW_TEST_EXTRAS option, to allow testing using extra_require
options. For example, set `CIBW_TEST_EXTRAS=test,qt` to make the wheel
installed with `pip install <wheel_file>[test,qt]`
- 🛠 Update Python from 3.7.2 to 3.7.4 on macOS
- 🛠 Update OpenSSL patch to 1.0.2t on macOS
### 0.11.1
_28 May 2019_
- 🐛 Fix missing file in the release tarball, that was causing problems with
Windows builds (#141)
### 0.11.0
_26 May 2019_
- ✨ Add support for building on Azure pipelines! This lets you build all
Linux, Mac and Windows wheels on one service, so it promises to be the
easiest to set up! Check out the quickstart in the docs, or
[cibuildwheel-azure-example](https://github.com/joerick/cibuildwheel-azure-example)
for an example project. (#126, #132)
- 🛠 Internal change - the end-to-end test projects format was updated, so we
can more precisely assert what should be produced for each one. (#136, #137).
### 0.10.2
_10 March 2019_
- 🛠 Revert temporary fix in macOS, that was working around a bug in pip 19 (#129)
- 🛠 Update Python to 2.7.16 on macOS
- 🛠 Update OpenSSL patch to 1.0.2r on macOS
### 0.10.1
_3 February 2019_
- 🐛 Fix build stalling on macOS (that was introduced in pip 19) (#122)
- 🐛 Fix "AttributeError: 'Popen' object has no attribute 'args'" on Python 2.7 for Linux builds (#108)
- 🛠 Update Python from 3.6.7, 3.7.1 to 3.6.8, 3.7.2 on macOS
- 🛠 Update openssl patch from 1.0.2p to 1.0.2q on macOS
- 🛠 Sorting build options dict items when printing preamble (#114)
### 0.10.0
_23 September 2018_
- ✨ Add `CIBW_BUILD` option, for specifying which specific builds to perform (#101)
- ✨ Add support for building Mac and Linux on CircleCI (#91, #97)
- 🛠 Improved support for building universal wheels (#95)
- 🛠 Ensure log output is unbuffered and therefore in the correct order (#92)
- 🛠 Improved error reporting for errors that occur inside a package's setup.py (#88)
- ⚠️ Removed support for Python 3.3 on Windows.
### 0.9.4
_29 July 2018_
- 🛠 CIBW_TEST_COMMAND now runs in a shell on Mac (as well as Linux) (#81)
### 0.9.3
_10 July 2018_
- 🛠 Update to Python 3.6.6 on macOS (#82)
- ✨ Add support for building Python 3.7 wheels on Windows (#76)
- ⚠️ Deprecated support for Python 3.3 on Windows.
### 0.9.2
_1 July 2018_
- 🛠 Update Python 3.7.0rc1 to 3.7.0 on macOS (#79)
### 0.9.1
_18 June 2018_
- 🛠 Removed the need to use `{python}` and `{pip}` in `CIBW_BEFORE_BUILD` statements, by ensuring the correct version is always on the path at `python` and `pip` instead. (#60)
- 🛠 We now patch the _ssl module on Python 3.4 and 3.5 so these versions can still make SSL web requests using TLS 1.2 while building. (#71)
### 0.9.0
_18 June 2018_
- ✨ Add support for Python 3.7 (#73)
### 0.8.0
_4 May 2018_
- ⚠️ Drop support for Python 3.3 on Linux (#67)
- 🐛 Fix TLS by updating setuptools (#69)
### 0.7.1
_2 April 2017_
- 🐛 macOS: Fix Pip bugs resulting from PyPI TLS 1.2 enforcement
- 🐛 macOS: Fix brew Python3 version problems in the CI
### 0.7.0
_7 January 2018_
- ✨ You can now specify a custom docker image using the `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE` options. (#46)
- 🐛 Fixed a bug where cibuildwheel would download and build a package from PyPI(!) instead of building the package on the local machine. (#51)
### 0.6.0
_9 October 2017_
- ✨ On the Linux build, the host filesystem is now accessible via `/host` (#36)
- 🐛 Fixed a bug where setup.py scripts would run the wrong version of Python when running subprocesses on Linux (#35)
### 0.5.1
_10 September 2017_
- 🐛 Fixed a couple of bugs on Python 3.
- ✨ Added experimental support for Mac builds on [Bitrise.io](https://www.bitrise.io)
### 0.5.0
_7 September 2017_
- ✨ `CIBW_ENVIRONMENT` added. You can now set environment variables for each build, even within the Docker container on Linux. This is a big one! (#21)
- ✨ `CIBW_BEFORE_BUILD` now runs in a system shell on all platforms. You can now do things like `CIBW_BEFORE_BUILD="cmd1 && cmd2"`. (#32)
### 0.4.1
_14 August 2017_
- 🐛 Fixed a bug on Windows where subprocess' output was hidden (#23)
- 🐛 Fixed a bug on AppVeyor where logs would appear in the wrong order due to output buffering (#24, thanks @YannickJadoul!)
### 0.4.0
_23 July 2017_
- 🐛 Fixed a bug that was increasing the build time by building the wheel twice. This was a problem for large projects that have a long build time. If you're upgrading and you need the old behaviour, use `CIBW_BEFORE_BUILD={pip} install .`, or install exactly the dependencies you need in `CIBW_BEFORE_BUILD`. See #18.
### 0.3.0
_27 June 2017_
- ⚠️ Removed Python 2.6 support on Linux (#12)
### 0.2.1
_11 June 2017_
- 🛠 Changed the build process to install the package before building the wheel - this allows direct dependencies to be installed first (#9, thanks @tgarc!)
- ✨ Added Python 3 support for the main process, for systems where Python 3 is the default (#8, thanks @tgarc).
### 0.2.0
_13 April 2017_
- ✨ Added `CIBW_SKIP` option, letting users explicitly skip a build
- ✨ Added `CIBW_BEFORE_BUILD` option, letting users run a shell command before the build starts
### 0.1.3
_31 March 2017_
- 🌟 First public release!
docs/contributing.md
+21
View File
@@ -0,0 +1,21 @@
Contributing
============
Wheel-building is pretty complex. I expect users to find many edge-cases - please help the rest of the community out by documenting these, adding features to support them, and reporting bugs.
I plan to be pretty liberal in accepting pull requests, as long as they align with the design goals below.
`cibuildwheel` is indie open source. I'm not paid to work on this.
Design Goals
------------
- `cibuildwheel` should wrap the complexity of wheel building.
- The user interface to `cibuildwheel` is the build script (e.g. `.travis.yml`). Feature additions should not increase the complexity of this script.
- Options should be environment variables (these lend themselves better to YML config files). They should be prefixed with `CIBW_`.
- Options should be generalise to all platforms. If platform-specific options are required, they should be namespaced e.g. `CIBW_TEST_COMMAND_MACOS`
Other notes:
- The platforms are very similar, until they're not. I'd rather have straight-forward code than totally DRY code, so let's keep airy platfrom abstractions to a minimum.
- I might want to break the options into a shared config file one day, so that config is more easily shared. That has motivated some of the design decisions.
docs/deliver-to-pypi.md
+35
View File
@@ -0,0 +1,35 @@
---
title: Deliver to PyPI
---
After you've built your wheels, you'll probably want to deliver them to PyPI.
### Manual method
On your development machine, do the following...
```bash
# Clear out your 'dist' folder.
rm -rf dist
# Make a source distribution
python setup.py sdist
# 🏃🏻
# Go and download your wheel files from wherever you put them. Put
# them all into the 'dist' folder.
# Upload using 'twine' (you may need to 'pip install twine')
twine upload dist/*
```
### Semi-automatic method using wheelhouse-uploader
Obviously, manual steps are for chumps, so we can automate this a little by using [wheelhouse-uploader](https://github.com/ogrisel/wheelhouse-uploader).
> Quick note from me - using S3 as a storage didn't work due to a [bug](https://issues.apache.org/jira/browse/LIBCLOUD-792) in libcloud. Feel free to use my fork of that package that fixes the bug `pip install https://github.com/joerick/libcloud/archive/v1.5.0-s3fix.zip`
### Automatic method
If you don't need much control over the release of a package, you can set up cibuildwheel to deliver the wheels straight to PyPI. This doesn't require any cloud storage to work - you just need to bump the version and tag it.
Check out [this example repo](https://github.com/joerick/cibuildwheel-autopypi-example) for instructions on how to set this up.
docs/faq.md
+23
View File
@@ -0,0 +1,23 @@
---
title: It didn't work!
---
If your wheel didn't compile, check the list below for some debugging tips.
- A mistake in your config. To quickly test your config without doing a git push and waiting for your code to build on CI, you can test the Linux build in a Docker container. On Mac or Linux, with Docker running, try `cibuildwheel --platform linux`. You'll have to bring your config into the current environment first.
- Missing dependency. You might need to install something on the build machine. You can do this in `.travis.yml`, `appveyor.yml`, or `.circleci/config.yml`, with apt-get, brew or whatever Windows uses :P . Given how the Linux build works, we'll probably have to build something into `cibuildwheel`. Let's chat about that over in the issues!
- Windows: missing C feature. The Windows C compiler doesn't support C language features invented after 1990, so you'll have to backport your C code to C90. For me, this mostly involved putting my variable declarations at the top of the function like an animal.
- MacOS: calling cibuildwheel from a python3 script and getting a `ModuleNotFoundError`? Due to a [bug](https://bugs.python.org/issue22490) in CPython, you'll need to [unset the `__PYVENV_LAUNCHER__` variable](https://github.com/joerick/cibuildwheel/issues/133#issuecomment-478288597) before activating a venv.
### Linux builds on Docker
Linux wheels are built in the [`manylinux1` docker images](https://github.com/pypa/manylinux) to provide binary compatible wheels on Linux, according to [PEP 513](https://www.python.org/dev/peps/pep-0513/). Because of this, when building with `cibuildwheel` on Linux, a few things should be taken into account:
- Programs and libraries cannot be installed on the Travis CI Ubuntu host with `apt-get`, but can be installed inside of the Docker image using `yum` or manually. The same goes for environment variables that are potentially needed to customize the wheel building. `cibuildwheel` supports this by providing the `CIBW_ENVIRONMENT` and `CIBW_BEFORE_BUILD` options to setup the build environment inside the running Docker image. See [below](options.md#build-environment) for details on these options.
- The project directory is mounted in the running Docker instance as `/project`, the output directory for the wheels as `/output`. In general, this is handled transparently by `cibuildwheel`. For a more finegrained level of control however, the root of the host file system is mounted as `/host`, allowing for example to access shared files, caches, etc. on the host file system. Note that this is not available on CircleCI due to their Docker policies.
- Alternative dockers images can be specified with the `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE` options to allow for a custom, preconfigured build environment for the Linux builds. See [options](options.md#manylinux-image) for more details.
docs/index.md
+2 -35
View File
@@ -1,39 +1,6 @@
Home
====
Python wheels are great. Building them across **Mac, Linux, Windows**, on **multiple versions of Python**, is not.
Welcome to the cibuildwheel docs.
`cibuildwheel` is here to help. `cibuildwheel` runs on your CI server - currently it supports Azure Pipelines, Travis CI, AppVeyor, and CircleCI - and it builds and tests your wheels across all of your platforms.
**`cibuildwheel` is in beta**. It's brand new - I'd love for you to try it and help make it better!
What does it do?
----------------
| | macOS 10.6+ | manylinux i686 | manylinux x86_64 | Windows 32bit | Windows 64bit |
|---|---|---|---|---|---|
| Python 2.7 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.4 | ✅ | ✅ | ✅ | ✅* | ✅* |
| Python 3.5 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.6 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.7 | ✅ | ✅ | ✅ | ✅ | ✅ |
> \* Not supported on Azure Pipelines
- Builds manylinux, macOS and Windows (32 and 64bit) wheels using Azure Pipelines, Travis CI, AppVeyor, and CircleCI
- Bundles shared library dependencies on Linux and macOS through [auditwheel](https://github.com/pypa/auditwheel) and [delocate](https://github.com/matthew-brett/delocate)
- Runs the library test suite against the wheel-installed version of your library
Usage
-----
`cibuildwheel` currently works **Travis CI** and **CircleCI** to build Linux and Mac wheels, and **AppVeyor** to build Windows wheels. **Azure Pipelines** supports all three.
| | Linux | macOS | Windows |
|-----------------|-------|-------|---------|
| Azure Pipelines | ✅ | ✅ | ✅ |
| Travis CI | ✅ | ✅ | |
| AppVeyor | | | ✅ |
| CircleCI | ✅ | ✅ | |
`cibuildwheel` is not intended to run on your development machine. It will try to install packages globally; this is no good. Travis CI, CircleCI, and AppVeyor run their builds in isolated environments, so are ideal for this kind of script.
To get started, head over to the [setup guide](setup.md).
docs/options.md
+33 -31
View File
@@ -1,26 +1,28 @@
## Options summary
cibuildwheel is configured using environment variables, that can be set using
your CI config.
`cibuildwheel` allows for easy customization of the various phases of the build process demonstrated above:
<table>
<tr><td><i>example .travis.yml environment variables</i><pre><code>env:
global:
- CIBW_TEST_REQUIRES=nose
- CIBW_TEST_COMMAND="nosetests {project}/tests"
</code></pre></td>
<td><i>example appveyor.yml environment variables</i><pre><code>environment:
global:
CIBW_TEST_REQUIRES: nose
CIBW_TEST_COMMAND: "nosetests {project}\\tests"
</code></pre></td>
<td><i>example .circleci/config.yml environment variables</i><pre><code>jobs:
job_name:
environment:
CIBW_TEST_REQUIRES: nose
CIBW_TEST_COMMAND: "nosetests {project}\\tests"
</code></pre></td>
</tr></table>
| | Option | |
|---|---|---|
| **Target wheels** | `CIBW_PLATFORM` | Override the auto-detected target platform |
| | `CIBW_BUILD` | Build only certain Python versions |
| | `CIBW_SKIP` | Skip certain Python versions |
| **Build parameters** | `CIBW_BUILD_VERBOSITY` | Increase or decrease the output of `pip wheel` |
| **Build environment** | `CIBW_ENVIRONMENT` | Set environment variables needed during the build |
| | `CIBW_BEFORE_BUILD` | Execute a shell command preparing each wheel's build |
| | `CIBW_MANYLINUX1_X86_64_IMAGE` | Specify an alternative manylinx1 x86_64 docker image |
| | `CIBW_MANYLINUX1_I686_IMAGE` | Specify an alternative manylinux1 i686 docker image |
| **Tests** | `CIBW_TEST_COMMAND` | Execute a shell command to test all built wheels |
| | `CIBW_TEST_REQUIRES` | Install Python dependencies before running the tests |
| | `CIBW_TEST_EXTRAS` | Install Python dependencies before running the tests using ``extras_require``|
## 🚩Build selection
Most of the config is via environment variables. These go into `.travis.yml`, `appveyor.yml`, and `.circleci/config.yml` nicely.
## Build selection
### CIBW_PLATFORM
### CIBW_PLATFORM - Override the auto-detected target platform {: #platform}
Options: `auto` `linux` `macos` `windows`
@@ -30,7 +32,7 @@ Default: `auto`
For `linux` you need Docker running, on Mac or Linux. For `macos`, you need a Mac machine, and note that this script is going to automatically install MacPython on your system, so don't run on your development machine. For `windows`, you need to run in Windows, and it will build and test for all versions of Python at `C:\PythonXX[-x64]`.
### CIBW_BUILD and CIBW_SKIP
### CIBW_BUILD, CIBW_SKIP - Choose the Python versions to build {: #build-skip}
Space-separated list of builds to build and skip. Each build has an identifier like `cp27-manylinux1_x86_64` or `cp34-macosx_10_6_intel` - you can list specific ones to build and `cibuildwheel` will only build those, and/or list ones to skip and `cibuildwheel` won't try to build them.
@@ -55,9 +57,9 @@ Examples:
- Skip Python 3.6 on Linux: `CIBW_SKIP=cp36-manylinux*`
- Only build on Python 3 and skip 32-bit builds: `CIBW_BUILD=cp3?-*` and `CIBW_SKIP=*-win32 *-manylinux1_i686`
## Build environment
## 🌎 Build environment
### CIBW_ENVIRONMENT
### CIBW_ENVIRONMENT - Set environment variables needed during the build {: #environment}
A space-separated list of environment variables to set during the build. Bash syntax should be used (even on Windows!).
@@ -75,7 +77,7 @@ Platform-specific variants also available:
In addition to the above, `cibuildwheel` always defines the environment variable `CIBUILDWHEEL=1`. This can be useful for [building wheels with optional extensions](https://github.com/joerick/cibuildwheel/wiki/Building-packages-with-optional-C-extensions).
### CIBW_BEFORE_BUILD
### CIBW_BEFORE_BUILD - Execute a shell command preparing each wheel's build {: #before-build}
A shell command to run before building the wheel. This option allows you to run a command in **each** Python environment before the `pip wheel` command. This is useful if you need to set up some dependency so it's available during the build.
@@ -90,7 +92,7 @@ Example: `yum install -y libffi-dev && pip install .`
Platform-specific variants also available:
`CIBW_BEFORE_BUILD_MACOS` | `CIBW_BEFORE_BUILD_WINDOWS` | `CIBW_BEFORE_BUILD_LINUX`
### CIBW_MANYLINUX1_X86_64_IMAGE and CIBW_MANYLINUX1_I686_IMAGE
### CIBW_MANYLINUX1_X86_64_IMAGE, CIBW_MANYLINUX1_I686_IMAGE - Specify alternative manylinux1 x86_64 docker images {: #manylinux-image}
An alternative docker image to be used for building [`manylinux1`](https://github.com/pypa/manylinux) wheels. `cibuildwheel` will then pull these instead of the official images, [`quay.io/pypa/manylinux1_x86_64`](https://quay.io/pypa/manylinux1_i686) and [`quay.io/pypa/manylinux1_i686`](https://quay.io/pypa/manylinux1_i686).
@@ -99,9 +101,9 @@ Beware to specify a valid docker image that can be used the same as the official
Example: `dockcross/manylinux-x64`
Example: `dockcross/manylinux-x86`
## Testing
## 🔬 Testing
### CIBW_TEST_COMMAND
### CIBW_TEST_COMMAND - Execute a shell command to test all built wheels {: #test-command}
Shell command to run tests after the build. The wheel will be installed automatically and available for import from the tests. `{project}` can be used as a placeholder for the absolute path to the project's root and will be replaced by `cibuildwheel`.
@@ -112,7 +114,7 @@ Example: `nosetests {project}/tests`
Platform-specific variants also available:
`CIBW_TEST_COMMAND_MACOS` | `CIBW_TEST_COMMAND_WINDOWS` | `CIBW_TEST_COMMAND_LINUX`
### CIBW_TEST_REQUIRES
### CIBW_TEST_REQUIRES - Install Python dependencies before running the tests {: #test-requires}
Space-separated list of dependencies required for running the tests.
@@ -122,7 +124,7 @@ Example: `nose==1.3.7 moto==0.4.31`
Platform-specific variants also available:
`CIBW_TEST_REQUIRES_MACOS` | `CIBW_TEST_REQUIRES_WINDOWS` | `CIBW_TEST_REQUIRES_LINUX`
### CIBW_TEST_EXTRAS
### CIBW_TEST_EXTRAS - Install your wheel for testing using `extras_require` {: #test-extras}
Comma-separated list of
[extras_require](https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies)
@@ -137,9 +139,9 @@ Example: `test,qt` (will cause the wheel to be installed with `pip install <whee
Platform-specific variants also available:
`CIBW_TEST_EXTRAS_MACOS` | `CIBW_TEST_EXTRAS_WINDOWS` | `CIBW_TEST_EXTRAS_LINUX`
## Other
## 💭 Other
### CIBW_BUILD_VERBOSITY
### CIBW_BUILD_VERBOSITY - Increase/decrease the output of pip wheel
An number from 1 to 3 to increase the level of verbosity (corresponding to invoking pip with `-v`, `-vv`, and `-vvv`), between -1 and -3 (`-q`, `-qq`, and `-qqq`), or just 0 (default verbosity). These flags are useful while debugging a build when the output of the actual build invoked by `pip wheel` is required.
docs/setup.md
+2 -1
View File
@@ -115,7 +115,8 @@ workflows:
- osx-wheels
```
Note: CircleCI doesn't enable free macOS containers for open source by default, but you can ask for access. See [here](https://circleci.com/docs/2.0/oss/#overview) for more information.
!!! note
CircleCI doesn't enable free macOS containers for open source by default, but you can ask for access. See [here](https://circleci.com/docs/2.0/oss/#overview) for more information.
CircleCI will store the built wheels for you - you can access them from the project console.
mkdocs.yml
+11 -2
View File
@@ -1,10 +1,19 @@
site_name: cibuildwheel
docs_dir: docs
theme: readthedocs
repo_url: https://github.com/joerick/cibuildwheel
nav:
- Home: index.md
- "Setup guide": setup.md
- "Options": options.md
- deliver-to-pypi.md
- faq.md
- contributing.md
- changelog.md
markdown_extensions:
- codehilite
- fenced_code
- fenced_code
- toc:
permalink: True
- attr_list
- admonition