Compare commits

..
113 Commits
Author SHA1 Message Date
Joe Rickerby a1fc0cb117 Bump version 2019-05-28 19:17:28 +01:00
Joe Rickerby 430b71c767 Ensure resources are packaged in release tarballs and wheels
Fix #141
2019-05-28 19:17:05 +01:00
Joe Rickerby 14f14a24d3 Add config for universal wheel support so we can post wheels to PyPI too 2019-05-28 10:34:31 +01:00
Joe Rickerby b732026734 Update changelog 2019-05-26 12:53:51 +01:00
Joe Rickerby 0d85db7111 Bump version 2019-05-26 12:48:43 +01:00
Joe Rickerby 477fc7e060 Merge pull request #132 from joerick/azure-windows
Support Windows on Azure Pipelines [continued...]
2019-05-26 11:38:36 +01:00
Joe Rickerby 17566cdee7 Don't expect Python 3.4 wheels on Azure 2019-04-27 12:12:02 +01:00
Joe Rickerby 49bc742add Remove unused imports 2019-04-27 10:23:07 +01:00
Joe Rickerby 4cbb67fac1 Merge remote-tracking branch 'origin/master' into azure-windows 2019-04-27 10:22:00 +01:00
Joe Rickerby 2e3155decf Merge branch 'master' into azure-windows 2019-04-27 10:11:30 +01:00
Joe Rickerby 719352b1d2 Merge pull request #137 from joerick/custom-test-checks
Custom test checks let individual tests assert exactly which wheels they expect to produce
2019-04-25 14:56:28 +01:00
Joe Rickerby f7d93cc9e4 Fix function name 2019-04-23 22:23:54 +01:00
Joe Rickerby 3f5572e956 Make included scripts virtualenv-friendly 2019-04-23 22:17:59 +01:00
Joe Rickerby bab8b6668e Full error output during pytest fails 2019-04-23 22:17:25 +01:00
Joe Rickerby f29697d623 Retain PYTHONPATH if already in env 2019-04-23 22:17:03 +01:00
Joe Rickerby e87032c830 Rework tests check all wheels produced against a reference list
Also, wrap the cibuildwheel call to make building env easier
2019-04-23 22:15:02 +01:00
Joe Rickerby e360a04dc1 Use utils.platform for platform checks 2019-04-22 23:25:14 +01:00
Joe Rickerby 9050137d2b Neaten the environment modification using triple-quotes and dict.update 2019-04-22 23:23:49 +01:00
Joe Rickerby 857aafbd3f Move the pythonpath fiddling to the test runner
instead of including it in every test
2019-04-22 23:13:34 +01:00
Joe Rickerby 0cd3e426bc Minor formatting fixes 2019-04-22 19:05:34 +01:00
Joe Rickerby 8b632bcb6d Skip cleanup if its not needed (when a test is skipped) 2019-04-22 18:55:55 +01:00
Joe Rickerby 364fc5220e another trailing comma 2019-04-22 18:44:09 +01:00
Joe Rickerby 0ed37f268a In general, prefer single quotes 2019-04-22 18:35:57 +01:00
Joe Rickerby f410fbdd03 Remove trailing commas from environment declarations 2019-04-22 18:33:03 +01:00
Joe Rickerby 21fb27faf7 Rework custom test machinery to give the individual tests more control
on how they're run, and combine environment.json and check.py
2019-04-22 18:22:43 +01:00
Yannick Jadoul ac4b62ca07 Update README.md with --print-build-identifiers switch 2019-04-22 16:15:47 +02:00
Yannick Jadoul 3f7a4d753f Skip non-linux builds for 06_docker_images test, now that we have custom checks 2019-04-20 22:12:45 +02:00
Yannick Jadoul 95d2086ced Adding check.py to tests, rather than a simple generic assert on number of wheels 2019-04-20 20:49:56 +02:00
Yannick Jadoul 52e8f41b5c Added --print-build-identifiers command line option 2019-04-20 19:02:35 +02:00
Joe Rickerby 1e2a2e7d4d Revert "Add azure-template.yml for more compact config"
This reverts commit 6377854545.
2019-04-06 18:03:05 +01:00
Joe Rickerby 6377854545 Add azure-template.yml for more compact config 2019-04-06 17:34:36 +01:00
Joe Rickerby be57269d5d Remove 3.4 from the pipelines config because 3.4 isn't supported 2019-04-06 16:23:14 +01:00
Joe Rickerby d5856884a3 README updates 2019-04-06 16:18:26 +01:00
Joe Rickerby c2d3dc3b0f Update README.md
Fix #133. Thanks @zackw for the report!
2019-04-05 21:31:50 +01:00
Joe Rickerby 9116217b60 Update README.md
Readme improvements, adding Azure config and putting services behind disclosure triangles
2019-03-11 22:37:59 +00:00
Joe Rickerby a8f6684bfd Skip Python 3.4 builds on Azure 2019-03-11 19:59:31 +00:00
Joe Rickerby 6852d87d2e Update changelog 2019-03-10 23:37:36 +00:00
Joe Rickerby 054c273997 Bump version 2019-03-10 23:32:06 +00:00
Joe Rickerby 652d094e46 Fix typo 2019-03-10 23:19:22 +00:00
Joe Rickerby 1bfe6d45bd Add auto-detect platform for Azure 2019-03-10 23:07:14 +00:00
Joe Rickerby aefde9535b Add azure-pipelines test run config 2019-03-10 22:50:07 +00:00
Joe Rickerby d46fb62e95 Refactor for code style 2019-03-10 22:41:55 +00:00
Joe Rickerby 87a9d170b0 Merge pull request #131 from mayeut/openssl-1.0.2r
Update openssl patch from 1.0.2q to 1.0.2r on macOS
2019-03-10 22:38:47 +00:00
Joe Rickerby dcce949d3c Vendor appveyor_run_with_cmd, but don't use it on azure pipelines 2019-03-10 22:03:31 +00:00
mayeut 1d3c59d804 Update openssl patch from 1.0.2q to 1.0.2r on macOS 2019-03-10 22:28:43 +01:00
Joe Rickerby 0b791dbea5 Merge pull request #130 from YannickJadoul/macos-python-2.7.16
Updating version of Python 2.7 to 2.7.16 on macOS
2019-03-10 21:26:04 +00:00
Yannick Jadoul 44256e4aa5 Revert "Running 'pip install --upgrade setuptools' with 'sudo' on macOS, fixing #122" (#129)
This reverts commit d79ca07bbb.
2019-03-10 21:25:28 +00:00
Joe Rickerby 243103762b Merge branch 'master' into azure-windows 2019-03-10 20:29:05 +00:00
Yannick Jadoul b579cd5f2a Updating version of Python 2.7 to 2.7.16 on macOS 2019-03-10 21:22:41 +01:00
Joe Rickerby 8523b25a18 Merge pull request #128 from YannickJadoul/fix-bashlex-install-permission
Fixing issue #127 by avoiding bashlex 0.13 in 'install_requires'
2019-03-10 19:52:32 +00:00
Yannick Jadoul 637a251b79 Fixing issue #127 by avoiding bashlex 0.13 in 'install_requires' 2019-03-10 14:57:43 +01:00
Thomas Robitaille ac76af4f75 Support Python location on Windows in Azure Pipelines 2019-02-22 22:19:41 +00:00
Joe Rickerby a6ff864010 Merge pull request #125 from ushuz/patch-1
Add missing date to changelog
2019-02-15 22:28:18 +00:00
John Hu 79f044314b Add missing date to changelog 2019-02-14 11:58:56 +08:00
Joe Rickerby 0c0e21aa83 Update changelog 2019-02-03 21:58:53 +00:00
Joe Rickerby f1bba58f86 Bump version 2019-02-03 21:54:00 +00:00
Joe Rickerby 56218d896c Merge pull request #123 from YannickJadoul/fix-issue-122
Running 'pip install --upgrade setuptools' with 'sudo' on macOS, fixing #122
2019-02-03 21:37:57 +00:00
Yannick Jadoul d79ca07bbb Running 'pip install --upgrade setuptools' with 'sudo' on macOS, fixing #122 2019-01-30 17:36:17 +01:00
Joe Rickerby c6f4f01732 Update README.md
ref #93
2019-01-07 22:17:19 +00:00
mayeut 586b3c7197 Update openssl patch from 1.0.2p to 1.0.2q on macOS 2019-01-07 18:34:46 +01:00
mayeut 0b4475ed07 Update Python from 3.6.7, 3.7.1 to 3.6.8, 3.7.2 on macOS 2019-01-06 23:52:57 +01:00
Yannick Jadoul 38b3b0a3da Merge pull request #114 from YannickJadoul/deterministic-preamble-order
Sorting build options dict items when printing preamble
2018-11-30 16:09:53 +01:00
Yannick Jadoul 5adfce9838 Merge pull request #113 from xanderyzwich/master
Refactoring: remove package_name
2018-11-30 00:03:03 +01:00
Yannick Jadoul bc83839387 Sorting build options dict items when printing preamble 2018-11-30 00:02:18 +01:00
Yannick Jadoul 481b1cb060 Revert "Uninstalling oclint in Travis CI config to fix HomeBrew upgrade of gcc (triggered by upgrading Python)"
This reverts commit c43f1b6e3d.
2018-11-29 23:37:04 +01:00
Yannick Jadoul ccfc3c3aa1 Fixing indentation error, and keeping only check for setup.py in project root directory in __main__.py 2018-11-29 23:26:10 +01:00
unknown 3d23cbf2a6 corrected as per notations from @YannickJadoul on #113 2018-11-29 02:44:19 +00:00
Joe Rickerby b9a296324d Merge pull request #108 from YannickJadoul/py27-popen-args
Fixing "AttributeError: 'Popen' object has no attribute 'args'" on Python 2.7 for Linux builds
2018-11-06 21:22:22 +00:00
Corey McCarty c4822ebdd6 Remove Try/Except for package_name #102
recommended by @YannickJadoul
2018-11-05 16:30:34 -06:00
corey.d.mccarty 2ca6500bba removed '--name' pieces for #102 2018-11-05 11:47:31 -05:00
corey.d.mccarty 8dd71189b2 removed 'package-name' as per #102 2018-11-05 11:16:42 -05:00
Yannick Jadoul a448c02d7f Fixing "AttributeError: 'Popen' object has no attribute 'args'" on Python 2.7 for Linux builds 2018-10-28 11:51:10 +01:00
Yannick Jadoul 696866bfe1 Merge pull request #111 from mayeut/openssl-update
Update openssl patch from 1.0.2o to 1.0.2p on macOS
2018-10-28 11:46:20 +01:00
Yannick Jadoul 1faa2fe5df Merge pull request #110 from mayeut/python-update
Update Python from 3.6.6, 3.7.0 to 3.6.7, 3.7.1 on macOS
2018-10-28 11:46:05 +01:00
Yannick Jadoul 602acebcc8 Merge pull request #112 from YannickJadoul/brew-upgrade-gcc
Fixing Travic CI build with HomeBrew upgrade of gcc
2018-10-28 11:44:38 +01:00
Yannick Jadoul c43f1b6e3d Uninstalling oclint in Travis CI config to fix HomeBrew upgrade of gcc (triggered by upgrading Python) 2018-10-27 19:22:53 +02:00
mayeut 0d3e67b1f9 Update openssl patch from 1.0.2o to 1.0.2p on macOS 2018-10-26 19:25:25 +02:00
mayeut e3b565320c Update Python from 3.6.6, 3.7.0 to 3.6.7, 3.7.1 on macOS 2018-10-26 19:23:49 +02:00
Yannick Jadoul 56973d34df Merge pull request #109 from safijari/add_apriltags2_to_readme
Add apriltags2-ethz library to readme
2018-10-18 11:13:02 +02:00
Jariullah Safi 66e9a1cc19 Update README.md 2018-10-18 00:04:25 -07:00
Joe Rickerby 2a91396064 Bump version 2018-09-23 22:08:39 +01:00
Joe Rickerby 1c98a9ed22 Update changelog 2018-09-23 22:07:47 +01:00
Joe Rickerby dbcf5ef64b Clarify README shared library note 2018-09-23 22:07:36 +01:00
Joe Rickerby cc030312a3 Prepare changelog update 2018-09-23 21:38:36 +01:00
Joe Rickerby 31c162fa59 Remove Python 3.3 support on windows 2018-09-23 21:38:36 +01:00
Joe Rickerby d80636aa22 Merge pull request #101 from YannickJadoul/cibw-build
Adding CIBW_BUILD option
2018-09-23 21:31:42 +01:00
Yannick Jadoul 22a6f0c7a9 Fixing tests by reducing minimum number of built wheels 2018-09-23 13:08:26 +02:00
Yannick Jadoul 7362075b61 Changing variable name of BuildSelector instance, updating Python version in test 04_build_skip, and adding unit tests for BuildSelector 2018-09-23 13:08:26 +02:00
Yannick Jadoul c108807e16 Renaming test 04_skip to 04_build_skip and amending it to test CIBW_BUILD as well 2018-09-23 13:07:28 +02:00
Yannick Jadoul 05a054245a Updating README.md to include CIBW_BUILD and interation with CIBW_SKIP 2018-09-23 13:07:28 +02:00
Yannick Jadoul a5f89fa7aa Adding CIBW_BUILD option and changing util.BuildSkipper to util.BuildSelector 2018-09-23 13:07:28 +02:00
Yannick Jadoul cf73851c61 Merge pull request #97 from YannickJadoul/docker-without-timeout
Reordering the Linux docker commands to get rid of timeout logic
2018-09-23 12:18:32 +02:00
Joe Rickerby 9372e37b18 Merge pull request #95 from anthrotype/overwrite-existing-wheel
overwrite existing wheel file with same name
2018-09-16 19:24:21 +01:00
Joe Rickerby fbb447c3aa Merge pull request #100 from msanders/patch-1
Add AutoPy
2018-09-15 11:08:42 +01:00
Michael Sanders 1e978d1b35 Add AutoPy 2018-09-14 12:36:46 -07:00
Yannick Jadoul 1dbda7af7e Add '--force' to 'docker rm' command in Linux builds 2018-09-12 11:52:41 +02:00
Yannick Jadoul b64770a560 Cleaning up use of run_docker and removing some debug output 2018-09-09 16:11:42 +02:00
Yannick Jadoul 4cc09a35b4 Removing monotonic from dependencies (until necessary again) 2018-09-09 01:24:07 +02:00
Yannick Jadoul d4885fe44a Fixing outdated Python 2 syntax error 2018-09-09 01:24:07 +02:00
Yannick Jadoul fcf1e9cd84 Moving stdin communication with docker to run_docker function 2018-09-09 01:23:48 +02:00
Yannick Jadoul f417a91a57 First try at getting rid of timeout logic in Linux docker handling, by swapping splitting docker run into create and start 2018-09-09 00:53:08 +02:00
Cosimo Lupo 1b9c74aabc macos: no need to call os.remove, shutil.move overwrites on posix 2018-09-08 18:41:39 +01:00
Joe Rickerby 35832c4309 Readme tweaks 2018-09-08 17:13:39 +01:00
Cosimo Lupo c9fffab6ea remove same-named wheel before shutil.move
In https://github.com/adobe-type-tools/afdko/pull/595 we attempt to build a py2.py3 wheel
thus the filename of the wheel is the same as the previous ones, and it causes a shutil.Error
https://travis-ci.org/adobe-type-tools/afdko/jobs/424691105#L1460

just remove the previous existing file before moving the new one (they'll be identical anyway,
or at least they *should*).
2018-09-06 11:34:50 +01:00
Kyle Altendorf 9f7fdbf456 Add CircleCI support (#91)
* Initial guess at supporting CircleCI

joerick/cibuildwheel#90

* Add note about setup_remote_docker on Circle

* Add diagnostic print of entire docker command

* Add more diagnostic prints

* Retrigger for Circle build

* Preliminary .circleci/config.yml

* Add workflows to Circle

* drop osx on circle

* call python3 directly on circle

* drop repeat pip install

* use a virtualenv on circle

* use a virtualenv on circle... again

* use a virtualenv on circle... again... again

* setup_remote_docker on circle

* try to manually copy into docker on circle

* correct Host

* correct docker calls

* refactor docker calls

* take 2 on the docker cp destination

* more docker cp debugging

* more docker cp debugging... again

* docker cp explicitly to /project

* wait until docker container is up

* wait until docker container is up... again

* and copy results back

* and copy results back... again

* create /output in container

* create /output in container... again

* copy contents of container's /output

* Retrigger ci

* fix merge typo

* fix monotonic

* always install monotonic

* always install monotonic... again

* correct to use project_dir

* wait a bit longer and a bit smarter

* wait a bit longer and a bit smarter... again

* add back circle osx

* add back circle osx... again

* py2/3 for circle

* py2/3 for circle... again

* use python2.7 image on circle linux

* (mostly) remove unused ci host identification code

* skip inspecting docker for the container, just retry copy on failure

* cleanup docker container and images

* fully remove ci host detection

* docs and tidy

* add circle badge
2018-09-05 09:15:32 +01:00
Yannick Jadoul 7c6b331e15 Merge pull request #84 from YannickJadoul/unbuffered-output
Applying unbuffered output to all platforms instead of only Windows
2018-09-02 22:49:26 +02:00
Tommy 22f69e4ced Added a repo using cibuildwheel and fixed some spelling (#92)
* Added KDEpy and fixed some spelling

* Reversed spelling, added repo to README

* Fixing single 'progams' typo in README
2018-09-02 22:40:49 +02:00
Joe Rickerby 4083a7600a Merge pull request #88 from miguelsousa/show-err-output
Improve error reporting during package name fetching
2018-08-21 22:52:45 +01:00
Miguel Sousa 20624e44a3 Improve error reporting during package name fetching 2018-08-21 14:11:28 -07:00
Yannick Jadoul 332b8ec6ac Merge pull request #86 from miguelsousa/patch-1
Update README.md
2018-08-20 09:09:51 +02:00
Miguel Sousa eaf59e385c Update README.md
Add horizontal rules between option descriptions; they make it easier to understand what text belongs to each option.
2018-08-19 21:06:43 -07:00
Yannick Jadoul 57454e75b0 Applying unbuffered output to all platforms instead of only Windows 2018-08-02 12:04:39 +02:00
Joe Rickerby 391c95d355 Update README.md 2018-07-29 18:21:19 +01:00
38 changed files with 1033 additions and 252 deletions
+81
View File
@@ -0,0 +1,81 @@
version: 2
jobs:
osx-python2:
macos:
xcode: "10.0.0"
environment:
PYTHON: python2
steps:
- checkout
- run:
name: Prepare the environment.
command: |
bash .circleci/prepare.sh
- run:
name: Test.
command: |
venv/bin/python ./bin/run_tests.py
osx-python3:
macos:
xcode: "10.0.0"
environment:
PYTHON: python3
steps:
- checkout
- run:
name: Prepare the environment.
command: |
bash .circleci/prepare.sh
- run:
name: Test.
command: |
venv/bin/python ./bin/run_tests.py
linux-python2:
docker:
- image: circleci/python:2.7
environment:
PYTHON: python2
steps:
- checkout
- setup_remote_docker
- run:
name: Prepare the environment.
command: |
bash .circleci/prepare.sh
- run:
name: Test.
command: |
venv/bin/python ./bin/run_tests.py
linux-python3:
docker:
- image: circleci/python:3.6
environment:
PYTHON: python3
steps:
- checkout
- setup_remote_docker
- run:
name: Prepare the environment.
command: |
bash .circleci/prepare.sh
- run:
name: Test.
command: |
venv/bin/python ./bin/run_tests.py
workflows:
version: 2
all-tests:
jobs:
- osx-python2
- osx-python3
- linux-python2
- linux-python3
+7
View File
@@ -0,0 +1,7 @@
$PYTHON --version
$PYTHON -m pip --version
$PYTHON -m pip install -q --user --ignore-installed --upgrade virtualenv
$PYTHON -m virtualenv -p $PYTHON venv
venv/bin/python -m pip install -r requirements-dev.txt
venv/bin/python -m pip freeze
venv/bin/python --version
+276 -40
View File
@@ -1,11 +1,11 @@
cibuildwheel
============
[![PyPI](https://img.shields.io/pypi/v/cibuildwheel.svg)](https://pypi.python.org/pypi/cibuildwheel) [![Build Status](https://travis-ci.org/joerick/cibuildwheel.svg?branch=master)](https://travis-ci.org/joerick/cibuildwheel) [![Build status](https://ci.appveyor.com/api/projects/status/wbsgxshp05tt1tif/branch/master?svg=true)](https://ci.appveyor.com/project/joerick/cibuildwheel/branch/master)
[![PyPI](https://img.shields.io/pypi/v/cibuildwheel.svg)](https://pypi.python.org/pypi/cibuildwheel) [![Build Status](https://travis-ci.org/joerick/cibuildwheel.svg?branch=master)](https://travis-ci.org/joerick/cibuildwheel) [![Build status](https://ci.appveyor.com/api/projects/status/wbsgxshp05tt1tif/branch/master?svg=true)](https://ci.appveyor.com/project/joerick/cibuildwheel/branch/master) [![CircleCI](https://circleci.com/gh/joerick/cibuildwheel.svg?style=svg)](https://circleci.com/gh/joerick/cibuildwheel)
Python wheels are great. Building them across **Mac, Linux, Windows**, on **multiple versions of Python**, is not.
`cibuildwheel` is here to help. `cibuildwheel` runs on your CI server - currently it supports Travis CI and Appveyor - and it builds and tests your wheels across all of your platforms.
`cibuildwheel` is here to help. `cibuildwheel` runs on your CI server - currently it supports Azure Pipelines, Travis CI, Appveyor, and CircleCI - and it builds and tests your wheels across all of your platforms.
**`cibuildwheel` is in beta**. It's brand new - I'd love for you to try it and help make it better!
@@ -14,29 +14,96 @@ What does it do?
| | macOS 10.6+ | manylinux i686 | manylinux x86_64 | Windows 32bit | Windows 64bit |
|---|---|---|---|---|---|
| Python 2.7 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.3 | | | | ⚠️ | ⚠️ |
| Python 3.4 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.5 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.6 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.7 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 2.7 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.4 | | | | ✅* | ✅* |
| Python 3.5 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.6 | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python 3.7 | ✅ | ✅ | ✅ | ✅ | ✅ |
<small>[⚠️=deprecated]</small>
> \* Not supported on Azure Pipelines
- Builds manylinux, macOS and Windows (32 and 64bit) wheels using Travis CI and Appveyor
- Builds manylinux, macOS and Windows (32 and 64bit) wheels using Azure Pipelines, Travis CI, Appveyor, and CircleCI
- Bundles shared library dependencies on Linux and macOS through [auditwheel](https://github.com/pypa/auditwheel) and [delocate](https://github.com/matthew-brett/delocate)
- Runs the library test suite against the wheel-installed version of your library
Usage
-----
`cibuildwheel` currently works on **Travis CI** to build Linux and Mac wheels, and **Appveyor** to build Windows wheels.
`cibuildwheel` currently works **Travis CI** and **CircleCI** to build Linux and Mac wheels, and **Appveyor** to build Windows wheels. **Azure Pipelines** supports all three.
`cibuildwheel` is not intended to run on your development machine. It will try to install packages globally; this is no good. Travis CI and Appveyor run their builds in isolated environments, so are ideal for this kind of script.
| | Linux | macOS | Windows |
|-----------------|-------|-------|---------|
| Azure Pipelines | ✅ | ✅ | ✅ |
| Travis CI | ✅ | ✅ | |
| Appveyor | | | ✅ |
| CircleCI | ✅ | ✅ | |
`cibuildwheel` is not intended to run on your development machine. It will try to install packages globally; this is no good. Travis CI, CircleCI, and Appveyor run their builds in isolated environments, so are ideal for this kind of script.
### Minimal setup
- Create a `.travis.yml` file in your repo.
<details>
<summary><b>Azure Pipelines</b>
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/apple.svg" />
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/windows.svg" />
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/linux.svg" />
</summary>
- Using Azure pipelines, you can build all three platforms on the same service. Create a `azure-pipelines.yml` file in your repo.
**azure-pipelines.yml**
```yaml
jobs:
- job: linux
pool: {vmImage: 'Ubuntu-16.04'}
steps:
- task: UsePythonVersion@0
- bash: |
python -m pip install --upgrade pip
pip install cibuildwheel==0.10.1
cibuildwheel --output-dir wheelhouse .
- task: PublishBuildArtifacts@1
inputs: {pathtoPublish: 'wheelhouse'}
- job: macos
pool: {vmImage: 'macOS-10.13'}
steps:
- task: UsePythonVersion@0
- bash: |
python -m pip install --upgrade pip
pip install cibuildwheel==0.10.1
cibuildwheel --output-dir wheelhouse .
- task: PublishBuildArtifacts@1
inputs: {pathtoPublish: 'wheelhouse'}
- job: windows
pool: {vmImage: 'vs2017-win2016'}
steps:
- {task: UsePythonVersion@0, inputs: {versionSpec: '2.7', architecture: x86}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '2.7', architecture: x64}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.5', architecture: x86}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.5', architecture: x64}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.6', architecture: x86}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.6', architecture: x64}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.7', architecture: x86}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.7', architecture: x64}}
- script: choco install vcpython27 -f -y
displayName: Install Visual C++ for Python 2.7
- bash: |
python -m pip install --upgrade pip
pip install cibuildwheel==0.10.1
cibuildwheel --output-dir wheelhouse .
- task: PublishBuildArtifacts@1
inputs: {pathtoPublish: 'wheelhouse'}
```
</details>
<details>
<summary><b>Travis CI</b>
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/apple.svg" />
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/linux.svg" />
</summary>
- To build Linux and Mac wheels on Travis CI, create a `.travis.yml` file in your repo.
```
language: python
@@ -52,17 +119,80 @@ Usage
env: PIP=pip2
script:
- $PIP install cibuildwheel==0.9.4
- $PIP install cibuildwheel==0.11.1
- cibuildwheel --output-dir wheelhouse
```
Then setup a deployment method by following the [Travis CI deployment docs](https://docs.travis-ci.com/user/deployment/), or see [Delivering to PyPI](#delivering-to-pypi) below.
- Create an `appveyor.yml` file in your repo.
</details>
<details>
<summary><b>CircleCI</b>
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/apple.svg" />
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/linux.svg" />
</summary>
- To build Linux and Mac wheels on CircleCI, create a `.circleci/config.yml` file in your repo,
```
version: 2
jobs:
linux-wheels:
working_directory: ~/linux-wheels
docker:
- image: circleci/python:3.6
steps:
- checkout
- setup_remote_docker
- run:
name: Build the Linux wheels.
command: |
pip install --user cibuildwheel
cibuildwheel --output-dir wheelhouse
- store_artifacts:
path: wheelhouse/
osx-wheels:
working_directory: ~/osx-wheels
macos:
xcode: "10.0.0"
steps:
- checkout
- run:
name: Build the OS X wheels.
command: |
pip install --user cibuildwheel
cibuildwheel --output-dir wheelhouse
- store_artifacts:
path: wheelhouse/
workflows:
version: 2
all-tests:
jobs:
- linux-wheels
- osx-wheels
```
Note: CircleCI doesn't enable free macOS containers for open source by default, but you can ask for access. See [here](https://circleci.com/docs/2.0/oss/#overview) for more information.
CircleCI will store the built wheels for you - you can access them from the project console.
</details>
<details>
<summary><b>Appveyor</b>
<img width="16" src="https://unpkg.com/simple-icons@latest/icons/windows.svg" />
</summary>
- To build Windows wheels on Appveyor, create an `appveyor.yml` file in your repo.
```
build_script:
- pip install cibuildwheel==0.9.4
- pip install cibuildwheel==0.11.1
- cibuildwheel --output-dir wheelhouse
artifacts:
- path: "wheelhouse\\*.whl"
@@ -71,6 +201,8 @@ Usage
Appveyor will store the built wheels for you - you can access them from the project console. Alternatively, you may want to store them in the same place as the Travis CI build. See [Appveyor deployment docs](https://www.appveyor.com/docs/deployment/) for more info, or see [Delivering to PyPI](#delivering-to-pypi) below.
</details>
- Commit those files, enable building of your repo on Travis CI and Appveyor, and push.
All being well, you should get wheels delivered to you in a few minutes.
@@ -84,6 +216,7 @@ All being well, you should get wheels delivered to you in a few minutes.
| | Option | |
|---|---|---|
| **Target wheels** | `CIBW_PLATFORM` | Override the auto-detected target platform |
| | `CIBW_BUILD` | Build only certain Python versions |
| | `CIBW_SKIP` | Skip certain Python versions |
| **Build parameters** | `CIBW_BUILD_VERBOSITY` | Increase or decrease the output of `pip wheel` |
| **Build environment** | `CIBW_ENVIRONMENT` | Set environment variables needed during the build |
@@ -98,8 +231,8 @@ A more detailed description of the options, the allowed values, and some example
### Linux builds on Docker
Linux wheels are built in the [`manylinux1` docker images](https://github.com/pypa/manylinux) to provide binary compatible wheels on Linux, according to [PEP 513](https://www.python.org/dev/peps/pep-0513/). Because of this, when building with `cibuildwheel` on Linux, a few things should be taken into account:
- Progams and libraries cannot be installed on the Travis CI Ubuntu host with `apt-get`, but can be installed inside of the Docker image using `yum` or manually. The same goes for environment variables that are potentially needed to customize the wheel building. `cibuildwheel` supports this by providing the `CIBW_ENVIRONMENT` and `CIBW_BEFORE_BUILD` options to setup the build environment inside the running Docker image. See [below](#options) for details on these options.
- The project directory is mounted in the running Docker instance as `/project`, the output directory for the wheels as `/output`. In general, this is handled transparently by `cibuildwheel`. For a more finegrained level of control however, the root of the host file system is mounted as `/host`, allowing for example to access shared files, caches, etc. on the host file system.
- Programs and libraries cannot be installed on the Travis CI Ubuntu host with `apt-get`, but can be installed inside of the Docker image using `yum` or manually. The same goes for environment variables that are potentially needed to customize the wheel building. `cibuildwheel` supports this by providing the `CIBW_ENVIRONMENT` and `CIBW_BEFORE_BUILD` options to setup the build environment inside the running Docker image. See [below](#options) for details on these options.
- The project directory is mounted in the running Docker instance as `/project`, the output directory for the wheels as `/output`. In general, this is handled transparently by `cibuildwheel`. For a more finegrained level of control however, the root of the host file system is mounted as `/host`, allowing for example to access shared files, caches, etc. on the host file system. Note that this is not available on CircleCI due to their Docker policies.
- Alternative dockers images can be specified with the `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE` options to allow for a custom, preconfigured build environment for the Linux builds. See [below](#options) for more details.
@@ -107,9 +240,8 @@ Options
-------
```
usage: cibuildwheel [-h]
[--output-dir OUTPUT_DIR]
[--platform PLATFORM]
usage: cibuildwheel [-h] [--platform {auto,linux,macos,windows}]
[--output-dir OUTPUT_DIR] [--print-build-identifiers]
[project_dir]
Build wheels for all the platforms.
@@ -129,11 +261,16 @@ optional arguments:
you need to run in Windows, and it will build and test
for all versions of Python at C:\PythonXX[-x64].
--output-dir OUTPUT_DIR
Destination folder for the wheels.
Destination folder for the wheels.
--print-build-identifiers
Print the build identifiers matched by the current
invocation and exit.
```
Most of the config is via environment variables. These go into `.travis.yml` and `appveyor.yml` nicely.
Most of the config is via environment variables. These go into `.travis.yml`, `appveyor.yml`, and `.circleci/config.yml` nicely.
***
| Environment variable: `CIBW_PLATFORM` | Command line argument: `--platform`
| --- | ---
@@ -142,16 +279,20 @@ Options: `auto` `linux` `macos` `windows`
Default: `auto`
`auto` will auto-detect platform using environment variables, such as `TRAVIS_OS_NAME`/`APPVEYOR`.
`auto` will auto-detect platform using environment variables, such as `TRAVIS_OS_NAME`/`APPVEYOR`/`CIRCLECI`.
For `linux` you need Docker running, on Mac or Linux. For `macos`, you need a Mac machine, and note that this script is going to automatically install MacPython on your system, so don't run on your development machine. For `windows`, you need to run in Windows, and it will build and test for all versions of Python at `C:\PythonXX[-x64]`.
| Environment variable: `CIBW_SKIP`
***
| Environment variables: `CIBW_BUILD` and `CIBW_SKIP`
| ---
Optional.
Space-separated list of builds to skip. Each build has an identifier like `cp27-manylinux1_x86_64` or `cp34-macosx_10_6_intel` - you can list ones to skip here and `cibuildwheel` won't try to build them.
Space-separated list of builds to build and skip. Each build has an identifier like `cp27-manylinux1_x86_64` or `cp34-macosx_10_6_intel` - you can list specific ones to build and `cibuildwheel` will only build those, and/or list ones to skip and `cibuildwheel` won't try to build them.
When both options are specified, both conditions are applied and only builds with a tag that matches `CIBW_BUILD` and does not match `CIBW_SKIP` will be built.
The format is `python_tag-platform_tag`. The tags are as defined in [PEP 0425](https://www.python.org/dev/peps/pep-0425/#details).
@@ -162,13 +303,16 @@ Platform tags look like `macosx_10_6_intel` `manylinux1_x86_64` `manylinux1_i686
You can also use shell-style globbing syntax (as per `fnmatch`)
Examples:
- Skip building on Python 2.7 on the Mac: `cp27-macosx_10_6_intel`
- Skip building on Python 2.7 on all platforms: `cp27-*`
- Skip Python 2.7 on Windows: `cp27-win*`
- Skip Python 2.7 on 32bit Windows: `cp27-win32`
- Skip Python 3.4 and Python 3.5: `cp34-* cp35-*`
- Skip Python 3.6 on Linux: `cp36-manylinux*`
- Only build on Python 3.6: `cp27-* cp34-* cp35-*`
- Only build on Python 3.6: `CIBW_BUILD`:`cp36-*`
- Skip building on Python 2.7 on the Mac: `CIBW_SKIP`:`cp27-macosx_10_6_intel`
- Skip building on Python 2.7 on all platforms: `CIBW_SKIP`:`cp27-*`
- Skip Python 2.7 on Windows: `CIBW_SKIP`:`cp27-win*`
- Skip Python 2.7 on 32-bit Windows: `CIBW_SKIP`:`cp27-win32`
- Skip Python 3.4 and Python 3.5: `CIBW_SKIP`:`cp34-* cp35-*`
- Skip Python 3.6 on Linux: `CIBW_SKIP`:`cp36-manylinux*`
- Only build on Python 3 and skip 32-bit builds: `CIBW_BUILD`:`cp3?-*` and `CIBW_SKIP`:`*-win32 *-manylinux1_i686`
***
| Environment variable: `CIBW_BUILD_VERBOSITY`
| ---
@@ -180,6 +324,8 @@ An number from 1 to 3 to increase the level of verbosity (corresponding to invok
Platform-specific variants also available:
`CIBW_BUILD_VERBOSITY_MACOS` | `CIBW_BUILD_VERBOSITY_WINDOWS` | `CIBW_BUILD_VERBOSITY_LINUX`
***
| Environment variable: `CIBW_ENVIRONMENT`
| ---
@@ -201,6 +347,8 @@ Platform-specific variants also available:
In addition to the above, `cibuildwheel` always defines the environment variable `CIBUILDWHEEL=1`. This can be useful for [building wheels with optional extensions](https://github.com/joerick/cibuildwheel/wiki/Building-packages-with-optional-C-extensions).
***
| Environment variable: `CIBW_BEFORE_BUILD`
| ---
@@ -219,6 +367,8 @@ Example: `yum install -y libffi-dev && pip install .`
Platform-specific variants also available:
`CIBW_BEFORE_BUILD_MACOS` | `CIBW_BEFORE_BUILD_WINDOWS` | `CIBW_BEFORE_BUILD_LINUX`
***
| Environment variables: `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE`
| ---
@@ -231,6 +381,8 @@ Beware to specify a valid docker image that can be used the same as the official
Example: `dockcross/manylinux-x64`
Example: `dockcross/manylinux-x86`
***
| Environment variable: `CIBW_TEST_COMMAND`
| ---
@@ -245,6 +397,8 @@ Example: `nosetests {project}/tests`
Platform-specific variants also available:
`CIBW_TEST_COMMAND_MACOS` | `CIBW_TEST_COMMAND_WINDOWS` | `CIBW_TEST_COMMAND_LINUX`
***
| Environment variable: `CIBW_TEST_REQUIRES`
| ---
@@ -258,9 +412,8 @@ Example: `nose==1.3.7 moto==0.4.31`
Platform-specific variants also available:
`CIBW_TEST_REQUIRES_MACOS` | `CIBW_TEST_REQUIRES_WINDOWS` | `CIBW_TEST_REQUIRES_LINUX`
--
#### Example YML syntax
### Example YML syntax
<table>
<tr><td><i>example .travis.yml environment variables</i><pre><code>env:
@@ -316,8 +469,9 @@ It didn't work!
If your wheel didn't compile, check the list below for some debugging tips.
- A mistake in your config. To quickly test your config without doing a git push and waiting for your code to build on CI, you can run the Linux build in a Docker container. On Mac or Linux, with Docker running, try `cibuildwheel --platform linux`. You'll have to bring your config into the current environment first.
- Missing dependency. You might need to install something on the build machine. You can do this in `.travis.yml` or `appveyor.yml`, with apt-get, brew or whatever Windows uses :P . Given how the Linux build works, we'll probably have to build something into `cibuildwheel`. Let's chat about that over in the issues!
- Missing dependency. You might need to install something on the build machine. You can do this in `.travis.yml`, `appveyor.yml`, or `.circleci/config.yml`, with apt-get, brew or whatever Windows uses :P . Given how the Linux build works, we'll probably have to build something into `cibuildwheel`. Let's chat about that over in the issues!
- Windows: missing C feature. The Windows C compiler doesn't support C language features invented after 1990, so you'll have to backport your C code to C90. For me, this mostly involved putting my variable declarations at the top of the function like an animal.
- MacOS: calling cibuildwheel from a python3 script and getting a `ModuleNotFoundError`? Due to a [bug](https://bugs.python.org/issue22490) in CPython, you'll need to [unset the `__PYVENV_LAUNCHER__` variable](https://github.com/joerick/cibuildwheel/issues/133#issuecomment-478288597) before activating a venv.
Working examples
----------------
@@ -329,96 +483,176 @@ Here are some repos that use cibuildwheel.
- [Parselmouth](https://github.com/YannickJadoul/Parselmouth)
- [python-admesh](https://github.com/admesh/python-admesh)
- [pybase64](https://github.com/mayeut/pybase64)
- [KDEpy](https://github.com/tommyod/KDEpy)
- [AutoPy](https://github.com/autopilot-rs/autopy)
- [apriltags2-ethz](https://github.com/safijari/apriltags2_ethz)
> Add repo here! Send a PR.
Legal note
----------
Since `cibuildwheel` runs the wheel through delocate or auditwheel, it will automatically bundle library dependencies. This is similar to static linking, so it might have some licence implications. Check the license for any code you're pulling in to make sure that's allowed.
Since `cibuildwheel` runs the wheel through delocate or auditwheel, it might automatically bundle dynamically linked libraries from the build machine.
It helps ensure that the library can run without any dependencies outside of the pip toolchain.
This is similar to static linking, so it might have some licence implications. Check the license for any code you're pulling in to make sure that's allowed.
Changelog
=========
### 0.11.0
_26 May 2019_
- ✨ Add support for building on Azure pipelines! This lets you build all
Linux, Mac and Windows wheels on one service, so it promises to be the
easiest to set up! Check out the quickstart in the docs, or
[cibuildwheel-azure-example](https://github.com/joerick/cibuildwheel-azure-example)
for an example project. (#126, #132)
- 🛠 Internal change - the end-to-end test projects format was updated, so we
can more precisely assert what should be produced for each one. (#136, #137).
### 0.10.2
_10 March 2019_
- 🛠 Revert temporary fix in macOS, that was working around a bug in pip 19 (#129)
- 🛠 Update Python to 2.7.16 on macOS
- 🛠 Update OpenSSL patch to 1.0.2r on macOS
### 0.10.1
_3 February 2019_
- 🐛 Fix build stalling on macOS (that was introduced in pip 19) (#122)
- 🐛 Fix "AttributeError: 'Popen' object has no attribute 'args'" on Python 2.7 for Linux builds (#108)
- 🛠 Update Python from 3.6.7, 3.7.1 to 3.6.8, 3.7.2 on macOS
- 🛠 Update openssl patch from 1.0.2p to 1.0.2q on macOS
- 🛠 Sorting build options dict items when printing preamble (#114)
### 0.10.0
_23 September 2018_
- ✨ Add `CIBW_BUILD` option, for specifying which specific builds to perform (#101)
- ✨ Add support for building Mac and Linux on CircleCI (#91, #97)
- 🛠 Improved support for building universal wheels (#95)
- 🛠 Ensure log output is unbuffered and therefore in the correct order (#92)
- 🛠 Improved error reporting for errors that occur inside a package's setup.py (#88)
- ⚠️ Removed support for Python 3.3 on Windows.
### 0.9.4
_29 July 2018_
- 🛠 CIBW_TEST_COMMAND now runs in a shell on Mac (as well as Linux) (#81)
### 0.9.3
_10 July 2018_
- 🛠 Update to Python 3.6.6 on macOS (#82)
- ✨ Add support for building Python 3.7 wheels on Windows (#76)
- ⚠️ Deprecated support for Python 3.3 on Windows.
### 0.9.2
_1 July 2018_
- 🛠 Update Python 3.7.0rc1 to 3.7.0 on macOS (#79)
### 0.9.1
_18 June 2018_
- 🛠 Removed the need to use `{python}` and `{pip}` in `CIBW_BEFORE_BUILD` statements, by ensuring the correct version is always on the path at `python` and `pip` instead. (#60)
- 🛠 We now patch the _ssl module on Python 3.4 and 3.5 so these versions can still make SSL web requests using TLS 1.2 while building. (#71)
### 0.9.0
_18 June 2018_
- ✨ Add support for Python 3.7 (#73)
### 0.8.0
_4 May 2018_
- ⚠️ Drop support for Python 3.3 on Linux (#67)
- 🐛 Fix TLS by updating setuptools (#69)
### 0.7.1
_2 April 2017_
- 🐛 macOS: Fix Pip bugs resulting from PyPI TLS 1.2 enforcement
- 🐛 macOS: Fix brew Python3 version problems in the CI
### 0.7.0
_7 January 2018_
- ✨ You can now specify a custom docker image using the `CIBW_MANYLINUX1_X86_64_IMAGE` and `CIBW_MANYLINUX1_I686_IMAGE` options. (#46)
- 🐛 Fixed a bug where cibuildwheel would download and build a package from PyPI(!) instead of building the package on the local machine. (#51)
### 0.6.0
_9 October 2017_
- ✨ On the Linux build, the host filesystem is now accessible via `/host` (#36)
- 🐛 Fixed a bug where setup.py scripts would run the wrong version of Python when running subprocesses on Linux (#35)
### 0.5.1
_10 September 2017_
- 🐛 Fixed a couple of bugs on Python 3.
- ✨ Added experimental support for Mac builds on [Bitrise.io](https://www.bitrise.io)
### 0.5.0
_7 September 2017_
- ✨ `CIBW_ENVIRONMENT` added. You can now set environment variables for each build, even within the Docker container on Linux. This is a big one! (#21)
- ✨ `CIBW_BEFORE_BUILD` now runs in a system shell on all platforms. You can now do things like `CIBW_BEFORE_BUILD="cmd1 && cmd2"`. (#32)
### 0.4.1
_14 August 2017_
- 🐛 Fixed a bug on Windows where subprocess' output was hidden (#23)
- 🐛 Fixed a bug on Appveyor where logs would appear in the wrong order due to output buffering (#24, thanks @YannickJadoul!)
### 0.4.0
_23 July 2017_
- 🐛 Fixed a bug that was increasing the build time by building the wheel twice. This was a problem for large projects that have a long build time. If you're upgrading and you need the old behaviour, use `CIBW_BEFORE_BUILD={pip} install .`, or install exactly the dependencies you need in `CIBW_BEFORE_BUILD`. See #18.
### 0.3.0
_27 June 2017_
- ⚠️ Removed Python 2.6 support on Linux (#12)
### 0.2.1
11 June 2017
_11 June 2017_
- 🛠 Changed the build process to install the package before building the wheel - this allows direct dependencies to be installed first (#9, thanks @tgarc!)
- ✨ Added Python 3 support for the main process, for systems where Python 3 is the default (#8, thanks @tgarc).
### 0.2.0
13 April 2017
_13 April 2017_
- ✨ Added `CIBW_SKIP` option, letting users explicitly skip a build
- ✨ Added `CIBW_BEFORE_BUILD` option, letting users run a shell command before the build starts
### 0.1.3
31 March 2017
_31 March 2017_
- 🌟 First public release!
@@ -470,4 +704,6 @@ Massive props also to-
See also
--------
If you'd like to keep wheel building separate from the package itself, check out [astrofrog/autowheel](https://github.com/astrofrog/autowheel). It builds packages using cibuildwheel from source distributions on PyPI.
If `cibuildwheel` is too limited for your needs, consider [matthew-brett/multibuild](http://github.com/matthew-brett/multibuild). `multibuild` is a toolbox for building a wheel on various platforms. It can do a lot more than this project - it's used to build SciPy!
+34
View File
@@ -0,0 +1,34 @@
jobs:
- job: linux
pool: {vmImage: 'Ubuntu-16.04'}
steps:
- task: UsePythonVersion@0
- bash: |
python -m pip install -r requirements-dev.txt
python ./bin/run_tests.py
- job: macos
pool: {vmImage: 'macOS-10.13'}
steps:
- task: UsePythonVersion@0
- bash: |
python -m pip install -r requirements-dev.txt
python ./bin/run_tests.py
- job: windows
pool: {vmImage: 'vs2017-win2016'}
steps:
- {task: UsePythonVersion@0, inputs: {versionSpec: '2.7', architecture: x86}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '2.7', architecture: x64}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.5', architecture: x86}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.5', architecture: x64}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.6', architecture: x86}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.6', architecture: x64}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.7', architecture: x86}}
- {task: UsePythonVersion@0, inputs: {versionSpec: '3.7', architecture: x64}}
- script: choco install vcpython27 -f -y
displayName: Install Visual C++ for Python 2.7
- bash: |
python -m pip install -r requirements-dev.txt
python ./bin/run_tests.py
+18 -22
View File
@@ -1,31 +1,29 @@
#!/usr/bin/python
#!/usr/bin/env python
from __future__ import print_function
import os, sys, subprocess, shutil, json
from glob import glob
import os, sys, subprocess, shutil
project_root = os.path.dirname(os.path.dirname(__file__))
test_utils_dir = os.path.join(project_root, 'test', 'shared')
def single_run(test_project):
# load project settings into environment
env_file = os.path.join(test_project, 'environment.json')
project_env = {}
if os.path.exists(env_file):
with open(env_file) as f:
project_env = json.load(f)
# run the build
# set up an environment that gives access to the test utils
env = os.environ.copy()
project_env = {str(k): str(v) for k, v in project_env.items()} # unicode not allowed in env
env.update(project_env)
print('Building %s with environment %s' % (test_project, project_env))
subprocess.check_call([sys.executable, '-m', 'cibuildwheel', test_project], env=env)
wheels = glob('wheelhouse/*.whl')
print('%s built successfully. %i wheels built.' % (test_project, len(wheels)))
# check some wheels were actually built
assert len(wheels) >= 4
if 'PYTHONPATH' in env:
env['PYTHONPATH'] += os.pathsep + test_utils_dir
else:
env['PYTHONPATH'] = test_utils_dir
# run the test
subprocess.check_call(
[sys.executable, '-m', 'pytest', '-vv', os.path.join(test_project, 'cibuildwheel_test.py')],
env=env,
)
# clean up
shutil.rmtree('wheelhouse')
if os.path.exists('wheelhouse'):
shutil.rmtree('wheelhouse')
if __name__ == '__main__':
import argparse
@@ -41,5 +39,3 @@ if __name__ == '__main__':
exit(2)
single_run(project_path)
print('Project built successfully.')
+2 -2
View File
@@ -1,4 +1,4 @@
#!/usr/bin/python
#!/usr/bin/env python
from __future__ import print_function
import os, sys, subprocess, shutil, json
@@ -14,7 +14,7 @@ if __name__ == '__main__':
### run the integration tests
test_projects = glob('test/??_*')
test_projects = sorted(glob('test/??_*'))
if len(test_projects) == 0:
print('No test projects found. Aborting.', file=sys.stderr)
+1 -1
View File
@@ -1 +1 @@
__version__ = '0.9.4'
__version__ = '0.11.1'
+51 -26
View File
@@ -4,7 +4,7 @@ import argparse, os, subprocess, sys, textwrap
import cibuildwheel
import cibuildwheel.linux, cibuildwheel.windows, cibuildwheel.macos
from cibuildwheel.environment import parse_environment, EnvironmentParseError
from cibuildwheel.util import BuildSkipper
from cibuildwheel.util import BuildSelector, Unbuffered
def get_option_from_environment(option_name, platform=None, default=None):
'''
@@ -48,11 +48,17 @@ def main():
help=('Path to the project that you want wheels for. Default: the current '
'directory.'))
parser.add_argument('--print-build-identifiers',
action='store_true',
help='Print the build identifiers matched by the current invocation and exit.')
args = parser.parse_args()
if args.platform != 'auto':
platform = args.platform
else:
platform = None
if os.environ.get('TRAVIS_OS_NAME') == 'linux':
platform = 'linux'
elif os.environ.get('TRAVIS_OS_NAME') == 'osx':
@@ -61,9 +67,22 @@ def main():
platform = 'windows'
elif 'BITRISE_BUILD_NUMBER' in os.environ:
platform = 'macos'
else:
elif os.environ.get('CIRCLECI'):
if sys.platform.startswith('linux'):
platform = 'linux'
elif sys.platform.startswith('darwin'):
platform = 'macos'
elif 'AZURE_HTTP_USER_AGENT' in os.environ:
if os.environ['AGENT_OS'] == 'Linux':
platform = 'linux'
elif os.environ['AGENT_OS'] == 'Darwin':
platform = 'macos'
elif os.environ['AGENT_OS'] == 'Windows_NT':
platform = 'windows'
if platform is None:
print('cibuildwheel: Unable to detect platform. cibuildwheel should run on your CI server, '
'Travis CI and Appveyor are supported. You can run on your development '
'Travis CI, Appveyor, and CircleCI are supported. You can run on your development '
'machine using the --platform argument. Check --help output for more '
'information.',
file=sys.stderr)
@@ -75,7 +94,7 @@ def main():
project_dir = args.project_dir
before_build = get_option_from_environment('CIBW_BEFORE_BUILD', platform=platform)
build_verbosity = get_option_from_environment('CIBW_BUILD_VERBOSITY', platform=platform, default='')
skip_config = os.environ.get('CIBW_SKIP', '')
build_config, skip_config = os.environ.get('CIBW_BUILD', '*'), os.environ.get('CIBW_SKIP', '')
environment_config = get_option_from_environment('CIBW_ENVIRONMENT', platform=platform, default='')
try:
@@ -91,41 +110,28 @@ def main():
traceback.print_exc(None, sys.stderr)
exit(2)
skip = BuildSkipper(skip_config)
build_selector = BuildSelector(build_config, skip_config)
# Add CIBUILDWHEEL environment variable
# This needs to be passed on to the docker container in linux.py
os.environ['CIBUILDWHEEL'] = '1'
try:
project_setup_py = os.path.join(project_dir, 'setup.py')
name_output = subprocess.check_output([sys.executable, project_setup_py, '--name'],
universal_newlines=True)
# the last line of output is the name
package_name = name_output.strip().splitlines()[-1]
except subprocess.CalledProcessError as err:
if not os.path.exists(project_setup_py):
print('cibuildwheel: Could not find setup.py at root of project', file=sys.stderr)
exit(2)
else:
print('cibuildwheel: Failed to get name of the package. Command was %s' % err.cmd,
file=sys.stderr)
exit(2)
if package_name == '' or package_name == 'UNKNOWN':
print('cibuildwheel: Invalid package name "%s". Check your setup.py' % package_name,
file=sys.stderr)
if not os.path.exists(os.path.join(project_dir, 'setup.py')):
print('cibuildwheel: Could not find setup.py at root of project', file=sys.stderr)
exit(2)
if args.print_build_identifiers:
print_build_identifiers(platform, build_selector)
exit(0)
build_options = dict(
project_dir=project_dir,
package_name=package_name,
output_dir=output_dir,
test_command=test_command,
test_requires=test_requires,
before_build=before_build,
build_verbosity=build_verbosity,
skip=skip,
build_selector=build_selector,
environment=environment,
)
@@ -141,6 +147,9 @@ def main():
elif platform == 'windows':
pass
# Python is buffering by default when running on the CI platforms, giving problems interleaving subprocess call output with unflushed calls to 'print'
sys.stdout = Unbuffered(sys.stdout)
print_preamble(platform, build_options)
if not os.path.exists(output_dir):
@@ -155,6 +164,7 @@ def main():
else:
raise Exception('Unsupported platform')
def print_preamble(platform, build_options):
print(textwrap.dedent('''
_ _ _ _ _ _ _
@@ -168,7 +178,7 @@ def print_preamble(platform, build_options):
print('Build options:')
print(' platform: %r' % platform)
for option, value in build_options.items():
for option, value in sorted(build_options.items()):
print(' %s: %r' % (option, value))
warnings = detect_warnings(platform, build_options)
@@ -179,6 +189,21 @@ def print_preamble(platform, build_options):
print('\nHere we go!\n')
def print_build_identifiers(platform, build_selector):
if platform == 'linux':
python_configurations = cibuildwheel.linux.get_python_configurations(build_selector)
elif platform == 'windows':
python_configurations = cibuildwheel.windows.get_python_configurations(build_selector)
elif platform == 'macos':
python_configurations = cibuildwheel.macos.get_python_configurations(build_selector)
else:
python_configurations = []
for config in python_configurations:
print(config.identifier)
def detect_warnings(platform, build_options):
warnings = []
+44 -31
View File
@@ -1,5 +1,5 @@
from __future__ import print_function
import os, subprocess, sys
import os, subprocess, sys, uuid
from collections import namedtuple
from .util import prepare_command, get_build_verbosity_extra_flags
@@ -9,15 +9,7 @@ except ImportError:
from pipes import quote as shlex_quote
def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, build_verbosity, skip, environment, manylinux1_images):
try:
subprocess.check_call(['docker', '--version'])
except:
print('cibuildwheel: Docker not found. Docker is required to run Linux builds. '
'If you\'re building on Travis CI, add `services: [docker]` to your .travis.yml.',
file=sys.stderr)
exit(2)
def get_python_configurations(build_selector):
PythonConfiguration = namedtuple('PythonConfiguration', ['identifier', 'path'])
python_configurations = [
PythonConfiguration(identifier='cp27-manylinux1_x86_64', path='/opt/python/cp27-cp27m'),
@@ -35,8 +27,20 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
]
# skip builds as required
python_configurations = [c for c in python_configurations if not skip(c.identifier)]
return [c for c in python_configurations if build_selector(c.identifier)]
def build(project_dir, output_dir, test_command, test_requires, before_build, build_verbosity, build_selector, environment, manylinux1_images):
try:
subprocess.check_call(['docker', '--version'])
except:
print('cibuildwheel: Docker not found. Docker is required to run Linux builds. '
'If you\'re building on Travis CI, add `services: [docker]` to your .travis.yml.'
'If you\'re building on Circle CI in Linux, add a `setup_remote_docker` step to your .circleci/config.yml',
file=sys.stderr)
exit(2)
python_configurations = get_python_configurations(build_selector)
platforms = [
('manylinux1_x86_64', manylinux1_images.get('x86_64') or 'quay.io/pypa/manylinux1_x86_64'),
('manylinux1_i686', manylinux1_images.get('i686') or 'quay.io/pypa/manylinux1_i686'),
@@ -50,6 +54,7 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
bash_script = '''
set -o errexit
set -o xtrace
mkdir /output
cd /project
{environment_exports}
@@ -100,7 +105,6 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
chown {uid}:{gid} "/output/$(basename "$delocated_wheel")"
done
'''.format(
package_name=package_name,
pybin_paths=' '.join(c.path+'/bin' for c in platform_configs),
test_requires=' '.join(test_requires),
test_command=shlex_quote(
@@ -115,25 +119,34 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
gid=os.getgid(),
)
docker_process = subprocess.Popen([
'docker',
'run',
'--env',
'CIBUILDWHEEL',
'--rm',
'-i',
'-v', '%s:/project' % os.path.abspath(project_dir),
'-v', '%s:/output' % os.path.abspath(output_dir),
'-v', '/:/host',
docker_image,
'/bin/bash'],
stdin=subprocess.PIPE, universal_newlines=True)
def run_docker(command, stdin_str=None):
print('docker command: docker {}'.format(' '.join(map(shlex_quote, command))))
if stdin_str is None:
subprocess.check_call(['docker'] + command)
else:
args = ['docker'] + command
process = subprocess.Popen(args, stdin=subprocess.PIPE, universal_newlines=True)
try:
process.communicate(stdin_str)
except KeyboardInterrupt:
process.kill()
process.wait()
if process.returncode != 0:
raise subprocess.CalledProcessError(process.returncode, args)
container_name = 'cibuildwheel-{}'.format(uuid.uuid4())
try:
docker_process.communicate(bash_script)
except KeyboardInterrupt:
docker_process.kill()
docker_process.wait()
if docker_process.returncode != 0:
run_docker(['create',
'--env', 'CIBUILDWHEEL',
'--name', container_name,
'-i',
'-v', '/:/host', # ignored on Circle
docker_image, '/bin/bash'])
run_docker(['cp', os.path.abspath(project_dir) + '/.', container_name + ':/project'])
run_docker(['start', '-i', '-a', container_name], stdin_str=bash_script)
run_docker(['cp', container_name + ':/output/.', os.path.abspath(output_dir)])
except subprocess.CalledProcessError:
exit(1)
finally:
# Still gets executed, even when 'exit(1)' gets called
run_docker(['rm', '--force', '-v', container_name])
+15 -11
View File
@@ -10,15 +10,22 @@ except ImportError:
from .util import prepare_command, get_build_verbosity_extra_flags
def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, build_verbosity, skip, environment):
def get_python_configurations(build_selector):
PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'identifier', 'url'])
python_configurations = [
PythonConfiguration(version='2.7', identifier='cp27-macosx_10_6_intel', url='https://www.python.org/ftp/python/2.7.15/python-2.7.15-macosx10.6.pkg'),
PythonConfiguration(version='2.7', identifier='cp27-macosx_10_6_intel', url='https://www.python.org/ftp/python/2.7.16/python-2.7.16-macosx10.6.pkg'),
PythonConfiguration(version='3.4', identifier='cp34-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.4.4/python-3.4.4-macosx10.6.pkg'),
PythonConfiguration(version='3.5', identifier='cp35-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.5.4/python-3.5.4-macosx10.6.pkg'),
PythonConfiguration(version='3.6', identifier='cp36-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.6.6/python-3.6.6-macosx10.6.pkg'),
PythonConfiguration(version='3.7', identifier='cp37-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.7.0/python-3.7.0-macosx10.6.pkg'),
PythonConfiguration(version='3.6', identifier='cp36-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.6.8/python-3.6.8-macosx10.6.pkg'),
PythonConfiguration(version='3.7', identifier='cp37-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.7.2/python-3.7.2-macosx10.6.pkg'),
]
# skip builds as required
return [c for c in python_configurations if build_selector(c.identifier)]
def build(project_dir, output_dir, test_command, test_requires, before_build, build_verbosity, build_selector, environment):
python_configurations = get_python_configurations(build_selector)
get_pip_url = 'https://bootstrap.pypa.io/get-pip.py'
get_pip_script = '/tmp/get-pip.py'
@@ -42,10 +49,6 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
call(['curl', '-L', '-o', get_pip_script, get_pip_url])
for config in python_configurations:
if skip(config.identifier):
print('cibuildwheel: Skipping build %s' % config.identifier, file=sys.stderr)
continue
# if this version of python isn't installed, get it from python.org and install
python_package_identifier = 'org.python.Python.PythonFramework-%s' % config.version
if python_package_identifier not in installed_system_packages:
@@ -55,7 +58,7 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
call(['sudo', 'installer', '-pkg', '/tmp/Python.pkg', '-target', '/'])
# patch open ssl
if config.version in ('3.4', '3.5'):
call(['curl', '-fsSLo', '/tmp/python-patch.tar.gz', 'https://github.com/mayeut/patch-macos-python-openssl/releases/download/v0.1.0/patch-macos-python-%s-openssl-v0.1.0.tar.gz' % config.version])
call(['curl', '-fsSLo', '/tmp/python-patch.tar.gz', 'https://github.com/mayeut/patch-macos-python-openssl/releases/download/v1.0.2r/patch-macos-python-%s-openssl-v1.0.2r.tar.gz' % config.version])
call(['sudo', 'tar', '-C', '/Library/Frameworks/Python.framework/Versions/%s/' % config.version, '-xmf', '/tmp/python-patch.tar.gz'])
installation_bin_path = '/Library/Frameworks/Python.framework/Versions/{}/bin'.format(config.version)
@@ -130,5 +133,6 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
test_command_prepared = prepare_command(test_command, project=abs_project_dir)
call(test_command_prepared, cwd=os.environ['HOME'], env=env, shell=True)
# we're all done here; move it to output
shutil.move(delocated_wheel, output_dir)
# we're all done here; move it to output (overwrite existing)
dst = os.path.join(output_dir, os.path.basename(delocated_wheel))
shutil.move(delocated_wheel, dst)
@@ -0,0 +1,88 @@
:: To build extensions for 64 bit Python 3, we need to configure environment
:: variables to use the MSVC 2010 C++ compilers from GRMSDKX_EN_DVD.iso of:
:: MS Windows SDK for Windows 7 and .NET Framework 4 (SDK v7.1)
::
:: To build extensions for 64 bit Python 2, we need to configure environment
:: variables to use the MSVC 2008 C++ compilers from GRMSDKX_EN_DVD.iso of:
:: MS Windows SDK for Windows 7 and .NET Framework 3.5 (SDK v7.0)
::
:: 32 bit builds, and 64-bit builds for 3.5 and beyond, do not require specific
:: environment configurations.
::
:: Note: this script needs to be run with the /E:ON and /V:ON flags for the
:: cmd interpreter, at least for (SDK v7.0)
::
:: More details at:
:: https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows
:: http://stackoverflow.com/a/13751649/163740
::
:: Author: Olivier Grisel
:: License: CC0 1.0 Universal: http://creativecommons.org/publicdomain/zero/1.0/
::
:: Notes about batch files for Python people:
::
:: Quotes in values are literally part of the values:
:: SET FOO="bar"
:: FOO is now five characters long: " b a r "
:: If you don't want quotes, don't include them on the right-hand side.
::
:: The CALL lines at the end of this file look redundant, but if you move them
:: outside of the IF clauses, they do not run properly in the SET_SDK_64==Y
:: case, I don't know why.
@ECHO OFF
SET COMMAND_TO_RUN=%*
SET WIN_SDK_ROOT=C:\Program Files\Microsoft SDKs\Windows
SET WIN_WDK=c:\Program Files (x86)\Windows Kits\10\Include\wdf
:: Extract the major and minor versions, and allow for the minor version to be
:: more than 9. This requires the version number to have two dots in it.
SET MAJOR_PYTHON_VERSION=%PYTHON_VERSION:~0,1%
IF "%PYTHON_VERSION:~3,1%" == "." (
SET MINOR_PYTHON_VERSION=%PYTHON_VERSION:~2,1%
) ELSE (
SET MINOR_PYTHON_VERSION=%PYTHON_VERSION:~2,2%
)
:: Based on the Python version, determine what SDK version to use, and whether
:: to set the SDK for 64-bit.
IF %MAJOR_PYTHON_VERSION% == 2 (
SET WINDOWS_SDK_VERSION="v7.0"
SET SET_SDK_64=Y
) ELSE (
IF %MAJOR_PYTHON_VERSION% == 3 (
SET WINDOWS_SDK_VERSION="v7.1"
IF %MINOR_PYTHON_VERSION% LEQ 4 (
SET SET_SDK_64=Y
) ELSE (
SET SET_SDK_64=N
IF EXIST "%WIN_WDK%" (
:: See: https://connect.microsoft.com/VisualStudio/feedback/details/1610302/
REN "%WIN_WDK%" 0wdf
)
)
) ELSE (
ECHO Unsupported Python version: "%MAJOR_PYTHON_VERSION%"
EXIT 1
)
)
IF %PYTHON_ARCH% == 64 (
IF %SET_SDK_64% == Y (
ECHO Configuring Windows SDK %WINDOWS_SDK_VERSION% for Python %MAJOR_PYTHON_VERSION% on a 64 bit architecture
SET DISTUTILS_USE_SDK=1
SET MSSdk=1
"%WIN_SDK_ROOT%\%WINDOWS_SDK_VERSION%\Setup\WindowsSdkVer.exe" -q -version:%WINDOWS_SDK_VERSION%
"%WIN_SDK_ROOT%\%WINDOWS_SDK_VERSION%\Bin\SetEnv.cmd" /x64 /release
ECHO Executing: %COMMAND_TO_RUN%
call %COMMAND_TO_RUN% || EXIT 1
) ELSE (
ECHO Using default MSVC build environment for 64 bit architecture
ECHO Executing: %COMMAND_TO_RUN%
call %COMMAND_TO_RUN% || EXIT 1
)
) ELSE (
ECHO Using default MSVC build environment for 32 bit architecture
ECHO Executing: %COMMAND_TO_RUN%
call %COMMAND_TO_RUN% || EXIT 1
)
+8 -5
View File
@@ -21,15 +21,18 @@ def get_build_verbosity_extra_flags(level):
return []
class BuildSkipper(object):
def __init__(self, skip_config):
self.patterns = skip_config.split()
class BuildSelector(object):
def __init__(self, build_config, skip_config):
self.build_patterns = build_config.split()
self.skip_patterns = skip_config.split()
def __call__(self, build_id):
return any(fnmatch(build_id, pattern) for pattern in self.patterns)
def match_any(patterns):
return any(fnmatch(build_id, pattern) for pattern in patterns)
return match_any(self.build_patterns) and not match_any(self.skip_patterns)
def __repr__(self):
return 'BuildSkipper(%r)' % ' '.join(self.patterns)
return 'BuildSelector({!r} - {!r})'.format(' '.join(self.build_patterns), ' '.join(self.skip_patterns))
# Taken from https://stackoverflow.com/a/107717
+65 -33
View File
@@ -1,39 +1,42 @@
from __future__ import print_function
import os, tempfile, subprocess, sys, shutil
try:
from urllib2 import urlopen
except ImportError:
from urllib.request import urlopen
import os, tempfile, subprocess, shutil
from collections import namedtuple
from glob import glob
from .util import prepare_command, get_build_verbosity_extra_flags, Unbuffered
from .util import prepare_command, get_build_verbosity_extra_flags
def build(project_dir, package_name, output_dir, test_command, test_requires, before_build, build_verbosity, skip, environment):
# Python under AppVeyor/Windows seems to be buffering by default, giving problems interleaving subprocess call output with unflushed calls to 'print'
sys.stdout.flush()
sys.stdout = Unbuffered(sys.stdout)
IS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache')
# run_with_env is a cmd file that sets the right environment variables to
run_with_env = os.path.join(tempfile.gettempdir(), 'appveyor_run_with_env.cmd')
if not os.path.exists(run_with_env):
with open(run_with_env, 'wb') as f:
request = urlopen('https://github.com/ogrisel/python-appveyor-demo/raw/09a1c8672e5015a74d8f69d07add6ee803c176ec/appveyor/run_with_env.cmd')
f.write(request.read())
def get_python_path(config):
if IS_RUNNING_ON_AZURE:
# We can't hard-code the paths because on Azure, we don't know which
# bugfix release of Python we are getting so we need to check which
# ones exist. We just use the first one that is found since there should
# only be one.
path_pattern = 'C:\\hostedtoolcache\\windows\\Python\\{version}\\{arch}'.format(
version=config.version.replace('x', '*'),
arch='x86' if config.arch == '32' else 'x64'
)
try:
return glob(path_pattern)[0]
except IndexError:
raise Exception('Could not find a Python install at ' + path_pattern)
else:
# Assume we're running on Appveyor
major, minor = config.version.split('.')[:2]
return 'C:\\Python{major}{minor}{arch}'.format(
major=major,
minor=minor,
arch = '-x64' if config.arch == '64' else ''
)
def shell(args, env=None, cwd=None):
# print the command executing for the logs
print('+ ' + ' '.join(args))
args = ['cmd', '/E:ON', '/V:ON', '/C', run_with_env] + args
return subprocess.check_call(' '.join(args), env=env, cwd=cwd)
def get_python_configurations(build_selector):
PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'arch', 'identifier', 'path'])
python_configurations = [
PythonConfiguration(version='2.7.x', arch="32", identifier='cp27-win32', path='C:\Python27'),
PythonConfiguration(version='2.7.x', arch="64", identifier='cp27-win_amd64', path='C:\Python27-x64'),
PythonConfiguration(version='3.3.x', arch="32", identifier='cp33-win32', path='C:\Python33'),
PythonConfiguration(version='3.3.x', arch="64", identifier='cp33-win_amd64', path='C:\Python33-x64'),
PythonConfiguration(version='3.4.x', arch="32", identifier='cp34-win32', path='C:\Python34'),
PythonConfiguration(version='3.4.x', arch="64", identifier='cp34-win_amd64', path='C:\Python34-x64'),
PythonConfiguration(version='3.5.x', arch="32", identifier='cp35-win32', path='C:\Python35'),
@@ -44,18 +47,44 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
PythonConfiguration(version='3.7.x', arch="64", identifier='cp37-win_amd64', path='C:\Python37-x64'),
]
if IS_RUNNING_ON_AZURE:
# Python 3.4 isn't supported on Azure.
# See https://github.com/Microsoft/azure-pipelines-tasks/issues/9674
python_configurations = [c for c in python_configurations if c.version != '3.4.x']
# skip builds as required
return [c for c in python_configurations if build_selector(c.identifier)]
def build(project_dir, output_dir, test_command, test_requires, before_build, build_verbosity, build_selector, environment):
if IS_RUNNING_ON_AZURE:
def shell(args, env=None, cwd=None):
print('+ ' + ' '.join(args))
args = ['cmd', '/E:ON', '/V:ON', '/C'] + args
return subprocess.check_call(' '.join(args), env=env, cwd=cwd)
else:
run_with_env = os.path.abspath(os.path.join(os.path.dirname(__file__), 'resources', 'appveyor_run_with_env.cmd'))
# run_with_env is a cmd file that sets the right environment variables
# to build on Appveyor.
def shell(args, env=None, cwd=None):
# print the command executing for the logs
print('+ ' + ' '.join(args))
args = ['cmd', '/E:ON', '/V:ON', '/C', run_with_env] + args
return subprocess.check_call(' '.join(args), env=env, cwd=cwd)
abs_project_dir = os.path.abspath(project_dir)
temp_dir = tempfile.mkdtemp(prefix='cibuildwheel')
built_wheel_dir = os.path.join(temp_dir, 'built_wheel')
python_configurations = get_python_configurations(build_selector)
for config in python_configurations:
if skip(config.identifier):
print('cibuildwheel: Skipping build %s' % config.identifier, file=sys.stderr)
continue
config_python_path = get_python_path(config)
# check python & pip exist for this configuration
assert os.path.exists(os.path.join(config.path, 'python.exe'))
assert os.path.exists(os.path.join(config.path, 'Scripts', 'pip.exe'))
assert os.path.exists(os.path.join(config_python_path, 'python.exe'))
assert os.path.exists(os.path.join(config_python_path, 'Scripts', 'pip.exe'))
# setup dirs
if os.path.exists(built_wheel_dir):
@@ -67,8 +96,8 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
env['PYTHON_VERSION'] = config.version
env['PYTHON_ARCH'] = config.arch
env['PATH'] = os.pathsep.join([
config.path,
os.path.join(config.path, 'Scripts'),
config_python_path,
os.path.join(config_python_path, 'Scripts'),
env['PATH']
])
env = environment.as_dictionary(prev_environment=env)
@@ -105,5 +134,8 @@ def build(project_dir, package_name, output_dir, test_command, test_requires, be
test_command_prepared = prepare_command(test_command, project=abs_project_dir)
shell([test_command_prepared], cwd='c:\\', env=env)
# we're all done here; move it to output
shutil.move(built_wheel, output_dir)
# we're all done here; move it to output (remove if already exists)
dst = os.path.join(output_dir, os.path.basename(built_wheel))
if os.path.isfile(dst):
os.remove(dst)
shutil.move(built_wheel, dst)
+4 -1
View File
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.9.4
current_version = 0.11.1
commit = True
tag = True
message = Bump version
@@ -12,3 +12,6 @@ message = Bump version
search = cibuildwheel=={current_version}
replace = cibuildwheel=={new_version}
[bdist_wheel]
universal = 1
+5 -2
View File
@@ -8,8 +8,8 @@ except ImportError:
setup(
name='cibuildwheel',
version='0.9.4',
install_requires=['bashlex'],
version='0.11.1',
install_requires=['bashlex!=0.13'],
description="Build Python wheels on CI with minimal configuration.",
long_description='For readme please see http://github.com/joerick/cibuildwheel',
author="Joe Rickerby",
@@ -18,6 +18,9 @@ setup(
packages=['cibuildwheel',],
license="BSD",
zip_safe=False,
package_data={
'cibuildwheel': ['resources/*'],
},
keywords='ci wheel packaging pypi travis appveyor macos linux windows',
classifiers=[
'Intended Audience :: Developers',
+22
View File
@@ -0,0 +1,22 @@
import os
import utils
project_dir = os.path.dirname(__file__)
def test():
# build the wheels
utils.cibuildwheel_run(project_dir)
# check that the expected wheels are produced
expected_wheels = utils.expected_wheels('spam', '0.1.0')
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
def test_build_identifiers():
# check that the number of expected wheels matches the number of build
# identifiers
expected_wheels = utils.expected_wheels('spam', '0.1.0')
build_identifiers = utils.cibuildwheel_get_build_identifiers(project_dir)
assert len(expected_wheels) == len(build_identifiers)
+19
View File
@@ -0,0 +1,19 @@
import os
import utils
def test():
project_dir = os.path.dirname(__file__)
# build and test the wheels
utils.cibuildwheel_run(project_dir, add_env={
'CIBW_TEST_REQUIRES': 'nose',
# the 'false ||' bit is to ensure this command runs in a shell on
# mac/linux.
'CIBW_TEST_COMMAND': 'false || nosetests {project}/test',
'CIBW_TEST_COMMAND_WINDOWS': 'nosetests {project}/test',
})
# also check that we got the right wheels
expected_wheels = utils.expected_wheels('spam', '0.1.0')
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
-6
View File
@@ -1,6 +0,0 @@
{
"CIBW_TEST_REQUIRES": "nose",
"CIBW_TEST_COMMAND": "false || nosetests {project}/test",
"comment": "The 'false ||' bit is to ensure this command runs in a shell on Mac and Linux",
"CIBW_TEST_COMMAND_WINDOWS": "nosetests {project}/test"
}
+18
View File
@@ -0,0 +1,18 @@
import os
import utils
def test():
project_dir = os.path.dirname(__file__)
# build the wheels
utils.cibuildwheel_run(project_dir, add_env={
# write python version information to a temporary file, this is
# checked in setup.py
'CIBW_BEFORE_BUILD': '''python -c "import sys; open('/tmp/pythonversion.txt', 'w').write(sys.version)" && python -c "import sys; open('/tmp/pythonexecutable.txt', 'w').write(sys.executable)"''',
'CIBW_BEFORE_BUILD_WINDOWS': '''python -c "import sys; open('c:\\pythonversion.txt', 'w').write(sys.version)" && python -c "import sys; open('c:\\pythonexecutable.txt', 'w').write(sys.executable)"''',
})
# also check that we got the right wheels
expected_wheels = utils.expected_wheels('spam', '0.1.0')
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
-4
View File
@@ -1,4 +0,0 @@
{
"CIBW_BEFORE_BUILD": "python -c \"import sys; open('/tmp/pythonversion.txt', 'w').write(sys.version)\" && python -c \"import sys; open('/tmp/pythonexecutable.txt', 'w').write(sys.executable)\"",
"CIBW_BEFORE_BUILD_WINDOWS": "python -c \"import sys; open('c:\\pythonversion.txt', 'w').write(sys.version)\" && python -c \"import sys; open('c:\\pythonexecutable.txt', 'w').write(sys.executable)\""
}
+16 -17
View File
@@ -1,24 +1,23 @@
from setuptools import setup, Extension
import sys, os
if sys.argv[-1] != '--name':
# assert that the Python version as written to pythonversion.txt in the CIBW_BEFORE_BUILD step
# is the same one as is currently running.
version_file = 'c:\\pythonversion.txt' if sys.platform == 'win32' else '/tmp/pythonversion.txt'
with open(version_file) as f:
stored_version = f.read()
print('stored_version', stored_version)
print('sys.version', sys.version)
assert stored_version == sys.version
# assert that the Python version as written to pythonversion.txt in the CIBW_BEFORE_BUILD step
# is the same one as is currently running.
version_file = 'c:\\pythonversion.txt' if sys.platform == 'win32' else '/tmp/pythonversion.txt'
with open(version_file) as f:
stored_version = f.read()
print('stored_version', stored_version)
print('sys.version', sys.version)
assert stored_version == sys.version
# check that the executable also was written
executable_file = 'c:\\pythonexecutable.txt' if sys.platform == 'win32' else '/tmp/pythonexecutable.txt'
with open(executable_file) as f:
stored_executable = f.read()
print('stored_executable', stored_executable)
print('sys.executable', sys.executable)
# windows/mac are case insensitive
assert os.path.realpath(stored_executable).lower() == os.path.realpath(sys.executable).lower()
# check that the executable also was written
executable_file = 'c:\\pythonexecutable.txt' if sys.platform == 'win32' else '/tmp/pythonexecutable.txt'
with open(executable_file) as f:
stored_executable = f.read()
print('stored_executable', stored_executable)
print('sys.executable', sys.executable)
# windows/mac are case insensitive
assert os.path.realpath(stored_executable).lower() == os.path.realpath(sys.executable).lower()
setup(
name="spam",
-2
View File
@@ -1,2 +0,0 @@
3.6.0 (default, Feb 7 2017, 23:55:32)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
+17
View File
@@ -0,0 +1,17 @@
import os
import utils
def test():
project_dir = os.path.dirname(__file__)
# build the wheels
utils.cibuildwheel_run(project_dir, add_env={
'CIBW_BUILD': 'cp3?-*',
'CIBW_SKIP': 'cp34-*',
})
# check that we got the right wheels. There should be no 2.7 or 3.4.
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if ('-cp3' in w) and ('-cp34' not in w)]
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
+14
View File
@@ -0,0 +1,14 @@
from setuptools import setup, Extension
import sys
# explode if run on Python 2.7 or Python 3.4 (these should be skipped)
if sys.version_info[0:2] == (2, 7):
raise Exception('Python 2.7 should not be built')
if sys.version_info[0:2] == (3, 4):
raise Exception('Python 3.4 should be skipped')
setup(
name="spam",
ext_modules=[Extension('spam', sources=['spam.c'])],
version="0.1.0",
)
-3
View File
@@ -1,3 +0,0 @@
{
"CIBW_SKIP": "cp26-* cp33-*"
}
-15
View File
@@ -1,15 +0,0 @@
from setuptools import setup, Extension
import sys
if sys.argv[-1] != '--name':
# explode if run on Python 2.6 or Python 3.3 (these should be skipped)
if sys.version_info[0:2] == (2, 6):
raise Exception('Python 2.6 should be skipped')
if sys.version_info[0:2] == (3, 3):
raise Exception('Python 3.3 should be skipped')
setup(
name="spam",
ext_modules=[Extension('spam', sources=['spam.c'])],
version="0.1.0",
)
-2
View File
@@ -1,2 +0,0 @@
3.6.0 (default, Feb 7 2017, 23:55:32)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
+18
View File
@@ -0,0 +1,18 @@
import os
import utils
def test():
project_dir = os.path.dirname(__file__)
# write some information into the CIBW_ENVIRONMENT, for expansion and
# insertion into the environment by cibuildwheel. This is checked
# in setup.py
utils.cibuildwheel_run(project_dir, add_env={
'CIBW_ENVIRONMENT': '''CIBW_TEST_VAR="a b c" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3="$(echo 'test string 3')" PATH=$PATH:/opt/cibw_test_path''',
'CIBW_ENVIRONMENT_WINDOWS': '''CIBW_TEST_VAR="a b c" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3="$(echo 'test string 3')" PATH="$PATH;/opt/cibw_test_path"''',
})
# also check that we got the right wheels built
expected_wheels = utils.expected_wheels('spam', '0.1.0')
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
-4
View File
@@ -1,4 +0,0 @@
{
"CIBW_ENVIRONMENT": "CIBW_TEST_VAR=\"a b c\" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3=\"$(echo 'test string 3')\" PATH=$PATH:/opt/cibw_test_path",
"CIBW_ENVIRONMENT_WINDOWS": "CIBW_TEST_VAR=\"a b c\" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3=\"$(echo 'test string 3')\" PATH=\"$PATH;/opt/cibw_test_path\""
}
+12 -13
View File
@@ -1,20 +1,19 @@
from setuptools import setup, Extension
import sys, os
if sys.argv[-1] != '--name':
# explode if environment isn't correct, as set in CIBW_ENVIRONMENT
CIBW_TEST_VAR = os.environ.get('CIBW_TEST_VAR')
CIBW_TEST_VAR_2 = os.environ.get('CIBW_TEST_VAR_2')
PATH = os.environ.get('PATH')
# explode if environment isn't correct, as set in CIBW_ENVIRONMENT
CIBW_TEST_VAR = os.environ.get('CIBW_TEST_VAR')
CIBW_TEST_VAR_2 = os.environ.get('CIBW_TEST_VAR_2')
PATH = os.environ.get('PATH')
if CIBW_TEST_VAR != 'a b c':
raise Exception('CIBW_TEST_VAR should equal "a b c". It was "%s"' % CIBW_TEST_VAR)
if CIBW_TEST_VAR_2 != '1':
raise Exception('CIBW_TEST_VAR_2 should equal "1". It was "%s"' % CIBW_TEST_VAR_2)
if '/opt/cibw_test_path' not in PATH:
raise Exception('PATH should contain "/opt/cibw_test_path". It was "%s"' % PATH)
if '$PATH' in PATH:
raise Exception('$PATH should be expanded in PATH. It was "%s"' % PATH)
if CIBW_TEST_VAR != 'a b c':
raise Exception('CIBW_TEST_VAR should equal "a b c". It was "%s"' % CIBW_TEST_VAR)
if CIBW_TEST_VAR_2 != '1':
raise Exception('CIBW_TEST_VAR_2 should equal "1". It was "%s"' % CIBW_TEST_VAR_2)
if '/opt/cibw_test_path' not in PATH:
raise Exception('PATH should contain "/opt/cibw_test_path". It was "%s"' % PATH)
if '$PATH' in PATH:
raise Exception('$PATH should be expanded in PATH. It was "%s"' % PATH)
setup(
-2
View File
@@ -1,2 +0,0 @@
3.6.0 (default, Feb 7 2017, 23:55:32)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
@@ -0,0 +1,18 @@
import os, pytest
import utils
def test():
project_dir = os.path.dirname(__file__)
if utils.platform != 'linux':
pytest.skip('the docker test is only relevant to the linux build')
utils.cibuildwheel_run(project_dir, add_env={
'CIBW_MANYLINUX1_X86_64_IMAGE': 'dockcross/manylinux-x64',
'CIBW_MANYLINUX1_I686_IMAGE': 'dockcross/manylinux-x86',
})
# also check that we got the right wheels built
expected_wheels = utils.expected_wheels('spam', '0.1.0')
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
-5
View File
@@ -1,5 +0,0 @@
{
"CIBW_MANYLINUX1_X86_64_IMAGE": "dockcross/manylinux-x64",
"CIBW_MANYLINUX1_I686_IMAGE": "dockcross/manylinux-x86",
"CIBW_SKIP": "cp37-manylinux*"
}
+4 -5
View File
@@ -2,11 +2,10 @@ import os, sys
from setuptools import setup, Extension
if sys.argv[-1] != '--name':
# check that we're running in the correct docker image as specified in the
# environment options CIBW_MANYLINUX1_*_IMAGE
if 'linux' in sys.platform and not os.path.exists('/dockcross'):
raise Exception('/dockcross directory not found. Is this test running in the correct docker image?')
# check that we're running in the correct docker image as specified in the
# environment options CIBW_MANYLINUX1_*_IMAGE
if 'linux' in sys.platform and not os.path.exists('/dockcross'):
raise Exception('/dockcross directory not found. Is this test running in the correct docker image?')
setup(
name="spam",
+9
View File
@@ -0,0 +1,9 @@
import os
import utils
def test():
project_dir = os.path.dirname(__file__)
# this test checks that SSL is working in the build environment using
# some checks in setup.py.
utils.cibuildwheel_run(project_dir)
+106
View File
@@ -0,0 +1,106 @@
'''
Utility functions used by the cibuildwheel tests.
This file is added to the PYTHONPATH in the test runner at bin/run_test.py.
'''
import subprocess, sys, os
IS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache')
def cibuildwheel_get_build_identifiers(project_path, env=None):
'''
Returns the list of build identifiers that cibuildwheel will try to build
for the current platform.
'''
cmd_output = subprocess.check_output(
[sys.executable, '-m', 'cibuildwheel', '--print-build-identifiers', project_path],
universal_newlines=True,
env=env,
)
return cmd_output.strip().split('\n')
def cibuildwheel_run(project_path, env=None, add_env=None):
'''
Runs cibuildwheel as a subprocess, building the project at project_path.
Uses the current Python interpreter.
Configure settings using env.
'''
if env is None:
env = os.environ.copy()
if add_env is not None:
env.update(add_env)
subprocess.check_call(
[sys.executable, '-m', 'cibuildwheel', project_path],
env=env,
)
def expected_wheels(package_name, package_version):
'''
Returns a list of expected wheels from a run of cibuildwheel.
'''
if platform == 'linux':
templates = [
'{package_name}-{package_version}-cp27-cp27m-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp27-cp27mu-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp34-cp34m-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp35-cp35m-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp36-cp36m-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp37-cp37m-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp27-cp27m-manylinux1_i686.whl',
'{package_name}-{package_version}-cp27-cp27mu-manylinux1_i686.whl',
'{package_name}-{package_version}-cp34-cp34m-manylinux1_i686.whl',
'{package_name}-{package_version}-cp35-cp35m-manylinux1_i686.whl',
'{package_name}-{package_version}-cp36-cp36m-manylinux1_i686.whl',
'{package_name}-{package_version}-cp37-cp37m-manylinux1_i686.whl',
]
elif platform == 'windows':
templates = [
'{package_name}-{package_version}-cp27-cp27m-win32.whl',
'{package_name}-{package_version}-cp34-cp34m-win32.whl',
'{package_name}-{package_version}-cp35-cp35m-win32.whl',
'{package_name}-{package_version}-cp36-cp36m-win32.whl',
'{package_name}-{package_version}-cp37-cp37m-win32.whl',
'{package_name}-{package_version}-cp27-cp27m-win_amd64.whl',
'{package_name}-{package_version}-cp34-cp34m-win_amd64.whl',
'{package_name}-{package_version}-cp35-cp35m-win_amd64.whl',
'{package_name}-{package_version}-cp36-cp36m-win_amd64.whl',
'{package_name}-{package_version}-cp37-cp37m-win_amd64.whl',
]
elif platform == 'macos':
templates = [
'{package_name}-{package_version}-cp27-cp27m-macosx_10_6_intel.whl',
'{package_name}-{package_version}-cp34-cp34m-macosx_10_6_intel.whl',
'{package_name}-{package_version}-cp35-cp35m-macosx_10_6_intel.whl',
'{package_name}-{package_version}-cp36-cp36m-macosx_10_6_intel.whl',
'{package_name}-{package_version}-cp37-cp37m-macosx_10_6_intel.whl',
]
else:
raise Exception('unsupported platform')
if IS_RUNNING_ON_AZURE:
# Python 3.4 isn't supported on Azure.
templates = [t for t in templates if '-cp34-' not in t]
return [filename.format(package_name=package_name, package_version=package_version)
for filename in templates]
platform = None
if 'CIBW_PLATFORM' in os.environ:
platform = os.environ['CIBW_PLATFORM']
elif sys.platform.startswith('linux'):
platform = 'linux'
elif sys.platform.startswith('darwin'):
platform = 'macos'
elif sys.platform in ['win32', 'cygwin']:
platform = 'windows'
else:
raise Exception('Unsupported platform')
+61
View File
@@ -0,0 +1,61 @@
from cibuildwheel.util import BuildSelector
def test_build():
build_selector = BuildSelector(build_config="cp3?-* *-manylinux1*", skip_config="")
assert build_selector('cp27-manylinux1_x86_64')
assert build_selector('cp36-manylinux1_x86_64')
assert build_selector('cp37-manylinux1_x86_64')
assert build_selector('cp27-manylinux1_i686')
assert build_selector('cp36-manylinux1_i686')
assert build_selector('cp37-manylinux1_i686')
assert not build_selector('cp27-macosx_10_6_intel')
assert build_selector('cp36-macosx_10_6_intel')
assert build_selector('cp37-macosx_10_6_intel')
assert not build_selector('cp27-win32')
assert build_selector('cp36-win32')
assert build_selector('cp37-win32')
assert not build_selector('cp27-win_amd64')
assert build_selector('cp36-win_amd64')
assert build_selector('cp37-win_amd64')
def test_skip():
build_selector = BuildSelector(build_config="*", skip_config="cp27-* cp3?-manylinux1_i686 cp36-win* *-win32")
assert not build_selector('cp27-manylinux1_x86_64')
assert build_selector('cp36-manylinux1_x86_64')
assert build_selector('cp37-manylinux1_x86_64')
assert not build_selector('cp27-manylinux1_i686')
assert not build_selector('cp36-manylinux1_i686')
assert not build_selector('cp37-manylinux1_i686')
assert not build_selector('cp27-macosx_10_6_intel')
assert build_selector('cp36-macosx_10_6_intel')
assert build_selector('cp37-macosx_10_6_intel')
assert not build_selector('cp27-win32')
assert not build_selector('cp36-win32')
assert not build_selector('cp37-win32')
assert not build_selector('cp27-win_amd64')
assert not build_selector('cp36-win_amd64')
assert build_selector('cp37-win_amd64')
def test_build_and_skip():
build_selector = BuildSelector(build_config="cp36-* cp37-macosx* *-manylinux1*", skip_config="cp27-* cp37-manylinux1_i686")
assert not build_selector('cp27-manylinux1_x86_64')
assert build_selector('cp36-manylinux1_x86_64')
assert build_selector('cp37-manylinux1_x86_64')
assert not build_selector('cp27-manylinux1_i686')
assert build_selector('cp36-manylinux1_i686')
assert not build_selector('cp37-manylinux1_i686')
assert not build_selector('cp27-macosx_10_6_intel')
assert build_selector('cp36-macosx_10_6_intel')
assert build_selector('cp37-macosx_10_6_intel')
assert not build_selector('cp27-win32')
assert build_selector('cp36-win32')
assert not build_selector('cp37-win32')
assert not build_selector('cp27-win_amd64')
assert build_selector('cp36-win_amd64')
assert not build_selector('cp37-win_amd64')