Compare commits

...
366 Commits
Author SHA1 Message Date
Joe Rickerby 783dedad22 Bump version: v1.4.0 2020-05-02 17:55:50 +01:00
Joe Rickerby dd3bec5ba9 Fix #323 2020-05-02 16:57:28 +01:00
Joe Rickerby c9c1ea6fec Update dependencies (#332) 2020-05-02 16:37:38 +01:00
Yannick Jadoul 2e4a29bf1f Updating the description of C++17 support on macOS (#330)
* Updating the description of C++17 support on macOS, adding nuances about runtime library support vs. C++17 compiler/STL features

* Improve phrasing of macOS C++17 docs with suggested edits
2020-05-02 09:58:48 +01:00
Yannick JadoulandJoe Rickerby dadf6a7ac9 PyPy 7.3.1 (#304)
* Testing PyPy 7.3.1rc1/7.3.1rc2

* Try PyPy's fix for macOS

* 7.3.1rc3

* Update PyPy to 7.3.1

* Using latest pypywheels/manylinux2010-pypy_x86_64:2020-04-15-4ce2a6f with PyPy 7.3.1

* Rehost pypy-7.3.1 on Github to guard against bitbucket links going away

The pypy team plan to have a CDN in front of their buildbot downloads
page, but the timeline is unknown. In the meantime, this should be more
reliable than the bitbucket links!

* Updating macOS and Windows download links for PyPy to https://downloads.python.org/pypy/

Co-authored-by: Joe Rickerby <joerick@mac.com>
2020-04-28 19:05:28 +01:00
Matthieu Darbois 9880c4fbb7 Update openssl patch from 1.0.2t to 1.0.2u on macOS (#328) 2020-04-25 16:14:31 +02:00
Matthieu Darbois 2970faf732 Update CPython 3.7 from 3.7.6 to 3.7.7 (#327)
https://docs.python.org/3.7/whatsnew/changelog.html#python-3-7-7-final
2020-04-25 13:07:57 +02:00
Matthieu Darbois a57ef07f72 Update CPython 2.7 from 2.7.17 to 2.7.18 (#326)
change log: https://github.com/python/cpython/blob/v2.7.18/Misc/NEWS.d/2.7.18rc1.rst
2020-04-25 11:16:36 +02:00
Joe Rickerby 0224d72931 Merge pull request #322 from ax3l/doc-fixTravisArm64
Travis-CI Example: Fix aarch64
2020-04-16 07:41:28 +01:00
Axel Huebl 98c4a395e9 Travis-CI Example: Fix aarch64
The `arch:` `aarch64` is called `arm64` on Travis-CI.
https://docs.travis-ci.com/user/multi-cpu-architectures/

Otherwise this falls back to a generic Linux amd64 host.
2020-04-15 18:15:31 -07:00
Joe Rickerby 95678f3d5d Merge pull request #319 from joerick/support-subdirectory-joerick
Provide `package_dir` for setting subdirectory of project [alternative]
2020-04-14 19:12:50 +01:00
Joe Rickerby f7cda33d96 Merge pull request #321 from MaxHalford/master
Update README.md
2020-04-14 19:00:59 +01:00
Joe Rickerby 603f77a0c1 Assert the files were run using capfd 2020-04-12 23:15:49 +01:00
Max Halford a290e00282 Update README.md
Add creme to working examples
2020-04-12 13:13:02 +02:00
Joe Rickerby 7ebe8fef55 Fix tests on windows - shebang don't work so great there! 2020-04-11 09:39:25 +01:00
Joe Rickerby 21e6b99884 Style fixes 2020-04-10 17:44:18 +01:00
Joe Rickerby f4e911028c Add test for building a package at a subpath within a project 2020-04-10 17:42:39 +01:00
Joe Rickerby 3e335044c6 Merge pull request #318 from joerick/appveyor-pty-fix
Try Appveyor config workaround
2020-04-10 17:23:42 +01:00
Joe Rickerby 407a1c0a6e Merge pull request #320 from joerick/reenable-linux-troubleshoot
Add back missing 'troubleshoot' function call
2020-04-10 17:23:21 +01:00
Joe Rickerby fa605d080b Add back missing 'troubleshoot' function call 2020-04-10 14:18:09 +01:00
Joe Rickerby 7b3bcfae34 Remove allow_failures block and add comment 2020-04-10 14:13:42 +01:00
Joe Rickerby 4b5c743a64 And again..? 2020-04-10 13:00:09 +01:00
Joe Rickerby 83b39bcda2 Update docs on the package/project split 2020-04-10 12:59:22 +01:00
Joe Rickerby a1983b94d8 Merge commit '882ea3426a837acfe205cc5de98793c98b6a1bf7' into support-subdirectory-joerick 2020-04-10 12:33:05 +01:00
Joe Rickerby 397393ba91 Use cwd to run test projects, not package_dir 2020-04-10 12:28:05 +01:00
Joe Rickerby e07d8038c6 Remove lingering project_dir references 2020-04-10 12:27:26 +01:00
Joe Rickerby f84388fe26 Changes that let me run the entire the entire linux test suite locally 2020-04-10 12:18:21 +01:00
Joe Rickerby 7e59d8c61c Change how paths are handled, using 'project is working dir' mentality 2020-04-09 16:19:36 +01:00
Joe Rickerby 95bef46832 Update docs for package_dir 2020-04-09 15:51:00 +01:00
Yannick Jadoul 882ea3426a Merge remote-tracking branch 'upstream/master' 2020-04-09 15:20:05 +02:00
Joe Rickerby a0cc4f6347 Fix comma/dot typo 2020-04-09 14:15:56 +01:00
Joe Rickerby d5010b8242 Merge remote-tracking branch 'origin/master' into support-subdirectory 2020-04-09 14:05:09 +01:00
Yannick Jadoul 25f2e8cb46 Fix flake8 2020-04-09 14:37:12 +02:00
Joe Rickerby 7eb0de0dd2 Try another one 2020-04-09 11:39:46 +01:00
Joe Rickerby eff997d12e Merge pull request #311 from Czaki/pack_arguments
Pack build arguments
2020-04-09 10:46:42 +01:00
Joe Rickerby b5175e6159 Try Appveyor config workaround
APPVEYOR_CONSOLE_DISABLE_PTY, as suggested in https://github.com/appveyor/ci/issues/3376#issuecomment-611314299
2020-04-09 10:43:21 +01:00
Yannick Jadoul b8dbedcdbe Remove explicit object base class of DependencyConstraints 2020-04-08 16:43:01 +02:00
Yannick Jadoul 8be6d47eec Merge pull request #313 from YannickJadoul/less-appveyor
Only run unit tests and 01_basic on AppVeyor
2020-04-08 16:24:44 +02:00
Yannick Jadoul b0d99df007 Allow failure for macOS job on AppVeyor 2020-04-08 15:10:48 +02:00
Yannick Jadoul f38eb2644f Run all tests on AppVeyor on master and appveyor-* branches 2020-04-08 13:18:27 +02:00
Grzegorz BokotaandJoe Rickerby ef935187d0 Update BuildOptions description
Co-Authored-By: Joe Rickerby <joerick@mac.com>
2020-04-08 12:54:58 +02:00
Grzegorz Bokota 6113136a1d fix types annotation with optional 2020-04-08 12:54:58 +02:00
Grzegorz Bokota 13a8c549bf rename opt to options 2020-04-08 12:54:58 +02:00
Grzegorz Bokota 8d7d41cd83 python 3.5 fix 2020-04-08 12:54:58 +02:00
Grzegorz Bokota 6554c3cf86 pack build argument in named tuple, introduce typing arguments 2020-04-08 12:54:58 +02:00
Joe Rickerby 3836bb2919 Merge pull request #306 from joerick/min-macosx-deployment-target
Set MACOSX_DEPLOYMENT_TARGET to 10.9, if not set by the user
2020-04-08 11:53:29 +01:00
Joe Rickerby 2345cf20da Merge pull request #314 from joerick/conserve-ci-on-origin-branches
Build PR branches only once (for origin branches) and skip docs changes
2020-04-08 11:40:57 +01:00
Yannick Jadoul 3a3d1a4d2b Only run unit tests and 01_basic on AppVeyor 2020-04-08 10:45:12 +02:00
Yannick Jadoul 49bbbec7f0 Fix flake8 whitespace warnings 2020-04-08 10:24:16 +02:00
Joe Rickerby 2b4bd815ed Fix some whitespace problems with the merge 2020-04-07 18:52:49 +01:00
Joe Rickerby e90e94a18e Merge branch 'master' of github.com:joerick/cibuildwheel into min-macosx-deployment-target 2020-04-07 18:47:22 +01:00
Joe Rickerby bff3e8e37a Merge pull request #307 from Czaki/setup_python_separate
Setup python extracted from build function
2020-04-07 18:24:55 +01:00
Joe Rickerby 1701bc6bde Limit the master-only rule to joerick/cibuildwheel 2020-04-07 18:11:12 +01:00
Joe Rickerby a2eff759e8 Fix github config 2020-04-07 11:54:37 +01:00
Joe Rickerby 1c88697e7c Merge pull request #312 from joerick/refactor-expected-wheels
Refactor expected_wheels test function
2020-04-07 11:50:30 +01:00
Joe Rickerby 97628673d2 Build PR branches only once (for origin branches) and skip docs changes 2020-04-07 11:49:10 +01:00
Grzegorz Bokota 3aa97f09e2 appveyor retries 2020-04-07 12:30:53 +02:00
Joe Rickerby 5a7dc042b6 Add Windows x86 machine values 2020-04-07 10:01:05 +01:00
Grzegorz Bokota a4602c32dd move calculate constraints outside python setup 2020-04-07 00:56:00 +02:00
Joe Rickerby 3018e7a1f8 Refactor expected_wheels function 2020-04-06 23:10:19 +01:00
Joe Rickerby ea6ada0728 Fix update deps script bugs 2020-04-06 14:30:43 +01:00
Joe Rickerby 90634ed872 Fix flake8 exclude list 2020-04-06 11:50:30 +01:00
Joe Rickerby e78a731e2d Fix bug in Docker pin script causing the latest tag to be pinned 2020-04-06 11:47:23 +01:00
Joe Rickerby 8986b2d468 Add dependency update script 2020-04-06 11:45:18 +01:00
Joe Rickerby 626a045770 Merge origin/master into min-macosx-deployment-target 2020-04-05 17:13:53 +01:00
Grzegorz Bokota 8b465bc94a move python setup to separated function 2020-04-05 16:03:35 +02:00
Joe Rickerby 317c0f4baf Merge pull request #256 from joerick/deterministic-builds
Deterministic builds
2020-04-05 12:58:36 +01:00
Joe Rickerby b52edf89ef Tidy up cpp_standards tests, removing 10.9 references as this is default 2020-04-05 12:40:35 +01:00
Joe Rickerby 27b79f8616 Fix tests to always default to 10.9 2020-04-05 12:34:52 +01:00
Joe Rickerby 93df99e43b Remove potentially confusing line in docs 2020-04-05 12:20:07 +01:00
Joe Rickerby 0bdafc5649 Set MACOSX_DEPLOYMENT_TARGET to 10.9, if not set by the user 2020-04-05 12:15:34 +01:00
Joe Rickerby 1eb5fa2311 Merge pull request #302 from henryiii/patch-1
Add GHA example
2020-04-02 22:51:11 +01:00
Henry Schreiner 5367e687f9 Replace most occurrences of nose with pytest 2020-03-31 15:14:27 -04:00
Henry Schreiner abdd1f71d1 Add GHA example 2020-03-31 14:51:32 -04:00
Filip Š ed50cfd20a Fix tests 2020-03-28 16:25:57 +01:00
Joe Rickerby 3fa9935d5b Ensure the correct versions of constraint files are used for each version 2020-03-28 09:52:28 +00:00
Joe Rickerby 13554d0490 Call get-pip.py on Windows with dependency constraint flags 2020-03-24 21:09:07 +00:00
Joe Rickerby 4b6df458fd Docs wording edit 2020-03-23 22:16:47 +00:00
Joe Rickerby de4bf6dafe Add missing version-specific constraint tests 2020-03-23 22:16:47 +00:00
Joe Rickerby ffaad9134b Remove PyPy version from PythonConfiguration 2020-03-18 10:11:09 +00:00
Joe Rickerby 33da3d65c7 Fix CUSTOM_COMPILE_COMMAND and add comment 2020-03-17 20:28:53 +00:00
Joe Rickerby 36a2f00e68 Revert "See stdout during tests to debug circleCI"
This reverts commit dc5dbea4c9.
2020-03-14 18:09:16 +00:00
Joe Rickerby 98dbbaecf4 Fix docker build on CircleCI 2020-03-14 17:27:51 +00:00
Joe Rickerby 7316d69139 Improve logging of docker calls 2020-03-14 17:21:47 +00:00
Joe Rickerby dc5dbea4c9 See stdout during tests to debug circleCI 2020-03-14 17:21:22 +00:00
Joe Rickerby 580b065bb0 Add no_output_timeout to CircleCI tests to prevent them timing out 2020-03-13 17:12:23 +00:00
Joe Rickerby 7090bfa4f1 Stop isn't necessary, docker rm will stop the container 2020-03-13 15:08:37 +00:00
Joe Rickerby 2db9d9a089 The docker container needs a main process 2020-03-13 15:08:10 +00:00
Joe Rickerby 94824f95af Merge remote-tracking branch 'origin/master' into deterministic-builds 2020-03-13 14:20:10 +00:00
Joe Rickerby 4e31c65d09 Workaround docker cp bug
https://github.com/moby/moby/issues/38995
2020-03-13 12:58:23 +00:00
Filip Š bbb77aa488 Update docs 2020-03-13 13:46:51 +01:00
Filip Š 1e08d29cfc Use package_dir to specify path to Python package 2020-03-13 13:45:14 +01:00
Joe Rickerby 40ff745321 Bump version: v1.3.0 2020-03-12 19:53:34 +00:00
Joe Rickerby 8f9a68fe94 Improve setup docs for Github actions 2020-03-12 19:46:58 +00:00
Joe Rickerby 840da16d39 Merge pull request #194 from mayeut/github-workflows
Add support for GitHub Actions
2020-03-12 09:25:31 +00:00
Joe Rickerby e49644edf2 Some docker fixes 2020-03-12 09:00:40 +00:00
Joe Rickerby c4f54c6aa3 Run each build using docker exec, for faster builds 2020-03-11 21:55:53 +00:00
Joe Rickerby 7efec9a0fc Try updating more things 2020-03-11 21:50:26 +00:00
Joe Rickerby b4ed192e88 Run dependency tests on pypy too, and update constraint for 3.6 2020-03-11 20:56:11 +00:00
Joe Rickerby b55e98fb2c Update test-pinned version of pip 2020-03-11 11:51:46 +00:00
Joe Rickerby cc956ea6cd Update version of setuptools used to test constraint file 2020-03-11 08:46:01 +00:00
Joe Rickerby 4fb4e44c23 Fix some errors introduced after the merge 2020-03-10 22:33:50 +00:00
Joe Rickerby 8603deac93 Update constraints 2020-03-10 21:50:20 +00:00
Matthieu Darbois b797430d21 Add support for GitHub Actions
Closes #159
2020-03-10 22:45:29 +01:00
Joe Rickerby 7ab2d599b9 Merge branch 'master' of github.com:joerick/cibuildwheel into deterministic-builds
# Conflicts:
#	cibuildwheel/__main__.py
#	cibuildwheel/linux.py
#	cibuildwheel/macos.py
#	cibuildwheel/windows.py
2020-03-10 21:16:45 +00:00
Yannick Jadoul c22d50427c Merge pull request #297 from YannickJadoul/travis-yml-reference-allow_failures
Use yaml anchors and references to not repeat full job in allow_failures
2020-03-10 01:05:29 +01:00
Yannick Jadoul ce85c8730d Use yaml anchors and references to not repeat full job in allow_failures 2020-03-09 21:03:15 +01:00
Joe Rickerby 34f8427aab Merge pull request #296 from myselfhimself/patch-1
Referencing gmic-py as Working example (for the macosx Github Actions…
2020-03-09 19:21:31 +00:00
Jonathan-David Schröder bb12d3dc31 Referencing gmic-py as Working example (for the macosx Github Actions builds) 2020-03-09 19:10:18 +01:00
Joe Rickerby 77fb97caf3 Fix typo 2020-03-08 18:08:48 +00:00
Joe Rickerby 23870b6780 Update options.md
Docs edit- now Windows does run in a shell too
2020-03-08 18:06:14 +00:00
Joe Rickerby 4caee57c21 Merge pull request #242 from Czaki/before_test
Add CIBW_BEFORE_TEST to customize test environment
2020-03-08 17:55:59 +00:00
Grzegorz Bokota 7e566e46ee uses sys.prefix instead of sys.executable in before_test 2020-03-08 14:25:42 +01:00
Joe Rickerby 02a52791c9 Fix incorrect constraints file on pypy2.7 2020-03-08 13:12:27 +00:00
Joe Rickerby 9ae2ae4ac6 Produce a python-3-only wheel 2020-03-08 13:10:06 +00:00
Joe Rickerby 42489436a2 Remove obsolete bumpversion config 2020-03-08 12:52:22 +00:00
Joe Rickerby 4628c1334e Bump version: v1.2.0 2020-03-08 12:50:23 +00:00
Joe Rickerby 18b4d42f83 Add bump_version.py script 2020-03-08 12:49:35 +00:00
Joe Rickerby dd2c27ab01 Some test fixes related to PyPy 2020-03-07 19:35:17 +00:00
Joe Rickerby ff99cb37f8 Update changelog 2020-03-07 19:09:28 +00:00
Joe Rickerby d0e5781832 Merge pull request #293 from joerick/windows-chain-commands
Allow chaining of commands on Windows
2020-03-07 18:39:04 +00:00
Joe Rickerby fd1cd2442c Merge remote-tracking branch 'origin/master' into deterministic-builds 2020-03-07 18:37:57 +00:00
Joe Rickerby ad1b59febf Remove hashes from constraint files 2020-03-07 18:31:39 +00:00
Joe Rickerby 7dfc173bca Update docs 2020-03-07 18:05:30 +00:00
Joe Rickerby 2f1ec582e9 Add constraints for python3.6 too 2020-03-07 17:53:12 +00:00
Grzegorz Bokota 79cf7e47ae fix test after rebase, fix typos in readme 2020-03-07 18:18:01 +01:00
Joe Rickerby 7c6e69ea28 Add separate constraints for Python3.5
The new virtualenv requires some packages whose latest versions are 3.6+
2020-03-07 17:05:14 +00:00
Joe Rickerby dbc2f06b9a Fix constraint resolution on pypy-2.7 2020-03-07 17:04:28 +00:00
Joe Rickerby 99dc079380 Merge pull request #292 from joerick/s390x-allow-failures
Allow failures on s390x on Travis
2020-03-07 14:55:53 +00:00
Joe Rickerby 04531d9058 Fix unit tests 2020-03-07 14:53:22 +00:00
Joe Rickerby 696067f19e Merge pull request #230 from mayeut/macos-appveyor
Add macOS support on AppVeyor
2020-03-07 14:17:07 +00:00
Joe Rickerby e934c8219e Merge pull request #291 from mayeut/update-cpython3.8
Update CPython 3.8 from 3.8.1 to 3.8.2 on macOS/Windows
2020-03-07 14:16:01 +00:00
Joe Rickerby e06b7696a0 Add missing function call, which causing tests to be skipped
(maybe types are not such a bad idea...)
2020-03-07 14:15:09 +00:00
Joe Rickerby 54fda3f965 Add back the version field to manylinux PythonConfiguration
(needed for constraints)
2020-03-07 14:13:04 +00:00
Joe Rickerby 219266258e Fix missing pypy image name 2020-03-07 14:12:15 +00:00
Joe Rickerby 298df76b8b Consume new config file with pypy and extra architectures 2020-03-07 13:39:57 +00:00
Joe Rickerby 7e583dcab7 Update constraints 2020-03-07 13:24:52 +00:00
Joe Rickerby 9768897764 Add other architectures and pypy to pinned image config.
Also, restructure the file to have platform first, then manylinux version
2020-03-07 13:24:38 +00:00
Joe Rickerby 8de326a67a Another test 2020-03-07 12:22:44 +00:00
Joe Rickerby c33fa2ada0 Add test for boolean ops on windows 2020-03-07 12:21:31 +00:00
Joe Rickerby 6265d87d09 Try removing the explicit cmd call 2020-03-07 12:17:49 +00:00
Grzegorz Bokota baba7706e0 add example with non pip python package 2020-03-07 12:24:15 +01:00
Grzegorz Bokota d613a8c1a6 remove clean files from setup.py, fixes in doc 2020-03-07 12:24:15 +01:00
Grzegorz Bokota 01a7c2d6df reorder before_test 2020-03-07 12:24:15 +01:00
Grzegorz BokotaandJoe Rickerby 6461074ec9 Apply spell suggestions from joerick code review
Co-Authored-By: Joe Rickerby <joerick@mac.com>
2020-03-07 12:24:15 +01:00
Grzegorz Bokota d9e6bbfa4f add documentation 2020-03-07 12:24:15 +01:00
Grzegorz Bokota 4e9316036c add before test variable 2020-03-07 12:24:15 +01:00
Joe Rickerby 93913ef2fa Remove debugging fail on s390x 2020-03-07 11:19:52 +00:00
mayeut 7903ff60f9 Update CPython 3.8 from 3.8.1 to 3.8.2 on macOS/Windows
Changelog: https://docs.python.org/release/3.8.2/whatsnew/changelog.html#python-3-8-2-final
2020-03-07 12:12:08 +01:00
mayeut 028c1dbf10 Add macOS support on AppVeyor 2020-03-07 12:03:00 +01:00
Joe Rickerby ce5fc12b5f Merge pull request #283 from Czaki/test_virtualenv2
change 02_test to check if virtualenv is properly used another version
2020-03-07 10:54:47 +00:00
Joe Rickerby 07fae7e0e5 Try neatening windows config 2020-03-07 10:52:53 +00:00
Joe Rickerby d0692932fb Fix travis deprecation warnings 2020-03-07 10:46:26 +00:00
Joe Rickerby 1238c9d7e5 Try 'minimal' build image 2020-03-07 10:38:27 +00:00
Joe Rickerby 38fef4e1d8 Consistent build names 2020-03-07 10:37:21 +00:00
Joe Rickerby 863829f67b Allow the failure of x390x and update Travis job names 2020-03-07 10:31:51 +00:00
Joe Rickerby 4456d51e03 Merge pull request #290 from joerick/remove-run-with-env
Try removing appveyor_run_with_env.cmd
2020-03-07 10:14:20 +00:00
Joe Rickerby ec42cb6f42 Fix example references to python3 on Windows, where it's only python
Fix #285
2020-03-06 10:31:01 +00:00
Joe Rickerby 850db69eb7 Try removing appveyor_run_with_env.cmd 2020-03-06 10:15:30 +00:00
Joe Rickerby 4e3cef81d7 Fix style errors 2020-03-06 09:08:20 +00:00
Joe Rickerby 83452b75dd Apply suggestions from code review 2020-03-05 18:28:16 +00:00
Grzegorz Bokota be9eae6828 test path_contains 2020-03-03 23:06:27 +01:00
Grzegorz Bokota 281caf3d2e bugfix for pypy 2020-03-02 15:01:02 +01:00
Grzegorz Bokota 6e41e729fe pin virtualenv to <20 on windows
list dir in test 

use bin directory in pypy
2020-03-02 15:01:02 +01:00
Grzegorz Bokota e0c456604d verify if __CIBW_VIRTUALENV_PATH__ is set 2020-03-01 19:49:46 +01:00
Joe Rickerby 1880b6d14d Merge remote-tracking branch 'origin/master' into deterministic-builds 2020-03-01 12:53:07 +00:00
Joe Rickerby 4c7453df6c Neater test implementation based on review 2020-03-01 12:32:41 +00:00
Joe Rickerby 5e16e86355 Fix some comments from review 2020-03-01 11:09:22 +00:00
Grzegorz Bokota 01dd18bbf8 change 02_test to check if virtualenv is properly used 2020-03-01 10:14:32 +01:00
Yannick Jadoul 774af30d71 Merge pull request #274 from YannickJadoul/undo-virtualenv-workaround
Revert "Keeping virtualenv<20 on macOS, until problems get fixed by new release"
2020-02-28 18:51:20 +01:00
Joe Rickerby 958fa1b824 Fix lint errors
Serves me right for editing code in the github editor!!
2020-02-28 16:26:08 +00:00
Joe Rickerby 8d8b91a9f6 Remove commented line of code and move comment 2020-02-28 16:23:38 +00:00
Joe Rickerby bae011950c Add note about changing python after cibuildwheel run
See #266
2020-02-28 11:30:58 +00:00
Joe Rickerby 9275e7431a Merge pull request #279 from joerick/fix-266
Change twine references to be immediately preceded by pip install in examples
2020-02-28 09:31:05 +00:00
Joe Rickerby 16e23410b4 Change twine references to be immediately preceded by the pip install
line

This ensures that the python that installs is the same python that
should execute the module (this sometimes isn't true on macOS because
the python.org installs that cibuildwheel makes take precedence over
the system versions)

Fix #266
2020-02-27 20:30:02 +00:00
Joe Rickerby d32055e8a9 Merge pull request #278 from metabolize/bump-macos-image
Bump MacOS build image on Azure Pipelines
2020-02-27 20:12:55 +00:00
Yannick Jadoul 1199d78d88 Always unset __PYVENV_LAUNCHER__ environment variable to avoid unwanted surprises 2020-02-27 16:56:47 +01:00
Joe Rickerby be72b8b85d Update README.md 2020-02-27 09:12:50 +00:00
Joe Rickerby 12f0eff609 Readme improvements 2020-02-26 22:29:53 +00:00
Joe Rickerby 4bc7e4f40d Use python3 instead of python in examples 2020-02-26 22:10:42 +00:00
Paul Melnikow a68f8b23e5 Bump MacOS build image on Azure Pipelines
The 10.13 image is no longer supported and will be removed on 3/23.

Closes #276
2020-02-26 16:47:26 -05:00
Joe Rickerby 6ad0aebaf8 Merge pull request #273 from mtreinish/non-x86-support
Add support for manylinux wheels on non-x86 platforms
2020-02-26 20:43:37 +00:00
Yannick Jadoul 7d0cc37b17 Revert "Keeping virtualenv<20 on macOS, until problems get fixed by new release"
This reverts commit 691174cf11.
2020-02-25 13:36:21 +01:00
Matthew Treinish 05df9bbfd4 Add comment and update test_06 skip message 2020-02-21 09:32:09 -05:00
Matthew TreinishandJoe Rickerby f8d2982166 Update CI.md
Co-Authored-By: Joe Rickerby <joerick@mac.com>
2020-02-21 09:27:47 -05:00
Matthew Treinish 36e0618992 Update options.md 2020-02-21 06:48:10 -05:00
Matthew Treinish 549fc8a93e Extract platform check to separate function 2020-02-21 06:43:56 -05:00
Matthew Treinish 5549eb3165 Update examples and CI.md 2020-02-21 06:37:45 -05:00
Matthew Treinish 7b906fb5dc Update README table 2020-02-20 17:43:38 -05:00
Matthew Treinish 90da096b14 More test updates 2020-02-20 17:29:28 -05:00
Matthew Treinish 43c45b6011 Update tests again 2020-02-20 16:24:52 -05:00
Matthew Treinish 6ddac481c0 Fix typo 2020-02-20 16:18:51 -05:00
Matthew Treinish a83ac06c8d Fix small oversights 2020-02-20 16:17:51 -05:00
Matthew Treinish ec4ea4e011 Adjust tests when running on non-x86 2020-02-20 16:12:54 -05:00
Matthew Treinish ec5670a476 Use platform instead of distutils 2020-02-20 16:12:29 -05:00
Matthew Treinish ce76b74251 Add support for manylinux wheels on non-x86 platforms
This commit adds config options and identifiers for building manylinux
wheels on non-x86 platforms. Based on what travis and manylinux2014
support this includes aarch64 (ARMv8), ppce64le (power pc 64bit little
endian), and the s390x platforms.

Fixes #269
2020-02-20 13:57:00 -05:00
Yannick Jadoul 65792bd700 Merge pull request #264 from YannickJadoul/check-python-path
Checking python in PATH is the one `cibuildwheel` installed/expects
2020-02-20 12:10:37 +01:00
Yannick Jadoul d14427d781 Adding more informative error messages 2020-02-20 00:17:06 +01:00
Yannick Jadoul 2ce304d5d9 Add test checking cibuildwheel detects when PATH is messed up by CIBW_ENVIRONMENT 2020-02-20 00:16:09 +01:00
Yannick Jadoul 42a231f830 Making sure that python and pip are the ones we expect 2020-02-20 00:16:03 +01:00
Yannick Jadoul 4f47ab8052 Merge pull request #185 from YannickJadoul/pypy
Adding PyPy support
2020-02-20 00:06:04 +01:00
Yannick Jadoul cf73c3355e Addressing a few more minor remarks of @joerick 2020-02-19 17:10:17 +01:00
Grzegorz Bokota 0d2bec3cb8 Make sure PyPy is only patched once, and use check_call to execute 'patch' 2020-02-19 17:10:17 +01:00
Yannick Jadoul 60dd3f0807 Fixing paths of PyPy on manylinux after PyPy's manylinux image's update 2020-02-19 17:10:17 +01:00
Yannick Jadoul 8c6abf3811 Pushing creation of symlinks into install_cpython and install_pypy 2020-02-19 17:10:17 +01:00
Yannick Jadoul fc0dbe15d5 Fixing flake8 errors and warnings after rebasing 2020-02-19 17:10:17 +01:00
Yannick Jadoul 0423d79e63 Dropping PyPy version from PyPy build identifiers 2020-02-19 17:10:17 +01:00
Yannick Jadoul fd0a789988 Fix minor review comment from @Czaki and @joerick 2020-02-19 17:10:13 +01:00
Joe Rickerby 8a450b7655 Fix expected wheel test logic 2020-02-18 22:19:13 +00:00
Joe Rickerby 5187633e26 Add the --no-download flag to virtualenv, to prevent it getting the latest
version of pip, setuptools, wheel from PyPI on creation.

Virtualenv will use bundled version of those tools instead
2020-02-18 22:03:14 +00:00
Joe Rickerby fc20273f8b Remove the unit test check - the versions are determined by virtualenv at creation time 2020-02-18 21:55:03 +00:00
Joe Rickerby f55a5d44a3 Fix typo and add debug logging 2020-02-18 21:03:00 +00:00
Joe Rickerby fd69aed369 Remove dead code 2020-02-18 20:45:13 +00:00
Joe Rickerby ddc4fb2ab8 Add tests for pinned software in setup.py of test project 2020-02-18 20:44:52 +00:00
Joe Rickerby 3c20f70c5d Add test for user-specified constraints file 2020-02-18 20:38:14 +00:00
Joe Rickerby b7fc45afd9 Fix expected wheels in test 2020-02-18 20:21:08 +00:00
Joe Rickerby c9e4a66915 Add a test case for pinned tool versions 2020-02-18 09:33:07 +00:00
Joe Rickerby fe3abfba92 Docs updates 2020-02-18 09:32:02 +00:00
Yannick Jadoul 1e317895fe Adding PyPy images to main_options_test::test_manylinux_images 2020-02-15 23:47:07 +01:00
Yannick Jadoul b743b8ae5b Fixing Windows tests by skipping pp27 in 10_cpp_standards tests (and pp36 for C++17) on Windows 2020-02-15 23:47:03 +01:00
Grzegorz Bokota 176f4aca1f Apply PyPy patch with default MACOSX_DEPLOYMENT_TARGET to local installation 2020-02-15 23:24:51 +01:00
Yannick Jadoul 60f198c90d Changing the expected wheel filenames to the new pip>=20 tags 2020-02-15 23:24:51 +01:00
Yannick Jadoul 11423a47f6 Reverting PyPy manylinux docker image back to the official version, now that it supports 7.3.0 2020-02-15 23:24:51 +01:00
Yannick Jadoul aac5db8a46 Cleaning up Python installations on Windows and macOS, and creating PyPy executable python symlinks outside of installation directory 2020-02-15 23:24:51 +01:00
Yannick Jadoul 17137c3c2f Changing *-manylinux_pypy_x86_64 build identifiers to *-manylinux_x86_64 2020-02-15 23:24:51 +01:00
Yannick Jadoul bafb4fd294 Adding PyPy to the README and docs 2020-02-15 23:24:51 +01:00
Yannick Jadoul fea54e25ab PyPy 7.3.0 released! 2020-02-15 23:24:51 +01:00
Yannick Jadoul 4c718706c8 Trying out PyPy 7.3.0rc3 on manylinux2010 as well, with a temporary newer version of manylinux2010-pypy_x86_64 image 2020-02-15 23:24:51 +01:00
Yannick Jadoul 08d3ee1d91 Updating PyPy to v7.3.0rc3 on macOS and Windows 2020-02-15 23:24:51 +01:00
Yannick Jadoul e80f7f4774 Trying out PyPy 7.3.0rc1 on macOS and Windows 2020-02-15 23:24:51 +01:00
Yannick Jadoul 186e09d543 Implementing @mayeut's suggestion for alternative way of generating CPython/PyPy manylinux and Windows platform tags in test/shared/utils.py 2020-02-15 23:24:51 +01:00
Yannick Jadoul ae6832e18d Building PyPy wheels on macOS 2020-02-15 23:24:51 +01:00
Yannick Jadoul 743051a453 Using C:\cibw as base folder for PyPy installation (cfr. #197) 2020-02-15 23:24:51 +01:00
Yannick Jadoul 129cf601da Building PyPy wheels on Windows 2020-02-15 23:24:51 +01:00
Yannick Jadoul dd26ef8ac6 Updated PyPy from 7.1.1 to 7.2.0 2020-02-15 23:24:51 +01:00
Yannick Jadoul ef80beb433 Skipping PyPy for 06_docker_images test, and manylinux1 and manylinux2014 runs of 08_manylinuxXXXX_only test 2020-02-15 23:24:51 +01:00
Yannick Jadoul 100fc14411 Adding PyPy builds to Linux, using the manylinux2010-pypy_x86_64 image 2020-02-15 23:24:51 +01:00
Joe Rickerby bf87df5d62 Ensure subprocess.run commands succeed with check=True 2020-02-15 12:46:55 +00:00
Joe Rickerby 4c076b8d41 Fix piping script into docker bash 2020-02-15 12:32:33 +00:00
Joe Rickerby 50bd6f41da Merge branch 'master' into deterministic-builds 2020-02-15 12:22:34 +00:00
Joe Rickerby cf8fa693b8 Change flake8 config to make it easier to run locally 2020-02-15 12:15:44 +00:00
Joe Rickerby f2c2a06c19 Merge fixes and code style 2020-02-15 11:57:31 +00:00
Joe Rickerby 103381774c Merge remote-tracking branch 'origin/master' into deterministic-builds 2020-02-15 11:45:54 +00:00
Yannick Jadoul d9948da29a Merge pull request #212 from nsoranzo/flake8
Fix all PEP-008 issues
2020-02-14 18:14:36 +01:00
Nicola Soranzo c738481e82 Rename main_util_fixtures.py to conftest.py to allow fixture auto discovery
Also fix new PEP-008 issues.
2020-02-14 15:50:06 +00:00
Nicola Soranzo c15d656726 Add a CircleCI job for flake8 2020-02-14 15:50:06 +00:00
Nicola Soranzo cab6ecef24 Fix all PEP-008 issues 2020-02-14 15:50:06 +00:00
Yannick Jadoul 1e420a9b57 Merge pull request #270 from YannickJadoul/debug-virtualenv
Fixing macOS CI and tests after lastest release of virtualenv
2020-02-14 14:06:17 +01:00
Yannick Jadoul 691174cf11 Keeping virtualenv<20 on macOS, until problems get fixed by new release 2020-02-14 13:32:49 +01:00
Yannick Jadoul 0b0830fbbb Merge pull request #265 from YannickJadoul/drop-2.7
Dropping Python 2.7 host support
2020-02-04 23:35:19 +01:00
Yannick Jadoul 165e51aec5 Remove 'from __future__ import print_function' lines 2020-02-04 11:58:52 +01:00
Grzegorz Bokota 48055273f1 Remove inheritance from object base 2020-02-04 11:58:52 +01:00
Yannick Jadoul e8d121552c Remove a few more Python 2-compatibility workarounds 2020-02-04 11:58:52 +01:00
Yannick Jadoul 67ec8fe99f Removing Python 2-compatible imports of urllib.request.open and shlex.quote 2020-02-04 11:58:52 +01:00
Yannick Jadoul b900b2d51e Ditching run_docker for subprocess.run 2020-02-04 11:58:52 +01:00
Yannick Jadoul 9fe8d29545 Dropping Python 2 from setup.py 2020-02-04 11:58:52 +01:00
Yannick Jadoul 692c64ffd9 Removing Python 2 configurations for CI runs 2020-02-04 11:58:50 +01:00
Joe Rickerby ada46aaf43 Fix exports / version check ordering 2020-02-03 20:50:11 +00:00
Joe Rickerby 67022e620e Merge pull request #263 from joerick/cython-error-reporting
Raise a warning if .so files were found in a project that fails to build
2020-02-03 20:33:54 +00:00
Joe Rickerby 17a019b450 Neaten comment 2020-02-03 19:01:17 +00:00
Joe Rickerby bf65679cc3 Fix imports on Python 2 2020-02-03 18:59:32 +00:00
Joe Rickerby a953c5f4ee Remove auditwheel from pinned deps - it's preinstalled on manylinux
auditwheel's pinning of an old version of 'wheel' is causing problems on
mac
2020-02-03 09:02:17 +00:00
Joe Rickerby d8a245dcf6 Use abspath for constraints file 2020-02-02 23:05:58 +00:00
Joe Rickerby e32d0af532 Allow any version of the docker image in the unit test 2020-02-02 22:38:09 +00:00
Yannick Jadoul ffec9aecf8 Update line in docs/cpp_standards.md on setting MACOSX_DEPLOYMENT_TARGET >= 10.9 (see #220) 2020-02-02 20:07:22 +01:00
Joe Rickerby 6177408076 Consume our pinned manylinux image version file for the defaults 2020-02-02 18:09:29 +00:00
Joe Rickerby baa0897f1d Add unit test case and fix filename confusion 2020-02-02 17:46:12 +00:00
Joe Rickerby b9607ffae7 Add ability to pin different dependencies for each python version 2020-02-02 17:30:27 +00:00
Joe Rickerby 076987cb7e Move the config flow control of the linux build into Python
The use of separate bash processes for each build means we can be a bit
freer with environment pollution
2020-02-02 16:03:33 +00:00
Joe Rickerby bad72cbb15 Merge remote-tracking branch 'origin/master' into deterministic-builds 2020-02-02 15:21:42 +00:00
Joe Rickerby 17a224c525 Ignore a python2 virtualenv too 2020-02-02 15:16:36 +00:00
Joe Rickerby 4bc851ddf2 Add an editorconfig to standardise styles across projects
See https://editorconfig.org/
2020-02-02 15:16:23 +00:00
Joe Rickerby 0124d174be Raise a warning if .so files were found in a project that fails to build
Fix #139
2020-02-02 14:33:36 +00:00
Joe Rickerby 8236877f02 Merge pull request #220 from mayeut/macos-platform
Only build x86_64 / Intel 10.9+ on macOS
2020-02-01 23:07:47 +00:00
Joe Rickerby c3f628d03d Add migration for build identifier change, and test for migration 2020-02-01 21:44:07 +00:00
Joe Rickerby 8ead985cfd Generate separate python2 constraint files 2020-01-31 20:27:29 +00:00
Joe Rickerby 1b9b6e9000 Add the ability to pin docker image tags (this isn't consumed yet) 2020-01-31 20:00:30 +00:00
Joe Rickerby 042efd6090 Update constraints 2020-01-31 19:59:57 +00:00
Joe Rickerby b159617a3e Vendor get-pip (version 19.3.1) 2020-01-31 19:08:19 +00:00
Joe Rickerby a97b6a13f1 Fix test protection reference 2020-01-31 17:11:56 +00:00
Joe Rickerby a143ff2bfd Use the internal download() function rather than curl 2020-01-31 17:06:58 +00:00
Joe Rickerby 1a7d1d8cb1 Add MACOSX_DEPLOYMENT_TARGET flag and don't overwrite user-set env vars 2020-01-31 17:05:32 +00:00
Joe Rickerby 68d688c016 Merge pull request #257 from YannickJadoul/newer-dockcross-images
Clean up temporary workaround in test/06_docker_images/cibuildwheel_test.py
2020-01-31 15:50:42 +00:00
Yannick Jadoul 88029d2a28 Clean up temporary workaround in test/06_docker_images/cibuildwheel_test.py 2020-01-30 17:20:14 +01:00
mayeut 513ef49cd4 Only build x86_64 / Intel 10.9+ on macOS 2020-01-29 22:59:01 +01:00
Joe Rickerby c0967ad622 Fix docker copy 2020-01-28 22:26:34 +00:00
Joe Rickerby 1cae983592 Add docs for CIBW_DEPENDENCY_VERSIONS 2020-01-28 22:18:37 +00:00
Joe Rickerby 06696c46a0 Pin tool dependencies whenever we pip install using a constraints file 2020-01-28 22:04:37 +00:00
Joe Rickerby b779a88fde Add note about 254 {vmImage: 'vs2017-win2016'} on Azure to docs
See #254
2020-01-28 21:28:22 +00:00
Joe Rickerby 379cf2dc13 Merge pull request #156 from Czaki/master
upgrade macos version for support c++11
2020-01-27 18:53:32 +00:00
Grzegorz Bokota b64b106fbd use pypi wheel 2020-01-27 09:36:41 +01:00
Joe Rickerby 973ecf53b3 Merge pull request #249 from joerick/pypi-token-examples
Switch examples to use PyPI API tokens
2020-01-19 01:07:33 +00:00
Joe Rickerby 186599a1f0 Switch examples to use PyPI API tokens 2020-01-18 11:15:40 +00:00
Yannick Jadoul aca7611fc4 Merge pull request #248 from YannickJadoul/travis-ci-if-tagged
Remove check for branch of tags in examples/travis-ci-test-and-deploy.yml; fix #246
2020-01-17 11:00:00 +01:00
Yannick Jadoul 9d44d9edf0 Remove check for branch of tags in examples/travis-ci-test-and-deploy.yml; fix #246 2020-01-16 17:17:20 +01:00
Grzegorz Bokota 2473c405bc again change pinned version of wheel 2020-01-16 16:10:05 +01:00
Grzegorz Bokota 984df80ec9 change pinned version of wheel 2020-01-16 16:10:05 +01:00
Yannick Jadoul 0d5a2a7843 Moving MACOSX_DEPLOYMENT_TARGET from test utils' expected_wheels to run_cibuildwheel, and some minor style fixes 2020-01-16 16:10:05 +01:00
Grzegorz Bokota 1f784a1a29 fix conflicts with main branch 2020-01-16 16:10:05 +01:00
Yannick Jadoul e64916fe9c Rewording a few things in cpp_standards.md, and some other minor fixes 2020-01-16 16:10:05 +01:00
Grzegorz Bokota 9cb796edcd update docs 2020-01-16 16:10:05 +01:00
Grzegorz Bokota 758f158bbd Update information about need to use modern standards in c++
add test for c++ standards for all platform change macos platform tag on more generic.
2020-01-16 16:10:05 +01:00
Joe Rickerby d17517902e Merge pull request #241 from Czaki/fix_docs_link
fix link for CIBW_BUILD_VERBOSITY in README.md
2020-01-07 08:50:51 +00:00
Grzegorz Bokota 200d4e3b8c fix link for CIBW_BUILD_VERBOSITY in README.md 2020-01-07 09:04:58 +01:00
Yannick Jadoul 9648b44bdf Merge pull request #232 from Czaki/improve_platform_tag_tests
Test main function with unit tests
2019-12-30 20:12:38 +01:00
Yannick Jadoul 492b5f4039 Splitting off fake_project_dir fixture from mock_protection, only mocking os.path.exist for setup.py in the mocked project directory 2019-12-27 00:56:00 +01:00
Matthieu Darbois 651f6a9172 Merge pull request #238 from mayeut/update-python
Update CPython on macOS / Windows
2019-12-26 16:07:19 +01:00
Yannick Jadoul fabb82e9cf Add manylinux2014 to unit_test/main_options_test.py::test_manylinux_images 2019-12-26 15:03:13 +01:00
Yannick Jadoul 8bfe9ac6cd Fix @joerick's remarks in unit_test/main_options_test.py 2019-12-26 14:55:08 +01:00
Yannick Jadoul de55153931 Remove 'from cibuildwheel import linux, macos, windows' from new tests, as imports are done in fixtures 2019-12-26 14:55:08 +01:00
Yannick Jadoul a9c968868a Fix main_platform_test.py::test_platform_argument, making sure --platform overrides CIBW_PLATFORM 2019-12-26 14:55:08 +01:00
Yannick Jadoul 05e7858815 Refactoring main_function_test.py 2019-12-26 14:55:08 +01:00
Grzegorz Bokota 5e33011229 remove apply_mock_protection by making it fixture 2019-12-26 14:55:08 +01:00
Grzegorz Bokota 0b707b22ed split test function on parts to be more readable. 2019-12-26 14:55:08 +01:00
Grzegorz Bokota d66e201c3f test main function with unit tests 2019-12-26 14:55:08 +01:00
mayeut 6732812ad7 Update CPython on macOS / Windows
- Update CPython 3.7 from 3.7.5 to 3.7.6
- Update CPython 3.8 from 3.8.0 to 3.8.1
2019-12-26 14:26:58 +01:00
mayeut cbb7eafd1f Remove extras spaces 2019-12-26 14:25:33 +01:00
Joe Rickerby be721153af Merge pull request #231 from YannickJadoul/docs-include-examples
Including CI configurations from examples into docs
2019-12-26 11:57:52 +00:00
Joe Rickerby ef67ddd351 Merge pull request #237 from Czaki/repeat_get_pip_download
Repeat get-pip download on fail
2019-12-26 11:23:09 +00:00
Grzegorz BokotaandMatthieu Darbois 0af635fb86 Update cibuildwheel/macos.py
Co-Authored-By: Matthieu Darbois <mayeut@users.noreply.github.com>
2019-12-18 23:22:19 +01:00
Grzegorz Bokota 32182f3edc re raise last exception in windows 2019-12-17 23:23:57 +01:00
Grzegorz BokotaandMatthieu Darbois ebcda4d049 Update cibuildwheel/macos.py
Co-Authored-By: Matthieu Darbois <mayeut@users.noreply.github.com>
2019-12-17 22:40:00 +01:00
Grzegorz Bokota dd9b665c1d retry on macos with cli arguments 2019-12-17 21:34:26 +01:00
Grzegorz Bokota e00f03047a reduce repetition in download retry 2019-12-17 11:53:57 +01:00
Grzegorz Bokota e0b385a520 Move windows repetition download to download function. 2019-12-17 10:55:16 +01:00
Grzegorz Bokota a1832ae6c6 Repeat get-pip download on fail 2019-12-17 01:09:06 +01:00
Yannick Jadoul 795413e5c0 Fix typo in docs 2019-12-15 13:55:44 +01:00
Yannick Jadoul eda3dc1a4f Moving Azure, AppVeyor, and CircleCI minimal examples in docs also to their own file under examples/ 2019-12-10 14:57:25 +01:00
Yannick Jadoul 472fa32d9f Added {% include ... %} directive for including examples/travis-ci-minimal.yml in docs/setup.md 2019-12-08 23:51:09 +01:00
Yannick Jadoul 2f41683e85 Including example Travis CI configuration from examples/ directory in docs/setup.md 2019-12-08 23:50:47 +01:00
Joe Rickerby 7709989603 Bump version 2019-12-07 14:58:10 +00:00
Joe Rickerby 31e67c4f1f Update changelog 2019-12-07 14:58:00 +00:00
Joe Rickerby 11bde36627 Merge pull request #234 from Czaki/fix_setup_cfg
change directory to not conflict with setup.cfg of package
2019-12-07 14:35:38 +00:00
Grzegorz Bokota 5941ba7daf add tests 2019-12-05 21:13:11 +01:00
Grzegorz Bokota c1df977e11 change directory to not conflict with setup.cfg of package 2019-12-03 22:07:47 +01:00
Matthieu Darbois 1dbb12daaa Add manylinux2014 x86_64/i686 (#215) 2019-12-02 11:07:56 +01:00
Joe Rickerby a005e263fc Merge pull request #233 from Czaki/fix_doc_environment
Add example of usage more than one environment variable
2019-12-02 08:54:26 +00:00
Grzegorz Bokota 2a1081c5e5 update docs with information about more than one environment variable. 2019-12-01 21:45:08 +01:00
Joe Rickerby 9c908daf27 Merge pull request #226 from Czaki/fix_platform_bug
Add better protection to unittest to be shure that it will create no changes on developer machine
2019-11-29 09:34:52 +00:00
Joe Rickerby 054be2eff2 Merge pull request #228 from SethMMorton/update-travis-examples
Update travis examples
2019-11-28 22:45:09 +00:00
Seth Morton 20ba988b72 Add an example folder with Travis-CI examples
I have placed two example Travis-CI configurations in an examples/
folder. The first is the same as in the README, and the second shows how
to both test and deploy in two stages.

I am hoping that examples might be added for other CI providers as well,
so that users can see different approaches to building and deploying
wheels.
2019-11-25 18:59:07 -08:00
Seth Morton 190b013c61 Update Travis CI examples
For a more granular view of why each of these changes were made, please
see https://github.com/joerick/cibuildwheel-autopypi-example/pull/5.

High-level overview:
 - Adding Windows support
 - Remove "sudo: required" for linux
 - Use "python -m pip" instead of using an environment variable
 - Add "install:" and "after_success:" steps so that errors get handled
   and reported properly
 - Rename "matrix:" to "jobs:" as this is the now preferred keyword
2019-11-25 15:08:18 -08:00
Grzegorz Bokota c966c6d098 Add mock for better protection of unit tests launch.
change behavior of main function to always end with exit.
2019-11-24 12:17:08 +01:00
Grzegorz Bokota d2700bc3b4 fix UnboundLocalError instead of proper error message (#221)
* fix UnboundLocalError: local variable 'platform' referenced before assignment
* add test to protect against regression
2019-11-21 22:45:33 +01:00
Matthieu Darbois 09e72800c6 Run each test using a temporary directory (#217) 2019-11-20 22:20:53 +01:00
Joe Rickerby 2b62eef9e7 Merge pull request #211 from nsoranzo/delocate_command
Add `CIBW_REPAIR_WHEEL_COMMAND` env variable
2019-11-20 19:58:19 +00:00
Nicola Soranzo 85cba9b1eb Move Platform-specific variants before Examples 2019-11-17 19:08:22 +00:00
Nicola Soranzo 402ec9e4ac Address one more review comment 2019-11-15 02:56:30 +00:00
Nicola Soranzo e9fd9eaead Address @joerick's comments 2019-11-15 02:27:39 +00:00
Nicola Soranzo 0e04fafd3a Add CIBW_REPAIR_COMMAND env variable
to allow different options for auditwheel/delocate, alternative
commands and a future Windows equivalent.

Fix https://github.com/joerick/cibuildwheel/issues/191 .
2019-11-14 12:10:55 +00:00
Yannick Jadoul 34515b78c6 Merge pull request #209 from YannickJadoul/silence-script-not-on-path-warnings
Avoid script location warnings on Windows
2019-11-14 11:59:40 +01:00
Yannick Jadoul 7d99ccfc7b Collapse multiple 'pip install's into one in windows.py and macos.py, and replace full path by 'python' in windows.py 2019-11-12 23:47:06 +01:00
Yannick Jadoul dc01231301 Replace a few shell invocations by simple_shell in windows.py 2019-11-12 22:25:54 +01:00
Matthieu Darbois 96a2505e78 Merge pull request #207 from mayeut/doc-appveyor-linux
Update doc for AppVeyor on linux
2019-11-12 22:11:27 +01:00
Yannick Jadoul d3fd087166 Run python get-pip.py with Scripts directory in PATH to avoid warnings, and reorder steps to match macOS 2019-11-12 17:05:37 +01:00
mayeut b3f82d52cc Update doc for AppVeyor on linux 2019-11-12 01:20:06 +01:00
Matthieu Darbois 7d90a2f16b Merge pull request #204 from mayeut/appveyor-linux
Rework platform detection to ignore CI provider to determine windows/linux/macos.
A check is still present to check if CI is running or not.
2019-11-11 22:24:52 +01:00
mayeut f4318e15b7 Keep CI check for platform auto 2019-11-11 21:49:16 +01:00
mayeut d3450e84ac Rework platform detection to ignore CI provider.
CI provider is now mostly irrelevant, rework platform detection with that in mind.
Run linux test in AppVeyor
2019-11-10 23:40:20 +01:00
Joe Rickerby a3e9b7b8ec Embed readme in project for PyPI 2019-11-10 19:13:49 +00:00
Joe Rickerby 1a9c6a5f66 Add azure pipelines status badge 2019-11-10 19:02:06 +00:00
Joe Rickerby c44a084a44 Really fix link 2019-11-10 19:01:58 +00:00
Joe Rickerby 1e102b9bc5 Fix link 2019-11-10 15:54:17 +00:00
Joe Rickerby 46f1906e2d Add docs badge 2019-11-10 15:52:01 +00:00
Joe Rickerby 79c480b896 Update changelog 2019-11-10 15:47:33 +00:00
Joe Rickerby 734ca6bfbd Remove @tgarc from maintainers 2019-11-10 15:45:34 +00:00
94 changed files with 25932 additions and 997 deletions
+25 -25
View File
@@ -1,11 +1,24 @@
version: 2
jobs:
osx-python2:
flake8:
docker:
- image: circleci/python:3.6
steps:
- checkout
- run:
name: Install flake8
command: sudo python -m pip install flake8
- run:
name: Test.
command: flake8 .
osx-python3.6:
macos:
xcode: "10.0.0"
xcode: "9.4.1"
environment:
PYTHON: python2
PYTHON: python3
steps:
- checkout
@@ -15,8 +28,9 @@ jobs:
- run:
name: Test.
command: venv/bin/python ./bin/run_tests.py
no_output_timeout: 30m
osx-python3:
osx-python3.7:
macos:
xcode: "10.0.0"
environment:
@@ -30,24 +44,9 @@ jobs:
- run:
name: Test.
command: venv/bin/python ./bin/run_tests.py
no_output_timeout: 30m
linux-python2:
docker:
- image: circleci/python:2.7
environment:
PYTHON: python2
steps:
- checkout
- setup_remote_docker
- run:
name: Prepare the environment.
command: bash .circleci/prepare.sh
- run:
name: Test.
command: venv/bin/python ./bin/run_tests.py
linux-python3:
linux-python3.6:
docker:
- image: circleci/python:3.6
environment:
@@ -62,12 +61,13 @@ jobs:
- run:
name: Test.
command: venv/bin/python ./bin/run_tests.py
no_output_timeout: 30m
workflows:
version: 2
all-tests:
jobs:
- osx-python2
- osx-python3
- linux-python2
- linux-python3
- flake8
- osx-python3.6
- osx-python3.7
- linux-python3.6
+14
View File
@@ -0,0 +1,14 @@
# http://editorconfig.org
root = true
[*]
indent_style = space
indent_size = 2
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
# Use 4 spaces the Python files
[*.py]
indent_size = 4
+36
View File
@@ -0,0 +1,36 @@
name: Test
on:
push: {}
pull_request:
paths-ignore:
- 'docs/**'
jobs:
test:
# skip branch builds on joerick/cibuildwheel, but always build `master` and pull requests
if: github.repository != 'joerick/cibuildwheel' || github.ref == 'refs/heads/master' || github.event_name == 'pull_request'
name: Test cibuildwheel on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-18.04, windows-latest, macos-latest]
python_version: ['3.7']
steps:
- uses: actions/checkout@v1
- uses: actions/setup-python@v1
name: Install Python ${{ matrix.python_version }}
with:
python-version: ${{ matrix.python_version }}
- name: Install dependencies
run: |
python -m pip install -r requirements-dev.txt
- name: Install Visual C++ for Python 2.7
if: startsWith(matrix.os, 'windows')
run: |
choco install vcpython27 -f -y
- name: Test cibuildwheel
run: |
python ./bin/run_tests.py
+4
View File
@@ -86,9 +86,13 @@ celerybeat-schedule
.venv
venv/
venv3/
venv2/
ENV/
env/
env2/
env2?/
env3/
env3?/
# Spyder project settings
.spyderproject
+40 -19
View File
@@ -1,33 +1,54 @@
language: generic
language: minimal
matrix:
branches:
only:
- master
jobs:
include:
# Linux Python 2
- sudo: required
language: python
python: 2.7
services: docker
env: PYTHON=python
# Linux Python 3
- sudo: required
- name: Linux | x86_64 + i686 | Python 3.5
language: python
python: 3.5
services: docker
env: PYTHON=python
# macOS Python 3
- os: osx
- name: Linux | arm64 | Python 3.5
language: python
python: 3.5
services: docker
arch: arm64
env: PYTHON=python
- name: Linux | ppc64le | Python 3.5
language: python
python: 3.5
services: docker
arch: ppc64le
env: PYTHON=python
- name: macOS | x86_64 | Python 3.x
os: osx
env: PYTHON=python3
- os: windows
- name: Windows | x86_64 | Python 3.5
os: windows
language: shell
before_install:
- choco install python3 --version 3.6.8 --no-progress -y
install:
- C:\\Python36\\python -m pip install -r requirements-dev.txt
script:
- C:\\Python36\\python ./bin/run_tests.py
- choco install python3 --version 3.5.4 --no-progress -y
env:
- PYTHON=C:\\Python35\\python
- &linux_s390x_35
name: Linux | s390x | Python 3.5
language: python
python: 3.5
services: docker
arch: s390x
env: PYTHON=python
allow_failures:
# must repeat the s390x job above exactly to match
- *linux_s390x_35
install: $PYTHON -m pip install -r requirements-dev.txt
+11
View File
@@ -0,0 +1,11 @@
This is a summary of the Python versions and platforms covered by the different CI platforms:
| | 3.5 | 3.6 | 3.7 | 3.8 |
|----------|------------------|------------------|---------------------------------------------------|------------------|
| Linux | Travis CI | CircleCI | AppVeyor / GitHub Actions | Azure Pipelines |
| macOS | Azure Pipelines | CircleCI | AppVeyor / Travis CI¹ / CircleCI / GitHub Actions | Azure Pipelines |
| Windows | TravisCI | Azure Pipelines | AppVeyor / GitHub Actions | Azure Pipelines |
> ¹ Python version not really pinned, but dependent on the (default) version of image used.
Non-x86 architectures are covered on Travis CI using Python 3.5.
+4
View File
@@ -22,3 +22,7 @@ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Get-pip and Pip (bundled in cibuildwheel/resources) are licensed under the MIT
license. See https://github.com/pypa/get-pip/blob/master/LICENSE.txt
+141 -34
View File
@@ -1,7 +1,7 @@
cibuildwheel
============
[![PyPI](https://img.shields.io/pypi/v/cibuildwheel.svg)](https://pypi.python.org/pypi/cibuildwheel) [![Build Status](https://travis-ci.org/joerick/cibuildwheel.svg?branch=master)](https://travis-ci.org/joerick/cibuildwheel) [![Build status](https://ci.appveyor.com/api/projects/status/wbsgxshp05tt1tif/branch/master?svg=true)](https://ci.appveyor.com/project/joerick/cibuildwheel/branch/master) [![CircleCI](https://circleci.com/gh/joerick/cibuildwheel.svg?style=svg)](https://circleci.com/gh/joerick/cibuildwheel)
[![PyPI](https://img.shields.io/pypi/v/cibuildwheel.svg)](https://pypi.python.org/pypi/cibuildwheel) [![Documentation Status](https://readthedocs.org/projects/cibuildwheel/badge/?version=stable)](https://cibuildwheel.readthedocs.io/en/stable/?badge=stable) [![Build Status](https://travis-ci.org/joerick/cibuildwheel.svg?branch=master)](https://travis-ci.org/joerick/cibuildwheel) [![Build status](https://ci.appveyor.com/api/projects/status/wbsgxshp05tt1tif/branch/master?svg=true)](https://ci.appveyor.com/project/joerick/cibuildwheel/branch/master) [![CircleCI](https://circleci.com/gh/joerick/cibuildwheel.svg?style=svg)](https://circleci.com/gh/joerick/cibuildwheel) [![Build Status](https://dev.azure.com/joerick0429/cibuildwheel/_apis/build/status/joerick.cibuildwheel?branchName=master)](https://dev.azure.com/joerick0429/cibuildwheel/_build/latest?definitionId=2&branchName=master)
[Documentation](https://cibuildwheel.readthedocs.org)
@@ -9,36 +9,40 @@ cibuildwheel
Python wheels are great. Building them across **Mac, Linux, Windows**, on **multiple versions of Python**, is not.
`cibuildwheel` is here to help. `cibuildwheel` runs on your CI server - currently it supports Azure Pipelines, Travis CI, AppVeyor, and CircleCI - and it builds and tests your wheels across all of your platforms.
`cibuildwheel` is here to help. `cibuildwheel` runs on your CI server - currently it supports Azure Pipelines, Travis CI, AppVeyor, GitHub Actions and CircleCI - and it builds and tests your wheels across all of your platforms.
What does it do?
----------------
| | macOS 10.6+ intel | macOS 10.9+ x86_64 | manylinux i686 | manylinux x86_64 | Windows 32bit | Windows 64bit |
|---|---|---|---|---|---|---|
| Python 2.7 | ✅ | | ✅ | ✅ | ✅¹ | ✅¹ |
| Python 3.5 | ✅ | | ✅ | ✅ | ✅ | ✅ |
| Python 3.6 | ✅ | | ✅ | ✅ | ✅ | ✅ |
| Python 3.7 | ✅ | | ✅ | ✅ | ✅ | ✅ |
| Python 3.8 | | ✅ | ✅ | ✅ | ✅ | ✅ |
| | macOS x86_64 | Windows 64bit | Windows 32bit | manylinux x86_64 | manylinux i686 | manylinux aarch64 | manylinux ppc64le | manylinux s390x |
|---|---|---|---|---|---|---|---|---|
| CPython 2.7 | ✅ | ✅¹ | ✅¹ | ✅ | ✅ | | | |
| CPython 3.5 | ✅ | | ✅ | ✅ | ✅ | ✅ | ✅ | ✅² |
| CPython 3.6 | ✅ | | ✅ | ✅ | ✅ | ✅ | ✅ | ✅² |
| CPython 3.7 | ✅ | | ✅ | ✅ | ✅ | ✅ | ✅ | ✅² |
| CPython 3.8 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅² |
| PyPy 2.7 v7.3.0 | ✅ | | ✅ | ✅ | | | | |
| PyPy 3.6 v7.3.0 | ✅ | | ✅ | ✅ | | | | |
> ¹ Not supported on Travis
<sup>¹ Not supported on Travis</sup><br>
<sup>² Beta support until Travis CI fixes <a href="https://travis-ci.community/t/no-space-left-on-device-for-system-z/5954/11">a bug</a></sup>
- Builds manylinux, macOS and Windows (32 and 64bit) wheels using Azure Pipelines, Travis CI, AppVeyor, and CircleCI
- Builds manylinux, macOS and Windows wheels for CPython and PyPy using Azure Pipelines, Travis CI, AppVeyor, and CircleCI
- Bundles shared library dependencies on Linux and macOS through [auditwheel](https://github.com/pypa/auditwheel) and [delocate](https://github.com/matthew-brett/delocate)
- Runs the library test suite against the wheel-installed version of your library
Usage
-----
`cibuildwheel` currently works on **Travis CI** and **Azure Pipelines** to build wheels for all three supported platforms (Linux, macOS, Windows). On **CircleCI** Linux and macOS wheels can be built, and on **AppVeyor** Windows is supported.
`cibuildwheel` currently works on **Travis CI**, **Azure Pipelines**, **AppVeyor** and **GitHub Actions** to build wheels for all three supported platforms (Linux, macOS, Windows). On **CircleCI** Linux and macOS wheels can be built.
| | Linux | macOS | Windows |
|-----------------|-------|-------|---------|
| Azure Pipelines | ✅ | ✅ | ✅ |
| Travis CI | ✅ | ✅ | ✅ |
| AppVeyor | | | ✅ |
| AppVeyor | | | ✅ |
| GitHub Actions | ✅ | ✅ | ✅ |
| CircleCI | ✅ | ✅ | |
`cibuildwheel` is not intended to run on your development machine. Because it uses system Python from Python.org it will try to install packages globally - not what you expect from a build tool! Instead, isolated CI services like Travis CI, CircleCI, Azure Pipelines and AppVeyor are ideal.
@@ -48,43 +52,47 @@ Usage
Example setup
-------------
To build manylinux and macOS wheels on Travis CI, and upload them to PyPI whenever you tag a version, you could use this `.travis.yml`:
To build manylinux, macOS, and Windows wheels on Travis CI and upload them to PyPI whenever you tag a version, you could use this `.travis.yml`:
```yaml
language: python
matrix:
jobs:
include:
# perform a linux build
- sudo: required
services:
- docker
env: PIP=pip
- services: docker
# and a mac build
- os: osx
language: generic
env: PIP=pip2
language: shell
# and a windows build
- os: windows
language: shell
before_install:
- choco install python --version 3.8.0
- export PATH="/c/Python38:/c/Python38/Scripts:$PATH"
env:
global:
- TWINE_USERNAME=joerick
# Note: TWINE_PASSWORD is set in Travis settings
- TWINE_USERNAME=__token__
# Note: TWINE_PASSWORD is set to a PyPI API token in Travis settings
install:
- python3 -m pip install cibuildwheel==1.4.0
script:
- $PIP install cibuildwheel==1.0.0
# build the wheels, put them into './wheelhouse'
- cibuildwheel --output-dir wheelhouse
- python3 -m cibuildwheel --output-dir wheelhouse
after_success:
# if the release was tagged, upload them to PyPI
- |
if [[ $TRAVIS_TAG ]]; then
python -m pip install twine
python -m twine upload wheelhouse/*.whl
python3 -m pip install twine
python3 -m twine upload wheelhouse/*.whl
fi
```
For more information, including how to build on Appveyor, Azure, CircleCI, check out the [documentation](https://cibuildwheel.readthedocs.org).
For more information, including how to build on Appveyor, Azure, CircleCI, check out the [documentation](https://cibuildwheel.readthedocs.org) and also check out [the examples](https://github.com/joerick/cibuildwheel/tree/master/examples).
Options
-------
@@ -95,11 +103,13 @@ Options
| | [`CIBW_BUILD`](https://cibuildwheel.readthedocs.io/en/stable/options/#build-skip) [`CIBW_SKIP`](https://cibuildwheel.readthedocs.io/en/stable/options/#build-skip) | Choose the Python versions to build |
| **Build environment** | [`CIBW_ENVIRONMENT`](https://cibuildwheel.readthedocs.io/en/stable/options/#environment) | Set environment variables needed during the build |
| | [`CIBW_BEFORE_BUILD`](https://cibuildwheel.readthedocs.io/en/stable/options/#before-build) | Execute a shell command preparing each wheel's build |
| | [`CIBW_MANYLINUX_X86_64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) [`CIBW_MANYLINUX_I686_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) | Specify alternative manylinux docker images |
| **Testing** | [`CIBW_TEST_COMMAND`](https://cibuildwheel.readthedocs.io/en/stable/options/#test-command) | Execute a shell command to test all built wheels |
| | [`CIBW_REPAIR_WHEEL_COMMAND`](https://cibuildwheel.readthedocs.io/en/stable/options/#repair-wheel-command) | Execute a shell command to repair each (non-pure Python) built wheel |
| | [`CIBW_MANYLINUX_X86_64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) [`CIBW_MANYLINUX_I686_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) [`CIBW_MANYLINUX_PYPY_X86_64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) | Specify alternative manylinux docker images |
| **Testing** | [`CIBW_TEST_COMMAND`](https://cibuildwheel.readthedocs.io/en/stable/options/#test-command) | Execute a shell command to test each built wheel |
| | [`CIBW_BEFORE_TEST`](https://cibuildwheel.readthedocs.io/en/stable/options/#before-test) | Execute shell command to prepare test environment |
| | [`CIBW_TEST_REQUIRES`](https://cibuildwheel.readthedocs.io/en/stable/options/#test-requires) | Install Python dependencies before running the tests |
| | [`CIBW_TEST_EXTRAS`](https://cibuildwheel.readthedocs.io/en/stable/options/#test-extras) | Install your wheel for testing using extras_require |
| **Other** | [`CIBW_BUILD_VERBOSITY`](https://cibuildwheel.readthedocs.io/en/stable/options/#test-extras) | Increase/decrease the output of pip wheel |
| **Other** | [`CIBW_BUILD_VERBOSITY`](https://cibuildwheel.readthedocs.io/en/stable/options/#build-verbosity) | Increase/decrease the output of pip wheel |
Working examples
----------------
@@ -116,13 +126,15 @@ Here are some repos that use cibuildwheel.
- [apriltags2-ethz](https://github.com/safijari/apriltags2_ethz)
- [TgCrypto](https://github.com/pyrogram/tgcrypto)
- [Twisted](https://github.com/twisted/twisted)
- [gmic-py](https://github.com/dtschump/gmic-py)
- [creme](https://github.com/creme-ml/creme)
> Add your repo here! Send a PR.
Legal note
----------
Since `cibuildwheel` runs the wheel through delocate or auditwheel, it might automatically bundle dynamically linked libraries from the build machine.
Since `cibuildwheel` repairs the wheel with `delocate` or `auditwheel`, it might automatically bundle dynamically linked libraries from the build machine.
It helps ensure that the library can run without any dependencies outside of the pip toolchain.
@@ -131,6 +143,102 @@ This is similar to static linking, so it might have some licence implications. C
Changelog
=========
### 1.4.0
_2 May 2020_
- ✨ Deterministic builds. cibuildwheel now locks the versions of the tools it
uses. This means that pinning your version of cibuildwheel pins the versions
of pip, setuptools, manylinux etc. that are used under the hood. This should
make things more reliable. But note that we don't control the entire build
environment on macOS and Windows, where the version of Xcode and Visual
Studio can still effect things.
This can be controlled using the [CIBW_DEPENDENCY_VERSIONS](https://cibuildwheel.readthedocs.io/en/stable/options/#dependency-versions)
and [manylinux image](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image)
options - if you always want to use the latest toolchain, you can still do
that, or you can specify your own pip constraints file and manylinux image.
(#256)
- ✨ Added `package_dir` command line option, meaning we now support building
a package that lives in a subdirectory and pulls in files from the wider
project. See [the `package_dir` option help](https://cibuildwheel.readthedocs.io/en/stable/options/#command-line-options)
for more information.
Note that this change makes the working directory (where you call
cibuildwheel from) relevant on Linux, as it's considered the 'project' and
will be copied into the Docker container. If your builds are slower on this
version, that's likely the reason. `cd` to your project and then call
`cibuildwheel` from there. (#319, #295)
- 🛠 On macOS, we make `MACOSX_DEPLOYMENT_TARGET` default to `10.9` if it's
not set. This should make things more consistent between Python versions.
- 🛠 Dependency updates - CPython 3.7.7, CPython 2.7.18, Pypy 7.3.1.
### 1.3.0
_12 March 2020_
- ✨ Add support for building on Github Actions! Check out the
[docs](https://cibuildwheel.readthedocs.io/en/stable/setup/#github-actions)
for information on how to set it up. (#194)
- ✨ Add the `CIBW_BEFORE_TEST` option, which lets you run a command to
prepare the environment before your tests are run. (#242)
### 1.2.0
_8 March 2020_
- ✨ Add support for building PyPy wheels, across Manylinux, macOS, and
Windows. (#185)
- ✨ Added the ability to build ARM64 (aarch64), ppc64le, and s390x wheels,
using manylinux2014 and Travis CI. (#273)
- ✨ You can now build macOS wheels on Appveyor. (#230)
- 🛠 Changed default macOS minimum target to 10.9, from 10.6. This allows the
use of more modern C++ libraries, among other things. (#156)
- 🛠 Stop building universal binaries on macOS. We now only build x86_64
wheels on macOS. (#220)
- ✨ Allow chaining of commands using `&&` and `||` on Windows inside
CIBW_BEFORE_BUILD and CIBW_TEST_COMMAND. (#293)
- 🛠 Improved error reporting for failed Cython builds due to stale .so files
(#263)
- 🛠 Update CPython from 3.7.5 to 3.7.6 and from 3.8.0 to 3.8.2 on Mac/Windows
- 🛠 Improved error messages when a bad config breaks cibuildwheel's PATH
variable. (#264)
- ⚠️ Removed support for *running* cibuildwheel on Python 2.7. cibuildwheel
will continue to build Python 2.7 wheels for a little while. (#265)
### 1.1.0
_7 December 2019_
- ✨ Add support for building manylinux2014 wheels. To use, set
`CIBW_MANYLINUX_X86_64_IMAGE` and CIBW_MANYLINUX_I686_IMAGE to
`manylinux2014`.
- ✨ Add support for [Linux on Appveyor](https://www.appveyor.com/blog/2018/03/06/appveyor-for-linux/) (#204, #207)
- ✨ Add `CIBW_REPAIR_WHEEL_COMMAND` env variable, for changing how
`auditwheel` or `delocate` are invoked, or testing an equivalent on
Windows. (#211)
- 📚 Added some travis example configs - these are available in /examples. (#228)
### 1.0.0
_10 November 2019_
- ✨ Add support for building Python 3.8 wheels! (#180)
- ✨ Add support for building manylinux2010 wheels. cibuildwheel will now
build using the manylinux2010 images by default. If your project is still
manylinux1 compatible, you should get both manylinux1 and manylinux2010
wheels - you can upload both to PyPI. If you always require manylinux1 wheels, you can
build using the old manylinux1 image using the [manylinux image](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) option.
(#155)
- 📚 Documentation is now on its [own mini-site](https://cibuildwheel.readthedocs.io),
rather than on the README (#169)
- ✨ Add support for building Windows wheels on Travis CI. (#160)
- 🛠 If you set `CIBW_TEST_COMMAND`, your tests now run in a virtualenv. (#164)
- 🛠 Windows now uses Python as installed by nuget, rather than the versions
installed by the various CI providers. (#180)
- 🛠 Update Python from 2.7.16 to 2.7.17 and 3.7.4 to 3.7.5 on macOS (#171)
- ⚠️ Removed support for Python 3.4 (#168)
### 0.12.0
_29 September 2019_
@@ -312,7 +420,6 @@ Maintainers
-----------
- Joe Rickerby [@joerick](https://github.com/joerick)
- Tomas Garcia [@tgarc](https://github.com/tgarc)
- Yannick Jadoul [@YannickJadoul](https://github.com/YannickJadoul)
- Matthieu Darbois [@mayeut](https://github.com/mayeut)
+34 -7
View File
@@ -1,10 +1,37 @@
environment:
# workaround an appveyor bug on macos
# see https://github.com/appveyor/ci/issues/3376#issuecomment-611314299
APPVEYOR_CONSOLE_DISABLE_PTY: true
matrix:
- PYTHON: "C:\\Python27\\python.exe"
- PYTHON: "C:\\Python35\\python.exe"
- APPVEYOR_BUILD_WORKER_IMAGE: Ubuntu
APPVEYOR_JOB_NAME: "python37-x64-ubuntu"
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
APPVEYOR_JOB_NAME: "python37-x64-vs2015"
- APPVEYOR_BUILD_WORKER_IMAGE: macos-mojave
APPVEYOR_JOB_NAME: "python37-x64-macos-mojave"
build_script:
- "%PYTHON% -m pip install -r requirements-dev.txt"
# the '-u' flag is required so the output is in the correct order.
# See https://github.com/joerick/cibuildwheel/pull/24 for more info.
- "%PYTHON% -u ./bin/run_tests.py"
stack: python 3.7
build: off
init:
- cmd: set PATH=C:\Python37;C:\Python37\Scripts;%PATH%
- ps: |
$BRANCH = if ($env:APPVEYOR_PULL_REQUEST_HEAD_REPO_BRANCH) { $env:APPVEYOR_PULL_REQUEST_HEAD_REPO_BRANCH } else { $env:APPVEYOR_REPO_BRANCH }
if (-not ($BRANCH -eq 'master' -or $BRANCH.ToLower().StartsWith('appveyor-'))) {
$env:PYTEST_ADDOPTS='-k "unit_test or _basic" --suppress-no-test-exit-code'
}
install: python -m pip install -r requirements-dev.txt pytest-custom-exit-code
# the '-u' flag is required so the output is in the correct order.
# See https://github.com/joerick/cibuildwheel/pull/24 for more info.
test_script: python -u ./bin/run_tests.py
branches:
only:
- master
skip_commits:
files:
- docs/*
+26 -4
View File
@@ -1,5 +1,5 @@
jobs:
- job: linux
- job: linux_38
pool: {vmImage: 'Ubuntu-18.04'}
steps:
- task: UsePythonVersion@0
@@ -9,8 +9,18 @@ jobs:
python -m pip install -r requirements-dev.txt
python ./bin/run_tests.py
- job: macos
pool: {vmImage: 'macOS-10.13'}
- job: macos_35
pool: {vmImage: 'macOS-10.15'}
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.5'
- bash: |
python -m pip install -r requirements-dev.txt
python ./bin/run_tests.py
- job: macos_38
pool: {vmImage: 'macOS-10.15'}
steps:
- task: UsePythonVersion@0
inputs:
@@ -19,7 +29,19 @@ jobs:
python -m pip install -r requirements-dev.txt
python ./bin/run_tests.py
- job: windows
- job: windows_36
pool: {vmImage: 'vs2017-win2016'}
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.6'
- script: choco install vcpython27 -f -y
displayName: Install Visual C++ for Python 2.7
- bash: |
python -m pip install -r requirements-dev.txt
python ./bin/run_tests.py
- job: windows_38
pool: {vmImage: 'vs2017-win2016'}
steps:
- task: UsePythonVersion@0
+113
View File
@@ -0,0 +1,113 @@
#!/usr/bin/env python3
import click
from pathlib import Path
import os
import cibuildwheel
from packaging.version import Version, InvalidVersion
import subprocess
import glob
config = [
# file path, version find/replace format
('cibuildwheel/__init__.py', "__version__ = '{}'"),
('setup.py', "version='{}'"),
('README.md', "cibuildwheel=={}"),
('examples/*', "cibuildwheel=={}"),
]
@click.command()
def bump_version():
git_changes_result = subprocess.run(['git diff-index --quiet HEAD --'], shell=True)
repo_has_uncommitted_changes = git_changes_result.returncode != 0
if repo_has_uncommitted_changes:
print('error: Uncommitted changes detected.')
exit(1)
current_version = cibuildwheel.__version__
print( 'Current version:', current_version) # noqa
new_version = input(' New version: ').strip()
try:
Version(new_version)
except InvalidVersion:
print("error: This version doesn't conform to PEP440")
print(' https://www.python.org/dev/peps/pep-0440/')
exit(1)
actions = []
for path_pattern, version_pattern in config:
paths = [Path(p) for p in glob.glob(path_pattern)]
if not paths:
print(f'error: Pattern {path_pattern} didnt match any files')
exit(1)
find_pattern = version_pattern.format(current_version)
replace_pattern = version_pattern.format(new_version)
found_at_least_one_file_needing_update = False
for path in paths:
contents = path.read_text(encoding='utf8')
if find_pattern in contents:
found_at_least_one_file_needing_update = True
actions.append(
(path, find_pattern, replace_pattern)
)
if not found_at_least_one_file_needing_update:
print('error: Didnt find any occurences of “{}” in “{}'.format(find_pattern, path_pattern))
exit(1)
print()
print("Here's the plan:")
print()
for action in actions:
print('{} {red}{}{off}{green}{}{off}'.format(
*action,
red="\u001b[31m", green="\u001b[32m", off="\u001b[0m"
))
print('Then commit, and tag as v{}'.format(new_version))
answer = input('Proceed? [y/N] ').strip()
if answer != 'y':
print('Aborted')
exit(1)
for path, find, replace in actions:
contents = path.read_text(encoding='utf8')
contents = contents.replace(find, replace)
path.write_text(contents, encoding='utf8')
print('Files updated. If you want to update the changelog as part of this')
print('commit, do that now.')
while input('Type "done" to continue: ').strip().lower() != 'done':
pass
subprocess.run([
'git', 'commit',
'-a',
'-m', f'Bump version: v{new_version}'
], check=True)
subprocess.run([
'git', 'tag',
'-a',
'-m', f'v{new_version}',
f'v{new_version}'
], check=True)
print('Done.')
if __name__ == '__main__':
os.chdir(os.path.dirname(__file__))
os.chdir('..')
bump_version()
+75
View File
@@ -0,0 +1,75 @@
#!/usr/bin/env python3
import os
import time
from pathlib import Path
from subprocess import run
import click
import textwrap
def shell(cmd, **kwargs):
return run([cmd], shell=True, **kwargs)
def git_repo_has_changes():
unstaged_changes = shell('git diff-index --quiet HEAD --').returncode != 0
staged_changes = shell('git diff-index --quiet --cached HEAD --').returncode != 0
return unstaged_changes or staged_changes
@click.command()
def main():
project_root = Path(__file__).parent / '..'
os.chdir(project_root)
if git_repo_has_changes():
print('Your git repo has uncommitted changes. Commit or stash before continuing.')
exit(1)
previous_branch = shell('git rev-parse --abbrev-ref HEAD',
check=True,
capture_output=True,
encoding='utf8').stdout.strip()
shell('git fetch origin', check=True)
timestamp = time.strftime('%Y-%m-%dT%H-%M-%S', time.gmtime())
branch_name = f'update-constraints-{timestamp}'
shell(f'git checkout -b {branch_name} origin/master', check=True)
try:
shell('bin/update_dependencies.py', check=True)
if not git_repo_has_changes():
print('Done: no constraint updates required.')
return
shell('git commit -a -m "Update dependencies"', check=True)
run(
[
'gh', 'pr', 'create',
'--repo', 'joerick/cibuildwheel',
'--base', 'master',
'--title', 'Update dependencies',
'--body', textwrap.dedent(f'''
Update the versions of our dependencies.
PR generated by `{os.path.basename(__file__)}`.
''')
],
check=True
)
print('Done.')
finally:
# remove any local changes
shell('git checkout -- .')
shell(f'git checkout {previous_branch}', check=True)
shell(f'git branch -D --force {branch_name}', check=True)
if __name__ == '__main__':
main.main(standalone_mode=True)
+6 -20
View File
@@ -1,39 +1,25 @@
#!/usr/bin/env python
#!/usr/bin/env python3
from __future__ import print_function
import os, sys, subprocess, shutil
import argparse
import os
import subprocess
import sys
project_root = os.path.dirname(os.path.dirname(__file__))
test_utils_dir = os.path.join(project_root, 'test', 'shared')
def single_run(test_project):
# set up an environment that gives access to the test utils
env = os.environ.copy()
if 'PYTHONPATH' in env:
env['PYTHONPATH'] += os.pathsep + test_utils_dir
else:
env['PYTHONPATH'] = test_utils_dir
# run the test
subprocess.check_call(
[sys.executable, '-m', 'pytest', '-vv', os.path.join(test_project, 'cibuildwheel_test.py')],
env=env,
)
# clean up
if os.path.exists('wheelhouse'):
shutil.rmtree('wheelhouse')
if __name__ == '__main__':
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("test_project_dir")
args = parser.parse_args()
project_path = os.path.abspath(args.test_project_dir)
if not os.path.exists(project_path):
print('No test project not found.', file=sys.stderr)
exit(2)
+6 -5
View File
@@ -1,18 +1,19 @@
#!/usr/bin/env python
#!/usr/bin/env python3
from __future__ import print_function
import os, sys, subprocess, shutil, json
import os
import subprocess
import sys
from glob import glob
if __name__ == '__main__':
# move cwd to the project root
os.chdir(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
### run the unit tests
# run the unit tests
subprocess.check_call([sys.executable, '-m', 'pytest', 'unit_test'])
### run the integration tests
# run the integration tests
test_projects = sorted(glob('test/??_*'))
+99
View File
@@ -0,0 +1,99 @@
#!/usr/bin/env python3
import configparser
import os
import subprocess
from collections import namedtuple
import requests
os.chdir(os.path.dirname(__file__))
os.chdir('..')
# CUSTOM_COMPILE_COMMAND is a pip-compile option that tells users how to
# regenerate the constraints files
os.environ['CUSTOM_COMPILE_COMMAND'] = "bin/update_constraints.py"
subprocess.check_call([
'pip-compile',
'--allow-unsafe',
'--upgrade',
'cibuildwheel/resources/constraints.in',
])
for python_version in ['27', '35', '36']:
subprocess.check_call([
f'./env{python_version}/bin/pip-compile',
'--allow-unsafe',
'--upgrade',
'cibuildwheel/resources/constraints.in',
'--output-file', f'cibuildwheel/resources/constraints-python{python_version}.txt'
])
Image = namedtuple('Image', [
'manylinux_version',
'platform',
'image_name',
])
images = [
Image('manylinux1', 'x86_64', 'quay.io/pypa/manylinux1_x86_64'),
Image('manylinux1', 'i686', 'quay.io/pypa/manylinux1_i686'),
Image('manylinux2010', 'x86_64', 'quay.io/pypa/manylinux2010_x86_64'),
Image('manylinux2010', 'i686', 'quay.io/pypa/manylinux2010_i686'),
Image('manylinux2010', 'pypy_x86_64', 'pypywheels/manylinux2010-pypy_x86_64'),
Image('manylinux2014', 'x86_64', 'quay.io/pypa/manylinux2014_x86_64'),
Image('manylinux2014', 'i686', 'quay.io/pypa/manylinux2014_i686'),
Image('manylinux2014', 'aarch64', 'quay.io/pypa/manylinux2014_aarch64'),
Image('manylinux2014', 'ppc64le', 'quay.io/pypa/manylinux2014_ppc64le'),
Image('manylinux2014', 's390x', 'quay.io/pypa/manylinux2014_s390x'),
]
config = configparser.ConfigParser()
for image in images:
# get the tag name whose digest matches 'latest'
if image.image_name.startswith('quay.io/'):
_, _, repository_name = image.image_name.partition('/')
response = requests.get(
f'https://quay.io/api/v1/repository/{repository_name}?includeTags=true'
)
response.raise_for_status()
repo_info = response.json()
tags_dict = repo_info['tags']
latest_tag = tags_dict.pop('latest')
# find the tag whose manifest matches 'latest'
tag_name = next(
name
for (name, info) in tags_dict.items()
if info['manifest_digest'] == latest_tag['manifest_digest']
)
else:
response = requests.get(
f'https://hub.docker.com/v2/repositories/{image.image_name}/tags'
)
response.raise_for_status()
tags = response.json()['results']
latest_tag = next(
tag for tag in tags if tag['name'] == 'latest'
)
# i don't know what it would mean to have multiple images per tag
assert len(latest_tag['images']) == 1
digest = latest_tag['images'][0]['digest']
pinned_tag = next(
tag
for tag in tags
if tag != latest_tag and tag['images'][0]['digest'] == digest
)
tag_name = pinned_tag['name']
if not config.has_section(image.platform):
config[image.platform] = {}
config[image.platform][image.manylinux_version] = f'{image.image_name}:{tag_name}'
with open('cibuildwheel/resources/pinned_docker_images.cfg', 'w') as f:
config.write(f)
+1 -1
View File
@@ -1 +1 @@
__version__ = '1.0.0'
__version__ = '1.4.0'
+139 -82
View File
@@ -1,10 +1,25 @@
from __future__ import print_function
import argparse, os, subprocess, sys, textwrap
import argparse
import os
import sys
import textwrap
import traceback
from configparser import ConfigParser
import cibuildwheel
import cibuildwheel.linux, cibuildwheel.windows, cibuildwheel.macos
from cibuildwheel.environment import parse_environment, EnvironmentParseError
from cibuildwheel.util import BuildSelector, Unbuffered
import cibuildwheel.linux
import cibuildwheel.macos
import cibuildwheel.windows
from cibuildwheel.environment import (
EnvironmentParseError,
parse_environment,
)
from cibuildwheel.util import (
BuildSelector,
DependencyConstraints,
Unbuffered,
BuildOptions
)
def get_option_from_environment(option_name, platform=None, default=None):
'''
@@ -24,29 +39,43 @@ def get_option_from_environment(option_name, platform=None, default=None):
return os.environ.get(option_name, default)
def strtobool(val):
if val.lower() in ('y', 'yes', 't', 'true', 'on', '1'):
return True
return False
def main():
parser = argparse.ArgumentParser(
description='Build wheels for all the platforms.',
epilog=('Most options are supplied via environment variables. '
'See https://github.com/joerick/cibuildwheel#options for info.'))
epilog='''
Most options are supplied via environment variables.
See https://github.com/joerick/cibuildwheel#options for info.
''')
parser.add_argument('--platform',
choices=['auto', 'linux', 'macos', 'windows'],
default=os.environ.get('CIBW_PLATFORM', 'auto'),
help=('Platform to build for. For "linux" you need docker running, on Mac '
'or Linux. For "macos", you need a Mac machine, and note that this '
'script is going to automatically install MacPython on your system, '
'so don\'t run on your development machine. For "windows", you need to '
'run in Windows, and it will build and test for all versions of '
'Python at C:\\PythonXX[-x64]. Default: auto.'))
help='''
Platform to build for. For "linux" you need docker running, on Mac
or Linux. For "macos", you need a Mac machine, and note that this
script is going to automatically install MacPython on your system,
so don't run on your development machine. For "windows", you need to
run in Windows, and it will build and test for all versions of
Python. Default: auto.
''')
parser.add_argument('--output-dir',
default=os.environ.get('CIBW_OUTPUT_DIR', 'wheelhouse'),
help='Destination folder for the wheels.')
parser.add_argument('project_dir',
parser.add_argument('package_dir',
default='.',
nargs='?',
help=('Path to the project that you want wheels for. Default: the current '
'directory.'))
help='''
Path to the package that you want wheels for. Must be a subdirectory of
the working directory. When set, the working directory is still
considered the 'project' and is copied into the Docker container on
Linux. Default: the working directory.
''')
parser.add_argument('--print-build-identifiers',
action='store_true',
@@ -59,36 +88,23 @@ def main():
if args.platform != 'auto':
platform = args.platform
else:
platform = None
if os.environ.get('TRAVIS_OS_NAME') == 'linux':
platform = 'linux'
elif os.environ.get('TRAVIS_OS_NAME') == 'osx':
platform = 'macos'
elif os.environ.get('TRAVIS_OS_NAME') == 'windows':
platform = 'windows'
elif 'APPVEYOR' in os.environ:
platform = 'windows'
elif 'BITRISE_BUILD_NUMBER' in os.environ:
platform = 'macos'
elif os.environ.get('CIRCLECI'):
if sys.platform.startswith('linux'):
platform = 'linux'
elif sys.platform.startswith('darwin'):
platform = 'macos'
elif 'AZURE_HTTP_USER_AGENT' in os.environ:
if os.environ['AGENT_OS'] == 'Linux':
platform = 'linux'
elif os.environ['AGENT_OS'] == 'Darwin':
platform = 'macos'
elif os.environ['AGENT_OS'] == 'Windows_NT':
platform = 'windows'
if platform is None:
ci = strtobool(os.environ.get('CI', 'false')) or 'BITRISE_BUILD_NUMBER' in os.environ or 'AZURE_HTTP_USER_AGENT' in os.environ or 'GITHUB_WORKFLOW' in os.environ
if not ci:
print('cibuildwheel: Unable to detect platform. cibuildwheel should run on your CI server, '
'Travis CI, AppVeyor, and CircleCI are supported. You can run on your development '
'machine using the --platform argument. Check --help output for more '
'information.',
'Travis CI, AppVeyor, Azure Pipelines, GitHub Actions and CircleCI are supported. You '
'can run on your development machine or other CI providers using the --platform argument. '
'Check --help output for more information.',
file=sys.stderr)
exit(2)
if sys.platform.startswith('linux'):
platform = 'linux'
elif sys.platform == 'darwin':
platform = 'macos'
elif sys.platform == 'win32':
platform = 'windows'
else:
print('cibuildwheel: Unable to detect platform from "sys.platform" in a CI environment. You can run '
'cibuildwheel using the --platform argument. Check --help output for more information.',
file=sys.stderr)
exit(2)
@@ -96,11 +112,27 @@ def main():
test_command = get_option_from_environment('CIBW_TEST_COMMAND', platform=platform)
test_requires = get_option_from_environment('CIBW_TEST_REQUIRES', platform=platform, default='').split()
test_extras = get_option_from_environment('CIBW_TEST_EXTRAS', platform=platform, default='')
project_dir = args.project_dir
package_dir = args.package_dir
before_build = get_option_from_environment('CIBW_BEFORE_BUILD', platform=platform)
build_verbosity = get_option_from_environment('CIBW_BUILD_VERBOSITY', platform=platform, default='')
build_config, skip_config = os.environ.get('CIBW_BUILD', '*'), os.environ.get('CIBW_SKIP', '')
if platform == 'linux':
repair_command_default = 'auditwheel repair -w {dest_dir} {wheel}'
elif platform == 'macos':
repair_command_default = 'delocate-listdeps {wheel} && delocate-wheel --require-archs x86_64 -w {dest_dir} {wheel}'
else:
repair_command_default = ''
repair_command = get_option_from_environment('CIBW_REPAIR_WHEEL_COMMAND', platform=platform, default=repair_command_default)
environment_config = get_option_from_environment('CIBW_ENVIRONMENT', platform=platform, default='')
before_test = get_option_from_environment('CIBW_BEFORE_TEST', platform=platform, default='')
dependency_versions = get_option_from_environment('CIBW_DEPENDENCY_VERSIONS', platform=platform, default='pinned')
if dependency_versions == 'pinned':
dependency_constraints = DependencyConstraints.with_defaults()
elif dependency_versions == 'latest':
dependency_constraints = None
else:
dependency_constraints = DependencyConstraints(dependency_versions)
if test_extras:
test_extras = '[{0}]'.format(test_extras)
@@ -112,9 +144,8 @@ def main():
try:
environment = parse_environment(environment_config)
except (EnvironmentParseError, ValueError) as e:
except (EnvironmentParseError, ValueError):
print('cibuildwheel: Malformed environment option "%s"' % environment_config, file=sys.stderr)
import traceback
traceback.print_exc(None, sys.stderr)
exit(2)
@@ -124,16 +155,49 @@ def main():
# This needs to be passed on to the docker container in linux.py
os.environ['CIBUILDWHEEL'] = '1'
if not os.path.exists(os.path.join(project_dir, 'setup.py')):
print('cibuildwheel: Could not find setup.py at root of project', file=sys.stderr)
if not os.path.exists(os.path.join(package_dir, 'setup.py')):
print('cibuildwheel: Could not find setup.py at root of package', file=sys.stderr)
exit(2)
if args.print_build_identifiers:
print_build_identifiers(platform, build_selector)
exit(0)
build_options = dict(
project_dir=project_dir,
if platform == 'linux':
pinned_docker_images_file = os.path.join(
os.path.dirname(__file__), 'resources', 'pinned_docker_images.cfg'
)
all_pinned_docker_images = ConfigParser()
all_pinned_docker_images.read(pinned_docker_images_file)
# all_pinned_docker_images looks like a dict of dicts, e.g.
# { 'x86_64': {'manylinux1': '...', 'manylinux2010': '...', 'manylinux2014': '...'},
# 'i686': {'manylinux1': '...', 'manylinux2010': '...', 'manylinux2014': '...'},
# 'pypy_x86_64': {'manylinux2010': '...' }
# ... }
manylinux_images = {}
for build_platform in ['x86_64', 'i686', 'pypy_x86_64', 'aarch64', 'ppc64le', 's390x']:
pinned_images = all_pinned_docker_images[build_platform]
config_name = 'CIBW_MANYLINUX_{}_IMAGE'.format(build_platform.upper())
config_value = os.environ.get(config_name)
if config_value is None:
# default to manylinux2010 if it's available, otherwise manylinux2014
image = pinned_images.get('manylinux2010') or pinned_images.get('manylinux2014')
elif config_value in pinned_images:
image = pinned_images[config_value]
else:
image = config_value
manylinux_images[build_platform] = image
else:
manylinux_images = None
build_options = BuildOptions(
package_dir=package_dir,
output_dir=output_dir,
test_command=test_command,
test_requires=test_requires,
@@ -141,27 +205,13 @@ def main():
before_build=before_build,
build_verbosity=build_verbosity,
build_selector=build_selector,
repair_command=repair_command,
environment=environment,
before_test=before_test,
dependency_constraints=dependency_constraints,
manylinux_images=manylinux_images,
)
if platform == 'linux':
manylinux_x86_64_image = os.environ.get('CIBW_MANYLINUX_X86_64_IMAGE', 'manylinux2010')
manylinux_i686_image = os.environ.get('CIBW_MANYLINUX_I686_IMAGE', 'manylinux2010')
default_manylinux_images_x86_64 = {'manylinux1': 'quay.io/pypa/manylinux1_x86_64',
'manylinux2010': 'quay.io/pypa/manylinux2010_x86_64'}
default_manylinux_images_i686 = {'manylinux1': 'quay.io/pypa/manylinux1_i686',
'manylinux2010': 'quay.io/pypa/manylinux2010_i686'}
build_options.update(
manylinux_images={'x86_64': default_manylinux_images_x86_64.get(manylinux_x86_64_image) or manylinux_x86_64_image,
'i686': default_manylinux_images_i686.get(manylinux_i686_image) or manylinux_i686_image},
)
elif platform == 'macos':
pass
elif platform == 'windows':
pass
# Python is buffering by default when running on the CI platforms, giving problems interleaving subprocess call output with unflushed calls to 'print'
sys.stdout = Unbuffered(sys.stdout)
@@ -171,13 +221,14 @@ def main():
os.makedirs(output_dir)
if platform == 'linux':
cibuildwheel.linux.build(**build_options)
cibuildwheel.linux.build(build_options)
elif platform == 'windows':
cibuildwheel.windows.build(**build_options)
cibuildwheel.windows.build(build_options)
elif platform == 'macos':
cibuildwheel.macos.build(**build_options)
cibuildwheel.macos.build(build_options)
else:
raise Exception('Unsupported platform')
print('cibuildwheel: Unsupported platform: {}'.format(platform), file=sys.stderr)
exit(2)
def detect_obsolete_options():
@@ -193,11 +244,18 @@ def detect_obsolete_options():
print("Option '{}' is not empty. Please unset '{}'".format(alternative, deprecated))
exit(2)
# Check for 'manylinux1' in the 'CIBW_BUILD' and 'CIBW_SKIP' options
for deprecated in ['CIBW_BUILD', 'CIBW_SKIP']:
if deprecated in os.environ and 'manylinux1' in os.environ[deprecated]:
print("Build identifiers with 'manylinux1' been deprecated. Replacing all occurences of 'manylinux1' by 'manylinux' in the option '{}'".format(deprecated))
os.environ[deprecated] = os.environ[deprecated].replace('manylinux1', 'manylinux')
# Check for deprecated identifiers in 'CIBW_BUILD' and 'CIBW_SKIP' options
for option in ['CIBW_BUILD', 'CIBW_SKIP']:
for deprecated, alternative in [('manylinux1', 'manylinux'),
('macosx_10_6_intel', 'macosx_x86_64'),
('macosx_10_9_x86_64', 'macosx_x86_64')]:
if option in os.environ and deprecated in os.environ[option]:
print("Build identifiers with '{deprecated}' have been deprecated. Replacing all occurences of '{deprecated}' with '{alternative}' in the option '{option}'".format(
deprecated=deprecated,
alternative=alternative,
option=option,
))
os.environ[option] = os.environ[option].replace(deprecated, alternative)
def print_preamble(platform, build_options):
@@ -210,10 +268,9 @@ def print_preamble(platform, build_options):
print('cibuildwheel version %s\n' % cibuildwheel.__version__)
print('Build options:')
print(' platform: %r' % platform)
for option, value in sorted(build_options.items()):
for option, value in sorted(build_options._asdict().items()):
print(' %s: %r' % (option, value))
warnings = detect_warnings(platform, build_options)
@@ -244,7 +301,7 @@ def detect_warnings(platform, build_options):
# warn about deprecated {python} and {pip}
for option_name in ['test_command', 'before_build']:
option_value = build_options.get(option_name)
option_value = getattr(build_options, option_name)
if option_value:
if '{python}' in option_value or '{pip}' in option_value:
+7 -8
View File
@@ -1,9 +1,12 @@
import subprocess, shlex, sys
import shlex
import subprocess
from collections import namedtuple
import bashlex
NodeExecutionContext = namedtuple('NodeExecutionContext', ['environment', 'input'])
def evaluate(value, environment):
if not value:
# empty string evaluates to empty string
@@ -16,9 +19,9 @@ def evaluate(value, environment):
raise ValueError('"%s" has too many parts' % value)
value_word_node = command_node.parts[0]
return evaluate_node(
value_word_node,
value_word_node,
context=NodeExecutionContext(environment=environment, input=value)
)
@@ -60,12 +63,8 @@ def evaluate_word_node(node, context):
def evaluate_command_node(node, context):
words = [evaluate_node(part, context=context) for part in node.parts]
command = ' '.join(words)
output = subprocess.check_output(shlex.split(command), env=context.environment)
return subprocess.check_output(shlex.split(command), env=context.environment, universal_newlines=True)
if sys.version_info[0] >= 3:
return output.decode('utf8', 'replace')
else:
return output
def evaluate_parameter_node(node, context):
return context.environment.get(node.value, '')
+3 -2
View File
@@ -1,4 +1,5 @@
import bashlex
from . import bashlex_eval
@@ -41,7 +42,7 @@ def split_env_items(env_string):
return result
class EnvironmentAssignment(object):
class EnvironmentAssignment:
def __init__(self, assignment):
name, equals, value = assignment.partition('=')
if not equals:
@@ -60,7 +61,7 @@ class EnvironmentAssignment(object):
return '%s=%s' % (self.name, self.value)
class ParsedEnvironment(object):
class ParsedEnvironment:
def __init__(self, assignments):
self.assignments = assignments
+259 -138
View File
@@ -1,179 +1,300 @@
from __future__ import print_function
import os, subprocess, sys, uuid
import os
import platform
import shlex
import subprocess
import sys
import textwrap
import uuid
from collections import namedtuple
from .util import prepare_command, get_build_verbosity_extra_flags
try:
from shlex import quote as shlex_quote
except ImportError:
from pipes import quote as shlex_quote
from .util import (
get_build_verbosity_extra_flags,
prepare_command,
BuildOptions
)
def call(args, input=None, universal_newlines=False):
print('+ ' + ' '.join(shlex.quote(a) for a in args))
subprocess.run(
args, input=input, universal_newlines=universal_newlines, check=True
)
def matches_platform(identifier):
pm = platform.machine()
if pm == "x86_64":
# x86_64 machines can run i686 docker containers
if identifier.endswith('x86_64') or identifier.endswith('i686'):
return True
elif pm == "i686":
if identifier.endswith('i686'):
return True
elif pm == "aarch64":
if identifier.endswith('aarch64'):
return True
elif pm == "ppc64le":
if identifier.endswith('ppc64le'):
return True
elif pm == "s390x":
if identifier.endswith('s390x'):
return True
return False
def get_python_configurations(build_selector):
PythonConfiguration = namedtuple('PythonConfiguration', ['identifier', 'path'])
PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'identifier', 'path'])
python_configurations = [
PythonConfiguration(identifier='cp27-manylinux_x86_64', path='/opt/python/cp27-cp27m'),
PythonConfiguration(identifier='cp27-manylinux_x86_64', path='/opt/python/cp27-cp27mu'),
PythonConfiguration(identifier='cp35-manylinux_x86_64', path='/opt/python/cp35-cp35m'),
PythonConfiguration(identifier='cp36-manylinux_x86_64', path='/opt/python/cp36-cp36m'),
PythonConfiguration(identifier='cp37-manylinux_x86_64', path='/opt/python/cp37-cp37m'),
PythonConfiguration(identifier='cp38-manylinux_x86_64', path='/opt/python/cp38-cp38'),
PythonConfiguration(identifier='cp27-manylinux_i686', path='/opt/python/cp27-cp27m'),
PythonConfiguration(identifier='cp27-manylinux_i686', path='/opt/python/cp27-cp27mu'),
PythonConfiguration(identifier='cp35-manylinux_i686', path='/opt/python/cp35-cp35m'),
PythonConfiguration(identifier='cp36-manylinux_i686', path='/opt/python/cp36-cp36m'),
PythonConfiguration(identifier='cp37-manylinux_i686', path='/opt/python/cp37-cp37m'),
PythonConfiguration(identifier='cp38-manylinux_i686', path='/opt/python/cp38-cp38'),
PythonConfiguration(version='2.7', identifier='cp27-manylinux_x86_64', path='/opt/python/cp27-cp27m'),
PythonConfiguration(version='2.7', identifier='cp27-manylinux_x86_64', path='/opt/python/cp27-cp27mu'),
PythonConfiguration(version='3.5', identifier='cp35-manylinux_x86_64', path='/opt/python/cp35-cp35m'),
PythonConfiguration(version='3.6', identifier='cp36-manylinux_x86_64', path='/opt/python/cp36-cp36m'),
PythonConfiguration(version='3.7', identifier='cp37-manylinux_x86_64', path='/opt/python/cp37-cp37m'),
PythonConfiguration(version='3.8', identifier='cp38-manylinux_x86_64', path='/opt/python/cp38-cp38'),
PythonConfiguration(version='2.7', identifier='cp27-manylinux_i686', path='/opt/python/cp27-cp27m'),
PythonConfiguration(version='2.7', identifier='cp27-manylinux_i686', path='/opt/python/cp27-cp27mu'),
PythonConfiguration(version='3.5', identifier='cp35-manylinux_i686', path='/opt/python/cp35-cp35m'),
PythonConfiguration(version='3.6', identifier='cp36-manylinux_i686', path='/opt/python/cp36-cp36m'),
PythonConfiguration(version='3.7', identifier='cp37-manylinux_i686', path='/opt/python/cp37-cp37m'),
PythonConfiguration(version='3.8', identifier='cp38-manylinux_i686', path='/opt/python/cp38-cp38'),
PythonConfiguration(version='2.7', identifier='pp27-manylinux_x86_64', path='/opt/python/pp27-pypy_73'),
PythonConfiguration(version='3.6', identifier='pp36-manylinux_x86_64', path='/opt/python/pp36-pypy36_pp73'),
PythonConfiguration(version='3.5', identifier='cp35-manylinux_aarch64', path='/opt/python/cp35-cp35m'),
PythonConfiguration(version='3.6', identifier='cp36-manylinux_aarch64', path='/opt/python/cp36-cp36m'),
PythonConfiguration(version='3.7', identifier='cp37-manylinux_aarch64', path='/opt/python/cp37-cp37m'),
PythonConfiguration(version='3.8', identifier='cp38-manylinux_aarch64', path='/opt/python/cp38-cp38'),
PythonConfiguration(version='3.5', identifier='cp35-manylinux_ppc64le', path='/opt/python/cp35-cp35m'),
PythonConfiguration(version='3.6', identifier='cp36-manylinux_ppc64le', path='/opt/python/cp36-cp36m'),
PythonConfiguration(version='3.7', identifier='cp37-manylinux_ppc64le', path='/opt/python/cp37-cp37m'),
PythonConfiguration(version='3.8', identifier='cp38-manylinux_ppc64le', path='/opt/python/cp38-cp38'),
PythonConfiguration(version='3.5', identifier='cp35-manylinux_s390x', path='/opt/python/cp35-cp35m'),
PythonConfiguration(version='3.6', identifier='cp36-manylinux_s390x', path='/opt/python/cp36-cp36m'),
PythonConfiguration(version='3.7', identifier='cp37-manylinux_s390x', path='/opt/python/cp37-cp37m'),
PythonConfiguration(version='3.8', identifier='cp38-manylinux_s390x', path='/opt/python/cp38-cp38'),
]
# skip builds as required
return [c for c in python_configurations if build_selector(c.identifier)]
return [c for c in python_configurations if matches_platform(c.identifier) and build_selector(c.identifier)]
def build(project_dir, output_dir, test_command, test_requires, test_extras, before_build, build_verbosity, build_selector, environment, manylinux_images):
def build(options: BuildOptions):
try:
subprocess.check_call(['docker', '--version'])
except:
except Exception:
print('cibuildwheel: Docker not found. Docker is required to run Linux builds. '
'If you\'re building on Travis CI, add `services: [docker]` to your .travis.yml.'
'If you\'re building on Circle CI in Linux, add a `setup_remote_docker` step to your .circleci/config.yml',
file=sys.stderr)
exit(2)
python_configurations = get_python_configurations(build_selector)
python_configurations = get_python_configurations(options.build_selector)
platforms = [
('manylinux_x86_64', manylinux_images['x86_64']),
('manylinux_i686', manylinux_images['i686']),
('cp', 'manylinux_x86_64', options.manylinux_images['x86_64']),
('cp', 'manylinux_i686', options.manylinux_images['i686']),
('cp', 'manylinux_aarch64', options.manylinux_images['aarch64']),
('cp', 'manylinux_ppc64le', options.manylinux_images['ppc64le']),
('cp', 'manylinux_s390x', options.manylinux_images['s390x']),
('pp', 'manylinux_x86_64', options.manylinux_images['pypy_x86_64']),
]
for platform_tag, docker_image in platforms:
platform_configs = [c for c in python_configurations if c.identifier.endswith(platform_tag)]
if not os.path.realpath(options.package_dir).startswith(os.path.realpath('.')):
raise Exception('package_dir must be inside the working directory')
container_package_dir = os.path.join('/project', os.path.relpath(options.package_dir, '.'))
for implementation, platform_tag, docker_image in platforms:
platform_configs = [c for c in python_configurations if c.identifier.startswith(implementation) and c.identifier.endswith(platform_tag)]
if not platform_configs:
continue
bash_script = '''
set -o errexit
set -o xtrace
mkdir /output
cd /project
container_name = 'cibuildwheel-{}'.format(uuid.uuid4())
{environment_exports}
try:
call(['docker', 'create',
'--env', 'CIBUILDWHEEL',
'--name', container_name,
'-i',
'-v', '/:/host', # ignored on CircleCI
docker_image,
'/bin/bash'])
for PYBIN in {pybin_paths}; do
# Setup
rm -rf /tmp/built_wheel
rm -rf /tmp/delocated_wheels
mkdir /tmp/built_wheel
mkdir /tmp/delocated_wheels
call(['docker', 'cp', '.', container_name + ':/project'])
if [ ! -z {before_build} ]; then
PATH="$PYBIN:$PATH" sh -c {before_build}
fi
call(['docker', 'start', container_name])
# Build that wheel
PATH="$PYBIN:$PATH" "$PYBIN/pip" wheel . -w /tmp/built_wheel --no-deps {build_verbosity_flag}
built_wheel=(/tmp/built_wheel/*.whl)
for config in platform_configs:
if options.dependency_constraints:
constraints_file = options.dependency_constraints.get_for_python_version(config.version)
# Delocate the wheel
# NOTE: 'built_wheel' here is a bash array of glob matches; "$built_wheel" returns
# the first element
if [[ "$built_wheel" == *none-any.whl ]]; then
# pure python wheel - just copy
mv "$built_wheel" /tmp/delocated_wheels
else
auditwheel repair "$built_wheel" -w /tmp/delocated_wheels
fi
delocated_wheels=(/tmp/delocated_wheels/*.whl)
# `docker cp` causes 'no space left on device' error when
# a container is running and the host filesystem is
# mounted. https://github.com/moby/moby/issues/38995
# Use `docker exec` instead.
with open(constraints_file, 'rb') as f:
call(
['docker', 'exec', '-i', container_name, 'sh', '-c', 'cat > /constraints.txt'],
input=f.read(),
)
if [ ! -z {test_command} ]; then
# Set up a virtual environment to install and test from, to make sure
# there are no dependencies that were pulled in at build time.
"$PYBIN/pip" install virtualenv
venv_dir=`mktemp -d`/venv
"$PYBIN/python" -m virtualenv "$venv_dir"
call(
['docker', 'exec', '-i', container_name, '/bin/bash'],
universal_newlines=True,
input='''
# give xtrace output an extra level of indent inside docker
PS4=' + '
# run the tests in a subshell to keep that `activate`
# script from polluting the env
(
source "$venv_dir/bin/activate"
set -o errexit
set -o xtrace
mkdir -p /output
cd /project
echo "Running tests using `which python`"
PYBIN="{config_python_bin}"
# Install the wheel we just built
# Note: If auditwheel produced two wheels, it's because the earlier produced wheel
# conforms to multiple manylinux standards. These multiple versions of the wheel are
# functionally the same, differing only in name, wheel metadata, and possibly include
# different external shared libraries. so it doesn't matter which one we run the tests on.
# Let's just pick the first one.
pip install "${{delocated_wheels[0]}}"{test_extras}
export PATH="$PYBIN:$PATH"
{environment_exports}
# Install any requirements to run the tests
if [ ! -z "{test_requires}" ]; then
pip install {test_requires}
# check the active python and pip are in PYBIN
if [ "$(which pip)" != "$PYBIN/pip" ]; then
echo "cibuildwheel: python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it."
exit 1
fi
if [ "$(which python)" != "$PYBIN/python" ]; then
echo "cibuildwheel: pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it."
exit 1
fi
# Run the tests from a different directory
pushd $HOME
sh -c {test_command}
popd
if [ ! -z {before_build} ]; then
sh -c {before_build}
fi
# Build the wheel
rm -rf /tmp/built_wheel
mkdir /tmp/built_wheel
pip wheel {package_dir} -w /tmp/built_wheel --no-deps {build_verbosity_flag}
built_wheel=(/tmp/built_wheel/*.whl)
# repair the wheel
rm -rf /tmp/repaired_wheels
mkdir /tmp/repaired_wheels
# NOTE: 'built_wheel' here is a bash array of glob matches; "$built_wheel" returns
# the first element
if [[ "$built_wheel" == *none-any.whl ]] || [ -z {repair_command} ]; then
# pure Python wheel or empty repair command
mv "$built_wheel" /tmp/repaired_wheels
else
sh -c {repair_command} repair_command "$built_wheel"
fi
repaired_wheels=(/tmp/repaired_wheels/*.whl)
if [ ! -z {test_command} ]; then
# Set up a virtual environment to install and test from, to make sure
# there are no dependencies that were pulled in at build time.
pip install {dependency_install_flags} virtualenv
venv_dir=`mktemp -d`/venv
python -m virtualenv --no-download "$venv_dir"
export __CIBW_VIRTUALENV_PATH__=$venv_dir
# run the tests in a subshell to keep that `activate`
# script from polluting the env
(
source "$venv_dir/bin/activate"
echo "Running tests using `which python`"
if [ ! -z {before_test} ]; then
sh -c {before_test}
fi
# Install the wheel we just built
# Note: If auditwheel produced two wheels, it's because the earlier produced wheel
# conforms to multiple manylinux standards. These multiple versions of the wheel are
# functionally the same, differing only in name, wheel metadata, and possibly include
# different external shared libraries. so it doesn't matter which one we run the tests on.
# Let's just pick the first one.
pip install "${{repaired_wheels[0]}}"{test_extras}
# Install any requirements to run the tests
if [ ! -z "{test_requires}" ]; then
pip install {test_requires}
fi
# Run the tests from a different directory
pushd $HOME
sh -c {test_command}
popd
)
# exit if tests failed (needed for older bash versions)
if [ $? -ne 0 ]; then
exit 1;
fi
# clean up
rm -rf "$venv_dir"
fi
# we're all done here; move it to output
mv "${{repaired_wheels[@]}}" /output
for repaired_wheel in "${{repaired_wheels[@]}}"; do
chown {uid}:{gid} "/output/$(basename "$repaired_wheel")"
done
'''.format(
config_python_bin=config.path + '/bin',
package_dir=container_package_dir,
test_requires=' '.join(options.test_requires),
test_extras=options.test_extras,
test_command=shlex.quote(
prepare_command(options.test_command, project='/project', package=container_package_dir) if options.test_command else ''
),
before_build=shlex.quote(
prepare_command(options.before_build, project='/project', package=container_package_dir) if options.before_build else ''
),
build_verbosity_flag=' '.join(get_build_verbosity_extra_flags(options.build_verbosity)),
repair_command=shlex.quote(
prepare_command(options.repair_command, wheel='"$1"', dest_dir='/tmp/repaired_wheels') if options.repair_command else ''
),
environment_exports='\n'.join(options.environment.as_shell_commands()),
uid=os.getuid(),
gid=os.getgid(),
before_test=shlex.quote(
prepare_command(options.before_test, project='/project', package=container_package_dir) if options.before_test else ''
),
dependency_install_flags='-c /constraints.txt' if options.dependency_constraints else '',
)
# exit if tests failed (needed for older bash versions)
if [ $? -ne 0 ]; then
exit 1;
fi
)
# clean up
rm -rf "$venv_dir"
fi
# we're all done here; move it to output
mv "${{delocated_wheels[@]}}" /output
for delocated_wheel in "${{delocated_wheels[@]}}"; do chown {uid}:{gid} "/output/$(basename "$delocated_wheel")"; done
done
'''.format(
pybin_paths=' '.join(c.path+'/bin' for c in platform_configs),
test_requires=' '.join(test_requires),
test_extras=test_extras,
test_command=shlex_quote(
prepare_command(test_command, project='/project') if test_command else ''
),
before_build=shlex_quote(
prepare_command(before_build, project='/project') if before_build else ''
),
build_verbosity_flag=' '.join(get_build_verbosity_extra_flags(build_verbosity)),
environment_exports='\n'.join(environment.as_shell_commands()),
uid=os.getuid(),
gid=os.getgid(),
)
def run_docker(command, stdin_str=None):
print('docker command: docker {}'.format(' '.join(map(shlex_quote, command))))
if stdin_str is None:
subprocess.check_call(['docker'] + command)
else:
args = ['docker'] + command
process = subprocess.Popen(args, stdin=subprocess.PIPE, universal_newlines=True)
try:
process.communicate(stdin_str)
except KeyboardInterrupt:
process.kill()
process.wait()
if process.returncode != 0:
raise subprocess.CalledProcessError(process.returncode, args)
container_name = 'cibuildwheel-{}'.format(uuid.uuid4())
try:
run_docker(['create',
'--env', 'CIBUILDWHEEL',
'--name', container_name,
'-i',
'-v', '/:/host', # ignored on Circle
docker_image, '/bin/bash'])
run_docker(['cp', os.path.abspath(project_dir) + '/.', container_name + ':/project'])
run_docker(['start', '-i', '-a', container_name], stdin_str=bash_script)
run_docker(['cp', container_name + ':/output/.', os.path.abspath(output_dir)])
except subprocess.CalledProcessError:
# copy the output back into the host
call(['docker', 'cp',
container_name + ':/output/.',
os.path.abspath(options.output_dir)])
except subprocess.CalledProcessError as error:
troubleshoot(options.package_dir, error)
exit(1)
finally:
# Still gets executed, even when 'exit(1)' gets called
run_docker(['rm', '--force', '-v', container_name])
call(['docker', 'rm', '--force', '-v', container_name])
def troubleshoot(package_dir, error):
if (isinstance(error, subprocess.CalledProcessError) and 'exec' in error.cmd):
# the bash script failed
print('Checking for common errors...')
so_files = []
for root, dirs, files in os.walk(package_dir):
for name in files:
_, ext = os.path.splitext(name)
if ext == '.so':
so_files.append(os.path.join(root, name))
if so_files:
print(textwrap.dedent('''
NOTE: Shared object (.so) files found in this project.
These files might be built against the wrong OS, causing problems with
auditwheel.
If you're using Cython and have previously done an in-place build,
remove those build files (*.so and *.c) before starting cibuildwheel.
'''))
print(' Files detected:')
print('\n'.join([' ' + f for f in so_files]))
print('')
+193 -104
View File
@@ -1,161 +1,250 @@
from __future__ import print_function
import os
import shlex
import shutil
import subprocess
import sys
import tempfile
import os, subprocess, shlex, sys, shutil
from collections import namedtuple
from glob import glob
try:
from shlex import quote as shlex_quote
except ImportError:
from pipes import quote as shlex_quote
from .util import prepare_command, get_build_verbosity_extra_flags
from .util import (
download,
get_build_verbosity_extra_flags,
prepare_command,
get_pip_script,
BuildOptions
)
def call(args, env=None, cwd=None, shell=False):
# print the command executing for the logs
if shell:
print('+ %s' % args)
else:
print('+ ' + ' '.join(shlex.quote(a) for a in args))
return subprocess.check_call(args, env=env, cwd=cwd, shell=shell)
def get_python_configurations(build_selector):
PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'identifier', 'url'])
python_configurations = [
PythonConfiguration(version='2.7', identifier='cp27-macosx_10_6_intel', url='https://www.python.org/ftp/python/2.7.17/python-2.7.17-macosx10.6.pkg'),
PythonConfiguration(version='3.5', identifier='cp35-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.5.4/python-3.5.4-macosx10.6.pkg'),
PythonConfiguration(version='3.6', identifier='cp36-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.6.8/python-3.6.8-macosx10.6.pkg'),
PythonConfiguration(version='3.7', identifier='cp37-macosx_10_6_intel', url='https://www.python.org/ftp/python/3.7.5/python-3.7.5-macosx10.6.pkg'),
PythonConfiguration(version='3.8', identifier='cp38-macosx_10_9_x86_64', url='https://www.python.org/ftp/python/3.8.0/python-3.8.0-macosx10.9.pkg'),
# CPython
PythonConfiguration(version='2.7', identifier='cp27-macosx_x86_64', url='https://www.python.org/ftp/python/2.7.18/python-2.7.18-macosx10.9.pkg'),
PythonConfiguration(version='3.5', identifier='cp35-macosx_x86_64', url='https://www.python.org/ftp/python/3.5.4/python-3.5.4-macosx10.6.pkg'),
PythonConfiguration(version='3.6', identifier='cp36-macosx_x86_64', url='https://www.python.org/ftp/python/3.6.8/python-3.6.8-macosx10.9.pkg'),
PythonConfiguration(version='3.7', identifier='cp37-macosx_x86_64', url='https://www.python.org/ftp/python/3.7.7/python-3.7.7-macosx10.9.pkg'),
PythonConfiguration(version='3.8', identifier='cp38-macosx_x86_64', url='https://www.python.org/ftp/python/3.8.2/python-3.8.2-macosx10.9.pkg'),
# PyPy
PythonConfiguration(version='2.7', identifier='pp27-macosx_x86_64', url='https://downloads.python.org/pypy/pypy2.7-v7.3.1-osx64.tar.bz2'),
PythonConfiguration(version='3.6', identifier='pp36-macosx_x86_64', url='https://downloads.python.org/pypy/pypy3.6-v7.3.1-osx64.tar.bz2'),
]
# skip builds as required
return [c for c in python_configurations if build_selector(c.identifier)]
def build(project_dir, output_dir, test_command, test_requires, test_extras, before_build, build_verbosity, build_selector, environment):
python_configurations = get_python_configurations(build_selector)
get_pip_url = 'https://bootstrap.pypa.io/get-pip.py'
get_pip_script = '/tmp/get-pip.py'
SYMLINKS_DIR = '/tmp/cibw_bin'
pkgs_output = subprocess.check_output(['pkgutil', '--pkgs'])
if sys.version_info[0] >= 3:
pkgs_output = pkgs_output.decode('utf8')
installed_system_packages = pkgs_output.splitlines()
def call(args, env=None, cwd=None, shell=False):
# print the command executing for the logs
if shell:
print('+ %s' % args)
else:
print('+ ' + ' '.join(shlex_quote(a) for a in args))
def make_symlinks(installation_bin_path, python_executable, pip_executable):
assert os.path.exists(os.path.join(installation_bin_path, python_executable))
return subprocess.check_call(args, env=env, cwd=cwd, shell=shell)
# Python bin folders on Mac don't symlink `python3` to `python`, and neither
# does PyPy for `pypy` or `pypy3`, so we do that so `python` and `pip` always
# point to the active configuration.
if os.path.exists(SYMLINKS_DIR):
shutil.rmtree(SYMLINKS_DIR)
os.makedirs(SYMLINKS_DIR)
abs_project_dir = os.path.abspath(project_dir)
os.symlink(os.path.join(installation_bin_path, python_executable), os.path.join(SYMLINKS_DIR, 'python'))
os.symlink(os.path.join(installation_bin_path, python_executable + '-config'), os.path.join(SYMLINKS_DIR, 'python-config'))
os.symlink(os.path.join(installation_bin_path, pip_executable), os.path.join(SYMLINKS_DIR, 'pip'))
# get latest pip once and for all
call(['curl', '-L', '-o', get_pip_script, get_pip_url])
def install_cpython(version, url):
installed_system_packages = subprocess.check_output(['pkgutil', '--pkgs'], universal_newlines=True).splitlines()
# if this version of python isn't installed, get it from python.org and install
python_package_identifier = 'org.python.Python.PythonFramework-{}'.format(version)
if python_package_identifier not in installed_system_packages:
# download the pkg
download(url, '/tmp/Python.pkg')
# install
call(['sudo', 'installer', '-pkg', '/tmp/Python.pkg', '-target', '/'])
# patch open ssl
if version == '3.5':
open_ssl_patch_url = 'https://github.com/mayeut/patch-macos-python-openssl/releases/download/v1.0.2u/patch-macos-python-%s-openssl-v1.0.2u.tar.gz' % version
download(open_ssl_patch_url, '/tmp/python-patch.tar.gz')
call(['sudo', 'tar', '-C', '/Library/Frameworks/Python.framework/Versions/{}/'.format(version), '-xmf', '/tmp/python-patch.tar.gz'])
installation_bin_path = '/Library/Frameworks/Python.framework/Versions/{}/bin'.format(version)
python_executable = 'python3' if version[0] == '3' else 'python'
pip_executable = 'pip3' if version[0] == '3' else 'pip'
make_symlinks(installation_bin_path, python_executable, pip_executable)
return installation_bin_path
def install_pypy(version, url):
pypy_tar_bz2 = url.rsplit('/', 1)[-1]
assert pypy_tar_bz2.endswith(".tar.bz2")
pypy_base_filename = os.path.splitext(os.path.splitext(pypy_tar_bz2)[0])[0]
installation_path = os.path.join('/tmp', pypy_base_filename)
if not os.path.exists(installation_path):
download(url, os.path.join("/tmp", pypy_tar_bz2))
call(['tar', '-C', '/tmp', '-xf', os.path.join("/tmp", pypy_tar_bz2)])
installation_bin_path = os.path.join(installation_path, 'bin')
python_executable = 'pypy3' if version[0] == '3' else 'pypy'
pip_executable = 'pip3' if version[0] == '3' else 'pip'
make_symlinks(installation_bin_path, python_executable, pip_executable)
return installation_bin_path
def setup_python(python_configuration, dependency_constraint_flags, environment):
if python_configuration.identifier.startswith('cp'):
installation_bin_path = install_cpython(python_configuration.version, python_configuration.url)
elif python_configuration.identifier.startswith('pp'):
installation_bin_path = install_pypy(python_configuration.version, python_configuration.url)
else:
raise ValueError("Unknown Python implementation")
env = os.environ.copy()
env['PATH'] = os.pathsep.join([
SYMLINKS_DIR,
installation_bin_path,
env['PATH'],
])
# Fix issue with site.py setting the wrong `sys.prefix`, `sys.exec_prefix`,
# `sys.path`, ... for PyPy: https://foss.heptapod.net/pypy/pypy/issues/3175
# Also fix an issue with the shebang of installed scripts inside the
# testing virtualenv- see https://github.com/theacodes/nox/issues/44 and
# https://github.com/pypa/virtualenv/issues/620
# Also see https://github.com/python/cpython/pull/9516
env.pop('__PYVENV_LAUNCHER__', None)
env = environment.as_dictionary(prev_environment=env)
# check what version we're on
call(['which', 'python'], env=env)
call(['python', '--version'], env=env)
which_python = subprocess.check_output(['which', 'python'], env=env, universal_newlines=True).strip()
if which_python != '/tmp/cibw_bin/python':
print("cibuildwheel: python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it.", file=sys.stderr)
exit(1)
# install pip & wheel
call(['python', get_pip_script] + dependency_constraint_flags, env=env, cwd="/tmp")
assert os.path.exists(os.path.join(installation_bin_path, 'pip'))
call(['which', 'pip'], env=env)
call(['pip', '--version'], env=env)
which_pip = subprocess.check_output(['which', 'pip'], env=env, universal_newlines=True).strip()
if which_pip != '/tmp/cibw_bin/pip':
print("cibuildwheel: pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it.", file=sys.stderr)
exit(1)
call(['pip', 'install', '--upgrade', 'setuptools', 'wheel', 'delocate'] + dependency_constraint_flags, env=env)
# Set MACOSX_DEPLOYMENT_TARGET to 10.9, if the user didn't set it.
# CPython 3.5 defaults to 10.6, and pypy defaults to 10.7, causing
# inconsistencies if it's left unset.
env.setdefault('MACOSX_DEPLOYMENT_TARGET', '10.9')
if python_configuration.version == '3.5':
# Cross-compilation platform override - CPython 3.5 has an
# i386/x86_64 version of Python, but we only want a x64_64 build
env.setdefault('_PYTHON_HOST_PLATFORM', 'macosx-10.9-x86_64')
# https://github.com/python/cpython/blob/a5ed2fe0eedefa1649aa93ee74a0bafc8e628a10/Lib/_osx_support.py#L260
env.setdefault('ARCHFLAGS', '-arch x86_64')
return env
def build(options: BuildOptions):
temp_dir = tempfile.mkdtemp(prefix='cibuildwheel')
built_wheel_dir = os.path.join(temp_dir, 'built_wheel')
repaired_wheel_dir = os.path.join(temp_dir, 'repaired_wheel')
python_configurations = get_python_configurations(options.build_selector)
for config in python_configurations:
# if this version of python isn't installed, get it from python.org and install
python_package_identifier = 'org.python.Python.PythonFramework-%s' % config.version
if python_package_identifier not in installed_system_packages:
# download the pkg
call(['curl', '-L', '-o', '/tmp/Python.pkg', config.url])
# install
call(['sudo', 'installer', '-pkg', '/tmp/Python.pkg', '-target', '/'])
# patch open ssl
if config.version == '3.5':
call(['curl', '-fsSLo', '/tmp/python-patch.tar.gz', 'https://github.com/mayeut/patch-macos-python-openssl/releases/download/v1.0.2t/patch-macos-python-%s-openssl-v1.0.2t.tar.gz' % config.version])
call(['sudo', 'tar', '-C', '/Library/Frameworks/Python.framework/Versions/%s/' % config.version, '-xmf', '/tmp/python-patch.tar.gz'])
dependency_constraint_flags = []
if options.dependency_constraints:
dependency_constraint_flags = [
'-c', options.dependency_constraints.get_for_python_version(config.version)
]
installation_bin_path = '/Library/Frameworks/Python.framework/Versions/{}/bin'.format(config.version)
# Python bin folders on Mac don't symlink python3 to python, so we do that
# so `python` and `pip` always point to the active configuration.
if os.path.exists('/tmp/cibw_bin'):
shutil.rmtree('/tmp/cibw_bin')
os.makedirs('/tmp/cibw_bin')
if config.version[0] == '3':
os.symlink(os.path.join(installation_bin_path, 'python3'), '/tmp/cibw_bin/python')
os.symlink(os.path.join(installation_bin_path, 'python3-config'), '/tmp/cibw_bin/python-config')
os.symlink(os.path.join(installation_bin_path, 'pip3'), '/tmp/cibw_bin/pip')
env = os.environ.copy()
env['PATH'] = os.pathsep.join([
'/tmp/cibw_bin',
installation_bin_path,
env['PATH'],
])
env = environment.as_dictionary(prev_environment=env)
# check what version we're on
call(['which', 'python'], env=env)
call(['python', '--version'], env=env)
# install pip & wheel
call(['python', get_pip_script, '--no-setuptools', '--no-wheel'], env=env)
call(['pip', '--version'], env=env)
call(['pip', 'install', '--upgrade', 'setuptools'], env=env)
call(['pip', 'install', 'wheel'], env=env)
call(['pip', 'install', 'delocate'], env=env)
# setup dirs
if os.path.exists('/tmp/built_wheel'):
shutil.rmtree('/tmp/built_wheel')
os.makedirs('/tmp/built_wheel')
if os.path.exists('/tmp/delocated_wheel'):
shutil.rmtree('/tmp/delocated_wheel')
os.makedirs('/tmp/delocated_wheel')
env = setup_python(config, dependency_constraint_flags, options.environment)
# run the before_build command
if before_build:
before_build_prepared = prepare_command(before_build, project=abs_project_dir)
if options.before_build:
before_build_prepared = prepare_command(options.before_build, project='.', package=options.package_dir)
call(before_build_prepared, env=env, shell=True)
# build the wheel
call(['pip', 'wheel', abs_project_dir, '-w', '/tmp/built_wheel', '--no-deps'] + get_build_verbosity_extra_flags(build_verbosity), env=env)
built_wheel = glob('/tmp/built_wheel/*.whl')[0]
if os.path.exists(built_wheel_dir):
shutil.rmtree(built_wheel_dir)
os.makedirs(built_wheel_dir)
call(['pip', 'wheel', options.package_dir, '-w', built_wheel_dir, '--no-deps'] + get_build_verbosity_extra_flags(options.build_verbosity), env=env)
built_wheel = glob(os.path.join(built_wheel_dir, '*.whl'))[0]
if built_wheel.endswith('none-any.whl'):
# pure python wheel - just move
shutil.move(built_wheel, '/tmp/delocated_wheel')
# repair the wheel
if os.path.exists(repaired_wheel_dir):
shutil.rmtree(repaired_wheel_dir)
os.makedirs(repaired_wheel_dir)
if built_wheel.endswith('none-any.whl') or not options.repair_command:
# pure Python wheel or empty repair command
shutil.move(built_wheel, repaired_wheel_dir)
else:
# list the dependencies
call(['delocate-listdeps', built_wheel], env=env)
# rebuild the wheel with shared libraries included and place in output dir
call(['delocate-wheel', '-w', '/tmp/delocated_wheel', built_wheel], env=env)
delocated_wheel = glob('/tmp/delocated_wheel/*.whl')[0]
repair_command_prepared = prepare_command(options.repair_command, wheel=built_wheel, dest_dir=repaired_wheel_dir)
call(repair_command_prepared, env=env, shell=True)
repaired_wheel = glob(os.path.join(repaired_wheel_dir, '*.whl'))[0]
if test_command:
if options.test_command:
# set up a virtual environment to install and test from, to make sure
# there are no dependencies that were pulled in at build time.
call(['pip', 'install', 'virtualenv'], env=env)
call(['pip', 'install', 'virtualenv'] + dependency_constraint_flags, env=env)
venv_dir = tempfile.mkdtemp()
call(['python', '-m', 'virtualenv', venv_dir], env=env)
# Use --no-download to ensure determinism by using seed libraries
# built into virtualenv
call(['python', '-m', 'virtualenv', '--no-download', venv_dir], env=env)
virtualenv_env = env.copy()
virtualenv_env['PATH'] = os.pathsep.join([
os.path.join(venv_dir, 'bin'),
virtualenv_env['PATH'],
])
# Fix some weird issue with the shebang of installed scripts
# See https://github.com/theacodes/nox/issues/44 and https://github.com/pypa/virtualenv/issues/620
virtualenv_env.pop('__PYVENV_LAUNCHER__', None)
virtualenv_env["__CIBW_VIRTUALENV_PATH__"] = venv_dir
# check that we are using the Python from the virtual environment
call(['which', 'python'], env=virtualenv_env)
if options.before_test:
before_test_prepared = prepare_command(options.before_test, project='.', package=options.package_dir)
call(before_test_prepared, env=virtualenv_env, shell=True)
# install the wheel
call(['pip', 'install', delocated_wheel + test_extras], env=virtualenv_env)
call(['pip', 'install', repaired_wheel + options.test_extras], env=virtualenv_env)
# test the wheel
if test_requires:
call(['pip', 'install'] + test_requires, env=virtualenv_env)
if options.test_requires:
call(['pip', 'install'] + options.test_requires, env=virtualenv_env)
# run the tests from $HOME, with an absolute path in the command
# (this ensures that Python runs the tests against the installed wheel
# and not the repo code)
test_command_prepared = prepare_command(test_command, project=abs_project_dir)
test_command_prepared = prepare_command(
options.test_command,
project=os.path.abspath('.'),
package=os.path.abspath(options.package_dir)
)
call(test_command_prepared, cwd=os.environ['HOME'], env=virtualenv_env, shell=True)
# clean up
shutil.rmtree(venv_dir)
# we're all done here; move it to output (overwrite existing)
dst = os.path.join(output_dir, os.path.basename(delocated_wheel))
shutil.move(delocated_wheel, dst)
dst = os.path.join(options.output_dir, os.path.basename(repaired_wheel))
shutil.move(repaired_wheel, dst)
@@ -1,88 +0,0 @@
:: To build extensions for 64 bit Python 3, we need to configure environment
:: variables to use the MSVC 2010 C++ compilers from GRMSDKX_EN_DVD.iso of:
:: MS Windows SDK for Windows 7 and .NET Framework 4 (SDK v7.1)
::
:: To build extensions for 64 bit Python 2, we need to configure environment
:: variables to use the MSVC 2008 C++ compilers from GRMSDKX_EN_DVD.iso of:
:: MS Windows SDK for Windows 7 and .NET Framework 3.5 (SDK v7.0)
::
:: 32 bit builds, and 64-bit builds for 3.5 and beyond, do not require specific
:: environment configurations.
::
:: Note: this script needs to be run with the /E:ON and /V:ON flags for the
:: cmd interpreter, at least for (SDK v7.0)
::
:: More details at:
:: https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows
:: http://stackoverflow.com/a/13751649/163740
::
:: Author: Olivier Grisel
:: License: CC0 1.0 Universal: http://creativecommons.org/publicdomain/zero/1.0/
::
:: Notes about batch files for Python people:
::
:: Quotes in values are literally part of the values:
:: SET FOO="bar"
:: FOO is now five characters long: " b a r "
:: If you don't want quotes, don't include them on the right-hand side.
::
:: The CALL lines at the end of this file look redundant, but if you move them
:: outside of the IF clauses, they do not run properly in the SET_SDK_64==Y
:: case, I don't know why.
@ECHO OFF
SET COMMAND_TO_RUN=%*
SET WIN_SDK_ROOT=C:\Program Files\Microsoft SDKs\Windows
SET WIN_WDK=c:\Program Files (x86)\Windows Kits\10\Include\wdf
:: Extract the major and minor versions, and allow for the minor version to be
:: more than 9. This requires the version number to have two dots in it.
SET MAJOR_PYTHON_VERSION=%PYTHON_VERSION:~0,1%
IF "%PYTHON_VERSION:~3,1%" == "." (
SET MINOR_PYTHON_VERSION=%PYTHON_VERSION:~2,1%
) ELSE (
SET MINOR_PYTHON_VERSION=%PYTHON_VERSION:~2,2%
)
:: Based on the Python version, determine what SDK version to use, and whether
:: to set the SDK for 64-bit.
IF %MAJOR_PYTHON_VERSION% == 2 (
SET WINDOWS_SDK_VERSION="v7.0"
SET SET_SDK_64=Y
) ELSE (
IF %MAJOR_PYTHON_VERSION% == 3 (
SET WINDOWS_SDK_VERSION="v7.1"
IF %MINOR_PYTHON_VERSION% LEQ 4 (
SET SET_SDK_64=Y
) ELSE (
SET SET_SDK_64=N
IF EXIST "%WIN_WDK%" (
:: See: https://connect.microsoft.com/VisualStudio/feedback/details/1610302/
REN "%WIN_WDK%" 0wdf
)
)
) ELSE (
ECHO Unsupported Python version: "%MAJOR_PYTHON_VERSION%"
EXIT 1
)
)
IF %PYTHON_ARCH% == 64 (
IF %SET_SDK_64% == Y (
ECHO Configuring Windows SDK %WINDOWS_SDK_VERSION% for Python %MAJOR_PYTHON_VERSION% on a 64 bit architecture
SET DISTUTILS_USE_SDK=1
SET MSSdk=1
"%WIN_SDK_ROOT%\%WINDOWS_SDK_VERSION%\Setup\WindowsSdkVer.exe" -q -version:%WINDOWS_SDK_VERSION%
"%WIN_SDK_ROOT%\%WINDOWS_SDK_VERSION%\Bin\SetEnv.cmd" /x64 /release
ECHO Executing: %COMMAND_TO_RUN%
call %COMMAND_TO_RUN% || EXIT 1
) ELSE (
ECHO Using default MSVC build environment for 64 bit architecture
ECHO Executing: %COMMAND_TO_RUN%
call %COMMAND_TO_RUN% || EXIT 1
)
) ELSE (
ECHO Using default MSVC build environment for 32 bit architecture
ECHO Executing: %COMMAND_TO_RUN%
call %COMMAND_TO_RUN% || EXIT 1
)
@@ -0,0 +1,26 @@
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bin/update_constraints.py
#
appdirs==1.4.3 # via virtualenv
configparser==4.0.2 # via importlib-metadata
contextlib2==0.6.0.post1 # via importlib-metadata, importlib-resources, virtualenv
delocate==0.8.0 # via -r cibuildwheel/resources/constraints.in
distlib==0.3.0 # via virtualenv
filelock==3.0.12 # via virtualenv
importlib-metadata==1.6.0 # via importlib-resources, virtualenv
importlib-resources==1.5.0 # via virtualenv
pathlib2==2.3.5 # via importlib-metadata, importlib-resources, virtualenv
scandir==1.10.0 # via pathlib2
singledispatch==3.4.0.3 # via importlib-resources
six==1.14.0 # via pathlib2, virtualenv
typing==3.7.4.1 # via importlib-resources
virtualenv==20.0.18 # via -r cibuildwheel/resources/constraints.in
wheel==0.34.2 # via -r cibuildwheel/resources/constraints.in, delocate
zipp==1.2.0 # via importlib-metadata, importlib-resources
# The following packages are considered to be unsafe in a requirements file:
pip==20.1 # via -r cibuildwheel/resources/constraints.in
setuptools==44.1.0 # via -r cibuildwheel/resources/constraints.in
@@ -0,0 +1,20 @@
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bin/update_constraints.py
#
appdirs==1.4.3 # via virtualenv
delocate==0.8.0 # via -r cibuildwheel/resources/constraints.in
distlib==0.3.0 # via virtualenv
filelock==3.0.12 # via virtualenv
importlib-metadata==1.6.0 # via importlib-resources, virtualenv
importlib-resources==1.5.0 # via virtualenv
six==1.14.0 # via virtualenv
virtualenv==20.0.18 # via -r cibuildwheel/resources/constraints.in
wheel==0.34.2 # via -r cibuildwheel/resources/constraints.in, delocate
zipp==1.2.0 # via importlib-metadata, importlib-resources
# The following packages are considered to be unsafe in a requirements file:
pip==20.1 # via -r cibuildwheel/resources/constraints.in
setuptools==46.1.3 # via -r cibuildwheel/resources/constraints.in
@@ -0,0 +1,20 @@
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bin/update_constraints.py
#
appdirs==1.4.3 # via virtualenv
delocate==0.8.0 # via -r cibuildwheel/resources/constraints.in
distlib==0.3.0 # via virtualenv
filelock==3.0.12 # via virtualenv
importlib-metadata==1.6.0 # via importlib-resources, virtualenv
importlib-resources==1.5.0 # via virtualenv
six==1.14.0 # via virtualenv
virtualenv==20.0.18 # via -r cibuildwheel/resources/constraints.in
wheel==0.34.2 # via -r cibuildwheel/resources/constraints.in, delocate
zipp==3.1.0 # via importlib-metadata, importlib-resources
# The following packages are considered to be unsafe in a requirements file:
pip==20.1 # via -r cibuildwheel/resources/constraints.in
setuptools==46.1.3 # via -r cibuildwheel/resources/constraints.in
+5
View File
@@ -0,0 +1,5 @@
pip
setuptools
wheel
delocate
virtualenv
+19
View File
@@ -0,0 +1,19 @@
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bin/update_constraints.py
#
appdirs==1.4.3 # via virtualenv
delocate==0.8.0 # via -r cibuildwheel/resources/constraints.in
distlib==0.3.0 # via virtualenv
filelock==3.0.12 # via virtualenv
importlib-metadata==1.6.0 # via virtualenv
six==1.14.0 # via virtualenv
virtualenv==20.0.18 # via -r cibuildwheel/resources/constraints.in
wheel==0.34.2 # via -r cibuildwheel/resources/constraints.in, delocate
zipp==3.1.0 # via importlib-metadata
# The following packages are considered to be unsafe in a requirements file:
pip==20.1 # via -r cibuildwheel/resources/constraints.in
setuptools==46.1.3 # via -r cibuildwheel/resources/constraints.in
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,22 @@
[x86_64]
manylinux1 = quay.io/pypa/manylinux1_x86_64:2020-04-25-37c204c
manylinux2010 = quay.io/pypa/manylinux2010_x86_64:2020-04-29-0e1afc5
manylinux2014 = quay.io/pypa/manylinux2014_x86_64:2020-05-01-b37d76b
[i686]
manylinux1 = quay.io/pypa/manylinux1_i686:2020-04-25-37c204c
manylinux2010 = quay.io/pypa/manylinux2010_i686:2020-04-29-0e1afc5
manylinux2014 = quay.io/pypa/manylinux2014_i686:2020-05-01-b37d76b
[pypy_x86_64]
manylinux2010 = pypywheels/manylinux2010-pypy_x86_64:2020-04-25-eb2cdff
[aarch64]
manylinux2014 = quay.io/pypa/manylinux2014_aarch64:2020-05-01-b37d76b
[ppc64le]
manylinux2014 = quay.io/pypa/manylinux2014_ppc64le:2020-05-01-b37d76b
[s390x]
manylinux2014 = quay.io/pypa/manylinux2014_s390x:2020-05-01-b37d76b
+7
View File
@@ -0,0 +1,7 @@
--- a/lib_pypy/_sysconfigdata.py Tue Jan 28 22:54:59 2020 +0200
+++ b/lib_pypy/_sysconfigdata.py Wed Jan 29 19:21:23 2020 +0200
@@ -47,4 +47,5 @@
build_time_vars['CC'] += ' -arch %s' % (arch,)
if "CXX" in build_time_vars:
build_time_vars['CXX'] += ' -arch %s' % (arch,)
+ build_time_vars['MACOSX_DEPLOYMENT_TARGET'] = '10.7'
+87 -8
View File
@@ -1,15 +1,20 @@
import os
import urllib.request
from fnmatch import fnmatch
import warnings
from time import sleep
from typing import NamedTuple, List, Optional, Dict
from .environment import ParsedEnvironment
def prepare_command(command, project):
def prepare_command(command, **kwargs):
'''
Preprocesses a command by expanding variables like {project}.
Preprocesses a command by expanding variables like {python}.
For example, used in the test_command option, to specify the path to the
tests directory.
For example, used in the test_command option to specify the path to the
project's root.
'''
return command.format(python='python', pip='pip', project=project)
return command.format(python='python', pip='pip', **kwargs)
def get_build_verbosity_extra_flags(level):
@@ -21,7 +26,7 @@ def get_build_verbosity_extra_flags(level):
return []
class BuildSelector(object):
class BuildSelector:
def __init__(self, build_config, skip_config):
self.build_patterns = build_config.split()
self.skip_patterns = skip_config.split()
@@ -36,7 +41,7 @@ class BuildSelector(object):
# Taken from https://stackoverflow.com/a/107717
class Unbuffered(object):
class Unbuffered:
def __init__(self, stream):
self.stream = stream
@@ -50,3 +55,77 @@ class Unbuffered(object):
def __getattr__(self, attr):
return getattr(self.stream, attr)
def download(url, dest):
print('+ Download ' + url + ' to ' + dest)
dest_dir = os.path.dirname(dest)
if not os.path.exists(dest_dir):
os.makedirs(dest_dir)
repeat_num = 3
for i in range(repeat_num):
try:
response = urllib.request.urlopen(url)
except Exception:
if i == repeat_num - 1:
raise
sleep(3)
continue
break
try:
with open(dest, 'wb') as file:
file.write(response.read())
finally:
response.close()
class DependencyConstraints:
def __init__(self, base_file_path):
assert os.path.exists(base_file_path)
self.base_file_path = os.path.abspath(base_file_path)
@classmethod
def with_defaults(cls):
return cls(
base_file_path=os.path.join(os.path.dirname(__file__), 'resources', 'constraints.txt')
)
def get_for_python_version(self, version):
version_parts = version.split('.')
# try to find a version-specific dependency file e.g. if
# ./constraints.txt is the base, look for ./constraints-python27.txt
base, ext = os.path.splitext(self.base_file_path)
specific = base + '-python{}{}'.format(version_parts[0], version_parts[1])
specific_file_path = specific + ext
if os.path.exists(specific_file_path):
return specific_file_path
else:
return self.base_file_path
BuildOptions = NamedTuple("BuildOptions", [
("package_dir", str),
("output_dir", str),
("test_command", Optional[str]),
("test_requires", List[str]),
("test_extras", str),
("before_build", Optional[str]),
("build_verbosity", int),
("build_selector", BuildSelector),
("repair_command", str),
("environment", ParsedEnvironment),
("before_test", str),
("dependency_constraints", Optional[DependencyConstraints]),
("manylinux_images", Optional[Dict[str, str]]),
])
"""
Replace this definition with a class-style NamedTuple in the
PEP526 style when Python 3.5 host support is dropped
"""
resources_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), 'resources'))
get_pip_script = os.path.join(resources_dir, 'get-pip.py')
+174 -111
View File
@@ -1,182 +1,245 @@
from __future__ import print_function
import os, tempfile, subprocess, shutil, sys
import os
import shutil
import subprocess
import sys
import tempfile
from collections import namedtuple
from glob import glob
from zipfile import ZipFile
try:
from shlex import quote as shlex_quote
except ImportError:
from pipes import quote as shlex_quote
try:
from urllib.request import urlopen
except ImportError:
from urllib2 import urlopen
from .util import prepare_command, get_build_verbosity_extra_flags
from .util import (
download,
get_build_verbosity_extra_flags,
prepare_command,
get_pip_script,
BuildOptions
)
IS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache')
IS_RUNNING_ON_TRAVIS = os.environ.get('TRAVIS_OS_NAME') == 'windows'
def get_python_path(config):
nuget_args = get_nuget_args(config)
return os.path.join(nuget_args[-1], nuget_args[0] + "." + config.version, "tools")
def shell(args, env=None, cwd=None):
print('+ ' + ' '.join(args))
return subprocess.check_call(' '.join(args), env=env, cwd=cwd, shell=True)
def get_nuget_args(configuration):
python_name = "python" if configuration.version[0] == '3' else "python2"
if configuration.arch == "32":
python_name = python_name + "x86"
return [python_name, "-Version", configuration.version, "-OutputDirectory", "C:/cibw/python"]
def get_nuget_args(version, arch):
python_name = 'python' if version[0] == '3' else 'python2'
if arch == '32':
python_name = python_name + 'x86'
return [python_name, '-Version', version, '-OutputDirectory', 'C:\\cibw\\python']
def get_python_configurations(build_selector):
PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'arch', 'identifier'])
PythonConfiguration = namedtuple('PythonConfiguration', ['version', 'arch', 'identifier', 'url'])
python_configurations = [
PythonConfiguration(version='2.7.17', arch="32", identifier='cp27-win32'),
PythonConfiguration(version='2.7.17', arch="64", identifier='cp27-win_amd64'),
PythonConfiguration(version='3.5.4', arch="32", identifier='cp35-win32'),
PythonConfiguration(version='3.5.4', arch="64", identifier='cp35-win_amd64'),
PythonConfiguration(version='3.6.8', arch="32", identifier='cp36-win32'),
PythonConfiguration(version='3.6.8', arch="64", identifier='cp36-win_amd64'),
PythonConfiguration(version='3.7.5', arch="32", identifier='cp37-win32'),
PythonConfiguration(version='3.7.5', arch="64", identifier='cp37-win_amd64'),
PythonConfiguration(version='3.8.0', arch="32", identifier='cp38-win32'),
PythonConfiguration(version='3.8.0', arch="64", identifier='cp38-win_amd64'),
# CPython
PythonConfiguration(version='2.7.18', arch='32', identifier='cp27-win32', url=None),
PythonConfiguration(version='2.7.18', arch='64', identifier='cp27-win_amd64', url=None),
PythonConfiguration(version='3.5.4', arch='32', identifier='cp35-win32', url=None),
PythonConfiguration(version='3.5.4', arch='64', identifier='cp35-win_amd64', url=None),
PythonConfiguration(version='3.6.8', arch='32', identifier='cp36-win32', url=None),
PythonConfiguration(version='3.6.8', arch='64', identifier='cp36-win_amd64', url=None),
PythonConfiguration(version='3.7.7', arch='32', identifier='cp37-win32', url=None),
PythonConfiguration(version='3.7.7', arch='64', identifier='cp37-win_amd64', url=None),
PythonConfiguration(version='3.8.2', arch='32', identifier='cp38-win32', url=None),
PythonConfiguration(version='3.8.2', arch='64', identifier='cp38-win_amd64', url=None),
# PyPy
PythonConfiguration(version='2.7', arch='32', identifier='pp27-win32', url='https://downloads.python.org/pypy/pypy2.7-v7.3.1-win32.zip'),
PythonConfiguration(version='3.6', arch='32', identifier='pp36-win32', url='https://downloads.python.org/pypy/pypy3.6-v7.3.1-win32.zip'),
]
if IS_RUNNING_ON_TRAVIS:
# cannot install VCForPython27.msi which is needed for compiling C software
# try with (and similar): msiexec /i VCForPython27.msi ALLUSERS=1 ACCEPT=YES /passive
python_configurations = [c for c in python_configurations if not c.version.startswith('2.7.')]
python_configurations = [c for c in python_configurations if not c.version.startswith('2.7')]
# skip builds as required
# skip builds as required
python_configurations = [c for c in python_configurations if build_selector(c.identifier)]
return python_configurations
def extract_zip(zip_src, dest):
with ZipFile(zip_src) as zip:
zip.extractall(dest)
def build(project_dir, output_dir, test_command, test_requires, test_extras, before_build, build_verbosity, build_selector, environment):
def simple_shell(args, env=None, cwd=None):
print('+ ' + ' '.join(args))
args = ['cmd', '/E:ON', '/V:ON', '/C'] + args
return subprocess.check_call(' '.join(args), env=env, cwd=cwd)
def download(url, dest):
print('+ Download ' + url + ' to ' + dest)
dest_dir = os.path.dirname(dest)
if not os.path.exists(dest_dir):
os.makedirs(dest_dir)
response = urlopen(url)
try:
with open(dest, 'wb') as file:
file.write(response.read())
finally:
response.close()
if IS_RUNNING_ON_AZURE or IS_RUNNING_ON_TRAVIS:
shell = simple_shell
def install_cpython(version, arch, nuget):
nuget_args = get_nuget_args(version, arch)
installation_path = os.path.join(nuget_args[-1], nuget_args[0] + '.' + version, 'tools')
shell([nuget, 'install'] + nuget_args)
return installation_path
def install_pypy(version, arch, url):
assert arch == '32'
# Inside the PyPy zip file is a directory with the same name
zip_filename = url.rsplit('/', 1)[-1]
installation_path = os.path.join('C:\\cibw', os.path.splitext(zip_filename)[0])
if not os.path.exists(installation_path):
pypy_zip = os.path.join('C:\\cibw', zip_filename)
download(url, pypy_zip)
# Extract to the parent directory because the zip file still contains a directory
extract_zip(pypy_zip, os.path.dirname(installation_path))
pypy_exe = 'pypy3.exe' if version[0] == '3' else 'pypy.exe'
shell(['mklink', os.path.join(installation_path, 'python.exe'), os.path.join(installation_path, pypy_exe)])
return installation_path
def setup_python(python_configuration, dependency_constraint_flags, environment):
nuget = 'C:\\cibw\\nuget.exe'
if not os.path.exists(nuget):
download('https://dist.nuget.org/win-x86-commandline/latest/nuget.exe', nuget)
if python_configuration.identifier.startswith('cp'):
installation_path = install_cpython(python_configuration.version, python_configuration.arch, nuget)
elif python_configuration.identifier.startswith('pp'):
installation_path = install_pypy(python_configuration.version, python_configuration.arch, python_configuration.url)
else:
run_with_env = os.path.abspath(os.path.join(os.path.dirname(__file__), 'resources', 'appveyor_run_with_env.cmd'))
raise ValueError("Unknown Python implementation")
# run_with_env is a cmd file that sets the right environment variables
# to build on AppVeyor.
def shell(args, env=None, cwd=None):
# print the command executing for the logs
print('+ ' + ' '.join(args))
args = ['cmd', '/E:ON', '/V:ON', '/C', run_with_env] + args
return subprocess.check_call(' '.join(args), env=env, cwd=cwd)
assert os.path.exists(os.path.join(installation_path, 'python.exe'))
abs_project_dir = os.path.abspath(project_dir)
# set up PATH and environment variables for run_with_env
env = os.environ.copy()
env['PYTHON_VERSION'] = python_configuration.version
env['PYTHON_ARCH'] = python_configuration.arch
env['PATH'] = os.pathsep.join([
installation_path,
os.path.join(installation_path, 'Scripts'),
env['PATH']
])
# update env with results from CIBW_ENVIRONMENT
env = environment.as_dictionary(prev_environment=env)
# for the logs - check we're running the right version of python
shell(['where', 'python'], env=env)
shell(['python', '--version'], env=env)
shell(['python', '-c', '"import struct; print(struct.calcsize(\'P\') * 8)"'], env=env)
where_python = subprocess.check_output(['where', 'python'], env=env, universal_newlines=True).splitlines()[0].strip()
if where_python != os.path.join(installation_path, 'python.exe'):
print("cibuildwheel: python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it.", file=sys.stderr)
exit(1)
# make sure pip is installed
if not os.path.exists(os.path.join(installation_path, 'Scripts', 'pip.exe')):
shell(['python', get_pip_script] + dependency_constraint_flags, env=env, cwd="C:\\cibw")
assert os.path.exists(os.path.join(installation_path, 'Scripts', 'pip.exe'))
where_pip = subprocess.check_output(['where', 'pip'], env=env, universal_newlines=True).splitlines()[0].strip()
if where_pip.strip() != os.path.join(installation_path, 'Scripts', 'pip.exe'):
print("cibuildwheel: pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it.", file=sys.stderr)
exit(1)
# prepare the Python environment
shell(['python', '-m', 'pip', 'install', '--upgrade', 'pip'] + dependency_constraint_flags, env=env)
shell(['pip', '--version'], env=env)
shell(['pip', 'install', '--upgrade', 'setuptools', 'wheel'] + dependency_constraint_flags, env=env)
return env
def build(options: BuildOptions):
temp_dir = tempfile.mkdtemp(prefix='cibuildwheel')
built_wheel_dir = os.path.join(temp_dir, 'built_wheel')
repaired_wheel_dir = os.path.join(temp_dir, 'repaired_wheel')
# install nuget as best way to provide python
nuget = 'C:\\cibw\\nuget.exe'
download('https://dist.nuget.org/win-x86-commandline/latest/nuget.exe', nuget)
# get pip fo this installation which not have.
get_pip_script = 'C:\\cibw\\get-pip.py'
download('https://bootstrap.pypa.io/get-pip.py', get_pip_script)
python_configurations = get_python_configurations(build_selector)
python_configurations = get_python_configurations(options.build_selector)
for config in python_configurations:
config_python_path = get_python_path(config)
simple_shell([nuget, "install"] + get_nuget_args(config))
if not os.path.exists(os.path.join(config_python_path, 'Scripts', 'pip.exe')):
simple_shell([os.path.join(config_python_path, 'python.exe'), get_pip_script ])
dependency_constraint_flags = []
if options.dependency_constraints:
dependency_constraint_flags = [
'-c', options.dependency_constraints.get_for_python_version(config.version)
]
# check python & pip exist for this configuration
assert os.path.exists(os.path.join(config_python_path, 'python.exe'))
assert os.path.exists(os.path.join(config_python_path, 'Scripts', 'pip.exe'))
# setup dirs
if os.path.exists(built_wheel_dir):
shutil.rmtree(built_wheel_dir)
os.makedirs(built_wheel_dir)
env = os.environ.copy()
# set up environment variables for run_with_env
env['PYTHON_VERSION'] = config.version
env['PYTHON_ARCH'] = config.arch
env['PATH'] = os.pathsep.join([
config_python_path,
os.path.join(config_python_path, 'Scripts'),
env['PATH']
])
env = environment.as_dictionary(prev_environment=env)
# for the logs - check we're running the right version of python
shell(['python', '--version'], env=env)
shell(['python', '-c', '"import struct; print(struct.calcsize(\'P\') * 8)\"'], env=env)
# prepare the Python environment
shell(['python', '-m', 'pip', 'install', '--upgrade', 'pip'], env=env)
shell(['pip', 'install', '--upgrade', 'setuptools'], env=env)
shell(['pip', 'install', 'wheel'], env=env)
# install Python
env = setup_python(config, dependency_constraint_flags, options.environment)
# run the before_build command
if before_build:
before_build_prepared = prepare_command(before_build, project=abs_project_dir)
if options.before_build:
before_build_prepared = prepare_command(options.before_build, project='.', package=options.package_dir)
shell([before_build_prepared], env=env)
# build the wheel
shell(['pip', 'wheel', abs_project_dir, '-w', built_wheel_dir, '--no-deps'] + get_build_verbosity_extra_flags(build_verbosity), env=env)
built_wheel = glob(built_wheel_dir+'/*.whl')[0]
if os.path.exists(built_wheel_dir):
shutil.rmtree(built_wheel_dir)
os.makedirs(built_wheel_dir)
shell(['pip', 'wheel', options.package_dir, '-w', built_wheel_dir, '--no-deps'] + get_build_verbosity_extra_flags(options.build_verbosity), env=env)
built_wheel = glob(os.path.join(built_wheel_dir, '*.whl'))[0]
if test_command:
# repair the wheel
if os.path.exists(repaired_wheel_dir):
shutil.rmtree(repaired_wheel_dir)
os.makedirs(repaired_wheel_dir)
if built_wheel.endswith('none-any.whl') or not options.repair_command:
# pure Python wheel or empty repair command
shutil.move(built_wheel, repaired_wheel_dir)
else:
repair_command_prepared = prepare_command(options.repair_command, wheel=built_wheel, dest_dir=repaired_wheel_dir)
shell([repair_command_prepared], env=env)
repaired_wheel = glob(os.path.join(repaired_wheel_dir, '*.whl'))[0]
if options.test_command:
# set up a virtual environment to install and test from, to make sure
# there are no dependencies that were pulled in at build time.
shell(['pip', 'install', 'virtualenv'], env=env)
shell(['pip', 'install', 'virtualenv'] + dependency_constraint_flags, env=env)
venv_dir = tempfile.mkdtemp()
shell(['python', '-m', 'virtualenv', venv_dir], env=env)
# Use --no-download to ensure determinism by using seed libraries
# built into virtualenv
shell(['python', '-m', 'virtualenv', '--no-download', venv_dir], env=env)
virtualenv_env = env.copy()
venv_script_path = os.path.join(venv_dir, 'Scripts')
if os.path.exists(os.path.join(venv_dir, 'bin')):
# pypy2.7 bugfix
venv_script_path = os.pathsep.join([venv_script_path, os.path.join(venv_dir, 'bin')])
virtualenv_env['PATH'] = os.pathsep.join([
os.path.join(venv_dir, 'Scripts'),
venv_script_path,
virtualenv_env['PATH'],
])
virtualenv_env["__CIBW_VIRTUALENV_PATH__"] = venv_dir
# check that we are using the Python from the virtual environment
shell(['which', 'python'], env=virtualenv_env)
if options.before_test:
before_test_prepared = prepare_command(
options.before_test,
project='.',
package=options.package_dir
)
shell([before_test_prepared], env=virtualenv_env)
# install the wheel
shell(['pip', 'install', built_wheel + test_extras], env=virtualenv_env)
shell(['pip', 'install', repaired_wheel + options.test_extras], env=virtualenv_env)
# test the wheel
if test_requires:
shell(['pip', 'install'] + test_requires, env=virtualenv_env)
if options.test_requires:
shell(['pip', 'install'] + options.test_requires, env=virtualenv_env)
# run the tests from c:\, with an absolute path in the command
# (this ensures that Python runs the tests against the installed wheel
# and not the repo code)
test_command_prepared = prepare_command(test_command, project=abs_project_dir)
test_command_prepared = prepare_command(
options.test_command,
project=os.path.abspath('.'),
package=os.path.abspath(options.package_dir)
)
shell([test_command_prepared], cwd='c:\\', env=virtualenv_env)
# clean up
shutil.rmtree(venv_dir)
# we're all done here; move it to output (remove if already exists)
dst = os.path.join(output_dir, os.path.basename(built_wheel))
dst = os.path.join(options.output_dir, os.path.basename(repaired_wheel))
if os.path.isfile(dst):
os.remove(dst)
shutil.move(built_wheel, dst)
shutil.move(repaired_wheel, dst)
+35
View File
@@ -0,0 +1,35 @@
---
title: Modern C++ standards
---
Building Python wheels with modern C++ standards (C++11 and later) requires a few tricks.
## Python 2.7 and C++17
The Python 2.7 header files use the `register` keyword, which is [reserved and unused from C+17 onwards](https://en.cppreference.com/w/cpp/keyword/register). Compiling a wheel for Python 2.7 with the C++17 standard is still possible to allow usage of `register` using proper flag `-Wno-register` for gcc/clang and `/wd5033` for MSVC.
## manylinux1 and C++14
The default `manylinux1` image (based on CentOS 5) contains a version of GCC and libstdc++ that only supports C++11 and earlier standards. There are however ways to compile wheels with the C++14 standard (and later): https://github.com/pypa/manylinux/issues/118
`manylinux2010` and `manylinux2014` are newer and support all C++ standards (up to C++17).
## macOS and deployment target versions
OS X/macOS allows you to specify a so-called "deployment target" version that will ensure backwards compatibility with older versions of macOS. One way to do this is by setting the `MACOSX_DEPLOYMENT_TARGET` environment variable.
However, to enable modern C++ standards, the deploment target needs to be set high enough (since older OS X/macOS versions did not have the necessary modern C++ standard library).
To get C++11 and C++14 support, `MACOSX_DEPLOYMENT_TARGET` needs to be set to (at least) `"10.9"`. By default, `cibuildwheel` already does this, building 64-bit-only wheels for macOS 10.9 and later.
To get C++17 support, Xcode 9.3+ is needed, requiring at least macOS 10.13 on the build machine. To use C++17 library features and link against the C++ runtime library, set `MACOSX_DEPLOYMENT_TARGET` to `"10.13"` or `"10.14"` (or higher) - macOS 10.13 offers partial C++17 support (e.g., the filesystem header is in experimental, offering `#include <experimental/filesystem>` instead of `#include <filesystem>`); macOS 10.14 has full C++17 support.
However, if only C++17 compiler and standard template library (STL) features are used (not needing a C++17 runtime) it might be possible to set `MACOSX_DEPLOYMENT_TARGET` to a lower value, such as `"10.9"`. To find out if this is the case, try compiling and running with a lower `MACOSX_DEPLOYMENT_TARGET`: if C++17 features are used that require a more recent deployment target, building the wheel should fail.
For more details see https://en.cppreference.com/w/cpp/compiler_support, https://en.wikipedia.org/wiki/Xcode, and https://xcodereleases.com/: Xcode 10 needs macOS 10.13 and Xcode 11 needs macOS 10.14.
## Windows and Python 2.7
Visual C++ for Python 2.7 does not support modern standards of C++. When building on Appveyor, you will need to either use the "Visual Studio 2017" or "Visual Studio 2019" image, but Python 2.7 is not supported on these images - skip it by setting `CIBW_SKIP=cp27-win*`.
There is an optional workaround for this, though: the pybind11 project argues and shows that it is [possible to compile Python 2.7 extension with a newer compiler](https://pybind11.readthedocs.io/en/stable/faq.html#working-with-ancient-visual-studio-2008-builds-on-windows) and has an example project showing how to do this: https://github.com/pybind/python_example. The main catch is that a user might need to install a newer "Microsoft Visual C++ Redistributable", since the newer C++ standard libraries are not included by default with the Python 2.7 installation.
+25 -1
View File
@@ -22,7 +22,7 @@ Linux wheels are built in the [`manylinux` docker images](https://github.com/pyp
- The project directory is mounted in the running Docker instance as `/project`, the output directory for the wheels as `/output`. In general, this is handled transparently by `cibuildwheel`. For a more finegrained level of control however, the root of the host file system is mounted as `/host`, allowing for example to access shared files, caches, etc. on the host file system. Note that this is not available on CircleCI due to their Docker policies.
- Alternative dockers images can be specified with the `CIBW_MANYLINUX_X86_64_IMAGE` and `CIBW_MANYLINUX_I686_IMAGE` options to allow for a custom, preconfigured build environment for the Linux builds. See [options](options.md#manylinux-image) for more details.
- Alternative dockers images can be specified with the `CIBW_MANYLINUX_X86_64_IMAGE`, `CIBW_MANYLINUX_I686_IMAGE`, and `CIBW_MANYLINUX_PYPY_X86_64_IMAGE` options to allow for a custom, preconfigured build environment for the Linux builds. See [options](options.md#manylinux-image) for more details.
### Building packages with optional C extensions
@@ -37,3 +37,27 @@ myextension = Extension(
optional=os.environ.get('CIBUILDWHEEL', '0') != '1',
)
```
### 'No module named XYZ' errors after running cibuildwheel on macOS
`cibuildwheel` on Mac installs the distributions from Python.org system-wide during its operation. This is necessary, but it can cause some confusing errors after cibuildwheel has finished.
Consider the build script:
```bash
python3 -m pip install twine cibuildwheel
python3 -m cibuildwheel --output-dir wheelhouse
python3 -m twine upload wheelhouse/*.whl
# error: no module named 'twine'
```
This doesn't work because while `cibuildwheel` was running, it installed a few new versions of 'python3', so the `python3` run on line 3 isn't the same as the `python3` that ran on line 1.
Solutions to this vary, but the simplest is to install tools immediately before they're used:
```bash
python3 -m pip install cibuildwheel
python3 -m cibuildwheel --output-dir wheelhouse
python3 -m pip install twine
python3 -m twine upload wheelhouse/*.whl
```
@@ -1,6 +1,24 @@
import mkdocs, re, os, io, cgi
import cgi
import io
import os
import re
TAG_REGEX_PATTERN = re.compile(
import mkdocs
INCLUDE_TAG_REGEX = re.compile(
r'''
{% # opening tag
\s*
include # directive name
\s+
"(?P<filename>[^"]+)" # "filename"
\s*
%} # closing tag
''',
flags=re.VERBOSE,
)
INCLUDEMARKDOWN_TAG_REGEX = re.compile(
r'''
{% # opening tag
\s*
@@ -15,11 +33,29 @@ TAG_REGEX_PATTERN = re.compile(
flags=re.VERBOSE,
)
class ImportMarkdownPlugin(mkdocs.plugins.BasePlugin):
def on_page_markdown(self, markdown, page, **kwargs):
page_src_path = page.file.abs_src_path
def found_import_markdown_tag(match):
def found_include_tag(match):
filename = match.group('filename')
file_path_abs = os.path.join(os.path.dirname(page_src_path), filename)
if not os.path.exists(file_path_abs):
raise ValueError('file not found', filename)
with io.open(file_path_abs, encoding='utf8') as f:
text_to_include = f.read()
# Allow good practice of having a final newline in the file
if text_to_include.endswith('\n'):
text_to_include = text_to_include[:-1]
return text_to_include
def found_includemarkdown_tag(match):
filename = match.group('filename')
start = match.group('start')
end = match.group('end')
@@ -31,21 +67,21 @@ class ImportMarkdownPlugin(mkdocs.plugins.BasePlugin):
with io.open(file_path_abs, encoding='utf8') as f:
text_to_include = f.read()
if start:
_, _, text_to_include = text_to_include.partition(start)
if end:
text_to_include, _, _ = text_to_include.partition(end)
return (
'<!-- BEGIN INCLUDE %s %s %s -->' % (
filename, cgi.escape(start), cgi.escape(end)
'<!-- BEGIN INCLUDE %s %s %s -->\n' % (
filename, cgi.escape(start or ''), cgi.escape(end or '')
)
+ text_to_include
+ '<!-- END INCLUDE -->'
+ '\n<!-- END INCLUDE -->'
)
markdown = re.sub(TAG_REGEX_PATTERN, found_import_markdown_tag, markdown)
markdown = re.sub(INCLUDE_TAG_REGEX, found_include_tag, markdown)
markdown = re.sub(INCLUDEMARKDOWN_TAG_REGEX, found_includemarkdown_tag, markdown)
return markdown
+220 -57
View File
@@ -4,7 +4,7 @@
## Setting options
cibuildwheel is configured using environment variables, that can be set using
cibuildwheel is configured using environment variables, that can be set using
your CI config.
For example, to configure cibuildwheel to run tests, add the following YAML to
@@ -14,16 +14,16 @@ your CI config file:
```yaml
env:
global:
- CIBW_TEST_REQUIRES=nose
- CIBW_TEST_COMMAND="nosetests {project}/tests"
- CIBW_TEST_REQUIRES=pytest
- CIBW_TEST_COMMAND="pytest {project}/tests"
```
> appveyor.yml ([docs](https://www.appveyor.com/docs/build-configuration/#environment-variables))
```yaml
environment:
global:
CIBW_TEST_REQUIRES: nose
CIBW_TEST_COMMAND: "nosetests {project}\\tests"
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_COMMAND: "pytest {project}\\tests"
```
> .circleci/config.yml ([docs](https://circleci.com/docs/2.0/configuration-reference/#environment))
@@ -31,15 +31,22 @@ environment:
jobs:
job_name:
environment:
CIBW_TEST_REQUIRES: nose
CIBW_TEST_COMMAND: "nosetests {project}/tests"
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_COMMAND: "pytest {project}/tests"
```
> azure-pipelines.yml ([docs](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables))
```yaml
variables:
CIBW_TEST_REQUIRES: nose
CIBW_TEST_COMMAND: "nosetests {project}/tests"
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_COMMAND: "pytest {project}/tests"
```
> .github/workflows/*.yml ([docs](https://help.github.com/en/actions/configuring-and-managing-workflows/using-environment-variables)) (can be global, in job, or in step)
```yaml
env:
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_COMMAND: "pytest {project}/tests"
```
@@ -56,7 +63,7 @@ Default: `auto`
`auto` will auto-detect platform using environment variables, such as `TRAVIS_OS_NAME`/`APPVEYOR`/`CIRCLECI`.
For `linux` you need Docker running, on Mac or Linux. For `macos`, you need a Mac machine, and note that this script is going to automatically install MacPython on your system, so don't run on your development machine. For `windows`, you need to run in Windows, and it will build and test for all versions of Python at `C:\PythonXX[-x64]`.
For `linux` you need Docker running, on macOS or Linux. For `macos`, you need a Mac machine, and note that this script is going to automatically install MacPython on your system, so don't run on your development machine. For `windows`, you need to run in Windows, and `cibuildwheel` will install required versions of Python to `C:\cibw\python` using NuGet.
This option can also be set using the command-line option `--platform`.
@@ -64,25 +71,29 @@ This option can also be set using the command-line option `--platform`.
> Choose the Python versions to build
Space-separated list of builds to build and skip. Each build has an identifier like `cp27-manylinux_x86_64` or `cp35-macosx_10_6_intel` - you can list specific ones to build and `cibuildwheel` will only build those, and/or list ones to skip and `cibuildwheel` won't try to build them.
Space-separated list of builds to build and skip. Each build has an identifier like `cp27-manylinux_x86_64` or `cp35-macosx_x86_64` - you can list specific ones to build and `cibuildwheel` will only build those, and/or list ones to skip and `cibuildwheel` won't try to build them.
When both options are specified, both conditions are applied and only builds with a tag that matches `CIBW_BUILD` and does not match `CIBW_SKIP` will be built.
When setting the options, you can use shell-style globbing syntax (as per `fnmatch`). All the build identifiers supported by cibuildwheel are shown below:
When setting the options, you can use shell-style globbing syntax (as per [`fnmatch`](https://docs.python.org/3/library/fnmatch.html)). All the build identifiers supported by cibuildwheel are shown below:
<div class="build-id-table-marker"></div>
| | macOS 64bit | macOS 32/64bit | Manylinux 64bit | Manylinux 32bit | Windows 64bit | Windows 32bit |
|------------|-------------------------|------------------------|------------------------|----------------------|-----------------|----------------|
| Python 2.7 | | cp27-macosx_10_6_intel | cp27-manylinux_x86_64 | cp27-manylinux_i686 | cp27-win_amd64 | cp27-win32 |
| Python 3.5 | | cp35-macosx_10_6_intel | cp35-manylinux_x86_64 | cp35-manylinux_i686 | cp35-win_amd64 | cp35-win32 |
| Python 3.6 | | cp36-macosx_10_6_intel | cp36-manylinux_x86_64 | cp36-manylinux_i686 | cp36-win_amd64 | cp36-win32 |
| Python 3.7 | | cp37-macosx_10_6_intel | cp37-manylinux_x86_64 | cp37-manylinux_i686 | cp37-win_amd64 | cp37-win32 |
| Python 3.8 | cp38-macosx_10_9_x86_64 | | cp38-manylinux_x86_64 | cp38-manylinux_i686 | cp38-win_amd64 | cp38-win32 |
| | macOS 64bit | Manylinux x86 64bit | Manylinux x86 32bit | Windows 64bit | Windows 32bit | Manylinux Armv8 64bit | Manylinux PPC64LE | Manylinux s390x |
|-----------------|---------------------|------------------------|----------------------|-----------------|----------------|------------------------|------------------------|----------------------|
| Python 2.7 | cp27-macosx_x86_64 | cp27-manylinux_x86_64 | cp27-manylinux_i686 | cp27-win_amd64 | cp27-win32 | | | |
| Python 3.5 | cp35-macosx_x86_64 | cp35-manylinux_x86_64 | cp35-manylinux_i686 | cp35-win_amd64 | cp35-win32 | cp35-manylinux_aarch64 | cp35-manylinux_ppc64le | cp35-manylinux_s390x |
| Python 3.6 | cp36-macosx_x86_64 | cp36-manylinux_x86_64 | cp36-manylinux_i686 | cp36-win_amd64 | cp36-win32 | cp36-manylinux_aarch64 | cp36-manylinux_ppc64le | cp36-manylinux_s390x |
| Python 3.7 | cp37-macosx_x86_64 | cp37-manylinux_x86_64 | cp37-manylinux_i686 | cp37-win_amd64 | cp37-win32 | cp37-manylinux_aarch64 | cp37-manylinux_ppc64le | cp37-manylinux_s390x |
| Python 3.8 | cp38-macosx_x86_64 | cp38-manylinux_x86_64 | cp38-manylinux_i686 | cp38-win_amd64 | cp38-win32 | cp38-manylinux_aarch64 | cp38-manylinux_ppc64le | cp38-manylinux_s390x |
| PyPy 2.7 v7.3.0 | pp27-macosx_x86_64 | pp27-manylinux_x86_64 | | | pp27-win32 | | | |
| PyPy 3.6 v7.3.0 | pp36-macosx_x86_64 | pp36-manylinux_x86_64 | | | pp36-win32 | | | |
The list of supported and currently selected build identifiers can also be retrieved by passing the `--print-build-identifiers` flag to `cibuildwheel`.
The format is `python_tag-platform_tag`, with tags similar to those in [PEP 425](https://www.python.org/dev/peps/pep-0425/#details).
For CPython, the minimally supported macOS version is 10.9; for PyPy 2.7 and PyPy 3.6, respectively macOS 10.7 and 10.13 or higher is required.
#### Examples
```yaml
@@ -90,10 +101,10 @@ The format is `python_tag-platform_tag`, with tags similar to those in [PEP 425]
CIBW_BUILD: cp36-*
# Skip building on Python 2.7 on the Mac
CIBW_SKIP: cp27-macosx_10_6_intel
CIBW_SKIP: cp27-macosx_x86_64
# Skip building on Python 3.8 on the Mac
CIBW_SKIP: cp38-macosx_10_9_x86_64
CIBW_SKIP: cp38-macosx_x86_64
# Skip building on Python 2.7 on all platforms
CIBW_SKIP: cp27-*
@@ -113,6 +124,12 @@ CIBW_SKIP: cp36-manylinux*
# Only build on Python 3 and skip 32-bit builds
CIBW_BUILD: cp3?-*
CIBW_SKIP: "*-win32 *-manylinux_i686"
# Only build PyPy and CPython 3
CIBW_BUILD: pp* cp3?-*
# Disable building PyPy wheels on all platforms
CIBW_SKIP: pp*
```
<style>
@@ -120,7 +137,7 @@ CIBW_SKIP: "*-win32 *-manylinux_i686"
font-size: 90%;
white-space: nowrap;
}
.rst-content .build-id-table-marker + table td,
.rst-content .build-id-table-marker + table td,
.rst-content .build-id-table-marker + table th {
padding: 4px 4px;
}
@@ -148,6 +165,11 @@ You must set this variable to pass variables to Linux builds (since they execute
You can use `$PATH` syntax to insert other variables, or the `$(pwd)` syntax to insert the output of other shell commands.
To specify more than one environment variable, separate the assignments by spaces.
Platform-specific variants also available:<br/>
`CIBW_ENVIRONMENT_MACOS` | `CIBW_ENVIRONMENT_WINDOWS` | `CIBW_ENVIRONMENT_LINUX`
#### Examples
```yaml
# Set some compiler flags
@@ -161,10 +183,10 @@ CIBW_ENVIRONMENT: "BUILD_TIME=$(date)"
# Supply options to `pip` to affect how it downloads dependencies
CIBW_ENVIRONMENT: "PIP_EXTRA_INDEX_URL=https://pypi.myorg.com/simple"
```
Platform-specific variants also available:
`CIBW_ENVIRONMENT_MACOS` | `CIBW_ENVIRONMENT_WINDOWS` | `CIBW_ENVIRONMENT_LINUX`
# Set two flags
CIBW_ENVIRONMENT: "BUILD_TIME=$(date) SAMPLE_TEXT=\"sample text\""
```
!!! note
`cibuildwheel` always defines the environment variable `CIBUILDWHEEL=1`. This can be useful for [building wheels with optional extensions](faq.md#building-packages-with-optional-c-extensions).
@@ -177,7 +199,12 @@ A shell command to run before building the wheel. This option allows you to run
If dependencies are required to build your wheel (for example if you include a header from a Python module), set this to `pip install .`, and the dependencies will be installed automatically by pip. However, this means your package will be built twice - if your package takes a long time to build, you might wish to manually list the dependencies here instead.
The active Python binary can be accessed using `python`, and pip with `pip`; `cibuildwheel` makes sure the right version of Python and pip will be executed. `{project}` can be used as a placeholder for the absolute path to the project's root.
The active Python binary can be accessed using `python`, and pip with `pip`; `cibuildwheel` makes sure the right version of Python and pip will be executed. The placeholder `{package}` can be used here; it will be replaced by the path to the package being built by `cibuildwheel`.
The command is run in a shell, so you can write things like `cmd1 && cmd2`.
Platform-specific variants also available:<br/>
`CIBW_BEFORE_BUILD_MACOS` | `CIBW_BEFORE_BUILD_WINDOWS` | `CIBW_BEFORE_BUILD_LINUX`
#### Examples
```yaml
@@ -189,43 +216,150 @@ CIBW_BEFORE_BUILD: pip install pybind11
# chain commands using &&
CIBW_BEFORE_BUILD: yum install -y libffi-dev && pip install .
# run a script that's inside your project
CIBW_BEFORE_BUILD: bash scripts/prepare_for_build.sh
# if cibuildwheel is called with a package_dir argument, it's available as {package}
CIBW_BEFORE_BUILD: "{package}/bin/prepare_for_build.sh"
```
Platform-specific variants also available:
`CIBW_BEFORE_BUILD_MACOS` | `CIBW_BEFORE_BUILD_WINDOWS` | `CIBW_BEFORE_BUILD_LINUX`
### `CIBW_REPAIR_WHEEL_COMMAND` {: #repair-wheel-command}
> Execute a shell command to repair each (non-pure Python) built wheel
Default:
- on Linux: `'auditwheel repair -w {dest_dir} {wheel}'`
- on macOS: `'delocate-listdeps {wheel} && delocate-wheel --require-archs x86_64 -w {dest_dir} {wheel}'`
- on Windows: `''`
A shell command to repair a built wheel by copying external library dependencies into the wheel tree and relinking them.
The command is run on each built wheel (except for pure Python ones) before testing it.
The following placeholders must be used inside the command and will be replaced by `cibuildwheel`:
- `{wheel}` for the absolute path to the built wheel
- `{dest_dir}` for the absolute path of the directory where to create the repaired wheel.
The command is run in a shell, so you can run multiple commands like `cmd1 && cmd2`.
Platform-specific variants also available:<br/>
`CIBW_REPAIR_WHEEL_COMMAND_MACOS` | `CIBW_REPAIR_WHEEL_COMMAND_WINDOWS` | `CIBW_REPAIR_WHEEL_COMMAND_LINUX`
#### Examples
```yaml
# don't repair macOS wheels
CIBW_REPAIR_WHEEL_COMMAND_MACOS: ""
# pass the `--lib-sdir .` flag to auditwheel on Linux
CIBW_REPAIR_WHEEL_COMMAND_LINUX: "auditwheel repair --lib-sdir . -w {dest_dir} {wheel}"
```
### `CIBW_MANYLINUX_X86_64_IMAGE`, `CIBW_MANYLINUX_I686_IMAGE` {: #manylinux-image}
> Specify alternative manylinux docker images
### `CIBW_MANYLINUX_X86_64_IMAGE`, `CIBW_MANYLINUX_I686_IMAGE`, `CIBW_MANYLINUX_PYPY_X86_64_IMAGE`, `CIBW_MANYLINUX_AARCH64_IMAGE`, `CIBW_MANYLINUX_PPC64LE_IMAGE`, `CIBW_MANYLINUX_S390X_IMAGE` {: #manylinux-image}
> Specify alternative manylinux docker images
An alternative Docker image to be used for building [`manylinux`](https://github.com/pypa/manylinux) wheels. `cibuildwheel` will then pull these instead of the default images, [`quay.io/pypa/manylinux2010_x86_64`](https://quay.io/pypa/manylinux2010_x86_64) and [`quay.io/pypa/manylinux2010_i686`](https://quay.io/pypa/manylinux2010_i686).
An alternative Docker image to be used for building [`manylinux`](https://github.com/pypa/manylinux) wheels. `cibuildwheel` will then pull these instead of the default images, [`quay.io/pypa/manylinux2010_x86_64`](https://quay.io/pypa/manylinux2010_x86_64), [`quay.io/pypa/manylinux2010_i686`](https://quay.io/pypa/manylinux2010_i686), [`pypywheels/manylinux2010-pypy_x86_64`](https://hub.docker.com/r/pypywheels/manylinux2010-pypy_x86_64), [`quay.io/pypa/manylinux2014_aarch64`](https://quay.io/pypa/manylinux2014_aarch64), [`quay.io/pypa/manylinux2014_ppc64le`](https://quay.io/pypa/manylinux2014_ppc64le), and [`quay.io/pypa/manylinux2014_s390x`](https://quay.io/pypa/manylinux2010_s390x).
The value of this option can either be set to `manylinux1` or `manylinux2010` to use the [official `manylinux` images](https://github.com/pypa/manylinux), or any other valid Docker image name.
The value of this option can either be set to `manylinux1`, `manylinux2010` or `manylinux2014` to use a pinned version of the [official `manylinux` images](https://github.com/pypa/manylinux) and [PyPy `manylinux` images](https://github.com/pypy/manylinux). Alternatively, set these options to any other valid Docker image name. Note that for PyPy, only the official `manylinux2010` image is currently available. For architectures other
than x86 (x86\_64 and i686) manylinux2014 must be used because this is the first version of the manylinux specification that supports additional architectures.
Beware to specify a valid Docker image that can be used in the same way as the official, default Docker images: all necessary Python and pip versions need to be present in `/opt/python/`, and the `auditwheel` tool needs to be present for `cibuildwheel` to work. Apart from that, the architecture and relevant shared system libraries need to be manylinux1- or manylinux2010-compatible in order to produce valid `manylinux1`/`manylinux2010` wheels (see https://github.com/pypa/manylinux, [PEP 513](https://www.python.org/dev/peps/pep-0513/), and [PEP 571](https://www.python.org/dev/peps/pep-0571/) for more details).
Beware to specify a valid Docker image that can be used in the same way as the official, default Docker images: all necessary Python and pip versions need to be present in `/opt/python/`, and the `auditwheel` tool needs to be present for `cibuildwheel` to work. Apart from that, the architecture and relevant shared system libraries need to be manylinux1-, manylinux2010- or manylinux2014-compatible in order to produce valid `manylinux1`/`manylinux2010`/`manylinux2014` wheels (see [pypa/manylinux on GitHub](https://github.com/pypa/manylinux), [PEP 513](https://www.python.org/dev/peps/pep-0513/), [PEP 571](https://www.python.org/dev/peps/pep-0571/) and [PEP 599](https://www.python.org/dev/peps/pep-0599/) for more details).
Note that `auditwheel` detects the version of the `manylinux` standard in the Docker image through the `AUDITWHEEL_PLAT` environment variable, as `cibuildwheel` has no way of detecting the correct `--plat` command line argument to pass to `auditwheel` for a custom image. If a Docker image does not correctly set this `AUDITWHEEL_PLAT` environment variable, the `CIBW_ENVIRONMENT` option can be used to do so (e.g., `CIBW_ENVIRONMENT="manylinux2010_$(uname -m)"`).
Note that `auditwheel` detects the version of the `manylinux` standard in the Docker image through the `AUDITWHEEL_PLAT` environment variable, as `cibuildwheel` has no way of detecting the correct `--plat` command line argument to pass to `auditwheel` for a custom image. If a Docker image does not correctly set this `AUDITWHEEL_PLAT` environment variable, the `CIBW_ENVIRONMENT` option can be used to do so (e.g., `CIBW_ENVIRONMENT='AUDITWHEEL_PLAT="manylinux2010_$(uname -m)"'`).
Note that `manylinux2014` doesn't support builds with Python 2.7 - when building with `manylinux2014`, skip Python 2.7 using `CIBW_SKIP` (see example below).
#### Examples
```yaml
# build using the manylinux1 image to ensure manylinux1 wheels are produced
# skip PyPy, since there is no PyPy manylinux1 image
CIBW_MANYLINUX_X86_64_IMAGE: manylinux1
CIBW_MANYLINUX_I686_IMAGE: manylinux1
CIBW_SKIP: pp*
# build using the manylinux2014 image
CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
CIBW_MANYLINUX_I686_IMAGE: manylinux2014
CIBW_SKIP: cp27-manylinux*
# build using the latest manylinux2010 release, instead of the cibuildwheel
# pinned version
CIBW_MANYLINUX_X86_64_IMAGE: quay.io/pypa/manylinux2010_x86_64:latest
CIBW_MANYLINUX_I686_IMAGE: quay.io/pypa/manylinux2010_i686:latest
# build using a different image from the docker registry
CIBW_MANYLINUX_X86_64_IMAGE: dockcross/manylinux-x64
CIBW_MANYLINUX_I686_IMAGE: dockcross/manylinux-x86
```
### `CIBW_DEPENDENCY_VERSIONS` {: #dependency-versions}
> Specify how cibuildwheel controls the versions of the tools it uses
Options: `pinned` `latest` `<your constraints file>`
Default: `pinned`
If `CIBW_DEPENDENCY_VERSIONS` is `pinned`, cibuildwheel uses versions of tools
like `pip`, `setuptools`, `virtualenv` that were pinned with that release of
cibuildwheel. This represents a known-good set of dependencies, and is
recommended for build repeatability.
If set to `latest`, cibuildwheel will use the latest of these packages that
are available on PyPI. This might be preferable if these packages have bug
fixes that can't wait for a new cibuildwheel release.
To control the versions of dependencies yourself, you can supply a [pip
constraints](https://pip.pypa.io/en/stable/user_guide/#constraints-files) file
here and it will be used instead.
!!! note
If you need different dependencies for each python version, provide them
in the same folder with a `-pythonXY` suffix. e.g. if your
`CIBW_DEPENDENCY_VERSIONS=./constraints.txt`, cibuildwheel will use
`./constraints-python27.txt` on Python 2.7, or fallback to
`./constraints.txt` if that's not found.
Platform-specific variants also available:<br/>
`CIBW_DEPENDENCY_VERSIONS_MACOS` | `CIBW_DEPENDENCY_VERSIONS_WINDOWS`
!!! note
This option does not affect the tools used on the Linux build - those versions
are bundled with the manylinux image that cibuildwheel uses. To change
dependency versions on Linux, use the [CIBW_MANYLINUX_*](#manylinux-image)
options.
#### Examples
```yaml
# use tools versions that are bundled with cibuildwheel (this is the default)
CIBW_DEPENDENCY_VERSIONS: pinned
# use the latest versions available on PyPI
CIBW_DEPENDENCY_VERSIONS: latest
# use your own pip constraints file
CIBW_DEPENDENCY_VERSIONS: ./constraints.txt
```
## Testing
### `CIBW_TEST_COMMAND` {: #test-command}
> Execute a shell command to test all built wheels
> Execute a shell command to test each built wheel
Shell command to run tests after the build. The wheel will be installed automatically and available for import from the tests. `{project}` can be used as a placeholder for the absolute path to the project's root and will be replaced by `cibuildwheel`.
Shell command to run tests after the build. The wheel will be installed automatically and available for import from the tests. To ensure the wheel is imported by your tests (instead of your source copy), tests are run from a different directory. Use the placeholders `{project}` and `{package}` when specifying paths in your project.
On Linux and Mac, the command runs in a shell, so you can write things like `cmd1 && cmd2`.
- `{project}` is an absolute path to the project root - the working directory where cibuildwheel was called.
- `{package}` is the path to the package being built - the `package_dir` argument supplied to cibuildwheel on the command line.
The command is run in a shell, so you can write things like `cmd1 && cmd2`.
Platform-specific variants also available:<br/>
`CIBW_TEST_COMMAND_MACOS` | `CIBW_TEST_COMMAND_WINDOWS` | `CIBW_TEST_COMMAND_LINUX`
#### Examples
@@ -233,31 +367,29 @@ On Linux and Mac, the command runs in a shell, so you can write things like `cmd
# run the project tests against the installed wheel using `nose`
CIBW_TEST_COMMAND: nosetests {project}/tests
# run the project tests using `pytest`
CIBW_TEST_COMMAND: nosetests {project}/tests
# run the package tests using `pytest`
CIBW_TEST_COMMAND: pytest {package}/tests
```
Platform-specific variants also available:
`CIBW_TEST_COMMAND_MACOS` | `CIBW_TEST_COMMAND_WINDOWS` | `CIBW_TEST_COMMAND_LINUX`
### `CIBW_TEST_REQUIRES` {: #test-requires}
> Install Python dependencies before running the tests
Space-separated list of dependencies required for running the tests.
Platform-specific variants also available:<br/>
`CIBW_TEST_REQUIRES_MACOS` | `CIBW_TEST_REQUIRES_WINDOWS` | `CIBW_TEST_REQUIRES_LINUX`
#### Examples
```yaml
# install pytest before running CIBW_TEST_COMMAND
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_REQUIRES: pytest
# install specific versions of test dependencies
CIBW_TEST_REQUIRES: nose==1.3.7 moto==0.4.31
```
Platform-specific variants also available:
`CIBW_TEST_REQUIRES_MACOS` | `CIBW_TEST_REQUIRES_WINDOWS` | `CIBW_TEST_REQUIRES_LINUX`
### `CIBW_TEST_EXTRAS` {: #test-extras}
> Install your wheel for testing using `extras_require`
@@ -269,15 +401,43 @@ tests. This can be used to avoid having to redefine test dependencies in
`CIBW_TEST_REQUIRES` if they are already defined in `setup.py` or
`setup.cfg`.
Platform-specific variants also available:<br/>
`CIBW_TEST_EXTRAS_MACOS` | `CIBW_TEST_EXTRAS_WINDOWS` | `CIBW_TEST_EXTRAS_LINUX`
#### Examples
```yaml
# will cause the wheel to be installed with `pip install <wheel_file>[test,qt]`
CIBW_TEST_EXTRAS: test,qt
CIBW_TEST_EXTRAS: test,qt
```
Platform-specific variants also available:
`CIBW_TEST_EXTRAS_MACOS` | `CIBW_TEST_EXTRAS_WINDOWS` | `CIBW_TEST_EXTRAS_LINUX`
### `CIBW_BEFORE_TEST` {: #before-test}
> Execute a shell command before testing each wheel
A shell command to run in **each** test virtual environment, before your wheel is installed and tested. This is useful if you need to install a non pip package, change values of environment variables
or perform multi step pip installation (e.g. installing `scikit-build` or `cython` before install test package)
The active Python binary can be accessed using `python`, and pip with `pip`; `cibuildwheel` makes sure the right version of Python and pip will be executed. The placeholder `{package}` can be used here; it will be replaced by the path to the package being built by `cibuildwheel`.
The command is run in a shell, so you can write things like `cmd1 && cmd2`.
Platform-specific variants also available:<br/>
`CIBW_BEFORE_TEST_MACOS` | `CIBW_BEFORE_TEST_WINDOWS` | `CIBW_BEFORE_TEST_LINUX`
#### Examples
```yaml
# install test dependencies with overwritten environment variables.
CIBW_BEFORE_TEST: CC=gcc CXX=g++ pip install -r requirements.txt
# chain commands using &&
CIBW_BEFORE_TEST: rm -rf ./data/cache && mkdir -p ./data/cache
# install non pip python package
CIBW_BEFORE_TEST: cd some_dir; ./configure; make; make install
# install python packages that are required to install test dependencies
CIBW_BEFORE_TEST: pip install cmake scikit-build
```
## Other
@@ -286,6 +446,9 @@ Platform-specific variants also available:
An number from 1 to 3 to increase the level of verbosity (corresponding to invoking pip with `-v`, `-vv`, and `-vvv`), between -1 and -3 (`-q`, `-qq`, and `-qqq`), or just 0 (default verbosity). These flags are useful while debugging a build when the output of the actual build invoked by `pip wheel` is required.
Platform-specific variants also available:<br/>
`CIBW_BUILD_VERBOSITY_MACOS` | `CIBW_BUILD_VERBOSITY_WINDOWS` | `CIBW_BUILD_VERBOSITY_LINUX`
#### Examples
```yaml
@@ -293,21 +456,22 @@ An number from 1 to 3 to increase the level of verbosity (corresponding to invok
CIBW_BUILD_VERBOSITY: 1
```
Platform-specific variants also available:
`CIBW_BUILD_VERBOSITY_MACOS` | `CIBW_BUILD_VERBOSITY_WINDOWS` | `CIBW_BUILD_VERBOSITY_LINUX`
## Command line options
```text
usage: cibuildwheel [-h] [--platform {auto,linux,macos,windows}]
[--output-dir OUTPUT_DIR] [--print-build-identifiers]
[project_dir]
[package_dir]
Build wheels for all the platforms.
positional arguments:
project_dir Path to the project that you want wheels for.
Default: the current directory.
package_dir Path to the package that you want wheels for. Must be
a subdirectory of the working directory. When set, the
working directory is still considered the 'project'
and is copied into the Docker container on Linux.
Default: the working directory.
optional arguments:
-h, --help show this help message and exit
@@ -318,13 +482,12 @@ optional arguments:
automatically install MacPython on your system, so
don't run on your development machine. For "windows",
you need to run in Windows, and it will build and test
for all versions of Python at C:\PythonXX[-x64].
for all versions of Python. Default: auto.
--output-dir OUTPUT_DIR
Destination folder for the wheels.
--print-build-identifiers
Print the build identifiers matched by the current
invocation and exit.
```
<style>
@@ -381,7 +544,7 @@ optional arguments:
options[header].push({name: optionName, description, id});
});
// write the table of contents
var tocTable = $('.options-toc');
+27 -109
View File
@@ -2,82 +2,42 @@
title: 'Setup'
---
# GitHub Actions [linux/mac/windows] {: #github-actions}
To build Linux, Mac, and Windows wheels using Github Actions, create a `.github/workflows/build.yml` file in your repo.
> build.yml
```yaml
{% include "../examples/github-minimal.yml" %}
```
Commit this file, and push to Github - either to your default branch, or to a PR branch. The build should start automatically.
For more info on this file, check out the [docs](https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions).
# Azure Pipelines [linux/mac/windows] {: #azure-pipelines}
Using Azure pipelines, you can build all three platforms on the same service. Create a `azure-pipelines.yml` file in your repo.
To build Linux, Mac, and Windows wheels on Azure Pipelines, create a `azure-pipelines.yml` file in your repo.
> azure-pipelines.yml
```yaml
jobs:
- job: linux
pool: {vmImage: 'Ubuntu-16.04'}
steps:
- task: UsePythonVersion@0
- bash: |
python -m pip install --upgrade pip
pip install cibuildwheel==1.0.0
cibuildwheel --output-dir wheelhouse .
- task: PublishBuildArtifacts@1
inputs: {pathtoPublish: 'wheelhouse'}
- job: macos
pool: {vmImage: 'macOS-10.13'}
steps:
- task: UsePythonVersion@0
- bash: |
python -m pip install --upgrade pip
pip install cibuildwheel==1.0.0
cibuildwheel --output-dir wheelhouse .
- task: PublishBuildArtifacts@1
inputs: {pathtoPublish: 'wheelhouse'}
- job: windows
pool: {vmImage: 'vs2017-win2016'}
steps:
- task: UsePythonVersion@0
- script: choco install vcpython27 -f -y
displayName: Install Visual C++ for Python 2.7
- bash: |
python -m pip install --upgrade pip
pip install cibuildwheel==1.0.0
cibuildwheel --output-dir wheelhouse .
- task: PublishBuildArtifacts@1
inputs: {pathtoPublish: 'wheelhouse'}
{% include "../examples/azure-pipelines-minimal.yml" %}
```
!!! note
To support Python 3.5 on Windows, make sure to specify the use of `{vmImage: 'vs2017-win2016'}` on Windows, to ensure the required toolchain is available.
Commit this file, enable building of your repo on Azure Pipelines, and push.
Wheels will be stored for you and available through the Pipelines interface. For more info on this file, check out the [docs](https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema).
# Travis CI [linux/mac] {: #travis-ci}
# Travis CI [linux/mac/windows] {: #travis-ci}
To build Linux and Mac wheels on Travis CI, create a `.travis.yml` file in your repo.
To build Linux, Mac, and Windows wheels on Travis CI, create a `.travis.yml` file in your repo.
> .travis.yml
```yaml
language: python
matrix:
include:
- sudo: required
services:
- docker
env: PIP=pip
- os: osx
language: generic
env: PIP=pip2
script:
- $PIP install cibuildwheel==1.0.0
- cibuildwheel --output-dir wheelhouse
```
To build on Windows too, add this matrix entry:
```yaml
- os: windows
language: shell
before_install:
- choco install python3 --version 3.6.8 --no-progress -y
env:
- PATH=/c/Python36:/c/Python36/Scripts:$PATH
{% include "../examples/travis-ci-minimal.yml" %}
```
Note that building Windows Python 2.7 wheels on Travis is unsupported.
@@ -92,44 +52,7 @@ To build Linux and Mac wheels on CircleCI, create a `.circleci/config.yml` file
> .circleci/config.yml
```yaml
version: 2
jobs:
linux-wheels:
working_directory: ~/linux-wheels
docker:
- image: circleci/python:3.6
steps:
- checkout
- setup_remote_docker
- run:
name: Build the Linux wheels.
command: |
pip install --user cibuildwheel==1.0.0
cibuildwheel --output-dir wheelhouse
- store_artifacts:
path: wheelhouse/
osx-wheels:
working_directory: ~/osx-wheels
macos:
xcode: "10.0.0"
steps:
- checkout
- run:
name: Build the OS X wheels.
command: |
pip install --user cibuildwheel==1.0.0
cibuildwheel --output-dir wheelhouse
- store_artifacts:
path: wheelhouse/
workflows:
version: 2
all-tests:
jobs:
- linux-wheels
- osx-wheels
{% include "../examples/circleci-minimal.yml" %}
```
Commit this file, enable building of your repo on CircleCI, and push.
@@ -139,19 +62,14 @@ Commit this file, enable building of your repo on CircleCI, and push.
CircleCI will store the built wheels for you - you can access them from the project console. Check out the CircleCI [docs](https://circleci.com/docs/2.0/configuration-reference/#section=configuration) for more info on this config file.
# AppVeyor [windows] {: #appveyor}
# AppVeyor [linux/mac/windows] {: #appveyor}
To build Windows wheels on AppVeyor, create an `appveyor.yml` file in your repo.
To build Linux, Mac, and Windows wheels on AppVeyor, create an `appveyor.yml` file in your repo.
> appveyor.yml
```yaml
build_script:
- pip install cibuildwheel==1.0.0
- cibuildwheel --output-dir wheelhouse
artifacts:
- path: "wheelhouse\\*.whl"
name: Wheels
{% include "../examples/appveyor-minimal.yml" %}
```
Commit this file, enable building of your repo on AppVeyor, and push.
@@ -162,7 +80,7 @@ For more info on this config file, check out the [docs](https://www.appveyor.com
> ⚠️ Got an error? Check the [FAQ](faq.md).
<script>
<script>
document.addEventListener('DOMContentLoaded', function() {
$('.toctree-l2 a, .rst-content h1').each(function(i, el) {
var text = $(el).text()
@@ -191,4 +109,4 @@ For more info on this config file, check out the [docs](https://www.appveyor.com
}
});
});
</script>
</script>
+21
View File
@@ -0,0 +1,21 @@
environment:
matrix:
- APPVEYOR_BUILD_WORKER_IMAGE: Ubuntu
APPVEYOR_JOB_NAME: "python37-x64-ubuntu"
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
APPVEYOR_JOB_NAME: "python37-x64-vs2015"
- APPVEYOR_BUILD_WORKER_IMAGE: macos-mojave
APPVEYOR_JOB_NAME: "python37-x64-macos-mojave"
stack: python 3.7
init:
- cmd: set PATH=C:\Python37;C:\Python37\Scripts;%PATH%
install: python -m pip install cibuildwheel==1.4.0
build_script: python -m cibuildwheel --output-dir wheelhouse
artifacts:
- path: "wheelhouse\\*.whl"
name: Wheels
+35
View File
@@ -0,0 +1,35 @@
jobs:
- job: linux
pool: {vmImage: 'Ubuntu-16.04'}
steps:
- task: UsePythonVersion@0
- bash: |
python3 -m pip install --upgrade pip
pip3 install cibuildwheel==1.4.0
cibuildwheel --output-dir wheelhouse .
- task: PublishBuildArtifacts@1
inputs: {pathtoPublish: 'wheelhouse'}
- job: macos
pool: {vmImage: 'macOS-10.15'}
steps:
- task: UsePythonVersion@0
- bash: |
python3 -m pip install --upgrade pip
pip3 install cibuildwheel==1.4.0
cibuildwheel --output-dir wheelhouse .
- task: PublishBuildArtifacts@1
inputs: {pathtoPublish: 'wheelhouse'}
- job: windows
pool: {vmImage: 'vs2017-win2016'}
steps:
- task: UsePythonVersion@0
- script: choco install vcpython27 -f -y
displayName: Install Visual C++ for Python 2.7
- bash: |
python -m pip install --upgrade pip
pip install cibuildwheel==1.4.0
cibuildwheel --output-dir wheelhouse .
- task: PublishBuildArtifacts@1
inputs: {pathtoPublish: 'wheelhouse'}
+38
View File
@@ -0,0 +1,38 @@
version: 2
jobs:
linux-wheels:
working_directory: ~/linux-wheels
docker:
- image: circleci/python:3.6
steps:
- checkout
- setup_remote_docker
- run:
name: Build the Linux wheels.
command: |
pip3 install --user cibuildwheel==1.4.0
cibuildwheel --output-dir wheelhouse
- store_artifacts:
path: wheelhouse/
osx-wheels:
working_directory: ~/osx-wheels
macos:
xcode: "10.0.0"
steps:
- checkout
- run:
name: Build the OS X wheels.
command: |
pip3 install --user cibuildwheel==1.4.0
cibuildwheel --output-dir wheelhouse
- store_artifacts:
path: wheelhouse/
workflows:
version: 2
all-tests:
jobs:
- linux-wheels
- osx-wheels
+37
View File
@@ -0,0 +1,37 @@
name: Build
on: [push, pull_request]
jobs:
build_wheels:
name: Build wheel on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-18.04, windows-latest, macos-latest]
steps:
- uses: actions/checkout@v1
- uses: actions/setup-python@v1
name: Install Python
with:
python-version: '3.7'
- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==1.4.0
- name: Install Visual C++ for Python 2.7
if: startsWith(matrix.os, 'windows')
run: |
choco install vcpython27 -f -y
- name: Build wheel
run: |
python -m cibuildwheel --output-dir wheelhouse
- uses: actions/upload-artifact@v1
with:
name: wheels
path: ./wheelhouse
+40
View File
@@ -0,0 +1,40 @@
# As written, this configuration will build your wheels on every
# commit, but will only push to PyPI on tagged commits.
language: python
jobs:
include:
# perform a linux build
- services: docker
# and a mac build
- os: osx
language: shell
# and a windows build
- os: windows
language: shell
before_install:
- choco install python --version 3.8.0
- export PATH="/c/Python38:/c/Python38/Scripts:$PATH"
# make sure it's on PATH as 'python3'
- ln -s /c/Python38/python.exe /c/Python38/python3.exe
env:
global:
- TWINE_USERNAME=__token__
# Note: TWINE_PASSWORD is set to a PyPI API token in Travis settings
install:
- python3 -m pip install cibuildwheel==1.4.0
script:
# build the wheels, put them into './wheelhouse'
- python3 -m cibuildwheel --output-dir wheelhouse
after_success:
# if the release was tagged, upload them to PyPI
- |
if [[ $TRAVIS_TAG ]]; then
python3 -m pip install twine
python3 -m twine upload wheelhouse/*.whl
fi
+33
View File
@@ -0,0 +1,33 @@
language: python
jobs:
include:
# perform a linux build
- services: docker
# perform a linux ARMv8 build
- services: docker
arch: arm64
# perform a linux PPC64LE build
- services: docker
arch: ppc64le
# perform a linux S390X build
- services: docker
arch: s390x
# and a mac build
- os: osx
language: shell
# and a windows build
- os: windows
language: shell
before_install:
- choco install python --version 3.8.0
- export PATH="/c/Python38:/c/Python38/Scripts:$PATH"
# make sure it's on PATH as 'python3'
- ln -s /c/Python38/python.exe /c/Python38/python3.exe
install:
- python3 -m pip install cibuildwheel==1.4.0
script:
# build the wheels, put them into './wheelhouse'
- python3 -m cibuildwheel --output-dir wheelhouse
+88
View File
@@ -0,0 +1,88 @@
# In this configuration, the package is tested against multiple versions of python
# on Linux and on one version of Python on Windows. If and only if all tests pass
# will the wheels be built and deployed. Further, deployment is only initiated if
# the current branch is "master", the current commit is tagged, and the current
# repo is yours (e.g. it won't run on a Pull Request). For convenience, a source
# distribution is also created.
language: python
python:
- 3.5
- 3.6
- 3.7
- 3.8
before_install:
- |
if [[ "$TRAVIS_OS_NAME" = windows ]]; then
choco install python --version 3.8.0
export PATH="/c/Python38:/c/Python38/Scripts:$PATH"
# make sure it's on PATH as 'python3'
ln -s /c/Python38/python.exe /c/Python38/python3.exe
fi
install:
- python3 -m pip install pytest
script:
- python3 setup.py install
- pytest
stages:
- test
# Only execute deployment stage on tagged commits, and from your repository
# (e.g. not PRs). Replace with your repo name.
- name: deploy
if: tag IS PRESENT AND repo = joerick/cibuildwheel
# To only build tags that look like vX.Y.Z:
# if: tag =~ ^v\d+\.\d+\.\d+$ AND repo = joerick/cibuildwheel
jobs:
include:
# Optional: run a test on Windows
- os: windows
language: shell
name: Test on Windows
# Deploy source distribution
- stage: deploy
name: Deploy source distribution
install: skip
script: python3 setup.py sdist --formats=gztar
after_success: |
python3 -m pip install twine
python3 -m twine upload --skip-existing dist/*.tar.gz
# Deploy on linux
- stage: deploy
name: Build and deploy Linux wheels
services: docker
install: python3 -m pip install cibuildwheel==1.4.0
script: python3 -m cibuildwheel --output-dir wheelhouse
after_success: |
python3 -m pip install twine
python3 -m twine upload --skip-existing wheelhouse/*.whl
# Deploy on mac
- stage: deploy
name: Build and deploy macOS wheels
os: osx
language: shell
install: python3 -m pip install cibuildwheel==1.4.0
script: python3 -m cibuildwheel --output-dir wheelhouse
after_success: |
python3 -m pip install twine
python3 -m twine upload --skip-existing wheelhouse/*.whl
# Deploy on windows
- stage: deploy
name: Build and deploy Windows wheels
os: windows
language: shell
install: python3 -m pip install cibuildwheel==1.4.0
script: python3 -m cibuildwheel --output-dir wheelhouse
after_success: |
python3 -m pip install twine
python3 -m twine upload --skip-existing wheelhouse/*.whl
env:
global:
- TWINE_USERNAME=__token__
# Note: TWINE_PASSWORD is set to a PyPI API token in Travis settings
+1
View File
@@ -18,6 +18,7 @@ nav:
- setup.md
- options.md
- deliver-to-pypi.md
- cpp_standards.md
- faq.md
- contributing.md
+3
View File
@@ -2,3 +2,6 @@
-e ./docs/mkdocs_include_markdown_plugin
pytest
mkdocs==1.0.4
pip-tools
requests
click
+13 -21
View File
@@ -1,21 +1,13 @@
[bumpversion]
current_version = 1.0.0
commit = True
tag = True
message = Bump version
[bumpversion:file:setup.py]
[bumpversion:file:cibuildwheel/__init__.py]
[bumpversion:file:README.md]
search = cibuildwheel=={current_version}
replace = cibuildwheel=={new_version}
[bumpversion:file:docs/setup.md]
search = cibuildwheel=={current_version}
replace = cibuildwheel=={new_version}
[bdist_wheel]
universal = 1
[flake8]
ignore = E501,W503
application-import-names = cibuildwheel
exclude =
cibuildwheel/resources/,
dist/,
build/,
.git/,
env/,
env2/,
env??/,
.venv/,
site/
+12 -7
View File
@@ -1,35 +1,40 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import io
import os
try:
from setuptools import setup
except ImportError:
from distutils.core import setup
this_directory = os.path.dirname(__file__)
with io.open(os.path.join(this_directory, 'README.md'), encoding='utf-8') as f:
long_description = f.read()
setup(
name='cibuildwheel',
version='1.0.0',
version='1.4.0',
install_requires=['bashlex!=0.13'],
description="Build Python wheels on CI with minimal configuration.",
long_description='For readme please see http://github.com/joerick/cibuildwheel',
long_description=long_description,
long_description_content_type='text/markdown',
author="Joe Rickerby",
author_email='joerick@mac.com',
url='https://github.com/joerick/cibuildwheel',
packages=['cibuildwheel',],
packages=['cibuildwheel', ],
license="BSD",
zip_safe=False,
package_data={
'cibuildwheel': ['resources/*'],
},
# Supported python versions
python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*',
python_requires='>=3.5',
keywords='ci wheel packaging pypi travis appveyor macos linux windows',
classifiers=[
'Intended Audience :: Developers',
'Natural Language :: English',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 3',
'Development Status :: 4 - Beta',
'Development Status :: 5 - Production/Stable',
'License :: OSI Approved :: BSD License',
'Programming Language :: Python :: Implementation :: CPython',
'Topic :: Software Development :: Build Tools',
+9 -5
View File
@@ -1,16 +1,17 @@
import os
import utils
import platform
import utils
project_dir = os.path.dirname(__file__)
def test():
# build the wheels
utils.cibuildwheel_run(project_dir)
actual_wheels = utils.cibuildwheel_run(project_dir)
# check that the expected wheels are produced
expected_wheels = utils.expected_wheels('spam', '0.1.0')
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
@@ -20,7 +21,10 @@ def test_build_identifiers():
# after adding CIBW_MANYLINUX_IMAGE to support manylinux2010, there
# can be multiple wheels for each wheel, though, so we need to limit
# the expected wheels
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if not '-manylinux' in w or '-manylinux1' in w]
if platform.machine() in ['x86_64', 'i686']:
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if '-manylinux' not in w or '-manylinux1' in w]
else:
expected_wheels = utils.expected_wheels('spam', '0.1.0')
build_identifiers = utils.cibuildwheel_get_build_identifiers(project_dir)
assert len(expected_wheels) == len(build_identifiers)
+4 -1
View File
@@ -1,6 +1,9 @@
import os
from setuptools import setup, Extension
from setuptools import (
Extension,
setup,
)
if os.environ.get('CIBUILDWHEEL', '0') != '1':
raise Exception('CIBUILDWHEEL environment variable is not set to 1')
+13 -13
View File
@@ -1,22 +1,25 @@
import os, subprocess
import os
import subprocess
import pytest
import utils
def test():
project_dir = os.path.dirname(__file__)
# build and test the wheels
utils.cibuildwheel_run(project_dir, add_env={
actual_wheels = utils.cibuildwheel_run(project_dir, add_env={
'CIBW_TEST_REQUIRES': 'nose',
# the 'false ||' bit is to ensure this command runs in a shell on
# mac/linux.
'CIBW_TEST_COMMAND': 'false || nosetests {project}/test',
'CIBW_TEST_COMMAND_WINDOWS': 'nosetests {project}/test',
'CIBW_TEST_COMMAND_WINDOWS': 'COLOR 00 || nosetests {project}/test',
})
# also check that we got the right wheels
expected_wheels = utils.expected_wheels('spam', '0.1.0')
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
@@ -24,32 +27,29 @@ def test_extras_require():
project_dir = os.path.dirname(__file__)
# build and test the wheels
utils.cibuildwheel_run(project_dir, add_env={
actual_wheels = utils.cibuildwheel_run(project_dir, add_env={
'CIBW_TEST_EXTRAS': 'test',
# the 'false ||' bit is to ensure this command runs in a shell on
# mac/linux.
'CIBW_TEST_COMMAND': 'false || nosetests {project}/test',
'CIBW_TEST_COMMAND_WINDOWS': 'nosetests {project}/test',
'CIBW_TEST_COMMAND_WINDOWS': 'COLOR 00 || nosetests {project}/test',
})
# also check that we got the right wheels
expected_wheels = utils.expected_wheels('spam', '0.1.0')
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
def test_failing_test():
def test_failing_test(tmp_path):
'''Ensure a failing test causes cibuildwheel to error out and exit'''
project_dir = os.path.dirname(__file__)
with pytest.raises(subprocess.CalledProcessError):
utils.cibuildwheel_run(project_dir, add_env={
utils.cibuildwheel_run(project_dir, output_dir=tmp_path, add_env={
'CIBW_TEST_COMMAND': 'false',
# manylinux1 has a version of bash that's been shown to have
# manylinux1 has a version of bash that's been shown to have
# problems with this, so let's check that.
'CIBW_MANYLINUX_I686_IMAGE': 'manylinux1',
'CIBW_MANYLINUX_X86_64_IMAGE': 'manylinux1',
})
assert len(os.listdir('wheelhouse'))
assert len(os.listdir(str(tmp_path))) == 0
+4 -1
View File
@@ -1,4 +1,7 @@
from setuptools import setup, Extension
from setuptools import (
Extension,
setup,
)
setup(
name="spam",
+37
View File
@@ -1,8 +1,45 @@
from __future__ import print_function
import os
import sys
from unittest import TestCase
import spam
def path_contains(parent, child):
''' returns True if `child` is inside `parent`.
Works around path-comparison bugs caused by short-paths on Windows e.g.
vssadm~1 instead of vssadministrator
'''
parent = os.path.abspath(parent)
child = os.path.abspath(child)
while child != os.path.dirname(child):
child = os.path.dirname(child)
if os.stat(parent) == os.stat(child):
# parent and child refer to the same directory on the filesystem
return True
return False
class TestSpam(TestCase):
def test_system(self):
self.assertEqual(0, spam.system('python -c "exit(0)"'))
self.assertNotEqual(0, spam.system('python -c "exit(1)"'))
def test_virtualenv(self):
virtualenv_path = os.environ.get("__CIBW_VIRTUALENV_PATH__")
if not virtualenv_path:
self.fail("No virtualenv path defined in environment variable __CIBW_VIRTUALENV_PATH__")
print("=[executable]", sys.executable)
print("=[spam location]", spam.__file__)
print("=[virtualenv path]", virtualenv_path)
print("=[listdir]", os.listdir(virtualenv_path))
if os.path.exists(os.path.join(virtualenv_path, 'Scripts')):
print("=[listdir]2", os.listdir(os.path.join(virtualenv_path, 'Scripts')))
if os.path.exists(os.path.join(virtualenv_path, 'bin')):
print("=[listdir]2", os.listdir(os.path.join(virtualenv_path, 'bin')))
self.assertTrue(path_contains(virtualenv_path, sys.executable))
self.assertTrue(path_contains(virtualenv_path, spam.__file__))
+4 -3
View File
@@ -1,18 +1,19 @@
import os
import utils
def test():
project_dir = os.path.dirname(__file__)
# build the wheels
utils.cibuildwheel_run(project_dir, add_env={
actual_wheels = utils.cibuildwheel_run(project_dir, add_env={
# write python version information to a temporary file, this is
# checked in setup.py
'CIBW_BEFORE_BUILD': '''python -c "import sys; open('/tmp/pythonversion.txt', 'w').write(sys.version)" && python -c "import sys; open('/tmp/pythonexecutable.txt', 'w').write(sys.executable)"''',
'CIBW_BEFORE_BUILD_WINDOWS': '''python -c "import sys; open('c:\\pythonversion.txt', 'w').write(sys.version)" && python -c "import sys; open('c:\\pythonexecutable.txt', 'w').write(sys.executable)"''',
})
# also check that we got the right wheels
expected_wheels = utils.expected_wheels('spam', '0.1.0')
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
+7 -2
View File
@@ -1,5 +1,10 @@
from setuptools import setup, Extension
import sys, os
import os
import sys
from setuptools import (
Extension,
setup,
)
# assert that the Python version as written to pythonversion.txt in the CIBW_BEFORE_BUILD step
# is the same one as is currently running.
+3 -2
View File
@@ -1,11 +1,13 @@
import os
import utils
def test():
project_dir = os.path.dirname(__file__)
# build the wheels
utils.cibuildwheel_run(project_dir, add_env={
actual_wheels = utils.cibuildwheel_run(project_dir, add_env={
'CIBW_BUILD': 'cp3?-*',
'CIBW_SKIP': 'cp37-*',
})
@@ -13,5 +15,4 @@ def test():
# check that we got the right wheels. There should be no 2.7 or 3.7.
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if ('-cp3' in w) and ('-cp37' not in w)]
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
+5 -1
View File
@@ -1,6 +1,10 @@
from setuptools import setup, Extension
import sys
from setuptools import (
Extension,
setup,
)
# explode if run on Python 2.7 or Python 3.4 (these should be skipped)
if sys.version_info[0:2] == (2, 7):
raise Exception('Python 2.7 should not be built')
+16 -2
View File
@@ -1,18 +1,32 @@
import os
import pytest
import subprocess
import utils
def test():
project_dir = os.path.dirname(__file__)
# write some information into the CIBW_ENVIRONMENT, for expansion and
# insertion into the environment by cibuildwheel. This is checked
# in setup.py
utils.cibuildwheel_run(project_dir, add_env={
actual_wheels = utils.cibuildwheel_run(project_dir, add_env={
'CIBW_ENVIRONMENT': '''CIBW_TEST_VAR="a b c" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3="$(echo 'test string 3')" PATH=$PATH:/opt/cibw_test_path''',
'CIBW_ENVIRONMENT_WINDOWS': '''CIBW_TEST_VAR="a b c" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3="$(echo 'test string 3')" PATH="$PATH;/opt/cibw_test_path"''',
})
# also check that we got the right wheels built
expected_wheels = utils.expected_wheels('spam', '0.1.0')
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
def test_overridden_path(tmp_path):
project_dir = os.path.dirname(__file__)
# mess up PATH, somehow
with pytest.raises(subprocess.CalledProcessError):
utils.cibuildwheel_run(project_dir, output_dir=tmp_path, add_env={
'CIBW_ENVIRONMENT': '''SOMETHING="$(mkdir new_path && touch new_path/python)" PATH="$(realpath new_path):$PATH"''',
'CIBW_ENVIRONMENT_WINDOWS': '''SOMETHING="$(mkdir new_path && type nul > new_path/python.exe)" PATH="$CD\\new_path;$PATH"''',
})
assert len(os.listdir(str(tmp_path))) == 0
+6 -3
View File
@@ -1,5 +1,9 @@
from setuptools import setup, Extension
import sys, os
import os
from setuptools import (
Extension,
setup,
)
# explode if environment isn't correct, as set in CIBW_ENVIRONMENT
CIBW_TEST_VAR = os.environ.get('CIBW_TEST_VAR')
@@ -15,7 +19,6 @@ if '/opt/cibw_test_path' not in PATH:
if '$PATH' in PATH:
raise Exception('$PATH should be expanded in PATH. It was "%s"' % PATH)
setup(
name="spam",
ext_modules=[Extension('spam', sources=['spam.c'])],
+12 -7
View File
@@ -1,21 +1,26 @@
import os, pytest
import os
import platform
import pytest
import utils
def test():
project_dir = os.path.dirname(__file__)
if utils.platform != 'linux':
pytest.skip('the test is only relevant to the linux build')
if platform.machine() not in ['x86_64', 'i686']:
pytest.skip('this test is currently only possible on x86_64/i686 due to availability of alternative images')
utils.cibuildwheel_run(project_dir, add_env={
actual_wheels = utils.cibuildwheel_run(project_dir, add_env={
'CIBW_MANYLINUX_X86_64_IMAGE': 'dockcross/manylinux2010-x64',
'CIBW_MANYLINUX_I686_IMAGE': 'dockcross/manylinux1-x86',
'CIBW_BEFORE_BUILD': '/opt/python/cp36-cp36m/bin/pip install -U auditwheel', # Currently necessary on dockcross images to get auditwheel 2.1 supporting AUDITWHEEL_PLAT
'CIBW_ENVIRONMENT': 'AUDITWHEEL_PLAT=`if [ $(uname -i) == "x86_64" ]; then echo "manylinux2010_x86_64"; else echo "manylinux1_i686"; fi`',
'CIBW_MANYLINUX_I686_IMAGE': 'dockcross/manylinux2010-x86',
'CIBW_SKIP': 'pp*',
})
# also check that we got the right wheels built
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if '-manylinux2010_i686' not in w]
actual_wheels = os.listdir('wheelhouse')
if '-pp' not in w]
assert set(actual_wheels) == set(expected_wheels)
+6 -2
View File
@@ -1,6 +1,10 @@
import os, sys
import os
import sys
from setuptools import setup, Extension
from setuptools import (
Extension,
setup,
)
# check that we're running in the correct docker image as specified in the
# environment options CIBW_MANYLINUX1_*_IMAGE
+2
View File
@@ -1,6 +1,8 @@
import os
import utils
def test():
project_dir = os.path.dirname(__file__)
# this test checks that SSL is working in the build environment using
+5 -1
View File
@@ -1,6 +1,10 @@
import ssl
import sys
from setuptools import setup, Extension
from setuptools import (
Extension,
setup,
)
if sys.version_info[0] == 2:
from urllib2 import urlopen
@@ -1,21 +0,0 @@
import os, pytest
import utils
def test():
project_dir = os.path.dirname(__file__)
if utils.platform != 'linux':
pytest.skip('the docker test is only relevant to the linux build')
# build the wheels
# CFLAGS environment veriable is ecessary to fail on 'malloc_info' (on manylinux1) during compilation/linking,
# rather than when dynamically loading the Python
utils.cibuildwheel_run(project_dir, add_env={
'CIBW_ENVIRONMENT': 'CFLAGS="$CFLAGS -Werror=implicit-function-declaration"',
})
# also check that we got the right wheels
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if not '-manylinux' in w or '-manylinux2010' in w]
actual_wheels = os.listdir('wheelhouse')
assert set(actual_wheels) == set(expected_wheels)
@@ -0,0 +1,44 @@
import os
import platform
import pytest
import utils
@pytest.mark.parametrize('manylinux_image', ['manylinux1', 'manylinux2010', 'manylinux2014'])
def test(manylinux_image):
project_dir = os.path.dirname(__file__)
if utils.platform != 'linux':
pytest.skip('the docker test is only relevant to the linux build')
elif platform.machine() not in ['x86_64', 'i686']:
if manylinux_image in ['manylinux1', 'manylinux2010']:
pytest.skip("manylinux1 and 2010 doesn't exist for non-x86 architectures")
# build the wheels
# CFLAGS environment variable is necessary to fail on 'malloc_info' (on manylinux1) during compilation/linking,
# rather than when dynamically loading the Python
add_env = {
'CIBW_ENVIRONMENT': 'CFLAGS="$CFLAGS -Werror=implicit-function-declaration"',
'CIBW_MANYLINUX_X86_64_IMAGE': manylinux_image,
'CIBW_MANYLINUX_I686_IMAGE': manylinux_image,
'CIBW_MANYLINUX_PYPY_X86_64_IMAGE': manylinux_image,
'CIBW_MANYLINUX_AARCH64_IMAGE': manylinux_image,
'CIBW_MANYLINUX_PPC64LE_IMAGE': manylinux_image,
'CIBW_MANYLINUX_S390X_IMAGE': manylinux_image,
}
if manylinux_image == 'manylinux1':
# We don't have a manylinux1 image for PyPy
add_env['CIBW_SKIP'] = 'pp*'
elif manylinux_image == 'manylinux2014':
# We don't have a manylinux2014 image for PyPy (yet?)
add_env['CIBW_SKIP'] = 'cp27* pp*' # Python 2.7 not available on manylinux2014
actual_wheels = utils.cibuildwheel_run(project_dir, add_env=add_env)
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0', manylinux_versions=[manylinux_image])]
if manylinux_image == 'manylinux2014':
expected_wheels = [w for w in expected_wheels if '-cp27' not in w]
if manylinux_image in ['manylinux1', 'manylinux2014']:
expected_wheels = [w for w in expected_wheels if '-pp' not in w]
assert set(actual_wheels) == set(expected_wheels)
+10
View File
@@ -0,0 +1,10 @@
from setuptools import (
Extension,
setup,
)
setup(
name="spam",
ext_modules=[Extension('spam', sources=['spam.c'])],
version="0.1.0",
)
+68
View File
@@ -0,0 +1,68 @@
#include <Python.h>
#include <malloc.h>
#if !defined(__GLIBC_PREREQ)
#error "Must run on a glibc linux environment"
#endif
#if !__GLIBC_PREREQ(2, 5) /* manylinux1 is glibc 2.5 */
#error "Must run on a glibc >= 2.5 linux environment"
#endif
static PyObject *
spam_system(PyObject *self, PyObject *args)
{
const char *command;
int sts = 0;
if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
#if defined(__GLIBC_PREREQ) && __GLIBC_PREREQ(2, 17) /* manylinux2014 is glibc 2.17 */
// secure_getenv is only available in manylinux2014, ensuring
// that only a manylinux2014 wheel is produced
sts = (int)secure_getenv("NON_EXISTING_ENV_VARIABLE");
#elif defined(__GLIBC_PREREQ) && __GLIBC_PREREQ(2, 10) /* manylinux2010 is glibc 2.12 */
// malloc_info is only available on manylinux2010+
sts = malloc_info(0, stdout);
#endif
if (sts == 0) {
sts = system(command);
}
return PyLong_FromLong(sts);
}
/* Module initialization */
#if PY_MAJOR_VERSION >= 3
#define MOD_INIT(name) PyMODINIT_FUNC PyInit_##name(void)
#define MOD_DEF(m, name, doc, methods, module_state_size) \
static struct PyModuleDef moduledef = { \
PyModuleDef_HEAD_INIT, name, doc, module_state_size, methods, }; \
m = PyModule_Create(&moduledef);
#define MOD_RETURN(m) return m;
#else
#define MOD_INIT(name) PyMODINIT_FUNC init##name(void)
#define MOD_DEF(m, name, doc, methods, module_state_size) \
m = Py_InitModule3(name, methods, doc);
#define MOD_RETURN(m) return;
#endif
static PyMethodDef module_methods[] = {
{"system", (PyCFunction)spam_system, METH_VARARGS,
"Execute a shell command."},
{NULL} /* Sentinel */
};
MOD_INIT(spam)
{
PyObject* m;
MOD_DEF(m,
"spam",
"Example module",
module_methods,
-1)
MOD_RETURN(m)
}
+14
View File
@@ -0,0 +1,14 @@
import os
import utils
project_dir = os.path.dirname(__file__)
def test():
# build the wheels
actual_wheels = utils.cibuildwheel_run(project_dir)
# check that the expected wheels are produced
expected_wheels = utils.expected_wheels('spam', '0.1.0')
assert set(actual_wheels) == set(expected_wheels)
+6
View File
@@ -0,0 +1,6 @@
[metadata]
name = spam
version = attr: spam.__version__
[options]
packages = find:
+8
View File
@@ -0,0 +1,8 @@
from setuptools import (
Extension,
setup,
)
setup(
ext_modules=[Extension('spam.spam', sources=['spam/spam.c'])],
)
+1
View File
@@ -0,0 +1 @@
__version__ = "0.1.0"
@@ -1,23 +1,14 @@
#include <Python.h>
#if defined(__linux__)
#include <malloc.h>
#endif
static PyObject *
spam_system(PyObject *self, PyObject *args)
{
const char *command;
int sts = 0;
int sts;
if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
#if defined(__linux__)
sts = malloc_info(0, stdout);
#endif
if (sts == 0) {
sts = system(command);
}
sts = system(command);
return PyLong_FromLong(sts);
}
@@ -0,0 +1,61 @@
import os
import pytest
import utils
project_dir = os.path.dirname(__file__)
def test_cpp11(tmp_path):
# This test checks that the C++11 standard is supported
# VC++ for Python 2.7 does not support modern standards
add_env = {'CIBW_SKIP': 'cp27-win* pp27-win32', 'CIBW_ENVIRONMENT': 'STANDARD=11'}
actual_wheels = utils.cibuildwheel_run(project_dir, add_env=add_env)
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if 'cp27-cp27m-win' not in w and 'pp27-pypy_73-win32' not in w]
assert set(actual_wheels) == set(expected_wheels)
def test_cpp14():
# This test checks that the C++14 standard is supported
# VC++ for Python 2.7 does not support modern standards
# The manylinux1 docker image does not have a compiler which supports C++11
# Python 3.4 and 3.5 are compiled with MSVC 10, which does not support C++14
add_env = {'CIBW_SKIP': 'cp27-win* pp27-win32 cp35-win*', 'CIBW_ENVIRONMENT': 'STANDARD=14'}
actual_wheels = utils.cibuildwheel_run(project_dir, add_env=add_env)
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if 'cp27-cp27m-win' not in w
and 'pp27-pypy_73-win32' not in w
and 'cp35-cp35m-win' not in w]
assert set(actual_wheels) == set(expected_wheels)
def test_cpp17():
# This test checks that the C++17 standard is supported
# Python and PyPy 2.7 use the `register` keyword which is forbidden in the C++17 standard
# The manylinux1 docker image does not have a compiler which supports C++11
# Python 3.5 and PyPy 3.6 are compiled with MSVC 10, which does not support C++17
if os.environ.get('APPVEYOR_BUILD_WORKER_IMAGE', '') == 'Visual Studio 2015':
pytest.skip('Visual Studio 2015 does not support C++17')
add_env = {'CIBW_SKIP': 'cp27-win* pp27-win32 cp35-win* pp36-win32', 'CIBW_ENVIRONMENT': 'STANDARD=17'}
if utils.platform == 'macos':
add_env['MACOSX_DEPLOYMENT_TARGET'] = '10.13'
actual_wheels = utils.cibuildwheel_run(project_dir, add_env=add_env)
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0', macosx_deployment_target='10.13')
if 'cp27-cp27m-win' not in w
and 'pp27-pypy_73-win32' not in w
and 'cp35-cp35m-win' not in w
and 'pp36-pypy36_pp73-win32' not in w]
assert set(actual_wheels) == set(expected_wheels)
+25
View File
@@ -0,0 +1,25 @@
import os
import platform
from setuptools import (
Extension,
setup,
)
standard = os.environ["STANDARD"]
language_standard = "/std:c++" + standard if platform.system() == "Windows" else "-std=c++" + standard
extra_compile_args = [language_standard, "-DSTANDARD=" + standard]
if standard == "17":
if platform.system() == "Windows":
extra_compile_args.append("/wd5033")
else:
extra_compile_args.append("-Wno-register")
setup(
name="spam",
ext_modules=[Extension('spam', sources=['spam.cpp'], language="c++", extra_compile_args=extra_compile_args)],
version="0.1.0",
)
+62
View File
@@ -0,0 +1,62 @@
#include <Python.h>
#include <string>
// Depending on the requested standard, use a modern C++ feature
// that was introduced in that standard.
#if STANDARD == 11
#include <array>
#elif STANDARD == 14
int a = 100'000;
#elif STANDARD == 17
#include <utility>
auto a = std::pair(5.0, false);
#else
#error Standard needed
#endif
static PyObject *
spam_system(PyObject *self, PyObject *args)
{
const char *command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
sts = system(command);
return PyLong_FromLong(sts);
}
/* Module initialization */
#if PY_MAJOR_VERSION >= 3
#define MOD_INIT(name) PyMODINIT_FUNC PyInit_##name(void)
#define MOD_DEF(m, name, doc, methods, module_state_size) \
static struct PyModuleDef moduledef = { \
PyModuleDef_HEAD_INIT, name, doc, module_state_size, methods, }; \
m = PyModule_Create(&moduledef);
#define MOD_RETURN(m) return m;
#else
#define MOD_INIT(name) PyMODINIT_FUNC init##name(void)
#define MOD_DEF(m, name, doc, methods, module_state_size) \
m = Py_InitModule3(name, methods, doc);
#define MOD_RETURN(m) return;
#endif
static PyMethodDef module_methods[] = {
{"system", (PyCFunction)spam_system, METH_VARARGS,
"Execute a shell command."},
{NULL} /* Sentinel */
};
MOD_INIT(spam)
{
PyObject* m;
MOD_DEF(m,
"spam",
"Example module",
module_methods,
-1)
MOD_RETURN(m)
}
+23
View File
@@ -0,0 +1,23 @@
import os
import utils
def test():
project_dir = os.path.dirname(__file__)
# build the wheels
actual_wheels = utils.cibuildwheel_run(project_dir, add_env={
# write python version information to a temporary file, this is
# checked in setup.py
'CIBW_BEFORE_TEST': '''python -c "import sys; open('/tmp/pythonversion.txt', 'w').write(sys.version)" && python -c "import sys; open('/tmp/pythonprefix.txt', 'w').write(sys.prefix)"''',
'CIBW_BEFORE_TEST_WINDOWS': '''python -c "import sys; open('c:\\pythonversion.txt', 'w').write(sys.version)" && python -c "import sys; open('c:\\pythonprefix.txt', 'w').write(sys.prefix)"''',
'CIBW_TEST_REQUIRES': 'nose',
# the 'false ||' bit is to ensure this command runs in a shell on
# mac/linux.
'CIBW_TEST_COMMAND': 'false || nosetests {project}/test',
'CIBW_TEST_COMMAND_WINDOWS': 'nosetests {project}/test',
})
# also check that we got the right wheels
expected_wheels = utils.expected_wheels('spam', '0.1.0')
assert set(actual_wheels) == set(expected_wheels)
@@ -1,5 +1,6 @@
from setuptools import setup, Extension
setup(
name="spam",
ext_modules=[Extension('spam', sources=['spam.c'])],
+48
View File
@@ -0,0 +1,48 @@
#include <Python.h>
static PyObject *
spam_system(PyObject *self, PyObject *args)
{
const char *command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
sts = system(command);
return PyLong_FromLong(sts);
}
/* Module initialization */
#if PY_MAJOR_VERSION >= 3
#define MOD_INIT(name) PyMODINIT_FUNC PyInit_##name(void)
#define MOD_DEF(m, name, doc, methods, module_state_size) \
static struct PyModuleDef moduledef = { \
PyModuleDef_HEAD_INIT, name, doc, module_state_size, methods, }; \
m = PyModule_Create(&moduledef);
#define MOD_RETURN(m) return m;
#else
#define MOD_INIT(name) PyMODINIT_FUNC init##name(void)
#define MOD_DEF(m, name, doc, methods, module_state_size) \
m = Py_InitModule3(name, methods, doc);
#define MOD_RETURN(m) return;
#endif
static PyMethodDef module_methods[] = {
{"system", (PyCFunction)spam_system, METH_VARARGS,
"Execute a shell command."},
{NULL} /* Sentinel */
};
MOD_INIT(spam)
{
PyObject* m;
MOD_DEF(m,
"spam",
"Example module",
module_methods,
-1)
MOD_RETURN(m)
}
+28
View File
@@ -0,0 +1,28 @@
import sys
import os
from unittest import TestCase
class TestBeforeTest(TestCase):
def test_version(self):
# assert that the Python version as written to pythonversion.txt in the CIBW_BEFORE_TEST step
# is the same one as is currently running.
# because of use symlinks in MacOS run this test is also need
version_file = 'c:\\pythonversion.txt' if sys.platform == 'win32' else '/tmp/pythonversion.txt'
with open(version_file) as f:
stored_version = f.read()
print('stored_version', stored_version)
print('sys.version', sys.version)
assert stored_version == sys.version
def test_prefix(self):
# check that the prefix also was written
prefix_file = 'c:\\pythonprefix.txt' if sys.platform == 'win32' else '/tmp/pythonprefix.txt'
with open(prefix_file) as f:
stored_prefix = f.read()
print('stored_prefix', stored_prefix)
print('sys.prefix', sys.prefix)
# Works around path-comparison bugs caused by short-paths on Windows e.g.
# vssadm~1 instead of vssadministrator
assert os.stat(stored_prefix) == os.stat(sys.prefix)
@@ -0,0 +1,125 @@
import os
import re
import pytest
import textwrap
import cibuildwheel.util
import utils
VERSION_REGEX = r'([\w-]+)==([^\s]+)'
def get_versions_from_constraint_file(constraint_file):
with open(constraint_file, encoding='utf8') as f:
constraint_file_text = f.read()
versions = {}
for package, version in re.findall(VERSION_REGEX, constraint_file_text):
versions[package] = version
return versions
@pytest.mark.parametrize('python_version', ['2.7', '3.5', '3.8'])
def test_pinned_versions(python_version):
if utils.platform == 'linux':
pytest.skip('linux doesn\'t pin individual tool versions, it pins manylinux images instead')
project_dir = os.path.dirname(__file__)
build_environment = {}
if python_version == '2.7':
constraint_filename = 'constraints-python27.txt'
build_pattern = '[cp]p27-*'
elif python_version == '3.5':
constraint_filename = 'constraints-python35.txt'
build_pattern = '[cp]p35-*'
else:
constraint_filename = 'constraints.txt'
build_pattern = '[cp]p38-*'
constraint_file = os.path.join(cibuildwheel.util.resources_dir, constraint_filename)
constraint_versions = get_versions_from_constraint_file(constraint_file)
for package in ['pip', 'setuptools', 'wheel', 'virtualenv']:
env_name = 'EXPECTED_{}_VERSION'.format(package.upper())
build_environment[env_name] = constraint_versions[package]
cibw_environment_option = ' '.join(
['{}={}'.format(k, v) for k, v in build_environment.items()]
)
# build and test the wheels
actual_wheels = utils.cibuildwheel_run(project_dir, add_env={
'CIBW_BUILD': build_pattern,
'CIBW_ENVIRONMENT': cibw_environment_option,
})
# also check that we got the right wheels
if python_version == '2.7':
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if '-cp27' in w or '-pp27' in w]
elif python_version == '3.5':
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if '-cp35' in w or '-pp35' in w]
elif python_version == '3.8':
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if '-cp38' in w or '-pp38' in w]
else:
raise ValueError('unhandled python version')
assert set(actual_wheels) == set(expected_wheels)
@pytest.mark.parametrize('python_version', ['2.7', '3.x'])
def test_dependency_constraints_file(tmp_path, python_version):
if utils.platform == 'linux':
pytest.skip('linux doesn\'t pin individual tool versions, it pins manylinux images instead')
project_dir = os.path.dirname(__file__)
tool_versions = {
'pip': '20.0.2',
'setuptools': '44.0.0' if python_version == '2.7' else '46.0.0',
'wheel': '0.34.2',
'virtualenv': '20.0.10',
}
constraints_file = tmp_path / 'constraints.txt'
constraints_file.write_text(textwrap.dedent(
'''
pip=={pip}
setuptools=={setuptools}
wheel=={wheel}
virtualenv=={virtualenv}
'''.format(**tool_versions)
))
build_environment = {}
for package_name, version in tool_versions.items():
env_name = 'EXPECTED_{}_VERSION'.format(package_name.upper())
build_environment[env_name] = version
cibw_environment_option = ' '.join(
['{}={}'.format(k, v) for k, v in build_environment.items()]
)
# build and test the wheels
actual_wheels = utils.cibuildwheel_run(project_dir, add_env={
'CIBW_BUILD': '[cp]p27-*' if python_version == '2.7' else '[cp]p3?-*',
'CIBW_ENVIRONMENT': cibw_environment_option,
'CIBW_DEPENDENCY_VERSIONS': str(constraints_file),
})
# also check that we got the right wheels
if python_version == '2.7':
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if '-cp27' in w or '-pp27' in w]
else:
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if '-cp27' not in w and '-pp27' not in w]
assert set(actual_wheels) == set(expected_wheels)
+31
View File
@@ -0,0 +1,31 @@
import os
import subprocess
from setuptools import (
Extension,
setup,
)
versions_output_text = subprocess.check_output(
['pip', 'freeze', '--all', '-qq'],
universal_newlines=True,
)
versions = versions_output_text.strip().splitlines()
# `versions` now looks like:
# ['pip==x.x.x', 'setuptools==x.x.x', 'wheel==x.x.x']
print('Gathered versions', versions)
for package_name in ['pip', 'setuptools', 'wheel']:
env_name = 'EXPECTED_{}_VERSION'.format(package_name.upper())
expected_version = os.environ[env_name]
assert '{}=={}'.format(package_name, expected_version) in versions, (
'error: {} version should equal {}'.format(package_name, expected_version)
)
setup(
name="spam",
ext_modules=[Extension('spam', sources=['spam.c'])],
version="0.1.0",
)
+48
View File
@@ -0,0 +1,48 @@
#include <Python.h>
static PyObject *
spam_system(PyObject *self, PyObject *args)
{
const char *command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
sts = system(command);
return PyLong_FromLong(sts);
}
/* Module initialization */
#if PY_MAJOR_VERSION >= 3
#define MOD_INIT(name) PyMODINIT_FUNC PyInit_##name(void)
#define MOD_DEF(m, name, doc, methods, module_state_size) \
static struct PyModuleDef moduledef = { \
PyModuleDef_HEAD_INIT, name, doc, module_state_size, methods, }; \
m = PyModule_Create(&moduledef);
#define MOD_RETURN(m) return m;
#else
#define MOD_INIT(name) PyMODINIT_FUNC init##name(void)
#define MOD_DEF(m, name, doc, methods, module_state_size) \
m = Py_InitModule3(name, methods, doc);
#define MOD_RETURN(m) return;
#endif
static PyMethodDef module_methods[] = {
{"system", (PyCFunction)spam_system, METH_VARARGS,
"Execute a shell command."},
{NULL} /* Sentinel */
};
MOD_INIT(spam)
{
PyObject* m;
MOD_DEF(m,
"spam",
"Example module",
module_methods,
-1)
MOD_RETURN(m)
}
+1
View File
@@ -0,0 +1 @@
print('before_build.py executed!')
@@ -0,0 +1,25 @@
import os
import utils
project_dir = os.path.dirname(__file__)
def test(capfd):
package_dir = os.path.join(project_dir, 'src', 'spam')
# build the wheels
actual_wheels = utils.cibuildwheel_run(project_dir, package_dir=package_dir, add_env={
'CIBW_BEFORE_BUILD': 'python {project}/bin/before_build.py',
'CIBW_TEST_COMMAND': 'python {package}/test/run_tests.py',
# this shouldn't depend on the version of python, so build only
# CPython 3.6
'CIBW_BUILD': 'cp36-*',
})
# check that the expected wheels are produced
expected_wheels = [w for w in utils.expected_wheels('spam', '0.1.0')
if 'cp36' in w]
assert set(actual_wheels) == set(expected_wheels)
captured = capfd.readouterr()
assert "before_build.py executed!" in captured.out
assert "run_tests.py executed!" in captured.out
+8
View File
@@ -0,0 +1,8 @@
from setuptools import Extension, setup
setup(
name="spam",
ext_modules=[Extension('spam', sources=['spam.c'])],
version="0.1.0",
)
+48
View File
@@ -0,0 +1,48 @@
#include <Python.h>
static PyObject *
spam_system(PyObject *self, PyObject *args)
{
const char *command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
sts = system(command);
return PyLong_FromLong(sts);
}
/* Module initialization */
#if PY_MAJOR_VERSION >= 3
#define MOD_INIT(name) PyMODINIT_FUNC PyInit_##name(void)
#define MOD_DEF(m, name, doc, methods, module_state_size) \
static struct PyModuleDef moduledef = { \
PyModuleDef_HEAD_INIT, name, doc, module_state_size, methods, }; \
m = PyModule_Create(&moduledef);
#define MOD_RETURN(m) return m;
#else
#define MOD_INIT(name) PyMODINIT_FUNC init##name(void)
#define MOD_DEF(m, name, doc, methods, module_state_size) \
m = Py_InitModule3(name, methods, doc);
#define MOD_RETURN(m) return;
#endif
static PyMethodDef module_methods[] = {
{"system", (PyCFunction)spam_system, METH_VARARGS,
"Execute a shell command."},
{NULL} /* Sentinel */
};
MOD_INIT(spam)
{
PyObject* m;
MOD_DEF(m,
"spam",
"Example module",
module_methods,
-1)
MOD_RETURN(m)
}
@@ -0,0 +1 @@
print('run_tests.py executed!')
+4
View File
@@ -0,0 +1,4 @@
import os
import sys
sys.path.append(os.path.join(os.path.dirname(__file__), 'shared'))
+96 -61
View File
@@ -4,12 +4,29 @@ Utility functions used by the cibuildwheel tests.
This file is added to the PYTHONPATH in the test runner at bin/run_test.py.
'''
import subprocess, sys, os
import os
import platform as pm
import shutil
import subprocess
import sys
from contextlib import contextmanager
from tempfile import mkdtemp
IS_WINDOWS_RUNNING_ON_AZURE = os.path.exists('C:\\hostedtoolcache')
IS_WINDOWS_RUNNING_ON_TRAVIS = os.environ.get('TRAVIS_OS_NAME') == 'windows'
# Python 2 does not have a tempfile.TemporaryDirectory context manager
@contextmanager
def TemporaryDirectoryIfNone(path):
_path = path or mkdtemp()
try:
yield _path
finally:
if path is None:
shutil.rmtree(_path)
def cibuildwheel_get_build_identifiers(project_path, env=None):
'''
Returns the list of build identifiers that cibuildwheel will try to build
@@ -24,86 +41,104 @@ def cibuildwheel_get_build_identifiers(project_path, env=None):
return cmd_output.strip().split('\n')
def cibuildwheel_run(project_path, env=None, add_env=None):
def cibuildwheel_run(project_path, package_dir='.', env=None, add_env=None, output_dir=None):
'''
Runs cibuildwheel as a subprocess, building the project at project_path.
Uses the current Python interpreter.
Configure settings using env.
:param project_path: path of the project to be built.
:param package_dir: path of the package to be built. Can be absolute, or
relative to project_path.
:param env: full environment to be used, os.environ if None
:param add_env: environment used to update env
:param output_dir: directory where wheels are saved. If None, a temporary
directory will be used for the duration of the command.
:return: list of built wheels (file names).
'''
if env is None:
env = os.environ.copy()
# If present in the host environment, remove the MACOSX_DEPLOYMENT_TARGET for consistency
env.pop('MACOSX_DEPLOYMENT_TARGET', None)
if add_env is not None:
env.update(add_env)
subprocess.check_call(
[sys.executable, '-m', 'cibuildwheel', project_path],
env=env,
)
with TemporaryDirectoryIfNone(output_dir) as _output_dir:
subprocess.check_call(
[sys.executable, '-m', 'cibuildwheel', '--output-dir', str(_output_dir), package_dir],
env=env,
cwd=project_path,
)
wheels = os.listdir(_output_dir)
return wheels
def expected_wheels(package_name, package_version):
def expected_wheels(package_name, package_version, manylinux_versions=None,
macosx_deployment_target='10.9'):
'''
Returns a list of expected wheels from a run of cibuildwheel.
'''
if platform == 'linux':
templates = [
'{package_name}-{package_version}-cp27-cp27m-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp27-cp27mu-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp35-cp35m-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp36-cp36m-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp37-cp37m-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp38-cp38-manylinux1_x86_64.whl',
'{package_name}-{package_version}-cp27-cp27m-manylinux2010_x86_64.whl',
'{package_name}-{package_version}-cp27-cp27mu-manylinux2010_x86_64.whl',
'{package_name}-{package_version}-cp35-cp35m-manylinux2010_x86_64.whl',
'{package_name}-{package_version}-cp36-cp36m-manylinux2010_x86_64.whl',
'{package_name}-{package_version}-cp37-cp37m-manylinux2010_x86_64.whl',
'{package_name}-{package_version}-cp38-cp38-manylinux2010_x86_64.whl',
'{package_name}-{package_version}-cp27-cp27m-manylinux1_i686.whl',
'{package_name}-{package_version}-cp27-cp27mu-manylinux1_i686.whl',
'{package_name}-{package_version}-cp35-cp35m-manylinux1_i686.whl',
'{package_name}-{package_version}-cp36-cp36m-manylinux1_i686.whl',
'{package_name}-{package_version}-cp37-cp37m-manylinux1_i686.whl',
'{package_name}-{package_version}-cp38-cp38-manylinux1_i686.whl',
'{package_name}-{package_version}-cp27-cp27m-manylinux2010_i686.whl',
'{package_name}-{package_version}-cp27-cp27mu-manylinux2010_i686.whl',
'{package_name}-{package_version}-cp35-cp35m-manylinux2010_i686.whl',
'{package_name}-{package_version}-cp36-cp36m-manylinux2010_i686.whl',
'{package_name}-{package_version}-cp37-cp37m-manylinux2010_i686.whl',
'{package_name}-{package_version}-cp38-cp38-manylinux2010_i686.whl',
]
elif platform == 'windows':
templates = [
'{package_name}-{package_version}-cp27-cp27m-win32.whl',
'{package_name}-{package_version}-cp35-cp35m-win32.whl',
'{package_name}-{package_version}-cp36-cp36m-win32.whl',
'{package_name}-{package_version}-cp37-cp37m-win32.whl',
'{package_name}-{package_version}-cp38-cp38-win32.whl',
'{package_name}-{package_version}-cp27-cp27m-win_amd64.whl',
'{package_name}-{package_version}-cp35-cp35m-win_amd64.whl',
'{package_name}-{package_version}-cp36-cp36m-win_amd64.whl',
'{package_name}-{package_version}-cp37-cp37m-win_amd64.whl',
'{package_name}-{package_version}-cp38-cp38-win_amd64.whl',
]
elif platform == 'macos':
templates = [
'{package_name}-{package_version}-cp27-cp27m-macosx_10_6_intel.whl',
'{package_name}-{package_version}-cp35-cp35m-macosx_10_6_intel.whl',
'{package_name}-{package_version}-cp36-cp36m-macosx_10_6_intel.whl',
'{package_name}-{package_version}-cp37-cp37m-macosx_10_6_intel.whl',
'{package_name}-{package_version}-cp38-cp38-macosx_10_9_x86_64.whl',
]
else:
raise Exception('unsupported platform')
# per PEP 425 (https://www.python.org/dev/peps/pep-0425/), wheel files shall have name of the form
# {distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl
# {python tag} and {abi tag} are closely related to the python interpreter used to build the wheel
# so we'll merge them below as python_abi_tag
if manylinux_versions is None:
if pm.machine() == 'x86_64':
manylinux_versions = ['manylinux1', 'manylinux2010']
else:
manylinux_versions = ['manylinux2014']
python_abi_tags = ['cp35-cp35m', 'cp36-cp36m', 'cp37-cp37m', 'cp38-cp38']
if pm.machine() in ['x86_64', 'AMD64', 'x86']:
python_abi_tags += ['cp27-cp27m', 'pp27-pypy_73', 'pp36-pypy36_pp73']
if platform == 'linux':
python_abi_tags.append('cp27-cp27mu') # python 2.7 has 2 different ABI on manylinux
wheels = []
for python_abi_tag in python_abi_tags:
platform_tags = []
if platform == 'linux':
architectures = [pm.machine()]
if pm.machine() == 'x86_64' and python_abi_tag.startswith('cp'):
architectures.append('i686')
platform_tags = [
'{}_{}'.format(manylinux_version, architecture)
for architecture in architectures
for manylinux_version in manylinux_versions
]
elif platform == 'windows':
if python_abi_tag.startswith('cp'):
platform_tags = ['win32', 'win_amd64']
else:
platform_tags = ['win32']
elif platform == 'macos':
platform_tags = ['macosx_{}_x86_64'.format(macosx_deployment_target.replace('.', '_'))]
else:
raise Exception('unsupported platform')
for platform_tag in platform_tags:
wheels.append('{package_name}-{package_version}-{python_abi_tag}-{platform_tag}.whl'.format(
package_name=package_name, package_version=package_version,
python_abi_tag=python_abi_tag, platform_tag=platform_tag
))
if IS_WINDOWS_RUNNING_ON_TRAVIS:
# Python 2.7 isn't supported on Travis.
templates = [t for t in templates if '-cp27-' not in t]
wheels = [w for w in wheels if '-cp27-' not in w and '-pp2' not in w]
return wheels
return [filename.format(package_name=package_name, package_version=package_version)
for filename in templates]
platform = None
+30
View File
@@ -0,0 +1,30 @@
from cibuildwheel.util import DependencyConstraints
import os
def test_defaults():
dependency_constraints = DependencyConstraints.with_defaults()
project_root = os.path.dirname(os.path.dirname(__file__))
resources_dir = os.path.join(project_root, 'cibuildwheel', 'resources')
assert os.path.samefile(
dependency_constraints.base_file_path,
os.path.join(resources_dir, 'constraints.txt')
)
assert os.path.samefile(
dependency_constraints.get_for_python_version('3.8'),
os.path.join(resources_dir, 'constraints.txt')
)
assert os.path.samefile(
dependency_constraints.get_for_python_version('3.6'),
os.path.join(resources_dir, 'constraints-python36.txt')
)
assert os.path.samefile(
dependency_constraints.get_for_python_version('3.5'),
os.path.join(resources_dir, 'constraints-python35.txt')
)
assert os.path.samefile(
dependency_constraints.get_for_python_version('2.7'),
os.path.join(resources_dir, 'constraints-python27.txt')
)
+8
View File
@@ -1,4 +1,5 @@
import os
from cibuildwheel.environment import parse_environment
@@ -13,6 +14,7 @@ def test_basic_parsing():
assert environment_dict == {'VAR': '1', 'VBR': '2'}
assert environment_cmds == ['export VAR=1', 'export VBR=2']
def test_quotes():
environment_recipe = parse_environment('A=1 VAR="1 NOT_A_VAR=2" VBR=\'vbr\'')
@@ -24,6 +26,7 @@ def test_quotes():
assert environment_dict == {'A': '1', 'VAR': '1 NOT_A_VAR=2', 'VBR': 'vbr'}
assert environment_cmds == ['export A=1', 'export VAR="1 NOT_A_VAR=2"', 'export VBR=\'vbr\'']
def test_inheritance():
environment_recipe = parse_environment('PATH=$PATH:/usr/local/bin')
@@ -35,6 +38,7 @@ def test_inheritance():
assert environment_dict == {'PATH': '/usr/bin:/usr/local/bin'}
assert environment_cmds == ['export PATH=$PATH:/usr/local/bin']
def test_shell_eval():
environment_recipe = parse_environment('VAR="$(echo "a test" string)"')
@@ -49,6 +53,7 @@ def test_shell_eval():
assert environment_dict['VAR'] == 'a test string'
assert environment_cmds == ['export VAR="$(echo "a test" string)"']
def test_shell_eval_and_env():
environment_recipe = parse_environment('VAR="$(echo "$PREV_VAR" string)"')
@@ -60,6 +65,7 @@ def test_shell_eval_and_env():
assert environment_dict == {'PREV_VAR': '1 2 3', 'VAR': '1 2 3 string'}
assert environment_cmds == ['export VAR="$(echo "$PREV_VAR" string)"']
def test_empty_var():
environment_recipe = parse_environment('CFLAGS=')
@@ -71,6 +77,7 @@ def test_empty_var():
assert environment_dict == {'CFLAGS': ''}
assert environment_cmds == ['export CFLAGS=']
def test_no_vars():
environment_recipe = parse_environment('')
@@ -80,6 +87,7 @@ def test_no_vars():
assert environment_dict == {}
assert environment_cmds == []
def test_no_vars_pass_through():
environment_recipe = parse_environment('')
+69
View File
@@ -0,0 +1,69 @@
import os
import subprocess
import sys
import pytest
from cibuildwheel import (
linux,
macos,
util,
windows,
)
class ArgsInterceptor:
def __call__(self, *args, **kwargs):
self.args = args
self.kwargs = kwargs
MOCK_PACKAGE_DIR = 'some_package_dir'
@pytest.fixture(autouse=True)
def mock_protection(monkeypatch):
'''
Ensure that a unit test will never actually run a cibuildwheel 'build'
function, which shouldn't be run on a developer's machine
'''
def fail_on_call(*args, **kwargs):
raise RuntimeError("This should never be called")
monkeypatch.setattr(subprocess, 'Popen', fail_on_call)
monkeypatch.setattr(util, 'download', fail_on_call)
monkeypatch.setattr(windows, 'build', fail_on_call)
monkeypatch.setattr(linux, 'build', fail_on_call)
monkeypatch.setattr(macos, 'build', fail_on_call)
@pytest.fixture(autouse=True)
def fake_package_dir(monkeypatch):
'''
Monkey-patch enough for the main() function to run
'''
real_os_path_exists = os.path.exists
def mock_os_path_exists(path):
if path == os.path.join(MOCK_PACKAGE_DIR, 'setup.py'):
return True
else:
return real_os_path_exists(path)
monkeypatch.setattr(os.path, 'exists', mock_os_path_exists)
monkeypatch.setattr(sys, 'argv', ['cibuildwheel', MOCK_PACKAGE_DIR])
@pytest.fixture(params=['linux', 'macos', 'windows'])
def platform(request, monkeypatch):
platform_value = request.param
monkeypatch.setenv('CIBW_PLATFORM', platform_value)
return platform_value
@pytest.fixture
def intercepted_build_args(platform, monkeypatch):
intercepted = ArgsInterceptor()
monkeypatch.setattr(globals()[platform], 'build', intercepted)
return intercepted
+234
View File
@@ -0,0 +1,234 @@
import sys
from fnmatch import fnmatch
import pytest
from cibuildwheel.__main__ import main
from cibuildwheel.environment import ParsedEnvironment
from cibuildwheel.util import BuildSelector
# CIBW_PLATFORM is tested in main_platform_test.py
def test_output_dir(platform, intercepted_build_args, monkeypatch):
OUTPUT_DIR = 'some_output_dir'
monkeypatch.setenv('CIBW_OUTPUT_DIR', OUTPUT_DIR)
main()
assert intercepted_build_args.args[0].output_dir == OUTPUT_DIR
def test_output_dir_default(platform, intercepted_build_args, monkeypatch):
main()
assert intercepted_build_args.args[0].output_dir == 'wheelhouse'
@pytest.mark.parametrize('also_set_environment', [False, True])
def test_output_dir_argument(also_set_environment, platform, intercepted_build_args, monkeypatch):
OUTPUT_DIR = 'some_output_dir'
monkeypatch.setattr(sys, 'argv', sys.argv + ['--output-dir', OUTPUT_DIR])
if also_set_environment:
monkeypatch.setenv('CIBW_OUTPUT_DIR', 'not_this_output_dir')
main()
assert intercepted_build_args.args[0].output_dir == OUTPUT_DIR
def test_build_selector(platform, intercepted_build_args, monkeypatch):
BUILD = 'some build* *-selector'
SKIP = 'some skip* *-selector'
monkeypatch.setenv('CIBW_BUILD', BUILD)
monkeypatch.setenv('CIBW_SKIP', SKIP)
main()
intercepted_build_selector = intercepted_build_args.args[0].build_selector
assert isinstance(intercepted_build_selector, BuildSelector)
assert intercepted_build_selector('build-this')
assert not intercepted_build_selector('skip-that')
# This unit test is just testing the options of 'main'
# Unit tests for BuildSelector are in build_selector_test.py
@pytest.mark.parametrize('architecture, image, full_image', [
('x86_64', None, 'quay.io/pypa/manylinux2010_x86_64:*'),
('x86_64', 'manylinux1', 'quay.io/pypa/manylinux1_x86_64:*'),
('x86_64', 'manylinux2010', 'quay.io/pypa/manylinux2010_x86_64:*'),
('x86_64', 'manylinux2014', 'quay.io/pypa/manylinux2014_x86_64:*'),
('x86_64', 'custom_image', 'custom_image'),
('i686', None, 'quay.io/pypa/manylinux2010_i686:*'),
('i686', 'manylinux1', 'quay.io/pypa/manylinux1_i686:*'),
('i686', 'manylinux2010', 'quay.io/pypa/manylinux2010_i686:*'),
('i686', 'manylinux2014', 'quay.io/pypa/manylinux2014_i686:*'),
('i686', 'custom_image', 'custom_image'),
('pypy_x86_64', None, 'pypywheels/manylinux2010-pypy_x86_64:*'),
('pypy_x86_64', 'manylinux1', 'manylinux1'), # Does not exist
('pypy_x86_64', 'manylinux2010', 'pypywheels/manylinux2010-pypy_x86_64:*'),
('pypy_x86_64', 'manylinux2014', 'manylinux2014'), # Does not exist (yet)
('pypy_x86_64', 'custom_image', 'custom_image'),
])
def test_manylinux_images(architecture, image, full_image, platform, intercepted_build_args, monkeypatch):
if image is not None:
monkeypatch.setenv('CIBW_MANYLINUX_' + architecture.upper() + '_IMAGE', image)
main()
if platform == 'linux':
assert fnmatch(
intercepted_build_args.args[0].manylinux_images[architecture],
full_image
)
else:
assert intercepted_build_args.args[0].manylinux_images is None
def get_default_repair_command(platform):
if platform == 'linux':
return 'auditwheel repair -w {dest_dir} {wheel}'
elif platform == 'macos':
return 'delocate-listdeps {wheel} && delocate-wheel --require-archs x86_64 -w {dest_dir} {wheel}'
elif platform == 'windows':
return ''
else:
raise ValueError('Unknown platform', platform)
@pytest.mark.parametrize('repair_command', [None, 'repair', 'repair -w {dest_dir} {wheel}'])
@pytest.mark.parametrize('platform_specific', [False, True])
def test_repair_command(repair_command, platform_specific, platform, intercepted_build_args, monkeypatch):
if repair_command is not None:
if platform_specific:
monkeypatch.setenv('CIBW_REPAIR_WHEEL_COMMAND_' + platform.upper(), repair_command)
monkeypatch.setenv('CIBW_REPAIR_WHEEL_COMMAND', 'overwritten')
else:
monkeypatch.setenv('CIBW_REPAIR_WHEEL_COMMAND', repair_command)
main()
expected_repair = repair_command or get_default_repair_command(platform)
assert intercepted_build_args.args[0].repair_command == expected_repair
@pytest.mark.parametrize('environment', [
{},
{'something': 'value'},
{'something': 'value', 'something_else': 'other_value'}
])
@pytest.mark.parametrize('platform_specific', [False, True])
def test_environment(environment, platform_specific, platform, intercepted_build_args, monkeypatch):
env_string = ' '.join(['{}={}'.format(k, v) for k, v in environment.items()])
if platform_specific:
monkeypatch.setenv('CIBW_ENVIRONMENT_' + platform.upper(), env_string)
monkeypatch.setenv('CIBW_ENVIRONMENT', 'overwritten')
else:
monkeypatch.setenv('CIBW_ENVIRONMENT', env_string)
main()
intercepted_environment = intercepted_build_args.args[0].environment
assert isinstance(intercepted_environment, ParsedEnvironment)
assert intercepted_environment.as_dictionary(prev_environment={}) == environment
@pytest.mark.parametrize('test_requires', [None, 'requirement other_requirement'])
@pytest.mark.parametrize('platform_specific', [False, True])
def test_test_requires(test_requires, platform_specific, platform, intercepted_build_args, monkeypatch):
if test_requires is not None:
if platform_specific:
monkeypatch.setenv('CIBW_TEST_REQUIRES_' + platform.upper(), test_requires)
monkeypatch.setenv('CIBW_TEST_REQUIRES', 'overwritten')
else:
monkeypatch.setenv('CIBW_TEST_REQUIRES', test_requires)
main()
assert intercepted_build_args.args[0].test_requires == (test_requires or '').split()
@pytest.mark.parametrize('test_extras', [None, 'extras'])
@pytest.mark.parametrize('platform_specific', [False, True])
def test_test_extras(test_extras, platform_specific, platform, intercepted_build_args, monkeypatch):
if test_extras is not None:
if platform_specific:
monkeypatch.setenv('CIBW_TEST_EXTRAS_' + platform.upper(), test_extras)
monkeypatch.setenv('CIBW_TEST_EXTRAS', 'overwritten')
else:
monkeypatch.setenv('CIBW_TEST_EXTRAS', test_extras)
main()
assert intercepted_build_args.args[0].test_extras == ('[' + test_extras + ']' if test_extras else '')
@pytest.mark.parametrize('test_command', [None, 'test --command'])
@pytest.mark.parametrize('platform_specific', [False, True])
def test_test_command(test_command, platform_specific, platform, intercepted_build_args, monkeypatch):
if test_command is not None:
if platform_specific:
monkeypatch.setenv('CIBW_TEST_COMMAND_' + platform.upper(), test_command)
monkeypatch.setenv('CIBW_TEST_COMMAND', 'overwritten')
else:
monkeypatch.setenv('CIBW_TEST_COMMAND', test_command)
main()
assert intercepted_build_args.args[0].test_command == test_command
@pytest.mark.parametrize('before_build', [None, 'before --build'])
@pytest.mark.parametrize('platform_specific', [False, True])
def test_before_build(before_build, platform_specific, platform, intercepted_build_args, monkeypatch):
if before_build is not None:
if platform_specific:
monkeypatch.setenv('CIBW_BEFORE_BUILD_' + platform.upper(), before_build)
monkeypatch.setenv('CIBW_BEFORE_BUILD', 'overwritten')
else:
monkeypatch.setenv('CIBW_BEFORE_BUILD', before_build)
main()
assert intercepted_build_args.args[0].before_build == before_build
@pytest.mark.parametrize('build_verbosity', [None, 0, 2, -2, 4, -4])
@pytest.mark.parametrize('platform_specific', [False, True])
def test_build_verbosity(build_verbosity, platform_specific, platform, intercepted_build_args, monkeypatch):
if build_verbosity is not None:
if platform_specific:
monkeypatch.setenv('CIBW_BUILD_VERBOSITY_' + platform.upper(), str(build_verbosity))
monkeypatch.setenv('CIBW_BUILD_VERBOSITY', 'overwritten')
else:
monkeypatch.setenv('CIBW_BUILD_VERBOSITY', str(build_verbosity))
main()
expected_verbosity = max(-3, min(3, int(build_verbosity or 0)))
assert intercepted_build_args.args[0].build_verbosity == expected_verbosity
@pytest.mark.parametrize('option_name', ['CIBW_BUILD', 'CIBW_SKIP'])
@pytest.mark.parametrize('option_value, build_selector_patterns', [
('*-manylinux1_*', ['*-manylinux_*']),
('*-macosx_10_6_intel', ['*-macosx_x86_64']),
('*-macosx_10_9_x86_64', ['*-macosx_x86_64']),
('cp37-macosx_10_9_x86_64', ['cp37-macosx_x86_64']),
])
def test_build_selector_migrations(intercepted_build_args, monkeypatch, option_name, option_value, build_selector_patterns):
monkeypatch.setenv(option_name, option_value)
main()
intercepted_build_selector = intercepted_build_args.args[0].build_selector
assert isinstance(intercepted_build_selector, BuildSelector)
if option_name == 'CIBW_BUILD':
assert intercepted_build_selector.build_patterns == build_selector_patterns
else:
assert intercepted_build_selector.skip_patterns == build_selector_patterns
@@ -0,0 +1,62 @@
import sys
import pytest
from cibuildwheel.__main__ import main
from conftest import MOCK_PACKAGE_DIR # noqa: I100
def test_unknown_platform_non_ci(monkeypatch, capsys):
monkeypatch.delenv('CI', raising=False)
monkeypatch.delenv('BITRISE_BUILD_NUMBER', raising=False)
monkeypatch.delenv('AZURE_HTTP_USER_AGENT', raising=False)
monkeypatch.delenv('GITHUB_WORKFLOW', raising=False)
monkeypatch.delenv('CIBW_PLATFORM', raising=False)
with pytest.raises(SystemExit) as exit:
main()
assert exit.value.code == 2
_, err = capsys.readouterr()
assert 'cibuildwheel: Unable to detect platform.' in err
assert 'cibuildwheel should run on your CI server' in err
def test_unknown_platform_on_ci(monkeypatch, capsys):
monkeypatch.setenv('CI', 'true')
monkeypatch.setattr(sys, 'platform', 'nonexistent')
monkeypatch.delenv('CIBW_PLATFORM', raising=False)
with pytest.raises(SystemExit) as exit:
main()
assert exit.value.code == 2
_, err = capsys.readouterr()
assert 'cibuildwheel: Unable to detect platform from "sys.platform"' in err
def test_unknown_platform(monkeypatch, capsys):
monkeypatch.setenv('CIBW_PLATFORM', 'nonexistent')
with pytest.raises(SystemExit) as exit:
main()
_, err = capsys.readouterr()
assert exit.value.code == 2
assert 'cibuildwheel: Unsupported platform: nonexistent' in err
def test_platform_argument(platform, intercepted_build_args, monkeypatch):
monkeypatch.setenv('CIBW_PLATFORM', 'nonexistent')
monkeypatch.setattr(sys, 'argv', sys.argv + ['--platform', platform])
main()
assert intercepted_build_args.args[0].package_dir == MOCK_PACKAGE_DIR
def test_platform_environment(platform, intercepted_build_args, monkeypatch):
main()
assert intercepted_build_args.args[0].package_dir == MOCK_PACKAGE_DIR