Compare commits

...
19 Commits
Author SHA1 Message Date
Joe Rickerby 63720bbc67 Bump version: v2.0.0b1 2021-07-09 18:56:12 +01:00
Henry SchreinerandJoe Rickerby 87499d4230 ci: fix circle ci image (#750)
* ci: fix circle min image

* Apply suggestions from code review

Co-authored-by: Joe Rickerby <joerick@mac.com>

Co-authored-by: Joe Rickerby <joerick@mac.com>
2021-07-09 13:30:06 -04:00
Moritz 0c32cc162e docs: fix toml syntax in example config code. (#751) 2021-07-09 12:23:06 -04:00
Henry SchreinerandJoe Rickerby 4307b52ff2 fix: support space in constraints file path (#738)
* tests: add space in constraints file

* fix: use uri strings in PIP_CONSTRAINTS

* WIP: try using quotes instead of uri

* Update cibuildwheel/windows.py

* Revert "Update cibuildwheel/windows.py"

This reverts commit 0951425524166677fdfd8df7e6bbbc9782ea2fb0.

* Revert "WIP: try using quotes instead of uri"

This reverts commit 5790eda4c6f231d5958f4cbd720683f787eea042.

* fix: work around pip bug with URI handling

* WIP: try biger hack

* fix: try using temporary file

* refactor: use common prefix for install dir on Windows

* Use more flexible type PathOrStr to simplify

* Rename variables to make slightly clearer

Co-authored-by: Joe Rickerby <joerick@mac.com>
2021-07-09 09:16:20 -04:00
Joe Rickerby 30954282fd Merge pull request #739 from pypa/options-table
Update options table in README for CIBW_BUILD_FRONTEND
2021-07-08 09:03:35 +01:00
pre-commit-ci[bot]andpre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> 9e82a3ea32 [pre-commit.ci] pre-commit autoupdate (#748)
updates:
- [github.com/asottile/pyupgrade: v2.19.4 → v2.20.0](https://github.com/asottile/pyupgrade/compare/v2.19.4...v2.20.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-06 00:48:50 +02:00
Joe Rickerby c2ab8392f5 Merge pull request #747 from pypa/update-dependencies-pr
[Bot] Update dependencies
2021-07-05 11:27:46 +01:00
mayeut a49312635c Update dependencies 2021-07-05 06:10:47 +00:00
Matthieu Darbois fcf8892abc chore: update packaging dev requirement (#746)
* chore: update packaging dev requirement

Requires packaging>=21.0
This allows to simplify a bit `bin/update_pythons.py` with regards to typing.

* update packaging dev requirement for pre-commit
2021-07-03 19:07:52 +02:00
Henry Schreiner b552fbd016 fix(action): Output a single stream (#741) 2021-06-30 18:10:01 -04:00
Henry Schreiner 46fd5d9de4 docs: clarify location of dependabot file (#731)
* docs: clarify location of dependabot file

* Update faq.md
2021-06-29 23:33:09 -04:00
Joe Rickerby 79bf489a08 Update options table in README for CIBW_BUILD_FRONTEND 2021-06-29 20:50:16 +01:00
pre-commit-ci[bot]andpre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> ec424751f5 [pre-commit.ci] pre-commit autoupdate (#737)
updates:
- [github.com/pre-commit/mirrors-mypy: v0.902 → v0.910](https://github.com/pre-commit/mirrors-mypy/compare/v0.902...v0.910)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-06-28 14:13:33 -04:00
cibuildwheel-bot[bot]andmayeut f879db2966 Update dependencies (#736)
Co-authored-by: mayeut <mayeut@users.noreply.github.com>
2021-06-28 09:17:01 -04:00
Joe Rickerby 489d560fff Update changelog 2021-06-25 13:59:08 +01:00
Joe Rickerby d6cf349b1a Bump version: v2.0.0a4 2021-06-25 13:55:15 +01:00
Henry SchreinerandJoe Rickerby 515e9be2c4 feat: support pypa build (#521)
* feat: support pypa build

fix: update `cibuildwheel.linux.troubleshoot` to know about build

Workaround issue with PyPy venv module by installing build[virtualenv]

fix: test_dependency_constraints_file when using build

fix: test/test_pep518.py

fix: use `strtobool` to parse `CIBW_PYPA_BUILD`

fix: update `cibuildwheel.linux.troubleshoot` to know about `python -m pip wheel`

test: use `build_mode` in `test/test_dependency_versions.py` tests

* refactor: use build-backend instead

* Apply suggestions from code review

Co-authored-by: Joe Rickerby <joerick@mac.com>

* refactor: use literal types for build_frontend

* refactor(tests): build_mode -> build_frontend_env

* Update test/test_before_build.py

Co-authored-by: Joe Rickerby <joerick@mac.com>
2021-06-23 10:47:18 -04:00
Henry Schreiner 9e1369e07d docs: add Python 3.8 to the identifer list (#728) 2021-06-22 18:14:57 -04:00
Henry Schreiner 122af66cd1 ci: also update projects displaly in workflow (#724) 2021-06-22 15:11:37 -04:00
42 changed files with 489 additions and 238 deletions
+1 -1
View File
@@ -3,7 +3,7 @@ version: 2
jobs:
osx-python3.6:
macos:
xcode: 9.4.1
xcode: 12.5.1
environment:
PYTHON: python3
steps:
@@ -17,6 +17,8 @@ jobs:
run: nox --force-color -s update_constraints
- name: "Run update: python configs"
run: nox --force-color -s update_pins
- name: "Run update: docs user projects"
run: nox --force-color -s update_proj
# we use this step to grab a Github App auth token, so that PRs generated by this workflow
# run the GHA tests.
+3 -3
View File
@@ -14,7 +14,7 @@ repos:
- id: trailing-whitespace
- repo: https://github.com/asottile/pyupgrade
rev: v2.19.4
rev: v2.20.0
hooks:
- id: pyupgrade
name: PyUpgrade 3.6+
@@ -42,7 +42,7 @@ repos:
stages: [manual]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.902
rev: v0.910
hooks:
- id: mypy
exclude: ^(bin|cibuildwheel/resources|docs)/.*py$
@@ -57,7 +57,7 @@ repos:
files: ^(bin|docs)/.*py$
args: ["--python-version=3.7", "--scripts-are-modules"]
additional_dependencies:
- packaging
- packaging>=21.0
- rich
- types-jinja2
- types-pyyaml
+77 -68
View File
@@ -85,7 +85,7 @@ jobs:
- uses: actions/setup-python@v2
- name: Install cibuildwheel
run: python -m pip install cibuildwheel==2.0.0a3
run: python -m pip install cibuildwheel==2.0.0b1
- name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse
@@ -107,14 +107,15 @@ Options
|---|--------|-------------|
| **Build selection** | [`CIBW_PLATFORM`](https://cibuildwheel.readthedocs.io/en/stable/options/#platform) | Override the auto-detected target platform |
| | [`CIBW_BUILD`](https://cibuildwheel.readthedocs.io/en/stable/options/#build-skip) <br> [`CIBW_SKIP`](https://cibuildwheel.readthedocs.io/en/stable/options/#build-skip) | Choose the Python versions to build |
| | [`CIBW_ARCHS`](https://cibuildwheel.readthedocs.io/en/stable/options/#archs) | Change the architectures built on your machine by default |
| | [`CIBW_ARCHS`](https://cibuildwheel.readthedocs.io/en/stable/options/#archs) | Change the architectures built on your machine by default. |
| | [`CIBW_PROJECT_REQUIRES_PYTHON`](https://cibuildwheel.readthedocs.io/en/stable/options/#requires-python) | Manually set the Python compatibility of your project |
| | [`CIBW_PRERELEASE_PYTHONS`](https://cibuildwheel.readthedocs.io/en/stable/options/#prerelease-pythons) | Enable building with pre-release versions of Python |
| **Build customization** | [`CIBW_ENVIRONMENT`](https://cibuildwheel.readthedocs.io/en/stable/options/#environment) | Set environment variables needed during the build |
| **Build customization** | [`CIBW_BUILD_FRONTEND`](https://cibuildwheel.readthedocs.io/en/stable/options/#build-frontend) | Set the tool to use to build, either "pip" (default for now) or "build" |
| | [`CIBW_ENVIRONMENT`](https://cibuildwheel.readthedocs.io/en/stable/options/#environment) | Set environment variables needed during the build |
| | [`CIBW_BEFORE_ALL`](https://cibuildwheel.readthedocs.io/en/stable/options/#before-all) | Execute a shell command on the build system before any wheels are built. |
| | [`CIBW_BEFORE_BUILD`](https://cibuildwheel.readthedocs.io/en/stable/options/#before-build) | Execute a shell command preparing each wheel's build |
| | [`CIBW_REPAIR_WHEEL_COMMAND`](https://cibuildwheel.readthedocs.io/en/stable/options/#repair-wheel-command) | Execute a shell command to repair each (non-pure Python) built wheel |
| | [`CIBW_MANYLINUX_X86_64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) <br> [`CIBW_MANYLINUX_I686_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) <br> [`CIBW_MANYLINUX_PYPY_X86_64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) <br> [`CIBW_MANYLINUX_AARCH64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) <br> [`CIBW_MANYLINUX_PPC64LE_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) <br> [`CIBW_MANYLINUX_S390X_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) <br> [`CIBW_MANYLINUX_PYPY_AARCH64_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) <br> [`CIBW_MANYLINUX_PYPY_I686_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) | Specify alternative manylinux docker images |
| | [`CIBW_MANYLINUX_*_IMAGE`](https://cibuildwheel.readthedocs.io/en/stable/options/#manylinux-image) | Specify alternative manylinux Docker images |
| | [`CIBW_DEPENDENCY_VERSIONS`](https://cibuildwheel.readthedocs.io/en/stable/options/#dependency-versions) | Specify how cibuildwheel controls the versions of the tools it uses |
| **Testing** | [`CIBW_TEST_COMMAND`](https://cibuildwheel.readthedocs.io/en/stable/options/#test-command) | Execute a shell command to test each built wheel |
| | [`CIBW_BEFORE_TEST`](https://cibuildwheel.readthedocs.io/en/stable/options/#before-test) | Execute a shell command before testing each wheel |
@@ -185,11 +186,11 @@ Here are some repos that use cibuildwheel.
| [TgCrypto][] | ![travisci icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Includes a Windows Travis build. |
| [pybase64][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | Fast Base64 encoding/decoding in Python |
| [etebase-py][] | ![travisci icon][] | ![linux icon][] | Python bindings to a Rust library using `setuptools-rust`, and `sccache` for improved speed. |
| [fathon][] | ![travisci icon][] | ![apple icon][] ![linux icon][] | python package for DFA (Detrended Fluctuation Analysis) and related algorithms |
| [pyjet][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | The interface between FastJet and NumPy |
| [numpythia][] | ![github icon][] | ![apple icon][] ![linux icon][] | The interface between PYTHIA and NumPy |
| [fathon][] | ![travisci icon][] | ![apple icon][] ![linux icon][] | python package for DFA (Detrended Fluctuation Analysis) and related algorithms |
| [GSD][] | ![github icon][] | ![apple icon][] ![linux icon][] ![windows icon][] | Cython and NumPy project with 64-bit wheels. |
| [polaroid][] | ![github icon][] | ![apple icon][] ![linux icon][] ![windows icon][] | Full range of wheels for setuptools rust, with auto release and PyPI deploy. |
| [GSD][] | ![github icon][] | ![apple icon][] ![linux icon][] ![windows icon][] | Cython and NumPy project with 64-bit wheels. |
| [pybind11 scikit_build_example][] | ![github icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | An example combining scikit-build and pybind11 |
| [pyinstrument_cext][] | ![travisci icon][] ![appveyor icon][] | ![windows icon][] ![apple icon][] ![linux icon][] | A simple C extension, without external dependencies |
| [Imagecodecs (fork)][] | ![azurepipelines icon][] | ![apple icon][] ![linux icon][] | Over 20 external dependencies in compiled libraries, custom docker image, `libomp`, `openblas` and `install_name_tool` for macOS. |
@@ -244,11 +245,11 @@ Here are some repos that use cibuildwheel.
[TgCrypto]: https://github.com/pyrogram/tgcrypto
[pybase64]: https://github.com/mayeut/pybase64
[etebase-py]: https://github.com/etesync/etebase-py
[fathon]: https://github.com/stfbnc/fathon
[pyjet]: https://github.com/scikit-hep/pyjet
[numpythia]: https://github.com/scikit-hep/numpythia
[fathon]: https://github.com/stfbnc/fathon
[GSD]: https://github.com/glotzerlab/gsd
[polaroid]: https://github.com/daggy1234/polaroid
[GSD]: https://github.com/glotzerlab/gsd
[pybind11 scikit_build_example]: https://github.com/pybind/scikit_build_example
[pyinstrument_cext]: https://github.com/joerick/pyinstrument_cext
[Imagecodecs (fork)]: https://github.com/czaki/imagecodecs_build
@@ -266,64 +267,64 @@ Here are some repos that use cibuildwheel.
[apple icon]: docs/data/readme_icons/apple.svg
[linux icon]: docs/data/readme_icons/linux.svg
<!-- scikit-learn: 46184, last pushed 0 days ago -->
<!-- Matplotlib: 13797, last pushed 0 days ago -->
<!-- MyPy: 10905, last pushed 0 days ago -->
<!-- psutil: 7420, last pushed 14 days ago -->
<!-- scikit-image: 4387, last pushed 0 days ago -->
<!-- twisted-iocpsupport: 4280, last pushed 0 days ago -->
<!-- websockets: 3418, last pushed 0 days ago -->
<!-- pyzmq: 2821, last pushed 3 days ago -->
<!-- aiortc: 2441, last pushed 9 days ago -->
<!-- River: 1669, last pushed 0 days ago -->
<!-- coverage.py: 1630, last pushed 0 days ago -->
<!-- numexpr: 1597, last pushed 109 days ago -->
<!-- h5py: 1530, last pushed 0 days ago -->
<!-- Dependency Injector: 1396, last pushed 0 days ago -->
<!-- PyAV: 1279, last pushed 15 days ago -->
<!-- PyTables: 1028, last pushed 87 days ago -->
<!-- ruptures: 739, last pushed 4 days ago -->
<!-- aioquic: 680, last pushed 6 days ago -->
<!-- pikepdf: 655, last pushed 0 days ago -->
<!-- google neuroglancer: 646, last pushed 0 days ago -->
<!-- DeepForest: 621, last pushed 21 days ago -->
<!-- AutoPy: 542, last pushed 4 days ago -->
<!-- Parselmouth: 521, last pushed 27 days ago -->
<!-- python-rapidjson: 423, last pushed 0 days ago -->
<!-- Rtree: 408, last pushed 92 days ago -->
<!-- python-snappy: 403, last pushed 116 days ago -->
<!-- markupsafe: 401, last pushed 4 days ago -->
<!-- H3-py: 392, last pushed 8 days ago -->
<!-- pybind11 cmake_example: 278, last pushed 21 days ago -->
<!-- KDEpy: 276, last pushed 18 days ago -->
<!-- cyvcf2: 266, last pushed 53 days ago -->
<!-- pybind11 python_example: 255, last pushed 21 days ago -->
<!-- sourmash: 250, last pushed 0 days ago -->
<!-- dd-trace-py: 248, last pushed 0 days ago -->
<!-- iminuit: 181, last pushed 17 days ago -->
<!-- time-machine: 173, last pushed 6 days ago -->
<!-- CTranslate2: 168, last pushed 0 days ago -->
<!-- matrixprofile: 166, last pushed 17 days ago -->
<!-- jq.py: 159, last pushed 10 days ago -->
<!-- Tokenizer: 146, last pushed 13 days ago -->
<!-- PyGLM: 104, last pushed 41 days ago -->
<!-- bx-python: 103, last pushed 7 days ago -->
<!-- iDynTree: 82, last pushed 6 days ago -->
<!-- boost-histogram: 80, last pushed 3 days ago -->
<!-- TgCrypto: 66, last pushed 22 days ago -->
<!-- pybase64: 62, last pushed 5 days ago -->
<!-- etebase-py: 50, last pushed 168 days ago -->
<!-- pyjet: 28, last pushed 32 days ago -->
<!-- numpythia: 28, last pushed 21 days ago -->
<!-- fathon: 26, last pushed 19 days ago -->
<!-- GSD: 17, last pushed 17 days ago -->
<!-- polaroid: 16, last pushed 12 days ago -->
<!-- pybind11 scikit_build_example: 14, last pushed 20 days ago -->
<!-- pyinstrument_cext: 10, last pushed 129 days ago -->
<!-- Imagecodecs (fork): 9, last pushed 13 days ago -->
<!-- xmlstarlet: 7, last pushed 63 days ago -->
<!-- CorrectionLib: 6, last pushed 9 days ago -->
<!-- SiPM: 3, last pushed 18 days ago -->
<!-- scikit-learn: 46313, last pushed 0 days ago -->
<!-- Matplotlib: 13847, last pushed 0 days ago -->
<!-- MyPy: 10969, last pushed 0 days ago -->
<!-- psutil: 7454, last pushed 4 days ago -->
<!-- scikit-image: 4405, last pushed 0 days ago -->
<!-- twisted-iocpsupport: 4293, last pushed 8 days ago -->
<!-- websockets: 3442, last pushed 8 days ago -->
<!-- pyzmq: 2832, last pushed 16 days ago -->
<!-- aiortc: 2457, last pushed 23 days ago -->
<!-- River: 1722, last pushed 5 days ago -->
<!-- coverage.py: 1639, last pushed 11 days ago -->
<!-- numexpr: 1600, last pushed 123 days ago -->
<!-- h5py: 1533, last pushed 6 days ago -->
<!-- Dependency Injector: 1422, last pushed 10 days ago -->
<!-- PyAV: 1286, last pushed 28 days ago -->
<!-- PyTables: 1037, last pushed 101 days ago -->
<!-- ruptures: 749, last pushed 6 days ago -->
<!-- aioquic: 693, last pushed 20 days ago -->
<!-- pikepdf: 671, last pushed 4 days ago -->
<!-- google neuroglancer: 655, last pushed 3 days ago -->
<!-- DeepForest: 627, last pushed 34 days ago -->
<!-- AutoPy: 544, last pushed 17 days ago -->
<!-- Parselmouth: 524, last pushed 11 days ago -->
<!-- python-rapidjson: 423, last pushed 9 days ago -->
<!-- Rtree: 412, last pushed 106 days ago -->
<!-- python-snappy: 405, last pushed 129 days ago -->
<!-- markupsafe: 402, last pushed 2 days ago -->
<!-- H3-py: 397, last pushed 22 days ago -->
<!-- pybind11 cmake_example: 285, last pushed 9 days ago -->
<!-- KDEpy: 280, last pushed 31 days ago -->
<!-- cyvcf2: 265, last pushed 66 days ago -->
<!-- pybind11 python_example: 255, last pushed 12 days ago -->
<!-- sourmash: 252, last pushed 0 days ago -->
<!-- dd-trace-py: 251, last pushed 3 days ago -->
<!-- iminuit: 181, last pushed 0 days ago -->
<!-- time-machine: 178, last pushed 1 days ago -->
<!-- CTranslate2: 170, last pushed 3 days ago -->
<!-- matrixprofile: 167, last pushed 9 days ago -->
<!-- jq.py: 159, last pushed 12 days ago -->
<!-- Tokenizer: 147, last pushed 9 days ago -->
<!-- PyGLM: 105, last pushed 54 days ago -->
<!-- bx-python: 103, last pushed 20 days ago -->
<!-- iDynTree: 82, last pushed 19 days ago -->
<!-- boost-histogram: 81, last pushed 4 days ago -->
<!-- TgCrypto: 66, last pushed 35 days ago -->
<!-- pybase64: 65, last pushed 0 days ago -->
<!-- etebase-py: 50, last pushed 181 days ago -->
<!-- fathon: 32, last pushed 32 days ago -->
<!-- pyjet: 29, last pushed 46 days ago -->
<!-- numpythia: 28, last pushed 11 days ago -->
<!-- polaroid: 18, last pushed 6 days ago -->
<!-- GSD: 16, last pushed 3 days ago -->
<!-- pybind11 scikit_build_example: 14, last pushed 12 days ago -->
<!-- pyinstrument_cext: 10, last pushed 4 days ago -->
<!-- Imagecodecs (fork): 10, last pushed 9 days ago -->
<!-- xmlstarlet: 7, last pushed 76 days ago -->
<!-- CorrectionLib: 6, last pushed 12 days ago -->
<!-- SiPM: 4, last pushed 1 days ago -->
<!-- END bin/projects.py -->
@@ -351,11 +352,19 @@ Changelog
_This release is currently in prerelease. The below release notes will be condensed into a single entry when we do a final release._
_v2.0.0a3 (22 Jun 2021)_
_v2.0.0b1 (9 July 2021)_
- 🐛 Bug fixes & docs improvements
_v2.0.0a4 (25 June 2021)_
- 🌟 Added support for building wheels with [build](https://github.com/pypa/build), as well as pip. This feature is controlled with the [`CIBW_BUILD_FRONTEND`](https://cibuildwheel.readthedocs.io/en/latest/options/#build-frontend) option.
_v2.0.0a3 (22 June 2021)_
- 🌟 You can now configure cibuildwheel options inside your project's `pyproject.toml`! Environment variables still work of course. Check out the [documentation](https://cibuildwheel.readthedocs.io/en/latest/options/#setting-options) for more info.
_v2.0.0a2 (1 Jun 2021)_
_v2.0.0a2 (1 June 2021)_
- 🌟 Added the ability to test building wheels on CPython 3.10! Because CPython 3.10 is in beta, these wheels should not be distributed, because they might not be compatible with the final release, but it's available to build for testing purposes. Use the flag [`--prerelease-pythons` or `CIBW_PRERELEASE_PYTHONS`](https://cibuildwheel.readthedocs.io/en/latest/options/#prerelease-pythons) to test. (#675)
- ✨ Added the ability to build CPython 3.8 wheels for Apple Silicon. (#704)
+2
View File
@@ -19,10 +19,12 @@ runs:
# This needs powershell-core because github.action path is a Windows-style
# path on Windows. Powershell-core understands both types of paths.
# Redirecting stderr to stdout to fix interleaving issue in Actions.
- run: >
pipx run
--spec ${{ github.action_path }}
cibuildwheel
${{ inputs.package-dir }}
--output-dir ${{ inputs.output-dir }}
2>&1
shell: pwsh
+3 -3
View File
@@ -6,7 +6,7 @@ import copy
import difflib
import logging
from pathlib import Path
from typing import Any, Iterable, Union, cast
from typing import Any, Union
import click
import requests
@@ -86,7 +86,7 @@ class WindowsVersions:
# Specifier.filter selects all non pre-releases that match the spec,
# unless there are only pre-releases, then it selects pre-releases
# instead (like pip)
unsorted_versions = cast(Iterable[Version], spec.filter(self.version_dict))
unsorted_versions = spec.filter(self.version_dict)
versions = sorted(unsorted_versions, reverse=True)
log.debug(f"Windows {self.arch} {spec} has {', '.join(str(v) for v in versions)}")
@@ -201,7 +201,7 @@ class CPythonVersions:
) -> ConfigMacOS | None:
# see note above on Specifier.filter
unsorted_versions = cast(Iterable[Version], spec.filter(self.versions_dict))
unsorted_versions = spec.filter(self.versions_dict)
sorted_versions = sorted(unsorted_versions, reverse=True)
if version <= Version("3.8.9999"):
+1 -1
View File
@@ -1 +1 @@
__version__ = "2.0.0a3"
__version__ = "2.0.0b1"
+13
View File
@@ -20,6 +20,7 @@ from cibuildwheel.options import ConfigOptions
from cibuildwheel.projectfiles import get_requires_python_str
from cibuildwheel.typing import PLATFORMS, PlatformName, assert_never
from cibuildwheel.util import (
BuildFrontend,
BuildOptions,
BuildSelector,
DependencyConstraints,
@@ -184,6 +185,7 @@ def main() -> None:
archs_config_str = args.archs or options("archs", sep=" ")
build_frontend_str = options("build-frontend", env_plat=False)
environment_config = options("environment", table={"item": '{k}="{v}"', "sep": " "})
before_all = options("before-all", sep=" && ")
before_build = options("before-build", sep=" && ")
@@ -200,6 +202,16 @@ def main() -> None:
os.environ.get("CIBW_PRERELEASE_PYTHONS", "0")
)
build_frontend: BuildFrontend
if build_frontend_str == "build":
build_frontend = "build"
elif build_frontend_str == "pip":
build_frontend = "pip"
else:
msg = f"cibuildwheel: Unrecognised build frontend '{build_frontend}', only 'pip' and 'build' are supported"
print(msg, file=sys.stderr)
sys.exit(2)
package_files = {"setup.py", "setup.cfg", "pyproject.toml"}
if not any(package_dir.joinpath(name).exists() for name in package_files):
@@ -308,6 +320,7 @@ def main() -> None:
environment=environment,
dependency_constraints=dependency_constraints,
manylinux_images=manylinux_images or None,
build_frontend=build_frontend,
)
# Python is buffering by default when running on the CI platforms, giving problems interleaving subprocess call output with unflushed calls to 'print'
+39 -16
View File
@@ -7,7 +7,7 @@ from typing import List, NamedTuple, Set
from .architecture import Architecture
from .docker_container import DockerContainer
from .logger import log
from .typing import PathOrStr
from .typing import PathOrStr, assert_never
from .util import (
BuildOptions,
BuildSelector,
@@ -145,7 +145,7 @@ def build(options: BuildOptions) -> None:
env, executor=docker.environment_executor
)
# check config python and pip are still on PATH
# check config python is still on PATH
which_python = docker.call(
["which", "python"], env=env, capture_output=True
).strip()
@@ -180,18 +180,38 @@ def build(options: BuildOptions) -> None:
docker.call(["rm", "-rf", built_wheel_dir])
docker.call(["mkdir", "-p", built_wheel_dir])
docker.call(
[
"pip",
"wheel",
container_package_dir,
"--wheel-dir",
built_wheel_dir,
"--no-deps",
*get_build_verbosity_extra_flags(options.build_verbosity),
],
env=env,
)
verbosity_flags = get_build_verbosity_extra_flags(options.build_verbosity)
if options.build_frontend == "pip":
docker.call(
[
"python",
"-m",
"pip",
"wheel",
container_package_dir,
f"--wheel-dir={built_wheel_dir}",
"--no-deps",
*verbosity_flags,
],
env=env,
)
elif options.build_frontend == "build":
config_setting = " ".join(verbosity_flags)
docker.call(
[
"python",
"-m",
"build",
container_package_dir,
"--wheel",
f"--outdir={built_wheel_dir}",
f"--config-setting={config_setting}",
],
env=env,
)
else:
assert_never(options.build_frontend)
built_wheel = docker.glob(built_wheel_dir, "*.whl")[0]
@@ -291,8 +311,11 @@ def build(options: BuildOptions) -> None:
def troubleshoot(package_dir: Path, error: Exception) -> None:
if isinstance(error, subprocess.CalledProcessError) and error.cmd[0:2] == ["pip", "wheel"]:
# the 'pip wheel' step failed.
if isinstance(error, subprocess.CalledProcessError) and (
error.cmd[0:4] == ["python", "-m", "pip", "wheel"]
or error.cmd[0:3] == ["python", "-m", "build"]
):
# the wheel build step failed
print("Checking for common errors...")
so_files = list(package_dir.glob("**/*.so"))
+77 -28
View File
@@ -12,13 +12,15 @@ from typing import Any, Dict, List, NamedTuple, Optional, Sequence, Set, Tuple,
from .architecture import Architecture
from .environment import ParsedEnvironment
from .logger import log
from .typing import Literal, PathOrStr
from .typing import Literal, PathOrStr, assert_never
from .util import (
BuildFrontend,
BuildOptions,
BuildSelector,
NonPlatformWheelError,
download,
get_build_verbosity_extra_flags,
get_pip_version,
install_certifi_script,
prepare_command,
read_python_configs,
@@ -178,7 +180,9 @@ def setup_python(
python_configuration: PythonConfiguration,
dependency_constraint_flags: Sequence[PathOrStr],
environment: ParsedEnvironment,
build_frontend: BuildFrontend,
) -> Dict[str, str]:
implementation_id = python_configuration.identifier.split("-")[0]
log.step(f"Installing Python {implementation_id}...")
@@ -308,18 +312,33 @@ def setup_python(
env.setdefault("SDKROOT", arm64_compatible_sdks[0])
log.step("Installing build tools...")
call(
[
"pip",
"install",
"--upgrade",
"setuptools",
"wheel",
"delocate",
*dependency_constraint_flags,
],
env=env,
)
if build_frontend == "pip":
call(
[
"pip",
"install",
"--upgrade",
"setuptools",
"wheel",
"delocate",
*dependency_constraint_flags,
],
env=env,
)
elif build_frontend == "build":
call(
[
"pip",
"install",
"--upgrade",
"delocate",
"build[virtualenv]",
*dependency_constraint_flags,
],
env=env,
)
else:
assert_never(build_frontend)
return env
@@ -356,7 +375,12 @@ def build(options: BuildOptions) -> None:
options.dependency_constraints.get_for_python_version(config.version),
]
env = setup_python(config, dependency_constraint_flags, options.environment)
env = setup_python(
config,
dependency_constraint_flags,
options.environment,
options.build_frontend,
)
if options.before_build:
log.step("Running before_build...")
@@ -370,20 +394,45 @@ def build(options: BuildOptions) -> None:
shutil.rmtree(built_wheel_dir)
built_wheel_dir.mkdir(parents=True)
# Path.resolve() is needed. Without it pip wheel may try to fetch package from pypi.org
# see https://github.com/pypa/cibuildwheel/pull/369
call(
[
"pip",
"wheel",
options.package_dir.resolve(),
"--wheel-dir",
built_wheel_dir,
"--no-deps",
*get_build_verbosity_extra_flags(options.build_verbosity),
],
env=env,
)
verbosity_flags = get_build_verbosity_extra_flags(options.build_verbosity)
if options.build_frontend == "pip":
# Path.resolve() is needed. Without it pip wheel may try to fetch package from pypi.org
# see https://github.com/pypa/cibuildwheel/pull/369
call(
[
"python",
"-m",
"pip",
"wheel",
options.package_dir.resolve(),
f"--wheel-dir={built_wheel_dir}",
"--no-deps",
*verbosity_flags,
],
env=env,
)
elif options.build_frontend == "build":
config_setting = " ".join(verbosity_flags)
build_env = env.copy()
if options.dependency_constraints:
constr = options.dependency_constraints.get_for_python_version(config.version)
build_env["PIP_CONSTRAINT"] = constr.as_uri()
build_env["VIRTUALENV_PIP"] = get_pip_version(env)
call(
[
"python",
"-m",
"build",
options.package_dir,
"--wheel",
f"--outdir={built_wheel_dir}",
f"--config-setting={config_setting}",
],
env=build_env,
)
else:
assert_never(options.build_frontend)
built_wheel = next(built_wheel_dir.glob("*.whl"))
+5 -5
View File
@@ -37,9 +37,9 @@ python_configurations = [
{ identifier = "cp38-macosx_x86_64", version = "3.8", url = "https://www.python.org/ftp/python/3.8.10/python-3.8.10-macosx10.9.pkg" },
{ identifier = "cp38-macosx_arm64", version = "3.8", url = "https://www.python.org/ftp/python/3.8.10/python-3.8.10-macosx10.9.pkg" },
{ identifier = "cp38-macosx_universal2", version = "3.8", url = "https://www.python.org/ftp/python/3.8.10/python-3.8.10-macosx10.9.pkg" },
{ identifier = "cp39-macosx_x86_64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.5/python-3.9.5-macos11.pkg" },
{ identifier = "cp39-macosx_arm64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.5/python-3.9.5-macos11.pkg" },
{ identifier = "cp39-macosx_universal2", version = "3.9", url = "https://www.python.org/ftp/python/3.9.5/python-3.9.5-macos11.pkg" },
{ identifier = "cp39-macosx_x86_64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.6/python-3.9.6-macos11.pkg" },
{ identifier = "cp39-macosx_arm64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.6/python-3.9.6-macos11.pkg" },
{ identifier = "cp39-macosx_universal2", version = "3.9", url = "https://www.python.org/ftp/python/3.9.6/python-3.9.6-macos11.pkg" },
{ identifier = "cp310-macosx_x86_64", version = "3.10", url = "https://www.python.org/ftp/python/3.10.0/python-3.10.0b3-macos11.pkg" },
{ identifier = "cp310-macosx_arm64", version = "3.10", url = "https://www.python.org/ftp/python/3.10.0/python-3.10.0b3-macos11.pkg" },
{ identifier = "cp310-macosx_universal2", version = "3.10", url = "https://www.python.org/ftp/python/3.10.0/python-3.10.0b3-macos11.pkg" },
@@ -54,8 +54,8 @@ python_configurations = [
{ identifier = "cp37-win_amd64", version = "3.7.9", arch = "64" },
{ identifier = "cp38-win32", version = "3.8.10", arch = "32" },
{ identifier = "cp38-win_amd64", version = "3.8.10", arch = "64" },
{ identifier = "cp39-win32", version = "3.9.5", arch = "32" },
{ identifier = "cp39-win_amd64", version = "3.9.5", arch = "64" },
{ identifier = "cp39-win32", version = "3.9.6", arch = "32" },
{ identifier = "cp39-win_amd64", version = "3.9.6", arch = "64" },
{ identifier = "cp310-win32", version = "3.10.0-b3", arch = "32" },
{ identifier = "cp310-win_amd64", version = "3.10.0-b3", arch = "64" },
{ identifier = "pp37-win_amd64", version = "3.7", arch = "64", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.5-win64.zip" },
@@ -1,5 +1,5 @@
#
# This file is autogenerated by pip-compile
# This file is autogenerated by pip-compile with python 3.6
# To update, run:
#
# bin/update_dependencies.py
@@ -12,9 +12,9 @@ distlib==0.3.2
# via virtualenv
filelock==3.0.12
# via virtualenv
importlib-metadata==4.5.0
importlib-metadata==4.6.1
# via virtualenv
importlib-resources==5.1.4
importlib-resources==5.2.0
# via virtualenv
six==1.16.0
# via virtualenv
@@ -26,13 +26,13 @@ wheel==0.36.2
# via
# -r cibuildwheel/resources/constraints.in
# delocate
zipp==3.4.1
zipp==3.5.0
# via
# importlib-metadata
# importlib-resources
# The following packages are considered to be unsafe in a requirements file:
pip==21.1.2
pip==21.1.3
# via -r cibuildwheel/resources/constraints.in
setuptools==57.0.0
setuptools==57.1.0
# via -r cibuildwheel/resources/constraints.in
@@ -1,5 +1,5 @@
#
# This file is autogenerated by pip-compile
# This file is autogenerated by pip-compile with python 3.7
# To update, run:
#
# bin/update_dependencies.py
@@ -12,7 +12,7 @@ distlib==0.3.2
# via virtualenv
filelock==3.0.12
# via virtualenv
importlib-metadata==4.5.0
importlib-metadata==4.6.1
# via virtualenv
six==1.16.0
# via virtualenv
@@ -24,11 +24,11 @@ wheel==0.36.2
# via
# -r cibuildwheel/resources/constraints.in
# delocate
zipp==3.4.1
zipp==3.5.0
# via importlib-metadata
# The following packages are considered to be unsafe in a requirements file:
pip==21.1.2
pip==21.1.3
# via -r cibuildwheel/resources/constraints.in
setuptools==57.0.0
setuptools==57.1.0
# via -r cibuildwheel/resources/constraints.in
@@ -1,5 +1,5 @@
#
# This file is autogenerated by pip-compile
# This file is autogenerated by pip-compile with python 3.8
# To update, run:
#
# bin/update_dependencies.py
@@ -22,7 +22,7 @@ wheel==0.36.2
# delocate
# The following packages are considered to be unsafe in a requirements file:
pip==21.1.2
pip==21.1.3
# via -r cibuildwheel/resources/constraints.in
setuptools==57.0.0
setuptools==57.1.0
# via -r cibuildwheel/resources/constraints.in
@@ -1,5 +1,5 @@
#
# This file is autogenerated by pip-compile
# This file is autogenerated by pip-compile with python 3.9
# To update, run:
#
# bin/update_dependencies.py
@@ -22,7 +22,7 @@ wheel==0.36.2
# delocate
# The following packages are considered to be unsafe in a requirements file:
pip==21.1.2
pip==21.1.3
# via -r cibuildwheel/resources/constraints.in
setuptools==57.0.0
setuptools==57.1.0
# via -r cibuildwheel/resources/constraints.in
+3 -3
View File
@@ -1,5 +1,5 @@
#
# This file is autogenerated by pip-compile
# This file is autogenerated by pip-compile with python 3.9
# To update, run:
#
# bin/update_dependencies.py
@@ -22,7 +22,7 @@ wheel==0.36.2
# delocate
# The following packages are considered to be unsafe in a requirements file:
pip==21.1.2
pip==21.1.3
# via -r cibuildwheel/resources/constraints.in
setuptools==57.0.0
setuptools==57.1.0
# via -r cibuildwheel/resources/constraints.in
+1
View File
@@ -4,6 +4,7 @@ skip = ""
test-skip = ""
archs = ["auto"]
build-frontend = "pip"
dependency-versions = "pinned"
environment = {}
build-verbosity = ""
+22 -22
View File
@@ -1,38 +1,38 @@
[x86_64]
manylinux1 = quay.io/pypa/manylinux1_x86_64:2021-05-31-f12b4c4
manylinux2010 = quay.io/pypa/manylinux2010_x86_64:2021-06-07-00faba2
manylinux2014 = quay.io/pypa/manylinux2014_x86_64:2021-06-07-00faba2
manylinux_2_24 = quay.io/pypa/manylinux_2_24_x86_64:2021-06-07-00faba2
manylinux1 = quay.io/pypa/manylinux1_x86_64:2021-07-04-13bcf48
manylinux2010 = quay.io/pypa/manylinux2010_x86_64:2021-07-04-1e3ce39
manylinux2014 = quay.io/pypa/manylinux2014_x86_64:2021-07-04-1e3ce39
manylinux_2_24 = quay.io/pypa/manylinux_2_24_x86_64:2021-07-04-1e3ce39
[i686]
manylinux1 = quay.io/pypa/manylinux1_i686:2021-05-31-f12b4c4
manylinux2010 = quay.io/pypa/manylinux2010_i686:2021-06-07-00faba2
manylinux2014 = quay.io/pypa/manylinux2014_i686:2021-06-07-00faba2
manylinux_2_24 = quay.io/pypa/manylinux_2_24_i686:2021-06-07-00faba2
manylinux1 = quay.io/pypa/manylinux1_i686:2021-07-04-13bcf48
manylinux2010 = quay.io/pypa/manylinux2010_i686:2021-07-04-1e3ce39
manylinux2014 = quay.io/pypa/manylinux2014_i686:2021-07-04-1e3ce39
manylinux_2_24 = quay.io/pypa/manylinux_2_24_i686:2021-07-04-1e3ce39
[pypy_x86_64]
manylinux2010 = quay.io/pypa/manylinux2010_x86_64:2021-06-07-00faba2
manylinux2014 = quay.io/pypa/manylinux2014_x86_64:2021-06-07-00faba2
manylinux_2_24 = quay.io/pypa/manylinux_2_24_x86_64:2021-06-07-00faba2
manylinux2010 = quay.io/pypa/manylinux2010_x86_64:2021-07-04-1e3ce39
manylinux2014 = quay.io/pypa/manylinux2014_x86_64:2021-07-04-1e3ce39
manylinux_2_24 = quay.io/pypa/manylinux_2_24_x86_64:2021-07-04-1e3ce39
[pypy_i686]
manylinux2010 = quay.io/pypa/manylinux2010_i686:2021-06-07-00faba2
manylinux2014 = quay.io/pypa/manylinux2014_i686:2021-06-07-00faba2
manylinux_2_24 = quay.io/pypa/manylinux_2_24_i686:2021-06-07-00faba2
manylinux2010 = quay.io/pypa/manylinux2010_i686:2021-07-04-1e3ce39
manylinux2014 = quay.io/pypa/manylinux2014_i686:2021-07-04-1e3ce39
manylinux_2_24 = quay.io/pypa/manylinux_2_24_i686:2021-07-04-1e3ce39
[aarch64]
manylinux2014 = quay.io/pypa/manylinux2014_aarch64:2021-06-07-00faba2
manylinux_2_24 = quay.io/pypa/manylinux_2_24_aarch64:2021-06-07-00faba2
manylinux2014 = quay.io/pypa/manylinux2014_aarch64:2021-07-04-1e3ce39
manylinux_2_24 = quay.io/pypa/manylinux_2_24_aarch64:2021-07-04-1e3ce39
[ppc64le]
manylinux2014 = quay.io/pypa/manylinux2014_ppc64le:2021-06-07-00faba2
manylinux_2_24 = quay.io/pypa/manylinux_2_24_ppc64le:2021-06-07-00faba2
manylinux2014 = quay.io/pypa/manylinux2014_ppc64le:2021-07-04-1e3ce39
manylinux_2_24 = quay.io/pypa/manylinux_2_24_ppc64le:2021-07-04-1e3ce39
[s390x]
manylinux2014 = quay.io/pypa/manylinux2014_s390x:2021-06-07-00faba2
manylinux_2_24 = quay.io/pypa/manylinux_2_24_s390x:2021-06-07-00faba2
manylinux2014 = quay.io/pypa/manylinux2014_s390x:2021-07-04-1e3ce39
manylinux_2_24 = quay.io/pypa/manylinux_2_24_s390x:2021-07-04-1e3ce39
[pypy_aarch64]
manylinux2014 = quay.io/pypa/manylinux2014_aarch64:2021-06-07-00faba2
manylinux_2_24 = quay.io/pypa/manylinux_2_24_aarch64:2021-06-07-00faba2
manylinux2014 = quay.io/pypa/manylinux2014_aarch64:2021-07-04-1e3ce39
manylinux_2_24 = quay.io/pypa/manylinux_2_24_aarch64:2021-07-04-1e3ce39
+21 -1
View File
@@ -4,6 +4,8 @@ import itertools
import os
import re
import ssl
import subprocess
import sys
import textwrap
import time
import urllib.request
@@ -20,12 +22,14 @@ from packaging.version import Version
from .architecture import Architecture
from .environment import ParsedEnvironment
from .typing import PathOrStr, PlatformName
from .typing import Literal, PathOrStr, PlatformName
resources_dir = Path(__file__).parent / "resources"
install_certifi_script = resources_dir / "install_certifi.py"
BuildFrontend = Literal["pip", "build"]
def prepare_command(command: str, **kwargs: PathOrStr) -> str:
"""
@@ -218,6 +222,7 @@ class BuildOptions(NamedTuple):
test_requires: List[str]
test_extras: str
build_verbosity: int
build_frontend: BuildFrontend
class NonPlatformWheelError(Exception):
@@ -300,3 +305,18 @@ def print_new_wheels(msg: str, output_dir: Path) -> Iterator[None]:
s = time.time() - start_time
m = s / 60
print(msg.format(n=n, s=s, m=m), *sorted(f" {f.name}" for f in new_contents), sep="\n")
def get_pip_version(env: Dict[str, str]) -> str:
# we use shell=True here for windows, even though we don't need a shell due to a bug
# https://bugs.python.org/issue8557
shell = sys.platform.startswith("win")
versions_output_text = subprocess.check_output(
["python", "-m", "pip", "freeze", "--all"], universal_newlines=True, shell=shell, env=env
)
(pip_version,) = (
version[5:]
for version in versions_output_text.strip().splitlines()
if version.startswith("pip==")
)
return pip_version
+97 -28
View File
@@ -10,20 +10,24 @@ from zipfile import ZipFile
from .architecture import Architecture
from .environment import ParsedEnvironment
from .logger import log
from .typing import PathOrStr
from .typing import PathOrStr, assert_never
from .util import (
BuildFrontend,
BuildOptions,
BuildSelector,
NonPlatformWheelError,
download,
get_build_verbosity_extra_flags,
get_pip_version,
prepare_command,
read_python_configs,
)
CIBW_INSTALL_PATH = Path("C:\\cibw")
def call(
args: Sequence[PathOrStr], env: Optional[Dict[str, str]] = None, cwd: Optional[str] = None
args: Sequence[PathOrStr], env: Optional[Dict[str, str]] = None, cwd: Optional[PathOrStr] = None
) -> None:
print("+ " + " ".join(str(a) for a in args))
# we use shell=True here, even though we don't need a shell due to a bug
@@ -31,7 +35,9 @@ def call(
subprocess.run([str(a) for a in args], env=env, cwd=cwd, shell=True, check=True)
def shell(command: str, env: Optional[Dict[str, str]] = None, cwd: Optional[str] = None) -> None:
def shell(
command: str, env: Optional[Dict[str, str]] = None, cwd: Optional[PathOrStr] = None
) -> None:
print(f"+ {command}")
subprocess.run(command, env=env, cwd=cwd, shell=True, check=True)
@@ -47,7 +53,7 @@ def get_nuget_args(version: str, arch: str) -> List[str]:
"-FallbackSource",
"https://api.nuget.org/v3/index.json",
"-OutputDirectory",
"C:\\cibw\\python",
str(CIBW_INSTALL_PATH / "python"),
]
@@ -100,9 +106,9 @@ def install_pypy(version: str, arch: str, url: str) -> Path:
zip_filename = url.rsplit("/", 1)[-1]
extension = ".zip"
assert zip_filename.endswith(extension)
installation_path = Path("C:\\cibw") / zip_filename[: -len(extension)]
installation_path = CIBW_INSTALL_PATH / zip_filename[: -len(extension)]
if not installation_path.exists():
pypy_zip = Path("C:\\cibw") / zip_filename
pypy_zip = CIBW_INSTALL_PATH / zip_filename
download(url, pypy_zip)
# Extract to the parent directory because the zip file still contains a directory
extract_zip(pypy_zip, installation_path.parent)
@@ -114,8 +120,10 @@ def setup_python(
python_configuration: PythonConfiguration,
dependency_constraint_flags: Sequence[PathOrStr],
environment: ParsedEnvironment,
build_frontend: BuildFrontend,
) -> Dict[str, str]:
nuget = Path("C:\\cibw\\nuget.exe")
nuget = CIBW_INSTALL_PATH / "nuget.exe"
if not nuget.exists():
log.step("Downloading nuget...")
download("https://dist.nuget.org/win-x86-commandline/latest/nuget.exe", nuget)
@@ -180,7 +188,7 @@ def setup_python(
requires_reinstall = not (installation_path / "Scripts" / "pip.exe").exists()
if requires_reinstall:
# maybe pip isn't installed at all. ensurepip resolves that.
call(["python", "-m", "ensurepip"], env=env, cwd="C:\\cibw")
call(["python", "-m", "ensurepip"], env=env, cwd=CIBW_INSTALL_PATH)
# upgrade pip to the version matching our constraints
# if necessary, reinstall it to ensure that it's available on PATH as 'pip.exe'
@@ -195,7 +203,7 @@ def setup_python(
*dependency_constraint_flags,
],
env=env,
cwd="C:\\cibw",
cwd=CIBW_INSTALL_PATH,
)
assert (installation_path / "Scripts" / "pip.exe").exists()
@@ -214,10 +222,26 @@ def setup_python(
sys.exit(1)
call(["pip", "--version"], env=env)
call(
["pip", "install", "--upgrade", "setuptools", "wheel", *dependency_constraint_flags],
env=env,
)
if build_frontend == "pip":
call(
[
"pip",
"install",
"--upgrade",
"setuptools",
"wheel",
*dependency_constraint_flags,
],
env=env,
)
elif build_frontend == "build":
call(
["pip", "install", "--upgrade", "build[virtualenv]", *dependency_constraint_flags],
env=env,
)
else:
assert_never(build_frontend)
return env
@@ -251,7 +275,12 @@ def build(options: BuildOptions) -> None:
]
# install Python
env = setup_python(config, dependency_constraint_flags, options.environment)
env = setup_python(
config,
dependency_constraint_flags,
options.environment,
options.build_frontend,
)
# run the before_build command
if options.before_build:
@@ -265,20 +294,60 @@ def build(options: BuildOptions) -> None:
if built_wheel_dir.exists():
shutil.rmtree(built_wheel_dir)
built_wheel_dir.mkdir(parents=True)
# Path.resolve() is needed. Without it pip wheel may try to fetch package from pypi.org
# see https://github.com/pypa/cibuildwheel/pull/369
call(
[
"pip",
"wheel",
options.package_dir.resolve(),
"-w",
built_wheel_dir,
"--no-deps",
*get_build_verbosity_extra_flags(options.build_verbosity),
],
env=env,
)
verbosity_flags = get_build_verbosity_extra_flags(options.build_verbosity)
if options.build_frontend == "pip":
# Path.resolve() is needed. Without it pip wheel may try to fetch package from pypi.org
# see https://github.com/pypa/cibuildwheel/pull/369
call(
[
"python",
"-m",
"pip",
"wheel",
options.package_dir.resolve(),
f"--wheel-dir={built_wheel_dir}",
"--no-deps",
*get_build_verbosity_extra_flags(options.build_verbosity),
],
env=env,
)
elif options.build_frontend == "build":
config_setting = " ".join(verbosity_flags)
build_env = env.copy()
if options.dependency_constraints:
constraints_path = options.dependency_constraints.get_for_python_version(
config.version
)
# Bug in pip <= 21.1.3 - we can't have a space in the
# constraints file, and pip doesn't support drive letters
# in uhi. After probably pip 21.2, we can use uri. For
# now, use a temporary file.
if " " in str(constraints_path):
tmp_file = tempfile.NamedTemporaryFile(
"w", suffix="constraints.txt", delete=False, dir=CIBW_INSTALL_PATH
)
with tmp_file as new_constraints_file, open(constraints_path) as f:
new_constraints_file.write(f.read())
constraints_path = Path(new_constraints_file.name)
build_env["PIP_CONSTRAINT"] = str(constraints_path)
build_env["VIRTUALENV_PIP"] = get_pip_version(env)
call(
[
"python",
"-m",
"build",
options.package_dir,
"--wheel",
f"--outdir={built_wheel_dir}",
f"--config-setting={config_setting}",
],
env=build_env,
)
else:
assert_never(options.build_frontend)
built_wheel = next(built_wheel_dir.glob("*.whl"))
+4 -4
View File
@@ -210,12 +210,12 @@ There are two suggested methods for keeping cibuildwheel up to date that instead
If you use GitHub Actions for builds, you can use cibuildwheel as an action:
```yaml
uses: pypa/cibuildwheel@v2.0.0a3
uses: pypa/cibuildwheel@v2.0.0b1
```
This is a composite step that just runs cibuildwheel using pipx. You can set command-line options as `with:` parameters, and use `env:` as normal.
Then, your `dependabot.yml` file could look like this:
Then, your `.github/dependabot.yml` file could look like this:
```yaml
version: 2
@@ -236,10 +236,10 @@ The second option, and the only one that supports other CI systems, is using a `
```bash
# requirements-cibw.txt
cibuildwheel==2.0.0a3
cibuildwheel==2.0.0b1
```
Then your install step would have `python -m pip install -r requirements-cibw.txt` in it. Your `dependabot.yml` file could look like this:
Then your install step would have `python -m pip install -r requirements-cibw.txt` in it. Your `.github/dependabot.yml` file could look like this:
```yaml
version: 2
+42 -3
View File
@@ -155,7 +155,7 @@ When setting the options, you can use shell-style globbing syntax, as per [fnmat
|--------------|---------------------------------------------------------------------|-------------------------------|-----------------------------------------------|----------------------------------------------------------------------------|
| Python 3.6 | cp36-macosx_x86_64 | cp36-win_amd64<br/>cp36-win32 | cp36-manylinux_x86_64<br/>cp36-manylinux_i686 | cp36-manylinux_aarch64<br/>cp36-manylinux_ppc64le<br/>cp36-manylinux_s390x |
| Python 3.7 | cp37-macosx_x86_64 | cp37-win_amd64<br/>cp37-win32 | cp37-manylinux_x86_64<br/>cp37-manylinux_i686 | cp37-manylinux_aarch64<br/>cp37-manylinux_ppc64le<br/>cp37-manylinux_s390x |
| Python 3.8 | cp38-macosx_x86_64 | cp38-win_amd64<br/>cp38-win32 | cp38-manylinux_x86_64<br/>cp38-manylinux_i686 | cp38-manylinux_aarch64<br/>cp38-manylinux_ppc64le<br/>cp38-manylinux_s390x |
| Python 3.8 | cp38-macosx_x86_64<br/>cp38-macosx_universal2<br/>cp38-macosx_arm64 | cp38-win_amd64<br/>cp38-win32 | cp38-manylinux_x86_64<br/>cp38-manylinux_i686 | cp38-manylinux_aarch64<br/>cp38-manylinux_ppc64le<br/>cp38-manylinux_s390x |
| Python 3.9 | cp39-macosx_x86_64<br/>cp39-macosx_universal2<br/>cp39-macosx_arm64 | cp39-win_amd64<br/>cp39-win32 | cp39-manylinux_x86_64<br/>cp39-manylinux_i686 | cp39-manylinux_aarch64<br/>cp39-manylinux_ppc64le<br/>cp39-manylinux_s390x |
| PyPy3.7 v7.3 | pp37-macosx_x86_64 | pp37-win_amd64 | pp37-manylinux_x86_64 | |
@@ -330,11 +330,11 @@ This option can also be set using the [command-line option](#command-line) `--ar
# Note that the `arm64` wheel and the `arm64` part of the `universal2`
# wheel cannot be tested in this configuration.
[tool.cibuildwheel.macos]
archs: ["x86_64", "universal2", "arm64"]
archs = ["x86_64", "universal2", "arm64"]
# On an Linux Intel runner with qemu installed, build Intel and ARM wheels
[tool.cibuildwheel.linux]
archs: ["auto", "aarch64"]
archs = ["auto", "aarch64"]
```
It is generally recommmended to use the environment variable or
@@ -435,6 +435,45 @@ This option can also be set using the [command-line option](#command-line) `--pr
## Build customization
### `CIBW_BUILD_FRONTEND` {: #build-frontend}
> Set the tool to use to build, either "pip" (default for now) or "build"
Choose which build backend to use. Can either be "pip", which will run
`python -m pip wheel`, or "build", which will run `python -m build --wheel`.
!!! tip
Until v2.0.0, [pip] was the only way to build wheels, and is still the
default. However, we expect that at some point in the future, cibuildwheel
will change the default to [build], in line with the PyPA's recommendation.
If you want to try `build` before this, you can use this option.
[pip]: https://pip.pypa.io/en/stable/cli/pip_wheel/
[build]: https://github.com/pypa/build/
#### Examples
!!! tab examples "Environment variables"
```yaml
# Switch to using build
CIBW_BUILD_FRONTEND: "build"
# Ensure pip is used even if the default changes in the future
CIBW_BUILD_FRONTEND: "pip"
```
!!! tab examples "pyproject.toml"
```toml
[tool.cibuildwheel]
# Switch to using build
build-frontend = "build"
# Ensure pip is used even if the default changes in the future
build-frontend = "pip"
```
### `CIBW_ENVIRONMENT` {: #environment}
> Set environment variables needed during the build
+2 -2
View File
@@ -46,7 +46,7 @@ To build Linux, Mac, and Windows wheels using GitHub Actions, create a `.github/
- uses: actions/checkout@v2
- name: Build wheels
run: pipx run cibuildwheel==2.0.0a3
run: pipx run cibuildwheel==2.0.0b1
- uses: actions/upload-artifact@v2
with:
@@ -81,7 +81,7 @@ To build Linux, Mac, and Windows wheels using GitHub Actions, create a `.github/
- uses: actions/setup-python@v2
- name: Install cibuildwheel
run: python -m pip install cibuildwheel==2.0.0a3
run: python -m pip install cibuildwheel==2.0.0b1
- name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse
+1 -1
View File
@@ -12,7 +12,7 @@ stack: python 3.7
init:
- cmd: set PATH=C:\Python37;C:\Python37\Scripts;%PATH%
install: python -m pip install cibuildwheel==2.0.0a3
install: python -m pip install cibuildwheel==2.0.0b1
build_script: python -m cibuildwheel --output-dir wheelhouse
+3 -3
View File
@@ -6,7 +6,7 @@ jobs:
- bash: |
set -o errexit
python3 -m pip install --upgrade pip
pip3 install cibuildwheel==2.0.0a3
pip3 install cibuildwheel==2.0.0b1
displayName: Install dependencies
- bash: cibuildwheel --output-dir wheelhouse .
displayName: Build wheels
@@ -20,7 +20,7 @@ jobs:
- bash: |
set -o errexit
python3 -m pip install --upgrade pip
python3 -m pip install cibuildwheel==2.0.0a3
python3 -m pip install cibuildwheel==2.0.0b1
displayName: Install dependencies
- bash: cibuildwheel --output-dir wheelhouse .
displayName: Build wheels
@@ -34,7 +34,7 @@ jobs:
- bash: |
set -o errexit
python -m pip install --upgrade pip
pip install cibuildwheel==2.0.0a3
pip install cibuildwheel==2.0.0b1
displayName: Install dependencies
- bash: cibuildwheel --output-dir wheelhouse .
displayName: Build wheels
+3 -3
View File
@@ -11,7 +11,7 @@ jobs:
- run:
name: Build the Linux wheels.
command: |
pip3 install --user cibuildwheel==2.0.0a3
pip3 install --user cibuildwheel==2.0.0b1
cibuildwheel --output-dir wheelhouse
- store_artifacts:
path: wheelhouse/
@@ -19,13 +19,13 @@ jobs:
osx-wheels:
working_directory: ~/osx-wheels
macos:
xcode: 10.0.0
xcode: 12.5.1
steps:
- checkout
- run:
name: Build the OS X wheels.
command: |
pip3 install cibuildwheel==2.0.0a3
pip3 install cibuildwheel==2.0.0b1
cibuildwheel --output-dir wheelhouse
- store_artifacts:
path: wheelhouse/
+1 -1
View File
@@ -10,7 +10,7 @@ jobs:
- uses: actions/checkout@v2
- name: Build wheels
uses: pypa/cibuildwheel@v2.0.0a3
uses: pypa/cibuildwheel@v2.0.0b1
env:
CIBW_ARCHS_MACOS: x86_64 universal2
+1 -1
View File
@@ -27,7 +27,7 @@ jobs:
python-version: '3.8'
- name: Build wheels
uses: pypa/cibuildwheel@v2.0.0a3
uses: pypa/cibuildwheel@v2.0.0b1
- uses: actions/upload-artifact@v2
with:
+1 -1
View File
@@ -14,7 +14,7 @@ jobs:
- uses: actions/checkout@v2
- name: Build wheels
uses: pypa/cibuildwheel@v2.0.0a3
uses: pypa/cibuildwheel@v2.0.0b1
# to supply options, put them in 'env', like:
# env:
# CIBW_SOME_OPTION: value
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
platforms: all
- name: Build wheels
uses: pypa/cibuildwheel@v2.0.0a3
uses: pypa/cibuildwheel@v2.0.0b1
env:
# configure cibuildwheel to build native archs ('auto'), and some
# emulated ones
+1 -1
View File
@@ -12,7 +12,7 @@ linux:
DOCKER_TLS_CERTDIR: ""
script:
- curl -sSL https://get.docker.com/ | sh
- python -m pip install cibuildwheel==2.0.0a3
- python -m pip install cibuildwheel==2.0.0b1
- cibuildwheel --output-dir wheelhouse
artifacts:
paths:
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
- ln -s /c/Python38/python.exe /c/Python38/python3.exe
install:
- python3 -m pip install cibuildwheel==2.0.0a3
- python3 -m pip install cibuildwheel==2.0.0b1
script:
# build the wheels, put them into './wheelhouse'
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
- ln -s /c/Python38/python.exe /c/Python38/python3.exe
install:
- python3 -m pip install cibuildwheel==2.0.0a3
- python3 -m pip install cibuildwheel==2.0.0b1
script:
# build the wheels, put them into './wheelhouse'
+2 -2
View File
@@ -57,7 +57,7 @@ jobs:
- stage: deploy
name: Build and deploy Linux wheels
services: docker
install: python3 -m pip install cibuildwheel==2.0.0a3
install: python3 -m pip install cibuildwheel==2.0.0b1
script: python3 -m cibuildwheel --output-dir wheelhouse
after_success: |
python3 -m pip install twine
@@ -67,7 +67,7 @@ jobs:
name: Build and deploy Windows wheels
os: windows
language: shell
install: python3 -m pip install cibuildwheel==2.0.0a3
install: python3 -m pip install cibuildwheel==2.0.0b1
script: python3 -m cibuildwheel --output-dir wheelhouse
after_success: |
python3 -m pip install twine
+1 -1
View File
@@ -1,6 +1,6 @@
[metadata]
name = cibuildwheel
version = 2.0.0a3
version = 2.0.0b1
description = Build Python wheels on CI with minimal configuration.
long_description = file: README.md
long_description_content_type = text/markdown
+1 -1
View File
@@ -20,7 +20,7 @@ extras = {
"pyyaml",
"requests",
"rich>=9.6",
"packaging>=20.8",
"packaging>=21.0",
],
"mypy": [
"mypy>=0.901",
+11 -2
View File
@@ -1,7 +1,9 @@
from typing import Dict
import pytest
def pytest_addoption(parser):
def pytest_addoption(parser) -> None:
parser.addoption(
"--run-emulation", action="store_true", default=False, help="run emulation tests"
)
@@ -11,7 +13,7 @@ def pytest_configure(config):
config.addinivalue_line("markers", "emulation: mark test requiring qemu binfmt_misc to run")
def pytest_collection_modifyitems(config, items):
def pytest_collection_modifyitems(config, items) -> None:
if config.getoption("--run-emulation"):
# --run-emulation given in cli: do not skip emulation tests
return
@@ -19,3 +21,10 @@ def pytest_collection_modifyitems(config, items):
for item in items:
if "emulation" in item.keywords:
item.add_marker(skip_emulation)
@pytest.fixture(
params=[{"CIBW_BUILD_FRONTEND": "pip"}, {"CIBW_BUILD_FRONTEND": "build"}], ids=["pip", "build"]
)
def build_frontend_env(request) -> Dict[str, str]:
return request.param # type: ignore
+2 -2
View File
@@ -18,12 +18,12 @@ basic_project = test_projects.new_c_project(
)
def test(tmp_path):
def test(tmp_path, build_frontend_env):
project_dir = tmp_path / "project"
basic_project.generate(project_dir)
# build the wheels
actual_wheels = utils.cibuildwheel_run(project_dir)
actual_wheels = utils.cibuildwheel_run(project_dir, add_env=build_frontend_env)
# check that the expected wheels are produced
expected_wheels = utils.expected_wheels("spam", "0.1.0")
+6 -1
View File
@@ -25,8 +25,13 @@ project_with_before_build_asserts = test_projects.new_c_project(
stored_executable = f.read()
print('stored_executable', stored_executable)
print('sys.executable', sys.executable)
# windows/mac are case insensitive
assert os.path.realpath(stored_executable).lower() == os.path.realpath(sys.executable).lower()
stored_path = os.path.realpath(stored_executable).lower()
current_path = os.path.realpath(sys.executable).lower()
# TODO: This is not valid in an virtual environment
assert stored_path == current_path, '{0} != {1}'.format(stored_path, current_path)
"""
)
)
+7 -4
View File
@@ -48,7 +48,7 @@ def get_versions_from_constraint_file(constraint_file):
@pytest.mark.parametrize("python_version", ["3.6", "3.8", "3.9"])
def test_pinned_versions(tmp_path, python_version):
def test_pinned_versions(tmp_path, python_version, build_frontend_env):
if utils.platform == "linux":
pytest.skip("linux doesn't pin individual tool versions, it pins manylinux images instead")
@@ -85,6 +85,7 @@ def test_pinned_versions(tmp_path, python_version):
add_env={
"CIBW_BUILD": build_pattern,
"CIBW_ENVIRONMENT": cibw_environment_option,
**build_frontend_env,
},
)
@@ -107,7 +108,7 @@ def test_pinned_versions(tmp_path, python_version):
assert set(actual_wheels) == set(expected_wheels)
def test_dependency_constraints_file(tmp_path):
def test_dependency_constraints_file(tmp_path, build_frontend_env):
if utils.platform == "linux":
pytest.skip("linux doesn't pin individual tool versions, it pins manylinux images instead")
@@ -118,10 +119,10 @@ def test_dependency_constraints_file(tmp_path):
"pip": "20.0.2",
"setuptools": "53.0.0",
"wheel": "0.34.2",
"virtualenv": "20.0.10",
"virtualenv": "20.0.35",
}
constraints_file = tmp_path / "constraints.txt"
constraints_file = tmp_path / "constraints file.txt"
constraints_file.write_text(
textwrap.dedent(
"""
@@ -129,6 +130,7 @@ def test_dependency_constraints_file(tmp_path):
setuptools=={setuptools}
wheel=={wheel}
virtualenv=={virtualenv}
importlib-metadata<3,>=0.12; python_version < "3.8"
""".format(
**tool_versions
)
@@ -149,6 +151,7 @@ def test_dependency_constraints_file(tmp_path):
add_env={
"CIBW_ENVIRONMENT": cibw_environment_option,
"CIBW_DEPENDENCY_VERSIONS": str(constraints_file),
**build_frontend_env,
},
)
+11 -4
View File
@@ -1,4 +1,3 @@
import os
import textwrap
from . import test_projects, utils
@@ -33,13 +32,13 @@ build-backend = "setuptools.build_meta"
"""
def test_pep518(tmp_path):
def test_pep518(tmp_path, build_frontend_env):
project_dir = tmp_path / "project"
basic_project.generate(project_dir)
# build the wheels
actual_wheels = utils.cibuildwheel_run(project_dir)
actual_wheels = utils.cibuildwheel_run(project_dir, add_env=build_frontend_env)
# check that the expected wheels are produced
expected_wheels = utils.expected_wheels("spam", "0.1.0")
@@ -50,4 +49,12 @@ def test_pep518(tmp_path):
assert not (project_dir / "42").exists()
assert not (project_dir / "4.1.2").exists()
assert len(os.listdir(project_dir)) == len(basic_project.files)
# pypa/build creates a "build" folder & a "*.egg-info" folder for the wheel being built,
# this should be harmless so remove them
contents = [
item
for item in project_dir.iterdir()
if item.name != "build" and not item.name.endswith(".egg-info")
]
assert len(contents) == len(basic_project.files)
+2 -2
View File
@@ -16,7 +16,7 @@ raise Exception('this build will fail')
"""
def test_failed_project_with_so_files(tmp_path, capfd):
def test_failed_project_with_so_files(tmp_path, capfd, build_frontend_env):
if utils.platform != "linux":
pytest.skip("this test is only relevant to the linux build")
@@ -24,7 +24,7 @@ def test_failed_project_with_so_files(tmp_path, capfd):
so_file_project.generate(project_dir)
with pytest.raises(subprocess.CalledProcessError):
utils.cibuildwheel_run(project_dir)
utils.cibuildwheel_run(project_dir, add_env=build_frontend_env)
captured = capfd.readouterr()
print("out", captured.out)