diff --git a/README.md b/README.md index 151820ea..2b2587cc 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ While cibuildwheel itself requires a recent Python version to run (we support th ¹ PyPy & GraalPy are only supported for manylinux wheels.
² Windows arm64 support is experimental.
-³ Free-threaded mode requires opt-in using [`CIBW_ENABLE`](https://cibuildwheel.pypa.io/en/stable/options/#enable).
+³ Free-threaded mode requires opt-in using [`enable`](https://cibuildwheel.pypa.io/en/stable/options/#enable).
⁴ Experimental, not yet supported on PyPI, but can be used directly in web deployment. Use `--platform pyodide` to build.
⁵ manylinux armv7l support is experimental. As there are no RHEL based image for this architecture, it's using an Ubuntu based image instead.
diff --git a/docs/contributing.md b/docs/contributing.md index 40ebaa8b..31c879a7 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -16,7 +16,7 @@ Everyone contributing to the cibuildwheel project is expected to follow the [PSF - `cibuildwheel` should wrap the complexity of wheel building. - The user interface to `cibuildwheel` is the build script (e.g. `.travis.yml`). Feature additions should not increase the complexity of this script. -- Options should be environment variables (these lend themselves better to YML config files). They should be prefixed with `CIBW_`. +- Options should be environment variables (these lend themselves better to YAML config files). They should be prefixed with `CIBW_`. - Options should be generalised to all platforms. If platform-specific options are required, they should be namespaced e.g. `CIBW_TEST_COMMAND_MACOS` Other notes: diff --git a/docs/faq.md b/docs/faq.md index 4db88301..5c92c02a 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -33,7 +33,7 @@ The CPython Limited API is a subset of the Python C Extension API that's declare To create a package that builds ABI3 wheels, you'll need to configure your build backend to compile libraries correctly create wheels with the right tags. [Check this repo](https://github.com/joerick/python-abi3-package-sample) for an example of how to do this with setuptools. -You could also consider running [abi3audit](https://github.com/trailofbits/abi3audit) against the produced wheels in order to check for abi3 violations or inconsistencies. You can run it alongside the default in your [CIBW_REPAIR_WHEEL_COMMAND](options.md#repair-wheel-command). +You could also consider running [abi3audit](https://github.com/trailofbits/abi3audit) against the produced wheels in order to check for abi3 violations or inconsistencies. You can run it alongside the default in your [repair-wheel-command](options.md#repair-wheel-command). ### Packages with optional C extensions {: #optional-extensions} @@ -108,7 +108,7 @@ package 'sdist' will also benefit. #### Missing build dependencies {: #cibw-options-alternatives-deps} -If your build needs Python dependencies, rather than using `CIBW_BEFORE_BUILD`, it's best to add these to the +If your build needs Python dependencies, rather than using `before-build`, it's best to add these to the [`build-system.requires`](https://www.python.org/dev/peps/pep-0518/#build-system-table) section of your pyproject.toml. For example, if your project requires Cython to build, your pyproject.toml might include a section like this: @@ -129,7 +129,7 @@ You might need to run some other commands before building, like running a script that performs codegen or downloading some data that's not stored in your source tree. -Rather than using `CIBW_BEFORE_ALL` or `CIBW_BEFORE_BUILD`, you could incorporate +Rather than using `before-all` or `before-build`, you could incorporate these steps into your package's build process. For example, if you're using setuptools, you can add steps to your package's `setup.py` using a structure like this: @@ -177,16 +177,16 @@ Sometimes a build will fail due to a missing dependency. **If the build is missing a Python package**, you should [add it to pyproject.toml](#cibw-options-alternatives-deps). -**If you need a build tool** (e.g. cmake, automake, ninja), you can install it through a package manager like apt/yum, brew or choco, using the [`CIBW_BEFORE_ALL`](options.md#before-all) option. +**If you need a build tool** (e.g. cmake, automake, ninja), you can install it through a package manager like apt/yum, brew or choco, using the [`before-all`](options.md#before-all) option. -**If your build is linking into a native library dependency**, you can build/install that in [`CIBW_BEFORE_ALL`](options.md#before-all). However, on Linux, Mac (and Windows if you're using [delvewheel]), the library that you install will be bundled into the wheel in the [repair step]. So take care to ensure that +**If your build is linking into a native library dependency**, you can build/install that in [`before-all`](options.md#before-all). However, on Linux, Mac (and Windows if you're using [delvewheel]), the library that you install will be bundled into the wheel in the [repair step]. So take care to ensure that - the bundled library doesn't accidentally increase the minimum system requirements (such as the minimum macOS version) - the bundled library matches the architecture of the wheel you're building when cross-compiling This is particularly an issue on macOS, where de facto package manager Homebrew will install libraries that are compiled for the specific version of macOS that the build machine is running, rendering the wheels useless for any previous version. And brew will not install the right arch for cross compilation of Apple Silicon wheels. -For these reasons, it's strongly recommended to not use brew for native library dependencies. Instead, we recommend compiling the library yourself. If you compile in the [`CIBW_BEFORE_ALL`](options.md#before-all) step, cibuildwheel will have already set the appropriate `MACOSX_DEPLOYMENT_TARGET` env var, so the library will target the correct version of macOS. +For these reasons, it's strongly recommended to not use brew for native library dependencies. Instead, we recommend compiling the library yourself. If you compile in the [`before-all`](options.md#before-all) step, cibuildwheel will have already set the appropriate `MACOSX_DEPLOYMENT_TARGET` env var, so the library will target the correct version of macOS. !!! tip For build steps, Homebrew is still a great resource - you can [look up the build formula](https://formulae.brew.sh/) and use that as a starting point. @@ -253,9 +253,9 @@ pipx run twine upload wheelhouse/*.whl ### macOS: Passing DYLD_LIBRARY_PATH to delocate -macOS has built-in [System Integrity protections](https://developer.apple.com/library/archive/documentation/Security/Conceptual/System_Integrity_Protection_Guide/RuntimeProtections/RuntimeProtections.html) which limits the use of `DYLD_LIBRARY_PATH` and `LD_LIBRARY_PATH` so that it does not automatically pass to children processes. This means if you set `DYLD_LIBRARY_PATH` before running cibuildwheel, or even set it in `CIBW_ENVIRONMENT`, it will be stripped out of the environment before delocate is called. +macOS has built-in [System Integrity protections](https://developer.apple.com/library/archive/documentation/Security/Conceptual/System_Integrity_Protection_Guide/RuntimeProtections/RuntimeProtections.html) which limits the use of `DYLD_LIBRARY_PATH` and `LD_LIBRARY_PATH` so that it does not automatically pass to children processes. This means if you set `DYLD_LIBRARY_PATH` before running cibuildwheel, or even set it in `environment`, it will be stripped out of the environment before delocate is called. -To work around this, use a different environment variable such as `REPAIR_LIBRARY_PATH` to store the library path, and set `DYLD_LIBRARY_PATH` in [`CIBW_REPAIR_WHEEL_COMMAND_MACOS`](https://cibuildwheel.pypa.io/en/stable/options/#repair-wheel-command), like this: +To work around this, use a different environment variable such as `REPAIR_LIBRARY_PATH` to store the library path, and set `DYLD_LIBRARY_PATH` in [`macos.repair-wheel-command`](https://cibuildwheel.pypa.io/en/stable/options/#repair-wheel-command), like this: !!! tab examples "Environment variables" @@ -307,7 +307,7 @@ Then cibuildwheel will detect that it's installed and use it instead. However, y ### macOS: Library dependencies do not satisfy target MacOS Since delocate 0.11.0 there is added verification that the library binary dependencies match the target macOS version. This is to prevent the situation where a wheel platform tag is lower than the actual minimum macOS version required by the library. To resolve this error you need to build the library to the same macOS version as the target wheel (for example using `MACOSX_DEPLOYMENT_TARGET` environment variable). -Alternatively, you could set `MACOSX_DEPLOYMENT_TARGET` in `CIBW_ENVIRONMENT` to correctly label the wheel as incompatible with older macOS versions. +Alternatively, you could set `MACOSX_DEPLOYMENT_TARGET` in `environment` to correctly label the wheel as incompatible with older macOS versions. This error may happen when you install a library using a package manager like Homebrew, which compiles the library for the macOS version of the build machine. This is not suitable for wheels, as the library will only work on the same macOS version as the build machine. You should compile the library yourself, or use a precompiled binary that matches the target macOS version. diff --git a/docs/platforms.md b/docs/platforms.md index ca9c0f90..d57b1b86 100644 --- a/docs/platforms.md +++ b/docs/platforms.md @@ -25,11 +25,11 @@ Linux wheels are built in [`manylinux`/`musllinux` containers](https://github.co - Programs and libraries are not installed on the CI runner host, but rather should be installed inside the container - using `yum` for `manylinux2014`, `apt-get` for `manylinux_2_31`, `dnf` for `manylinux_2_28` and `apk` for `musllinux_1_1` or `musllinux_1_2`, or manually. The same goes for environment variables that are potentially needed to customize the wheel building. - `cibuildwheel` supports this by providing the [`CIBW_ENVIRONMENT`](options.md#environment) and [`CIBW_BEFORE_ALL`](options.md#before-all) options to setup the build environment inside the running container. + `cibuildwheel` supports this by providing the [`environment`](options.md#environment) and [`before-all`](options.md#before-all) options to setup the build environment inside the running container. - The project directory is copied into the container as `/project`, the output directory for the wheels to be copied out is `/output`. In general, this is handled transparently by `cibuildwheel`. For a more finegrained level of control however, the root of the host file system is mounted as `/host`, allowing for example to access shared files, caches, etc. on the host file system. Note that `/host` is not available on CircleCI and GitLab CI due to their Docker policies. -- Alternative Docker images can be specified with the `CIBW_MANYLINUX_*_IMAGE`/`CIBW_MUSLLINUX_*_IMAGE` options to allow for a custom, preconfigured build environment for the Linux builds. See [options](options.md#linux-image) for more details. +- Alternative Docker images can be specified with the `manylinux-*-image`/`musllinux-*-image` options to allow for a custom, preconfigured build environment for the Linux builds. See [options](options.md#linux-image) for more details. ## macOS {: #macos} @@ -80,7 +80,7 @@ If you set the value lower, cibuildwheel will cap it to the lowest supported val `cibuildwheel` supports both native builds and cross-compiling between `arm64` (Apple Silicon) and `x86_64` (Intel) architectures, including the cross-compatible `universal2` format. By default, macOS builds will build a single architecture wheel, using the build machine's architecture. -If you need to support both x86_64 and Apple Silicon, you can use the `CIBW_ARCHS` environment variable to specify the architectures you want to build, or the value `universal2` to build a multi-architecture wheel. cibuildwheel _will_ test x86_64 wheels (or the x86_64 slice of a `universal2` wheel) when running on Apple Silicon hardware using Rosetta 2 emulation, but it is *not* possible to test Apple Silicon wheels on x86_64 hardware. +If you need to support both `x86_64` and Apple Silicon, you can use the [`macos.archs`](options.md#archs) setting to specify the architectures you want to build, or the value `universal2` to build a multi-architecture wheel. cibuildwheel _will_ test `x86_64` wheels (or the `x86_64` slice of a `universal2` wheel) when running on Apple Silicon hardware using Rosetta 2 emulation, but it is *not* possible to test Apple Silicon wheels on `x86_64` hardware. #### Overview of Mac architectures @@ -129,8 +129,8 @@ If your CI provider doesn't offer arm64 runners yet, or you want to create `univ Regarding testing, -- On an arm64 runner, it is possible to test x86_64 wheels and both parts of a universal2 wheel using Rosetta 2 emulation. -- On an x86_64 runner, arm64 code can be compiled but it can't be tested. `cibuildwheel` will raise a warning to notify you of this - these warnings can be silenced by skipping testing on these platforms: `CIBW_TEST_SKIP: "*_arm64 *_universal2:arm64"`. +- On an arm64 runner, it is possible to test `x86_64` wheels and both parts of a `universal2` wheel using Rosetta 2 emulation. +- On an `x86_64` runner, arm64 code can be compiled but it can't be tested. `cibuildwheel` will raise a warning to notify you of this - these warnings can be silenced by skipping testing on these platforms: `test-skip = ["*_arm64", "*_universal2:arm64"]`. !!! note If your project uses **Poetry** as a build backend, cross-compiling on macOS [does not currently work](https://github.com/python-poetry/poetry/issues/7107). In some cases arm64 wheels can be built but their tags will be incorrect, with the platform tag showing `x86_64` instead of `arm64`. @@ -150,11 +150,11 @@ In order to speed-up builds, cibuildwheel will cache the tools it needs to be re ### Windows ARM64 builds {: #windows-arm64} -`cibuildwheel` supports cross-compiling `ARM64` wheels on all Windows runners, but a native ARM64 runner is required for testing. On non-native runners, tests for ARM64 wheels will be automatically skipped with a warning. Add `"*-win_arm64"` to your `CIBW_TEST_SKIP` setting to suppress the warning. +`cibuildwheel` supports cross-compiling `ARM64` wheels on all Windows runners, but a native `ARM64` runner is required for testing. On non-native runners, tests for `ARM64` wheels will be automatically skipped with a warning. Add `"*-win_arm64"` to your `test-skip` setting to suppress the warning. Cross-compilation on Windows relies on a supported build backend. Supported backends use an environment variable to specify their target platform (the one they are compiling native modules for, as opposed to the one they are running on), which is set in [cibuildwheel's windows.py](https://github.com/pypa/cibuildwheel/blob/main/cibuildwheel/platforms/windows.py) before building. Currently, `setuptools>=65.4.1` and `setuptools_rust` are the only supported backends. -By default, `ARM64` is not enabled when running on non-ARM64 runners. Use [`CIBW_ARCHS`](options.md#archs) to select it. +By default, `ARM64` is not enabled when running on non-`ARM64` runners. Use [`CIBW_ARCHS`](options.md#archs) to select it. ## Pyodide/WebAssembly {: #pyodide} @@ -170,7 +170,7 @@ You must target pyodide with `--platform pyodide` (or use `--only` on the identi ### Choosing a Pyodide version {: #pyodide-choosing-a-version} -It is also possible to target a specific Pyodide version by setting the `CIBW_PYODIDE_VERSION` option to the desired version. Users are responsible for setting an appropriate Pyodide version according to the `pyodide-build` version. A list is available in Pyodide's [cross-build environments metadata file](https://github.com/pyodide/pyodide/blob/main/pyodide-cross-build-environments.json), which can be viewed more easily by installing `pyodide-build` from PyPI and using `pyodide xbuildenv search --all` to see a compatibility table. +It is also possible to target a specific Pyodide version by setting the `pyodide-version` option to the desired version. Users are responsible for setting an appropriate Pyodide version according to the `pyodide-build` version. A list is available in Pyodide's [cross-build environments metadata file](https://github.com/pyodide/pyodide/blob/main/pyodide-cross-build-environments.json), which can be viewed more easily by installing `pyodide-build` from PyPI and using `pyodide xbuildenv search --all` to see a compatibility table. ### Running tests @@ -196,7 +196,7 @@ By default, cibuildwheel will build wheels for all three of these targets. If you need to specify different compilation flags or other properties on a per-ABI or per-CPU basis, you can use [configuration overrides](configuration.md#overrides) with a `select` clause that targets the specific ABI or architecture. For example, consider the following example: -``` +```toml [tool.cibuildwheel.ios] test-sources = ["tests"] test-requires = ["pytest"] @@ -233,12 +233,12 @@ iOS builds support both the `pip` and `build` build frontends. In principle, sup The environment used to run builds does not inherit the full user environment - in particular, `PATH` is deliberately re-written. This is because UNIX C tooling doesn't do a great job differentiating between "macOS ARM64" and "iOS ARM64" binaries. If (for example) Homebrew is on the path when compilation commands are invoked, it's easy for a macOS version of a library to be linked into the iOS binary, rendering it unusable on iOS. To prevent this, iOS builds always force `PATH` to a "known minimal" path, that includes only the bare system utilities, and the iOS compiler toolchain. -If your project requires additional tools to build (such as `cmake`, `ninja`, or `rustc`), those tools must be explicitly declared as cross-build tools using [`CIBW_XBUILD_TOOLS`](options.md#xbuild-tools). *Any* tool used by the build process must be included in the `CIBW_XBUILD_TOOLS` list, not just tools that cibuildwheel will invoke directly. For example, if your build script invokes `cmake`, and the `cmake` script invokes `magick` to perform some image transformations, both `cmake` and `magick` must be included in your cross-build tools list. +If your project requires additional tools to build (such as `cmake`, `ninja`, or `rustc`), those tools must be explicitly declared as cross-build tools using [`xbuild-tools`](options.md#xbuild-tools). *Any* tool used by the build process must be included in the `xbuild-tools` list, not just tools that cibuildwheel will invoke directly. For example, if your build script invokes `cmake`, and the `cmake` script invokes `magick` to perform some image transformations, both `cmake` and `magick` must be included in your cross-build tools list. ### Tests If tests have been configured, the test suite will be executed on the simulator matching the architecture of the build machine - that is, if you're building on an ARM64 macOS machine, the ARM64 wheel will be tested on an ARM64 simulator. It is not possible to use cibuildwheel to test wheels on other simulators, or on physical devices. -The iOS test environment can't support running shell scripts, so the [`CIBW_TEST_COMMAND`](options.md#test-command) value must be specified as if it were a command line being passed to `python -m ...`. In addition, the project must use [`CIBW_TEST_SOURCES`](options.md#test-sources) to specify the minimum subset of files that should be copied to the test environment. This is because the test must be run "on device", and the simulator device will not have access to the local project directory. +The iOS test environment can't support running shell scripts, so the [`test-command`](options.md#test-command) value must be specified as if it were a command line being passed to `python -m ...`. In addition, the project must use [`test-sources`](options.md#test-sources) to specify the minimum subset of files that should be copied to the test environment. This is because the test must be run "on device", and the simulator device will not have access to the local project directory. -The test process uses the same testbed used by CPython itself to run the CPython test suite. It is an Xcode project that has been configured to have a single Xcode "XCUnit" test - the result of which reports the success or failure of running `python -m `. +The test process uses the same testbed used by CPython itself to run the CPython test suite. It is an Xcode project that has been configured to have a single Xcode "XCUnit" test - the result of which reports the success or failure of running `python -m `.