Commit Graph
16 Commits
Author SHA1 Message Date
79244d366c feet: general approach to auditing wheels with abi3audit default (#2805)
* WIP - initial punt at audit command

* Add `abi3audit` as a dependency

* Add helper functions to check stable ABI wheels

* Run `abi3audit` for macOS and Windows wheels

* Copy out of container for repairing?

* Add some notes that `cibuildwheel` runs `abi3audit`

* Add basic unit tests

* Add a basic C extension with `Py_LIMITED_API`

* Add a test project that violates Stable ABI

* Fix linux test

* Skip abi3 wheel tests for Pyodide

* Patch the correct subprocess module

* wrap cleanup of abi3audit dir

* Write the docs for the new options

* Move to above testing in docs

* Implement audit-requires and audit-command

* Some cleanups after self-review

* Add default value

* fix type errors

* the key is `audit-command`, not `audit`

* Add a variety of tests for audit requires options

* Add `test_audit_requires` similar to `test_test_requires`

* Add some configurability-related audit tests

* Fix parsing error with options docs leaving out commands

* Better way to extract version (maybe helps Pyodide?)

* Fix a case of unbound `use_uv`

* Standardise: rename to `abi3_wheel`

* Fix audit command run message

* Simplify custom audit command a bit

* Remove unnecessary skip for Pyodide

* Pyodide should have no default audit command

* More accurate skip messages for Pyodide skips

* Wheels are audited after they are repaired

* Regenerate constraints to include `abi3audit`

* Fix typos

* Some attempts for Windows fixes

* Check `pyvenv.cfg` instead of directory existence

* Add validation for lack of wheel placeholders

* Try yet another Windows `uv` fix

* Regenerate diagram and re-trigger Azure CI

* Add missing `import sys` for abi3 C extension tests

* Remove audit-command at the global level

* Clarify `abi3audit` pinning a little bit

* Regen constraints

* Discard changes to cibuildwheel/resources/constraints-pyodide312.txt

* Discard changes to cibuildwheel/resources/constraints-pyodide313.txt

* try opt-in uv again

* fix issue on windows on Python 3.13 related to nested venvs

On win / python 3.13, virtualenv creates a venv where the 'home'
points back to the venv that sys.executable was running in, rather
than the root install. that seemingly leads to problems with package
resolution, where pip.exe couldn't find the pip python package.
this appears to fix it!

* Update constraints

* chore: revert python-discovery bump

Assisted-by: OpenCode:glm-5.1
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

* fix: restore workaround for graalpy

Assisted-by: OpenCode:glm-5.1
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

---------

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Co-authored-by: Agriya Khetarpal <74401230+agriyakhetarpal@users.noreply.github.com>
Co-authored-by: Henry Schreiner <henryfs@princeton.edu>
2026-05-14 07:41:14 -07:00
Henry Schreinerandpre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> 097806b6b1 tests: fully type the test suite (#2794)
* tests: fully type the test suite

* chore: require more typing

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-04-01 15:23:02 +01:00
Agriya KhetarpalHood ChathamJoe RickerbyJoe RickerbyHenry Schreinerpre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
163864158d feat: Pyodide improvements: version setting, standalone environments (#2002)
* Fix a typo: pyoodide ➡️ pyodide

* Add `pyodide_build_version` attribute

* Add version to xbuildenv log step

* Add version to Emscripten log step

* Use `pyodide-build`'s version for updating constraints

* Bump Pyodide constraints by updating `pyodide-build`

* Add a schema for `pyodide-version`

* Update Pyodide constraints

* Bump `pyodide-build` to new 0.29.0

* Test out another Pyodide identifier

* Update outdated Pyodide constraints

* Add Pyodide version to temp directory name

* Remove Pyodide 0.26.1 from build configurations

* Retrieve + validate + install specific xbuildenvs

* Test wheel builds with Pyodide 0.26.2

* Add correct Pyodide version to identifier temp dir

* Don't pre-call Pyodide xbuildenv search

* Fetch just the stable Pyodide versions

* Refactor search + validation + install into one step

* Move all of it under a lock

* Reorder xbuildenv installation

* Add env and cwd to xbuildenv search call

* Temporarily lower to 0.26.2 target

* Separate out search, validate, install; again

* Run xbuildenv search in `CIBW_CACHE_PATH`

* Remove prior `PYODIDE_ROOT` env vars, copy envs

* Validate doesn't need to depend on searching

* Add file lock when searching xbuildenvs

* Test the original version: 0.26.1

* Update Pyodide constraints

* Update constraints for `pyodide-build` 0.29.0 again

* Bump Pyodide from version 0.26.1 ➡️ version 0.26.4

* Add note on compatibility for macOS + other archs

* Note Pyodide version for Pyodide identifier

* Docs about `CIBW_PYODIDE_VERSION`

* Don't fetch just the stable versions

* Discard a variable that's not used later

* Rename `search_xbuildenv` ➡️ `get_xbuildenv_versions`

* `validate_xbuildenv` ➡️ `validate_xbuildenv_version`

* Replace ordered comment, add newline

* Replace sentence on macOS support

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Capitalise: "pyodide" ➡️ "Pyodide"

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* "work" ➡️ "may succeed"

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Add another job to test a custom Pyodide version

* Handle "v"-prefixed + non-prefixed versions

* Convert to a proper toml-able option, and remove some hardcoded versions

This removes the enscripten and pyodide-build version specs from
pyproject.toml - pyodide-build is spec'd in the constraints file, and
the emscripten version can be read from the pyodide-build output.

* Add a schema entry

* Add docs for CIBW_PYODIDE_VERSION

* Rephrase

* Add tests for pyodide-version

* Apply suggestions from code review

* Add python_build_standalone util

* Hook up to python-build-standalone, removing dependency on host python

* Remove python hard-code in action.yml

* Add log step

* Add workaround for https://github.com/pyodide/pyodide-build/issues/143

* Add emscripten pytest test

* Remove unneeded checks

* Fix a pytest invoke for emscripten

* Generate pyodide-build constraints from the pinned pyodide version

* Remove pyodide python-build-standalone workaround

* Fixup paths from newer version of pyodide-build

* Fix/skip some failing tests

* Use `python -m pytest` on pyodide, even on Linux

* Docs fixes

* Don't call the github API at runtime, cache the release assets instead

* Ignore pylint false positive

* Add version auto-updating for pyodide

* Add support for pyodide 3.13.

* Fix tests for multiple pyodide wheels

* Remove workaround for unreleased pyodide-build

* Rename to "test_pyodide"

* Fix pathname confusion

* Remove extra github actions job

* Fix expectation for test_abi_none

* Fix the custom_repair_wheel test to actually have clashing names

* Fix pinned version test

* Document test-command limitation

* Remove pyodide 0.28.0a1 for now

* Update constraints files

* Docs/test fixes post removing pyodide cp313

* Fix ABI test expectation

* Docs improvements

* Improve some comments

* Remove logic duplication

* remove pyodide special casing

* Refactor constraints code to use a utility script, circumventing import issues

* chore: nicer nox env

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: typo in variable name found by copilot

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Apply suggestions from code review

* Some more Pyodide version updates in the docs section

* We haven't released Pyodide v0.27.6 yet

* Back to the Github URL for cross-build-environments

* `pyodide-build`, not `emsdk` for Windows skips

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Move to a separate `_json_request` function

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Rename "retries" ➡️ "retry_count"

* Add some type hints

* Copy env vars before `UV_CUSTOM_COMPILE_COMMAND`

* Use `HTTPError.headers.get` instead

* Remove extra end quote

* Change download tests URL to `https://badssl.com/`

Co-Authored-By: Joe Rickerby <1244307+joerick@users.noreply.github.com>

* Download size changes, too

* Use jsdelivr for Github asset mirroring

* Bump to Pyodide v0.27.6

* Fix unit tests

* Move to pyodide v0.27.6 again

* Bump to pyodide-build 0.30.4

* Use new URL for cross-build environments metadata

Co-authored-by: Joe Rickerby <joerick@mac.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
Co-authored-by: Joe Rickerby <joerick@mac.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Joe Rickerby <1244307+joerick@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-05-20 16:59:58 -04:00
Henry Schreiner 708cf1d32b style: remove from future import annotations
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-02-28 16:38:15 -05:00
Matthieu Darbois b605f8fec8 chore: refactor cibuildwheel.utils (#2252)
* chore: refactor cibuildwheel.utils

* rework build_frontend extra flags

* ci(fix): use tonistiigi/binfmt:qemu-v8.1.5 image for qemu
2025-01-27 14:35:56 -05:00
Russell Keith-MageeandJoe Rickerby 0b82613d6f Add a test-sources configuration option. (#2062)
* Add test_sources configuration option.

* Add pyodide implementation.

* Modify default test behavior to run in the project directory.

* Update the test for running in the project directory.

* Apply suggestions from code review

Co-authored-by: Joe Rickerby <joerick@mac.com>

* Add shlex quoting to test-sources.

* Restructure ctypes example to avoid issues running from the project folder.

---------

Co-authored-by: Joe Rickerby <joerick@mac.com>
2025-01-10 15:24:24 +00:00
Henry Schreiner b98602705f chore: stricter mypy (#2053)
* chore: improve mypy

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* chore(types): type functions in tests

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* chore(types): No partial types in tests

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2024-10-22 10:16:59 -04:00
zachlewispre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>mayeut
dfd01af9aa fix: more reliably validate Podman API version (#2016)
* fix: parse version strings that include dashes

It's possible for some container engines to report their versions with a dash (e.g., "4.9.4-rhel"), which breaks packaging.version.Version's ability to parse the string. This commit introduces a version_from_string method which santizies the version string and returns an instance of Version.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* cleanup: pacify ruff

* cleanup: further pacify ruff

* fix: properly define _version_from_string method

Also, lift the method up and prefix with a "_" to better match the existing conventions

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refactor: more robust podman ver check

Use the "podman --version" command instead of "podman version -f {{json .}}" for better reliability across distributions.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: oci engine version check

Lower Docker API check to 1.41
Podman versions are not PEP440 compliant, remove distro specific suffixes before parsing.
Add tests with real-world outputs and some made up ones.

* fix: UX on OCIEngineTooOldError

* Add FlexibleVersion

per review comment

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: mayeut <mayeut@users.noreply.github.com>
2024-10-01 01:10:58 -04:00
Joe Rickerby 687981060f Improve parse_key_value_string to allow colons in values.
Fix #1600
2023-09-19 14:05:36 +01:00
Henry Schreiner c8b037ba3e chore: use Ruff (#1405)
chore: use ruff

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2023-01-30 15:53:44 -05:00
Joe Rickerby ec878dc393 Add a unit test for the workaround, and fix a bug 2022-12-04 19:08:47 +00:00
mayeut 6d27f05c7c chore: use from __future__ import annotations 2022-07-19 07:47:45 +02:00
9a537512c1 feat: add support for py3-none-{platform} wheels (#1151)
* feature: add support for `py3-none-{platform}` wheels

This extends the mechanism introduced in #1091 for `abi3` wheels.
Most of the mentions to `abi3` have been removed and replaced by a more generic `compatible_wheel`.
This allows to build a wheel `foo-0.1-py3-none-win_amd64.whl` only once and still test with every configured python.

* Add integration test for py3-none abi wheels

* Fix expected_wheels for py3-none abi

* Limit test to three pythons and check for certain log messages

* chore: add some comments to explain filter process

Co-authored-by: Joe Rickerby <joerick@mac.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2022-07-03 14:00:00 -04:00
Joe Rickerby c824ed99d6 Use hash rather than backslash for escaping 2021-11-14 18:43:34 +00:00
Joe Rickerby f4b14d3da0 Fix backslash replacement issue 2021-11-14 13:38:41 +00:00
Henry Schreiner 97e3fbbd73 fix: pass through unexpected format strings
Uses a regex-based solution instead of str.format
2021-10-23 14:54:20 +01:00