Compare commits

..
21 Commits
Author SHA1 Message Date
Henry Schreiner da84127892 docs: add Python 3.8 to the identifer list (#728) 2021-06-22 18:15:25 -04:00
Joe Rickerby bf3a5590c9 Bump version: v1.12.0 2021-06-22 18:13:50 +01:00
Henry Schreiner 8734e0e4e6 ci: pin dockcross for tests 2021-06-21 12:27:18 -04:00
Henry Schreiner 744812b7d3 ci: pin dockcross for tests 2021-06-21 12:27:18 -04:00
Henry Schreiner 548e0cefe3 feat: support building universal2/arm64 from x64_86 Python 3.8 (#704) 2021-06-21 12:27:18 -04:00
Joe Rickerby 6c62f9ee62 Bump version: v1.11.1.post1 2021-05-28 15:58:38 +01:00
Joe Rickerby 9a608b0b94 Fix mkdocs "Edit in Github" links 2021-05-28 15:53:26 +01:00
Joe Rickerby 5ea360607d Bump version: v1.11.1 2021-05-28 15:50:45 +01:00
Matthieu Darbois dc127cd69d Revert "Temporarily disable CircleCI mac builds (#685)" (#687)
This reverts commit af45aface9.
2021-05-28 14:24:46 +01:00
Joe Rickerby d1898152f7 Fix bump_version script 2021-05-28 14:13:04 +01:00
Joe Rickerby e4a917f717 Revert "Remove travis-ci macOS from docs/examples"
This reverts commit 5de574d36820c2481f4e7557ed6aa0e431ee3e63.
2021-05-24 17:24:36 -04:00
Henry Schreiner 6d5c80c624 ci: use native actions syntax (#662) 2021-05-24 17:24:36 -04:00
Henry SchreinerandJoe Rickerby 6ab9906ff0 chore: rename joerick/cibuildwheel -> pypa/cibuildwheel (#682)
* chore: rename joerick/cibuildwheel -> pypa/cibuildwheel

* Update Appveyor status badge markup

Note that it still has joerick/cibuildwheel - confusingly it's still within my appveyor account, but this is the new project that points to the pypa github repo

* Update Appveyor badge markup again

I went through a few passes to enable that project, here's the latest.

* Update Azure badge, also

* Update Travis badge to reference the .com

Co-authored-by: Joe Rickerby <joerick@mac.com>
2021-05-24 17:24:36 -04:00
Matthieu Darbois 5150122a95 Fix Travis CI tests on ppc64le/s390x (#689)
* Disable manylinux only tests on Travis CI ppc64le

Those tests are taking too much disk space for the Travis CI runner right now.
c.f. https://travis-ci.community/t/running-out-of-disk-space-quota-when-using-docker-on-ppc64le/11634

* Use focal to run aarch64/ppc64le/s390x jobs on Travis CI

This removes allow_failures from the Travis CI s390x job

* Use `focal` distribution for all Travis CI configurations

* Revert "Disable manylinux only tests on Travis CI ppc64le"

This reverts commit d0a09c689c058985c7dcb60b1e11a85a7bfce5e0.

* Skip test_manylinuxXXXX_only on Travis CI ppc64le

* Add a comment on why test_manylinuxXXXX_only test is skipped
2021-05-24 17:24:36 -04:00
Matthieu Darbois e5556f67f4 Remove macOS build on travis-ci (#686)
* Remove macOS build on travis-ci
* Remove travis-ci macOS from docs/examples
2021-05-24 17:24:36 -04:00
Edoardopre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>Henry Schreiner
f6fc96400b docs: update projects.yml (#673)
* Update projects.yml

Added EdoPro98/SimSiPM project

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* docs: add more projects

* docs: rerun projects.py

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2021-05-24 17:24:36 -04:00
Joe Rickerby 7e3bbe6e04 Temporarily disable CircleCI mac builds (#685) 2021-05-24 17:24:36 -04:00
Henry Schreiner 13f875a04a chore: move to PyPA (PSF) CoC (#679) 2021-05-18 13:51:04 -04:00
Henry SchreinerandMatthieu Darbois 339beb061c [1.x] chore: bump to final Python 3.5 pin (#674)
* chore: bump to final Python 3.5 pin

* Update pinned_docker_images.cfg

* Apply suggestions from code review

Co-authored-by: Matthieu Darbois <mayeut@users.noreply.github.com>

Co-authored-by: Matthieu Darbois <mayeut@users.noreply.github.com>
2021-05-14 16:41:12 -04:00
Henry Schreiner fee39c2f35 fix: missing f for fstrings in bin (#659) 2021-05-07 18:17:44 -04:00
Joe Rickerby 73adde0b9f Enable tests on 1.x branch 2021-05-05 08:51:06 +01:00
242 changed files with 29576 additions and 34673 deletions
-122
View File
@@ -1,122 +0,0 @@
---
name: changelog-entry
description: Generate a new changelog entry for cibuildwheel based on all changes since the last tag. Use when updating the changelog or preparing a release.
---
# Generate Changelog Entry
Produce a new version section at the top of `docs/changelog.md` summarizing all changes since the last git tag.
## Steps
1. **Find the last tag** — run `git tag --sort=-version:refname | head -1` to get the most recent version tag.
2. **Gather changes** — run `git log <last-tag>..HEAD --oneline` and `git log <last-tag>..HEAD --format="%H %s"` to see all commits since that tag.
3. **Read each merge/commit** — for substantive commits, read the full message and any linked PRs to understand the change. Use `git log <last-tag>..HEAD --format="---%n%B"` for full messages. If a commit message is ambiguous, use `gh pr view <number>` to get richer context from GitHub.
4. **Classify and draft entries** — assign each change an emoji and write a one-line description following the style rules below. Combine related small PRs of the same type (bot dependency updates, dependabot CI action bumps, pre-commit autoupdates) into a single entry listing all PR numbers. Skip meta-changelog PRs (e.g., "Add missing CHANGELOG entries") — they don't describe user-facing changes. If a bot/dependency PR contains a substantive fix mixed in (e.g., a pip revert inside a dependency update), split it: mention the fix separately under its own category.
5. **Determine the new version number** — inspect commits for breaking changes or new features to decide patch/minor/major bump. Ask the user if unclear.
6. **Determine the date** — use today's date.
7. **Insert the new section** — add it at the top of `docs/changelog.md`, right after the `# Changelog` heading and a blank line, before any existing version sections.
## Style Rules
These are non-negotiable formatting conventions derived from the existing changelog.
### Version heading
```markdown
### v3.4.2
```
`###` heading, `v` prefix, full semver.
### Date line
```markdown
_14 May 2026_
```
Italic (underscore-wrapped), day without leading zero, full month name, 4-digit year. One blank line after the date.
### Entry format
```markdown
- <emoji> <Description> (#<PR number>)
```
- Each entry is a single bullet starting with `- `.
- Emoji immediately after the dash-space.
- Space between emoji and description text.
- PR number(s) in parens at end: `(#1234)` or `(#1234, #5678)`.
- No trailing period for single-sentence entries.
- Period at end of multi-sentence entries only.
### Emoji categories
Use exactly one emoji per entry, chosen by category:
| Emoji | Category | Used for |
|-------|----------|----------|
| 🌟 | Major feature | New platforms, significant new capabilities |
| ✨ | Feature | New features, additions, user-visible enhancements |
| 🐛 | Bug fix | Bug fixes |
| 🛠 | Maintenance | Dep updates, internal improvements, behavior tweaks |
| ⚠️ | Warning | Deprecations, breaking changes, dropped support |
| 📚 | Docs | Documentation changes |
| 💼 | Internal | Non-user-facing infra/tooling changes |
| 🧪 | Tests | Test changes |
| 🔐 | Security | Security-related changes (used in past changelogs) |
### Category disambiguation
When a change could fit multiple categories, use these tiebreakers:
- **CI/workflow fixes** → 💼 (not 🧪) — they fix infra, not test logic.
- **Test suite changes** → 🧪 — only for changes to the test code itself.
- **Diagnostic output changes** (e.g., printing more info during builds) → 🛠 (not 🐛) — they're improvements, not bug fixes.
- **A bug fix that also changes test code** → 🐛 — the user-facing fix takes priority; test changes are implicit.
### Writing style
- **Present tense**: "Adds", "Fixes", "Updates", not "Added", "Fixed".
- **Sentence case**: capitalize only the first word after the emoji.
- **Link option names to docs**: `[`option-name`](https://cibuildwheel.pypa.io/en/stable/options/#option-name)`. Option anchors match the option name — verify in `docs/options.md` by searching for `{: #option-name }`.
- Be specific about what changed and why, not just that something changed.
### Ordering
Within a version section, order entries by importance:
1. 🌟 entries first
2. ⚠️ entries next
3. ✨ entries
4. 🐛 entries
5. 🛠 entries
6. 📚 entries
7. 💼 entries
8. 🧪 entries last
### Multi-line entries
For complex features needing explanation, use an indented italic paragraph:
```markdown
- ✨ Short summary here. (#1234)
_Longer explanation with details and caveats._ (#1234)
```
Adding a new Python beta version always has a specific longer explanation, check for a previous addition (like 3.14) for the note to use.
### Blank lines
- One blank line between version sections.
- No blank lines between bullets within a version.
## Validation
After inserting the new section:
1. Check that the new section follows all style rules above.
2. Verify PR numbers match actual PRs in the commit log.
3. Ensure no duplicate entries — multiple commits to the same PR should produce one entry.
4. Run a final review of the file to confirm formatting is consistent with surrounding entries.
+22 -52
View File
@@ -1,72 +1,42 @@
version: 2.1
version: 2
commands:
cibw_prepare_environment:
description: "Prepare the environment for testing."
jobs:
osx-python3.6:
macos:
xcode: 9.4.1
environment:
PYTHON: python3
steps:
- checkout
- run:
name: Prepare the environment.
command: bash .circleci/prepare.sh
cibw_run_tests:
description: "Runs tests, with CIBW_ENABLE=all on the main branch"
steps:
- run:
name: Test
command: |
if [ "${CIRCLE_BRANCH}" == "main" ]; then
echo "INFO: Exporting CIBW_ENABLE=all for main branch test run."
export CIBW_ENABLE=all
else
echo "INFO: CIBW_ENABLE not set for this branch test run."
fi
venv/bin/python ./bin/run_tests.py
name: Test.
command: venv/bin/python ./bin/run_tests.py
no_output_timeout: 30m
jobs:
osx-python312:
macos:
xcode: 15.4.0
resource_class: macos.m1.medium.gen1
environment:
PYTHON: python3
steps:
- checkout
- cibw_prepare_environment
- cibw_run_tests
linux-python312:
linux-python3.6:
docker:
- image: cimg/python:3.12
- image: circleci/python:3.6
environment:
PYTHON: python3
# Temporarily restrict the tests that are run on CircleCI to prevent
# test timeouts.
PYTEST_ADDOPTS: -k "unit_test or main_tests or test_0_basic or test_docker_images"
steps:
- checkout
- setup_remote_docker
- cibw_prepare_environment
- cibw_run_tests
linux-aarch64:
machine:
image: default
resource_class: arm.medium
environment:
PYTHON: python3
# Temporarily restrict the tests that are run on CircleCI to prevent
# test timeouts.
PYTEST_ADDOPTS: -k "unit_test or main_tests or test_0_basic or test_docker_images"
steps:
- checkout
- cibw_prepare_environment
- cibw_run_tests
- run:
name: Prepare the environment.
command: bash .circleci/prepare.sh
- run:
name: Test.
command: venv/bin/python ./bin/run_tests.py
no_output_timeout: 30m
workflows:
version: 2
all-tests:
jobs:
- osx-python312
- linux-python312
- linux-aarch64
- osx-python3.6
- linux-python3.6
+3 -10
View File
@@ -1,16 +1,9 @@
#!/usr/bin/env bash
set -o errexit
set -o xtrace
if [ "$(uname -s)" == "Darwin" ]; then
sudo softwareupdate --install-rosetta --agree-to-license
else
docker run --rm --privileged docker.io/tonistiigi/binfmt:latest --install all
fi
$PYTHON --version
$PYTHON -m venv venv
venv/bin/python -m pip install -U pip
venv/bin/python -m pip install -e. --group test
$PYTHON -m pip --version
$PYTHON -m virtualenv -p $PYTHON venv
venv/bin/python -m pip install -e ".[dev]"
venv/bin/python -m pip freeze
venv/bin/python --version
-6
View File
@@ -1,6 +0,0 @@
* text=auto
*.py diff=python
*.md diff=markdown
*.svg -diff
-7
View File
@@ -1,7 +0,0 @@
# Specification files for platform maintainers
cibuildwheel/platforms/ios.py @freakboy3742
cibuildwheel/platforms/pyodide.py @hoodmane @ryanking13 @agriyakhetarpal
cibuildwheel/platforms/android.py @mhsmith
# Constraints files
cibuildwheel/resources/constraints-pyodide*.txt @hoodmane @ryanking13 @agriyakhetarpal
-31
View File
@@ -1,31 +0,0 @@
name: General issue
description: Open a general issue
body:
- type: markdown
attributes:
value: |
Hi! If you're not sure whether your problem is a bug in cibuildwheel or something else, please consider asking a question in the Discussions tab first.
- type: textarea
id: description
attributes:
label: Description
- type: input
id: log
attributes:
label: Build log
description: If relevant, please provide a link to a build log that shows the problem
placeholder: https://...
validations:
required: false
- type: input
id: ci-config
attributes:
label: CI config
description: Link to the CI config that produced the build log above. e.g. the Github Actions workflow file
placeholder: https://...
validations:
required: false
+7
View File
@@ -0,0 +1,7 @@
# config file for https://github.com/bobvanderlinden/probot-auto-merge
# no approvals required, just the label is enough
minApprovals: {}
requiredLabels:
- merge when green
-13
View File
@@ -1,13 +0,0 @@
version: 2
updates:
# Maintain dependencies for GitHub Actions
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
groups:
actions:
patterns:
- "*"
cooldown:
default-days: 7
-40
View File
@@ -1,40 +0,0 @@
name: "Copilot Setup Steps"
permissions: {}
on:
workflow_dispatch:
push:
paths:
- .github/workflows/copilot-setup-steps.yml
pull_request:
paths:
- .github/workflows/copilot-setup-steps.yml
jobs:
copilot-setup-steps:
name: Copilot Setup Steps
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
allow-prereleases: true
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
- name: Install tooling
run: |
uv tool install nox
uv tool install prek
- name: Pre-install checks
run: |
prek prepare-hooks
-46
View File
@@ -1,46 +0,0 @@
name: Release
on:
workflow_dispatch:
pull_request:
release:
types:
- published
permissions: {}
jobs:
dist:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- uses: hynek/build-and-inspect-python-package@d44ca7d91762de7a7d5436ddae667c6da6d1c3df # v2.18.0
publish:
needs: [dist]
runs-on: ubuntu-latest
if: github.event_name == 'release' && github.event.action == 'published'
environment:
name: pypi
url: https://pypi.org/p/cibuildwheel
permissions:
id-token: write
attestations: write
steps:
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: Packages
path: dist
- name: Generate artifact attestation for sdist and wheel
uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4.1.0
with:
subject-path: "dist/cibuildwheel-*"
- uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0
with:
attestations: true
+39 -324
View File
@@ -3,367 +3,82 @@ name: Test
on:
push:
branches:
- main
- 2.x
- master
- 1.x
pull_request:
types:
- opened
- synchronize
- reopened
- labeled
paths-ignore:
- .ci*
- bin/*
- docs/**
- examples/azure-pipelines-*
- examples/ci*
- examples/travis-ci-*
- .pre-commit-config.yaml
- .readthedocs.yml
- .travis.yml
- README.md
- azure-pipelines.yml
- mkdocs.yml
- noxfile.py
- 'docs/**'
workflow_dispatch:
# allow manual runs on branches without a PR
permissions: {}
concurrency:
group: test-${{ github.ref }}
cancel-in-progress: true
jobs:
lint:
name: Linters (mypy, ruff, etc.)
pre-commit:
name: Pre-commit checks (mypy, flake8, etc.)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
- uses: j178/prek-action@bdca6f102f98e2b4c7029491a53dfd366469e33d # v2.0.4
- name: PyLint checks
run: uvx nox -s pylint -- --output-format=github
sample:
name: Generate a sample project
needs: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
name: Install Python
with:
python-version: "3.x"
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
- name: Install dependencies
run: uv sync --no-dev --group test
- name: Generate a sample project
run: uv run --no-sync -m test.test_projects test.test_0_basic.basic_project sample_proj
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: sample_proj
path: sample_proj
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- uses: pre-commit/action@v2.0.0
test:
name: Test on ${{ matrix.os }} (${{ matrix.python_version }}) ${{ matrix.test_select }}
needs: sample
name: Test cibuildwheel on ${{ matrix.os }}
needs: pre-commit
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
# Min Python
- os: ubuntu-latest
python_version: '3.11'
# Max Python
- os: ubuntu-latest
python_version: '3.15'
- os: ubuntu-latest
python_version: '3.14'
test_select: android
- os: ubuntu-24.04-arm
python_version: '3.13'
- os: windows-latest
python_version: '3.13'
- os: windows-11-arm
python_version: '3.13'
- os: macos-15-intel
python_version: '3.13'
- os: macos-15
python_version: '3.13'
- os: macos-15-intel
python_version: '3.13'
test_select: ios
- os: macos-15
python_version: '3.13'
test_select: ios
# Exercise iOS on a non-default simulator.
test_runtime: 'args: --simulator "iPhone 16e,OS=18.5"'
- os: macos-15-intel
python_version: '3.13'
test_select: android
# Exercise Android on a non-default simulator
test_runtime: 'args: --managed minVersion'
- os: macos-15
python_version: '3.13'
test_select: android
os: [ubuntu-latest, windows-latest, macos-latest]
python_version: ['3.7']
timeout-minutes: 180
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
name: Install Python ${{ matrix.python_version }}
with:
python-version: ${{ matrix.python_version }}
allow-prereleases: true
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
- name: Free up disk space
if: runner.os == 'Linux' && matrix.test_select != 'android'
run: |
sudo rm -rf $ANDROID_HOME/ndk /opt/hostedtoolcache/CodeQL \
/usr/local/lib/node_modules /usr/local/share/chromium \
/usr/local/share/powershell
df -m
# for oci_container unit tests
- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@06116385d9baf250c9f4dcb4858b16962ea869c3 # v4.1.0
- name: Set up Xcode
if: ${{ startsWith(matrix.os, 'macos-15') }}
run: |
# GitHub recommends explicitly selecting the desired Xcode version
sudo xcode-select --switch /Applications/Xcode_16.4.app
- name: Install dependencies
run: uv sync --no-dev --group test
- uses: joerick/pr-labels-action@0543b277721e852d821c6738d449f2f4dea03d5f # v1.0.9
- name: Set CIBW_ENABLE
shell: bash
run: |
if [[ "$GITHUB_REF_NAME" == "main" ]]; then
CIBW_ENABLE=all
else
# get the default CIBW_ENABLE value from the test module
CIBW_ENABLE=$(uv run --no-sync python -c 'import sys, test.utils as c; sys.stdout.write(c.DEFAULT_CIBW_ENABLE)')
python -m pip install -e ".[dev]"
# if this is a PR, check for labels
if [[ -n "$GITHUB_PR_LABEL_CI_PYPY" ]]; then
CIBW_ENABLE+=" pypy"
fi
if [[ -n "$GITHUB_PR_LABEL_CI_GRAALPY" ]]; then
CIBW_ENABLE+=" graalpy"
fi
fi
echo "CIBW_ENABLE=${CIBW_ENABLE}" >> $GITHUB_ENV
- name: Sample build
if: "contains(github.event.pull_request.labels.*.name, 'CI: Sample build')"
run: |
python bin/sample_build.py
- name: Download a sample project
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: sample_proj
path: sample_proj
- name: Run a sample build (GitHub Action)
uses: ./
with:
package-dir: sample_proj
output-dir: wheelhouse
- name: Get some sample wheels
run: |
python -m test.test_projects test.test_0_basic.basic_project sample_proj
cibuildwheel --output-dir wheelhouse sample_proj
env:
CIBW_ARCHS_MACOS: x86_64 universal2 arm64
CIBW_BUILD_FRONTEND: ${{ matrix.test_select && 'build' || 'build[uv]' }}
CIBW_PLATFORM: ${{ matrix.test_select }}
CIBW_TEST_RUNTIME: ${{ matrix.test_runtime }}
- name: Run a sample build (GitHub Action, only)
uses: ./
if: matrix.test_select == ''
- uses: actions/upload-artifact@v2
with:
package-dir: sample_proj
output-dir: wheelhouse_only
only: cp313-${{ runner.os == 'Linux' && (runner.arch == 'ARM64' && 'manylinux_aarch64' || 'manylinux_x86_64') || (runner.os == 'Windows' && 'win_amd64' || 'macosx_x86_64') }}
- name: Create custom configuration file
shell: bash
run: |
cat > sample_proj/cibw.toml <<EOF
[tool.cibuildwheel]
# Only build on CPython 3.13 on native arch
archs = ["native"]
build = "cp313-*"
# Skip musllinux
skip = "*-musllinux*"
EOF
- name: Run a sample build (GitHub Action, config-file)
uses: ./
env:
CIBW_PLATFORM: ${{ matrix.test_select }}
CIBW_TEST_RUNTIME: ${{ matrix.test_runtime }}
with:
package-dir: sample_proj
output-dir: wheelhouse_config_file
config-file: sample_proj/cibw.toml
- name: Check Action artifacts
shell: bash
run: |
test $(find wheelhouse -name '*.whl' | wc -l) -ge 1
test $(find wheelhouse_config_file -name '*.whl' | wc -l) -eq 1
- name: Check Action artifacts (native build only)
if: matrix.test_select == ''
shell: bash
run: |
test $(find wheelhouse_only -name '*.whl' | wc -l) -eq 1
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: wheelhouse/*.whl
name: sample_wheels
path: wheelhouse
- name: Test cibuildwheel
env:
CIBW_TEST_RUNTIME: ${{ matrix.test_runtime }}
run: uv run --no-sync bin/run_tests.py --test-select=${{ matrix.test_select || 'native' }} ${{ (runner.os == 'Linux' && runner.arch == 'X64') && '--run-podman' || '' }}
emulated-archs:
name: Get qemu emulated architectures
needs: lint
runs-on: ubuntu-latest
outputs:
archs: ${{ steps.archs.outputs.archs }}
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
- name: Install dependencies
run: uv sync --no-dev --group test
- name: Get qemu emulated architectures
id: archs
run: |
OUTPUT=$(uv run --no-sync python -c "from json import dumps; from test.utils import EMULATED_ARCHS; print(dumps(EMULATED_ARCHS))")
echo "${OUTPUT}"
echo "archs=${OUTPUT}" >> "$GITHUB_OUTPUT"
python ./bin/run_tests.py
test-emulated:
name: Test Linux ${{ matrix.arch }} using qemu
needs: emulated-archs
name: Test emulated cibuildwheel using qemu
needs: pre-commit
runs-on: ubuntu-latest
timeout-minutes: 180
strategy:
matrix:
arch: ${{ fromJSON(needs.emulated-archs.outputs.archs) }}
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- name: Install dependencies
run: uv sync --no-dev --group test
run: |
python -m pip install -r requirements-dev.txt
- name: Set up QEMU
uses: docker/setup-qemu-action@06116385d9baf250c9f4dcb4858b16962ea869c3 # v4.1.0
id: qemu
uses: docker/setup-qemu-action@v1
with:
platforms: all
- name: Run the emulation tests
env:
MATRIX_ARCH: ${{ matrix.arch }}
run: uv run --no-sync pytest --run-emulation "$MATRIX_ARCH" test/test_emulation.py
test-pyodide:
name: Test pyodide
needs: sample
runs-on: ubuntu-24.04
timeout-minutes: 180
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
name: Install Python 3.12
with:
python-version: '3.12'
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
- name: Install dependencies
run: uv sync --no-dev --group test
- name: Download a sample project
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: sample_proj
path: sample_proj
- name: Run a sample build (GitHub Action)
uses: ./
with:
package-dir: sample_proj
output-dir: wheelhouse
env:
CIBW_PLATFORM: pyodide
CIBW_ENABLE: pyodide-prerelease
- name: Run tests with 'CIBW_PLATFORM' set to 'pyodide'
run: uv run --no-sync ./bin/run_tests.py
env:
CIBW_PLATFORM: pyodide
CIBW_ENABLE: pyodide-prerelease
test-uv-extras:
name: Test uv extra on ${{ matrix.os }} ${{ matrix.test_select }}
needs: sample
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
test_select: android
- os: windows-latest
- os: windows-11-arm
- os: macos-15-intel
- os: macos-15
- os: macos-15-intel
test_select: android
- os: macos-15
test_select: android
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Download a sample project
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: sample_proj
path: sample_proj
- name: Run a sample build (GitHub Action)
uses: ./
with:
extras: uv
package-dir: sample_proj
output-dir: wheelhouse
env:
CIBW_ARCHS_MACOS: x86_64 universal2 arm64
CIBW_BUILD: cp314-*
CIBW_BUILD_FRONTEND: 'build[uv]'
CIBW_PLATFORM: ${{ matrix.test_select }}
run: |
pytest --run-emulation test/test_emulation.py
+11 -47
View File
@@ -1,60 +1,30 @@
name: Update dependencies
on:
pull_request:
paths:
- '.github/workflows/update-dependencies.yml'
- 'bin/update_pythons.py'
- 'bin/update_docker.py'
- 'bin/update_virtualenv.py'
- 'bin/projects.py'
- 'docs/data/projects.yml'
- 'noxfile.py'
workflow_dispatch:
schedule:
- cron: '0 6 * * 1' # "At 06:00 on Monday."
permissions: {}
jobs:
update-dependencies:
name: Update dependencies
if: github.repository_owner == 'pypa' || github.event_name != 'schedule'
runs-on: ubuntu-latest
permissions:
contents: write
environment: ${{ github.ref == 'refs/heads/main' && github.repository == 'pypa/cibuildwheel' && 'update-dependencies-workflow' || '' }}
steps:
# we use this step to grab a Github App auth token, so that PRs generated by this workflow
# run the GHA tests.
- uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
id: generate-token
if: github.ref == 'refs/heads/main' && github.repository == 'pypa/cibuildwheel'
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
name: Install Python 3.9
with:
client-id: ${{ secrets.CIBUILDWHEEL_BOT_APP_ID }}
private-key: ${{ secrets.CIBUILDWHEEL_BOT_APP_PRIVATE_KEY }}
permission-contents: write
permission-pull-requests: write
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
python-version: "3.14"
python-version: 3.9
architecture: x64
- name: Install dependencies
run: python -m pip install ".[dev]"
- name: "Run update: dependencies"
run: uvx nox --force-color -s update_constraints
run: python ./bin/update_dependencies.py
- name: "Run update: python configs"
run: uvx nox --force-color -s update_pins
- name: "Run update: docs user projects"
run: uvx nox --force-color -s update_proj -- --auth=${{ secrets.GITHUB_TOKEN }}
run: python ./bin/update_pythons.py --force
- name: Create Pull Request
if: github.ref == 'refs/heads/main' && github.repository == 'pypa/cibuildwheel'
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
if: github.ref == 'refs/heads/master' && github.repository == 'pypa/cibuildwheel'
uses: peter-evans/create-pull-request@v3
with:
commit-message: Update dependencies
title: '[Bot] Update dependencies'
@@ -63,10 +33,4 @@ jobs:
PR generated by "Update dependencies" [workflow](https://github.com/${{github.repository}}/actions/runs/${{github.run_id}}).
branch: update-dependencies-pr
sign-commits: true
token: ${{ steps.generate-token.outputs.token }}
delete-branch: true
labels: |
CI: GraalPy
CI: PyPy
dependencies
@@ -1,32 +0,0 @@
name: Update the vX.Y tag
on:
release:
types: [released]
workflow_dispatch:
inputs:
TAG_NAME:
description: 'Tag name that the major.minor tag will point to'
required: true
env:
TAG_NAME: ${{ github.event.inputs.TAG_NAME || github.event.release.tag_name }}
jobs:
update_tag:
name: Update the major.minor tag to include the ${{ github.event.inputs.TAG_NAME || github.event.release.tag_name }} changes
environment:
name: releaseNewActionVersion
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Update the ${{ env.TAG_NAME }} tag
id: update-major-minor-tag
uses: joerick/update-vX.Y-tag-action@c4cefb60c33b82e4aa73a469e4acf30ee48d5812 # v1.0.2
with:
source-tag: ${{ env.TAG_NAME }}
-24
View File
@@ -1,24 +0,0 @@
name: zizmor - GitHub Actions Security Analysis
on:
push:
branches: ["main"]
pull_request:
branches: ["**"]
permissions: {}
jobs:
zizmor:
name: Run zizmor 🌈
runs-on: ubuntu-latest
permissions:
security-events: write
steps:
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Run zizmor 🌈
uses: zizmorcore/zizmor-action@5f14fd08f7cf1cb1609c1e344975f152c7ee938d # v0.5.6
+5 -13
View File
@@ -84,7 +84,9 @@ celerybeat-schedule
# virtualenv
.venv
venv*/
venv/
venv3/
venv2/
ENV/
env/
env2/
@@ -110,15 +112,5 @@ all_known_setup.yaml
# mkdocs
site/
# PyCharm
.idea/
# Lockfiles
*.lock
*.pylock
# OS files
.DS_Store
# This file should be a symlink or contain "See @AGENTS.md"
CLAUDE.md
# Virtual environments
venv*
+2 -45
View File
@@ -1,5 +1,5 @@
linux:
image: python:3.13
image: python:3.8
services:
- name: docker:dind
entrypoint: ["env", "-u", "DOCKER_HOST"]
@@ -9,50 +9,7 @@ linux:
DOCKER_DRIVER: overlay2
# See https://github.com/docker-library/docker/pull/166
DOCKER_TLS_CERTDIR: ""
# skip all but the basic tests
# (comment the below line in a PR to debug a Gitlab-specific issue)
PYTEST_ADDOPTS: -k "unit_test or test_0_basic" --suppress-no-test-exit-code
rules:
- if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH =~ /^gitlab/'
variables:
CIBW_ENABLE: "all"
script:
- curl -sSL https://get.docker.com/ | sh
- docker run --rm --privileged docker.io/tonistiigi/binfmt:latest --install all
- python -m pip install -U pip
- python -m pip install -e. pytest-custom-exit-code --group test
- python -m pip install -e ".[dev]"
- python ./bin/run_tests.py
windows:
variables:
PYTEST_ADDOPTS: -k "unit_test or test_0_basic" --suppress-no-test-exit-code
rules:
- if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH =~ /^gitlab/'
variables:
# Everything except graalpy. GraalPy is JVM-based and very slow to
# start; on the small (2-core) GitLab SaaS Windows runners,
# virtualenv's interpreter query for graalpy.exe times out, failing
# the build. (Other platforms run the full "all" group.)
CIBW_ENABLE: "cpython-prerelease pypy pypy-eol pyodide-eol pyodide-prerelease"
script:
- python -m pip install -U pip
- python -m pip install -e. pytest-custom-exit-code --group test
- python bin\run_tests.py
tags:
- saas-windows-medium-amd64
macos:
image: macos-15-xcode-16
variables:
PYTEST_ADDOPTS: -k "unit_test or test_0_basic" --suppress-no-test-exit-code
rules:
- if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH =~ /^gitlab/'
variables:
CIBW_ENABLE: "all"
script:
- python3 -m pip install -U pip
- python3 -m pip install -e. pytest-custom-exit-code --group test
- python3 ./bin/run_tests.py
tags:
- saas-macos-medium-m1
+44 -81
View File
@@ -1,10 +1,7 @@
ci:
autoupdate_schedule: monthly
exclude: "^cibuildwheel/resources/ios-support/(?!make_cross_venv\\.py)"
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: 3e8a8703264a2f4a69428a0aa4dcb512790b2c8c # frozen: v6.0.0
rev: v3.4.0
hooks:
- id: check-case-conflict
- id: check-merge-conflict
@@ -15,94 +12,60 @@ repos:
exclude: (cibuildwheel/resources/pinned_docker_images.cfg)|(.svg$)
- id: mixed-line-ending
- id: trailing-whitespace
exclude: ^cibuildwheel/resources/android/android.patch$
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: 0671d8ab202c4ac093b78433ae5baf74f3fc7246 # frozen: v0.15.15
- repo: https://github.com/PyCQA/isort
rev: 5.8.0
hooks:
- id: ruff-check
args: ["--fix"]
- id: ruff-format
- id: isort
- repo: https://github.com/psf/black
rev: 21.4b2
hooks:
- id: black
- repo: https://github.com/asottile/setup-cfg-fmt
rev: v1.17.0
hooks:
- id: setup-cfg-fmt
stages: [manual]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: d2823d321df3af8f878f7ee3414dc94d037145b9 # frozen: v2.1.0
rev: v0.812
hooks:
- id: mypy
name: mypy 3.11 on cibuildwheel/
args: ["--python-version=3.11"]
exclude: ^cibuildwheel/resources/android/_cross_venv.py$ # Requires Python 3.13 or later
additional_dependencies: &mypy-dependencies
- bracex
- build
- dependency-groups>=1.2
- humanize
- nox>=2025.2.9
- packaging
- pygithub
- pytest
- rich
- tomli_w
- types-certifi
- types-click
- types-jinja2
- types-pyyaml
- types-requests
- types-setuptools
- uv
- validate-pyproject
exclude: ^(bin/|cibuildwheel/resources/).*py$
additional_dependencies: [packaging]
args: ["--python-version=3.6", "--ignore-missing-imports", "--scripts-are-modules"]
- id: mypy
name: mypy 3.14
args: ["--python-version=3.14"]
additional_dependencies: *mypy-dependencies
name: mypy 3.7+ on bin/
files: ^bin/.*py$
additional_dependencies: [packaging]
args: ["--python-version=3.7", "--ignore-missing-imports", "--scripts-are-modules"]
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: 745eface02aef23e168a8afb6b5737818efbea95 # frozen: v0.11.0.1
- repo: https://github.com/asottile/pyupgrade
rev: v2.14.0
hooks:
- id: shellcheck
- id: pyupgrade
name: PyUpgrade 3.6+
args: ["--py36-plus"]
exclude: ^bin/
- id: pyupgrade
name: PyUpgrade 3.7+ on bin
exclude: ^(cibuildwheel|unit_test|test)/
args: ["--py37-plus"]
- repo: https://github.com/PyCQA/flake8
rev: 3.9.1
hooks:
- id: flake8
exclude: cibuildwheel/resources/
additional_dependencies: [flake8-bugbear]
- repo: local
hooks:
- id: disallow-caps
name: Disallow improper capitalization
language: pygrep
entry: PyBind|Numpy|Cmake|Github|PyTest
types: [markdown]
exclude: ^docs/working-examples\.md$ # Autogenerated
- id: cog
name: Cog the README
language: python
pass_filenames: false
entry: cog -c -P -r -I ./bin README.md
files: '^(README\.md|docs/changelog\.md|docs/options\.md|bin/readme.*)$'
additional_dependencies: [cogapp>=3.5]
- repo: https://github.com/henryiii/flake8-lazy
rev: 65b94e97b1091634368592d98072160842265fe4 # frozen: v0.8.2
hooks:
- id: flake8-lazy
args: ["--apply=set"]
files: '^cibuildwheel'
exclude: "^cibuildwheel/resources/"
- repo: https://github.com/codespell-project/codespell
rev: 2ccb47ff45ad361a21071a7eedda4c37e6ae8c5a # frozen: v2.4.2
hooks:
- id: codespell
args: ["-w"]
exclude: ^docs/working-examples\.md$ # Autogenerated
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 943377262562a12b57292fc98fabd7dbf81451fe # frozen: 0.37.2
hooks:
- id: check-dependabot
- id: check-github-actions
- id: check-github-workflows
- id: check-gitlab-ci
- id: check-readthedocs
- id: check-jsonschema
name: Check projects
args: [--schemafile, docs/data/projects.schema.json]
files: '^docs/data/projects.yml$'
- id: check-metaschema
files: '^docs/data/projects.schema.json$'
entry: PyBind|Numpy|Cmake|Github
types:
- markdown
+10 -7
View File
@@ -2,10 +2,13 @@
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
version: 2
build:
os: ubuntu-24.04
commands:
- asdf plugin add uv
- asdf install uv latest
- asdf global uv latest
- NO_COLOR=1 uv run --python 3.14 --managed-python --no-dev --group docs mkdocs build --strict --site-dir $READTHEDOCS_OUTPUT/html
mkdocs:
configuration: mkdocs.yml
python:
version: 3.8
install:
- method: pip
path: .
extra_requirements:
- docs
+50
View File
@@ -0,0 +1,50 @@
os: linux
dist: focal
language: python
branches:
only:
- master
jobs:
include:
- name: Linux | x86_64 + i686 | Python 3.6
python: 3.6
services: docker
env: PYTHON=python
- name: Linux | arm64 | Python 3.6
python: 3.6
services: docker
arch: arm64
env: PYTHON=python
- name: Linux | ppc64le | Python 3.6
python: 3.6
services: docker
arch: ppc64le
env:
- PYTHON=python
# skip test_manylinuxXXXX_only, it uses too much disk space
# c.f. https://travis-ci.community/t/running-out-of-disk-space-quota-when-using-docker-on-ppc64le/11634
- PYTEST_ADDOPTS='-k "not test_manylinuxXXXX_only"'
- name: Windows | x86_64 | Python 3.6
os: windows
language: shell
before_install:
- choco upgrade python3 -y --version 3.6.8 --limit-output
# Update root certificates to fix SSL error; see http://www.chawn.com/RootCerts.htm
- powershell "md C:\temp\certs; CertUtil -generateSSTFromWU C:\temp\certs\RootStore.sst; Get-ChildItem -Path C:\\temp\certs\Rootstore.sst | Import-Certificate -CertStoreLocation Cert:\\LocalMachine\\Root\\ | out-null"
env:
- PYTHON=C:\\Python36\\python
- name: Linux | s390x | Python 3.6
python: 3.6
services: docker
arch: s390x
env: PYTHON=python
install: $PYTHON -m pip install -e ".[dev]" pytest-custom-exit-code
script: $PYTHON ./bin/run_tests.py
-57
View File
@@ -1,57 +0,0 @@
# cibuildwheel — Agent Notes
## Always run
- `prek -a` should be run after changes to reformat, lint, and type check
## Developer commands
- `uv run pytest unit_test` — Quick run for unit tests
- `nox -s tests` — run doctests + unit tests + integration tests (slow, >30 min, requires system Python installs on macOS).
- `nox -s tests -- unit_test` — fast unit tests only.
- `nox -s tests -- test -k before_build` — single integration test/file via pytest `-k`.
- `nox -s lint` — run all linters (pre-commit/prek).
- `nox -s pylint` — run pylint separately (not in pre-commit).
- `nox -s docs` — mkdocs serve (interactive) or build (non-interactive).
- Set up local dev env at `.venv`: `uv sync` (dependency groups used).
## Project layout
- `cibuildwheel/` — main package. Entry point: `cibuildwheel.__main__:main`.
- `test/`**integration tests** (expensive, run actual wheel builds).
- `unit_test/`**unit tests** (fast, no wheel builds).
- `bin/` — maintainer scripts (update pins, generate README tables, schema, etc.).
- `docs/` — mkdocs source.
## Testing specifics
- Three test suites exist, run in this order by `bin/run_tests.py`:
1. `pytest cibuildwheel` — doctests.
2. `pytest unit_test [...]` — unit tests.
3. `pytest test [...]` — integration tests (split into `serial` and `not serial` runs).
- Serial integration tests **must not** run in parallel; non-serial use pytest-xdist by default.
- Custom pytest options:
- `--run-docker` (unit_test + test): run OCI container tests. Linux only.
- `--run-podman`: run podman tests (Linux).
- `--run-emulation` (test): run QEMU emulation tests (e.g., `--run-emulation aarch64`).
- `--platform linux` (test): force integration tests to target Linux container builds even on macOS/Windows.
- `--enable` (test): sets `CIBW_ENABLE` env var (e.g., `pypy`, `graalpy`).
- Integration tests auto-set a default `CIBW_ENABLE` if the env var is absent.
- The `build_frontend_env` fixture parameterizes over `pip`, `build`, `build[uv]`, `uv` and skips unsupported combos per platform.
- Some integration tests require system Python.org installs on macOS; missing them prints a download URL in the error.
- iOS/Android/pyodide tests have dedicated pytest marks (`ios`, `android`, `pyodide`) and need platform-specific runners/simulators.
## Lint / typecheck
- Ruff (lint + format) and mypy run via pre-commit. Pylint runs separately via `nox -s pylint`.
- Mypy is strict (`strict = true`) and targets Python 3.11 for the package, 3.14 for a second check in pre-commit.
- Ruff config in `pyproject.toml` (`line-length = 100`).
- Python 3.11 is the minimum supported version for the package itself.
## Generated / maintained files
- `README.md` contains two **cog-generated** tables (options table, changelog preview). Pre-commit runs `cog -c -P -r -I ./bin README.md`. Edit the source scripts (`bin/readme_*.py`) or the upstream files (`docs/options.md`, `docs/changelog.md`) — do not hand-edit the generated blocks. Note the identifier is _not_ cog generated, and can be edited.
- `cibuildwheel/resources/cibuildwheel.schema.json` is generated by `bin/generate_schema.py` (run via `nox -s generate_schema`).
- `cibuildwheel/resources/constraints-*.txt` are generated via `nox -s update_constraints`.
- `cibuildwheel/resources/pinned_docker_images.cfg` and other resource files are updated via `nox -s update_pins`.
## CI / release quirks
- CI uses `uv sync --no-dev --group test` for test installs, then `uv run --no-sync` to execute.
- The release workflow uses `hynek/build-and-inspect-python-package` for dist building.
- `test.yml` skips unrelated paths to avoid burning CI time on docs-only changes.
- A sample project artifact is built once and downloaded by downstream test jobs to avoid redundant work.
+10 -7
View File
@@ -1,9 +1,12 @@
This is a summary of the host Python versions and platforms covered by the different CI platforms:
This is a summary of the Python versions and platforms covered by the different CI platforms:
| | 3.11 | 3.12 | 3.13 | 3.14 |
|---------|----------------------------------|-----------------------|----------------|----------------|
| Linux | Azure Pipelines / GitHub Actions | CircleCI¹ / GitLab¹ | GitHub Actions | GitHub Actions |
| macOS | Azure Pipelines | CircleCI¹ / GitLab¹ | GitHub Actions | |
| Windows | Azure Pipelines | GitLab¹ | GitHub Actions | |
| | 3.6 | 3.7 | 3.8 |
|----------|------------------------------|------------------------------------------|------------------|
| Linux | Travis CI / CircleCI | AppVeyor² / GitHub Actions | Azure Pipelines |
| macOS | CircleCI | AppVeyor² / Travis CI¹ / GitHub Actions | Azure Pipelines |
| Windows | Travis CI / Azure Pipelines | AppVeyor² / GitHub Actions | Azure Pipelines |
> ¹ Runs a reduced set of tests to reduce CI load
> ¹ Python version not really pinned, but dependent on the (default) version of image used.
> ² AppVeyor only runs the "basic" test to reduce load.
Non-x86 architectures are covered on Travis CI using Python 3.6.
+24 -5
View File
@@ -1,9 +1,28 @@
Copyright 2017, Joe Rickerby and contributors. All rights reserved.
This project is licensed under the 'BSD 2-clause license'.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
Copyright (c) 2017-2019, Joe Rickerby and contributors. All rights reserved.
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Get-pip and Pip (bundled in cibuildwheel/resources) are licensed under the MIT
license. See https://github.com/pypa/get-pip/blob/master/LICENSE.txt
+765 -232
View File
File diff suppressed because it is too large Load Diff
+8 -127
View File
@@ -9,18 +9,6 @@ inputs:
description: 'Folder to place the outputs in, defaults to "wheelhouse"'
required: false
default: wheelhouse
config-file:
description: 'File containing the config, defaults to {package}/pyproject.toml'
required: false
default: ''
only:
description: 'Build a specific wheel only. No need for arch/platform if this is set'
required: false
default: ''
extras:
description: 'Comma-separated list of extras to install'
required: false
default: ''
branding:
icon: package
color: yellow
@@ -28,120 +16,13 @@ branding:
runs:
using: composite
steps:
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
id: python
with:
python-version: "3.11 - 3.14"
update-environment: false
- id: cibw
run: |
# Install cibuildwheel and build the command line
"$PYTHON" -u << "EOF"
import os
import shlex
import shutil
import sys
import venv
from pathlib import Path
from subprocess import run
EXTRAS = set(e.strip() for e in os.environ.get("INPUT_EXTRAS", "").split(",") if e.strip())
class EnvBuilder(venv.EnvBuilder):
def __init__(self):
super().__init__()
def setup_scripts(self, context):
pass
def post_setup(self, context):
super().post_setup(context)
self.bin_path = Path(context.env_exe).parent
install_spec = os.environ["GITHUB_ACTION_PATH"]
if EXTRAS:
install_spec += f"[{','.join(sorted(EXTRAS))}]"
run([sys.executable, "-m", "pip", "--python", context.env_exe, "install", install_spec], check=True)
print("::group::Install cibuildwheel")
venv_path = Path(os.environ["RUNNER_TEMP"]) / "cibw"
if venv_path.exists():
shutil.rmtree(venv_path)
builder = EnvBuilder()
builder.create(venv_path)
exposed_binaries = {"cibuildwheel"}
if "uv" in EXTRAS:
exposed_binaries.add("uv")
clean_bin_path = builder.bin_path.parent / f"{builder.bin_path.name}.clean"
clean_bin_path.mkdir()
for path in list(builder.bin_path.iterdir()):
if path.stem in exposed_binaries:
try:
os.symlink(path, clean_bin_path / path.name)
except OSError:
shutil.copy2(path, clean_bin_path / path.name)
cibw_bin = [p for p in builder.bin_path.glob("cibuildwheel*") if p.stem == "cibuildwheel"][0]
# Build the command line
cmd_args = [str(cibw_bin), os.environ["INPUT_PACKAGE_DIR"]]
if output_dir := os.environ.get("INPUT_OUTPUT_DIR"):
cmd_args += ["--output-dir", output_dir]
if config_file := os.environ.get("INPUT_CONFIG_FILE"):
cmd_args += ["--config-file", config_file]
if only := os.environ.get("INPUT_ONLY"):
cmd_args += ["--only", only]
cmd_bash = shlex.join(cmd_args)
def pwsh_quote(text):
# Wrap in single quotes and double-up any existing single quotes
return "'" + str(text).replace("'", "''") + "'"
# Prepend '& ' so PowerShell executes the quoted binary path
cmd_pwsh = "& " + " ".join(pwsh_quote(arg) for arg in cmd_args)
with open(os.environ["GITHUB_OUTPUT"], "at") as f:
f.write(f"prepend-path={clean_bin_path}\n")
f.write(f"cmd-bash={cmd_bash}\n")
f.write(f"cmd-pwsh={cmd_pwsh}\n")
print("::endgroup::")
EOF
shell: bash
env:
PYTHON: ${{ steps.python.outputs.python-path }}
INPUT_PACKAGE_DIR: ${{ inputs.package-dir }}
INPUT_OUTPUT_DIR: ${{ inputs.output-dir }}
INPUT_CONFIG_FILE: ${{ inputs.config-file }}
INPUT_ONLY: ${{ inputs.only }}
INPUT_EXTRAS: ${{ inputs.extras }}
# Redirecting stderr to stdout to fix interleaving issue in Actions.
- run: |
export PATH="$CIBW_PREPEND_PATH:$PATH"
eval "$CIBW_CMD_BASH" 2>&1
shell: bash
if: runner.os != 'Windows'
env:
CIBW_PREPEND_PATH: ${{ steps.cibw.outputs.prepend-path }}
CIBW_CMD_BASH: ${{ steps.cibw.outputs.cmd-bash }}
# Windows needs powershell to interact nicely with Meson
- run: |
$env:PATH = "$env:CIBW_PREPEND_PATH;$env:PATH"
Invoke-Expression $env:CIBW_CMD_PWSH
# This needs powershell-core because github.action path is a Windows-style
# path on Windows. Powershell-core understands both types of paths.
- run: >
pipx run
--spec ${{ github.action_path }}
cibuildwheel
${{ inputs.package-dir }}
--output-dir ${{ inputs.output-dir }}
shell: pwsh
if: runner.os == 'Windows'
env:
CIBW_PREPEND_PATH: ${{ steps.cibw.outputs.prepend-path }}
CIBW_CMD_PWSH: ${{ steps.cibw.outputs.cmd-pwsh }}
+34
View File
@@ -0,0 +1,34 @@
environment:
matrix:
- APPVEYOR_BUILD_WORKER_IMAGE: Ubuntu
APPVEYOR_JOB_NAME: "python37-x64-ubuntu"
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
APPVEYOR_JOB_NAME: "python37-x64-vs2015"
- APPVEYOR_BUILD_WORKER_IMAGE: macos-mojave
APPVEYOR_JOB_NAME: "python37-x64-macos-mojave"
stack: python 3.7
build: off
init:
- cmd: set PATH=C:\Python37;C:\Python37\Scripts;%PATH%
- ps: |
$BRANCH = if ($env:APPVEYOR_PULL_REQUEST_HEAD_REPO_BRANCH) { $env:APPVEYOR_PULL_REQUEST_HEAD_REPO_BRANCH } else { $env:APPVEYOR_REPO_BRANCH }
if (-not ($BRANCH -eq 'master' -or $BRANCH.ToLower().StartsWith('appveyor-'))) {
$env:PYTEST_ADDOPTS='-k "unit_test or test_0_basic" --suppress-no-test-exit-code'
}
install: python -m pip install -e ".[dev]" pytest-custom-exit-code
# the '-u' flag is required so the output is in the correct order.
# See https://github.com/pypa/cibuildwheel/pull/24 for more info.
test_script: python -u ./bin/run_tests.py
branches:
only:
- master
skip_commits:
files:
- docs/*
+37 -86
View File
@@ -1,91 +1,42 @@
pr:
paths:
exclude:
- .github/**
- bin/*
- docs/**
- examples/github-*
- examples/ci*
- examples/travis-ci-*
- .ci*
- .pre-commit-config.yaml
- .readthedocs.yml
- .travis.yml
- README.md
- mkdocs.yml
- noxfile.py
jobs:
- job: tests
strategy:
matrix:
linux_313:
imageName: "ubuntu-latest"
pythonVersion: "3.13"
testSelect: "native"
android_313:
imageName: "ubuntu-latest"
pythonVersion: "3.13"
testSelect: "android"
macos_313:
imageName: "macos-latest"
pythonVersion: "3.13"
testSelect: "native"
ios_313:
imageName: "macos-14" # See https://github.com/actions/runner-images/issues/12777
pythonVersion: "3.13"
testSelect: "ios"
android_macos_313:
imageName: "macos-latest"
pythonVersion: "3.13"
testSelect: "android"
windows_313:
imageName: "windows-latest"
pythonVersion: "3.13"
testSelect: "native"
timeoutInMinutes: 180
pool:
vmImage: $(imageName)
variables:
# Relocate cibuildwheel's download cache (interpreters) and the per-worker
# test pip caches (which live under it) to a stable, cacheable path.
CIBW_CACHE_PATH: $(Pipeline.Workspace)/.cibw-cache
- job: linux_38
pool: {vmImage: 'Ubuntu-18.04'}
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: $(pythonVersion)
- task: Cache@2
displayName: 'Cache cibuildwheel downloads'
inputs:
# build-platforms.toml carries the interpreter pins; bump the version
# prefix to force a cold cache. restoreKeys lets a stale cache still seed
# a new run (cibuildwheel re-downloads only what's missing).
key: 'cibw-cache | v1 | "$(imageName)" | "$(testSelect)" | cibuildwheel/resources/build-platforms.toml'
restoreKeys: |
cibw-cache | v1 | "$(imageName)" | "$(testSelect)"
path: $(CIBW_CACHE_PATH)
- task: JavaToolInstaller@0
condition: and(eq(variables['testSelect'], 'android'), eq(variables['Agent.OS'], 'Linux'))
inputs:
versionSpec: '17'
jdkArchitectureOption: 'x64'
jdkSourceOption: 'PreInstalled'
- bash: docker run --rm --privileged docker.io/tonistiigi/binfmt:latest --install all
condition: and(eq(variables['imageName'], 'ubuntu-latest'), eq(variables['testSelect'], 'native'))
displayName: 'Install binfmt on Linux'
- bash: python -m pip install -U pip && python -m pip install -e. --group test
displayName: 'Update pip and install cibuildwheel --group test'
- bash: echo "##vso[task.setvariable variable=CIBW_ENABLE;]all"
condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')
displayName: Set CIBW_ENABLE to all (main branch)
versionSpec: '3.8'
- bash: |
echo "CIBW_ENABLE = $CIBW_ENABLE"
python ./bin/run_tests.py --test-select $(testSelect)
displayName: 'Run tests'
python -m pip install -e ".[dev]" pytest-azurepipelines
python ./bin/run_tests.py
- job: macos_38
pool: {vmImage: 'macOS-10.15'}
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.8'
- bash: |
python -m pip install -e ".[dev]" pytest-azurepipelines
python ./bin/run_tests.py
- job: windows_36
pool: {vmImage: 'vs2017-win2016'}
timeoutInMinutes: 180
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.6'
- bash: |
python -m pip install -e ".[dev]" pytest-azurepipelines
python ./bin/run_tests.py
- job: windows_38
pool: {vmImage: 'vs2017-win2016'}
timeoutInMinutes: 180
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.8'
- bash: |
python -m pip install -e ".[dev]" pytest-azurepipelines
python ./bin/run_tests.py
+15 -46
View File
@@ -1,29 +1,30 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = ["click", "packaging", "prek"]
# ///
#!/usr/bin/env python3
from __future__ import annotations
import glob
import os
import subprocess
import sys
import tomllib
import urllib.parse
from pathlib import Path
import click
from packaging.version import InvalidVersion, Version
import cibuildwheel
config = [
# file path, version find/replace format
("pyproject.toml", 'version = "{}"'),
("README.md", "cibuildwheel=={}"),
("cibuildwheel/__init__.py", '__version__ = "{}"'),
("docs/faq.md", "cibuildwheel=={}"),
("docs/faq.md", "cibuildwheel@v{}"),
("docs/setup.md", "cibuildwheel=={}"),
("examples/*", "cibuildwheel=={}"),
("examples/*", "cibuildwheel@v{}"),
("setup.cfg", "version = {}"),
]
RED = "\u001b[31m"
@@ -33,8 +34,7 @@ OFF = "\u001b[0m"
@click.command()
def bump_version() -> None:
with open("pyproject.toml", "rb") as f:
current_version = tomllib.load(f)["project"]["version"]
current_version = cibuildwheel.__version__
try:
commit_date_str = subprocess.run(
@@ -60,7 +60,7 @@ def bump_version() -> None:
print(e)
print("Failed to get previous version tag information.")
git_changes_result = subprocess.run(["git diff-index --quiet HEAD --"], shell=True, check=False)
git_changes_result = subprocess.run(["git diff-index --quiet HEAD --"], shell=True)
repo_has_uncommitted_changes = git_changes_result.returncode != 0
if repo_has_uncommitted_changes:
@@ -68,8 +68,8 @@ def bump_version() -> None:
sys.exit(1)
# fmt: off
print( "Current version:", current_version)
new_version = input(" New version: ").strip()
print( 'Current version:', current_version) # noqa
new_version = input(' New version: ').strip()
# fmt: on
try:
@@ -82,7 +82,7 @@ def bump_version() -> None:
actions = []
for path_pattern, version_pattern in config:
paths = list(Path().glob(path_pattern))
paths = [Path(p) for p in glob.glob(path_pattern)]
if not paths:
print(f"error: Pattern {path_pattern} didn't match any files")
@@ -129,33 +129,13 @@ def bump_version() -> None:
contents = contents.replace(find, replace)
path.write_text(contents, encoding="utf8")
print("Files updated. If you want to update docs/changelog.md as part of")
print("this commit, do that now.")
print("Files updated. If you want to update the changelog as part of this")
print("commit, do that now.")
print()
while input('Type "done" to continue: ').strip().lower() != "done":
pass
# run pre-commit to update the README changelog
subprocess.run(
[
"prek",
"run",
"--files=docs/changelog.md",
],
check=False,
)
# run pre-commit to check that no errors occurred on the second run
subprocess.run(
[
"prek",
"run",
"--files=docs/changelog.md",
],
check=True,
)
subprocess.run(
[
"git",
@@ -178,17 +158,6 @@ def bump_version() -> None:
)
print("Done.")
print()
print("Push the new version to GitHub with:")
print(f" git push && git push origin v{new_version}")
print()
release_url = "https://github.com/pypa/cibuildwheel/releases/new?" + urllib.parse.urlencode(
{"tag": f"v{new_version}"}
)
print("Then create a release at the URL:")
print(f" {release_url}")
if __name__ == "__main__":
+3
View File
@@ -0,0 +1,3 @@
#!/bin/bash
CIBW_PLATFORM=linux pytest "$@"
-79
View File
@@ -1,79 +0,0 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "click",
# "cibuildwheel",
# ]
#
# [tool.uv.sources]
# cibuildwheel = { path = ".." }
# ///
import sys
import textwrap
from pathlib import Path
import click
from cibuildwheel.extra import get_pyodide_xbuildenv_info
@click.command()
@click.argument(
"pyodide-version",
type=str,
)
@click.option(
"--output-file",
type=click.Path(),
default=None,
help="Output file to write the constraints to. If not provided, the constraints will be printed to stdout.",
)
def generate_pyodide_constraints(pyodide_version: str, output_file: str | None = None) -> None:
"""
Generate constraints for a specific Pyodide version. The constraints are
generated based on the Pyodide version's xbuildenv info, which is retrieved
from the Pyodide repository.
These constraints should then be 'pinned' using `uv pip compile`.
Example usage:
bin/generate_pyodide_constraints.py 0.27.0
"""
xbuildenv_info = get_pyodide_xbuildenv_info()
try:
pyodide_version_xbuildenv_info = xbuildenv_info["releases"][pyodide_version]
except KeyError as e:
msg = f"Pyodide version {pyodide_version} not found in xbuildenv info. Versions available: {', '.join(xbuildenv_info['releases'].keys())}"
raise click.BadParameter(msg) from e
pyodide_build_min_version = pyodide_version_xbuildenv_info.get("min_pyodide_build_version")
pyodide_build_max_version = pyodide_version_xbuildenv_info.get("max_pyodide_build_version")
pyodide_build_specifier_parts: list[str] = []
if pyodide_build_min_version:
pyodide_build_specifier_parts.append(f">={pyodide_build_min_version}")
if pyodide_build_max_version:
pyodide_build_specifier_parts.append(f"<={pyodide_build_max_version}")
pyodide_build_specifier = ",".join(pyodide_build_specifier_parts)
constraints_txt = textwrap.dedent(f"""
pip
build[virtualenv]
pyodide-build{pyodide_build_specifier}
click<8.2
""")
if output_file is None:
print(constraints_txt)
else:
Path(output_file).write_text(constraints_txt)
print(f"Constraints written to {output_file}", file=sys.stderr)
if __name__ == "__main__":
generate_pyodide_constraints()
-405
View File
@@ -1,405 +0,0 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = ["pyyaml"]
# ///
import argparse
import copy
import json
from typing import Any
import yaml
parser = argparse.ArgumentParser(allow_abbrev=False)
parser.add_argument("--schemastore", action="store_true", help="Generate schema_store version")
args = parser.parse_args()
# The defaults in the schema are used by external tools for validation and IDE support. They
# should match the values in defaults.toml, which are used by cibuildwheel itself.
starter = """
$schema: http://json-schema.org/draft-07/schema#
$id: https://github.com/pypa/cibuildwheel/blob/main/cibuildwheel/resources/cibuildwheel.schema.json
$defs:
inherit:
enum:
- none
- prepend
- append
default: none
description: How to inherit the parent's value.
enable:
enum:
- cpython-prerelease
- graalpy
- pyodide-prerelease
- pypy
- pypy-eol
description: A Python version or flavor to enable.
additionalProperties: false
description: cibuildwheel's settings.
type: object
properties:
inherit:
type: object
additionalProperties: false
properties:
audit-command: {"$ref": "#/$defs/inherit"}
audit-requires: {"$ref": "#/$defs/inherit"}
before-all: {"$ref": "#/$defs/inherit"}
before-build: {"$ref": "#/$defs/inherit"}
xbuild-tools: {"$ref": "#/$defs/inherit"}
xbuild-files: {"$ref": "#/$defs/inherit"}
before-test: {"$ref": "#/$defs/inherit"}
config-settings: {"$ref": "#/$defs/inherit"}
container-engine: {"$ref": "#/$defs/inherit"}
environment: {"$ref": "#/$defs/inherit"}
environment-pass: {"$ref": "#/$defs/inherit"}
repair-wheel-command: {"$ref": "#/$defs/inherit"}
test-command: {"$ref": "#/$defs/inherit"}
test-extras: {"$ref": "#/$defs/inherit"}
test-sources: {"$ref": "#/$defs/inherit"}
test-requires: {"$ref": "#/$defs/inherit"}
test-environment: {"$ref": "#/$defs/inherit"}
test-runtime: {"$ref": "#/$defs/inherit"}
audit-command:
description: Execute a shell command to audit each wheel after it is repaired. Use {wheel} for each wheel path, or {abi3_wheel} to only audit abi3 wheels.
type: string_array
audit-requires:
description: Install Python dependencies for the audit step.
type: string_array
archs:
description: Change the architectures built on your machine by default.
type: string_array
before-all:
description: Execute a shell command on the build system before any wheels are built.
type: string_array
before-build:
description: Execute a shell command preparing each wheel's build.
type: string_array
before-test:
description: Execute a shell command before testing each wheel.
type: string_array
build:
default: ['*']
description: Choose the Python versions to build.
type: string_array
build-frontend:
default: default
description: Set the tool to use to build, either "build" (default), "build[uv]", "uv", or "pip"
oneOf:
- enum: [pip, build, "build[uv]", uv, default]
- type: string
pattern: '^pip; ?args:'
- type: string
pattern: '^build; ?args:'
- type: string
pattern: '^build\\[uv\\]; ?args:'
- type: string
pattern: '^uv; ?args:'
- type: object
additionalProperties: false
required: [name]
properties:
name:
enum: [pip, build, "build[uv]", uv]
args:
type: array
items:
type: string
build-verbosity:
type: integer
minimum: -3
maximum: 3
default: 0
description: Increase/decrease the output of pip wheel.
config-settings:
description: Specify config-settings for the build backend.
type: string_table_array
container-engine:
oneOf:
- enum: [docker, podman]
- type: string
pattern: '^docker; ?(create_args|disable_host_mount):'
- type: string
pattern: '^podman; ?(create_args|disable_host_mount):'
- type: object
additionalProperties: false
required: [name]
properties:
name:
enum: [docker, podman]
create-args:
type: array
items:
type: string
disable-host-mount:
type: boolean
dependency-versions:
default: pinned
description: Specify how cibuildwheel controls the versions of the tools it uses
oneOf:
- enum: [pinned, latest]
- type: string
description: Path to a file containing dependency versions, or inline package specifications, starting with "packages:"
not:
enum: [pinned, latest]
- type: object
additionalProperties: false
properties:
file:
type: string
- type: object
additionalProperties: false
properties:
packages:
type: array
items:
type: string
enable:
description: Enable or disable certain builds.
oneOf:
- $ref: "#/$defs/enable"
- type: array
items:
$ref: "#/$defs/enable"
environment:
description: Set environment variables needed during the build.
type: string_table
environment-pass:
description: Set environment variables on the host to pass-through to the container
during the build.
type: string_array
manylinux-aarch64-image:
type: string
description: Specify alternative manylinux / musllinux container images
manylinux-armv7l-image:
type: string
description: Specify alternative manylinux / musllinux container images
manylinux-i686-image:
type: string
description: Specify alternative manylinux / musllinux container images
manylinux-ppc64le-image:
type: string
description: Specify alternative manylinux / musllinux container images
manylinux-pypy_aarch64-image:
type: string
description: Specify alternative manylinux / musllinux container images
manylinux-pypy_i686-image:
type: string
description: Specify alternative manylinux / musllinux container images
manylinux-pypy_x86_64-image:
type: string
description: Specify alternative manylinux / musllinux container images
manylinux-riscv64-image:
type: string
description: Specify alternative manylinux / musllinux container images
manylinux-s390x-image:
type: string
description: Specify alternative manylinux / musllinux container images
manylinux-x86_64-image:
type: string
description: Specify alternative manylinux / musllinux container images
musllinux-aarch64-image:
type: string
description: Specify alternative manylinux / musllinux container images
musllinux-armv7l-image:
type: string
description: Specify alternative manylinux / musllinux container images
musllinux-i686-image:
type: string
description: Specify alternative manylinux / musllinux container images
musllinux-ppc64le-image:
type: string
description: Specify alternative manylinux / musllinux container images
musllinux-riscv64-image:
type: string
description: Specify alternative manylinux / musllinux container images
musllinux-s390x-image:
type: string
description: Specify alternative manylinux / musllinux container images
musllinux-x86_64-image:
type: string
description: Specify alternative manylinux / musllinux container images
xbuild-tools:
description: Binaries on the path that should be included in an isolated cross-build environment
type: string_array
xbuild-files:
description: Platform-specific files in the build environment
type: string_table_array
pyodide-version:
type: string
description: Specify the version of Pyodide to use
repair-wheel-command:
description: Execute a shell command to repair each built wheel.
type: string_array
skip:
description: Choose the Python versions to skip.
type: string_array
test-command:
description: Execute a shell command to test each built wheel.
type: string_array
test-extras:
description: Install your wheel for testing using `extras_require`
type: string_array
test-sources:
description: Test files that are required by the test environment
type: string_array
test-groups:
description: Install extra groups when testing
type: string_array
test-requires:
description: Install Python dependencies before running the tests
type: string_array
test-skip:
description: Skip running tests on some builds.
type: string_array
test-environment:
description: Set environment variables for the test environment
type: string_table
test-runtime:
description: Additional configuration for the test runner
oneOf:
- type: string
pattern: '^$'
- type: object
additionalProperties: false
- type: string
pattern: 'args:'
- type: object
additionalProperties: false
required: [args]
properties:
args:
type: array
items:
type: string
"""
schema = yaml.safe_load(starter)
string_array = yaml.safe_load(
"""
- type: string
- type: array
items:
type: string
"""
)
string_table_array = yaml.safe_load(
"""
- type: string
- type: object
additionalProperties: false
patternProperties:
.+:
oneOf:
- type: string
- type: array
items:
type: string
"""
)
string_table = yaml.safe_load(
"""
- type: string
- type: object
additionalProperties: false
patternProperties:
.+:
type: string
"""
)
for value in schema["properties"].values():
match value:
case {"type": "string_array"}:
del value["type"]
value["oneOf"] = string_array
case {"type": "string_table"}:
del value["type"]
value["oneOf"] = string_table
case {"type": "string_table_array"}:
del value["type"]
value["oneOf"] = string_table_array
overrides = yaml.safe_load(
"""
type: array
description: An overrides array
items:
type: object
required: ["select"]
minProperties: 2
additionalProperties: false
properties:
select: {}
"""
)
for key, value in schema["properties"].items():
value["title"] = f"CIBW_{key.replace('-', '_').upper()}"
non_global_options = {k: {"$ref": f"#/properties/{k}"} for k in schema["properties"]}
del non_global_options["build"]
del non_global_options["skip"]
del non_global_options["test-skip"]
del non_global_options["enable"]
overrides["items"]["properties"]["select"]["oneOf"] = string_array
overrides["items"]["properties"] |= non_global_options.copy()
del overrides["items"]["properties"]["archs"]
not_linux = non_global_options.copy()
del not_linux["environment-pass"]
del not_linux["container-engine"]
for key in list(not_linux):
if "linux-" in key:
del not_linux[key]
def as_object(d: dict[str, Any]) -> dict[str, Any]:
return {
"type": "object",
"additionalProperties": False,
"properties": copy.deepcopy(d),
}
oses = {
"linux": as_object(non_global_options),
"windows": as_object(not_linux),
"macos": as_object(not_linux),
"pyodide": as_object(not_linux),
"android": as_object(not_linux),
"ios": as_object(not_linux),
}
for os_name, command in [
("linux", "auditwheel repair -w {dest_dir} {wheel}"),
("windows", "delvewheel repair -w {dest_dir} -v {wheel}"),
("macos", "delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}"),
("android", "auditwheel repair --ldpaths {ldpaths} -w {dest_dir} {wheel}"),
]:
oses[os_name]["properties"]["repair-wheel-command"] = {
**schema["properties"]["repair-wheel-command"],
"default": command,
}
del oses["linux"]["properties"]["dependency-versions"]
schema["properties"]["overrides"] = overrides
schema["properties"] |= oses
if args.schemastore:
schema["$id"] = "https://json.schemastore.org/partial-cibuildwheel.json"
schema["description"] = (
"cibuildwheel's settings. Generated with ./bin/generate_schema.py --schemastore from cibuildwheel."
)
print(json.dumps(schema, indent=2))
+18 -51
View File
@@ -1,45 +1,17 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "click",
# "pygithub",
# "pyyaml",
# "rich",
# "cibuildwheel",
# ]
#
# [tool.uv.sources]
# cibuildwheel = { path = ".." }
# ///
"""
Check known projects for usage of requires-python.
Usage:
./bin/inspect_all_known_projects.py --online=$GITHUB_TOKEN
This will cache the results to all_known_setup.yaml; you can reprint
the results without the `--online` setting.
"""
#!/usr/bin/env python3
from __future__ import annotations
import ast
from pathlib import Path
from typing import Iterator
import click
import yaml
from github import Github, GithubException
from ghapi.core import GhApi, HTTP404NotFoundError # type: ignore
from rich import print
from cibuildwheel.projectfiles import Analyzer
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Iterable, Iterator
DIR = Path(__file__).parent.resolve()
@@ -75,44 +47,39 @@ def check_repo(name: str, contents: str) -> str:
class MaybeRemote:
github: Github | None
contents: dict[str, dict[str, str | None]]
def __init__(self, cached_file: Path | str, *, online: str | None) -> None:
if online is not None:
self.github = Github(online)
self.contents = {
def __init__(self, cached_file: Path | str, *, online: bool) -> None:
self.online = online
if self.online:
self.contents: dict[str, dict[str, str | None]] = {
"setup.py": {},
"setup.cfg": {},
"pyproject.toml": {},
}
else:
self.github = None
with open(cached_file) as f:
self.contents = yaml.safe_load(f)
def get(self, repo: str, filename: str) -> str | None:
if self.github:
if self.online:
try:
gh_file = self.github.get_repo(repo).get_contents(filename)
except GithubException:
self.contents[filename][repo] = (
GhApi(*repo.split("/")).get_content(filename).decode()
)
except HTTP404NotFoundError:
self.contents[filename][repo] = None
else:
assert not isinstance(gh_file, list)
self.contents[filename][repo] = gh_file.decoded_content.decode(encoding="utf-8")
return self.contents[filename][repo]
elif repo in self.contents[filename]:
return self.contents[filename][repo]
else:
msg = f"Trying to access {repo}:{filename} and not in cache, rebuild cache"
raise RuntimeError(msg)
raise RuntimeError(
f"Trying to access {repo}:{filename} and not in cache, rebuild cache"
)
def save(self, filename: Path | str) -> None:
with open(filename, "w") as f:
yaml.safe_dump(self.contents, f, default_flow_style=False)
def on_each(self, repos: Iterable[str]) -> Iterator[tuple[str, str, str | None]]:
def on_each(self, repos: list[str]) -> Iterator[tuple[str, str, str | None]]:
for repo in repos:
print(f"[bold]{repo}:")
for filename in sorted(self.contents, reverse=True):
@@ -120,8 +87,8 @@ class MaybeRemote:
@click.command()
@click.option("--online", help="Set to $GITHUB_TOKEN")
def main(online: str | None) -> None:
@click.option("--online", is_flag=True, help="Remember to set GITHUB_TOKEN")
def main(online: bool) -> None:
with open(DIR / "../docs/data/projects.yml") as f:
known = yaml.safe_load(f)
+14 -21
View File
@@ -1,36 +1,29 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "click",
# ]
# ///
#!/usr/bin/env python3
from __future__ import annotations
import os
import subprocess
import sys
import textwrap
import time
from pathlib import Path
from subprocess import run
import click
def shell(cmd: str, *, check: bool, **kwargs: object) -> subprocess.CompletedProcess[str]:
return subprocess.run([cmd], shell=True, check=check, **kwargs) # type: ignore[call-overload, no-any-return]
def shell(cmd, **kwargs):
return run([cmd], shell=True, **kwargs)
def git_repo_has_changes() -> bool:
unstaged_changes: bool = shell("git diff-index --quiet HEAD --", check=False).returncode != 0
staged_changes: bool = (
shell("git diff-index --quiet --cached HEAD --", check=False).returncode != 0
)
def git_repo_has_changes():
unstaged_changes = shell("git diff-index --quiet HEAD --").returncode != 0
staged_changes = shell("git diff-index --quiet --cached HEAD --").returncode != 0
return unstaged_changes or staged_changes
@click.command()
def main() -> None:
def main():
project_root = Path(__file__).parent / ".."
os.chdir(project_root)
@@ -47,7 +40,7 @@ def main() -> None:
timestamp = time.strftime("%Y-%m-%dT%H-%M-%S", time.gmtime())
branch_name = f"update-constraints-{timestamp}"
shell(f"git checkout -b {branch_name} origin/main", check=True)
shell(f"git checkout -b {branch_name} origin/master", check=True)
try:
shell("bin/update_dependencies.py", check=True)
@@ -61,16 +54,16 @@ def main() -> None:
f"""
Update the versions of our dependencies.
PR generated by `{Path(__file__).name}`.
PR generated by `{os.path.basename(__file__)}`.
"""
)
subprocess.run(
run(
[
"gh",
"pr",
"create",
"--repo=pypa/cibuildwheel",
"--base=main",
"--base=master",
"--title=Update dependencies",
f"--body='{body}'",
],
@@ -80,7 +73,7 @@ def main() -> None:
print("Done.")
finally:
# remove any local changes
shell("git checkout -- .", check=True)
shell("git checkout -- .")
shell(f"git checkout {previous_branch}", check=True)
shell(f"git branch -D --force {branch_name}", check=True)
+46 -107
View File
@@ -1,19 +1,11 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "click",
# "pyyaml",
# "pygithub",
# ]
# ///
#!/usr/bin/env python3
"""
Convert a yaml project list into a nice table.
Suggested usage:
./bin/projects.py docs/data/projects.yml --online --auth $GITHUB_API_TOKEN
./bin/projects.py docs/data/projects.yml --online --auth $GITHUB_API_TOKEN --readme README.md
git diff
"""
@@ -22,23 +14,19 @@ from __future__ import annotations
import builtins
import functools
import textwrap
import urllib.error
import urllib.request
import xml.dom.minidom
from datetime import UTC, datetime
from datetime import datetime
from io import StringIO
from pathlib import Path
from typing import Any, Self, TextIO
from typing import Any, TextIO
import click
import yaml
from github import Auth, Github, GithubException
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Iterable, Mapping, Sequence
from github import Github, GithubException
ICONS = (
"appveyor",
"github",
"azurepipelines",
"circleci",
@@ -47,16 +35,13 @@ ICONS = (
"windows",
"apple",
"linux",
"android",
"ios",
"pyodide",
)
class Project:
NAME: int = 0
def __init__(self, config: Mapping[str, Any], github: Github | None = None):
def __init__(self, config: dict[str, Any], github: Github | None = None):
try:
self.name: str = config["name"]
self.gh: str = config["gh"]
@@ -77,7 +62,7 @@ class Project:
print(f"Broken: {self.stars_repo}")
raise
self.num_stars: int = repo.stargazers_count
self.num_stars = repo.stargazers_count
self.pushed_at = repo.pushed_at
if not self.notes:
notes = repo.description
@@ -85,12 +70,12 @@ class Project:
self.notes = notes
else:
self.num_stars = 0
self.pushed_at = datetime.now(UTC)
self.pushed_at = datetime.utcnow()
name_len = len(self.name) + 4
self.__class__.NAME = max(self.__class__.NAME, name_len)
def __lt__(self, other: Self) -> bool:
def __lt__(self, other: Project) -> bool:
if self.online:
return self.num_stars < other.num_stars
else:
@@ -100,8 +85,8 @@ class Project:
def header(cls) -> str:
return textwrap.dedent(
f"""\
| {"Name":{cls.NAME}} | CI | OS | Notes |
|{"":-^{cls.NAME + 2}}|----|----|:------|"""
| {'Name':{cls.NAME}} | CI | OS | Notes |
|{'':-^{cls.NAME+2 }}|----|----|:------|"""
)
@property
@@ -132,23 +117,17 @@ class Project:
return f"[{self.name}]: {self.url}"
def info(self) -> str:
days = (datetime.now(UTC) - self.pushed_at).days
days = (datetime.utcnow() - self.pushed_at).days
return f"<!-- {self.name}: {self.num_stars}, last pushed {days} days ago -->"
def fetch_icon(icon_name: str) -> None:
url = f"https://cdn.jsdelivr.net/npm/simple-icons@v4/icons/{icon_name}.svg"
try:
with urllib.request.urlopen(url) as f:
original_svg_data = f.read()
except urllib.error.HTTPError as e:
if e.code == 404 and path_for_icon(icon_name).exists():
return
raise
with urllib.request.urlopen(url) as f:
original_svg_data = f.read()
document = xml.dom.minidom.parseString(original_svg_data)
svgElement = document.documentElement
assert svgElement is not None
assert svgElement.nodeName == "svg"
svgElement.setAttribute("width", "16px")
svgElement.setAttribute("fill", "#606060")
@@ -160,37 +139,28 @@ def fetch_icon(icon_name: str) -> None:
f.write(svgElement.toxml())
def path_for_icon(icon_name: str, relative_to: Path | None = None) -> Path:
from_path = Path.cwd() if relative_to is None else relative_to.parent
absolute_path = PROJECT_ROOT / "docs" / "data" / "readme_icons" / f"{icon_name}.svg"
return absolute_path.resolve().relative_to(from_path.resolve())
def path_for_icon(icon_name: str) -> Path:
return Path(".") / "docs" / "data" / "readme_icons" / f"{icon_name}.svg"
def get_projects(
config: Iterable[Mapping[str, Any]],
def str_projects(
config: list[dict[str, Any]],
*,
online: bool = True,
auth: str | None = None,
) -> list[Project]:
if online:
for icon in ICONS:
fetch_icon(icon)
github = Github(auth=Auth.Token(auth)) if (online and auth) else None
return sorted((Project(item, github) for item in config), reverse=online)
def render_projects(
projects: Sequence[Project], *, dest_path: Path, include_info: bool = True
) -> str:
io = StringIO()
print = functools.partial(builtins.print, file=io)
print(Project.header())
if online:
for icon in ICONS:
fetch_icon(icon)
github = Github(auth) if online else None
projects = sorted((Project(item, github) for item in config), reverse=online)
print(Project.header())
for project in projects:
print(project.table_row())
@@ -200,73 +170,42 @@ def render_projects(
print()
for icon in ICONS:
print(f"[{icon} icon]: {path_for_icon(icon, relative_to=dest_path).as_posix()}")
print(f"[{icon} icon]: {path_for_icon(icon).as_posix()}")
if include_info:
print()
for project in projects:
print(project.info())
print()
for project in projects:
print(project.info())
return io.getvalue()
def insert_projects_table(
file: Path,
*,
projects: Sequence[Project],
input_filename: str,
include_info: bool = True,
) -> None:
text = file.read_text()
projects_table = render_projects(projects, include_info=include_info, dest_path=file)
start_str = "<!-- START bin/projects.py -->\n"
start = text.find(start_str)
assert start != -1
end = text.find("<!-- END bin/projects.py -->\n")
assert end != -1
generated_note = f"<!-- this section is generated by bin/projects.py. Don't edit it directly, instead, edit {input_filename} -->"
new_text = (
f"{text[: start + len(start_str)]}\n{generated_note}\n\n{projects_table}\n{text[end:]}"
)
file.write_text(new_text)
PROJECT_ROOT = Path(__file__).parent / ".."
README_FILE = PROJECT_ROOT / "README.md"
DOCS_PAGE = PROJECT_ROOT / "docs" / "working-examples.md"
@click.command(help="Try ./bin/projects.py docs/data/projects.yml")
@click.command(help="Try ./bin/projects.py docs/data/projects.yml --readme README.md")
@click.argument("input", type=click.File("r"))
@click.option("--online/--no-online", default=True, help="Get info from GitHub")
@click.option("--auth", help="GitHub authentication token")
@click.option(
"--dry-run", default=False, help="Print the output, rather than writing it to files in the repo"
)
@click.option("--readme", type=click.File("r+"), help="Modify a readme file if given")
def projects(
input: TextIO,
online: bool,
auth: str | None,
dry_run: bool,
readme: TextIO | None,
) -> None:
config = yaml.safe_load(input)
projects = get_projects(config, online=online, auth=auth)
output = str_projects(config, online=online, auth=auth)
if dry_run:
output = render_projects(projects, dest_path=README_FILE)
if readme is None:
print(output)
else:
insert_projects_table(
README_FILE, projects=projects[:10], input_filename=input.name, include_info=False
)
insert_projects_table(
DOCS_PAGE, projects=projects, input_filename=input.name, include_info=False
)
text = readme.read()
start_str = "<!-- START bin/projects.py -->\n"
start = text.find(start_str)
end = text.find("<!-- END bin/projects.py -->\n")
generated_note = f"<!-- this section is generated by bin/projects.py. Don't edit it directly, instead, edit {input.name} -->"
new_text = f"{text[:start + len(start_str)]}\n{generated_note}\n\n{output}\n{text[end:]}"
readme.seek(0)
readme.write(new_text)
readme.truncate()
if __name__ == "__main__":
-23
View File
@@ -1,23 +0,0 @@
#!/usr/bin/env -S uv run --script
import re
from pathlib import Path
PROJECT_ROOT = Path(__file__).parent / ".."
CHANGELOG_FILE = PROJECT_ROOT / "docs" / "changelog.md"
# https://regexr.com/622ds
FIRST_5_CHANGELOG_ENTRIES_REGEX = re.compile(r"""(^###.*?(?=###)){5}""", re.DOTALL | re.MULTILINE)
def mini_changelog() -> str:
changelog_text = CHANGELOG_FILE.read_text()
mini_changelog_match = FIRST_5_CHANGELOG_ENTRIES_REGEX.search(changelog_text)
assert mini_changelog_match, "Failed to find the first few changelog entries"
return f"\n{mini_changelog_match.group(0).strip()}\n"
if __name__ == "__main__":
print(mini_changelog())
-68
View File
@@ -1,68 +0,0 @@
#!/usr/bin/env -S uv run --script
import dataclasses
import re
from pathlib import Path
from typing import Final
DIR: Final[Path] = Path(__file__).parent.parent.resolve()
OPTIONS_MD: Final[Path] = DIR / "docs" / "options.md"
SECTION_HEADER_REGEX = re.compile(r"^## (?P<name>.*?)$", re.MULTILINE)
# https://regexr.com/8f1ff
OPTION_HEADER_REGEX = re.compile(
r"^### (?P<name>.*?){.*#(?P<id>\S+).*}\n+> ?(?P<desc>.*)$", re.MULTILINE
)
@dataclasses.dataclass(kw_only=True)
class Option:
name: str
id: str
desc: str
section: str
def get_table() -> str:
options_md = OPTIONS_MD.read_text(encoding="utf-8")
sections = SECTION_HEADER_REGEX.split(options_md)[1:]
options = []
for section_name, section_content in zip(sections[0::2], sections[1::2], strict=True):
for match in OPTION_HEADER_REGEX.finditer(section_content):
option = Option(
name=match.group("name").strip(),
id=match.group("id").strip(),
desc=match.group("desc").strip(),
section=section_name.strip(),
)
options.append(option)
table_md = "\n<!-- This table is auto-generated from docs/options.md by bin/readme_options_table.py -->\n\n"
table_md += "| | Option | Description |\n"
table_md += "|---|---|---|\n"
last_section: str | None = None
for option in options:
cells: list[str] = []
cells.append(f"**{option.section}**" if option.section != last_section else "")
last_section = option.section
url = f"https://cibuildwheel.pypa.io/en/stable/options/#{option.id}"
name = option.name.replace(", ", "<br>") # Replace commas with line breaks
cells.append(f"[{name}]({url})")
cells.append(option.desc)
table_md += "| " + " | ".join(cells) + " |\n"
table_md += "\n"
return table_md
if __name__ == "__main__":
print(get_table())
+50 -118
View File
@@ -1,108 +1,50 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "click",
# "jinja2",
# ]
# ///
#!/usr/bin/env python3
from __future__ import annotations
import os
import re
import shutil
import subprocess
import sys
import textwrap
import time
import typing
from collections import namedtuple
from glob import glob
from pathlib import Path
from subprocess import run
from urllib.parse import quote
import click
DIR = Path(__file__).parent.parent.resolve()
BuildBackend = typing.Literal["setuptools", "meson"]
def shell(cmd, **kwargs):
return run([cmd], shell=True, **kwargs)
def shell(cmd: str, *, check: bool, **kwargs: object) -> subprocess.CompletedProcess[str]:
return subprocess.run([cmd], shell=True, check=check, **kwargs) # type: ignore[call-overload, no-any-return]
def git_repo_has_changes() -> bool:
unstaged_changes = shell("git diff-index --quiet HEAD --", check=False).returncode != 0
staged_changes = shell("git diff-index --quiet --cached HEAD --", check=False).returncode != 0
def git_repo_has_changes():
unstaged_changes = shell("git diff-index --quiet HEAD --").returncode != 0
staged_changes = shell("git diff-index --quiet --cached HEAD --").returncode != 0
return unstaged_changes or staged_changes
def generate_project(path: Path, build_backend: BuildBackend) -> None:
def generate_basic_project(path):
sys.path.insert(0, "")
match build_backend:
case "meson":
from test.test_projects.meson import new_meson_project # noqa: PLC0415
project = new_meson_project()
case "setuptools":
from test.test_projects.setuptools import new_c_project # noqa: PLC0415
project = new_c_project()
case _:
typing.assert_never(build_backend)
from test.test_projects.c import new_c_project
project = new_c_project()
project.generate(path)
class CIService(typing.NamedTuple):
name: str
dst_config_path: str
badge_md: str
config_file_transform: typing.Callable[[str, str], str] = lambda x, _: x # identity by default
def github_config_file_transform(content: str, git_ref: str) -> str:
# one of the the github configs only builds on main, so we need to remove that restriction
# so our example build will run on the test branch.
#
# replace:
# """
# push:
# branches:
# - main
# """
# with:
# """
# push:
# """"
content = re.sub(
r"push:\n\s+branches:\n\s+- main",
"push:",
content,
)
# use the version of cibuildwheel from the current commit, not the latest
# release
# replace:
# """
# uses: pypa/cibuildwheel@v3.3.1
# """
# with:
# """
# uses: pypa/cibuildwheel@<latest commit hash>
# """
content = re.sub(
r"uses: pypa/cibuildwheel@v.*",
f"uses: pypa/cibuildwheel@{git_ref}",
content,
)
return content
CIService = namedtuple("CIService", "name dst_config_path badge_md")
services = [
CIService(
name="appveyor",
dst_config_path="appveyor.yml",
badge_md="[![Build status](https://ci.appveyor.com/api/projects/status/wbsgxshp05tt1tif/branch/{branch}?svg=true)](https://ci.appveyor.com/project/pypa/cibuildwheel/branch/{branch})",
),
CIService(
name="azure-pipelines",
dst_config_path="azure-pipelines.yml",
badge_md="[![Build Status](https://dev.azure.com/joerick0429/cibuildwheel/_apis/build/status/pypa.cibuildwheel?branchName={branch})](https://dev.azure.com/joerick0429/cibuildwheel/_build/latest?definitionId=2&branchName={branch})",
badge_md="[![Build Status](https://dev.azure.com/joerick0429/cibuildwheel/_apis/build/status/joerick.cibuildwheel?branchName={branch})](https://dev.azure.com/joerick0429/cibuildwheel/_build/latest?definitionId=2&branchName={branch})",
),
CIService(
name="circleci",
@@ -112,55 +54,49 @@ services = [
CIService(
name="github",
dst_config_path=".github/workflows/example.yml",
badge_md="[![Build](https://github.com/pypa/cibuildwheel/actions/workflows/example.yml/badge.svg?branch={branch})](https://github.com/pypa/cibuildwheel/actions?query=branch%3A{branch})",
config_file_transform=github_config_file_transform,
badge_md="[![Build](https://github.com/pypa/cibuildwheel/workflows/Build/badge.svg?branch={branch})](https://github.com/pypa/cibuildwheel/actions)",
),
CIService(
name="travis-ci",
dst_config_path=".travis.yml",
badge_md="[![Build Status](https://app.travis-ci.com/pypa/cibuildwheel.svg?branch={branch})](https://app.travis-ci.com/pypa/cibuildwheel)",
badge_md="[![Build Status](https://travis-ci.org/pypa/cibuildwheel.svg?branch={branch})](https://travis-ci.org/pypa/cibuildwheel)",
),
CIService(
name="gitlab",
dst_config_path=".gitlab-ci.yml",
badge_md="[![Gitlab](https://gitlab.com/joerick/cibuildwheel/badges/{branch}/pipeline.svg)](https://gitlab.com/joerick/cibuildwheel/-/commits/{branch})",
badge_md="[![Gitlab](https://gitlab.com/pypa/cibuildwheel/badges/{branch}/pipeline.svg)](https://gitlab.com/pypa/cibuildwheel/-/commits/{branch})",
),
]
def ci_service_for_config_file(config_file: Path) -> CIService:
filename = config_file.name
try:
return next(s for s in services if filename.startswith(s.name))
except StopIteration:
msg = f"unknown ci service for config file {config_file}"
raise ValueError(msg) from None
def ci_service_for_config_file(config_file):
service_name = Path(config_file).name.rsplit("-", 1)[0]
for service in services:
if service.name == service_name:
return service
raise ValueError(f"unknown ci service for config file {config_file}")
@click.command()
@click.argument("config_files", nargs=-1, type=click.Path())
@click.option("--build-backend", type=click.Choice(["setuptools", "meson"]), default="setuptools")
def run_example_ci_configs(
config_files: list[str], build_backend: BuildBackend = "setuptools"
) -> None:
def run_example_ci_configs(config_files=None):
"""
Test the example configs. If no files are specified, will test
examples/*-minimal.yml
"""
if len(config_files) == 0:
config_file_paths = list(Path("examples").glob("*-minimal.yml"))
else:
config_file_paths = [Path(f) for f in config_files]
config_files = glob("examples/*-minimal.yml")
# check each CI service has at most 1 config file
configs_by_service = set()
for config_file in config_file_paths:
configs_by_service = {}
for config_file in config_files:
service = ci_service_for_config_file(config_file)
if service.name in configs_by_service:
msg = "You cannot specify more than one config per CI service"
raise Exception(msg)
configs_by_service.add(service.name)
raise Exception("You cannot specify more than one config per CI service")
configs_by_service[service.name] = config_file
if git_repo_has_changes():
print("Your git repo has uncommitted changes. Commit or stash before continuing.")
@@ -169,9 +105,6 @@ def run_example_ci_configs(
previous_branch = shell(
"git rev-parse --abbrev-ref HEAD", check=True, capture_output=True, encoding="utf8"
).stdout.strip()
git_ref = shell(
"git rev-parse HEAD", check=True, capture_output=True, encoding="utf8"
).stdout.strip()
timestamp = time.strftime("%Y-%m-%dT%H-%M-%S", time.gmtime())
branch_name = f"example-config-test---{previous_branch}-{timestamp}"
@@ -180,29 +113,27 @@ def run_example_ci_configs(
shell(f"git checkout --orphan {branch_name}", check=True)
example_project = Path("example_root")
generate_project(example_project, build_backend=build_backend)
generate_basic_project(example_project)
for config_file in config_file_paths:
for config_file in config_files:
service = ci_service_for_config_file(config_file)
src_config_file = Path(config_file)
dst_config_file = example_project / service.dst_config_path
dst_config_file.parent.mkdir(parents=True, exist_ok=True)
shutil.copyfile(src_config_file, dst_config_file)
contents = config_file.read_text(encoding="utf8")
contents = service.config_file_transform(contents, git_ref)
dst_config_file.write_text(contents, encoding="utf8")
subprocess.run(["git", "add", example_project], check=True)
run(["git", "add", example_project], check=True)
message = textwrap.dedent(
f"""\
Test example CI configs
Test example minimal configs
Testing files: {[str(f) for f in config_files]}
Testing files: {config_files}
Generated from branch: {previous_branch}
Time: {timestamp}
"""
)
subprocess.run(["git", "commit", "--no-verify", "--message", message], check=True)
run(["git", "commit", "--no-verify", "--message", message], check=True)
shell(f"git subtree --prefix={example_project} push origin {branch_name}", check=True)
print("---")
@@ -213,24 +144,25 @@ def run_example_ci_configs(
print("> ")
print("> | Service | Config | Status |")
print("> |---|---|---|")
for config_file in config_file_paths:
for config_file in config_files:
service = ci_service_for_config_file(config_file)
badge = service.badge_md.format(
branch=branch_name, branch_escaped=quote(branch_name, safe="")
)
print(f"> | {service.name} | `{config_file}` | {badge} |")
print("> ")
print(f"> Generated by `{' '.join(sys.argv)}`")
print("> Generated by `bin/run_example_ci_config.py`")
print()
print("---")
finally:
# remove any local changes
shutil.rmtree(example_project, ignore_errors=True)
shell("git checkout -- .", check=True)
shell("git checkout -- .")
shell(f"git checkout {previous_branch}", check=True)
shell(f"git branch -D --force {branch_name}", check=True)
if __name__ == "__main__":
os.chdir(DIR)
os.chdir(os.path.dirname(__file__))
os.chdir("..")
run_example_ci_configs(standalone_mode=True)
+8 -111
View File
@@ -1,126 +1,23 @@
#!/usr/bin/env -S uv run --script
#!/usr/bin/env python3
import argparse
import os
import subprocess
import sys
from pathlib import Path
if __name__ == "__main__":
if sys.version_info < (3, 13):
default_cpu_count = os.cpu_count() or 2
else:
default_cpu_count = os.process_cpu_count() or 2
parser = argparse.ArgumentParser(allow_abbrev=False)
parser.add_argument(
"--run-podman", action="store_true", default=False, help="run podman tests (linux only)"
)
parser.add_argument(
"--num-processes",
action="store",
default=default_cpu_count,
help="number of processes to use for testing",
)
parser.add_argument(
"--test-select",
choices={"all", "native", "android", "ios", "pyodide"},
default="all",
help="Either 'native' or 'android'/'ios'/'pyodide'",
)
args = parser.parse_args()
# move cwd to the project root
os.chdir(Path(__file__).resolve().parents[1])
# doc tests
doc_test_args = [sys.executable, "-m", "pytest", "cibuildwheel"]
print(
"\n\n================================== DOC TESTS ==================================",
flush=True,
)
result = subprocess.run(doc_test_args, check=False)
if result.returncode not in (0, 5):
# Allow case where no doctests are collected (returncode 5) because
# circleci sets an explicit "-k" filter that disables doctests. There
# isn't a pattern that will only select doctests. This can be removed
# and have check=True if the circleci PYTEST_ADDOPTS is removed.
raise subprocess.CalledProcessError(
result.returncode, result.args, output=result.stdout, stderr=result.stderr
)
# unit tests
print(
"\n\n================================== UNIT TESTS ==================================",
flush=True,
)
# run the unit tests
unit_test_args = [sys.executable, "-m", "pytest", "unit_test"]
if (
sys.platform.startswith("linux")
and os.environ.get("CIBW_PLATFORM", "linux") == "linux"
and args.test_select in ["all", "native"]
):
# run the docker unit tests only on Linux
# run the docker unit tests only on Linux
if sys.platform.startswith("linux"):
unit_test_args += ["--run-docker"]
if args.run_podman:
unit_test_args += ["--run-podman"]
subprocess.run(unit_test_args, check=True)
print(
"\n\n=========================== SERIAL INTEGRATION TESTS ===========================",
flush=True,
# run the integration tests
subprocess.run(
[sys.executable, "-m", "pytest", "-x", "--durations", "0", "--timeout=2400", "test"],
check=True,
)
match args.test_select:
case "all":
marks = []
case "native":
marks = ["not pyodide", "not android", "not ios"]
case mark:
marks = [f"{mark}"]
# Run the serial integration tests without multiple processes
serial_integration_test_args = [
sys.executable,
"-m",
"pytest",
"-m",
f"{' and '.join(['serial', *marks])}",
"-x",
"--durations",
"0",
"--timeout=2400",
"test",
"-vv",
]
subprocess.run(serial_integration_test_args, check=True)
print(
"\n\n========================= NON-SERIAL INTEGRATION TESTS =========================",
flush=True,
)
integration_test_args = [
sys.executable,
"-m",
"pytest",
"-m",
f"{' and '.join(['not serial', *marks])}",
f"--numprocesses={args.num_processes}",
"-x",
"--durations",
"0",
"--timeout=2400",
"test",
"-vv",
]
if sys.platform.startswith("linux") and args.run_podman:
integration_test_args += ["--run-podman"]
subprocess.run(integration_test_args, check=True)
+4 -7
View File
@@ -1,5 +1,6 @@
#!/usr/bin/env -S uv run --script
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import os
@@ -12,7 +13,7 @@ if __name__ == "__main__":
# move cwd to the project root
os.chdir(Path(__file__).resolve().parents[1])
parser = argparse.ArgumentParser(description="Runs a sample build", allow_abbrev=False)
parser = argparse.ArgumentParser(description="Runs a sample build")
parser.add_argument("project_python_path", nargs="?", default="test.test_0_basic.basic_project")
options = parser.parse_args()
@@ -23,8 +24,4 @@ if __name__ == "__main__":
check=True,
)
sys.exit(
subprocess.run(
[sys.executable, "-m", "cibuildwheel"], cwd=project_dir, check=False
).returncode
)
sys.exit(subprocess.run([sys.executable, "-m", "cibuildwheel"], cwd=project_dir).returncode)
+142
View File
@@ -0,0 +1,142 @@
#!/usr/bin/env python3
from __future__ import annotations
import configparser
import os
import shutil
import subprocess
import sys
from typing import NamedTuple
import requests
os.chdir(os.path.dirname(__file__))
os.chdir("..")
# CUSTOM_COMPILE_COMMAND is a pip-compile option that tells users how to
# regenerate the constraints files
os.environ["CUSTOM_COMPILE_COMMAND"] = "bin/update_dependencies.py"
PYTHON_VERSIONS = ["27", "35", "36", "37", "38", "39"]
if "--no-docker" in sys.argv:
for python_version in PYTHON_VERSIONS:
subprocess.run(
[
f"./env{python_version}/bin/pip-compile",
"--allow-unsafe",
"--upgrade",
"cibuildwheel/resources/constraints.in",
f"--output-file=cibuildwheel/resources/constraints-python{python_version}.txt",
],
check=True,
)
else:
# latest manylinux2010 image with cpython 2.7 support
image_runner = "quay.io/pypa/manylinux2010_x86_64:2021-02-06-3d322a5"
subprocess.run(["docker", "pull", image_runner], check=True)
for python_version in PYTHON_VERSIONS:
abi_flags = "" if int(python_version) >= 38 else "m"
python_path = f"/opt/python/cp{python_version}-cp{python_version}{abi_flags}/bin/"
command = (
f"{python_path}pip install pip-tools && "
f"{python_path}pip-compile --allow-unsafe --upgrade "
"cibuildwheel/resources/constraints.in "
f"--output-file cibuildwheel/resources/constraints-python{python_version}.txt"
)
subprocess.run(
[
"docker",
"run",
"--rm",
"--env=CUSTOM_COMPILE_COMMAND",
f"--volume={os.getcwd()}:/volume",
"--workdir=/volume",
image_runner,
"bash",
"-c",
command,
],
check=True,
)
# default constraints.txt
shutil.copyfile(
f"cibuildwheel/resources/constraints-python{PYTHON_VERSIONS[-1]}.txt",
"cibuildwheel/resources/constraints.txt",
)
class Image(NamedTuple):
manylinux_version: str
platform: str
image_name: str
tag: str | None
images = [
Image("manylinux1", "x86_64", "quay.io/pypa/manylinux1_x86_64", None),
Image("manylinux1", "i686", "quay.io/pypa/manylinux1_i686", None),
# Images for manylinux2010 are pinned to the latest tag supporting cp27
Image("manylinux2010", "x86_64", "quay.io/pypa/manylinux2010_x86_64", "2021-02-06-3d322a5"),
Image("manylinux2010", "i686", "quay.io/pypa/manylinux2010_i686", "2021-02-06-3d322a5"),
Image("manylinux2010", "pypy_x86_64", "pypywheels/manylinux2010-pypy_x86_64", None),
# 2014 images
Image("manylinux2014", "x86_64", "quay.io/pypa/manylinux2014_x86_64", None),
Image("manylinux2014", "i686", "quay.io/pypa/manylinux2014_i686", None),
Image("manylinux2014", "aarch64", "quay.io/pypa/manylinux2014_aarch64", None),
Image("manylinux2014", "ppc64le", "quay.io/pypa/manylinux2014_ppc64le", None),
Image("manylinux2014", "s390x", "quay.io/pypa/manylinux2014_s390x", None),
# 2_24 images
Image("manylinux_2_24", "x86_64", "quay.io/pypa/manylinux_2_24_x86_64", None),
Image("manylinux_2_24", "i686", "quay.io/pypa/manylinux_2_24_i686", None),
Image("manylinux_2_24", "aarch64", "quay.io/pypa/manylinux_2_24_aarch64", None),
Image("manylinux_2_24", "ppc64le", "quay.io/pypa/manylinux_2_24_ppc64le", None),
Image("manylinux_2_24", "s390x", "quay.io/pypa/manylinux_2_24_s390x", None),
]
config = configparser.ConfigParser()
for image in images:
# get the tag name whose digest matches 'latest'
if image.tag is not None:
# image has been pinned, do not update
tag_name = image.tag
elif image.image_name.startswith("quay.io/"):
_, _, repository_name = image.image_name.partition("/")
response = requests.get(
f"https://quay.io/api/v1/repository/{repository_name}?includeTags=true"
)
response.raise_for_status()
repo_info = response.json()
tags_dict = repo_info["tags"]
latest_tag = tags_dict.pop("latest")
# find the tag whose manifest matches 'latest'
tag_name = next(
name
for (name, info) in tags_dict.items()
if info["manifest_digest"] == latest_tag["manifest_digest"]
)
else:
response = requests.get(f"https://hub.docker.com/v2/repositories/{image.image_name}/tags")
response.raise_for_status()
tags = response.json()["results"]
latest_tag = next(tag for tag in tags if tag["name"] == "latest")
# i don't know what it would mean to have multiple images per tag
assert len(latest_tag["images"]) == 1
digest = latest_tag["images"][0]["digest"]
pinned_tag = next(
tag for tag in tags if tag != latest_tag and tag["images"][0]["digest"] == digest
)
tag_name = pinned_tag["name"]
if not config.has_section(image.platform):
config[image.platform] = {}
config[image.platform][image.manylinux_version] = f"{image.image_name}:{tag_name}"
with open("cibuildwheel/resources/pinned_docker_images.cfg", "w") as f:
config.write(f)
-153
View File
@@ -1,153 +0,0 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "packaging",
# "requests",
# ]
# ///
import configparser
import dataclasses
from pathlib import Path
import requests
from packaging.version import Version
DIR = Path(__file__).parent.resolve()
RESOURCES = DIR.parent / "cibuildwheel/resources"
@dataclasses.dataclass(frozen=True)
class Image:
manylinux_version: str
platforms: list[str]
image_name: str
tag: str | None = None # Set this to pin the image
use_platform_suffix: bool = False
class PyPAImage(Image):
def __init__(self, manylinux_version: str, platforms: list[str], tag: str | None = None):
image_name = f"quay.io/pypa/{manylinux_version}"
super().__init__(manylinux_version, platforms, image_name, tag, True)
images = [
# manylinux2014 images
PyPAImage(
"manylinux2014",
[
"x86_64",
"i686",
"aarch64",
"ppc64le",
"s390x",
"pypy_x86_64",
"pypy_i686",
"pypy_aarch64",
],
),
# manylinux_2_28 images
PyPAImage(
"manylinux_2_28",
[
"x86_64",
"i686",
"aarch64",
"ppc64le",
"s390x",
"pypy_x86_64",
"pypy_i686",
"pypy_aarch64",
],
),
# manylinux_2_31 images
PyPAImage("manylinux_2_31", ["armv7l"]),
# manylinux_2_34 images
PyPAImage(
"manylinux_2_34",
[
"x86_64",
"i686",
"aarch64",
"ppc64le",
"s390x",
"pypy_x86_64",
"pypy_i686",
"pypy_aarch64",
],
),
# manylinux_2_35 images
PyPAImage("manylinux_2_35", ["armv7l"]),
# manylinux_2_39 images
PyPAImage("manylinux_2_39", ["riscv64"]),
# musllinux_1_2 images
PyPAImage(
"musllinux_1_2", ["x86_64", "i686", "aarch64", "ppc64le", "s390x", "armv7l", "riscv64"]
),
]
config = configparser.ConfigParser()
for image in images:
# get the tag name whose digest matches 'latest'
if image.tag is not None:
# image has been pinned, do not update
tag_name = image.tag
elif image.image_name.startswith("quay.io/"):
_, _, repository_name = image.image_name.partition("/")
response = requests.get(
f"https://quay.io/api/v1/repository/{repository_name}?includeTags=true"
)
response.raise_for_status()
repo_info = response.json()
tags_dict = repo_info["tags"]
latest_tag = tags_dict.pop("latest")
# find the tag whose manifest matches 'latest'
tag_name = next(
name
for (name, info) in tags_dict.items()
if info["manifest_digest"] == latest_tag["manifest_digest"]
)
elif image.image_name.startswith("ghcr.io/"):
repository = image.image_name[8:]
response = requests.get(
"https://ghcr.io/token", params={"scope": f"repository:{repository}:pull"}
)
response.raise_for_status()
token = response.json()["token"]
response = requests.get(
f"https://ghcr.io/v2/{repository}/tags/list",
headers={"Authorization": f"Bearer {token}"},
)
response.raise_for_status()
ghcr_tags = [(Version(tag), tag) for tag in response.json()["tags"] if tag != "latest"]
ghcr_tags.sort(reverse=True)
tag_name = ghcr_tags[0][1]
else:
response = requests.get(f"https://hub.docker.com/v2/repositories/{image.image_name}/tags")
response.raise_for_status()
tags = response.json()["results"]
latest_tag = next(tag for tag in tags if tag["name"] == "latest")
# i don't know what it would mean to have multiple images per tag
assert len(latest_tag["images"]) == 1
digest = latest_tag["images"][0]["digest"]
pinned_tag = next(
tag for tag in tags if tag != latest_tag and tag["images"][0]["digest"] == digest
)
tag_name = pinned_tag["name"]
for platform in image.platforms:
if not config.has_section(platform):
config[platform] = {}
suffix = ""
if image.use_platform_suffix:
suffix = f"_{platform.removeprefix('pypy_')}"
config[platform][image.manylinux_version] = f"{image.image_name}{suffix}:{tag_name}"
with open(RESOURCES / "pinned_docker_images.cfg", "w") as f:
config.write(f)
-61
View File
@@ -1,61 +0,0 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "playwright",
# ]
# ///
import subprocess
import sys
import tempfile
from pathlib import Path
from playwright.sync_api import sync_playwright # type: ignore[import-not-found]
CSS = """
<link href="https://fonts.googleapis.com/css2?family=Lato&display=swap" rel="stylesheet">
<style>
html, body {
font-family: Lato, "Helvetica Neue", Helvetica, Arial, sans-serif;
font-weight: 400;
font-size: 16px;
color: #404040;
background: white;
margin: 0;
padding: 0;
}
* {
box-sizing: border-box;
}
</style>
"""
def main() -> None:
subprocess.run([sys.executable, "-m", "playwright", "install", "chromium"], check=True)
html_str = Path("docs/diagram.html").read_text()
html_str = f"<html><head>{CSS}</head><body>{html_str}</body></html>"
with tempfile.TemporaryDirectory() as tmp_dir_str:
html_path = Path(tmp_dir_str) / "diagram_screenshot.html"
html_path.write_text(html_str)
dest_path = Path("docs/data/how-it-works.png")
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page(device_scale_factor=2, viewport={"width": 830, "height": 600})
page.goto(html_path.as_uri())
page.wait_for_load_state("networkidle")
height = page.evaluate("document.body.scrollHeight")
page.set_viewport_size({"width": 830, "height": height})
page.screenshot(path=str(dest_path), full_page=True)
browser.close()
if __name__ == "__main__":
main()
-155
View File
@@ -1,155 +0,0 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "click",
# "packaging",
# "requests",
# "rich",
# ]
# ///
import dataclasses
import difflib
import logging
import tomllib
from pathlib import Path
from typing import Final
import click
import packaging.specifiers
import requests
import rich
from packaging.version import InvalidVersion, Version
from rich.logging import RichHandler
from rich.syntax import Syntax
log = logging.getLogger("cibw")
# Looking up the dir instead of using utils.resources_dir
# since we want to write to it.
DIR: Final[Path] = Path(__file__).parent.parent.resolve()
RESOURCES_DIR: Final[Path] = DIR / "cibuildwheel/resources"
NODEJS_DIST: Final[str] = "https://nodejs.org/dist/"
NODEJS_INDEX: Final[str] = f"{NODEJS_DIST}index.json"
@dataclasses.dataclass(frozen=True, order=True)
class VersionTuple:
version: Version
version_string: str
def parse_nodejs_index() -> list[VersionTuple]:
versions: list[VersionTuple] = []
response = requests.get(NODEJS_INDEX)
response.raise_for_status()
versions_info = response.json()
for version_info in versions_info:
version_string = version_info.get("version", "???")
if not version_info.get("lts", False):
log.debug("Ignoring non LTS release %r", version_string)
continue
if "linux-x64" not in version_info.get("files", []):
log.warning(
"Ignoring release %r which does not include a linux-x64 binary", version_string
)
continue
try:
version = Version(version_string)
if version.is_devrelease:
log.info("Ignoring development release %r", str(version))
continue
if version.is_prerelease:
log.info("Ignoring pre-release %r", str(version))
continue
versions.append(VersionTuple(version, version_string))
except InvalidVersion:
log.warning("Ignoring release %r", version_string)
versions.sort(reverse=True)
return versions
@click.command()
@click.option("--force", is_flag=True)
@click.option(
"--level", default="INFO", type=click.Choice(["WARNING", "INFO", "DEBUG"], case_sensitive=False)
)
def update_nodejs(force: bool, level: str) -> None:
logging.basicConfig(
level="INFO",
format="%(message)s",
datefmt="[%X]",
handlers=[RichHandler(rich_tracebacks=True, markup=True)],
)
log.setLevel(level)
toml_file_path = RESOURCES_DIR / "nodejs.toml"
original_toml = toml_file_path.read_text()
with toml_file_path.open("rb") as f:
nodejs_data = tomllib.load(f)
nodejs_data.pop("url")
major_versions = [VersionTuple(Version(key), key) for key in nodejs_data]
major_versions.sort(reverse=True)
versions = parse_nodejs_index()
# update existing versions, 1 per LTS
for major_version in major_versions:
current = Version(nodejs_data[major_version.version_string])
specifier = packaging.specifiers.SpecifierSet(
specifiers=f"=={major_version.version.major}.*"
)
for version in versions:
if specifier.contains(version.version) and version.version > current:
nodejs_data[major_version.version_string] = version.version_string
break
# check for a new major LTS to insert
if versions and versions[0].version.major > major_versions[0].version.major:
major_versions.insert(
0,
VersionTuple(Version(str(versions[0].version.major)), f"v{versions[0].version.major}"),
)
nodejs_data[major_versions[0].version_string] = versions[0].version_string
versions_toml = "\n".join(
f'{major_version.version_string} = "{nodejs_data[major_version.version_string]}"'
for major_version in major_versions
)
result_toml = f'url = "{NODEJS_DIST}"\n{versions_toml}\n'
rich.print() # spacer
if original_toml == result_toml:
rich.print("[green]Check complete, nodejs version unchanged.")
return
rich.print("nodejs version updated.")
rich.print("Changes:")
rich.print()
toml_relpath = toml_file_path.relative_to(DIR).as_posix()
diff_lines = difflib.unified_diff(
original_toml.splitlines(keepends=True),
result_toml.splitlines(keepends=True),
fromfile=toml_relpath,
tofile=toml_relpath,
)
rich.print(Syntax("".join(diff_lines), "diff", theme="ansi_light"))
rich.print()
if force:
toml_file_path.write_text(result_toml)
rich.print("[green]TOML file updated.")
else:
rich.print("[yellow]File left unchanged. Use --force flag to update.")
if __name__ == "__main__":
update_nodejs()
-98
View File
@@ -1,98 +0,0 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "cibuildwheel",
# "requests",
# ]
#
# [tool.uv.sources]
# cibuildwheel = { path = ".." }
# ///
import json
from pathlib import Path
from typing import Final
import requests
from cibuildwheel.extra import github_api_request
from cibuildwheel.util.python_build_standalone import (
PythonBuildStandaloneAsset,
PythonBuildStandaloneReleaseData,
)
# Resolve path relative to this script so writes go to the source checkout,
# not the uv-installed copy of the package.
DIR: Final[Path] = Path(__file__).parent.parent.resolve()
PYTHON_BUILD_STANDALONE_RELEASES: Final[Path] = (
DIR / "cibuildwheel/resources/python-build-standalone-releases.json"
)
def main() -> None:
"""
This script updates the vendored list of release assets to the latest
version of astral-sh/python-build-standalone.
"""
# Get the latest release tag from the GitHub API
latest_release = github_api_request("repos/astral-sh/python-build-standalone/releases/latest")
latest_tag = latest_release["tag_name"]
# Get the list of assets for the latest release
github_assets = github_api_request(
f"repos/astral-sh/python-build-standalone/releases/tags/{latest_tag}"
)["assets"]
# Build a sha256 map from the SHA256SUMS file in the release
sha256_sums_urls = [
ga["browser_download_url"] for ga in github_assets if ga["name"] == "SHA256SUMS"
]
name_to_sha256: dict[str, str] = {}
if sha256_sums_urls:
response = requests.get(sha256_sums_urls[0])
response.raise_for_status()
for line in response.text.splitlines():
parts = line.split()
if len(parts) == 2:
sha256_hex, filename = parts
# The filename may have a leading "./" or spaces - strip it
name_to_sha256[filename.lstrip("./")] = sha256_hex
assets = [
PythonBuildStandaloneAsset(
name=ga["name"],
url=ga["browser_download_url"],
sha256=name_to_sha256.get(ga["name"], ""),
)
for ga in github_assets
if ga["name"].endswith("install_only.tar.gz")
]
# Try to keep output order stable
assets = sorted(assets, key=lambda x: x["name"])
# Write the assets to the JSON file. One day, we might need to support
# multiple releases, but for now, we only support the latest one
json_file_contents = PythonBuildStandaloneReleaseData(
releases=[
{
"tag": latest_tag,
"assets": assets,
}
]
)
with PYTHON_BUILD_STANDALONE_RELEASES.open("w", encoding="utf-8") as f:
json.dump(json_file_contents, f, indent=2)
# Add a trailing newline, our pre-commit hook requires it
f.write("\n")
print(
f"Updated {PYTHON_BUILD_STANDALONE_RELEASES.name} with {len(assets)} assets for tag {latest_tag}"
)
if __name__ == "__main__":
main()
+117 -420
View File
@@ -1,43 +1,24 @@
#!/usr/bin/env -S uv run --script
#!/usr/bin/env python3
# /// script
# dependencies = [
# "click",
# "packaging",
# "requests",
# "rich",
# "cibuildwheel",
# ]
#
# [tool.uv.sources]
# cibuildwheel = { path = ".." }
# ///
from __future__ import annotations
import copy
import difflib
import hashlib
import logging
import operator
import re
import tomllib
from pathlib import Path
from typing import Any, Final, Literal, NotRequired, TypedDict
from xml.etree import ElementTree as ET
from typing import Union
import click
import requests
import rich
import toml
from packaging.specifiers import Specifier
from packaging.version import Version
from rich.logging import RichHandler
from rich.syntax import Syntax
from cibuildwheel.extra import dump_python_configurations, get_pyodide_xbuildenv_info
from cibuildwheel.platforms.android import android_triplet
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Mapping, MutableMapping
from cibuildwheel.extra import InlineArrayDictEncoder
from cibuildwheel.typing import Final, Literal, TypedDict
log = logging.getLogger("cibw")
@@ -47,23 +28,29 @@ DIR: Final[Path] = Path(__file__).parent.parent.resolve()
RESOURCES_DIR: Final[Path] = DIR / "cibuildwheel/resources"
ArchStr = Literal["32", "64", "ARM64"]
ArchStr = Literal["32", "64"]
class Config(TypedDict):
class ConfigWinCP(TypedDict):
identifier: str
version: str
arch: str
class ConfigUrl(Config):
class ConfigWinPP(TypedDict):
identifier: str
version: str
arch: str
url: str
sha256: NotRequired[str]
class ConfigPyodide(Config):
default_pyodide_version: str
node_version: str
sha256: str
class ConfigMacOS(TypedDict):
identifier: str
version: str
url: str
AnyConfig = Union[ConfigWinCP, ConfigWinPP, ConfigMacOS]
# The following set of "Versions" classes allow the initial call to the APIs to
@@ -71,7 +58,8 @@ class ConfigPyodide(Config):
class WindowsVersions:
def __init__(self, arch_str: ArchStr, free_threaded: bool) -> None:
def __init__(self, arch_str: ArchStr) -> None:
response = requests.get("https://api.nuget.org/v3/index.json")
response.raise_for_status()
api_info = response.json()
@@ -80,140 +68,41 @@ class WindowsVersions:
if resource["@type"] == "PackageBaseAddress/3.0.0":
endpoint = resource["@id"]
ARCH_DICT = {"32": "win32", "64": "win_amd64", "ARM64": "win_arm64"}
PACKAGE_DICT = {"32": "pythonx86", "64": "python", "ARM64": "pythonarm64"}
ARCH_DICT = {"32": "win32", "64": "win_amd64"}
PACKAGE_DICT = {"32": "pythonx86", "64": "python"}
self.arch_str = arch_str
self.arch = ARCH_DICT[arch_str]
self.free_threaded = free_threaded
package = PACKAGE_DICT[arch_str]
if free_threaded:
package = f"{package}-freethreaded"
response = requests.get(f"{endpoint}{package}/index.json")
response.raise_for_status()
cp_info = response.json()
self.version_dict = {Version(v): v for v in cp_info["versions"]}
versions = (Version(v) for v in cp_info["versions"])
self.versions = sorted(v for v in versions if not v.is_devrelease)
def update_version_windows(self, spec: Specifier) -> Config | None:
# Specifier.filter selects all non pre-releases that match the spec,
# unless there are only pre-releases, then it selects pre-releases
# instead (like pip)
unsorted_versions = spec.filter(self.version_dict)
versions = sorted(unsorted_versions, reverse=True)
log.debug("Windows %s %s has %s", self.arch, spec, ", ".join(str(v) for v in versions))
def update_version_windows(self, spec: Specifier) -> ConfigWinCP | None:
versions = sorted(v for v in self.versions if spec.contains(v))
if not all(v.is_prerelease for v in versions):
versions = [v for v in versions if not v.is_prerelease]
log.debug(f"Windows {self.arch} {spec} has {', '.join(str(v) for v in versions)}")
if not versions:
return None
flags = "t" if self.free_threaded else ""
version = versions[0]
identifier = f"cp{version.major}{version.minor}{flags}-{self.arch}"
return Config(
version = versions[-1]
identifier = f"cp{version.major}{version.minor}-{self.arch}"
return ConfigWinCP(
identifier=identifier,
version=self.version_dict[version],
version=str(version),
arch=self.arch_str,
)
class GraalPyVersions:
def __init__(self) -> None:
response = requests.get("https://api.github.com/repos/oracle/graalpython/releases")
response.raise_for_status()
releases = response.json()
gp_asset_re = re.compile(
r"^(?P<prefix>graalpy(?P<cpython>\d+\.\d+)?-(?P<graalpy>\d+\.\d+\.\d+))-"
)
cp_version_re = re.compile(r"Python (\d+\.\d+(?:\.\d+)?)")
for release in releases:
for asset in release["assets"]:
m = gp_asset_re.match(asset["name"])
if m:
release["asset_prefix"] = m.group("prefix")
release["graalpy_version"] = Version(m.group("graalpy"))
if m.group("cpython"):
release["python_version"] = Version(m.group("cpython"))
break
if "python_version" not in release and (
m := cp_version_re.search(release["body"] or "")
):
release["python_version"] = Version(m.group(1))
self.releases = [r for r in releases if "graalpy_version" in r and "python_version" in r]
def update_version(self, identifier: str, spec: Specifier) -> ConfigUrl | None:
if "x86_64" in identifier or "amd64" in identifier:
arch = "x86_64"
elif "arm64" in identifier or "aarch64" in identifier:
arch = "aarch64"
else:
msg = f"{identifier} not supported yet on GraalPy"
raise RuntimeError(msg)
gpspec_str = identifier.split("-", maxsplit=1)[0].split("_")[1]
if "." not in gpspec_str and len(gpspec_str) == 3:
gpspec_str = gpspec_str[:2] + "." + gpspec_str[-1]
gpspec = Specifier(f"=={gpspec_str}.*")
releases_tmp = (r for r in self.releases if spec.contains(r["python_version"]))
releases_tmp = (r for r in releases_tmp if gpspec.contains(r["graalpy_version"]))
releases = sorted(releases_tmp, key=lambda r: r["graalpy_version"])
if not releases:
msg = f"GraalPy {arch} not found for {spec}!"
raise RuntimeError(msg)
if "macosx" in identifier:
arch = "x86_64" if "x86_64" in identifier else "arm64"
platform = "macos"
elif "win" in identifier:
arch = "aarch64" if "arm64" in identifier else "x86_64"
platform = "windows"
else:
msg = "GraalPy provides downloads for macOS and Windows and is included for manylinux"
raise RuntimeError(msg)
arch = "amd64" if arch == "x86_64" else "aarch64"
ext = "zip" if "win" in identifier else "tar.gz"
for release in reversed(releases):
version = release["python_version"]
urls = [
rf["browser_download_url"]
for rf in release["assets"]
if rf["name"].endswith(f"{platform}-{arch}.{ext}")
and rf["name"].startswith(release["asset_prefix"])
]
if not urls:
continue
(url,) = urls
# Fetch sha256 from the ".sha256" sidecar asset in the same release.
sha256 = ""
sha256_asset_name = url.rsplit("/", 1)[-1] + ".sha256"
sha256_urls = [
rf["browser_download_url"]
for rf in release["assets"]
if rf["name"] == sha256_asset_name
]
if sha256_urls:
sha256_response = requests.get(sha256_urls[0])
sha256_response.raise_for_status()
sha256 = sha256_response.text.strip().split()[0]
return ConfigUrl(
identifier=identifier,
version=f"{version.major}.{version.minor}",
url=url,
sha256=sha256,
)
return None
class PyPyVersions:
def __init__(self, arch_str: ArchStr):
response = requests.get("https://downloads.python.org/pypy/versions.json")
response.raise_for_status()
@@ -229,61 +118,50 @@ class PyPyVersions:
]
self.arch = arch_str
def get_arch_file(self, release: Mapping[str, Any]) -> str:
urls: list[str] = [
rf["download_url"]
for rf in release["files"]
if "" in rf["platform"] == f"win{self.arch}"
]
return urls[0] if urls else ""
def update_version_windows(self, spec: Specifier) -> ConfigWinCP:
if self.arch != "32":
raise RuntimeError("64 bit releases not supported yet on Windows")
def update_version_windows(self, spec: Specifier) -> ConfigUrl:
releases = [r for r in self.releases if spec.contains(r["python_version"])]
releases = sorted(releases, key=operator.itemgetter("pypy_version"))
releases = [r for r in releases if self.get_arch_file(r)]
releases = sorted(releases, key=lambda r: r["pypy_version"])
if not releases:
msg = f"PyPy Win {self.arch} not found for {spec}! {self.releases}"
raise RuntimeError(msg)
version_arch = "win32" if self.arch == "32" else "win_amd64"
raise RuntimeError(f"PyPy Win {self.arch} not found for {spec}! {self.releases}")
release = releases[-1]
version = release["python_version"]
identifier = f"pp{version.major}{version.minor}-{version_arch}"
url = self.get_arch_file(release)
identifier = f"pp{version.major}{version.minor}-win32"
return ConfigUrl(
(url,) = [rf["download_url"] for rf in release["files"] if "" in rf["platform"] == "win32"]
return ConfigWinPP(
identifier=identifier,
version=f"{version.major}.{version.minor}",
arch="32",
url=url,
)
def update_version_macos(self, spec: Specifier) -> ConfigUrl:
if self.arch not in {"64", "ARM64"}:
msg = f"'{self.arch}' arch not supported yet on macOS"
raise RuntimeError(msg)
def update_version_macos(self, spec: Specifier) -> ConfigMacOS:
if self.arch != "64":
raise RuntimeError("Other archs not supported yet on macOS")
releases = [r for r in self.releases if spec.contains(r["python_version"])]
releases = sorted(releases, key=operator.itemgetter("pypy_version"))
releases = sorted(releases, key=lambda r: r["pypy_version"])
if not releases:
msg = f"PyPy macOS {self.arch} not found for {spec}!"
raise RuntimeError(msg)
raise RuntimeError(f"PyPy macOS {self.arch} not found for {spec}!")
release = releases[-1]
version = release["python_version"]
arch = "x86_64" if self.arch == "64" else self.arch.lower()
identifier = f"pp{version.major}{version.minor}-macosx_{arch}"
identifier = f"pp{version.major}{version.minor}-macosx_x86_64"
arch = "x64" if self.arch == "64" else self.arch.lower()
(url,) = (
(url,) = [
rf["download_url"]
for rf in release["files"]
if "" in rf["platform"] == "darwin" and rf["arch"] == arch
)
if "" in rf["platform"] == "darwin" and rf["arch"] == "x64"
]
return ConfigUrl(
return ConfigMacOS(
identifier=identifier,
version=f"{version.major}.{version.minor}",
url=url,
@@ -292,6 +170,7 @@ class PyPyVersions:
class CPythonVersions:
def __init__(self) -> None:
response = requests.get(
"https://www.python.org/api/v2/downloads/release/?is_published=true"
)
@@ -301,155 +180,37 @@ class CPythonVersions:
self.versions_dict: dict[Version, int] = {}
for release in releases_info:
# Skip the pymanager releases
if not release["slug"].startswith("python"):
continue
# Removing the prefix, Python 3.9 would use: release["name"].removeprefix("Python ")
version = Version(release["name"][7:])
# Removing the prefix
version = Version(release["name"].removeprefix("Python "))
self.versions_dict[version] = release["resource_uri"]
if not version.is_prerelease and not version.is_devrelease:
uri = int(release["resource_uri"].rstrip("/").split("/")[-1])
self.versions_dict[version] = uri
files_response = requests.get("https://www.python.org/api/v2/downloads/release_file/")
files_response.raise_for_status()
self.files_info = files_response.json()
def update_version_macos(self, identifier: str, spec: Specifier) -> ConfigMacOS | None:
file_idents = ("macos11.pkg", "macosx10.9.pkg", "macosx10.6.pkg")
sorted_versions = sorted(v for v in self.versions_dict if spec.contains(v))
def update_version(self, identifier: str, spec: Specifier, file_ident: str) -> ConfigUrl | None:
# see note above on Specifier.filter
unsorted_versions = spec.filter(self.versions_dict)
sorted_versions = sorted(unsorted_versions, reverse=True)
for new_version in sorted_versions:
for version in reversed(sorted_versions):
# Find the first patch version that contains the requested file
uri = self.versions_dict[new_version]
files = [rf for rf in self.files_info if rf["release"] == uri]
uri = self.versions_dict[version]
response = requests.get(
f"https://www.python.org/api/v2/downloads/release_file/?release={uri}"
)
response.raise_for_status()
file_info = response.json()
matching = [rf for rf in files if file_ident in rf["url"]]
if matching:
rf = matching[0]
return ConfigUrl(
identifier=identifier,
version=f"{new_version.major}.{new_version.minor}",
url=rf["url"],
sha256=rf.get("sha256_sum", ""),
)
for file_ident in file_idents:
urls = [rf["url"] for rf in file_info if file_ident in rf["url"]]
if urls:
return ConfigMacOS(
identifier=identifier,
version=f"{version.major}.{version.minor}",
url=urls[0],
)
return None
def update_version_macos(
self, identifier: str, version: Version, spec: Specifier
) -> ConfigUrl | None:
macver = "x10.9" if version <= Version("3.8.9999") else "11"
return self.update_version(identifier, spec, f"macos{macver}.pkg")
def update_version_android(self, identifier: str, spec: Specifier) -> ConfigUrl | None:
return self.update_version(identifier, spec, android_triplet(identifier))
class MavenVersions:
MAVEN_URL = "https://repo.maven.apache.org/maven2/com/chaquo/python/python"
def __init__(self) -> None:
response = requests.get(f"{self.MAVEN_URL}/maven-metadata.xml")
response.raise_for_status()
root = ET.fromstring(response.text)
self.versions: list[Version] = []
for version_elem in root.findall("./versioning/versions/version"):
version_str = version_elem.text
assert isinstance(version_str, str), version_str
self.versions.append(Version(version_str))
def update_version_android(self, identifier: str, spec: Specifier) -> ConfigUrl | None:
sorted_versions = sorted(spec.filter(self.versions), reverse=True)
# Return a config using the highest version for the given specifier.
if sorted_versions:
max_version = sorted_versions[0]
triplet = android_triplet(identifier)
return ConfigUrl(
identifier=identifier,
version=f"{max_version.major}.{max_version.minor}",
url=f"{self.MAVEN_URL}/{max_version}/python-{max_version}-{triplet}.tar.gz",
)
else:
return None
class CPythonIOSVersions:
def __init__(self) -> None:
response = requests.get(
"https://api.github.com/repos/beeware/Python-Apple-support/releases",
headers={
"Accept": "application/vnd.github+json",
"X-Github-Api-Version": "2022-11-28",
},
)
response.raise_for_status()
releases_info = response.json()
self.versions_dict: dict[Version, dict[int, str]] = {}
# Each release has a name like "3.13-b4"
for release in releases_info:
py_version, build = release["name"].split("-")
version = Version(py_version)
self.versions_dict.setdefault(version, {})
# There are several release assets associated with each release;
# The name of the asset will be something like
# "Python-3.11-iOS-support.b4.tar.gz". Store all builds that are
# "-iOS-support" builds, retaining the download URL.
for asset in release["assets"]:
filename, build, _, _ = asset["name"].rsplit(".", 3)
if filename.endswith("-iOS-support"):
self.versions_dict[version][int(build[1:])] = asset["browser_download_url"]
def update_version_ios(self, identifier: str, version: Version) -> ConfigUrl | None:
# Return a config using the highest build number for the given version.
urls = [url for _, url in sorted(self.versions_dict.get(version, {}).items())]
if urls:
return ConfigUrl(
identifier=identifier,
version=str(version),
url=urls[-1],
)
return None
class PyodideVersions:
def __init__(self) -> None:
xbuildenv_info = get_pyodide_xbuildenv_info()
self.releases = xbuildenv_info["releases"]
def update_version_pyodide(
self, identifier: str, version: Version, spec: Specifier, node_version: str
) -> ConfigPyodide | None:
# get releases that match the python version
releases = [
r for r in self.releases.values() if spec.contains(Version(r["python_version"]))
]
# sort by version, latest first
releases.sort(key=lambda r: Version(r["version"]), reverse=True)
if not releases:
msg = f"Pyodide not found for {spec}!"
raise ValueError(msg)
final_releases = [r for r in releases if not Version(r["version"]).is_prerelease]
# prefer a final release if available, otherwise use the latest
# pre-release
release = final_releases[0] if final_releases else releases[0]
return ConfigPyodide(
identifier=identifier,
version=str(version),
default_pyodide_version=release["version"],
node_version=node_version,
sha256=release["sha256"],
)
# This is a universal interface to all the above Versions classes. Given an
# identifier, it updates a config dict.
@@ -457,118 +218,53 @@ class PyodideVersions:
class AllVersions:
def __init__(self) -> None:
self.windows_32 = WindowsVersions("32", False)
self.windows_t_32 = WindowsVersions("32", True)
self.windows_64 = WindowsVersions("64", False)
self.windows_t_64 = WindowsVersions("64", True)
self.windows_arm64 = WindowsVersions("ARM64", False)
self.windows_t_arm64 = WindowsVersions("ARM64", True)
self.windows_pypy_64 = PyPyVersions("64")
self.windows_32 = WindowsVersions("32")
self.windows_64 = WindowsVersions("64")
self.windows_pypy = PyPyVersions("32")
self.cpython = CPythonVersions()
self.macos_cpython = CPythonVersions()
self.macos_pypy = PyPyVersions("64")
self.macos_pypy_arm64 = PyPyVersions("ARM64")
self.maven = MavenVersions()
self.ios_cpython = CPythonIOSVersions()
self.graalpy = GraalPyVersions()
self.pyodide = PyodideVersions()
def _stream_sha256(self, url: str) -> str:
"""Download a file (streaming) and return its SHA256 hex digest."""
log.debug("Computing sha256 for %s by streaming download...", url)
response = requests.get(url, stream=True)
response.raise_for_status()
hasher = hashlib.sha256()
for chunk in response.iter_content(65536):
hasher.update(chunk)
return hasher.hexdigest()
def update_config(self, config: MutableMapping[str, str]) -> None:
def update_config(self, config: dict[str, str]) -> None:
identifier = config["identifier"]
version = Version(config["version"])
spec = Specifier(f"=={version.major}.{version.minor}.*")
log.info("Reading in %r -> %s @ %s", str(identifier), spec, version)
config_update: Config | None = None
log.info(f"Reading in '{identifier}' -> {spec} @ {version}")
orig_config = copy.copy(config)
config_update: AnyConfig | None
# We need to use ** in update due to MyPy (probably a bug)
if "macosx" in identifier:
if identifier.startswith("cp"):
config_update = self.cpython.update_version_macos(identifier, version, spec)
elif identifier.startswith("pp"):
if "macosx_x86_64" in identifier:
config_update = self.macos_pypy.update_version_macos(spec)
elif "macosx_arm64" in identifier:
config_update = self.macos_pypy_arm64.update_version_macos(spec)
elif identifier.startswith("gp"):
config_update = self.graalpy.update_version(identifier, spec)
elif "t-win32" in identifier and identifier.startswith("cp"):
config_update = self.windows_t_32.update_version_windows(spec)
elif "win32" in identifier and identifier.startswith("cp"):
config_update = self.windows_32.update_version_windows(spec)
elif "t-win_amd64" in identifier and identifier.startswith("cp"):
config_update = self.windows_t_64.update_version_windows(spec)
elif "win_amd64" in identifier:
if identifier.startswith("cp"):
config_update = self.windows_64.update_version_windows(spec)
elif identifier.startswith("pp"):
config_update = self.windows_pypy_64.update_version_windows(spec)
elif identifier.startswith("gp"):
config_update = self.graalpy.update_version(identifier, spec)
elif "t-win_arm64" in identifier and identifier.startswith("cp"):
config_update = self.windows_t_arm64.update_version_windows(spec)
elif "win_arm64" in identifier and identifier.startswith("cp"):
config_update = self.windows_arm64.update_version_windows(spec)
elif "android" in identifier:
# Python 3.13 is released by Chaquopy on Maven Central.
# Python 3.14 and newer have official releases on python.org.
versions = self.maven if identifier.startswith("cp313") else self.cpython
config_update = versions.update_version_android(identifier, spec)
elif "ios" in identifier:
# Python 3.13 and 3.14 are released by BeeWare on GitHub.
# Python 3.15 and newer have official releases on python.org.
if identifier.startswith(("cp313", "cp314")):
config_update = self.ios_cpython.update_version_ios(identifier, version)
if "macos" in identifier:
if identifier.startswith("pp"):
config_update = self.macos_pypy.update_version_macos(spec)
else:
config_update = self.cpython.update_version(
identifier, spec, "iOS-XCframework.tar.gz"
)
elif "pyodide" in identifier:
config_update = self.pyodide.update_version_pyodide(
identifier, version, spec, config["node_version"]
)
config_update = self.macos_cpython.update_version_macos(identifier, spec)
assert config_update is not None, f"{identifier} not found!"
# Fill in sha256 for URL-based configs if not already provided by the
# update_version_* method (e.g. PyPy, BeeWare iOS, Maven have no sidecar).
# Also fills in sha256 when the CPython API doesn't return a sha256_sum
# (e.g. for older releases).
# Widen the type to allow arbitrary key access on the underlying dict.
config_update_dict: dict[str, str] = config_update # type: ignore[assignment]
if "url" in config_update_dict and not config_update_dict.get("sha256"):
url = config_update_dict["url"]
existing_sha256 = config.get("sha256", "")
if url == config.get("url") and existing_sha256:
# URL unchanged — preserve the existing sha256
config_update_dict["sha256"] = existing_sha256
else:
config_update_dict["sha256"] = self._stream_sha256(url)
if config_update != config:
log.info(" Updated %s to %s", config, config_update)
config.clear()
assert config_update is not None, f"MacOS {spec} not found!"
config.update(**config_update)
elif "win32" in identifier:
if identifier.startswith("pp"):
config.update(**self.windows_pypy.update_version_windows(spec))
else:
config_update = self.windows_32.update_version_windows(spec)
if config_update:
config.update(**config_update)
elif "win_amd64" in identifier:
config_update = self.windows_64.update_version_windows(spec)
if config_update:
config.update(**config_update)
if config != orig_config:
log.info(f" Updated {orig_config} to {config}")
@click.command()
@click.option("--force", is_flag=True)
@click.option(
"--level", default="INFO", type=click.Choice(["WARNING", "INFO", "DEBUG"], case_sensitive=False)
"--level", default="INFO", type=click.Choice(["INFO", "DEBUG", "TRACE"], case_sensitive=False)
)
def update_pythons(force: bool, level: str) -> None:
logging.basicConfig(
level="INFO",
format="%(message)s",
@@ -581,14 +277,15 @@ def update_pythons(force: bool, level: str) -> None:
toml_file_path = RESOURCES_DIR / "build-platforms.toml"
original_toml = toml_file_path.read_text()
with toml_file_path.open("rb") as f:
configs = tomllib.load(f)
configs = toml.loads(original_toml)
for platform in ["windows", "macos", "android", "ios", "pyodide"]:
for config in configs[platform]["python_configurations"]:
all_versions.update_config(config)
for config in configs["windows"]["python_configurations"]:
all_versions.update_config(config)
result_toml = dump_python_configurations(configs)
for config in configs["macos"]["python_configurations"]:
all_versions.update_config(config)
result_toml = toml.dumps(configs, encoder=InlineArrayDictEncoder()) # type: ignore
rich.print() # spacer
-152
View File
@@ -1,152 +0,0 @@
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "click",
# "packaging",
# "requests",
# "rich",
# "cibuildwheel",
# ]
#
# [tool.uv.sources]
# cibuildwheel = { path = ".." }
# ///
import dataclasses
import difflib
import hashlib
import logging
import tomllib
from pathlib import Path
from typing import Final
import click
import requests
import rich
from packaging.version import Version
from rich.logging import RichHandler
from rich.syntax import Syntax
from cibuildwheel.extra import github_api_request
log = logging.getLogger("cibw")
# Looking up the dir instead of using utils.resources_dir
# since we want to write to it.
DIR: Final[Path] = Path(__file__).parent.parent.resolve()
RESOURCES_DIR: Final[Path] = DIR / "cibuildwheel/resources"
GET_VIRTUALENV_GITHUB: Final[str] = "https://github.com/pypa/get-virtualenv"
GET_VIRTUALENV_URL_TEMPLATE: Final[str] = (
f"{GET_VIRTUALENV_GITHUB}/blob/{{version}}/public/virtualenv.pyz?raw=true"
)
@dataclasses.dataclass(frozen=True, order=True)
class VersionTuple:
name: str
download_url: str
version: Version
def get_latest_virtualenv_release() -> VersionTuple:
response = github_api_request("repos/pypa/get-virtualenv/releases/latest")
tag_name = response["tag_name"]
asset = next(
(asset for asset in response["assets"] if asset["name"] == "virtualenv.pyz"),
None,
)
if not asset:
msg = "No asset named 'virtualenv.pyz' found in the latest release of get-virtualenv."
raise RuntimeError(msg)
return VersionTuple(
version=Version(tag_name), name=tag_name, download_url=asset["browser_download_url"]
)
@click.command()
@click.option("--force", is_flag=True)
@click.option(
"--level", default="INFO", type=click.Choice(["WARNING", "INFO", "DEBUG"], case_sensitive=False)
)
def update_virtualenv(force: bool, level: str) -> None:
logging.basicConfig(
level="INFO",
format="%(message)s",
datefmt="[%X]",
handlers=[RichHandler(rich_tracebacks=True, markup=True)],
)
log.setLevel(level)
toml_file_path = RESOURCES_DIR / "virtualenv.toml"
original_toml = toml_file_path.read_text()
with toml_file_path.open("rb") as f:
configurations = tomllib.load(f)
default = configurations.pop("default")
local_version = str(default["version"])
latest_release = get_latest_virtualenv_release()
if latest_release.version > Version(local_version):
version = latest_release.name
url = latest_release.download_url
sha256 = "" # recomputed below
else:
version = local_version
url = default["url"]
sha256 = default["sha256"]
# Compute sha256 if not already stored (new version or first-time population)
if not sha256:
log.info("Computing sha256 for %s...", url)
response = requests.get(url, stream=True)
response.raise_for_status()
hasher = hashlib.sha256()
for chunk in response.iter_content(65536):
hasher.update(chunk)
sha256 = hasher.hexdigest()
configurations["default"] = {
"version": version,
"url": url,
"sha256": sha256,
}
result_toml = "".join(
f'{key} = {{ version = "{value["version"]}", url = "{value["url"]}", sha256 = "{value["sha256"]}" }}\n'
for key, value in configurations.items()
)
rich.print() # spacer
if original_toml == result_toml:
rich.print("[green]Check complete, virtualenv version unchanged.")
return
rich.print("virtualenv version updated.")
rich.print("Changes:")
rich.print()
toml_relpath = toml_file_path.relative_to(DIR).as_posix()
diff_lines = difflib.unified_diff(
original_toml.splitlines(keepends=True),
result_toml.splitlines(keepends=True),
fromfile=toml_relpath,
tofile=toml_relpath,
)
rich.print(Syntax("".join(diff_lines), "diff", theme="ansi_light"))
rich.print()
if force:
toml_file_path.write_text(result_toml)
rich.print("[green]TOML file updated.")
else:
rich.print("[yellow]File left unchanged. Use --force flag to update.")
if __name__ == "__main__":
update_virtualenv()
+1 -1
View File
@@ -1 +1 @@
__version__ = "4.1.0"
__version__ = "1.12.0"
+337 -451
View File
@@ -1,123 +1,90 @@
from __future__ import annotations
__lazy_modules__ = {
"argparse",
"cibuildwheel._compat",
"cibuildwheel._compat.tarfile",
"cibuildwheel.architecture",
"cibuildwheel.ci",
"cibuildwheel.logger",
"cibuildwheel.options",
"cibuildwheel.platforms",
"cibuildwheel.selector",
"cibuildwheel.typing",
"cibuildwheel.util",
"cibuildwheel.util.file",
"cibuildwheel.util.helpers",
"cibuildwheel.util.resources",
"contextlib",
"functools",
"io",
"pathlib",
"shutil",
"tempfile",
"textwrap",
"traceback",
}
import argparse
import contextlib
import dataclasses
import functools
import io
import os
import shutil
import sys
import textwrap
import traceback
import typing
from configparser import ConfigParser
from pathlib import Path
from tempfile import mkdtemp
from typing import Dict, List, Optional, Set, Union, overload
from packaging.specifiers import SpecifierSet
import cibuildwheel
from cibuildwheel import errors
from cibuildwheel._compat.tarfile import TarFile, safe_extractall
import cibuildwheel.linux
import cibuildwheel.macos
import cibuildwheel.util
import cibuildwheel.windows
from cibuildwheel.architecture import Architecture, allowed_architectures_check
from cibuildwheel.ci import CIProvider, detect_ci_provider, fix_ansi_codes_for_github_actions
from cibuildwheel.logger import log
from cibuildwheel.options import CommandLineArguments, Options, compute_options
from cibuildwheel.platforms import ALL_PLATFORM_MODULES, get_build_identifiers, native_platform
from cibuildwheel.selector import BuildSelector, EnableGroup, selector_matches
from cibuildwheel.typing import PLATFORMS, PlatformName
from cibuildwheel.util.file import CIBW_CACHE_PATH, ensure_cache_sentinel
from cibuildwheel.util.helpers import strtobool
from cibuildwheel.util.resources import read_all_configs
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Generator, Iterable, Sequence
from typing import Literal
from cibuildwheel.environment import EnvironmentParseError, parse_environment
from cibuildwheel.projectfiles import get_requires_python_str
from cibuildwheel.typing import PLATFORMS, PlatformName, assert_never
from cibuildwheel.util import (
BuildOptions,
BuildSelector,
DependencyConstraints,
TestSelector,
Unbuffered,
detect_ci_provider,
resources_dir,
)
@dataclasses.dataclass
class GlobalOptions:
print_traceback_on_error: bool = True # decides what happens when errors are hit.
@overload
def get_option_from_environment(
option_name: str, *, platform: Optional[str] = None, default: str
) -> str:
... # noqa: E704
@dataclasses.dataclass(frozen=True)
class FileReport:
name: str
size: str
@overload
def get_option_from_environment(
option_name: str, *, platform: Optional[str] = None, default: None = None
) -> Optional[str]:
... # noqa: E704 E302
def get_option_from_environment(
option_name: str, *, platform: Optional[str] = None, default: Optional[str] = None
) -> Optional[str]: # noqa: E302
"""
Returns an option from the environment, optionally scoped by the platform.
Example:
get_option_from_environment('CIBW_COLOR', platform='macos')
This will return the value of CIBW_COLOR_MACOS if it exists, otherwise the value of
CIBW_COLOR.
"""
if platform:
option = os.environ.get(f"{option_name}_{platform.upper()}")
if option is not None:
return option
return os.environ.get(option_name, default)
def main() -> None:
global_options = GlobalOptions()
try:
main_inner(global_options)
except errors.FatalError as e:
message = e.args[0]
if log.step_active:
log.step_end_with_error(message)
else:
log.error(message)
platform: PlatformName
if global_options.print_traceback_on_error:
traceback.print_exc(file=sys.stderr)
sys.exit(e.return_code)
def main_inner(global_options: GlobalOptions) -> None:
"""
`main_inner` is the same as `main`, but it raises FatalError exceptions
rather than exiting directly.
"""
# Default in 3.15+, only needed on 3.14
if sys.version_info >= (3, 14):
arg_parser = functools.partial(argparse.ArgumentParser, suggest_on_error=True)
else:
arg_parser = argparse.ArgumentParser
parser = arg_parser(
parser = argparse.ArgumentParser(
description="Build wheels for all the platforms.",
epilog="""
Most options are supplied via environment variables or in
--config-file (pyproject.toml usually). See
https://github.com/pypa/cibuildwheel#options for info.
Most options are supplied via environment variables.
See https://github.com/pypa/cibuildwheel#options for info.
""",
allow_abbrev=False,
)
parser.add_argument(
"--platform",
choices=["auto", "linux", "macos", "windows", "pyodide", "android", "ios"],
default=None,
choices=["auto", "linux", "macos", "windows"],
default=os.environ.get("CIBW_PLATFORM", "auto"),
help="""
Platform to build for. Use this option to override the auto-detected
platform. Specifying "macos" or "windows" only works on that
operating system. "linux" works on any desktop OS, as long as
Docker/Podman is installed. "pyodide" and "android" only work on
Linux and macOS. "ios" only works on macOS. Default: auto.
Platform to build for. For "linux" you need docker running, on Mac
or Linux. For "macos", you need a Mac machine, and note that this
script is going to automatically install MacPython on your system,
so don't run on your development machine. For "windows", you need to
run in Windows, and it will build and test for all versions of
Python. Default: auto.
""",
)
@@ -135,60 +102,21 @@ def main_inner(global_options: GlobalOptions) -> None:
""",
)
enable_groups_str = ", ".join(g.value for g in EnableGroup)
parser.add_argument(
"--enable",
action="append",
default=[],
metavar="GROUP",
help=f"""
Enable an additional category of builds. Use multiple times to select multiple groups. Choices: {enable_groups_str}.
""",
)
parser.add_argument(
"--only",
default=None,
choices=[v["identifier"] for vv in read_all_configs().values() for v in vv],
metavar="IDENTIFIER",
help="""
Force a single wheel build when given an identifier. Overrides
CIBW_BUILD/CIBW_SKIP. --platform and --arch cannot be specified
if this is given.
""",
)
parser.add_argument(
"--output-dir",
type=Path,
default=Path(os.environ.get("CIBW_OUTPUT_DIR", "wheelhouse")),
help="Destination folder for the wheels. Default: wheelhouse.",
)
parser.add_argument(
"--config-file",
default="",
help="""
TOML config file. Default: "", meaning {package}/pyproject.toml, if
it exists. To refer to a project inside your project, use {package};
this matters if you build from an SDist.
""",
default=os.environ.get("CIBW_OUTPUT_DIR", "wheelhouse"),
help="Destination folder for the wheels.",
)
parser.add_argument(
"package_dir",
metavar="PACKAGE",
default=Path(),
type=Path,
default=".",
nargs="?",
help="""
Path to the package that you want wheels for. Default: the working
directory. Can be a directory inside the working directory, or an
sdist. When set to a directory, the working directory is still
considered the 'project' and is copied into the build container
on Linux. When set to a tar.gz sdist file, --config-file
and --output-dir are relative to the current directory, and other
paths are relative to the expanded SDist directory.
Path to the package that you want wheels for. Must be a subdirectory of
the working directory. When set, the working directory is still
considered the 'project' and is copied into the Docker container on
Linux. Default: the working directory.
""",
)
@@ -198,219 +126,264 @@ def main_inner(global_options: GlobalOptions) -> None:
help="Print the build identifiers matched by the current invocation and exit.",
)
parser.add_argument(
"--clean-cache",
action="store_true",
help="Clear the cibuildwheel cache and exit.",
)
parser.add_argument(
"--allow-empty",
action="store_true",
help="Do not report an error code if the build does not match any wheels.",
)
parser.add_argument(
"--debug-traceback",
action="store_true",
default=strtobool(os.environ.get("CIBW_DEBUG_TRACEBACK", "0")),
help="Print a full traceback for all errors",
args = parser.parse_args()
detect_obsolete_options()
if args.platform != "auto":
platform = args.platform
else:
ci_provider = detect_ci_provider()
if ci_provider is None:
print(
textwrap.dedent(
"""
cibuildwheel: Unable to detect platform. cibuildwheel should run on your CI server;
Travis CI, AppVeyor, Azure Pipelines, GitHub Actions, CircleCI, and Gitlab are
supported. You can run on your development machine or other CI providers using the
--platform argument. Check --help output for more information.
"""
),
file=sys.stderr,
)
sys.exit(2)
if sys.platform.startswith("linux"):
platform = "linux"
elif sys.platform == "darwin":
platform = "macos"
elif sys.platform == "win32":
platform = "windows"
else:
print(
'cibuildwheel: Unable to detect platform from "sys.platform" in a CI environment. You can run '
"cibuildwheel using the --platform argument. Check --help output for more information.",
file=sys.stderr,
)
sys.exit(2)
if platform not in PLATFORMS:
print(f"cibuildwheel: Unsupported platform: {platform}", file=sys.stderr)
sys.exit(2)
package_dir = Path(args.package_dir)
output_dir = Path(args.output_dir)
if platform == "linux":
repair_command_default = "auditwheel repair -w {dest_dir} {wheel}"
elif platform == "macos":
repair_command_default = "delocate-listdeps {wheel} && delocate-wheel --require-archs {delocate_archs} -w {dest_dir} {wheel}"
elif platform == "windows":
repair_command_default = ""
else:
assert_never(platform)
build_config = os.environ.get("CIBW_BUILD") or "*"
skip_config = os.environ.get("CIBW_SKIP", "")
test_skip = os.environ.get("CIBW_TEST_SKIP", "")
environment_config = get_option_from_environment(
"CIBW_ENVIRONMENT", platform=platform, default=""
)
before_all = get_option_from_environment("CIBW_BEFORE_ALL", platform=platform, default="")
before_build = get_option_from_environment("CIBW_BEFORE_BUILD", platform=platform)
repair_command = get_option_from_environment(
"CIBW_REPAIR_WHEEL_COMMAND", platform=platform, default=repair_command_default
)
dependency_versions = get_option_from_environment(
"CIBW_DEPENDENCY_VERSIONS", platform=platform, default="pinned"
)
test_command = get_option_from_environment("CIBW_TEST_COMMAND", platform=platform)
before_test = get_option_from_environment("CIBW_BEFORE_TEST", platform=platform)
test_requires = get_option_from_environment(
"CIBW_TEST_REQUIRES", platform=platform, default=""
).split()
test_extras = get_option_from_environment("CIBW_TEST_EXTRAS", platform=platform, default="")
build_verbosity_str = get_option_from_environment(
"CIBW_BUILD_VERBOSITY", platform=platform, default=""
)
args = CommandLineArguments(**vars(parser.parse_args()))
global_options.print_traceback_on_error = args.debug_traceback
if args.clean_cache:
if not CIBW_CACHE_PATH.exists():
print(f"Cache directory does not exist: {CIBW_CACHE_PATH}")
sys.exit(0)
sentinel_file = CIBW_CACHE_PATH / "CACHEDIR.TAG"
if not sentinel_file.exists():
print(
f"Error: {CIBW_CACHE_PATH} does not appear to be a cibuildwheel cache directory.",
"Only directories with a CACHEDIR.TAG sentinel file can be cleaned.",
sep="\n",
file=sys.stderr,
)
sys.exit(1)
# Verify signature to ensure it's a proper cache dir
# See https://bford.info/cachedir/ for more
try:
sentinel_content = sentinel_file.read_text(encoding="utf-8")
except OSError as e:
print(f"Error reading cache directory tag: {e}", file=sys.stderr)
sys.exit(1)
if not sentinel_content.startswith("Signature: 8a477f597d28d172789f06886806bc55"):
print(
f"Error: {sentinel_file} does not contain a valid cache directory signature.",
"For safety, only properly signed cache directories can be cleaned.",
sep="\n",
file=sys.stderr,
)
sys.exit(1)
print(f"Clearing cache directory: {CIBW_CACHE_PATH}")
try:
shutil.rmtree(CIBW_CACHE_PATH)
print("Cache cleared successfully.")
except OSError as e:
print(f"Error clearing cache: {e}", file=sys.stderr)
sys.exit(1)
sys.exit(0)
args.package_dir = args.package_dir.resolve()
# This are always relative to the base directory, even in SDist builds
args.output_dir = args.output_dir.resolve()
# Standard builds if a directory or non-existent path is given
if not args.package_dir.is_file() and not args.package_dir.name.endswith("tar.gz"):
build_in_directory(args)
return
# Tarfile builds require extraction and changing the directory
temp_dir = Path(mkdtemp(prefix="cibw-sdist-")).resolve(strict=True)
try:
with TarFile.open(args.package_dir) as tar:
safe_extractall(tar, temp_dir)
# The extract directory is now the project dir
try:
(project_dir,) = temp_dir.iterdir()
except ValueError:
msg = "invalid sdist: didn't contain a single dir"
raise SystemExit(msg) from None
# This is now the new package dir
args.package_dir = project_dir.resolve()
with contextlib.chdir(project_dir):
build_in_directory(args)
finally:
# avoid https://github.com/python/cpython/issues/86962 by performing
# cleanup manually
shutil.rmtree(temp_dir, ignore_errors=sys.platform.startswith("win"))
if temp_dir.exists():
log.warning(f"Can't delete temporary folder '{temp_dir}'")
def _compute_platform_only(only: str) -> PlatformName:
if "linux_" in only:
return "linux"
if "macosx_" in only:
return "macos"
if "win_" in only or "win32" in only:
return "windows"
if "pyodide_" in only:
return "pyodide"
if "android_" in only:
return "android"
if "ios_" in only:
return "ios"
msg = f"Invalid --only='{only}', must be a build selector with a known platform"
raise errors.ConfigurationError(msg)
def _compute_platform(args: CommandLineArguments) -> PlatformName:
platform_option_value = args.platform or os.environ.get("CIBW_PLATFORM", "") or "auto"
if args.only and args.platform is not None:
msg = "--platform cannot be specified with --only, it is computed from --only"
raise errors.ConfigurationError(msg)
if args.only and args.archs is not None:
msg = "--arch cannot be specified with --only, it is computed from --only"
raise errors.ConfigurationError(msg)
if platform_option_value not in PLATFORMS | {"auto"}:
msg = f"Unsupported platform: {platform_option_value}"
raise errors.ConfigurationError(msg)
if args.only:
return _compute_platform_only(args.only)
elif platform_option_value != "auto":
return typing.cast("PlatformName", platform_option_value)
return native_platform()
def build_in_directory(args: CommandLineArguments) -> None:
platform: PlatformName = _compute_platform(args)
if platform == "pyodide" and sys.platform == "win32":
msg = "Building for pyodide is not supported on Windows"
raise errors.ConfigurationError(msg)
options = compute_options(platform=platform, command_line_arguments=args, env=os.environ)
package_dir = options.globals.package_dir
package_files = {"setup.py", "setup.cfg", "pyproject.toml"}
if not any(package_dir.joinpath(name).exists() for name in package_files):
names = ", ".join(sorted(package_files, reverse=True))
msg = f"Could not find any of {{{names}}} at root of package"
raise errors.ConfigurationError(msg)
print(
f"cibuildwheel: Could not find any of {{{names}}} at root of package", file=sys.stderr
)
sys.exit(2)
platform_module = ALL_PLATFORM_MODULES[platform]
identifiers = get_build_identifiers(
platform_module=platform_module,
build_selector=options.globals.build_selector,
architectures=options.globals.architectures,
# Passing this in as an environment variable will override pyproject.toml, setup.cfg, or setup.py
requires_python_str: Optional[str] = os.environ.get(
"CIBW_PROJECT_REQUIRES_PYTHON"
) or get_requires_python_str(package_dir)
requires_python = None if requires_python_str is None else SpecifierSet(requires_python_str)
build_selector = BuildSelector(
build_config=build_config, skip_config=skip_config, requires_python=requires_python
)
test_selector = TestSelector(skip_config=test_skip)
try:
environment = parse_environment(environment_config)
except (EnvironmentParseError, ValueError):
print(f'cibuildwheel: Malformed environment option "{environment_config}"', file=sys.stderr)
traceback.print_exc(None, sys.stderr)
sys.exit(2)
if dependency_versions == "pinned":
dependency_constraints: Optional[
DependencyConstraints
] = DependencyConstraints.with_defaults()
elif dependency_versions == "latest":
dependency_constraints = None
else:
dependency_versions_path = Path(dependency_versions)
dependency_constraints = DependencyConstraints(dependency_versions_path)
if test_extras:
test_extras = f"[{test_extras}]"
try:
build_verbosity = min(3, max(-3, int(build_verbosity_str)))
except ValueError:
build_verbosity = 0
# Add CIBUILDWHEEL environment variable
# This needs to be passed on to the docker container in linux.py
os.environ["CIBUILDWHEEL"] = "1"
if args.archs is not None:
archs_config_str = args.archs
else:
archs_config_str = get_option_from_environment(
"CIBW_ARCHS", platform=platform, default="auto"
)
archs = Architecture.parse_config(archs_config_str, platform=platform)
identifiers = get_build_identifiers(platform, build_selector, archs)
if args.print_build_identifiers:
for identifier in identifiers:
print(identifier)
sys.exit(0)
# Add CIBUILDWHEEL environment variable
os.environ["CIBUILDWHEEL"] = "1"
manylinux_images: Optional[Dict[str, str]] = None
if platform == "linux":
pinned_docker_images_file = resources_dir / "pinned_docker_images.cfg"
all_pinned_docker_images = ConfigParser()
all_pinned_docker_images.read(pinned_docker_images_file)
# all_pinned_docker_images looks like a dict of dicts, e.g.
# { 'x86_64': {'manylinux1': '...', 'manylinux2010': '...', 'manylinux2014': '...'},
# 'i686': {'manylinux1': '...', 'manylinux2010': '...', 'manylinux2014': '...'},
# 'pypy_x86_64': {'manylinux2010': '...' }
# ... }
# Python block-buffers stdout when it isn't a tty, which de-interleaves
# `print` output and direct fd-1 writes from subprocesses sharing this
# stdout. line_buffering allows each newline to flush all the way to fd-1.
if isinstance(sys.stdout, io.TextIOWrapper):
sys.stdout.reconfigure(line_buffering=True)
manylinux_images = {}
# create the cache dir before it gets printed & builds performed
CIBW_CACHE_PATH.mkdir(parents=True, exist_ok=True)
ensure_cache_sentinel(CIBW_CACHE_PATH)
for build_platform in ["x86_64", "i686", "pypy_x86_64", "aarch64", "ppc64le", "s390x"]:
pinned_images = all_pinned_docker_images[build_platform]
print_preamble(platform=platform, options=options, identifiers=identifiers)
config_name = f"CIBW_MANYLINUX_{build_platform.upper()}_IMAGE"
config_value = os.environ.get(config_name)
if config_value is None:
# default to manylinux2010 if it's available, otherwise manylinux2014
image = pinned_images.get("manylinux2010") or pinned_images.get("manylinux2014")
elif config_value in pinned_images:
image = pinned_images[config_value]
else:
image = config_value
manylinux_images[build_platform] = image
build_options = BuildOptions(
architectures=archs,
package_dir=package_dir,
output_dir=output_dir,
test_command=test_command,
test_requires=test_requires,
test_extras=test_extras,
before_test=before_test,
before_build=before_build,
before_all=before_all,
build_verbosity=build_verbosity,
build_selector=build_selector,
test_selector=test_selector,
repair_command=repair_command,
environment=environment,
dependency_constraints=dependency_constraints,
manylinux_images=manylinux_images,
)
# Python is buffering by default when running on the CI platforms, giving problems interleaving subprocess call output with unflushed calls to 'print'
sys.stdout = Unbuffered(sys.stdout) # type: ignore
print_preamble(platform, build_options)
try:
options.check_for_invalid_configuration(identifiers)
allowed_architectures_check(platform, options.globals.architectures)
allowed_architectures_check(platform, build_options.architectures)
except ValueError as err:
raise errors.DeprecationError(*err.args) from err
print("cibuildwheel:", *err.args, file=sys.stderr)
sys.exit(4)
if not identifiers:
message = f"No build identifiers selected: {options.globals.build_selector}"
if options.globals.allow_empty:
print(f"cibuildwheel: {message}", file=sys.stderr)
print(f"cibuildwheel: No build identifiers selected: {build_selector}", file=sys.stderr)
if not args.allow_empty:
sys.exit(3)
if not output_dir.exists():
output_dir.mkdir(parents=True)
with cibuildwheel.util.print_new_wheels(
"\n{n} wheels produced in {m:.0f} minutes:", output_dir
):
if platform == "linux":
cibuildwheel.linux.build(build_options)
elif platform == "windows":
cibuildwheel.windows.build(build_options)
elif platform == "macos":
cibuildwheel.macos.build(build_options)
else:
raise errors.NothingToDoError(message)
output_dir = options.globals.output_dir
output_dir.mkdir(parents=True, exist_ok=True)
tmp_path = Path(mkdtemp(prefix="cibw-run-")).resolve(strict=True)
try:
with log.print_summary(options=options):
platform_module.build(options, tmp_path)
finally:
# avoid https://github.com/python/cpython/issues/86962 by performing
# cleanup manually
shutil.rmtree(tmp_path, ignore_errors=sys.platform.startswith("win"))
if tmp_path.exists():
log.warning(f"Can't delete temporary folder '{tmp_path}'")
assert_never(platform)
def print_preamble(platform: str, options: Options, identifiers: Sequence[str]) -> None:
def detect_obsolete_options() -> None:
# Check the old 'MANYLINUX1_*_IMAGE' options
for (deprecated, alternative) in [
("CIBW_MANYLINUX1_X86_64_IMAGE", "CIBW_MANYLINUX_X86_64_IMAGE"),
("CIBW_MANYLINUX1_I686_IMAGE", "CIBW_MANYLINUX_I686_IMAGE"),
]:
if deprecated in os.environ:
print(
f"'{deprecated}' has been deprecated, and will be removed in a future release. Use the option '{alternative}' instead."
)
if alternative not in os.environ:
print(f"Using value of option '{deprecated}' as replacement for '{alternative}'")
os.environ[alternative] = os.environ[deprecated]
else:
print(f"Option '{alternative}' is not empty. Please unset '{deprecated}'")
sys.exit(2)
# Check for deprecated identifiers in 'CIBW_BUILD' and 'CIBW_SKIP' options
for option in ["CIBW_BUILD", "CIBW_SKIP"]:
for deprecated, alternative in [
("manylinux1", "manylinux"),
("macosx_10_6_intel", "macosx_x86_64"),
("macosx_10_9_x86_64", "macosx_x86_64"),
]:
if option in os.environ and deprecated in os.environ[option]:
print(
f"Build identifiers with '{deprecated}' have been deprecated. Replacing all occurences of '{deprecated}' with '{alternative}' in the option '{option}'"
)
os.environ[option] = os.environ[option].replace(deprecated, alternative)
def print_preamble(platform: str, build_options: BuildOptions) -> None:
print(
textwrap.dedent(
"""
@@ -425,146 +398,59 @@ def print_preamble(platform: str, options: Options, identifiers: Sequence[str])
print(f"cibuildwheel version {cibuildwheel.__version__}\n")
print("Build options:")
print(f" platform: {platform}")
options_summary = textwrap.indent(options.summary(identifiers), " ")
if detect_ci_provider() == CIProvider.github_actions:
options_summary = fix_ansi_codes_for_github_actions(options_summary)
print(options_summary)
print(f" platform: {platform!r}")
for option, value in sorted(build_options._asdict().items()):
print(f" {option}: {value!r}")
print()
print(f"Cache folder: {CIBW_CACHE_PATH}")
print()
warnings = detect_warnings(platform, build_options)
if warnings:
print("\nWarnings:")
for warning in warnings:
print(" " + warning)
warnings = detect_warnings(options=options)
for warning in warnings:
log.warning(warning)
error_list = list(detect_errors(options=options, identifiers=identifiers))
if error_list:
for error in error_list:
log.error(error)
msg = "\n".join(error_list)
raise errors.ConfigurationError(msg)
n = len(identifiers)
print(f"{n} build{'s' if n != 1 else ''} selected:")
print(", ".join(identifiers))
print()
print("Here we go!\n")
print("\nHere we go!\n")
def detect_errors(*, options: Options, identifiers: Iterable[str]) -> Generator[str, None, None]:
# Check for deprecated CIBW_FREE_THREADED_SUPPORT environment variable
if "CIBW_FREE_THREADED_SUPPORT" in os.environ:
yield "CIBW_FREE_THREADED_SUPPORT environment variable is no longer supported."
def get_build_identifiers(
platform: PlatformName, build_selector: BuildSelector, architectures: Set[Architecture]
) -> List[str]:
python_configurations: Union[
List[cibuildwheel.linux.PythonConfiguration],
List[cibuildwheel.windows.PythonConfiguration],
List[cibuildwheel.macos.PythonConfiguration],
]
# Deprecated {python} and {pip}
for option_name in ["test_command", "before_build"]:
option_values = [getattr(options.build_options(i), option_name) for i in identifiers]
if any(o and ("{python}" in o or "{pip}" in o) for o in option_values):
# Reminder: in an f-string, double braces means literal single brace
yield (
f"{option_name}: '{{python}}' and '{{pip}}' are no longer supported "
"and have been removed in cibuildwheel 3. Simply use 'python' or 'pip' instead."
)
def detect_warnings(*, options: Options) -> Generator[str, None, None]:
python_version_deprecation = ((3, 11), 3)
if sys.version_info[:2] < python_version_deprecation[0]:
python_version = ".".join(map(str, python_version_deprecation[0]))
yield (
f"cibuildwheel {python_version_deprecation[1]} will require Python {python_version}+, "
"please upgrade the Python version used to run cibuildwheel. "
"This does not affect the versions you can target when building wheels. See: https://cibuildwheel.pypa.io/en/stable/#what-does-it-do"
if platform == "linux":
python_configurations = cibuildwheel.linux.get_python_configurations(
build_selector, architectures
)
elif platform == "windows":
python_configurations = cibuildwheel.windows.get_python_configurations(
build_selector, architectures
)
elif platform == "macos":
python_configurations = cibuildwheel.macos.get_python_configurations(
build_selector, architectures
)
else:
assert_never(platform)
build_selector = options.globals.build_selector
test_selector = options.globals.test_selector
all_valid_identifiers = [
config.identifier
for module in ALL_PLATFORM_MODULES.values()
for config in module.all_python_configurations()
]
enabled_selector = BuildSelector(
build_config="*", skip_config="", enable=options.globals.build_selector.enable
)
all_enabled_identifiers = [
identifier for identifier in all_valid_identifiers if enabled_selector(identifier)
]
yield from check_for_invalid_selectors(
selector_name="build",
selector_value=build_selector.build_config,
all_valid_identifiers=all_valid_identifiers,
all_enabled_identifiers=all_enabled_identifiers,
)
yield from check_for_invalid_selectors(
selector_name="skip",
selector_value=build_selector.skip_config,
all_valid_identifiers=all_valid_identifiers,
all_enabled_identifiers=all_enabled_identifiers,
)
yield from check_for_invalid_selectors(
selector_name="test_skip",
selector_value=test_selector.skip_config,
all_valid_identifiers=all_valid_identifiers,
all_enabled_identifiers=all_enabled_identifiers,
)
return [config.identifier for config in python_configurations]
def check_for_invalid_selectors(
*,
selector_name: Literal["build", "skip", "test_skip"],
selector_value: str,
all_valid_identifiers: Sequence[str],
all_enabled_identifiers: Sequence[str],
) -> list[str]:
def detect_warnings(platform: str, build_options: BuildOptions) -> List[str]:
warnings = []
for selector in selector_value.split():
selector_ = selector
if selector_name == "test_skip":
# macosx_universal2 uses an additional identifier for tests which ends with ":{arch}"
values = selector.split(":")
universal2_identifiers = filter(
lambda x: x.endswith("-macosx_universal2"), all_valid_identifiers
# warn about deprecated {python} and {pip}
for option_name in ["test_command", "before_build"]:
option_value = getattr(build_options, option_name)
if option_value and ("{python}" in option_value or "{pip}" in option_value):
# Reminder: in an f-string, double braces means literal single brace
msg = (
f"{option_name}: '{{python}}' and '{{pip}}' are no longer needed, "
"and will be removed in a future release. Simply use 'python' or 'pip' instead."
)
if len(values) == 2 and any(
selector_matches(selector_, f"{i}:{arch}")
for i in universal2_identifiers
for arch in ["arm64", "x86_64"]
):
# just ignore the arch part in the rest of the check
selector_ = values[0]
if not any(selector_matches(selector_, i) for i in all_enabled_identifiers):
msg = f"Invalid {selector_name} selector: {selector!r}. "
error_type: type = errors.ConfigurationError
if any(selector_matches(selector_, i) for i in all_valid_identifiers):
msg += "This selector matches a group that wasn't enabled. Enable it using the `enable` option or remove this selector. "
if "p2" in selector_ or "p35" in selector_:
msg += f"cibuildwheel 4.x no longer supports Python < 3.9. Please use the 1.x series or update `{selector_name}`. "
error_type = errors.DeprecationError
if "p36" in selector_ or "p37" in selector_:
msg += f"cibuildwheel 4.x no longer supports Python < 3.9. Please use the 2.x series or update `{selector_name}`. "
error_type = errors.DeprecationError
if "p38" in selector_:
msg += f"cibuildwheel 4.x no longer supports Python < 3.9. Please use the 3.x series or update `{selector_name}`. "
error_type = errors.DeprecationError
if "cp313t" in selector_:
msg += f"cibuildwheel 4.x no longer supports Python 3.13 free-threading. Please use the 3.x series or update `{selector_name}`. "
error_type = errors.DeprecationError
if selector_name == "build":
raise error_type(msg)
msg += "This selector will have no effect. "
warnings.append(msg)
return warnings
View File
-71
View File
@@ -1,71 +0,0 @@
# Based on https://github.com/pypa/build/blob/f4ebd495cc0c2c74155bd4fe48b76399fb7927ac/src/build/_compat/tarfile.py
from __future__ import annotations
import sys
import tarfile
TYPE_CHECKING = False
if TYPE_CHECKING:
from pathlib import Path
TarFile = tarfile.TarFile
# Per https://peps.python.org/pep-0706/, the "data" filter will become
# the default in Python 3.14. The first series of releases with the filter
# had a broken filter that could not process symlinks correctly.
elif (3, 11, 5) <= sys.version_info < (3, 14):
class TarFile(tarfile.TarFile): # pragma: no cover
extraction_filter = staticmethod(tarfile.data_filter)
else:
TarFile = tarfile.TarFile # pragma: no cover
# Same availability matrix as the TarFile subclass above. On runtimes that
# ship the stdlib ``data`` filter we delegate to it; the fallback branch is
# only reached on 3.10.0-3.10.12 / 3.11.0-3.11.4 and validates each member
# manually before extraction.
if sys.version_info >= (3, 11, 5):
def safe_extractall(tar: tarfile.TarFile, path: Path) -> None: # pragma: no cover
"""Extract every member of ``tar`` into ``path`` via the PEP 706 ``data`` filter."""
tar.extractall(path, filter="data")
else:
def safe_extractall(tar: tarfile.TarFile, path: Path) -> None: # pragma: no cover
"""Validate every member of ``tar``, then extract into ``path``.
Reached on 3.10.0-3.10.12 / 3.11.0-3.11.4 where the stdlib ``data`` filter is missing. Device or special files,
paths that escape ``path``, and symlinks/hardlinks whose targets resolve outside ``path`` are rejected before
any write hits the disk.
"""
base = path.resolve()
for member in tar.getmembers():
_validate_safe_member(member, base)
tar.extractall(path)
def _validate_safe_member(member: tarfile.TarInfo, base: Path) -> None:
if member.ischr() or member.isblk() or member.isfifo():
msg = f"refusing to extract special device file {member.name!r}"
raise tarfile.TarError(msg)
target = (base / member.name).resolve(strict=False)
if not target.is_relative_to(base):
msg = f"refusing to extract {member.name!r}: path escapes destination"
raise tarfile.TarError(msg)
if member.issym() or member.islnk():
link_base = target.parent if member.issym() else base
link_target = (link_base / member.linkname).resolve(strict=False)
if not link_target.is_relative_to(base):
msg = f"refusing to extract {member.name!r}: link target escapes destination"
raise tarfile.TarError(msg)
__all__ = [
"TarFile",
"safe_extractall",
]
+73 -212
View File
@@ -1,249 +1,110 @@
from __future__ import annotations
__lazy_modules__ = {"platform", "re", "shutil", "subprocess"}
import functools
import platform as platform_module
import re
import shutil
import subprocess
import sys
import typing
from enum import StrEnum, auto
from enum import Enum
from typing import Set
from cibuildwheel import errors
from .typing import Literal, PlatformName, assert_never
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Set
from typing import Final, Literal, Self
from cibuildwheel.typing import PlatformName
PRETTY_NAMES: Final[dict[PlatformName, str]] = {
"linux": "Linux",
"macos": "macOS",
"windows": "Windows",
"pyodide": "Pyodide",
"android": "Android",
"ios": "iOS",
}
ARCH_SYNONYMS: Final[list[dict[PlatformName, str | None]]] = [
{"linux": "x86_64", "macos": "x86_64", "windows": "AMD64", "android": "x86_64"},
{"linux": "i686", "macos": None, "windows": "x86"},
{"linux": "aarch64", "macos": "arm64", "windows": "ARM64", "android": "arm64_v8a"},
]
PRETTY_NAMES = {"linux": "Linux", "macos": "macOS", "windows": "Windows"}
def arch_synonym(arch: str, from_platform: PlatformName, to_platform: PlatformName) -> str | None:
for arch_synonym_ in ARCH_SYNONYMS:
if arch == arch_synonym_.get(from_platform):
return arch_synonym_.get(to_platform, arch)
@functools.total_ordering
class Architecture(Enum):
value: str
return arch
def _check_aarch32_el0() -> bool:
"""Check if running armv7l natively on aarch64 is supported"""
if not sys.platform.startswith("linux"):
return False
if platform_module.machine() != "aarch64":
return False
executable = shutil.which("linux32")
if executable is None:
return False
check = subprocess.run([executable, "uname", "-m"], check=False, capture_output=True, text=True)
return check.returncode == 0 and check.stdout.startswith("armv")
@typing.final
class Architecture(StrEnum):
# mac/linux/android archs
x86_64 = auto()
# mac/linux archs
x86_64 = "x86_64"
# linux archs
i686 = auto()
aarch64 = auto()
ppc64le = auto()
s390x = auto()
armv7l = auto()
riscv64 = auto()
i686 = "i686"
aarch64 = "aarch64"
ppc64le = "ppc64le"
s390x = "s390x"
# mac archs
universal2 = auto()
arm64 = auto()
universal2 = "universal2"
arm64 = "arm64"
# windows archs
x86 = auto()
x86 = "x86"
AMD64 = "AMD64"
ARM64 = "ARM64"
# WebAssembly
wasm32 = auto()
# Allow this to be sorted
def __lt__(self, other: "Architecture") -> bool:
return self.value < other.value
# android archs
arm64_v8a = auto()
# iOS "multiarch" architectures that include both
# the CPU architecture and the ABI.
arm64_iphoneos = auto()
arm64_iphonesimulator = auto()
x86_64_iphonesimulator = auto()
@classmethod
def parse_config(cls, config: str, platform: PlatformName) -> set[Self]:
@staticmethod
def parse_config(config: str, platform: PlatformName) -> "Set[Architecture]":
result = set()
for arch_str in re.split(r"[\s,]+", config):
match arch_str:
case "auto":
result |= cls.auto_archs(platform=platform)
case "native":
if native_arch := cls.native_arch(platform=platform):
result.add(native_arch)
case "all":
result |= cls.all_archs(platform=platform)
case "auto64":
result |= cls.bitness_archs(platform=platform, bitness="64")
case "auto32":
result |= cls.bitness_archs(platform=platform, bitness="32")
case _:
if arch := cls._parse_arch_name(arch_str, platform=platform):
result.add(arch)
else:
msg = f"Invalid architecture '{arch_str}'"
raise errors.ConfigurationError(msg)
return result
@classmethod
def _parse_arch_name(cls, arch_str: str, platform: PlatformName) -> Self | None:
"""Resolve an architecture name case-insensitively.
The same value (e.g. "arm64"/"ARM64") can map to different members
depending on the platform, so prefer a member valid for ``platform``
before falling back to any case-insensitive match.
"""
for arch in cls.all_archs(platform):
if arch.value.lower() == arch_str.lower():
return arch
for arch in cls:
if arch.value.lower() == arch_str.lower():
return arch
return None
@classmethod
def native_arch(cls, platform: PlatformName) -> Self | None:
native_machine = platform_module.machine()
native_architecture = cls(native_machine)
# Cross-platform support. Used for --print-build-identifiers or docker builds.
host_platform: PlatformName = (
"windows"
if sys.platform.startswith("win")
else ("macos" if sys.platform.startswith("darwin") else "linux")
)
if platform == "pyodide":
return cls.wasm32
elif platform == "ios":
# Can only build for iOS on macOS. The "native" architecture is the
# simulator for the macOS native platform.
if host_platform == "macos":
if native_architecture == cls.x86_64:
return cls.x86_64_iphonesimulator
else:
return cls.arm64_iphonesimulator
if arch_str == "auto":
result |= Architecture.auto_archs(platform=platform)
elif arch_str == "native":
result.add(Architecture(platform_module.machine()))
elif arch_str == "all":
result |= Architecture.all_archs(platform=platform)
elif arch_str == "auto64":
result |= Architecture.bitness_archs(platform=platform, bitness="64")
elif arch_str == "auto32":
result |= Architecture.bitness_archs(platform=platform, bitness="32")
else:
return None
result.add(Architecture(arch_str))
return result
# we might need to rename the native arch to the machine we're running
# on, as the same arch can have different names on different platforms
if host_platform != platform:
synonym = arch_synonym(native_machine, host_platform, platform)
if synonym is None:
# can't build anything on this platform
return None
@staticmethod
def auto_archs(platform: PlatformName) -> "Set[Architecture]":
native_architecture = Architecture(platform_module.machine())
result = {native_architecture}
native_architecture = cls(synonym)
if platform == "linux" and native_architecture == Architecture.x86_64:
# x86_64 machines can run i686 docker containers
result.add(Architecture.i686)
return native_architecture
if platform == "windows" and native_architecture == Architecture.AMD64:
result.add(Architecture.x86)
@classmethod
def auto_archs(cls, platform: PlatformName) -> set[Self]:
native_arch = cls.native_arch(platform)
if native_arch is None:
return set() # can't build anything on this platform
result = {native_arch}
match platform:
case "windows" if cls.AMD64 in result:
result.add(cls.x86)
case "ios" if native_arch == cls.arm64_iphonesimulator:
# Also build the device wheel if we're on ARM64.
result.add(cls.arm64_iphoneos)
if platform == "macos" and native_architecture == Architecture.arm64:
# arm64 can build and test both archs of a universal2 wheel.
result.add(Architecture.universal2)
return result
@classmethod
def all_archs(cls, platform: PlatformName) -> set[Self]:
all_archs_map = {
"linux": {
cls.x86_64,
cls.i686,
cls.aarch64,
cls.ppc64le,
cls.s390x,
cls.armv7l,
cls.riscv64,
},
"macos": {cls.x86_64, cls.arm64, cls.universal2},
"windows": {cls.x86, cls.AMD64, cls.ARM64},
"pyodide": {cls.wasm32},
"android": {cls.x86_64, cls.arm64_v8a},
"ios": {
cls.x86_64_iphonesimulator,
cls.arm64_iphonesimulator,
cls.arm64_iphoneos,
},
}
return all_archs_map[platform]
@staticmethod
def all_archs(platform: PlatformName) -> "Set[Architecture]":
if platform == "linux":
return {
Architecture.x86_64,
Architecture.i686,
Architecture.aarch64,
Architecture.ppc64le,
Architecture.s390x,
}
elif platform == "macos":
return {Architecture.x86_64, Architecture.arm64, Architecture.universal2}
elif platform == "windows":
return {Architecture.x86, Architecture.AMD64}
else:
assert_never(platform)
@classmethod
def bitness_archs(cls, platform: PlatformName, bitness: Literal["64", "32"]) -> set[Self]:
# This map maps 64-bit architectures to their 32-bit equivalents.
archs_map = {
cls.x86_64: cls.i686,
cls.AMD64: cls.x86,
cls.aarch64: cls.armv7l,
}
native_arch = cls.native_arch(platform)
@staticmethod
def bitness_archs(platform: PlatformName, bitness: Literal["64", "32"]) -> "Set[Architecture]":
archs_32 = {Architecture.i686, Architecture.x86}
auto_archs = Architecture.auto_archs(platform)
if native_arch is None:
return set() # can't build anything on this platform
if native_arch == cls.wasm32:
return {native_arch} if bitness == "32" else set()
match bitness:
case "64":
return {native_arch} if native_arch not in archs_map.values() else set()
case "32":
if native_arch in archs_map.values():
return {native_arch}
elif native_arch in archs_map and platform in {"linux", "windows"}:
if native_arch == cls.aarch64 and not _check_aarch32_el0():
# If we're on aarch64, skip if we cannot build armv7l wheels.
return set()
return {archs_map[native_arch]}
else:
return set()
case _:
typing.assert_never(bitness)
if bitness == "64":
return auto_archs - archs_32
elif bitness == "32":
return auto_archs & archs_32
else:
assert_never(bitness)
def allowed_architectures_check(
platform: PlatformName,
architectures: Set[Architecture],
) -> None:
allowed_architectures = Architecture.all_archs(platform)
msg = f"{PRETTY_NAMES[platform]} only supports {sorted(allowed_architectures)} at the moment."
-148
View File
@@ -1,148 +0,0 @@
from __future__ import annotations
__lazy_modules__ = {
"cibuildwheel.logger",
"cibuildwheel.util",
"cibuildwheel.util.cmd",
"cibuildwheel.util.helpers",
"cibuildwheel.util.packaging",
"cibuildwheel.venv",
"pathlib",
"subprocess",
}
import subprocess
import sys
from pathlib import Path
from cibuildwheel import errors
from cibuildwheel.logger import log
from cibuildwheel.util.cmd import call, shell
from cibuildwheel.util.helpers import prepare_command
from cibuildwheel.util.packaging import is_abi3_wheel
from cibuildwheel.venv import activate_virtualenv, find_uv, virtualenv
TYPE_CHECKING = False
if TYPE_CHECKING:
from cibuildwheel.options import BuildOptions
def run_audit(
*,
tmp_dir: Path,
build_options: BuildOptions,
wheel: Path,
) -> None:
"""
Run the audit commands on a single wheel.
Creates a virtualenv (or reuses an existing one) and installs any
audit requirements, then runs each audit command template against
the wheel. Commands containing {abi3_wheel} are skipped for
non-abi3 wheels.
"""
if not needs_audit(build_options.audit_command, wheel.name):
return
log.step("Auditing wheel...")
use_uv = build_options.build_frontend.name in {"build[uv]", "uv"}
version = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}"
dependency_constraint = build_options.dependency_constraints.get_for_python_version(
version=version, tmp_dir=tmp_dir
)
# Use the base interpreter, not the venv python, to avoid nested-venv
# issues where pip can't be found (seen on Windows + Python 3.13).
host_python = Path(getattr(sys, "_base_executable", sys.executable))
audit_venv_dir = tmp_dir / "audit_venv"
if not (audit_venv_dir / "pyvenv.cfg").exists():
env = virtualenv(
version,
host_python,
audit_venv_dir,
dependency_constraint=dependency_constraint,
use_uv=use_uv,
)
else:
env = activate_virtualenv(audit_venv_dir)
# install audit requirements. This is run every time in case the user has
# defined overrides.
audit_requires = build_options.audit_requires
if audit_requires:
print(f"Installing audit dependencies: {', '.join(audit_requires)}")
pip: list[str]
if use_uv:
uv_path = find_uv()
assert uv_path is not None
pip = [str(uv_path), "pip"]
else:
pip = ["pip"]
# we pin if the audit-requires is left as the default "abi3audit"
should_pin = audit_requires == ["abi3audit"] and dependency_constraint
call(
*pip,
"install",
*(["--constraint", str(dependency_constraint)] if should_pin else []),
*audit_requires,
env=env,
)
audit_command = build_options.audit_command
for command_template in audit_command:
if "{abi3_wheel}" in command_template and "{wheel}" in command_template:
msg = (
f"Invalid audit command {command_template!r}: cannot contain both {{abi3_wheel}} "
"and {{wheel}} placeholders"
)
raise errors.ConfigurationError(msg)
if "{abi3_wheel}" in command_template and not is_abi3_wheel(wheel.name):
continue
prepared_command = prepare_command(
command_template,
abi3_wheel=wheel,
wheel=wheel,
project=".",
package=build_options.package_dir,
)
print(f"Running audit command: {prepared_command}")
try:
shell(prepared_command, env=env)
except subprocess.CalledProcessError as e:
print(f"Audit command failed with exit code {e.returncode}")
msg = f"Audit command failed: {prepared_command}"
raise errors.AuditCommandFailedError(msg) from e
def needs_audit(audit_commands: list[str], wheel_name: str) -> bool:
saw_abi3_placeholder = False
for audit_command in audit_commands:
if "{abi3_wheel}" not in audit_command and "{wheel}" not in audit_command:
msg = (
f"Invalid audit command {audit_command!r}: must contain either "
"{{abi3_wheel}} or {{wheel}} placeholder"
)
raise errors.ConfigurationError(msg)
if "{abi3_wheel}" in audit_command:
saw_abi3_placeholder = True
if is_abi3_wheel(wheel_name):
return True
elif "{wheel}" in audit_command:
return True
if saw_abi3_placeholder:
print("No audit required for this wheel, as it is not abi3")
else:
print("No audit configured")
return False
+20 -36
View File
@@ -1,38 +1,26 @@
from __future__ import annotations
__lazy_modules__ = {"bashlex", "subprocess"}
import dataclasses
import subprocess
from typing import Callable, Dict, List, NamedTuple, Optional, Sequence
import bashlex
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import (
Callable,
Iterable,
Mapping,
Sequence,
)
# a function that takes a command and the environment, and returns the result
EnvironmentExecutor = Callable[[list[str], dict[str, str]], str]
# a function that takes a command and the environment, and returns the result
EnvironmentExecutor = Callable[[List[str], Dict[str, str]], str]
def local_environment_executor(command: Sequence[str], env: Mapping[str, str]) -> str:
return subprocess.run(command, env=env, text=True, stdout=subprocess.PIPE, check=True).stdout
def local_environment_executor(command: List[str], env: Dict[str, str]) -> str:
return subprocess.run(
command, env=env, universal_newlines=True, stdout=subprocess.PIPE, check=True
).stdout
@dataclasses.dataclass(frozen=True, kw_only=True)
class NodeExecutionContext:
environment: dict[str, str]
class NodeExecutionContext(NamedTuple):
environment: Dict[str, str]
input: str
executor: EnvironmentExecutor
def evaluate(
value: str, environment: Mapping[str, str], executor: EnvironmentExecutor | None = None
value: str, environment: Dict[str, str], executor: Optional[EnvironmentExecutor] = None
) -> str:
if not value:
# empty string evaluates to empty string
@@ -42,17 +30,14 @@ def evaluate(
command_node = bashlex.parsesingle(value)
if len(command_node.parts) != 1:
msg = f"{value!r} has too many parts"
raise ValueError(msg)
raise ValueError(f'"{value}" has too many parts')
value_word_node = command_node.parts[0]
return evaluate_node(
value_word_node,
context=NodeExecutionContext(
environment=dict(environment),
input=value,
executor=executor or local_environment_executor,
environment=environment, input=value, executor=executor or local_environment_executor
),
)
@@ -67,8 +52,7 @@ def evaluate_node(node: bashlex.ast.node, context: NodeExecutionContext) -> str:
elif node.kind == "parameter":
return evaluate_parameter_node(node, context=context)
else:
msg = f"Unsupported bash construct: {node.kind!r}"
raise ValueError(msg)
raise ValueError(f'Unsupported bash construct: "{node.kind}"')
def evaluate_word_node(node: bashlex.ast.node, context: NodeExecutionContext) -> str:
@@ -79,8 +63,10 @@ def evaluate_word_node(node: bashlex.ast.node, context: NodeExecutionContext) ->
part_value = evaluate_node(part, context=context)
if part_string not in value:
msg = f"bash parse failed. part {part_string!r} not found in {value!r}. Word was {node.word!r}. Full input was {context.input!r}"
raise RuntimeError(msg)
raise RuntimeError(
f'bash parse failed. part "{part_string}" not found in "{value}". '
f'Word was "{node.word}". Full input was "{context.input}"'
)
value = value.replace(part_string, part_value, 1)
@@ -107,17 +93,15 @@ def evaluate_nodes_as_compound_command(
result += evaluate_command_node(node, context=context)
elif node.kind == "operator":
if node.op != ";":
msg = f"Unsupported bash operator: {node.op!r}"
raise ValueError(msg)
raise ValueError(f'Unsupported bash operator: "{node.op}"')
else:
msg = f"Unsupported bash node in compound command: {node.kind!r}"
raise ValueError(msg)
raise ValueError(f'Unsupported bash node in compound command: "{node.kind}"')
return result
def evaluate_nodes_as_simple_command(
nodes: Iterable[bashlex.ast.node], context: NodeExecutionContext
nodes: List[bashlex.ast.node], context: NodeExecutionContext
) -> str:
command = [evaluate_node(part, context=context) for part in nodes]
return context.executor(command, context.environment)
-78
View File
@@ -1,78 +0,0 @@
__lazy_modules__ = {"cibuildwheel.util", "cibuildwheel.util.helpers"}
import os
import re
from enum import Enum
from cibuildwheel.util.helpers import strtobool
ANSI_CODE_REGEX = re.compile(r"(\033\[[0-9;]*m)")
class CIProvider(Enum):
# official support
travis_ci = "travis"
circle_ci = "circle_ci"
azure_pipelines = "azure_pipelines"
github_actions = "github_actions"
gitlab = "gitlab"
# unofficial support
appveyor = "appveyor"
other = "other"
def detect_ci_provider() -> CIProvider | None:
if "TRAVIS" in os.environ:
return CIProvider.travis_ci
elif "APPVEYOR" in os.environ:
return CIProvider.appveyor
elif "CIRCLECI" in os.environ:
return CIProvider.circle_ci
elif "AZURE_HTTP_USER_AGENT" in os.environ:
return CIProvider.azure_pipelines
elif "GITHUB_ACTIONS" in os.environ:
return CIProvider.github_actions
elif "GITLAB_CI" in os.environ:
return CIProvider.gitlab
elif strtobool(os.environ.get("CI", "false")):
return CIProvider.other
else:
return None
def fix_ansi_codes_for_github_actions(text: str) -> str:
"""
Github Actions forgets the current ANSI style on every new line. This
function repeats the current ANSI style on every new line.
"""
ansi_codes: list[str] = []
output = ""
for line in text.splitlines(keepends=True):
# add the current ANSI codes to the beginning of the line
output += "".join(ansi_codes) + line
# split the line at each ANSI code
parts = ANSI_CODE_REGEX.split(line)
# if there are any ANSI codes, save them
if len(parts) > 1:
# iterate over the ANSI codes in this line
for code in parts[1::2]:
if code == "\033[0m":
# reset the list of ANSI codes when the clear code is found
ansi_codes = []
else:
ansi_codes.append(code)
return output
def filter_ansi_codes(text: str, /) -> str:
"""
Remove ANSI codes from text.
"""
return ANSI_CODE_REGEX.sub("", text)
+242
View File
@@ -0,0 +1,242 @@
import io
import json
import os
import shlex
import subprocess
import sys
import uuid
from pathlib import Path, PurePath
from types import TracebackType
from typing import IO, Dict, List, Optional, Sequence, Type, cast
from .typing import PathOrStr, PopenBytes
class DockerContainer:
"""
An object that represents a running Docker container.
Intended for use as a context manager e.g.
`with DockerContainer('ubuntu') as docker:`
A bash shell is running in the remote container. When `call()` is invoked,
the command is relayed to the remote shell, and the results are streamed
back to cibuildwheel.
"""
UTILITY_PYTHON = "/opt/python/cp38-cp38/bin/python"
process: PopenBytes
bash_stdin: IO[bytes]
bash_stdout: IO[bytes]
def __init__(
self, docker_image: str, simulate_32_bit: bool = False, cwd: Optional[PathOrStr] = None
):
if not docker_image:
raise ValueError("Must have a non-empty docker image to run.")
self.docker_image = docker_image
self.simulate_32_bit = simulate_32_bit
self.cwd = cwd
self.name: Optional[str] = None
def __enter__(self) -> "DockerContainer":
self.name = f"cibuildwheel-{uuid.uuid4()}"
cwd_args = ["-w", str(self.cwd)] if self.cwd else []
shell_args = ["linux32", "/bin/bash"] if self.simulate_32_bit else ["/bin/bash"]
subprocess.run(
[
"docker",
"create",
"--env=CIBUILDWHEEL",
f"--name={self.name}",
"--interactive",
"--volume=/:/host", # ignored on CircleCI
*cwd_args,
self.docker_image,
*shell_args,
],
check=True,
)
self.process = subprocess.Popen(
[
"docker",
"start",
"--attach",
"--interactive",
self.name,
],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
)
assert self.process.stdin and self.process.stdout
self.bash_stdin = self.process.stdin
self.bash_stdout = self.process.stdout
# run a noop command to block until the container is responding
self.call(["/bin/true"])
return self
def __exit__(
self,
exc_type: Optional[Type[BaseException]],
exc_val: Optional[BaseException],
exc_tb: Optional[TracebackType],
) -> None:
self.bash_stdin.close()
self.process.terminate()
self.process.wait()
assert isinstance(self.name, str)
subprocess.run(["docker", "rm", "--force", "-v", self.name], stdout=subprocess.DEVNULL)
self.name = None
def copy_into(self, from_path: Path, to_path: PurePath) -> None:
# `docker cp` causes 'no space left on device' error when
# a container is running and the host filesystem is
# mounted. https://github.com/moby/moby/issues/38995
# Use `docker exec` instead.
if from_path.is_dir():
self.call(["mkdir", "-p", to_path])
subprocess.run(
f"tar cf - . | docker exec -i {self.name} tar -xC {shell_quote(to_path)} -f -",
shell=True,
check=True,
cwd=from_path,
)
else:
subprocess.run(
f'cat {shell_quote(from_path)} | docker exec -i {self.name} sh -c "cat > {shell_quote(to_path)}"',
shell=True,
check=True,
)
def copy_out(self, from_path: PurePath, to_path: Path) -> None:
# note: we assume from_path is a dir
to_path.mkdir(parents=True, exist_ok=True)
subprocess.run(
f"docker exec -i {self.name} tar -cC {shell_quote(from_path)} -f - . | tar -xf -",
shell=True,
check=True,
cwd=to_path,
)
def glob(self, path: PurePath, pattern: str) -> List[PurePath]:
glob_pattern = os.path.join(str(path), pattern)
path_strs = json.loads(
self.call(
[
self.UTILITY_PYTHON,
"-c",
f"import sys, json, glob; json.dump(glob.glob({glob_pattern!r}), sys.stdout)",
],
capture_output=True,
)
)
return [PurePath(p) for p in path_strs]
def call(
self,
args: Sequence[PathOrStr],
env: Optional[Dict[str, str]] = None,
capture_output: bool = False,
cwd: Optional[PathOrStr] = None,
) -> str:
chdir = f"cd {cwd}" if cwd else ""
env_assignments = (
" ".join(f"{shlex.quote(k)}={shlex.quote(v)}" for k, v in env.items())
if env is not None
else ""
)
command = " ".join(shlex.quote(str(a)) for a in args)
end_of_message = str(uuid.uuid4())
# log the command we're executing
print(f" + {command}")
# Write a command to the remote shell. First we change the
# cwd, if that's required. Then, we use the `env` utility to run
# `command` inside the specified environment. We use `env` because it
# can cope with spaces and strange characters in the name or value.
# Finally, the remote shell is told to write a footer - this will show
# up in the output so we know when to stop reading, and will include
# the returncode of `command`.
self.bash_stdin.write(
bytes(
f"""(
{chdir}
env {env_assignments} {command}
printf "%04d%s\n" $? {end_of_message}
)
""",
encoding="utf8",
errors="surrogateescape",
)
)
self.bash_stdin.flush()
if capture_output:
output_io: IO[bytes] = io.BytesIO()
else:
output_io = sys.stdout.buffer
while True:
line = self.bash_stdout.readline()
if line.endswith(bytes(end_of_message, encoding="utf8") + b"\n"):
# fmt: off
footer_offset = (
len(line)
- 1 # newline character
- len(end_of_message) # delimiter
- 4 # 4 returncode decimals
)
# fmt: on
returncode_str = line[footer_offset : footer_offset + 4]
returncode = int(returncode_str)
# add the last line to output, without the footer
output_io.write(line[0:footer_offset])
break
else:
output_io.write(line)
if isinstance(output_io, io.BytesIO):
output = str(output_io.getvalue(), encoding="utf8", errors="surrogateescape")
else:
output = ""
if returncode != 0:
raise subprocess.CalledProcessError(returncode, args, output)
return output
def get_environment(self) -> Dict[str, str]:
env = json.loads(
self.call(
[
self.UTILITY_PYTHON,
"-c",
"import sys, json, os; json.dump(os.environ.copy(), sys.stdout)",
],
capture_output=True,
)
)
return cast(Dict[str, str], env)
def environment_executor(self, command: List[str], environment: Dict[str, str]) -> str:
# used as an EnvironmentExecutor to evaluate commands and capture output
return self.call(command, env=environment, capture_output=True)
def shell_quote(path: PurePath) -> str:
return shlex.quote(str(path))
+20 -78
View File
@@ -1,26 +1,15 @@
from __future__ import annotations
__lazy_modules__ = {"bashlex", "bashlex.errors"}
import dataclasses
from typing import Protocol
from typing import Dict, List, Mapping, Optional
import bashlex
import bashlex.errors
from cibuildwheel import bashlex_eval
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Mapping, Sequence
from typing import Any
from . import bashlex_eval
class EnvironmentParseError(Exception):
pass
def split_env_items(env_string: str) -> list[str]:
def split_env_items(env_string: str) -> List[str]:
"""Splits space-separated variable assignments into a list of individual assignments.
>>> split_env_items('VAR=abc')
@@ -39,11 +28,7 @@ def split_env_items(env_string: str) -> list[str]:
if not env_string:
return []
try:
command_node = bashlex.parsesingle(env_string)
except bashlex.errors.ParsingError as e:
raise EnvironmentParseError(env_string) from e
command_node = bashlex.parsesingle(env_string)
result = []
for word_node in command_node.parts:
@@ -53,44 +38,7 @@ def split_env_items(env_string: str) -> list[str]:
return result
class EnvironmentAssignment(Protocol):
name: str
def evaluated_value(
self,
*,
environment: Mapping[str, str],
executor: bashlex_eval.EnvironmentExecutor | None = None,
) -> str:
"""Returns the value of this assignment, as evaluated in the environment"""
class EnvironmentAssignmentRaw:
"""
An environment variable - a simple name/value pair
"""
def __init__(self, name: str, value: str):
self.name = name
self.value = value
def __repr__(self) -> str:
return f"{self.name}={self.value}"
def evaluated_value(self, **_: Any) -> str:
return self.value
@dataclasses.dataclass
class EnvironmentAssignmentBash:
"""
An environment variable, in bash syntax. The value can use bash constructs
like "$OTHER_VAR" and "$(command arg1 arg2)".
"""
name: str
value: str
class EnvironmentAssignment:
def __init__(self, assignment: str):
name, equals, value = assignment.partition("=")
if not equals:
@@ -100,28 +48,29 @@ class EnvironmentAssignmentBash:
def evaluated_value(
self,
environment: Mapping[str, str],
executor: bashlex_eval.EnvironmentExecutor | None = None,
environment: Dict[str, str],
executor: Optional[bashlex_eval.EnvironmentExecutor] = None,
) -> str:
"""Returns the value of this assignment, as evaluated in the environment"""
return bashlex_eval.evaluate(self.value, environment=environment, executor=executor)
def as_shell_assignment(self) -> str:
return f"export {self.name}={self.value}"
def __repr__(self) -> str:
return f"{self.name}={self.value}"
@dataclasses.dataclass(kw_only=True)
class ParsedEnvironment:
assignments: list[EnvironmentAssignment]
def __init__(self, assignments: Sequence[EnvironmentAssignment]) -> None:
self.assignments = list(assignments)
def __init__(self, assignments: List[EnvironmentAssignment]):
self.assignments = assignments
def as_dictionary(
self,
prev_environment: Mapping[str, str],
executor: bashlex_eval.EnvironmentExecutor | None = None,
) -> dict[str, str]:
environment = {**prev_environment}
executor: Optional[bashlex_eval.EnvironmentExecutor] = None,
) -> Dict[str, str]:
environment = dict(**prev_environment)
for assignment in self.assignments:
value = assignment.evaluated_value(environment=environment, executor=executor)
@@ -129,21 +78,14 @@ class ParsedEnvironment:
return environment
def add(self, name: str, value: str, prepend: bool = False) -> None:
assignment = EnvironmentAssignmentRaw(name=name, value=value)
if prepend:
self.assignments.insert(0, assignment)
else:
self.assignments.append(assignment)
def as_shell_commands(self) -> List[str]:
return [a.as_shell_assignment() for a in self.assignments]
def __repr__(self) -> str:
return f"{self.__class__.__name__}({[repr(a) for a in self.assignments]!r})"
def options_summary(self) -> list[EnvironmentAssignment]:
return self.assignments
return f"ParsedEnvironment({[repr(a) for a in self.assignments]!r})"
def parse_environment(env_string: str) -> ParsedEnvironment:
env_items = split_env_items(env_string)
assignments = [EnvironmentAssignmentBash(item) for item in env_items]
assignments = [EnvironmentAssignment(item) for item in env_items]
return ParsedEnvironment(assignments=assignments)
-129
View File
@@ -1,129 +0,0 @@
"""
Errors that can cause the build to fail. Each subclass of FatalError has
a different return code, by defining them all here, we can ensure that they're
semantically clear and unique.
"""
__lazy_modules__ = {"textwrap"}
import textwrap
class FatalError(BaseException):
"""
Raising an error of this type will cause the message to be printed to
stderr and the process to be terminated. Within cibuildwheel, raising this
exception produces a better error message, and optional traceback.
"""
return_code: int = 1
class ConfigurationError(FatalError):
return_code = 2
class NothingToDoError(FatalError):
return_code = 3
class DeprecationError(FatalError):
return_code = 4
class NonPlatformWheelError(FatalError):
def __init__(self) -> None:
message = textwrap.dedent(
"""
Build failed because a pure Python wheel was generated.
If you intend to build a pure-Python wheel, you don't need
cibuildwheel - use `pip wheel .`, `pipx run build --wheel`, `uv
build --wheel`, etc. instead. You only need cibuildwheel if you
have compiled (not Python) code in your wheels making them depend
on the platform.
If you expected a platform wheel, check your project configuration,
or run cibuildwheel with CIBW_BUILD_VERBOSITY=1 to view build logs.
"""
)
super().__init__(message)
self.return_code = 5
class AlreadyBuiltWheelError(FatalError):
def __init__(self, wheel_name: str) -> None:
message = textwrap.dedent(
f"""
Build failed because a wheel named {wheel_name} was already generated in the current run.
If you expected another wheel to be generated, check your project configuration, or run
cibuildwheel with CIBW_BUILD_VERBOSITY=1 to view build logs.
"""
)
super().__init__(message)
self.return_code = 6
class OCIEngineTooOldError(FatalError):
def __init__(self, message: str) -> None:
super().__init__(message)
self.return_code = 7
class RepairStepProducedNoWheelError(FatalError):
def __init__(self) -> None:
message = textwrap.dedent(
"""
Build failed because the repair step completed successfully but
did not produce a wheel.
Your `repair-wheel-command` must place the repaired
wheel in the {dest_dir} directory. See the documentation for
example configurations:
https://cibuildwheel.pypa.io/en/stable/options/#repair-wheel-command
"""
)
super().__init__(message)
self.return_code = 8
class RepairStepProducedMultipleWheelsError(FatalError):
def __init__(self, wheels: list[str]) -> None:
message = textwrap.dedent(
f"""
Build failed because the repair step completed successfully but
produced multiple wheels: {wheels}
Your `repair-wheel-command` must place exactly one repaired
wheel in the {{dest_dir}} directory. See the documentation for
example configurations:
https://cibuildwheel.pypa.io/en/stable/options/#repair-wheel-command
"""
)
super().__init__(message)
self.return_code = 8
class AuditCommandFailedError(FatalError):
def __init__(self, message: str) -> None:
super().__init__(message)
self.return_code = 9
class BuildProducedNoWheelError(FatalError):
def __init__(self) -> None:
message = textwrap.dedent(
"""
Build failed because the build frontend completed successfully but
did not produce a wheel.
This usually indicates a problem with your project configuration or
build backend. Check your project configuration, or run cibuildwheel
with CIBW_BUILD_VERBOSITY=1 to view build logs.
"""
)
super().__init__(message)
self.return_code = 10
+15 -103
View File
@@ -2,110 +2,22 @@
These are utilities for the `/bin` scripts, not for the `cibuildwheel` program.
"""
from __future__ import annotations
from typing import Any, Dict
__lazy_modules__ = {"io", "json", "urllib", "urllib.error", "urllib.request"}
import json
import time
import typing
import urllib.error
import urllib.request
from io import StringIO
from typing import NotRequired, Protocol
from cibuildwheel import __version__ as cibw_version
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Mapping, Sequence
from typing import Any
import toml.encoder
from packaging.version import Version
__all__ = ("Printable", "dump_python_configurations")
class InlineArrayDictEncoder(toml.encoder.TomlEncoder): # type: ignore
def __init__(self) -> None:
super().__init__()
self.dump_funcs[Version] = lambda v: f'"{v}"'
class Printable(Protocol):
def __str__(self) -> str: ...
def dump_python_configurations(
inp: Mapping[str, Mapping[str, Sequence[Mapping[str, Printable]]]],
) -> str:
output = StringIO()
for header, values in inp.items():
output.write(f"[{header}]\n")
for inner_header, listing in values.items():
output.write(f"{inner_header} = [\n")
for item in listing:
output.write(" { ")
dict_contents = (f'{key} = "{value}"' for key, value in item.items())
output.write(", ".join(dict_contents))
output.write(" },\n")
output.write("]\n")
output.write("\n")
# Strip the final newline, to avoid two blank lines at the end.
return output.getvalue()[:-1]
def _json_request(request: urllib.request.Request, timeout: int = 30) -> dict[str, Any]:
with urllib.request.urlopen(request, timeout=timeout) as response:
return typing.cast("dict[str, Any]", json.load(response))
def github_api_request(path: str, *, max_retries: int = 3) -> dict[str, Any]:
"""
Makes a GitHub API request to the given path and returns the JSON response.
"""
api_url = f"https://api.github.com/{path}"
headers = {
"Accept": "application/vnd.github.v3+json",
"User-Agent": f"cibuildwheel/{cibw_version}",
}
request = urllib.request.Request(api_url, headers=headers)
for retry_count in range(max_retries):
try:
return _json_request(request)
except (urllib.error.URLError, TimeoutError) as e:
# pylint: disable=E1101
if (
isinstance(e, urllib.error.HTTPError)
and (e.code in {403, 429})
and e.headers.get("x-ratelimit-remaining") == "0"
):
reset_time = int(e.headers.get("x-ratelimit-reset", 0))
wait_time = max(0, reset_time - int(e.headers.get("date", 0)))
print(f"Github rate limit exceeded. Waiting for {wait_time} seconds.")
time.sleep(wait_time)
else:
print(f"Retrying GitHub API request due to error: {e}")
if retry_count == max_retries - 1:
print(f"GitHub API request failed (Network error: {e}). Check network connection.")
raise e
# Should never be reached but to keep the type checker happy
msg = "Unexpected execution path in github_api_request"
raise RuntimeError(msg)
class PyodideXBuildEnvRelease(typing.TypedDict):
version: str
sha256: str
python_version: str
emscripten_version: str
min_pyodide_build_version: NotRequired[str]
max_pyodide_build_version: NotRequired[str]
class PyodideXBuildEnvInfo(typing.TypedDict):
releases: dict[str, PyodideXBuildEnvRelease]
def get_pyodide_xbuildenv_info() -> PyodideXBuildEnvInfo:
xbuildenv_info_url = (
"https://pyodide.github.io/pyodide/api/v2/pyodide-cross-build-environments.json"
)
with urllib.request.urlopen(xbuildenv_info_url) as response:
return typing.cast("PyodideXBuildEnvInfo", json.loads(response.read().decode("utf-8")))
def dump_sections(self, o: Dict[str, Any], sup: str) -> Any:
if not all(isinstance(a, list) for a in o.values()):
return super().dump_sections(o, sup)
val = ""
for k, v in o.items():
inner = ",\n ".join(self.dump_inline_table(d_i).strip() for d_i in v)
val += f"{k} = [\n {inner},\n]\n"
return val, self._dict()
-103
View File
@@ -1,103 +0,0 @@
from __future__ import annotations
__lazy_modules__ = {"cibuildwheel.util", "cibuildwheel.util.helpers", "shlex"}
import dataclasses
import shlex
import typing
from typing import Literal, get_args
from cibuildwheel.util.helpers import parse_key_value_string, prepare_command
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Sequence
from typing import Self
from cibuildwheel.typing import PathOrStr
BuildFrontendName = Literal["pip", "build", "build[uv]", "uv"]
@dataclasses.dataclass(frozen=True)
class BuildFrontendConfig:
name: BuildFrontendName
args: Sequence[str] = ()
@classmethod
def from_config_string(cls, config_string: str) -> Self:
config_dict = parse_key_value_string(config_string, ["name"], ["args"])
name = " ".join(config_dict["name"])
if name not in get_args(BuildFrontendName):
names = ", ".join(repr(n) for n in get_args(BuildFrontendName))
msg = f"Unrecognised build frontend {name!r}, must be one of {names}"
raise ValueError(msg)
name = typing.cast("BuildFrontendName", name)
args = config_dict.get("args") or []
return cls(name=name, args=args)
def options_summary(self) -> str | dict[str, str]:
if not self.args:
return self.name
else:
return {"name": self.name, "args": repr(self.args)}
def _get_verbosity_flags(level: int, frontend: BuildFrontendName) -> list[str]:
if level < 0:
return ["-" + -level * "q"]
if level > 0:
if frontend == "pip":
return ["-" + level * "v"]
if level > 1:
return ["-" + (level - 1) * "v"]
return []
def _split_config_settings(config_settings: str) -> list[str]:
config_settings_list = shlex.split(config_settings)
return [f"-C{setting}" for setting in config_settings_list]
def prepare_config_settings(config_settings: str, *, project: PathOrStr, package: PathOrStr) -> str:
# Substitute the {project}/{package} placeholders on each already-split
# token rather than on the raw string. A substituted path may contain
# spaces or backslashes (e.g. a Windows `{package}` path), and the result
# is later re-parsed with shlex.split (in _split_config_settings /
# parse_config_settings) — substituting on the whole string would let
# those characters be reinterpreted, splitting one setting into several or
# eating backslashes. shlex.join re-quotes each token so the round-trip is
# lossless.
settings = shlex.split(config_settings)
prepared = [prepare_command(setting, project=project, package=package) for setting in settings]
return shlex.join(prepared)
# Based on build.__main__.main.
def parse_config_settings(config_settings_str: str) -> dict[str, str | list[str]]:
config_settings: dict[str, str | list[str]] = {}
for arg in shlex.split(config_settings_str):
setting, _, value = arg.partition("=")
existing_value = config_settings.get(setting)
if existing_value is None:
config_settings[setting] = value
elif isinstance(existing_value, str):
config_settings[setting] = [existing_value, value]
else:
existing_value.append(value)
return config_settings
def get_build_frontend_extra_flags(
build_frontend: BuildFrontendConfig, verbosity_level: int, config_settings: str
) -> list[str]:
return [
*_split_config_settings(config_settings),
*build_frontend.args,
*_get_verbosity_flags(verbosity_level, build_frontend.name),
]
+336
View File
@@ -0,0 +1,336 @@
import subprocess
import sys
import textwrap
from pathlib import Path, PurePath
from typing import List, NamedTuple, Set
from .architecture import Architecture
from .docker_container import DockerContainer
from .logger import log
from .typing import PathOrStr
from .util import (
BuildOptions,
BuildSelector,
NonPlatformWheelError,
get_build_verbosity_extra_flags,
prepare_command,
read_python_configs,
resources_dir,
)
class PythonConfiguration(NamedTuple):
version: str
identifier: str
path_str: str
@property
def path(self) -> PurePath:
return PurePath(self.path_str)
def get_python_configurations(
build_selector: BuildSelector,
architectures: Set[Architecture],
) -> List[PythonConfiguration]:
full_python_configs = read_python_configs("linux")
python_configurations = [PythonConfiguration(**item) for item in full_python_configs]
# return all configurations whose arch is in our `architectures` set,
# and match the build/skip rules
return [
c
for c in python_configurations
if any(c.identifier.endswith(arch.value) for arch in architectures)
and build_selector(c.identifier)
]
def build(options: BuildOptions) -> None:
try:
# check docker is installed
subprocess.run(["docker", "--version"], check=True, stdout=subprocess.DEVNULL)
except Exception:
print(
"cibuildwheel: Docker not found. Docker is required to run Linux builds. "
"If you're building on Travis CI, add `services: [docker]` to your .travis.yml."
"If you're building on Circle CI in Linux, add a `setup_remote_docker` step to your .circleci/config.yml",
file=sys.stderr,
)
sys.exit(2)
assert options.manylinux_images is not None
python_configurations = get_python_configurations(options.build_selector, options.architectures)
platforms = [
("cp", "manylinux_x86_64", options.manylinux_images["x86_64"]),
("cp", "manylinux_i686", options.manylinux_images["i686"]),
("cp", "manylinux_aarch64", options.manylinux_images["aarch64"]),
("cp", "manylinux_ppc64le", options.manylinux_images["ppc64le"]),
("cp", "manylinux_s390x", options.manylinux_images["s390x"]),
("pp", "manylinux_x86_64", options.manylinux_images["pypy_x86_64"]),
]
cwd = Path.cwd()
abs_package_dir = options.package_dir.resolve()
if cwd != abs_package_dir and cwd not in abs_package_dir.parents:
raise Exception("package_dir must be inside the working directory")
container_project_path = PurePath("/project")
container_package_dir = container_project_path / abs_package_dir.relative_to(cwd)
container_output_dir = PurePath("/output")
for implementation, platform_tag, docker_image in platforms:
platform_configs = [
c
for c in python_configurations
if c.identifier.startswith(implementation) and c.identifier.endswith(platform_tag)
]
if not platform_configs:
continue
try:
log.step(f"Starting Docker image {docker_image}...")
with DockerContainer(
docker_image,
simulate_32_bit=platform_tag.endswith("i686"),
cwd=container_project_path,
) as docker:
log.step("Copying project into Docker...")
docker.copy_into(Path.cwd(), container_project_path)
if options.before_all:
log.step("Running before_all...")
env = docker.get_environment()
env["PATH"] = f'/opt/python/cp38-cp38/bin:{env["PATH"]}'
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
env = options.environment.as_dictionary(
env, executor=docker.environment_executor
)
before_all_prepared = prepare_command(
options.before_all,
project=container_project_path,
package=container_package_dir,
)
docker.call(["sh", "-c", before_all_prepared], env=env)
for config in platform_configs:
log.build_start(config.identifier)
dependency_constraint_flags: List[PathOrStr] = []
if config.identifier.startswith("pp"):
# Patch PyPy to make sure headers get installed into a venv
patch_version = "_27" if config.version == "2.7" else ""
patch_path = resources_dir / f"pypy_venv{patch_version}.patch"
patch_docker_path = PurePath("/pypy_venv.patch")
docker.copy_into(patch_path, patch_docker_path)
try:
docker.call(
[
"patch",
"--force",
"-p1",
"-d",
config.path,
"-i",
patch_docker_path,
]
)
except subprocess.CalledProcessError:
print("PyPy patch not applied", file=sys.stderr)
if options.dependency_constraints:
constraints_file = options.dependency_constraints.get_for_python_version(
config.version
)
container_constraints_file = PurePath("/constraints.txt")
docker.copy_into(constraints_file, container_constraints_file)
dependency_constraint_flags = ["-c", container_constraints_file]
log.step("Setting up build environment...")
env = docker.get_environment()
# put this config's python top of the list
python_bin = config.path / "bin"
env["PATH"] = f'{python_bin}:{env["PATH"]}'
env = options.environment.as_dictionary(
env, executor=docker.environment_executor
)
# check config python and pip are still on PATH
which_python = docker.call(
["which", "python"], env=env, capture_output=True
).strip()
if PurePath(which_python) != python_bin / "python":
print(
"cibuildwheel: python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it.",
file=sys.stderr,
)
sys.exit(1)
which_pip = docker.call(["which", "pip"], env=env, capture_output=True).strip()
if PurePath(which_pip) != python_bin / "pip":
print(
"cibuildwheel: pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it.",
file=sys.stderr,
)
sys.exit(1)
if options.before_build:
log.step("Running before_build...")
before_build_prepared = prepare_command(
options.before_build,
project=container_project_path,
package=container_package_dir,
)
docker.call(["sh", "-c", before_build_prepared], env=env)
log.step("Building wheel...")
temp_dir = PurePath("/tmp/cibuildwheel")
built_wheel_dir = temp_dir / "built_wheel"
docker.call(["rm", "-rf", built_wheel_dir])
docker.call(["mkdir", "-p", built_wheel_dir])
docker.call(
[
"pip",
"wheel",
container_package_dir,
"--wheel-dir",
built_wheel_dir,
"--no-deps",
*get_build_verbosity_extra_flags(options.build_verbosity),
],
env=env,
)
built_wheel = docker.glob(built_wheel_dir, "*.whl")[0]
repaired_wheel_dir = temp_dir / "repaired_wheel"
docker.call(["rm", "-rf", repaired_wheel_dir])
docker.call(["mkdir", "-p", repaired_wheel_dir])
if built_wheel.name.endswith("none-any.whl"):
raise NonPlatformWheelError()
if options.repair_command:
log.step("Repairing wheel...")
repair_command_prepared = prepare_command(
options.repair_command, wheel=built_wheel, dest_dir=repaired_wheel_dir
)
docker.call(["sh", "-c", repair_command_prepared], env=env)
else:
docker.call(["mv", built_wheel, repaired_wheel_dir])
repaired_wheels = docker.glob(repaired_wheel_dir, "*.whl")
if options.test_command and options.test_selector(config.identifier):
log.step("Testing wheel...")
# set up a virtual environment to install and test from, to make sure
# there are no dependencies that were pulled in at build time.
docker.call(
["pip", "install", "virtualenv", *dependency_constraint_flags], env=env
)
venv_dir = (
PurePath(docker.call(["mktemp", "-d"], capture_output=True).strip())
/ "venv"
)
docker.call(
["python", "-m", "virtualenv", "--no-download", venv_dir], env=env
)
virtualenv_env = env.copy()
virtualenv_env["PATH"] = f"{venv_dir / 'bin'}:{virtualenv_env['PATH']}"
if options.before_test:
before_test_prepared = prepare_command(
options.before_test,
project=container_project_path,
package=container_package_dir,
)
docker.call(["sh", "-c", before_test_prepared], env=virtualenv_env)
# Install the wheel we just built
# Note: If auditwheel produced two wheels, it's because the earlier produced wheel
# conforms to multiple manylinux standards. These multiple versions of the wheel are
# functionally the same, differing only in name, wheel metadata, and possibly include
# different external shared libraries. so it doesn't matter which one we run the tests on.
# Let's just pick the first one.
wheel_to_test = repaired_wheels[0]
docker.call(
["pip", "install", str(wheel_to_test) + options.test_extras],
env=virtualenv_env,
)
# Install any requirements to run the tests
if options.test_requires:
docker.call(
["pip", "install", *options.test_requires], env=virtualenv_env
)
# Run the tests from a different directory
test_command_prepared = prepare_command(
options.test_command,
project=container_project_path,
package=container_package_dir,
)
docker.call(
["sh", "-c", test_command_prepared], cwd="/root", env=virtualenv_env
)
# clean up test environment
docker.call(["rm", "-rf", venv_dir])
# move repaired wheels to output
docker.call(["mkdir", "-p", container_output_dir])
docker.call(["mv", *repaired_wheels, container_output_dir])
log.build_end()
log.step("Copying wheels back to host...")
# copy the output back into the host
docker.copy_out(container_output_dir, options.output_dir)
log.step_end()
except subprocess.CalledProcessError as error:
log.step_end_with_error(
f"Command {error.cmd} failed with code {error.returncode}. {error.stdout}"
)
troubleshoot(options.package_dir, error)
sys.exit(1)
def troubleshoot(package_dir: Path, error: Exception) -> None:
if isinstance(error, subprocess.CalledProcessError) and error.cmd[0:2] == ["pip", "wheel"]:
# the 'pip wheel' step failed.
print("Checking for common errors...")
so_files = list(package_dir.glob("**/*.so"))
if so_files:
print(
textwrap.dedent(
"""
NOTE: Shared object (.so) files found in this project.
These files might be built against the wrong OS, causing problems with
auditwheel.
If you're using Cython and have previously done an in-place build,
remove those build files (*.so and *.c) before starting cibuildwheel.
"""
),
file=sys.stderr,
)
print(" Files detected:")
print("\n".join(f" {f}" for f in so_files))
print("")
+80 -264
View File
@@ -1,142 +1,41 @@
from __future__ import annotations
__lazy_modules__ = {
"cibuildwheel.ci",
"contextlib",
"functools",
"hashlib",
"humanize",
"io",
"pathlib",
"re",
"textwrap",
}
import codecs
import contextlib
import dataclasses
import functools
import hashlib
import io
import os
import re
import sys
import textwrap
import time
from pathlib import Path
from typing import IO, AnyStr, Optional, Union
import humanize
from cibuildwheel.util import CIProvider, detect_ci_provider
from cibuildwheel.ci import CIProvider, detect_ci_provider, filter_ansi_codes
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Generator
from typing import IO, AnyStr, Final, Literal
from cibuildwheel.options import Options
FoldPattern = tuple[str, str]
DEFAULT_FOLD_PATTERN: Final[FoldPattern] = ("{name}", "")
FOLD_PATTERNS: Final[dict[str, FoldPattern]] = {
DEFAULT_FOLD_PATTERN = ("{name}", "")
FOLD_PATTERNS = {
"azure": ("##[group]{name}", "##[endgroup]"),
"travis": ("travis_fold:start:{identifier}\n{name}", "travis_fold:end:{identifier}"),
"github": ("::group::{name}", "::endgroup::{name}"),
}
PLATFORM_IDENTIFIER_DESCRIPTIONS: Final[dict[str, str]] = {
PLATFORM_IDENTIFIER_DESCIPTIONS = {
"manylinux_x86_64": "manylinux x86_64",
"manylinux_i686": "manylinux i686",
"manylinux_aarch64": "manylinux aarch64",
"manylinux_ppc64le": "manylinux ppc64le",
"manylinux_s390x": "manylinux s390x",
"manylinux_armv7l": "manylinux armv7l",
"manylinux_riscv64": "manylinux riscv64",
"musllinux_x86_64": "musllinux x86_64",
"musllinux_i686": "musllinux i686",
"musllinux_aarch64": "musllinux aarch64",
"musllinux_ppc64le": "musllinux ppc64le",
"musllinux_s390x": "musllinux s390x",
"musllinux_armv7l": "musllinux armv7l",
"musllinux_riscv64": "musllinux riscv64",
"win32": "Windows 32bit",
"win_amd64": "Windows 64bit",
"win_arm64": "Windows on ARM 64bit",
"macosx_x86_64": "macOS x86_64",
"macosx_universal2": "macOS Universal 2 - x86_64 and arm64",
"macosx_arm64": "macOS arm64 - Apple Silicon",
"pyodide_wasm32": "Pyodide",
"android_arm64_v8a": "Android arm64_v8a",
"android_x86_64": "Android x86_64",
"ios_arm64_iphoneos": "iOS Device (ARM64)",
"ios_arm64_iphonesimulator": "iOS Simulator (ARM64)",
"ios_x86_64_iphonesimulator": "iOS Simulator (x86_64)",
}
class Colors:
def __init__(self, *, enabled: bool) -> None:
self.red = "\033[31m" if enabled else ""
self.green = "\033[32m" if enabled else ""
self.yellow = "\033[33m" if enabled else ""
self.blue = "\033[34m" if enabled else ""
self.cyan = "\033[36m" if enabled else ""
self.bright_red = "\033[91m" if enabled else ""
self.bright_green = "\033[92m" if enabled else ""
self.white = "\033[37m\033[97m" if enabled else ""
self.gray = "\033[38;5;244m" if enabled else ""
self.bg_grey = "\033[48;5;235m" if enabled else ""
self.bold = "\033[1m" if enabled else ""
self.faint = "\033[2m" if enabled else ""
self.end = "\033[0m" if enabled else ""
class Symbols:
def __init__(self, *, unicode: bool) -> None:
self.done = "" if unicode else "done"
self.error = "" if unicode else "failed"
@dataclasses.dataclass(kw_only=True, frozen=True)
class BuildInfo:
identifier: str
filename: Path | None
duration: float
@functools.cached_property
def size(self) -> str | None:
if self.filename is None:
return None
return humanize.naturalsize(self.filename.stat().st_size)
@functools.cached_property
def sha256(self) -> str | None:
if self.filename is None:
return None
with self.filename.open("rb") as f:
digest = hashlib.file_digest(f, "sha256")
return digest.hexdigest()
def __str__(self) -> str:
duration = humanize.naturaldelta(self.duration)
if self.filename:
return f"{self.identifier}: {self.filename.name} {self.size} in {duration}, SHA256={self.sha256}"
return f"{self.identifier}: {duration} (test only)"
class Logger:
fold_mode: Literal["azure", "github", "travis", "disabled"]
fold_mode: str
colors_enabled: bool
unicode_enabled: bool
active_build_identifier: str | None = None
build_start_time: float | None = None
step_start_time: float | None = None
active_fold_group_name: str | None = None
summary: list[BuildInfo]
active_build_identifier: Optional[str] = None
build_start_time: Optional[float] = None
step_start_time: Optional[float] = None
active_fold_group_name: Optional[str] = None
def __init__(self) -> None:
if sys.platform == "win32" and hasattr(sys.stdout, "reconfigure"):
@@ -148,28 +47,25 @@ class Logger:
ci_provider = detect_ci_provider()
match ci_provider:
case CIProvider.azure_pipelines:
self.fold_mode = "azure"
self.colors_enabled = True
if ci_provider == CIProvider.azure_pipelines:
self.fold_mode = "azure"
self.colors_enabled = True
case CIProvider.github_actions:
self.fold_mode = "github"
self.colors_enabled = True
elif ci_provider == CIProvider.github_actions:
self.fold_mode = "github"
self.colors_enabled = True
case CIProvider.travis_ci:
self.fold_mode = "travis"
self.colors_enabled = True
elif ci_provider == CIProvider.travis_ci:
self.fold_mode = "travis"
self.colors_enabled = True
case CIProvider.appveyor:
self.fold_mode = "disabled"
self.colors_enabled = True
elif ci_provider == CIProvider.appveyor:
self.fold_mode = "disabled"
self.colors_enabled = True
case _:
self.fold_mode = "disabled"
self.colors_enabled = file_supports_color(sys.stdout)
self.summary = []
else:
self.fold_mode = "disabled"
self.colors_enabled = file_supports_color(sys.stdout)
def build_start(self, identifier: str) -> None:
self.step_end()
@@ -183,7 +79,7 @@ class Logger:
self.build_start_time = time.time()
self.active_build_identifier = identifier
def build_end(self, filename: Path | None) -> None:
def build_end(self) -> None:
assert self.build_start_time is not None
assert self.active_build_identifier is not None
self.step_end()
@@ -191,14 +87,11 @@ class Logger:
c = self.colors
s = self.symbols
duration = time.time() - self.build_start_time
duration_str = humanize.naturaldelta(duration, minimum_unit="milliseconds")
print()
print(f"{c.green}{s.done} {c.end}{self.active_build_identifier} finished in {duration_str}")
self.summary.append(
BuildInfo(identifier=self.active_build_identifier, filename=filename, duration=duration)
print(
f"{c.green}{s.done} {c.end}{self.active_build_identifier} finished in {duration:.2f}s"
)
self.build_start_time = None
self.active_build_identifier = None
@@ -213,7 +106,6 @@ class Logger:
c = self.colors
s = self.symbols
duration = time.time() - self.step_start_time
if success:
print(f"{c.green}{s.done} {c.end}{duration:.2f}s".rjust(78))
else:
@@ -221,58 +113,23 @@ class Logger:
self.step_start_time = None
def step_end_with_error(self, error: BaseException | str) -> None:
def step_end_with_error(self, error: Union[BaseException, str]) -> None:
self.step_end(success=False)
self.error(error)
def quiet(self, message: str) -> None:
c = self.colors
print(f"{c.gray}{message}{c.end}", file=sys.stderr)
def notice(self, message: str) -> None:
if self.fold_mode == "github":
print(f"::notice::cibuildwheel: {message}\n", file=sys.stderr)
else:
c = self.colors
print(f"cibuildwheel: {c.bold}note{c.end}: {message}\n", file=sys.stderr)
def warning(self, message: str) -> None:
if self.fold_mode == "github":
print(f"::warning::cibuildwheel: {message}\n", file=sys.stderr)
print(f"::warning::{message}\n", file=sys.stderr)
else:
c = self.colors
print(f"cibuildwheel: {c.yellow}warning{c.end}: {message}\n", file=sys.stderr)
print(f"{c.yellow}Warning{c.end}: {message}\n", file=sys.stderr)
def error(self, error: BaseException | str) -> None:
def error(self, error: Union[BaseException, str]) -> None:
if self.fold_mode == "github":
print(f"::error::cibuildwheel: {error}\n", file=sys.stderr)
print(f"::error::{error}\n", file=sys.stderr)
else:
c = self.colors
print(f"cibuildwheel: {c.bright_red}error{c.end}: {error}\n", file=sys.stderr)
@contextlib.contextmanager
def print_summary(self, *, options: Options) -> Generator[None, None, None]:
start = time.time()
yield
duration = time.time() - start
if summary_path := os.environ.get("GITHUB_STEP_SUMMARY"):
github_summary = self._github_step_summary(duration=duration, options=options)
Path(summary_path).write_text(filter_ansi_codes(github_summary), encoding="utf-8")
n_wheels = len([info for info in self.summary if info.filename])
s = "s" if n_wheels > 1 else ""
duration_str = humanize.naturaldelta(duration)
print()
self._start_fold_group(f"{n_wheels} wheel{s} produced in {duration_str}")
for build_info in self.summary:
print(" ", build_info)
self._end_fold_group()
self.summary = []
@property
def step_active(self) -> bool:
return self.step_start_time is not None
print(f"{c.bright_red}Error{c.end}: {error}\n", file=sys.stderr)
def _start_fold_group(self, name: str) -> None:
self._end_fold_group()
@@ -294,8 +151,7 @@ class Logger:
sys.stdout.flush()
self.active_fold_group_name = None
@staticmethod
def _fold_group_identifier(name: str) -> str:
def _fold_group_identifier(self, name: str) -> str:
"""
Travis doesn't like fold groups identifiers that have spaces in. This
method converts them to ascii identifiers
@@ -309,79 +165,19 @@ class Logger:
# lowercase, shorten
return identifier.lower()[:20]
def _github_step_summary(self, duration: float, options: Options) -> str:
"""
Returns the GitHub step summary, in markdown format.
"""
out = io.StringIO()
options_summary = options.summary(
identifiers=[bi.identifier for bi in self.summary], skip_unset=True
)
out.write(
textwrap.dedent("""\
### 🎡 cibuildwheel
@property
def colors(self) -> "Colors":
if self.colors_enabled:
return Colors(enabled=True)
else:
return Colors(enabled=False)
<details>
<summary>
Build options
</summary>
```yaml
{options_summary}
```
</details>
""").format(options_summary=options_summary)
)
n_wheels = len([b for b in self.summary if b.filename])
wheel_rows = "\n".join(
"<tr>"
f"<td nowrap>{'<samp>' + b.filename.name + '</samp>' if b.filename else '*Test only*'}</td>"
f"<td nowrap>{b.size or 'N/A'}</td>"
f"<td nowrap><samp>{b.identifier}</samp></td>"
f"<td nowrap>{humanize.naturaldelta(b.duration)}</td>"
f"<td nowrap><samp>{b.sha256 or 'N/A'}</samp></td>"
"</tr>"
for b in self.summary
)
out.write(
textwrap.dedent("""\
<table>
<thead>
<tr>
<th align="left">Wheel</th>
<th align="left">Size</th>
<th align="left">Build identifier</th>
<th align="left">Time</th>
<th align="left">SHA256</th>
</tr>
</thead>
<tbody>
{wheel_rows}
</tbody>
</table>
<div align="right"><sup>{n} wheel{s} created in {duration_str}</sup></div>
""").format(
wheel_rows=wheel_rows,
n=n_wheels,
duration_str=humanize.naturaldelta(duration),
s="s" if n_wheels > 1 else "",
)
)
out.write("\n")
out.write("---")
out.write("\n")
return out.getvalue()
@functools.cached_property
def colors(self) -> Colors:
return Colors(enabled=self.colors_enabled)
@functools.cached_property
def symbols(self) -> Symbols:
return Symbols(unicode=self.unicode_enabled)
@property
def symbols(self) -> "Symbols":
if self.unicode_enabled:
return Symbols(unicode=True)
else:
return Symbols(unicode=False)
def build_description_from_identifier(identifier: str) -> str:
@@ -390,32 +186,50 @@ def build_description_from_identifier(identifier: str) -> str:
build_description = ""
python_interpreter = python_identifier[0:2]
version_parts = python_identifier[2:].split("_")
python_version = version_parts[0]
python_version = python_identifier[2:4]
if python_interpreter == "cp":
build_description += "CPython"
elif python_interpreter == "pp":
build_description += "PyPy"
elif python_interpreter == "gp":
build_description += "GraalPy"
else:
msg = f"unknown python {python_interpreter!r}"
raise Exception(msg)
raise Exception("unknown python")
build_description += f" {python_version[0]}.{python_version[1:]} "
if len(version_parts) > 1:
build_description += f"(ABI {version_parts[1]}) "
build_description += f" {python_version[0]}.{python_version[1]} "
try:
build_description += PLATFORM_IDENTIFIER_DESCRIPTIONS[platform_identifier]
build_description += PLATFORM_IDENTIFIER_DESCIPTIONS[platform_identifier]
except KeyError as e:
msg = f"unknown platform {platform_identifier!r}"
raise Exception(msg) from e
raise Exception("unknown platform") from e
return build_description
class Colors:
def __init__(self, *, enabled: bool) -> None:
self.red = "\033[31m" if enabled else ""
self.green = "\033[32m" if enabled else ""
self.yellow = "\033[33m" if enabled else ""
self.blue = "\033[34m" if enabled else ""
self.cyan = "\033[36m" if enabled else ""
self.bright_red = "\033[91m" if enabled else ""
self.bright_green = "\033[92m" if enabled else ""
self.white = "\033[37m\033[97m" if enabled else ""
self.bg_grey = "\033[48;5;235m" if enabled else ""
self.bold = "\033[1m" if enabled else ""
self.faint = "\033[2m" if enabled else ""
self.end = "\033[0m" if enabled else ""
class Symbols:
def __init__(self, *, unicode: bool) -> None:
self.done = "" if unicode else "done"
self.error = "" if unicode else "failed"
def file_supports_color(file_obj: IO[AnyStr]) -> bool:
"""
Returns True if the running system's terminal supports color.
@@ -442,6 +256,8 @@ def file_supports_unicode(file_obj: IO[AnyStr]) -> bool:
return "utf" in codec_info.name
# Global instance of the Logger.
"""
Global instance of the Logger.
"""
# (there's only one stdout per-process, so a global instance is justified)
log = Logger()
+582
View File
@@ -0,0 +1,582 @@
import os
import platform
import re
import shlex
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
from typing import Any, Dict, List, NamedTuple, Optional, Sequence, Set, Tuple, cast
from .architecture import Architecture
from .environment import ParsedEnvironment
from .logger import log
from .typing import Literal, PathOrStr
from .util import (
BuildOptions,
BuildSelector,
NonPlatformWheelError,
download,
get_build_verbosity_extra_flags,
get_pip_script,
install_certifi_script,
prepare_command,
read_python_configs,
resources_dir,
unwrap,
)
def call(
args: Sequence[PathOrStr],
env: Optional[Dict[str, str]] = None,
cwd: Optional[str] = None,
shell: bool = False,
) -> None:
# print the command executing for the logs
if shell:
print(f"+ {args}")
else:
print("+ " + " ".join(shlex.quote(str(a)) for a in args))
subprocess.run(args, env=env, cwd=cwd, shell=shell, check=True)
def get_macos_version() -> Tuple[int, int]:
"""
Returns the macOS major/minor version, as a tuple, e.g. (10, 15) or (11, 0)
These tuples can be used in comparisons, e.g.
(10, 14) <= (11, 0) == True
(10, 14) <= (10, 16) == True
(11, 2) <= (11, 0) != True
"""
version_str, _, _ = platform.mac_ver()
version = tuple(map(int, version_str.split(".")[:2]))
return cast(Tuple[int, int], version)
def get_macos_sdks() -> List[str]:
output = subprocess.run(
["xcodebuild", "-showsdks"],
universal_newlines=True,
check=True,
stdout=subprocess.PIPE,
).stdout
return [m.group(1) for m in re.finditer(r"-sdk (macosx\S+)", output)]
class PythonConfiguration(NamedTuple):
version: str
identifier: str
url: str
def get_python_configurations(
build_selector: BuildSelector, architectures: Set[Architecture]
) -> List[PythonConfiguration]:
full_python_configs = read_python_configs("macos")
python_configurations = [PythonConfiguration(**item) for item in full_python_configs]
# filter out configs that don't match any of the selected architectures
python_configurations = [
c
for c in python_configurations
if any(c.identifier.endswith(a.value) for a in architectures)
]
# skip builds as required by BUILD/SKIP
python_configurations = [c for c in python_configurations if build_selector(c.identifier)]
# When running on macOS 11 and x86_64, the reported OS is '10.16', but
# there is no such OS - it really means macOS 11.
if get_macos_version() >= (10, 16):
if any(c.identifier.startswith("pp") for c in python_configurations):
# pypy doesn't work on macOS 11 yet
# See https://foss.heptapod.net/pypy/pypy/-/issues/3314
log.warning(
unwrap(
"""
PyPy is currently unsupported when building on macOS 11. To build macOS PyPy
wheels, build on an older OS, such as macOS 10.15. To silence this warning,
deselect PyPy by adding "pp*-macosx*" to your CIBW_SKIP option.
"""
)
)
python_configurations = [
c for c in python_configurations if not c.identifier.startswith("pp")
]
if any(c.identifier.startswith("cp35") for c in python_configurations):
# CPython 3.5 doesn't work on macOS 11
log.warning(
unwrap(
"""
CPython 3.5 is unsupported when building on macOS 11. To build CPython 3.5
wheels, build on an older OS, such as macOS 10.15. To silence this warning,
deselect CPython 3.5 by adding "cp35-macosx_x86_64" to your CIBW_SKIP option.
"""
)
)
python_configurations = [
c for c in python_configurations if not c.identifier.startswith("cp35")
]
return python_configurations
SYMLINKS_DIR = Path("/tmp/cibw_bin")
def make_symlinks(installation_bin_path: Path, python_executable: str, pip_executable: str) -> None:
assert (installation_bin_path / python_executable).exists()
# Python bin folders on Mac don't symlink `python3` to `python`, and neither
# does PyPy for `pypy` or `pypy3`, so we do that so `python` and `pip` always
# point to the active configuration.
if SYMLINKS_DIR.exists():
shutil.rmtree(SYMLINKS_DIR)
SYMLINKS_DIR.mkdir(parents=True)
(SYMLINKS_DIR / "python").symlink_to(installation_bin_path / python_executable)
(SYMLINKS_DIR / "python-config").symlink_to(
installation_bin_path / (python_executable + "-config")
)
(SYMLINKS_DIR / "pip").symlink_to(installation_bin_path / pip_executable)
def install_cpython(version: str, url: str) -> Path:
installed_system_packages = subprocess.run(
["pkgutil", "--pkgs"], universal_newlines=True, check=True, stdout=subprocess.PIPE
).stdout.splitlines()
# if this version of python isn't installed, get it from python.org and install
python_package_identifier = f"org.python.Python.PythonFramework-{version}"
python_executable = "python3" if version[0] == "3" else "python"
installation_bin_path = Path(f"/Library/Frameworks/Python.framework/Versions/{version}/bin")
if python_package_identifier not in installed_system_packages:
# download the pkg
download(url, Path("/tmp/Python.pkg"))
# install
call(["sudo", "installer", "-pkg", "/tmp/Python.pkg", "-target", "/"])
# patch open ssl
if version == "3.5":
open_ssl_patch_url = f"https://github.com/mayeut/patch-macos-python-openssl/releases/download/v1.1.1h/patch-macos-python-{version}-openssl-v1.1.1h.tar.gz"
download(open_ssl_patch_url, Path("/tmp/python-patch.tar.gz"))
call(
[
"sudo",
"tar",
"-C",
f"/Library/Frameworks/Python.framework/Versions/{version}/",
"-xmf",
"/tmp/python-patch.tar.gz",
]
)
call(["sudo", str(installation_bin_path / python_executable), str(install_certifi_script)])
pip_executable = "pip3" if version[0] == "3" else "pip"
make_symlinks(installation_bin_path, python_executable, pip_executable)
return installation_bin_path
def install_pypy(version: str, url: str) -> Path:
pypy_tar_bz2 = url.rsplit("/", 1)[-1]
extension = ".tar.bz2"
assert pypy_tar_bz2.endswith(extension)
pypy_base_filename = pypy_tar_bz2[: -len(extension)]
installation_path = Path("/tmp") / pypy_base_filename
if not installation_path.exists():
downloaded_tar_bz2 = Path("/tmp") / pypy_tar_bz2
download(url, downloaded_tar_bz2)
call(["tar", "-C", "/tmp", "-xf", downloaded_tar_bz2])
# Patch PyPy to make sure headers get installed into a venv
patch_version = "_27" if version == "2.7" else ""
patch_path = resources_dir / f"pypy_venv{patch_version}.patch"
call(["patch", "--force", "-p1", "-d", installation_path, "-i", patch_path])
installation_bin_path = installation_path / "bin"
python_executable = "pypy3" if version[0] == "3" else "pypy"
pip_executable = "pip3" if version[0] == "3" else "pip"
make_symlinks(installation_bin_path, python_executable, pip_executable)
return installation_bin_path
def setup_python(
python_configuration: PythonConfiguration,
dependency_constraint_flags: Sequence[PathOrStr],
environment: ParsedEnvironment,
) -> Dict[str, str]:
implementation_id = python_configuration.identifier.split("-")[0]
log.step(f"Installing Python {implementation_id}...")
if implementation_id.startswith("cp"):
installation_bin_path = install_cpython(
python_configuration.version, python_configuration.url
)
elif implementation_id.startswith("pp"):
installation_bin_path = install_pypy(python_configuration.version, python_configuration.url)
else:
raise ValueError("Unknown Python implementation")
log.step("Setting up build environment...")
env = os.environ.copy()
env["PATH"] = os.pathsep.join(
[
str(SYMLINKS_DIR),
str(installation_bin_path),
env["PATH"],
]
)
# Fix issue with site.py setting the wrong `sys.prefix`, `sys.exec_prefix`,
# `sys.path`, ... for PyPy: https://foss.heptapod.net/pypy/pypy/issues/3175
# Also fix an issue with the shebang of installed scripts inside the
# testing virtualenv- see https://github.com/theacodes/nox/issues/44 and
# https://github.com/pypa/virtualenv/issues/620
# Also see https://github.com/python/cpython/pull/9516
env.pop("__PYVENV_LAUNCHER__", None)
env = environment.as_dictionary(prev_environment=env)
# we version pip ourselves, so we don't care about pip version checking
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
# check what version we're on
call(["which", "python"], env=env)
call(["python", "--version"], env=env)
which_python = subprocess.run(
["which", "python"], env=env, universal_newlines=True, check=True, stdout=subprocess.PIPE
).stdout.strip()
if which_python != "/tmp/cibw_bin/python":
print(
"cibuildwheel: python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it.",
file=sys.stderr,
)
sys.exit(1)
# install pip & wheel
call(["python", get_pip_script, *dependency_constraint_flags], env=env, cwd="/tmp")
assert (installation_bin_path / "pip").exists()
call(["which", "pip"], env=env)
call(["pip", "--version"], env=env)
which_pip = subprocess.run(
["which", "pip"], env=env, universal_newlines=True, check=True, stdout=subprocess.PIPE
).stdout.strip()
if which_pip != "/tmp/cibw_bin/pip":
print(
"cibuildwheel: pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it.",
file=sys.stderr,
)
sys.exit(1)
# Set MACOSX_DEPLOYMENT_TARGET to 10.9, if the user didn't set it.
# CPython 3.5 defaults to 10.6, and pypy defaults to 10.7, causing
# inconsistencies if it's left unset.
env.setdefault("MACOSX_DEPLOYMENT_TARGET", "10.9")
config_is_arm64 = python_configuration.identifier.endswith("arm64")
config_is_universal2 = python_configuration.identifier.endswith("universal2")
if python_configuration.version == "3.5":
# Cross-compilation platform override - CPython 3.5 has an
# i386/x86_64 version of Python, but we only want a x64_64 build
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-10.9-x86_64")
# https://github.com/python/cpython/blob/a5ed2fe0eedefa1649aa93ee74a0bafc8e628a10/Lib/_osx_support.py#L260
env.setdefault("ARCHFLAGS", "-arch x86_64")
elif python_configuration.version not in {"3.6", "3.7"}:
if config_is_arm64:
# macOS 11 is the first OS with arm64 support, so the wheels
# have that as a minimum.
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-11.0-arm64")
env.setdefault("ARCHFLAGS", "-arch arm64")
elif config_is_universal2:
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-10.9-universal2")
env.setdefault("ARCHFLAGS", "-arch arm64 -arch x86_64")
elif python_configuration.identifier.endswith("x86_64"):
# even on the macos11.0 Python installer, on the x86_64 side it's
# compatible back to 10.9.
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-10.9-x86_64")
env.setdefault("ARCHFLAGS", "-arch x86_64")
building_arm64 = config_is_arm64 or config_is_universal2
if building_arm64 and get_macos_version() < (10, 16) and "SDKROOT" not in env:
# xcode 12.2 or higher can build arm64 on macos 10.15 or below, but
# needs the correct SDK selected.
sdks = get_macos_sdks()
# Different versions of Xcode contain different SDK versions...
# we're happy with anything newer than macOS 11.0
arm64_compatible_sdks = [s for s in sdks if not s.startswith("macosx10.")]
if not arm64_compatible_sdks:
log.warning(
unwrap(
"""
SDK for building arm64-compatible wheels not found. You need Xcode 12.2 or later
to build universal2 or arm64 wheels.
"""
)
)
else:
env.setdefault("SDKROOT", arm64_compatible_sdks[0])
log.step("Installing build tools...")
call(
[
"pip",
"install",
"--upgrade",
"setuptools",
"wheel",
"delocate",
*dependency_constraint_flags,
],
env=env,
)
return env
def build(options: BuildOptions) -> None:
temp_dir = Path(tempfile.mkdtemp(prefix="cibuildwheel"))
built_wheel_dir = temp_dir / "built_wheel"
repaired_wheel_dir = temp_dir / "repaired_wheel"
try:
if options.before_all:
log.step("Running before_all...")
env = options.environment.as_dictionary(prev_environment=os.environ)
env.setdefault("MACOSX_DEPLOYMENT_TARGET", "10.9")
before_all_prepared = prepare_command(
options.before_all, project=".", package=options.package_dir
)
call([before_all_prepared], shell=True, env=env)
python_configurations = get_python_configurations(
options.build_selector, options.architectures
)
for config in python_configurations:
log.build_start(config.identifier)
config_is_arm64 = config.identifier.endswith("arm64")
config_is_universal2 = config.identifier.endswith("universal2")
dependency_constraint_flags: Sequence[PathOrStr] = []
if options.dependency_constraints:
dependency_constraint_flags = [
"-c",
options.dependency_constraints.get_for_python_version(config.version),
]
env = setup_python(config, dependency_constraint_flags, options.environment)
if options.before_build:
log.step("Running before_build...")
before_build_prepared = prepare_command(
options.before_build, project=".", package=options.package_dir
)
call(before_build_prepared, env=env, shell=True)
log.step("Building wheel...")
if built_wheel_dir.exists():
shutil.rmtree(built_wheel_dir)
built_wheel_dir.mkdir(parents=True)
# Path.resolve() is needed. Without it pip wheel may try to fetch package from pypi.org
# see https://github.com/pypa/cibuildwheel/pull/369
call(
[
"pip",
"wheel",
options.package_dir.resolve(),
"--wheel-dir",
built_wheel_dir,
"--no-deps",
*get_build_verbosity_extra_flags(options.build_verbosity),
],
env=env,
)
built_wheel = next(built_wheel_dir.glob("*.whl"))
if repaired_wheel_dir.exists():
shutil.rmtree(repaired_wheel_dir)
repaired_wheel_dir.mkdir(parents=True)
if built_wheel.name.endswith("none-any.whl"):
raise NonPlatformWheelError()
if options.repair_command:
log.step("Repairing wheel...")
if config_is_universal2:
delocate_archs = "x86_64,arm64"
elif config_is_arm64:
delocate_archs = "arm64"
else:
delocate_archs = "x86_64"
repair_command_prepared = prepare_command(
options.repair_command,
wheel=built_wheel,
dest_dir=repaired_wheel_dir,
delocate_archs=delocate_archs,
)
call(repair_command_prepared, env=env, shell=True)
else:
shutil.move(str(built_wheel), repaired_wheel_dir)
repaired_wheel = next(repaired_wheel_dir.glob("*.whl"))
log.step_end()
if options.test_command and options.test_selector(config.identifier):
machine_arch = platform.machine()
testing_archs: List[Literal["x86_64", "arm64"]] = []
if config_is_arm64:
testing_archs = ["arm64"]
elif config_is_universal2:
testing_archs = ["x86_64", "arm64"]
else:
testing_archs = ["x86_64"]
for testing_arch in testing_archs:
if config_is_universal2:
arch_specific_identifier = f"{config.identifier}:{testing_arch}"
if not options.test_selector(arch_specific_identifier):
continue
if machine_arch == "x86_64" and testing_arch == "arm64":
if config_is_arm64:
log.warning(
unwrap(
"""
While arm64 wheels can be built on x86_64, they cannot be
tested. The ability to test the arm64 wheels will be added in a
future release of cibuildwheel, once Apple Silicon CI runners
are widely available. To silence this warning, set
`CIBW_TEST_SKIP: *-macosx_arm64`.
"""
)
)
elif config_is_universal2:
log.warning(
unwrap(
"""
While universal2 wheels can be built on x86_64, the arm64 part
of them cannot currently be tested. The ability to test the
arm64 part of a universal2 wheel will be added in a future
release of cibuildwheel, once Apple Silicon CI runners are
widely available. To silence this warning, set
`CIBW_TEST_SKIP: *-macosx_universal2:arm64`.
"""
)
)
else:
raise RuntimeError("unreachable")
# skip this test
continue
log.step(
"Testing wheel..."
if testing_arch == machine_arch
else f"Testing wheel on {testing_arch}..."
)
# set up a virtual environment to install and test from, to make sure
# there are no dependencies that were pulled in at build time.
call(["pip", "install", "virtualenv", *dependency_constraint_flags], env=env)
venv_dir = Path(tempfile.mkdtemp())
arch_prefix = []
if testing_arch != machine_arch:
if machine_arch == "arm64" and testing_arch == "x86_64":
# rosetta2 will provide the emulation with just the arch prefix.
arch_prefix = ["arch", "-x86_64"]
else:
raise RuntimeError(
"don't know how to emulate {testing_arch} on {machine_arch}"
)
# define a custom 'call' function that adds the arch prefix each time
def call_with_arch(args: Sequence[PathOrStr], **kwargs: Any) -> None:
if isinstance(args, str):
args = " ".join(arch_prefix) + " " + args
else:
args = [*arch_prefix, *args]
call(args, **kwargs)
# Use --no-download to ensure determinism by using seed libraries
# built into virtualenv
call_with_arch(
["python", "-m", "virtualenv", "--no-download", venv_dir], env=env
)
virtualenv_env = env.copy()
virtualenv_env["PATH"] = os.pathsep.join(
[
str(venv_dir / "bin"),
virtualenv_env["PATH"],
]
)
# check that we are using the Python from the virtual environment
call_with_arch(["which", "python"], env=virtualenv_env)
if options.before_test:
before_test_prepared = prepare_command(
options.before_test, project=".", package=options.package_dir
)
call_with_arch(before_test_prepared, env=virtualenv_env, shell=True)
# install the wheel
call_with_arch(
["pip", "install", f"{repaired_wheel}{options.test_extras}"],
env=virtualenv_env,
)
# test the wheel
if options.test_requires:
call_with_arch(
["pip", "install"] + options.test_requires, env=virtualenv_env
)
# run the tests from $HOME, with an absolute path in the command
# (this ensures that Python runs the tests against the installed wheel
# and not the repo code)
test_command_prepared = prepare_command(
options.test_command,
project=Path(".").resolve(),
package=options.package_dir.resolve(),
)
call_with_arch(
test_command_prepared,
cwd=os.environ["HOME"],
env=virtualenv_env,
shell=True,
)
# clean up
shutil.rmtree(venv_dir)
# we're all done here; move it to output (overwrite existing)
shutil.move(str(repaired_wheel), options.output_dir)
log.build_end()
except subprocess.CalledProcessError as error:
log.step_end_with_error(
f"Command {error.cmd} failed with code {error.returncode}. {error.stdout}"
)
sys.exit(1)
-615
View File
@@ -1,615 +0,0 @@
from __future__ import annotations
__lazy_modules__ = {
"cibuildwheel.ci",
"cibuildwheel.errors",
"cibuildwheel.logger",
"cibuildwheel.util",
"cibuildwheel.util.cmd",
"cibuildwheel.util.helpers",
"contextlib",
"io",
"json",
"platform",
"shlex",
"shutil",
"subprocess",
"textwrap",
"uuid",
}
import contextlib
import dataclasses
import io
import json
import os
import platform
import shlex
import shutil
import subprocess
import sys
import textwrap
import typing
import uuid
from enum import Enum
from pathlib import PurePosixPath
from typing import Literal, assert_never
from cibuildwheel.ci import CIProvider, detect_ci_provider
from cibuildwheel.errors import OCIEngineTooOldError
from cibuildwheel.logger import log
from cibuildwheel.util.cmd import call
from cibuildwheel.util.helpers import FlexibleVersion, parse_key_value_string, strtobool
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Mapping, Sequence
from pathlib import Path, PurePath
from types import TracebackType
from typing import IO, Self
from cibuildwheel.typing import PathOrStr
ContainerEngineName = Literal["docker", "podman"]
# Order of the enum matters for tests. 386 shall appear before amd64.
class OCIPlatform(Enum):
i386 = "linux/386"
AMD64 = "linux/amd64"
ARMV7 = "linux/arm/v7"
ARM64 = "linux/arm64"
PPC64LE = "linux/ppc64le"
RISCV64 = "linux/riscv64"
S390X = "linux/s390x"
@classmethod
def native(cls) -> Self:
"""Return the current OCI platform, or raise ValueError if unknown."""
arch = platform.machine().lower()
mapping = {
"i386": cls.i386,
"i686": cls.i386,
"x86_64": cls.AMD64,
"amd64": cls.AMD64,
"armv7l": cls.ARMV7,
"aarch64": cls.ARM64,
"arm64": cls.ARM64,
"ppc64le": cls.PPC64LE,
"riscv64": cls.RISCV64,
"s390x": cls.S390X,
}
try:
return mapping[arch]
except KeyError as ex:
msg = f"Unsupported platform architecture: {arch}"
raise OSError(msg) from ex
@dataclasses.dataclass(frozen=True)
class OCIContainerEngineConfig:
name: ContainerEngineName
_: dataclasses.KW_ONLY
create_args: tuple[str, ...] = dataclasses.field(default_factory=tuple)
disable_host_mount: bool = False
@classmethod
def from_config_string(cls, config_string: str) -> Self:
config_dict = parse_key_value_string(
config_string,
["name"],
["create_args", "create-args", "disable_host_mount", "disable-host-mount"],
)
name = " ".join(config_dict["name"])
if name not in {"docker", "podman"}:
msg = f"unknown container engine {name}"
raise ValueError(msg)
name = typing.cast("ContainerEngineName", name)
# some flexibility in the option names to cope with TOML conventions
create_args = config_dict.get("create_args") or config_dict.get("create-args") or []
disable_host_mount_options = (
config_dict.get("disable_host_mount") or config_dict.get("disable-host-mount") or []
)
disable_host_mount = (
strtobool(disable_host_mount_options[-1]) if disable_host_mount_options else False
)
if "--platform" in create_args or any(arg.startswith("--platform=") for arg in create_args):
msg = "Using '--platform' in 'container-engine::create_args' is deprecated. It will be ignored."
log.warning(msg)
if "--platform" in create_args:
index = create_args.index("--platform")
create_args.pop(index)
create_args.pop(index)
else:
create_args = [arg for arg in create_args if not arg.startswith("--platform=")]
return cls(name=name, create_args=tuple(create_args), disable_host_mount=disable_host_mount)
def options_summary(self) -> str | dict[str, str]:
if not self.create_args:
return self.name
else:
return {
"name": self.name,
"create_args": repr(self.create_args),
"disable_host_mount": str(self.disable_host_mount),
}
DEFAULT_ENGINE = OCIContainerEngineConfig("docker")
def _check_engine_version(engine: OCIContainerEngineConfig) -> None:
try:
version_string = call(engine.name, "version", "-f", "{{json .}}", capture_stdout=True)
# We are using lowercase keys for all dicts so we are not affected by casing of keys
version_info = json.loads(
version_string.strip(), object_pairs_hook=lambda inp: {k.lower(): v for k, v in inp}
)
match engine.name:
case "docker":
client_api_version = FlexibleVersion(version_info["client"]["apiversion"])
server_api_version = FlexibleVersion(version_info["server"]["apiversion"])
# --platform support was introduced in 1.32 as experimental, 1.41 removed the experimental flag
version = min(client_api_version, server_api_version)
minimum_version = FlexibleVersion("1.41")
minimum_version_str = "20.10.0" # docker version
error_msg = textwrap.dedent(
f"""
Build failed because {engine.name} is too old.
cibuildwheel requires {engine.name}>={minimum_version_str} running API version {minimum_version}.
The API version found by cibuildwheel is {version}.
"""
)
case "podman":
# podman uses the same version string for "Version" & "ApiVersion"
client_version = FlexibleVersion(version_info["client"]["version"])
if "server" in version_info:
server_version = FlexibleVersion(version_info["server"]["version"])
else:
server_version = client_version
# --platform support was introduced in v3
version = min(client_version, server_version)
minimum_version = FlexibleVersion("3")
error_msg = textwrap.dedent(
f"""
Build failed because {engine.name} is too old.
cibuildwheel requires {engine.name}>={minimum_version}.
The version found by cibuildwheel is {version}.
"""
)
case _:
assert_never(engine.name)
if version < minimum_version:
raise OCIEngineTooOldError(error_msg) from None
except (subprocess.CalledProcessError, KeyError, ValueError) as e:
msg = f"Build failed because {engine.name} is too old or is not working properly."
raise OCIEngineTooOldError(msg) from e
class OCIContainer:
"""
An object that represents a running OCI (e.g. Docker) container.
Intended for use as a context manager e.g.
`with OCIContainer(image = 'ubuntu') as docker:`
A bash shell is running in the remote container. When `call()` is invoked,
the command is relayed to the remote shell, and the results are streamed
back to cibuildwheel.
Example:
>>> # xdoctest: +REQUIRES(LINUX)
>>> from cibuildwheel.oci_container import * # NOQA
>>> from cibuildwheel.options import _get_pinned_container_images
>>> import pytest
>>> try:
... oci_platform = OCIPlatform.native()
... except OSError as ex:
... pytest.skip(str(ex))
>>> if oci_platform != OCIPlatform.AMD64:
... pytest.skip('only runs on amd64')
>>> image = _get_pinned_container_images()['x86_64']['manylinux2014']
>>> # Test the default container
>>> with OCIContainer(image=image, oci_platform=oci_platform) as self:
... self.call(["echo", "hello world"])
... self.call(["cat", "/proc/1/cgroup"])
... print(self.get_environment())
... print(self.debug_info())
"""
UTILITY_PYTHON = "/opt/python/cp39-cp39/bin/python"
bash_stdin: IO[bytes]
bash_stdout: IO[bytes]
def __init__(
self,
*,
image: str,
oci_platform: OCIPlatform,
cwd: PathOrStr | None = None,
engine: OCIContainerEngineConfig = DEFAULT_ENGINE,
):
if not image:
msg = "Must have a non-empty image to run."
raise ValueError(msg)
self.image = image
self.oci_platform = oci_platform
self.cwd = cwd
self.name: str | None = None
self.process: subprocess.Popen[bytes] | None = None
self.engine = engine
self.host_tar_format = ""
if sys.platform.startswith("darwin"):
self.host_tar_format = "--format gnutar"
def _get_platform_args(self, *, oci_platform: OCIPlatform | None = None) -> tuple[str, str]:
if oci_platform is None:
oci_platform = self.oci_platform
# we need '--pull=always' otherwise some images with the wrong platform get reused (e.g. 386 image for amd64)
# c.f. https://github.com/moby/moby/issues/48197#issuecomment-2282802313
pull = "always"
try:
image_platform = call(
self.engine.name,
"image",
"inspect",
self.image,
"--format",
(
"{{.Os}}/{{.Architecture}}/{{.Variant}}"
if len(oci_platform.value.split("/")) == 3
else "{{.Os}}/{{.Architecture}}"
),
capture_stdout=True,
).strip()
if image_platform == oci_platform.value:
# in case the correct image is already present, don't pull
# this allows to run local only images
pull = "never"
except subprocess.CalledProcessError:
# silently fallback to "--pull=always"
pass
return f"--platform={oci_platform.value}", f"--pull={pull}"
def __enter__(self) -> Self:
assert self.process is None
self.name = f"cibuildwheel-{uuid.uuid4()}"
_check_engine_version(self.engine)
# work-around for Travis-CI PPC64le Docker runs since 2021:
# this avoids network splits
# https://github.com/pypa/cibuildwheel/issues/904
# https://github.com/conda-forge/conda-smithy/pull/1520
network_args = []
if detect_ci_provider() == CIProvider.travis_ci and platform.machine() == "ppc64le":
network_args = ["--network=host"]
platform_args = self._get_platform_args()
simulate_32_bit = False
if self.oci_platform in {OCIPlatform.i386, OCIPlatform.ARMV7}:
# If the architecture running the image is already the right one
# or the image entrypoint takes care of enforcing this, then we don't need to
# simulate this
run_cmd = [self.engine.name, "run", "--rm"]
ctr_cmd = ["uname", "-m"]
try:
container_machine = call(
*run_cmd, *platform_args, self.image, *ctr_cmd, capture_stdout=True
).strip()
except subprocess.CalledProcessError:
if self.oci_platform == OCIPlatform.i386:
# The image might have been built with amd64 architecture
# Let's try that
platform_args = self._get_platform_args(oci_platform=OCIPlatform.AMD64)
container_machine = call(
*run_cmd, *platform_args, self.image, *ctr_cmd, capture_stdout=True
).strip()
else:
raise
if container_machine not in {"i686", "armv7l", "armv8l"}:
simulate_32_bit = True
# sanity check to ensure no deadlock waiting for container to start
call(
*run_cmd,
*platform_args,
self.image,
"linux32",
"/bin/true",
capture_stdout=True,
)
shell_args = ["linux32", "/bin/bash"] if simulate_32_bit else ["/bin/bash"]
subprocess.run(
[
self.engine.name,
"create",
"--env=CIBUILDWHEEL",
"--env=SOURCE_DATE_EPOCH",
f"--name={self.name}",
"--interactive",
*(["--volume=/:/host"] if not self.engine.disable_host_mount else []),
*network_args,
*platform_args,
*self.engine.create_args,
self.image,
*shell_args,
],
check=True,
)
try:
self.process = subprocess.Popen(
[
self.engine.name,
"start",
"--attach",
"--interactive",
self.name,
],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
)
assert self.process.stdin
assert self.process.stdout
self.bash_stdin = self.process.stdin
self.bash_stdout = self.process.stdout
# run a noop command to block until the container is responding
self.call(["/bin/true"], cwd="/")
if self.cwd:
# Although `docker create -w` does create the working dir if it
# does not exist, podman does not. There does not seem to be a way
# to setup a workdir for a container running in podman.
self.call(["mkdir", "-p", os.fspath(self.cwd)], cwd="/")
except BaseException:
# clean-up
if self.process is not None:
if self.process.poll() is None:
self.process.kill()
self.process.communicate()
self.process = None
self._remove_container()
raise
return self
def __exit__(
self,
exc_type: type[BaseException] | None,
exc_val: BaseException | None,
exc_tb: TracebackType | None,
) -> None:
assert self.process is not None
try:
# Ask bash to exit cleanly and wait for the `start` process to finish.
# If the container/bash has already died, the write raises
# BrokenPipeError; if bash refuses to exit, `wait` raises
# TimeoutExpired. Both are handled below so we always reach the
# cleanup in `finally` rather than leaking the process or container.
self.bash_stdin.write(b"exit 0\n")
self.bash_stdin.flush()
self.process.wait(timeout=30)
if self.engine.name == "podman":
# This works around what seems to be a race condition in the
# podman backend. The full reason is not understood. See PR #966
# for a discussion on possible causes and attempts to remove this
# line. For now, this seems to work "well enough".
self.process.wait()
except (OSError, subprocess.TimeoutExpired):
# bash didn't shut down cleanly; force the process down so it isn't
# leaked, then continue to cleanup.
if self.process.poll() is None:
self.process.kill()
self.process.wait()
finally:
with contextlib.suppress(OSError):
self.bash_stdin.close()
with contextlib.suppress(OSError):
self.bash_stdout.close()
self.process = None
keep_container = strtobool(os.environ.get("CIBW_DEBUG_KEEP_CONTAINER", ""))
if not keep_container:
self._remove_container()
def _remove_container(self) -> None:
assert self.name is not None
result = subprocess.run(
[self.engine.name, "rm", "--force", "-v", self.name],
stdout=subprocess.DEVNULL,
check=False,
)
# only warn when not running in CI
if result.returncode != 0 and detect_ci_provider() is None:
msg = f"Failed to remove {self.name!r} container."
log.warning(msg)
self.name = None
def copy_into(self, from_path: Path, to_path: PurePath) -> None:
if from_path.is_dir():
self.call(["mkdir", "-p", to_path])
subprocess.run(
f"tar -c {self.host_tar_format} -f - . | {self.engine.name} exec -i {self.name} tar --no-same-owner -xC {shell_quote(to_path)} -f -",
shell=True,
check=True,
cwd=from_path,
)
else:
exec_process: subprocess.Popen[bytes]
self.call(["mkdir", "-p", to_path.parent])
with subprocess.Popen(
[
self.engine.name,
"exec",
"-i",
str(self.name),
"sh",
"-c",
f"cat > {shell_quote(to_path)}",
],
stdin=subprocess.PIPE,
) as exec_process:
assert exec_process.stdin
with open(from_path, "rb") as from_file:
shutil.copyfileobj(from_file, exec_process.stdin)
exec_process.stdin.close()
exec_process.wait()
if exec_process.returncode:
raise subprocess.CalledProcessError(
exec_process.returncode, exec_process.args, None, None
)
def copy_out(self, from_path: PurePath, to_path: Path) -> None:
# note: we assume from_path is a dir
to_path.mkdir(parents=True, exist_ok=True)
call(self.engine.name, "cp", f"{self.name}:{from_path}/.", to_path)
def glob(self, path: PurePosixPath, pattern: str) -> list[PurePosixPath]:
glob_pattern = path.joinpath(pattern)
path_strings = json.loads(
self.call(
[
self.UTILITY_PYTHON,
"-c",
f"import sys, json, glob; json.dump(glob.glob({str(glob_pattern)!r}), sys.stdout)",
],
capture_output=True,
)
)
return [PurePosixPath(p) for p in path_strings]
def call(
self,
args: Sequence[PathOrStr],
env: Mapping[str, str] | None = None,
capture_output: bool = False,
cwd: PathOrStr | None = None,
) -> str:
if cwd is None:
# Podman does not start the a container in a specific working dir
# so we always need to specify it when making calls.
cwd = self.cwd
chdir = f"cd {cwd}" if cwd else ""
env_assignments = (
" ".join(f"{shlex.quote(k)}={shlex.quote(v)}" for k, v in env.items())
if env is not None
else ""
)
command = " ".join(shlex.quote(str(a)) for a in args)
end_of_message = str(uuid.uuid4())
# log the command we're executing
print(f" + {command}")
# Write a command to the remote shell. First we change the
# cwd, if that's required. Then, we use the `env` utility to run
# `command` inside the specified environment. We use `env` because it
# can cope with spaces and strange characters in the name or value.
# Finally, the remote shell is told to write a footer - this will show
# up in the output so we know when to stop reading, and will include
# the return code of `command`.
self.bash_stdin.write(
bytes(
f"""(
{chdir}
env {env_assignments} {command}
printf "%04d%s\n" $? {end_of_message}
)
""",
encoding="utf8",
errors="surrogateescape",
)
)
self.bash_stdin.flush()
if capture_output:
output_io: IO[bytes] = io.BytesIO()
else:
output_io = sys.stdout.buffer
while True:
line = self.bash_stdout.readline()
if line.endswith(bytes(end_of_message, encoding="utf8") + b"\n"):
# fmt: off
footer_offset = (
len(line)
- 1 # newline character
- len(end_of_message) # delimiter
- 4 # 4 return code decimals
)
# fmt: on
return_code_str = line[footer_offset : footer_offset + 4]
return_code = int(return_code_str)
# add the last line to output, without the footer
output_io.write(line[0:footer_offset])
output_io.flush()
break
else:
output_io.write(line)
output_io.flush()
if isinstance(output_io, io.BytesIO):
output = str(output_io.getvalue(), encoding="utf8", errors="surrogateescape")
else:
output = ""
if return_code != 0:
raise subprocess.CalledProcessError(return_code, args, output)
return output
def get_environment(self) -> dict[str, str]:
env = json.loads(
self.call(
[
self.UTILITY_PYTHON,
"-c",
"import sys, json, os; json.dump(os.environ.copy(), sys.stdout)",
],
capture_output=True,
)
)
return typing.cast("dict[str, str]", env)
def environment_executor(self, command: Sequence[str], environment: dict[str, str]) -> str:
# used as an EnvironmentExecutor to evaluate commands and capture output
return self.call(command, env=environment, capture_output=True)
def debug_info(self) -> str:
command = [self.engine.name, "info"]
if self.engine.name == "podman":
command.append("--debug")
completed = subprocess.run(
command,
check=True,
cwd=self.cwd,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
)
output = str(completed.stdout, encoding="utf8", errors="surrogateescape")
return output
def shell_quote(path: PurePath) -> str:
return shlex.quote(os.fspath(path))
File diff suppressed because it is too large Load Diff
-65
View File
@@ -1,65 +0,0 @@
from __future__ import annotations
import sys
from typing import Protocol
from cibuildwheel import errors
from cibuildwheel.platforms import android, ios, linux, macos, pyodide, windows
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Sequence
from pathlib import Path
from typing import Final
from cibuildwheel.architecture import Architecture
from cibuildwheel.options import Options
from cibuildwheel.selector import BuildSelector
from cibuildwheel.typing import GenericPythonConfiguration, PlatformName
class PlatformModule(Protocol):
# note that as per PEP544, the self argument is ignored when the protocol
# is applied to a module
def all_python_configurations(self) -> Sequence[GenericPythonConfiguration]: ...
def get_python_configurations(
self, build_selector: BuildSelector, architectures: set[Architecture]
) -> Sequence[GenericPythonConfiguration]: ...
def build(self, options: Options, tmp_path: Path) -> None: ...
ALL_PLATFORM_MODULES: Final[dict[PlatformName, PlatformModule]] = {
"linux": linux,
"windows": windows,
"macos": macos,
"pyodide": pyodide,
"android": android,
"ios": ios,
}
def native_platform() -> PlatformName:
if sys.platform.startswith("linux"):
return "linux"
elif sys.platform == "darwin":
return "macos"
elif sys.platform == "win32":
return "windows"
else:
msg = (
'Unable to detect platform from "sys.platform". cibuildwheel doesn\'t '
"support building wheels for this platform. You might be able to build for a different "
"platform using the --platform argument. Check --help output for more information."
)
raise errors.ConfigurationError(msg)
def get_build_identifiers(
platform_module: PlatformModule,
build_selector: BuildSelector,
architectures: set[Architecture],
) -> list[str]:
python_configurations = platform_module.get_python_configurations(build_selector, architectures)
return [config.identifier for config in python_configurations]
-785
View File
@@ -1,785 +0,0 @@
from __future__ import annotations
__lazy_modules__ = {
"build",
"build.env",
"cibuildwheel.architecture",
"cibuildwheel.audit",
"cibuildwheel.frontend",
"cibuildwheel.logger",
"cibuildwheel.util.cmd",
"cibuildwheel.util.file",
"cibuildwheel.util.helpers",
"cibuildwheel.util.packaging",
"cibuildwheel.util.python_build_standalone",
"cibuildwheel.venv",
"filelock",
"packaging",
"packaging.utils",
"pathlib",
"platform",
"pprint",
"re",
"runpy",
"shlex",
"shutil",
"subprocess",
"textwrap",
"typing",
}
import os
import platform
import re
import shlex
import shutil
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
from pprint import pprint
from runpy import run_path
from textwrap import dedent
from typing import Any
from build import ProjectBuilder
from build.env import IsolatedEnv
from filelock import FileLock
from packaging.utils import canonicalize_name
from cibuildwheel import errors, platforms # pylint: disable=cyclic-import
from cibuildwheel.architecture import Architecture, arch_synonym
from cibuildwheel.audit import run_audit
from cibuildwheel.frontend import (
get_build_frontend_extra_flags,
parse_config_settings,
prepare_config_settings,
)
from cibuildwheel.logger import log
from cibuildwheel.util import resources
from cibuildwheel.util.cmd import call, shell
from cibuildwheel.util.file import (
CIBW_CACHE_PATH,
copy_test_sources,
download,
move_file,
remove_on_error,
)
from cibuildwheel.util.helpers import prepare_command
from cibuildwheel.util.packaging import find_compatible_wheel
from cibuildwheel.util.python_build_standalone import create_python_build_standalone_environment
from cibuildwheel.venv import constraint_flags, find_uv, virtualenv
TYPE_CHECKING = False
if TYPE_CHECKING:
from cibuildwheel.options import BuildOptions, Options
from cibuildwheel.selector import BuildSelector
from cibuildwheel.typing import PathOrStr
RESOURCES_ANDROID = resources.PATH / "android"
ANDROID_TRIPLET = {
"arm64_v8a": "aarch64-linux-android",
"x86_64": "x86_64-linux-android",
}
def parse_identifier(identifier: str) -> tuple[str, str]:
match = re.fullmatch(r"cp(\d)(\d+)-android_(.+)", identifier)
if not match:
msg = f"invalid Android identifier: '{identifier}'"
raise ValueError(msg)
major, minor, arch = match.groups()
return (f"{major}.{minor}", arch)
def android_triplet(identifier: str) -> str:
return ANDROID_TRIPLET[parse_identifier(identifier)[1]]
@dataclass(frozen=True)
class PythonConfiguration:
version: str
identifier: str
url: str
sha256: str
@property
def arch(self) -> str:
return parse_identifier(self.identifier)[1]
def all_python_configurations() -> list[PythonConfiguration]:
return [PythonConfiguration(**item) for item in resources.read_python_configs("android")]
def get_python_configurations(
build_selector: BuildSelector, architectures: set[Architecture]
) -> list[PythonConfiguration]:
return [
c
for c in all_python_configurations()
if c.arch in architectures and build_selector(c.identifier)
]
def shell_prepared(command: str, *, build_options: BuildOptions, env: dict[str, str]) -> None:
shell(
prepare_command(command, project=".", package=build_options.package_dir),
env=env,
)
def before_all(options: Options, python_configurations: list[PythonConfiguration]) -> None:
before_all_options = options.build_options(python_configurations[0].identifier)
if before_all_options.before_all:
log.step("Running before_all...")
shell_prepared(
before_all_options.before_all,
build_options=before_all_options,
env=before_all_options.environment.as_dictionary(os.environ),
)
@dataclass(frozen=True)
class BuildState:
config: PythonConfiguration
options: BuildOptions
build_path: Path
python_dir: Path
build_env: dict[str, str]
android_env: dict[str, str]
def build(options: Options, tmp_path: Path) -> None:
if "ANDROID_HOME" not in os.environ:
msg = (
"ANDROID_HOME environment variable is not set. For instructions, see "
"https://cibuildwheel.pypa.io/en/stable/platforms/#android"
)
raise errors.FatalError(msg)
configs = get_python_configurations(
options.globals.build_selector, options.globals.architectures
)
if not configs:
return
try:
before_all(options, configs)
built_wheels: list[Path] = []
for config in configs:
log.build_start(config.identifier)
build_options = options.build_options(config.identifier)
build_path = tmp_path / config.identifier
build_path.mkdir()
python_dir = setup_target_python(config, build_path)
build_env, android_env = setup_env(config, build_options, build_path, python_dir)
state = BuildState(
config, build_options, build_path, python_dir, build_env, android_env
)
setup_xbuild_files(state)
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
if compatible_wheel:
print(
f"\nFound previously built wheel {compatible_wheel.name} that is "
f"compatible with {config.identifier}. Skipping build step..."
)
repaired_wheel = compatible_wheel
else:
before_build(state)
built_wheel = build_wheel(state)
repaired_wheel = repair_wheel(state, built_wheel)
run_audit(tmp_dir=tmp_path, build_options=build_options, wheel=repaired_wheel)
test_wheel(state, repaired_wheel)
output_wheel: Path | None = None
if compatible_wheel is None:
output_wheel = move_file(
repaired_wheel, build_options.output_dir / repaired_wheel.name
)
built_wheels.append(output_wheel)
shutil.rmtree(build_path)
log.build_end(output_wheel)
except subprocess.CalledProcessError as error:
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
raise errors.FatalError(msg) from error
def setup_target_python(config: PythonConfiguration, build_path: Path) -> Path:
log.step("Installing target Python...")
python_tgz = CIBW_CACHE_PATH / config.url.rpartition("/")[-1]
with FileLock(f"{python_tgz}.lock"):
if not python_tgz.exists():
with remove_on_error(python_tgz):
download(config.url, python_tgz, sha256=config.sha256)
python_dir = build_path / "python"
python_dir.mkdir()
shutil.unpack_archive(python_tgz, python_dir)
# Patch a testbed bug. This code and the patch file can both be removed once we've
# updated to Python versions that include the fix.
call("patch", "-p1", "-i", RESOURCES_ANDROID / "android.patch", cwd=python_dir)
# Work around https://github.com/python/cpython/issues/138800. This can be removed
# once we've updated to Python versions that include the fix.
pc_path = python_dir / f"prefix/lib/pkgconfig/python-{config.version}.pc"
pc_path.write_text(pc_path.read_text().replace("$(BLDLIBRARY)", f"-lpython{config.version}"))
return python_dir
def setup_env(
config: PythonConfiguration, build_options: BuildOptions, build_path: Path, python_dir: Path
) -> tuple[dict[str, str], dict[str, str]]:
"""
Returns two environment dicts, both pointing at the same virtual environment:
* build_env, which uses the environment normally.
* android_env, which uses the environment while simulating running on Android.
"""
log.step("Setting up build environment...")
use_uv, pip = find_pip(build_options)
# Create virtual environment
python_exe = create_python_build_standalone_environment(
config.version, build_path, CIBW_CACHE_PATH
)
venv_dir = build_path / "venv"
dependency_constraint = build_options.dependency_constraints.get_for_python_version(
version=config.version, tmp_dir=build_path
)
build_env = virtualenv(
config.version, python_exe, venv_dir, dependency_constraint, use_uv=use_uv
)
create_cmake_toolchain(config, build_path, python_dir, build_env)
# See platforms.md for the reason why we use this default API level.
build_env.setdefault("ANDROID_API_LEVEL", "24")
# Apply custom environment variables, and check environment is still valid
build_env = build_options.environment.as_dictionary(build_env)
build_env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
build_env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
for command in ["python"] if use_uv else ["python", "pip"]:
command_path = call("which", command, env=build_env, capture_stdout=True).strip()
if command_path != f"{venv_dir}/bin/{command}":
msg = (
f"{command} available on PATH doesn't match our installed instance. If you "
f"have modified PATH, ensure that you don't overwrite cibuildwheel's entry "
f"or insert {command} above it."
)
raise errors.FatalError(msg)
if command == "python":
call(command, "-V", "-V", env=build_env)
else:
call(command, "--version", env=build_env)
# Install build tools
tools = ["auditwheel", "patchelf", "pkgconf"]
if build_options.build_frontend.name in {"build", "build[uv]"}:
tools.append("build")
call(*pip, "install", *tools, *constraint_flags(dependency_constraint), env=build_env)
# Construct an altered environment which simulates running on Android.
android_env = setup_android_env(config, python_dir, build_env)
# Build-time requirements must be queried within android_env, because
# `get_requires_for_build` can run arbitrary code in setup.py scripts, which may be
# affected by the target platform. However, the requirements must be installed
# within build_env, because they're going to run on the build machine.
#
# The `build` CLI doesn't support this combination, so we use its API to query the
# requirements, and then install them ourselves with pip. We'll later run `build` in
# the same environment, passing the `--no-isolation` option.
class AndroidEnv(IsolatedEnv):
@property
def python_executable(self) -> str:
return f"{venv_dir}/bin/python"
def make_extra_environ(self) -> dict[str, str]:
return android_env
pb = ProjectBuilder.from_isolated_env(AndroidEnv(), build_options.package_dir)
if pb.build_system_requires:
call(*pip, "install", *pb.build_system_requires, env=build_env)
requires_for_build = pb.get_requires_for_build(
"wheel",
parse_config_settings(
prepare_config_settings(
build_options.config_settings, project=".", package=build_options.package_dir
)
),
)
if requires_for_build:
call(*pip, "install", *requires_for_build, env=build_env)
return build_env, android_env
def create_cmake_toolchain(
config: PythonConfiguration, build_path: Path, python_dir: Path, build_env: dict[str, str]
) -> None:
toolchain_path = build_path / "toolchain.cmake"
build_env["CMAKE_TOOLCHAIN_FILE"] = str(toolchain_path)
with open(toolchain_path, "w", encoding="UTF-8") as toolchain_file:
print(
dedent(
f"""\
# To support as many build systems as possible, we use environment
# variables as the single source of truth for compiler flags and paths,
# so they don't need to be specified here.
set(CMAKE_SYSTEM_NAME Android)
set(CMAKE_SYSTEM_PROCESSOR {android_triplet(config.identifier).split("-")[0]})
# Inhibit all of CMake's own NDK handling code.
set(CMAKE_SYSTEM_VERSION 1)
# Tell CMake where to look for headers and libraries.
set(CMAKE_FIND_ROOT_PATH "{python_dir}/prefix")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE BOTH)
# Allow CMake to run Python in the simulated Android environment when
# policy CMP0190 is active.
set(CMAKE_CROSSCOMPILING_EMULATOR /bin/sh -c [["$0" "$@"]])
"""
),
file=toolchain_file,
)
def localize_sysconfigdata(
python_dir: Path, build_env: dict[str, str], sysconfigdata_path: Path
) -> dict[str, Any]:
sysconfigdata: dict[str, Any] = run_path(str(sysconfigdata_path))["build_time_vars"]
with sysconfigdata_path.open("w", encoding="UTF-8") as f:
f.write("# Generated by cibuildwheel\n")
f.write("build_time_vars = ")
sysconfigdata = localized_vars(build_env, sysconfigdata, python_dir / "prefix")
pprint(sysconfigdata, stream=f, compact=True)
return sysconfigdata
def localized_vars(
build_env: dict[str, str], orig_vars: dict[str, Any], prefix: Path
) -> dict[str, Any]:
orig_prefix = orig_vars["prefix"]
localized_vars_ = {}
for key, value in orig_vars.items():
# The host's sysconfigdata will include references to build-time paths.
# Update these to refer to the current prefix.
final = value
if isinstance(final, str):
final = final.replace(orig_prefix, str(prefix))
if key == "ANDROID_API_LEVEL":
try:
final = int(build_env[key])
except ValueError as e:
msg = f"ANDROID_API_LEVEL: {e}. This variable must be an integer."
raise errors.FatalError(msg) from e
# Build systems vary in whether FLAGS variables are read from sysconfig, and if so,
# whether they're replaced by environment variables or combined with them. Even
# setuptools has changed its behavior here
# (https://github.com/pypa/setuptools/issues/4836).
#
# Ensure consistency by clearing the sysconfig variables and letting the environment
# variables take effect alone. This will also work for any non-Python build systems
# which the build script may call.
elif key in ["CFLAGS", "CXXFLAGS", "LDFLAGS"]:
final = ""
# These variables contain an embedded copy of LDFLAGS.
elif key in ["LDSHARED", "LDCXXSHARED"]:
final = final.removesuffix(" " + orig_vars["LDFLAGS"])
localized_vars_[key] = final
return localized_vars_
def setup_android_env(
config: PythonConfiguration, python_dir: Path, build_env: dict[str, str]
) -> dict[str, str]:
site_packages = find_site_packages(build_env)
for suffix in ["pth", "py"]:
shutil.copy(RESOURCES_ANDROID / f"_cross_venv.{suffix}", site_packages)
sysconfigdata_path = Path(
shutil.copy(
glob1(python_dir, "prefix/lib/python*/_sysconfigdata_*.py"),
site_packages,
)
)
sysconfigdata = localize_sysconfigdata(python_dir, build_env, sysconfigdata_path)
# Activate the code in _cross_venv.py.
android_env = build_env.copy()
android_env["CIBW_HOST_TRIPLET"] = android_triplet(config.identifier)
# Get the environment variables needed to build for Android (CC, CFLAGS, etc). These are
# generated by https://github.com/python/cpython/blob/main/Android/android-env.sh.
env_output = call(python_dir / "android.py", "env", env=build_env, capture_stdout=True)
# shlex.split should produce a sequence alternating between:
# * the word "export"
# * a key=value string, without quotes
for i, token in enumerate(shlex.split(env_output)):
if i % 2 == 0:
if token != "export":
msg = f"Unexpected output from android.py env: expected 'export', got {token!r}"
raise errors.FatalError(msg)
else:
key, sep, value = token.partition("=")
if sep != "=":
msg = f"Unexpected output from android.py env: expected 'key=value', got {token!r}"
raise errors.FatalError(msg)
android_env[key] = value
# localized_vars cleared the CFLAGS and CXXFLAGS in the sysconfigdata, but most
# packages take their optimization flags from these variables. Pass these flags via
# environment variables instead.
#
# We don't enable debug information, because it significantly increases binary size,
# and most Android app developers don't have the NDK installed, so they would have no
# way to strip it.
opt = " ".join(word for word in sysconfigdata["OPT"].split() if not word.startswith("-g"))
for key in ["CFLAGS", "CXXFLAGS"]:
android_env[key] += " " + opt
# Cargo target linker needs to be specified after CC is set
setup_rust(config, python_dir, android_env)
# Create shims which install additional build tools on first use.
setup_fortran(android_env)
# `android.py env` returns PKG_CONFIG="pkg-config --define-prefix", but some build
# systems can't handle arguments in that variable. Since we have a known version
# of pkgconf, it's safe to use PKG_CONFIG_RELOCATE_PATHS instead.
android_env["PKG_CONFIG"] = call(
"which", "pkgconf-pypi", env=build_env, capture_stdout=True
).strip()
android_env["PKG_CONFIG_RELOCATE_PATHS"] = "1"
# Format the environment so it can be pasted into a shell when debugging.
for key, value in sorted(android_env.items()):
if os.environ.get(key) != value:
print(f"export {key}={shlex.quote(value)}")
return android_env
def setup_rust(config: PythonConfiguration, python_dir: Path, env: dict[str, str]) -> None:
cargo_target = android_triplet(config.identifier)
# CARGO_BUILD_TARGET is the variable used by Cargo and setuptools_rust
env["CARGO_BUILD_TARGET"] = cargo_target
# The linker needs to be specified after CC is set by android-env.sh
cargo_target_linker_env_name = f"CARGO_TARGET_{cargo_target.upper().replace('-', '_')}_LINKER"
# CC has already been set by calling android.py (it calls android-env.sh)
env[cargo_target_linker_env_name] = env["CC"]
# All Python extension modules must be explicitly linked against libpython3.x.so when building for Android.
# See: https://peps.python.org/pep-0738/#linkage
# For projects using PyO3, this requires setting PYO3_CROSS_LIB_DIR to the directory containing libpython3.x.so.
# See: https://pyo3.rs/v0.27.1/building-and-distribution.html#cross-compiling
env["PYO3_CROSS_LIB_DIR"] = str(python_dir / "prefix" / "lib")
venv_bin = Path(env["VIRTUAL_ENV"]) / "bin"
for tool in ["cargo", "rustup"]:
shim_path = venv_bin / tool
shutil.copy(RESOURCES_ANDROID / "rust_shim.py", shim_path)
shim_path.chmod(0o755)
def setup_fortran(env: dict[str, str]) -> None:
# In case there's any autodetection based on the executable name, use the same name
# as the real executable (see fortran_shim.run_flang)
shim_in = RESOURCES_ANDROID / "fortran_shim.py"
shim_out = Path(env["VIRTUAL_ENV"]) / "bin/flang-new"
# The hashbang line runs the shim in cibuildwheel's own virtual environment, so it
# has access to utility functions for downloading and caching files.
shim_out.write_text(f"#!{sys.executable}\n\n" + shim_in.read_text())
shim_out.chmod(0o755)
env["FC"] = str(shim_out)
def setup_xbuild_files(state: BuildState) -> None:
_, pip = find_pip(state.options)
xbf_dir = state.build_path / "xbuild_files"
xbf_dir.mkdir()
for requirement in call(*pip, "freeze", env=state.build_env, capture_stdout=True).splitlines():
name, _, _ = requirement.strip().partition("==")
xbuild_files = state.options.xbuild_files.get(canonicalize_name(name), [])
if xbuild_files:
log.step(f"Installing xbuild-files for {name}...")
pip_install_android(state, xbf_dir, "--no-deps", requirement)
for xbf in xbuild_files:
if (xbf_dir / xbf).exists():
shutil.copy(
xbf_dir / xbf,
find_site_packages(state.build_env) / xbf,
)
else:
log.warning(f"{xbf_dir / xbf} does not exist")
def pip_install_android(state: BuildState, target: Path, *args: PathOrStr) -> None:
use_uv, pip = find_pip(state.options)
call(
*pip,
"install",
"--only-binary=:all:",
*(["--python-platform", android_triplet(state.config.identifier)] if use_uv else []),
"--target",
target,
*args,
env=state.android_env,
)
def find_site_packages(env: dict[str, str]) -> Path:
return glob1(Path(env["VIRTUAL_ENV"]), "lib/python*/site-packages")
def glob1(base: Path, pattern: str) -> Path:
results = list(base.glob(pattern))
if len(results) != 1:
msg = f"{base} contains {len(results)} paths matching '{pattern}'; expected 1"
raise errors.FatalError(msg)
return results[0]
def find_pip(build_options: BuildOptions) -> tuple[bool, list[str]]:
use_uv = build_options.build_frontend.name in {"build[uv]", "uv"}
uv_path = find_uv()
if use_uv and uv_path is None:
msg = "uv not found"
raise AssertionError(msg)
pip = ["pip"] if not use_uv else [str(uv_path), "pip"]
return use_uv, pip
def before_build(state: BuildState) -> None:
if state.options.before_build:
log.step("Running before_build...")
shell_prepared(
state.options.before_build,
build_options=state.options,
env=state.android_env,
)
def build_wheel(state: BuildState) -> Path:
log.step("Building wheel...")
built_wheel_dir = state.build_path / "built_wheel"
match state.options.build_frontend.name:
case "build" | "build[uv]":
call(
"python",
"-m",
"build",
state.options.package_dir,
"--wheel",
"--no-isolation",
"--skip-dependency-check",
f"--outdir={built_wheel_dir}",
*get_build_frontend_extra_flags(
state.options.build_frontend,
state.options.build_verbosity,
prepare_config_settings(
state.options.config_settings,
project=".",
package=state.options.package_dir,
),
),
env=state.android_env,
)
case "uv":
uv_path = find_uv()
assert uv_path is not None
call(
uv_path,
"build",
state.options.package_dir,
"--wheel",
"--no-build-isolation",
f"--out-dir={built_wheel_dir}",
*get_build_frontend_extra_flags(
state.options.build_frontend,
state.options.build_verbosity,
prepare_config_settings(
state.options.config_settings,
project=".",
package=state.options.package_dir,
),
),
env=state.android_env,
)
case x:
msg = f"Android requires the build frontend to be 'build' or 'uv', not {x!r}"
raise errors.FatalError(msg)
built_wheel = glob1(built_wheel_dir, "*.whl")
if built_wheel.name.endswith("none-any.whl"):
raise errors.NonPlatformWheelError()
return built_wheel
def repair_wheel(state: BuildState, built_wheel: Path) -> Path:
log.step("Repairing wheel...")
repaired_wheel_dir = state.build_path / "repaired_wheel"
repaired_wheel_dir.mkdir()
if state.options.repair_command:
# Tell auditwheel the locations of compiler libraries.
toolchain = Path(state.android_env["CC"]).parent.parent
triplet = android_triplet(state.config.identifier)
ldpaths = ":".join(
str(glob1(toolchain, pattern))
for pattern in [
f"lib/clang/*/lib/linux/{triplet.split('-')[0]}", # libomp
f"sysroot/usr/lib/{triplet}", # libc++_shared
]
)
shell(
prepare_command(
state.options.repair_command,
ldpaths=ldpaths,
wheel=built_wheel,
dest_dir=repaired_wheel_dir,
package=state.options.package_dir,
project=".",
),
env=state.build_env,
)
else:
shutil.move(built_wheel, repaired_wheel_dir)
repaired_wheels = list(repaired_wheel_dir.glob("*.whl"))
if len(repaired_wheels) == 0:
raise errors.RepairStepProducedNoWheelError()
if len(repaired_wheels) != 1:
raise errors.RepairStepProducedMultipleWheelsError(
[rw.name for rw in repaired_wheels],
)
repaired_wheel = repaired_wheels[0]
if repaired_wheel.name.endswith("none-any.whl"):
raise errors.NonPlatformWheelError()
return repaired_wheel
def test_wheel(state: BuildState, wheel: Path) -> None:
test_command = state.options.test_command
if not (test_command and state.options.test_selector(state.config.identifier)):
return
log.step("Testing wheel...")
native_arch = arch_synonym(platform.machine(), platforms.native_platform(), "android")
if state.config.arch != native_arch:
log.warning(
f"Skipping tests for {state.config.arch}, as the build machine only "
f"supports {native_arch}"
)
return
if state.options.before_test:
shell_prepared(
state.options.before_test,
build_options=state.options,
env=state.android_env,
)
# Install the wheel and test-requires.
site_packages_dir = state.build_path / "site-packages"
site_packages_dir.mkdir()
pip_install_android(
state,
site_packages_dir,
f"{wheel}{state.options.test_extras}",
*state.options.test_requires,
)
# Copy test-sources.
cwd_dir = state.build_path / "cwd"
cwd_dir.mkdir()
if state.options.test_sources:
copy_test_sources(state.options.test_sources, Path.cwd(), cwd_dir)
else:
(cwd_dir / "test_fail.py").write_text(
resources.TEST_FAIL_CWD_FILE.read_text(),
)
# Android doesn't support placeholders in the test command.
if any(("{" + placeholder + "}") in test_command for placeholder in ("project", "package")):
msg = (
f"Test command {test_command!r} with a "
"'{project}' or '{package}' placeholder is not supported on Android, "
"because the source directory is not visible on the emulator."
)
raise errors.FatalError(msg)
# Parse test-command.
test_args = shlex.split(test_command)
if test_args[0] in {"python", "python3"} and any(arg in test_args for arg in ("-c", "-m")):
# Forward the args to the CPython testbed script. We require '-c' or '-m'
# to be in the command, because without those flags, the testbed script
# will prepend '-m test', which will run Python's own test suite.
del test_args[0]
elif test_args[0] == "pytest":
# We transform some commands into the `python -m` form, but this is deprecated.
msg = (
f"Test command {test_command!r} is not supported on Android. "
"cibuildwheel will try to execute it as if it started with 'python -m'. "
"If this works, all you need to do is add that to your test command."
)
log.warning(msg)
test_args.insert(0, "-m")
else:
msg = (
f"Test command {test_command!r} is not supported on Android. "
f"Command must begin with 'python' or 'python3', and contain '-m' or '-c'."
)
raise errors.FatalError(msg)
# By default, run on a testbed managed emulator running the newest supported
# Android version. However, if the user specifies a --managed or --connected
# test execution argument, that argument takes precedence.
test_runtime_args = state.options.test_runtime.args
if any(arg.startswith(("--managed", "--connected")) for arg in test_runtime_args):
emulator_args = []
else:
emulator_args = ["--managed", "maxVersion"]
# Run the test app.
call(
state.python_dir / "android.py",
"test",
"--site-packages",
site_packages_dir,
"--cwd",
cwd_dir,
*emulator_args,
*(["-v"] if state.options.build_verbosity > 0 else []),
*test_runtime_args,
"--",
*test_args,
env=state.build_env,
)
-810
View File
@@ -1,810 +0,0 @@
from __future__ import annotations
__lazy_modules__ = {
"cibuildwheel.audit",
"cibuildwheel.frontend",
"cibuildwheel.logger",
"cibuildwheel.platforms.macos",
"cibuildwheel.util",
"cibuildwheel.util.cmd",
"cibuildwheel.util.file",
"cibuildwheel.util.helpers",
"cibuildwheel.util.packaging",
"cibuildwheel.venv",
"filelock",
"packaging",
"packaging.version",
"pathlib",
"platform",
"shlex",
"shutil",
"subprocess",
"textwrap",
"typing",
}
import dataclasses
import os
import platform
import shlex
import shutil
import subprocess
import sys
import textwrap
from pathlib import Path
from typing import assert_never
from filelock import FileLock
from packaging.version import Version
from cibuildwheel import errors
from cibuildwheel.audit import run_audit
from cibuildwheel.frontend import (
BuildFrontendName,
get_build_frontend_extra_flags,
prepare_config_settings,
)
from cibuildwheel.logger import log
from cibuildwheel.platforms.macos import install_cpython as install_build_cpython
from cibuildwheel.util import resources
from cibuildwheel.util.cmd import call, shell, split_command
from cibuildwheel.util.file import (
CIBW_CACHE_PATH,
copy_test_sources,
download,
move_file,
remove_on_error,
)
from cibuildwheel.util.helpers import prepare_command, unwrap_preserving_paragraphs
from cibuildwheel.util.packaging import find_compatible_wheel
from cibuildwheel.venv import constraint_flags, virtualenv
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Sequence, Set
from cibuildwheel.architecture import Architecture
from cibuildwheel.environment import ParsedEnvironment
from cibuildwheel.options import Options
from cibuildwheel.selector import BuildSelector
@dataclasses.dataclass(frozen=True, kw_only=True)
class PythonConfiguration:
version: str
identifier: str
url: str
build_url: str
build_sha256: str
sha256: str
@property
def sdk(self) -> str:
return self.multiarch.rsplit("-", 1)[1]
@property
def arch(self) -> str:
return self.multiarch.rsplit("-", 1)[0]
@property
def multiarch(self) -> str:
# The multiarch identifier, as reported by `sys.implementation._multiarch`
return "-".join(self.identifier.split("-ios_")[1].rsplit("_", 1))
@property
def is_simulator(self) -> bool:
return self.identifier.endswith("_iphonesimulator")
@property
def xcframework_slice(self) -> str:
"XCframeworks include binaries for multiple ABIs; which ABI section should be used?"
return "ios-arm64_x86_64-simulator" if self.is_simulator else "ios-arm64"
def all_python_configurations() -> list[PythonConfiguration]:
# iOS builds are always cross builds; we need to install a macOS Python as
# well. Rather than duplicate the location of the URL of macOS installers,
# load the macos configurations, determine the macOS configuration that
# matches the platform we're building, and embed that URL in the parsed iOS
# configuration.
macos_python_configs = resources.read_python_configs("macos")
def build_python_config(config_dict: dict[str, str]) -> dict[str, str]:
# The iOS identifier will be something like cp313-ios_arm64_iphoneos.
# Drop the iphoneos suffix, then replace ios with macosx to yield
# cp313-macosx_arm64, which will be a macOS build identifier.
modified_ios_identifier = config_dict["identifier"].rsplit("_", 1)[0]
macos_identifier = modified_ios_identifier.replace("ios", "macosx")
matching = [
config for config in macos_python_configs if config["identifier"] == macos_identifier
]
if not matching:
msg = (
f"Internal error: no macOS configuration found matching "
f"{macos_identifier!r}, needed to build the iOS configuration "
f"{config_dict['identifier']!r}. The bundled build-platforms.toml "
f"resources are inconsistent."
)
raise errors.FatalError(msg)
return matching[0]
# Load the platform configuration
full_python_configs = resources.read_python_configs("ios")
# Build the configurations, annotating with macOS URL details.
python_configurations = []
for item in full_python_configs:
build_config = build_python_config(item)
python_configurations.append(
PythonConfiguration(
**item,
build_url=build_config["url"],
build_sha256=build_config["sha256"],
)
)
return python_configurations
def get_python_configurations(
build_selector: BuildSelector,
architectures: Set[Architecture],
) -> list[PythonConfiguration]:
python_configurations = all_python_configurations()
# Filter out configs that don't match any of the selected architectures
python_configurations = [
c
for c in python_configurations
if any(c.identifier.endswith(f"-ios_{a.value}") for a in architectures)
]
# Skip builds as required by BUILD/SKIP
python_configurations = [c for c in python_configurations if build_selector(c.identifier)]
return python_configurations
def install_target_cpython(tmp: Path, config: PythonConfiguration, free_threading: bool) -> Path:
if free_threading:
msg = "Free threading builds aren't available for iOS (yet)"
raise errors.FatalError(msg)
# Install an iOS build of CPython
ios_python_tar_gz = config.url.rsplit("/", 1)[-1]
extension = ".tar.gz"
assert ios_python_tar_gz.endswith(extension)
installation_path = CIBW_CACHE_PATH / ios_python_tar_gz[: -len(extension)]
with FileLock(str(installation_path) + ".lock"):
if not installation_path.exists():
downloaded_tar_gz = tmp / ios_python_tar_gz
download(config.url, downloaded_tar_gz, sha256=config.sha256)
with remove_on_error(installation_path):
installation_path.mkdir(parents=True)
call("tar", "-C", installation_path, "-xf", downloaded_tar_gz)
downloaded_tar_gz.unlink()
return installation_path
def cross_virtualenv(
*,
py_version: str,
target_python: Path,
multiarch: str,
build_python: Path,
venv_path: Path,
dependency_constraint: Path | None,
xbuild_tools: Sequence[str] | None,
) -> dict[str, str]:
"""Create a cross-compilation virtual environment.
In a cross-compilation environment, the *target* is the platform where the
code will ultimately run, and the *build* is the platform where you're
running the compilation. When building iOS wheels, iOS is the target machine
and macOS is the build machine. The terminology around these machines varies
between build tools (configure uses "host" and "build"; cmake uses "target" and
"build host").
A cross-compilation virtualenv is an environment that is based on the
*build* python (so that binaries can execute); but it modifies the
environment at startup so that any request for platform details (such as
`sys.platform` or `sysconfig.get_platform()`) return details of the target
platform. It also applies a loader patch so that any virtualenv created by
the cross-compilation environment will also be a cross-compilation
environment.
:param py_version: The Python version (major.minor) in use
:param target_python: The path to the python binary for the target platform
:param multiarch: The multiarch tag for the target platform (i.e., the value
of `sys.implementation._multiarch`)
:param build_python: The path to the python binary for the build platform
:param venv_path: The path where the cross virtual environment should be
created.
:param dependency_constraint: A path to a constraint file that should be
used when constraining dependencies in the environment.
:param xbuild_tools: A list of executable names (without paths) that are
on the path, but must be preserved in the cross environment.
"""
# Create an initial macOS virtual environment
env = virtualenv(
py_version,
build_python,
venv_path,
dependency_constraint,
use_uv=False,
)
# Convert the macOS virtual environment into an iOS virtual environment
# using make_cross_venv.py.
# target_python is the path to the Python binary;
# determine the root of the XCframework slice that is being used.
slice_path = target_python.parent.parent
if Version(py_version) >= Version("3.15"):
# python.org 3.15+ distributions: sysconfig data lives in the
# stdlib directory (lib-<arch>/python<version>); cross-venv scripts
# come from cibuildwheel resources.
arch = multiarch.split("-", maxsplit=1)[0]
stdlib_path = slice_path / f"lib-{arch}" / f"python{py_version}"
call(
"python",
str(resources.IOS_SUPPORT_FILES / "make_cross_venv.py"),
str(venv_path),
str(stdlib_path),
str(resources.IOS_SUPPORT_FILES),
env=env,
cwd=venv_path,
)
else:
# BeeWare distributions: platform-config already contains both
# sysconfig data and the make_cross_venv.py script.
platform_config_path = slice_path / f"platform-config/{multiarch}"
call(
"python",
str(platform_config_path / "make_cross_venv.py"),
str(venv_path),
env=env,
cwd=venv_path,
)
# When running on macOS, it's easy for the build environment to leak into
# the target environment, especially when building for ARM64 (because the
# build architecture is the same as the target architecture). The primary
# culprit for this is Homebrew libraries leaking in as dependencies for iOS
# libraries.
#
# To prevent problems, set the PATH to isolate the build environment from
# sources that could introduce incompatible binaries.
#
# However, there may be some tools on the path that are needed for the
# build. Find their location on the path, and link the underlying binaries
# (fully resolving symlinks) to a "safe" location that will *only* contain
# those tools. This avoids needing to add *all* of Homebrew to the path just
# to get access to (for example) cmake for build purposes. A value of None
# means the user hasn't provided a list of xbuild tools.
xbuild_tools_path = venv_path / "cibw_xbuild_tools"
xbuild_tools_path.mkdir()
if xbuild_tools is None:
log.warning(
textwrap.dedent(
"""
Your project configuration does not define any cross-build tools.
iOS builds use an isolated build environment; if your build process requires any
third-party tools (such as cmake, ninja, or rustc), you must explicitly declare
that those tools are required using xbuild-tools/CIBW_XBUILD_TOOLS. This will
likely manifest as a "somebuildtool: command not found" error.
If the build succeeds, you can silence this warning by setting adding
`xbuild-tools = []` to your pyproject.toml configuration, or exporting
CIBW_XBUILD_TOOLS as an empty string into your environment.
"""
)
)
else:
for tool in xbuild_tools:
tool_path = shutil.which(tool)
if tool_path is None:
msg = f"Could not find a {tool!r} executable on the path."
raise errors.FatalError(msg)
# Link the binary into the safe tools directory
original = Path(tool_path).resolve()
print(f"{tool!r} will be included in the cross-build environment (using {original})")
(xbuild_tools_path / tool).symlink_to(original)
env["PATH"] = os.pathsep.join(
[
# The target python's binary directory
str(target_python.parent),
# The cross-platform environment's binary directory
str(venv_path / "bin"),
# The directory of cross-build tools
str(xbuild_tools_path),
# The bare minimum Apple system paths.
"/usr/bin",
"/bin",
"/usr/sbin",
"/sbin",
"/Library/Apple/usr/bin",
]
)
# Also unset DYLD_LIBRARY_PATH to ensure that no macOS libraries will be
# found and linked.
env.pop("DYLD_LIBRARY_PATH", None)
return env
def setup_python(
tmp: Path,
*,
python_configuration: PythonConfiguration,
dependency_constraint: Path | None,
environment: ParsedEnvironment,
build_frontend: BuildFrontendName,
xbuild_tools: Sequence[str] | None,
) -> tuple[Path, dict[str, str]]:
# Not using set because mypy can't narrow it
if build_frontend == "build[uv]" or build_frontend == "uv": # noqa: PLR1714
msg = "uv doesn't support iOS"
raise errors.FatalError(msg)
# An iOS environment requires 2 python installs - one for the build machine
# (macOS), and one for the target (iOS). We'll only ever interact with the
# *target* python, but the build Python needs to exist to act as the base
# for a cross venv.
tmp.mkdir()
implementation_id = python_configuration.identifier.split("-")[0]
log.step(f"Installing Build Python {implementation_id}...")
if implementation_id.startswith("cp"):
free_threading = "t-ios" in python_configuration.identifier
build_python = install_build_cpython(
tmp,
python_configuration.version,
python_configuration.build_url,
free_threading,
python_configuration.build_sha256,
)
else:
msg = f"Unknown Python implementation: {implementation_id}"
raise errors.FatalError(msg)
assert build_python.exists(), (
f"{build_python.name} not found, has {list(build_python.parent.iterdir())}"
)
log.step(f"Installing Target Python {implementation_id}...")
target_install_path = install_target_cpython(tmp, python_configuration, free_threading)
target_python = (
target_install_path
/ "Python.xcframework"
/ python_configuration.xcframework_slice
/ "bin"
/ f"python{python_configuration.version}"
)
assert target_python.parent.exists(), (
f"{target_python.parent} not found, has {list(target_install_path.iterdir())}"
)
log.step("Creating cross build environment...")
venv_path = tmp / "venv"
env = cross_virtualenv(
py_version=python_configuration.version,
target_python=target_python,
multiarch=python_configuration.multiarch,
build_python=build_python,
venv_path=venv_path,
dependency_constraint=dependency_constraint,
xbuild_tools=xbuild_tools,
)
venv_bin_path = venv_path / "bin"
assert venv_bin_path.exists()
# We version pip ourselves, so we don't care about pip version checking
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
# upgrade pip to the version matching our constraints
# if necessary, reinstall it to ensure that it's available on PATH as 'pip'
pip = ["python", "-m", "pip"]
call(
*pip,
"install",
"--upgrade",
"pip",
*constraint_flags(dependency_constraint),
env=env,
cwd=venv_path,
)
# Apply our environment after pip is ready
env = environment.as_dictionary(prev_environment=env)
# Check what Python version we're on
which_python = call("which", "python", env=env, capture_stdout=True).strip()
print(which_python)
if which_python != str(venv_bin_path / "python"):
msg = (
"cibuildwheel: python available on PATH doesn't match our installed instance. "
"If you have modified PATH, ensure that you don't overwrite cibuildwheel's "
"entry or insert python above it."
)
raise errors.FatalError(msg)
call("python", "-V", "-V", env=env)
# Check what pip version we're on
assert (venv_bin_path / "pip").exists()
which_pip = call("which", "pip", env=env, capture_stdout=True).strip()
print(which_pip)
if which_pip != str(venv_bin_path / "pip"):
msg = (
"cibuildwheel: pip available on PATH doesn't match our installed instance. "
"If you have modified PATH, ensure that you don't overwrite cibuildwheel's "
"entry or insert pip above it."
)
raise errors.FatalError(msg)
call("pip", "--version", env=env)
# Ensure that IPHONEOS_DEPLOYMENT_TARGET is set in the environment
env.setdefault("IPHONEOS_DEPLOYMENT_TARGET", "13.0")
log.step("Installing build tools...")
match build_frontend:
case "pip":
# No additional build tools required
pass
case "build":
call(
"pip",
"install",
"--upgrade",
"build[virtualenv]",
*constraint_flags(dependency_constraint),
env=env,
)
case _:
assert_never(build_frontend)
return target_install_path, env
def build(options: Options, tmp_path: Path) -> None:
if sys.platform != "darwin":
msg = "iOS binaries can only be built on macOS"
raise errors.FatalError(msg)
python_configurations = get_python_configurations(
build_selector=options.globals.build_selector,
architectures=options.globals.architectures,
)
if not python_configurations:
return
try:
before_all_options_identifier = python_configurations[0].identifier
before_all_options = options.build_options(before_all_options_identifier)
if before_all_options.before_all:
log.step("Running before_all...")
env = before_all_options.environment.as_dictionary(prev_environment=os.environ)
env.setdefault("IPHONEOS_DEPLOYMENT_TARGET", "13.0")
before_all_prepared = prepare_command(
before_all_options.before_all,
project=".",
package=before_all_options.package_dir,
)
shell(before_all_prepared, env=env)
built_wheels: list[Path] = []
for config in python_configurations:
build_options = options.build_options(config.identifier)
build_frontend = build_options.build_frontend
# uv doesn't support iOS
# Not using set because mypy can't narrow it
if build_frontend.name == "build[uv]" or build_frontend.name == "uv": # noqa: PLR1714
msg = "uv doesn't support iOS"
raise errors.FatalError(msg)
log.build_start(config.identifier)
identifier_tmp_dir = tmp_path / config.identifier
identifier_tmp_dir.mkdir()
built_wheel_dir = identifier_tmp_dir / "built_wheel"
repaired_wheel_dir = identifier_tmp_dir / "repaired_wheel"
constraints_path = build_options.dependency_constraints.get_for_python_version(
version=config.version, tmp_dir=identifier_tmp_dir
)
target_install_path, env = setup_python(
identifier_tmp_dir / "build",
python_configuration=config,
dependency_constraint=constraints_path,
environment=build_options.environment,
build_frontend=build_frontend.name,
xbuild_tools=build_options.xbuild_tools,
)
env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
if compatible_wheel:
log.step_end()
print(
f"\nFound previously built wheel {compatible_wheel.name} "
f"that is compatible with {config.identifier}. "
"Skipping build step..."
)
test_wheel = compatible_wheel
else:
if build_options.before_build:
log.step("Running before_build...")
before_build_prepared = prepare_command(
build_options.before_build,
project=".",
package=build_options.package_dir,
)
shell(before_build_prepared, env=env)
log.step("Building wheel...")
built_wheel_dir.mkdir()
extra_flags = get_build_frontend_extra_flags(
build_frontend,
build_options.build_verbosity,
prepare_config_settings(
build_options.config_settings,
project=".",
package=build_options.package_dir,
),
)
match build_frontend.name:
case "pip":
# Path.resolve() is needed. Without it pip wheel may try to
# fetch package from pypi.org. See
# https://github.com/pypa/cibuildwheel/pull/369
call(
"python",
"-m",
"pip",
"wheel",
build_options.package_dir.resolve(),
f"--wheel-dir={built_wheel_dir}",
"--no-deps",
*extra_flags,
env=env,
)
case "build":
call(
"python",
"-m",
"build",
build_options.package_dir,
"--wheel",
f"--outdir={built_wheel_dir}",
*extra_flags,
env=env,
)
case _:
assert_never(build_frontend)
try:
built_wheel = next(built_wheel_dir.glob("*.whl"))
except StopIteration:
raise errors.BuildProducedNoWheelError() from None
if built_wheel.name.endswith("none-any.whl"):
raise errors.NonPlatformWheelError()
repaired_wheel_dir.mkdir()
if build_options.repair_command:
log.step("Repairing wheel...")
repair_command_prepared = prepare_command(
build_options.repair_command,
wheel=built_wheel,
dest_dir=repaired_wheel_dir,
package=build_options.package_dir,
project=".",
)
shell(repair_command_prepared, env=env)
else:
shutil.move(str(built_wheel), repaired_wheel_dir)
try:
repaired_wheel = next(repaired_wheel_dir.glob("*.whl"))
except StopIteration:
raise errors.RepairStepProducedNoWheelError() from None
if repaired_wheel.name in {wheel.name for wheel in built_wheels}:
raise errors.AlreadyBuiltWheelError(repaired_wheel.name)
log.step_end()
run_audit(tmp_dir=tmp_path, build_options=build_options, wheel=repaired_wheel)
test_wheel = repaired_wheel
if build_options.test_command and build_options.test_selector(config.identifier):
if not config.is_simulator:
log.step("Skipping tests on non-simulator SDK")
elif config.arch != os.uname().machine:
log.step("Skipping tests on non-native simulator architecture")
else:
test_env = build_options.test_environment.as_dictionary(prev_environment=env)
if build_options.before_test:
before_test_prepared = prepare_command(
build_options.before_test,
project=".",
package=build_options.package_dir,
)
shell(before_test_prepared, env=test_env)
log.step("Setting up test harness...")
# Clone the testbed project into the build directory
testbed_path = identifier_tmp_dir / "testbed"
call(
"python",
target_install_path / "testbed",
"clone",
testbed_path,
env=test_env,
)
testbed_app_path = testbed_path / "iOSTestbed" / "app"
# Copy the test sources to the testbed app
if build_options.test_sources:
copy_test_sources(
build_options.test_sources,
Path.cwd(),
testbed_app_path,
)
else:
(testbed_app_path / "test_fail.py").write_text(
resources.TEST_FAIL_CWD_FILE.read_text()
)
log.step("Installing test requirements...")
# Install the compiled wheel (with any test extras), plus
# the test requirements. Use the --platform tag to force
# the installation of iOS wheels; this requires the use of
# --only-binary=:all:
ios_version = test_env["IPHONEOS_DEPLOYMENT_TARGET"]
platform_tag = f"ios_{ios_version.replace('.', '_')}_{config.arch}_{config.sdk}"
call(
"python",
"-m",
"pip",
"install",
"--only-binary=:all:",
"--platform",
platform_tag,
"--target",
testbed_path / "iOSTestbed" / "app_packages",
f"{test_wheel}{build_options.test_extras}",
*build_options.test_requires,
env=test_env,
)
log.step("Running test suite...")
# iOS doesn't support placeholders in the test command,
# because the source dir isn't visible on the simulator.
if (
"{project}" in build_options.test_command
or "{package}" in build_options.test_command
):
msg = unwrap_preserving_paragraphs(
f"""
iOS tests configured with a test command that uses the "{{project}}" or
"{{package}}" placeholder. iOS tests cannot use placeholders, because the
source directory is not visible on the simulator.
In addition, iOS tests must run as a Python module, so the test command
must begin with 'python -m'.
Test command: {build_options.test_command!r}
"""
)
raise errors.FatalError(msg)
test_command_list = shlex.split(build_options.test_command)
try:
for test_command_parts in split_command(test_command_list):
match test_command_parts:
case ["python", "-m", *rest]:
final_command = rest
case ["pytest", *rest]:
# pytest works exactly the same as a module, so we
# can just run it as a module.
msg = unwrap_preserving_paragraphs(f"""
iOS tests configured with a test command which doesn't start
with 'python -m'. iOS tests must execute python modules - other
entrypoints are not supported.
cibuildwheel will try to execute it as if it started with
'python -m'. If this works, all you need to do is add that to
your test command.
Test command: {build_options.test_command!r}
""")
log.warning(msg)
final_command = ["pytest", *rest]
case _:
msg = unwrap_preserving_paragraphs(
f"""
iOS tests configured with a test command which doesn't start
with 'python -m'. iOS tests must execute python modules - other
entrypoints are not supported.
Test command: {build_options.test_command!r}
"""
)
raise errors.FatalError(msg)
test_runtime_args = build_options.test_runtime.args
# 2025-10: The GitHub Actions macos-15 runner has a known issue where
# the default simulator won't start due to a disk performance issue;
# see https://github.com/actions/runner-images/issues/12777 for details.
# In the meantime, if it looks like we're running on a GitHub Actions
# macos-15 runner, use a simulator that is known to work, unless the
# user explicitly specifies a simulator.
os_version, _, arch = platform.mac_ver()
if (
"GITHUB_ACTIONS" in os.environ
and os_version.startswith("15.")
and arch == "arm64"
and not any(
arg.startswith("--simulator") for arg in test_runtime_args
)
):
test_runtime_args = [
"--simulator",
"iPhone 16e,OS=18.5",
*test_runtime_args,
]
call(
"python",
testbed_path,
"run",
*(["--verbose"] if build_options.build_verbosity > 0 else []),
*test_runtime_args,
"--",
*final_command,
env=test_env,
)
except subprocess.CalledProcessError:
# catches the first test command failure in the loop,
# implementing short-circuiting
log.step_end(success=False)
log.error(f"Test suite failed on {config.identifier}")
sys.exit(1)
log.step_end()
# We're all done here; move it to output (overwrite existing)
output_wheel: Path | None = None
if compatible_wheel is None:
output_wheel = build_options.output_dir.joinpath(repaired_wheel.name)
moved_wheel = move_file(repaired_wheel, output_wheel)
if moved_wheel != output_wheel.resolve():
log.warning(
f"{repaired_wheel} was moved to {moved_wheel} instead of {output_wheel}"
)
built_wheels.append(output_wheel)
# Clean up
shutil.rmtree(identifier_tmp_dir)
log.build_end(output_wheel)
except subprocess.CalledProcessError as error:
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
raise errors.FatalError(msg) from error
-613
View File
@@ -1,613 +0,0 @@
from __future__ import annotations
__lazy_modules__ = {
"cibuildwheel.audit",
"cibuildwheel.frontend",
"cibuildwheel.logger",
"cibuildwheel.util",
"cibuildwheel.util.file",
"cibuildwheel.util.helpers",
"cibuildwheel.util.packaging",
"collections",
"contextlib",
"pathlib",
"shutil",
"subprocess",
"textwrap",
"typing",
}
import contextlib
import dataclasses
import shutil
import subprocess
import sys
import textwrap
from collections import OrderedDict
from pathlib import Path, PurePath, PurePosixPath
from typing import assert_never
from cibuildwheel import errors
from cibuildwheel.architecture import Architecture
from cibuildwheel.audit import needs_audit, run_audit
from cibuildwheel.frontend import get_build_frontend_extra_flags, prepare_config_settings
from cibuildwheel.logger import log
from cibuildwheel.oci_container import OCIContainer, OCIContainerEngineConfig, OCIPlatform
from cibuildwheel.util import resources
from cibuildwheel.util.file import copy_test_sources
from cibuildwheel.util.helpers import prepare_command, unwrap
from cibuildwheel.util.packaging import find_compatible_wheel
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Iterable, Iterator, Sequence, Set
from cibuildwheel.options import BuildOptions, Options
from cibuildwheel.selector import BuildSelector
from cibuildwheel.typing import PathOrStr
ARCHITECTURE_OCI_PLATFORM_MAP = {
Architecture.x86_64: OCIPlatform.AMD64,
Architecture.i686: OCIPlatform.i386,
Architecture.aarch64: OCIPlatform.ARM64,
Architecture.ppc64le: OCIPlatform.PPC64LE,
Architecture.s390x: OCIPlatform.S390X,
Architecture.armv7l: OCIPlatform.ARMV7,
Architecture.riscv64: OCIPlatform.RISCV64,
}
@dataclasses.dataclass(frozen=True, kw_only=True)
class PythonConfiguration:
version: str
identifier: str
path_str: str
@property
def path(self) -> PurePosixPath:
return PurePosixPath(self.path_str)
@dataclasses.dataclass(frozen=True, kw_only=True)
class BuildStep:
platform_configs: list[PythonConfiguration]
platform_tag: str
container_engine: OCIContainerEngineConfig
container_image: str
def all_python_configurations() -> list[PythonConfiguration]:
config_dicts = resources.read_python_configs("linux")
return [PythonConfiguration(**item) for item in config_dicts]
def get_python_configurations(
build_selector: BuildSelector,
architectures: Set[Architecture],
) -> list[PythonConfiguration]:
python_configurations = all_python_configurations()
# return all configurations whose arch is in our `architectures` set,
# and match the build/skip rules
return [
c
for c in python_configurations
if any(c.identifier.endswith(arch.value) for arch in architectures)
and build_selector(c.identifier)
]
def container_image_for_python_configuration(
config: PythonConfiguration, build_options: BuildOptions
) -> str:
# e.g
# identifier is 'cp310-manylinux_x86_64'
# platform_tag is 'manylinux_x86_64'
# platform_arch is 'x86_64'
_, platform_tag = config.identifier.split("-", 1)
_, platform_arch = platform_tag.split("_", 1)
assert build_options.manylinux_images is not None
assert build_options.musllinux_images is not None
return (
build_options.manylinux_images[platform_arch]
if platform_tag.startswith("manylinux")
else build_options.musllinux_images[platform_arch]
)
def get_build_steps(
options: Options, python_configurations: list[PythonConfiguration]
) -> Iterator[BuildStep]:
"""
Groups PythonConfigurations into BuildSteps. Each BuildStep represents a
separate container instance.
"""
steps = OrderedDict[tuple[str, str, str, OCIContainerEngineConfig], BuildStep]()
for config in python_configurations:
_, platform_tag = config.identifier.split("-", 1)
build_options = options.build_options(config.identifier)
before_all = build_options.before_all
container_image = container_image_for_python_configuration(config, build_options)
container_engine = build_options.container_engine
step_key = (platform_tag, container_image, before_all, container_engine)
if step_key in steps:
steps[step_key].platform_configs.append(config)
else:
steps[step_key] = BuildStep(
platform_configs=[config],
platform_tag=platform_tag,
container_engine=container_engine,
container_image=container_image,
)
yield from steps.values()
def check_all_python_exist(
*, platform_configs: Iterable[PythonConfiguration], container: OCIContainer
) -> None:
exist = True
has_manylinux_interpreters = False
messages = []
with contextlib.suppress(subprocess.CalledProcessError):
# use capture_output to keep quiet
container.call(["manylinux-interpreters", "--help"], capture_output=True)
has_manylinux_interpreters = True
for config in platform_configs:
python_path = config.path / "bin" / "python"
if has_manylinux_interpreters:
try:
container.call(["manylinux-interpreters", "ensure", config.path.name])
except subprocess.CalledProcessError:
messages.append(
f" 'manylinux-interpreters ensure {config.path.name}' needed to build '{config.identifier}' failed in container running image '{container.image}'."
" Either the installation failed or this interpreter is not available in that image. Please check the logs."
)
exist = False
else:
try:
container.call(["test", "-x", python_path])
except subprocess.CalledProcessError:
messages.append(
f" '{python_path}' executable doesn't exist in image '{container.image}' to build '{config.identifier}'."
)
exist = False
if not exist:
message = "\n".join(messages)
raise errors.FatalError(message)
def build_in_container(
*,
options: Options,
platform_configs: Sequence[PythonConfiguration],
container: OCIContainer,
container_project_path: PurePath,
container_package_dir: PurePath,
local_tmp_dir: Path,
) -> None:
container_output_dir = PurePosixPath("/output")
check_all_python_exist(platform_configs=platform_configs, container=container)
log.step("Copying project into container...")
container.copy_into(Path.cwd(), container_project_path)
before_all_options_identifier = platform_configs[0].identifier
before_all_options = options.build_options(before_all_options_identifier)
if before_all_options.before_all:
log.step("Running before_all...")
env = container.get_environment()
env["PATH"] = f"/opt/python/cp39-cp39/bin:{env['PATH']}"
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
env["PIP_ROOT_USER_ACTION"] = "ignore"
env = before_all_options.environment.as_dictionary(
env, executor=container.environment_executor
)
before_all_prepared = prepare_command(
before_all_options.before_all,
project=container_project_path,
package=container_package_dir,
)
container.call(["sh", "-c", before_all_prepared], env=env)
built_wheels: list[PurePosixPath] = []
for config in platform_configs:
log.build_start(config.identifier)
local_identifier_tmp_dir = local_tmp_dir / config.identifier
build_options = options.build_options(config.identifier)
build_frontend = build_options.build_frontend
use_uv = build_frontend.name in {"build[uv]", "uv"}
pip = ["uv", "pip"] if use_uv else ["pip"]
log.step("Setting up build environment...")
dependency_constraint_flags: list[PathOrStr] = []
local_constraints_file = build_options.dependency_constraints.get_for_python_version(
version=config.version,
tmp_dir=local_identifier_tmp_dir,
)
if local_constraints_file:
container_constraints_file = PurePosixPath("/constraints.txt")
container.copy_into(local_constraints_file, container_constraints_file)
dependency_constraint_flags = ["-c", container_constraints_file]
env = container.get_environment()
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
env["PIP_ROOT_USER_ACTION"] = "ignore"
# put this config's python top of the list
python_bin = config.path / "bin"
env["PATH"] = f"{python_bin}:{env['PATH']}"
env = build_options.environment.as_dictionary(env, executor=container.environment_executor)
env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
# check config python is still on PATH
which_python = container.call(["which", "python"], env=env, capture_output=True).strip()
if PurePosixPath(which_python) != python_bin / "python":
msg = "python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it."
raise errors.FatalError(msg)
container.call(["python", "-V", "-V"], env=env)
if use_uv:
which_uv = container.call(["which", "uv"], env=env, capture_output=True).strip()
if not which_uv:
msg = "uv not found on PATH. You must use a supported manylinux or musllinux environment with uv."
raise errors.FatalError(msg)
else:
which_pip = container.call(["which", "pip"], env=env, capture_output=True).strip()
if PurePosixPath(which_pip) != python_bin / "pip":
msg = "pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it."
raise errors.FatalError(msg)
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
if compatible_wheel:
log.step_end()
print(
f"\nFound previously built wheel {compatible_wheel.name}, that's compatible with {config.identifier}. Skipping build step..."
)
repaired_wheel = compatible_wheel
else:
if build_options.before_build:
log.step("Running before_build...")
before_build_prepared = prepare_command(
build_options.before_build,
project=container_project_path,
package=container_package_dir,
)
before_build_env = env.copy()
if use_uv:
# On Linux, no virtualenv is created for the build environment
# (unlike macOS/Windows, where one is set up before before_build
# runs). uv requires either an active venv or an explicit Python
# target to install packages. Pin UV_PYTHON to the exact interpreter
# for this build so that `uv pip install` works in before_build
# without requiring users to pass --system.
before_build_env["UV_PYTHON"] = str(python_bin / "python")
container.call(["sh", "-c", before_build_prepared], env=before_build_env)
log.step("Building wheel...")
temp_dir = PurePosixPath("/tmp/cibuildwheel")
built_wheel_dir = temp_dir / "built_wheel"
container.call(["rm", "-rf", built_wheel_dir])
container.call(["mkdir", "-p", built_wheel_dir])
extra_flags = get_build_frontend_extra_flags(
build_frontend,
build_options.build_verbosity,
prepare_config_settings(
build_options.config_settings,
project=container_project_path,
package=container_package_dir,
),
)
match build_frontend.name:
case "pip":
container.call(
[
"python",
"-m",
"pip",
"wheel",
container_package_dir,
f"--wheel-dir={built_wheel_dir}",
"--no-deps",
*extra_flags,
],
env=env,
)
case "build" | "build[uv]":
if use_uv and "--no-isolation" not in extra_flags and "-n" not in extra_flags:
extra_flags += ["--installer=uv"]
container.call(
[
"python",
"-m",
"build",
container_package_dir,
"--wheel",
f"--outdir={built_wheel_dir}",
*extra_flags,
],
env=env,
)
case "uv":
container.call(
[
"uv",
"build",
f"--python={python_bin / 'python'}",
container_package_dir,
"--wheel",
f"--out-dir={built_wheel_dir}",
*extra_flags,
],
env=env,
)
case _:
assert_never(build_frontend)
try:
built_wheel = container.glob(built_wheel_dir, "*.whl")[0]
except IndexError:
raise errors.BuildProducedNoWheelError() from None
repaired_wheel_dir = temp_dir / "repaired_wheel"
container.call(["rm", "-rf", repaired_wheel_dir])
container.call(["mkdir", "-p", repaired_wheel_dir])
if built_wheel.name.endswith("none-any.whl"):
raise errors.NonPlatformWheelError()
if build_options.repair_command:
log.step("Repairing wheel...")
repair_command_prepared = prepare_command(
build_options.repair_command,
wheel=built_wheel,
dest_dir=repaired_wheel_dir,
package=container_package_dir,
project=container_project_path,
)
container.call(["sh", "-c", repair_command_prepared], env=env)
else:
container.call(["mv", built_wheel, repaired_wheel_dir])
match container.glob(repaired_wheel_dir, "*.whl"):
case []:
raise errors.RepairStepProducedNoWheelError()
case [repaired_wheel]:
pass
case too_many:
raise errors.RepairStepProducedMultipleWheelsError([p.name for p in too_many])
if repaired_wheel.name in {wheel.name for wheel in built_wheels}:
raise errors.AlreadyBuiltWheelError(repaired_wheel.name)
log.step_end()
if needs_audit(build_options.audit_command, repaired_wheel.name):
local_abi3audit_dir = local_identifier_tmp_dir / "audit"
local_abi3audit_dir.mkdir(parents=True, exist_ok=True)
try:
container.copy_out(repaired_wheel_dir, local_abi3audit_dir)
local_wheel = local_abi3audit_dir / repaired_wheel.name
run_audit(tmp_dir=local_tmp_dir, build_options=build_options, wheel=local_wheel)
finally:
shutil.rmtree(local_abi3audit_dir, ignore_errors=True)
if build_options.test_command and build_options.test_selector(config.identifier):
log.step("Testing wheel...")
# set up a virtual environment to install and test from, to make sure
# there are no dependencies that were pulled in at build time.
if not use_uv:
container.call(
["pip", "install", "virtualenv", *dependency_constraint_flags], env=env
)
testing_temp_dir = PurePosixPath(
container.call(["mktemp", "-d"], capture_output=True).strip()
)
venv_dir = testing_temp_dir / "venv"
if use_uv:
container.call(["uv", "venv", venv_dir, "--python", python_bin / "python"], env=env)
else:
# Use embedded dependencies from virtualenv to ensure determinism
venv_args = ["--no-periodic-update", "--pip=embed", "--no-setuptools"]
if "38" in config.identifier:
venv_args.append("--no-wheel")
container.call(["python", "-m", "virtualenv", *venv_args, venv_dir], env=env)
virtualenv_env = env.copy()
virtualenv_env["PATH"] = f"{venv_dir / 'bin'}:{virtualenv_env['PATH']}"
virtualenv_env["VIRTUAL_ENV"] = str(venv_dir)
virtualenv_env = build_options.test_environment.as_dictionary(
prev_environment=virtualenv_env
)
if build_options.before_test:
before_test_prepared = prepare_command(
build_options.before_test,
project=container_project_path,
package=container_package_dir,
)
container.call(["sh", "-c", before_test_prepared], env=virtualenv_env)
# Install the wheel we just built
container.call(
[*pip, "install", str(repaired_wheel) + build_options.test_extras],
env=virtualenv_env,
)
# Install any requirements to run the tests
if build_options.test_requires:
container.call([*pip, "install", *build_options.test_requires], env=virtualenv_env)
# Run the tests from a different directory
test_command_prepared = prepare_command(
build_options.test_command,
project=container_project_path,
package=container_package_dir,
wheel=repaired_wheel,
)
test_cwd = testing_temp_dir / "test_cwd"
container.call(["mkdir", "-p", test_cwd])
if build_options.test_sources:
copy_test_sources(
build_options.test_sources,
Path.cwd(),
test_cwd,
copy_into=container.copy_into,
)
else:
# Use the test_fail.py file to raise a nice error if the user
# tries to run tests in the cwd
container.copy_into(resources.TEST_FAIL_CWD_FILE, test_cwd / "test_fail.py")
container.call(["sh", "-c", test_command_prepared], cwd=test_cwd, env=virtualenv_env)
# clean up test environment
container.call(["rm", "-rf", testing_temp_dir])
# move repaired wheel to output
output_wheel: Path | None = None
if compatible_wheel is None:
container.call(["mkdir", "-p", container_output_dir])
container.call(["mv", repaired_wheel, container_output_dir])
built_wheels.append(container_output_dir / repaired_wheel.name)
output_wheel = options.globals.output_dir / repaired_wheel.name
log.build_end(output_wheel)
log.step("Copying wheels back to host...")
# copy the output back into the host
container.copy_out(container_output_dir, options.globals.output_dir)
log.step_end()
def build(options: Options, tmp_path: Path) -> None:
python_configurations = get_python_configurations(
options.globals.build_selector, options.globals.architectures
)
cwd = Path.cwd()
abs_package_dir = options.globals.package_dir.resolve()
if cwd != abs_package_dir and cwd not in abs_package_dir.parents:
msg = "package_dir must be inside the working directory"
raise errors.ConfigurationError(msg)
container_project_path = PurePosixPath("/project")
container_package_dir = container_project_path / abs_package_dir.relative_to(cwd)
for build_step in get_build_steps(options, python_configurations):
try:
# check the container engine is installed
subprocess.run(
[build_step.container_engine.name, "--version"],
check=True,
stdout=subprocess.DEVNULL,
)
except subprocess.CalledProcessError as error:
msg = unwrap(
f"""
{build_step.container_engine.name} not found. An OCI exe like
Docker or Podman is required to run Linux builds. If you're
building on Travis CI, add `services: [docker]` to your
.travis.yml. If you're building on Circle CI in Linux, add a
`setup_remote_docker` step to your .circleci/config.yml.
"""
)
raise errors.ConfigurationError(msg) from error
try:
ids_to_build = [x.identifier for x in build_step.platform_configs]
log.step(f"Starting container image {build_step.container_image}...")
print(f"info: This container will host the build for {', '.join(ids_to_build)}...")
architecture = Architecture(build_step.platform_tag.split("_", 1)[1])
with OCIContainer(
image=build_step.container_image,
oci_platform=ARCHITECTURE_OCI_PLATFORM_MAP[architecture],
cwd=container_project_path,
engine=build_step.container_engine,
) as container:
build_in_container(
options=options,
platform_configs=build_step.platform_configs,
container=container,
container_project_path=container_project_path,
container_package_dir=container_package_dir,
local_tmp_dir=tmp_path,
)
except subprocess.CalledProcessError as error:
troubleshoot(options, error)
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
raise errors.FatalError(msg) from error
def _matches_prepared_command(error_cmd: Sequence[str], command_template: str) -> bool:
if len(error_cmd) < 3 or error_cmd[0:2] != ["sh", "-c"]:
return False
command_prefix = command_template.split("{", maxsplit=1)[0].strip()
return error_cmd[2].startswith(command_prefix)
def troubleshoot(options: Options, error: Exception) -> None:
if isinstance(error, subprocess.CalledProcessError) and (
error.cmd[0:4] == ["python", "-m", "pip", "wheel"]
or error.cmd[0:2] == ["uv", "build"]
or error.cmd[0:3] == ["python", "-m", "build"]
or _matches_prepared_command(
error.cmd, options.build_options(None).repair_command
) # TODO allow matching of overrides too?
):
# the wheel build step or the repair step failed
so_files = list(options.globals.package_dir.glob("**/*.so"))
if so_files:
print(
textwrap.dedent(
"""
NOTE: Shared object (.so) files found in this project.
These files might be built against the wrong OS, causing problems with
auditwheel. If possible, run cibuildwheel in a clean checkout.
If you're using Cython and have previously done an in-place build,
remove those build files (*.so and *.c) before starting cibuildwheel.
setuptools uses the build/ folder to store its build cache. It
may be necessary to remove those build files (*.so and *.o) before
starting cibuildwheel.
Files that belong to a virtual environment are probably not an issue
unless you used a custom command telling cibuildwheel to activate it.
"""
),
file=sys.stderr,
)
print(" Files detected:")
print("\n".join(f" {f}" for f in so_files))
print()
-801
View File
@@ -1,801 +0,0 @@
from __future__ import annotations
__lazy_modules__ = {
"cibuildwheel.audit",
"cibuildwheel.ci",
"cibuildwheel.frontend",
"cibuildwheel.logger",
"cibuildwheel.util",
"cibuildwheel.util.cmd",
"cibuildwheel.util.file",
"cibuildwheel.util.helpers",
"cibuildwheel.util.packaging",
"cibuildwheel.venv",
"filelock",
"inspect",
"packaging",
"packaging.version",
"pathlib",
"platform",
"re",
"shutil",
"subprocess",
}
import dataclasses
import functools
import inspect
import os
import platform
import re
import shutil
import subprocess
import sys
import typing
from pathlib import Path
from typing import assert_never
from filelock import FileLock
from packaging.version import Version
from cibuildwheel import errors
from cibuildwheel.audit import run_audit
from cibuildwheel.ci import detect_ci_provider
from cibuildwheel.frontend import (
BuildFrontendName,
get_build_frontend_extra_flags,
prepare_config_settings,
)
from cibuildwheel.logger import log
from cibuildwheel.util import resources
from cibuildwheel.util.cmd import call, shell
from cibuildwheel.util.file import (
CIBW_CACHE_PATH,
copy_test_sources,
download,
move_file,
remove_on_error,
)
from cibuildwheel.util.helpers import prepare_command, unwrap
from cibuildwheel.util.packaging import find_compatible_wheel, get_pip_version
from cibuildwheel.venv import constraint_flags, find_uv, target_marker_env, virtualenv
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Set
from typing import Literal
from cibuildwheel.architecture import Architecture
from cibuildwheel.environment import ParsedEnvironment
from cibuildwheel.options import Options
from cibuildwheel.selector import BuildSelector
@functools.cache
def get_macos_version() -> tuple[int, int]:
"""
Returns the macOS major/minor version, as a tuple, e.g. (10, 15) or (11, 0)
These tuples can be used in comparisons, e.g.
(10, 14) <= (11, 0) == True
(10, 14) <= (10, 16) == True
(11, 2) <= (11, 0) != True
"""
version_str, _, _ = platform.mac_ver()
version = tuple(map(int, version_str.split(".")[:2]))
if (10, 15) < version < (11, 0):
# When built against an older macOS SDK, Python will report macOS 10.16
# instead of the real version.
version_str = call(
sys.executable,
"-sS",
"-c",
"import platform; print(platform.mac_ver()[0])",
env={"SYSTEM_VERSION_COMPAT": "0"},
capture_stdout=True,
)
version = tuple(map(int, version_str.split(".")[:2]))
return typing.cast("tuple[int, int]", version)
@functools.cache
def get_test_macosx_deployment_target() -> str:
version = get_macos_version()
if version >= (11, 0):
return f"{version[0]}.0"
return f"{version[0]}.{version[1]}"
def get_macos_sdks() -> list[str]:
output = call("xcodebuild", "-showsdks", capture_stdout=True)
return [m.group(1) for m in re.finditer(r"-sdk (macosx\S+)", output)]
@dataclasses.dataclass(frozen=True, kw_only=True)
class PythonConfiguration:
version: str
identifier: str
url: str
sha256: str
def all_python_configurations() -> list[PythonConfiguration]:
config_dicts = resources.read_python_configs("macos")
return [PythonConfiguration(**item) for item in config_dicts]
def get_python_configurations(
build_selector: BuildSelector, architectures: Set[Architecture]
) -> list[PythonConfiguration]:
python_configurations = all_python_configurations()
# filter out configs that don't match any of the selected architectures
python_configurations = [
c
for c in python_configurations
if any(c.identifier.endswith(a.value) for a in architectures)
]
# skip builds as required by BUILD/SKIP
python_configurations = [c for c in python_configurations if build_selector(c.identifier)]
# filter-out some cross-compilation configs with PyPy and GraalPy:
# can't build arm64 on x86_64
# rosetta allows to build x86_64 on arm64
if platform.machine() == "x86_64":
python_configurations_before = set(python_configurations)
python_configurations = [
c
for c in python_configurations
if not (c.identifier.startswith(("pp", "gp")) and c.identifier.endswith("arm64"))
]
removed_elements = python_configurations_before - set(python_configurations)
if removed_elements:
ids = ", ".join(c.identifier for c in removed_elements)
log.quiet(
unwrap(
f"""
Note: {ids} {"was" if len(removed_elements) == 1 else "were"}
selected, but can't be built on x86_64 so will be skipped automatically.
"""
)
)
return python_configurations
def install_cpython(_tmp: Path, version: str, url: str, free_threading: bool, sha256: str) -> Path:
ft = "T" if free_threading else ""
installation_path = Path(f"/Library/Frameworks/Python{ft}.framework/Versions/{version}")
with FileLock(CIBW_CACHE_PATH / f"cpython{version}.lock"):
installed_system_packages = call("pkgutil", "--pkgs", capture_stdout=True).splitlines()
# if this version of python isn't installed, get it from python.org and install
python_package_identifier = f"org.python.Python.Python{ft}Framework-{version}"
if python_package_identifier not in installed_system_packages:
if detect_ci_provider() is None:
# if running locally, we don't want to install CPython with sudo
# let the user know & provide a link to the installer
msg = inspect.cleandoc(
f"""
Framework CPython {version} not detected as installed in:
{installation_path}
cibuildwheel will not perform system-wide installs when running outside of CI.
To build locally, install CPython {version} on this machine, or, disable this
version of Python using CIBW_SKIP=cp{version.replace(".", "")}-macosx_*
For portable builds, cibuildwheel needs the official builds from python.org.
Download link: {url}
"""
)
raise errors.FatalError(msg)
python_filename = url.rsplit("/", maxsplit=1)[-1]
pkg_path = CIBW_CACHE_PATH / "cpython-installer" / python_filename
if not pkg_path.exists():
download(url, pkg_path, sha256=sha256)
args = []
if version.startswith("3.14"):
args += ["-applyChoiceChangesXML", str(resources.FREE_THREAD_ENABLE_314.resolve())]
elif version.startswith("3.15"):
args += ["-applyChoiceChangesXML", str(resources.FREE_THREAD_ENABLE_315.resolve())]
call("sudo", "installer", "-pkg", pkg_path, *args, "-target", "/")
pkg_path.unlink()
env = os.environ.copy()
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
if free_threading:
call(installation_path / f"bin/python{version}t", "-m", "ensurepip", env=env)
call(
installation_path / f"bin/python{version}t",
resources.INSTALL_CERTIFI_SCRIPT,
env=env,
)
else:
call(installation_path / "bin/python3", resources.INSTALL_CERTIFI_SCRIPT, env=env)
return installation_path / "bin" / (f"python{version}t" if free_threading else "python3")
def install_pypy(tmp: Path, url: str, sha256: str) -> Path:
pypy_tar_bz2 = url.rsplit("/", 1)[-1]
extension = ".tar.bz2"
assert pypy_tar_bz2.endswith(extension)
installation_path = CIBW_CACHE_PATH / pypy_tar_bz2[: -len(extension)]
with FileLock(str(installation_path) + ".lock"):
if not installation_path.exists():
downloaded_tar_bz2 = tmp / pypy_tar_bz2
download(url, downloaded_tar_bz2, sha256=sha256)
installation_path.parent.mkdir(parents=True, exist_ok=True)
with remove_on_error(installation_path):
call("tar", "-C", installation_path.parent, "-xf", downloaded_tar_bz2)
downloaded_tar_bz2.unlink()
return installation_path / "bin" / "pypy3"
def install_graalpy(tmp: Path, url: str, sha256: str) -> Path:
graalpy_archive = url.rsplit("/", 1)[-1]
extension = ".tar.gz"
assert graalpy_archive.endswith(extension)
installation_path = CIBW_CACHE_PATH / graalpy_archive[: -len(extension)]
with FileLock(str(installation_path) + ".lock"):
if not installation_path.exists():
downloaded_archive = tmp / graalpy_archive
download(url, downloaded_archive, sha256=sha256)
with remove_on_error(installation_path):
installation_path.mkdir(parents=True)
# GraalPy top-folder name is inconsistent with archive name
call(
"tar",
"-C",
installation_path,
"--strip-components=1",
"-xzf",
downloaded_archive,
)
downloaded_archive.unlink()
return installation_path / "bin" / "graalpy"
def setup_python(
tmp: Path,
python_configuration: PythonConfiguration,
dependency_constraint: Path | None,
environment: ParsedEnvironment,
build_frontend: BuildFrontendName,
) -> tuple[Path, dict[str, str]]:
uv_path = find_uv()
use_uv = build_frontend in {"build[uv]", "uv"}
tmp.mkdir()
implementation_id = python_configuration.identifier.split("-")[0]
log.step(f"Installing Python {implementation_id}...")
if implementation_id.startswith("cp"):
free_threading = "t-macos" in python_configuration.identifier
base_python = install_cpython(
tmp,
python_configuration.version,
python_configuration.url,
free_threading,
python_configuration.sha256,
)
elif implementation_id.startswith("pp"):
base_python = install_pypy(tmp, python_configuration.url, python_configuration.sha256)
elif implementation_id.startswith("gp"):
base_python = install_graalpy(tmp, python_configuration.url, python_configuration.sha256)
else:
msg = "Unknown Python implementation"
raise ValueError(msg)
assert base_python.exists(), (
f"{base_python.name} not found, has {list(base_python.parent.iterdir())}"
)
log.step("Setting up build environment...")
venv_path = tmp / "venv"
env = virtualenv(
python_configuration.version,
base_python,
venv_path,
dependency_constraint,
use_uv=use_uv,
marker_env=target_marker_env(implementation_id=implementation_id),
)
venv_bin_path = venv_path / "bin"
assert venv_bin_path.exists()
# Fix issue with site.py setting the wrong `sys.prefix`, `sys.exec_prefix`,
# `sys.path`, ... for PyPy: https://foss.heptapod.net/pypy/pypy/issues/3175
# Also fix an issue with the shebang of installed scripts inside the
# testing virtualenv- see https://github.com/theacodes/nox/issues/44 and
# https://github.com/pypa/virtualenv/issues/620
# Also see https://github.com/python/cpython/pull/9516
env.pop("__PYVENV_LAUNCHER__", None)
# uv uses this over the current environment's python, so remove it to avoid confusion
env.pop("PYTHON_VERSION", None)
env.pop("PYTHON_ARCH", None)
env.pop("UV_PYTHON", None)
# we version pip ourselves, so we don't care about pip version checking
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
# Apply our environment after pip is ready
env = environment.as_dictionary(prev_environment=env)
# check what Python version we're on
which_python = call("which", "python", env=env, capture_stdout=True).strip()
print(which_python)
if which_python != str(venv_bin_path / "python"):
msg = "python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it."
raise errors.FatalError(msg)
call("python", "-V", "-V", env=env)
# check what pip version we're on
if not use_uv:
assert (venv_bin_path / "pip").exists()
which_pip = call("which", "pip", env=env, capture_stdout=True).strip()
print(which_pip)
if which_pip != str(venv_bin_path / "pip"):
msg = "pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it."
raise errors.FatalError(msg)
call("pip", "--version", env=env)
config_is_arm64 = python_configuration.identifier.endswith("arm64")
config_is_universal2 = python_configuration.identifier.endswith("universal2")
# Set MACOSX_DEPLOYMENT_TARGET, if the user didn't set it.
# For arm64, the minimal deployment target is 11.0.
# On x86_64 (or universal2), use 10.9 as a default.
# CPython 3.12.6+ needs 10.13.
# CPython 3.14.0 needs 10.15.
if config_is_arm64:
default_target = "11.0"
elif Version(python_configuration.version) >= Version("3.14"):
default_target = "10.15"
elif Version(python_configuration.version) >= Version("3.12"):
default_target = "10.13"
elif python_configuration.identifier.startswith("pp") and Version(
python_configuration.version
) >= Version("3.9"):
default_target = "10.15"
else:
default_target = "10.9"
env.setdefault("MACOSX_DEPLOYMENT_TARGET", default_target)
# This is a floor, it can't be set lower than the default_target.
if Version(env["MACOSX_DEPLOYMENT_TARGET"]) < Version(default_target):
log.warning(
f"Bumping MACOSX_DEPLOYMENT_TARGET ({env['MACOSX_DEPLOYMENT_TARGET']}) to the minimum required ({default_target})."
)
env["MACOSX_DEPLOYMENT_TARGET"] = default_target
if config_is_arm64:
# macOS 11 is the first OS with arm64 support, so the wheels
# have that as a minimum.
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-11.0-arm64")
env.setdefault("ARCHFLAGS", "-arch arm64")
elif config_is_universal2:
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-10.9-universal2")
env.setdefault("ARCHFLAGS", "-arch arm64 -arch x86_64")
elif python_configuration.identifier.endswith("x86_64"):
# even on the macos11.0 Python installer, on the x86_64 side it's
# compatible back to 10.9.
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-10.9-x86_64")
env.setdefault("ARCHFLAGS", "-arch x86_64")
building_arm64 = config_is_arm64 or config_is_universal2
if building_arm64 and get_macos_version() < (10, 16) and "SDKROOT" not in env:
# xcode 12.2 or higher can build arm64 on macos 10.15 or below, but
# needs the correct SDK selected.
sdks = get_macos_sdks()
# Different versions of Xcode contain different SDK versions...
# we're happy with anything newer than macOS 11.0
arm64_compatible_sdks = [s for s in sdks if not s.startswith("macosx10.")]
if not arm64_compatible_sdks:
log.warning(
unwrap(
"""
SDK for building arm64-compatible wheels not found. You need Xcode 12.2 or later
to build universal2 or arm64 wheels.
"""
)
)
else:
env.setdefault("SDKROOT", arm64_compatible_sdks[0])
log.step("Installing build tools...")
match build_frontend:
case "pip":
call(
"pip",
"install",
"--upgrade",
"delocate",
*constraint_flags(dependency_constraint),
env=env,
)
case "build":
call(
"pip",
"install",
"--upgrade",
"delocate",
"build[virtualenv]",
*constraint_flags(dependency_constraint),
env=env,
)
case "build[uv]":
assert uv_path is not None
call(
uv_path,
"pip",
"install",
"--python",
which_python,
"--upgrade",
"delocate",
"build[virtualenv, uv]",
*constraint_flags(dependency_constraint),
env=env,
)
case "uv":
assert uv_path is not None
call(
uv_path,
"pip",
"install",
"--python",
which_python,
"--upgrade",
"delocate",
*constraint_flags(dependency_constraint),
env=env,
)
case _:
assert_never(build_frontend)
return base_python, env
def build(options: Options, tmp_path: Path) -> None:
python_configurations = get_python_configurations(
options.globals.build_selector, options.globals.architectures
)
if not python_configurations:
return
try:
before_all_options_identifier = python_configurations[0].identifier
before_all_options = options.build_options(before_all_options_identifier)
if before_all_options.before_all:
log.step("Running before_all...")
env = before_all_options.environment.as_dictionary(prev_environment=os.environ)
env.setdefault("MACOSX_DEPLOYMENT_TARGET", "10.9")
before_all_prepared = prepare_command(
before_all_options.before_all, project=".", package=before_all_options.package_dir
)
shell(before_all_prepared, env=env)
built_wheels: list[Path] = []
for config in python_configurations:
build_options = options.build_options(config.identifier)
build_frontend = build_options.build_frontend
use_uv = build_frontend.name in {"build[uv]", "uv"}
uv_path = find_uv()
if use_uv and uv_path is None:
msg = "uv not found"
raise AssertionError(msg)
pip = ["pip"] if not use_uv else [str(uv_path), "pip"]
log.build_start(config.identifier)
identifier_tmp_dir = tmp_path / config.identifier
identifier_tmp_dir.mkdir()
built_wheel_dir = identifier_tmp_dir / "built_wheel"
repaired_wheel_dir = identifier_tmp_dir / "repaired_wheel"
config_is_arm64 = config.identifier.endswith("arm64")
config_is_universal2 = config.identifier.endswith("universal2")
constraints_path = build_options.dependency_constraints.get_for_python_version(
version=config.version, tmp_dir=identifier_tmp_dir
)
base_python, env = setup_python(
identifier_tmp_dir / "build",
config,
constraints_path,
build_options.environment,
build_frontend.name,
)
env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
pip_version = None if use_uv else get_pip_version(env)
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
if compatible_wheel:
log.step_end()
print(
f"\nFound previously built wheel {compatible_wheel.name}, that's compatible with {config.identifier}. Skipping build step..."
)
repaired_wheel = compatible_wheel
else:
if build_options.before_build:
log.step("Running before_build...")
before_build_prepared = prepare_command(
build_options.before_build, project=".", package=build_options.package_dir
)
shell(before_build_prepared, env=env)
log.step("Building wheel...")
built_wheel_dir.mkdir()
extra_flags = get_build_frontend_extra_flags(
build_frontend,
build_options.build_verbosity,
prepare_config_settings(
build_options.config_settings,
project=".",
package=build_options.package_dir,
),
)
build_env = env.copy()
match build_frontend.name:
case "pip":
# Path.resolve() is needed. Without it pip wheel may try to fetch package from pypi.org
# see https://github.com/pypa/cibuildwheel/pull/369
call(
"python",
"-m",
"pip",
"wheel",
build_options.package_dir.resolve(),
f"--wheel-dir={built_wheel_dir}",
"--no-deps",
*extra_flags,
env=build_env,
)
case "build" | "build[uv]":
if (
use_uv
and "--no-isolation" not in extra_flags
and "-n" not in extra_flags
):
extra_flags.append("--installer=uv")
call(
"python",
"-m",
"build",
build_options.package_dir,
"--wheel",
f"--outdir={built_wheel_dir}",
*extra_flags,
env=build_env,
)
case "uv":
assert uv_path is not None
call(
uv_path,
"build",
f"--python={base_python}",
build_options.package_dir,
"--wheel",
f"--out-dir={built_wheel_dir}",
*extra_flags,
env=build_env,
)
case _:
assert_never(build_frontend)
try:
built_wheel = next(built_wheel_dir.glob("*.whl"))
except StopIteration:
raise errors.BuildProducedNoWheelError() from None
repaired_wheel_dir.mkdir()
if built_wheel.name.endswith("none-any.whl"):
raise errors.NonPlatformWheelError()
if build_options.repair_command:
log.step("Repairing wheel...")
if config_is_universal2:
delocate_archs = "x86_64,arm64"
elif config_is_arm64:
delocate_archs = "arm64"
else:
delocate_archs = "x86_64"
repair_command_prepared = prepare_command(
build_options.repair_command,
wheel=built_wheel,
dest_dir=repaired_wheel_dir,
delocate_archs=delocate_archs,
package=build_options.package_dir,
project=".",
)
shell(repair_command_prepared, env=env)
else:
shutil.move(str(built_wheel), repaired_wheel_dir)
try:
repaired_wheel = next(repaired_wheel_dir.glob("*.whl"))
except StopIteration:
raise errors.RepairStepProducedNoWheelError() from None
if repaired_wheel.name in {wheel.name for wheel in built_wheels}:
raise errors.AlreadyBuiltWheelError(repaired_wheel.name)
log.step_end()
run_audit(tmp_dir=tmp_path, build_options=build_options, wheel=repaired_wheel)
if build_options.test_command and build_options.test_selector(config.identifier):
machine_arch = platform.machine()
testing_archs: list[Literal["x86_64", "arm64"]]
if config_is_arm64:
testing_archs = ["arm64"]
elif config_is_universal2:
testing_archs = ["x86_64", "arm64"]
else:
testing_archs = ["x86_64"]
for testing_arch in testing_archs:
if config_is_universal2:
arch_specific_identifier = f"{config.identifier}:{testing_arch}"
if not build_options.test_selector(arch_specific_identifier):
continue
if machine_arch == "x86_64" and testing_arch == "arm64":
if config_is_arm64:
log.warning(
unwrap(
"""
While arm64 wheels can be built on x86_64, they cannot be
tested. Consider building arm64 wheels natively, if your CI
provider offers this. To silence this warning, set
`CIBW_TEST_SKIP: "*-macosx_arm64"`.
"""
)
)
elif config_is_universal2:
log.warning(
unwrap(
"""
While universal2 wheels can be built on x86_64, the arm64 part
of the wheel cannot be tested on x86_64. Consider building
universal2 wheels on an arm64 runner, if your CI provider offers
this. Notably, an arm64 runner can also test the x86_64 part of
the wheel, through Rosetta emulation. To silence this warning,
set `CIBW_TEST_SKIP: "*-macosx_universal2:arm64"`.
"""
)
)
else:
msg = "unreachable"
raise RuntimeError(msg)
# skip this test
continue
log.step(
"Testing wheel..."
if testing_arch == machine_arch
else f"Testing wheel on {testing_arch}..."
)
arch_prefix = []
uv_arch_args = []
if testing_arch != machine_arch:
if machine_arch == "arm64" and testing_arch == "x86_64":
# rosetta2 will provide the emulation with just the arch prefix.
arch_prefix = ["arch", "-x86_64"]
uv_arch_args = ["--python-platform", "x86_64-apple-darwin"]
else:
msg = f"don't know how to emulate {testing_arch} on {machine_arch}"
raise RuntimeError(msg)
# define a custom 'call' function that adds the arch prefix each time
call_with_arch = functools.partial(call, *arch_prefix)
shell_with_arch = functools.partial(call, *arch_prefix, "/bin/sh", "-c")
# set up a virtual environment to install and test from, to make sure
# there are no dependencies that were pulled in at build time.
venv_dir = identifier_tmp_dir / f"venv-test-{testing_arch}"
virtualenv_env = virtualenv(
config.version,
base_python,
venv_dir,
None,
use_uv=use_uv,
env=env,
pip_version=pip_version,
)
if use_uv:
pip_install = functools.partial(call, *pip, "install", *uv_arch_args)
else:
pip_install = functools.partial(call_with_arch, *pip, "install")
virtualenv_env["MACOSX_DEPLOYMENT_TARGET"] = get_test_macosx_deployment_target()
virtualenv_env = build_options.test_environment.as_dictionary(
prev_environment=virtualenv_env
)
# check that we are using the Python from the virtual environment
call_with_arch("which", "python", env=virtualenv_env)
if build_options.before_test:
before_test_prepared = prepare_command(
build_options.before_test,
project=".",
package=build_options.package_dir,
)
shell_with_arch(before_test_prepared, env=virtualenv_env)
# install the wheel
pip_install(
f"{repaired_wheel}{build_options.test_extras}",
env=virtualenv_env,
)
# test the wheel
if build_options.test_requires:
pip_install(
*build_options.test_requires,
env=virtualenv_env,
)
# run the tests from a temp dir, with an absolute path in the command
# (this ensures that Python runs the tests against the installed wheel
# and not the repo code)
test_command_prepared = prepare_command(
build_options.test_command,
project=Path.cwd(),
package=build_options.package_dir.resolve(),
wheel=repaired_wheel,
)
test_cwd = identifier_tmp_dir / "test_cwd"
if build_options.test_sources:
# only create test_cwd if it doesn't already exist - it
# may have been created during a previous `testing_arch`
if not test_cwd.exists():
test_cwd.mkdir()
copy_test_sources(
build_options.test_sources,
Path.cwd(),
test_cwd,
)
else:
# Use the test_fail.py file to raise a nice error if the user
# tries to run tests in the cwd
test_cwd.mkdir(exist_ok=True)
(test_cwd / "test_fail.py").write_text(
resources.TEST_FAIL_CWD_FILE.read_text()
)
shell_with_arch(test_command_prepared, cwd=test_cwd, env=virtualenv_env)
# we're all done here; move it to output (overwrite existing)
output_wheel = None
if compatible_wheel is None:
output_wheel = build_options.output_dir.joinpath(repaired_wheel.name)
moved_wheel = move_file(repaired_wheel, output_wheel)
if moved_wheel != output_wheel.resolve():
log.warning(
f"{repaired_wheel} was moved to {moved_wheel} instead of {output_wheel}"
)
built_wheels.append(output_wheel)
# clean up
shutil.rmtree(identifier_tmp_dir)
log.build_end(output_wheel)
except subprocess.CalledProcessError as error:
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
raise errors.FatalError(msg) from error
-640
View File
@@ -1,640 +0,0 @@
from __future__ import annotations
__lazy_modules__ = {
"cibuildwheel.architecture",
"cibuildwheel.audit",
"cibuildwheel.frontend",
"cibuildwheel.logger",
"cibuildwheel.util",
"cibuildwheel.util.cmd",
"cibuildwheel.util.file",
"cibuildwheel.util.helpers",
"cibuildwheel.util.packaging",
"cibuildwheel.util.python_build_standalone",
"cibuildwheel.venv",
"filelock",
"json",
"pathlib",
"shutil",
"subprocess",
"tempfile",
"tomllib",
}
import dataclasses
import functools
import json
import os
import shutil
import subprocess
import sys
import tomllib
import typing
from pathlib import Path
from tempfile import TemporaryDirectory
from typing import Final, TypedDict
from filelock import FileLock
from cibuildwheel import errors
from cibuildwheel.architecture import Architecture
from cibuildwheel.audit import run_audit
from cibuildwheel.frontend import get_build_frontend_extra_flags, prepare_config_settings
from cibuildwheel.logger import log
from cibuildwheel.util import resources
from cibuildwheel.util.cmd import call, shell
from cibuildwheel.util.file import (
CIBW_CACHE_PATH,
copy_test_sources,
download,
extract_tar,
extract_zip,
move_file,
remove_on_error,
)
from cibuildwheel.util.helpers import prepare_command, unwrap, unwrap_preserving_paragraphs
from cibuildwheel.util.packaging import find_compatible_wheel, get_pip_version
from cibuildwheel.util.python_build_standalone import (
PythonBuildStandaloneError,
create_python_build_standalone_environment,
)
from cibuildwheel.venv import constraint_flags, virtualenv
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import Set
from cibuildwheel.environment import ParsedEnvironment
from cibuildwheel.options import Options
from cibuildwheel.selector import BuildSelector
IS_WIN: Final[bool] = sys.platform.startswith("win")
@dataclasses.dataclass(frozen=True, kw_only=True)
class PythonConfiguration:
version: str
identifier: str
default_pyodide_version: str
node_version: str
sha256: str = ""
class PyodideXBuildEnvInfoVersionRange(TypedDict):
min: str | None
max: str | None
class PyodideXBuildEnvInfo(TypedDict):
version: str
python: str
emscripten: str
pyodide_build: PyodideXBuildEnvInfoVersionRange
compatible: bool
@functools.cache
def ensure_node(major_version: str) -> Path:
with resources.NODEJS.open("rb") as f:
loaded_file = tomllib.load(f)
version = str(loaded_file[major_version])
base_url = str(loaded_file["url"])
ext = "zip" if IS_WIN else "tar.xz"
platform = "win" if IS_WIN else ("darwin" if sys.platform.startswith("darwin") else "linux")
linux_arch = Architecture.native_arch("linux")
assert linux_arch is not None
arch = {"x86_64": "x64", "i686": "x86", "aarch64": "arm64"}.get(
linux_arch.value, linux_arch.value
)
name = f"node-{version}-{platform}-{arch}"
path = CIBW_CACHE_PATH / name
with FileLock(str(path) + ".lock"):
if not path.exists():
url = f"{base_url}{version}/{name}.{ext}"
with TemporaryDirectory() as tmp_path:
archive = Path(tmp_path) / f"{name}.{ext}"
download(url, archive)
with remove_on_error(path):
if ext == "zip":
extract_zip(archive, path.parent)
else:
extract_tar(archive, path.parent)
assert path.exists()
if not IS_WIN:
return path / "bin"
return path
def install_emscripten(env: dict[str, str], version: str, xbuildenv_cache_path: Path) -> Path:
"""Install Emscripten via pyodide-build, which also applies Pyodide-specific patches."""
emscripten_dir = Path(
call("pyodide", "config", "get", "emscripten_dir", env=env, capture_stdout=True).strip()
)
with FileLock(CIBW_CACHE_PATH / "emscripten.lock"):
if emscripten_dir.exists():
return emscripten_dir
with remove_on_error(emscripten_dir):
call(
"pyodide",
"xbuildenv",
"install-emscripten",
"--force",
"--version",
version,
"--path",
str(xbuildenv_cache_path),
env=env,
cwd=CIBW_CACHE_PATH,
)
assert emscripten_dir.exists()
return emscripten_dir
def get_all_xbuildenv_version_info(env: dict[str, str]) -> list[PyodideXBuildEnvInfo]:
xbuildenvs_info_str = call(
"pyodide",
"xbuildenv",
"search",
"--json",
"--all",
env=env,
cwd=CIBW_CACHE_PATH,
capture_stdout=True,
).strip()
xbuildenvs_info = json.loads(xbuildenvs_info_str)
if "environments" not in xbuildenvs_info:
msg = f"Invalid xbuildenvs info, got {xbuildenvs_info}"
raise ValueError(msg)
return typing.cast("list[PyodideXBuildEnvInfo]", xbuildenvs_info["environments"])
def get_xbuildenv_version_info(
env: dict[str, str], version: str, pyodide_build_version: str
) -> PyodideXBuildEnvInfo:
xbuildenvs_info = get_all_xbuildenv_version_info(env)
for xbuildenv_info in xbuildenvs_info:
if xbuildenv_info["version"] == version:
return xbuildenv_info
msg = unwrap(f"""
Could not find Pyodide cross-build environment version {version} in the available
versions as reported by pyodide-build v{pyodide_build_version}.
Available pyodide xbuildenv versions are:
{", ".join(e["version"] for e in xbuildenvs_info if e["compatible"])}
""")
raise errors.FatalError(msg)
# The default pyodide xbuildenv version that's specified in
# build-platforms.toml is compatible with the pyodide-build version that's
# pinned in the bundled constraints file. But if the user changes
# pyodide-version and/or dependency-constraints in the cibuildwheel config, we
# need to check if the xbuildenv version is compatible with the pyodide-build
# version.
def validate_pyodide_build_version(
xbuildenv_info: PyodideXBuildEnvInfo, pyodide_build_version: str
) -> None:
"""
Validate the Pyodide version is compatible with the installed
pyodide-build version.
"""
pyodide_version = xbuildenv_info["version"]
if not xbuildenv_info["compatible"]:
msg = unwrap_preserving_paragraphs(f"""
The Pyodide xbuildenv version {pyodide_version} is not compatible
with the pyodide-build version {pyodide_build_version}. Please use
the 'pyodide xbuildenv search --all' command to find a compatible
version.
Set the pyodide-build version using the `dependency-constraints`
option, or set the Pyodide xbuildenv version using the
`pyodide-version` option.
""")
raise errors.FatalError(msg)
def validate_pyodide_target_python(
xbuildenv_info: PyodideXBuildEnvInfo, python_configuration: PythonConfiguration
) -> None:
"""
Validate that the resolved Pyodide xbuildenv targets the same Python version
as the identifier being built.
This is to catch the case where a global ``pyodide-version`` is applied to an
identifier whose Python version does not match, such as setting
``pyodide-version = "3Y.0.0"`` (a Python 3.Y environment) while
building ``cp3X-pyodide_wasm32``.
"""
xbuildenv_version = xbuildenv_info["version"]
identifier = python_configuration.identifier
expected_version = python_configuration.version
default_version = python_configuration.default_pyodide_version
xbuildenv_python_minor = ".".join(xbuildenv_info["python"].split(".")[:2])
if xbuildenv_python_minor != expected_version:
msg = unwrap_preserving_paragraphs(f"""
The `pyodide-version` option is set to {xbuildenv_version}, which
provides Python {xbuildenv_python_minor}, but the {identifier} build
needs Python {expected_version}.
Either remove `pyodide-version` so {identifier} uses its default
({default_version}), or stop building {identifier} so that
`pyodide-version` only applies to an identifier it matches.
""")
raise errors.ConfigurationError(msg)
def install_xbuildenv(env: dict[str, str], xbuildenv_cache_path: Path, pyodide_version: str) -> str:
"""Install a particular Pyodide xbuildenv version and set a path to the Pyodide root."""
pyodide_root = xbuildenv_cache_path / pyodide_version / "xbuildenv" / "pyodide-root"
with FileLock(CIBW_CACHE_PATH / "xbuildenv.lock"):
if pyodide_root.exists():
return str(pyodide_root)
# We don't want to mutate env but we need to delete any existing
# PYODIDE_ROOT so copy it first.
env = dict(env)
env.pop("PYODIDE_ROOT", None)
# Install the xbuildenv
with remove_on_error(xbuildenv_cache_path / pyodide_version):
call(
"pyodide",
"xbuildenv",
"install",
"--path",
str(xbuildenv_cache_path),
pyodide_version,
env=env,
cwd=CIBW_CACHE_PATH,
)
assert pyodide_root.exists()
return str(pyodide_root)
def get_base_python(tmp: Path, python_configuration: PythonConfiguration) -> Path:
try:
return create_python_build_standalone_environment(
python_version=python_configuration.version,
temp_dir=tmp,
cache_dir=CIBW_CACHE_PATH,
)
except PythonBuildStandaloneError as e:
msg = unwrap(f"""
Failed to create a Python build environment:
{e}
""")
raise errors.FatalError(msg) from e
def setup_python(
tmp: Path,
python_configuration: PythonConfiguration,
constraints_path: Path | None,
environment: ParsedEnvironment,
user_pyodide_version: str | None,
) -> dict[str, str]:
log.step("Installing a base python environment...")
base_python = get_base_python(tmp / "base", python_configuration)
log.step("Setting up build environment...")
pyodide_version = user_pyodide_version or python_configuration.default_pyodide_version
venv_path = tmp / "venv"
env = virtualenv(python_configuration.version, base_python, venv_path, None, use_uv=False)
venv_bin_path = venv_path / "bin"
assert venv_bin_path.exists()
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
# upgrade pip to the version matching our constraints
# if necessary, reinstall it to ensure that it's available on PATH as 'pip'
call(
"python",
"-m",
"pip",
"install",
"--upgrade",
"pip",
*constraint_flags(constraints_path),
env=env,
cwd=venv_path,
)
env = environment.as_dictionary(prev_environment=env)
# check what Python version we're on
which_python = call("which", "python", env=env, capture_stdout=True).strip()
print(which_python)
if which_python != str(venv_bin_path / "python"):
msg = "python available on PATH doesn't match our venv instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it."
raise errors.FatalError(msg)
call("python", "-V", "-V", env=env)
# check what pip version we're on
assert (venv_bin_path / "pip").exists()
which_pip = call("which", "pip", env=env, capture_stdout=True).strip()
print(which_pip)
if which_pip != str(venv_bin_path / "pip"):
msg = "pip available on PATH doesn't match our venv instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it."
raise errors.FatalError(msg)
call("pip", "--version", env=env)
log.step("Installing build tools...")
call(
"pip",
"install",
"--upgrade",
"pyodide-build",
*constraint_flags(constraints_path),
env=env,
)
pyodide_build_version = call(
"python",
"-c",
"from importlib.metadata import version; print(version('pyodide-build'))",
env=env,
capture_stdout=True,
).strip()
xbuildenv_info = get_xbuildenv_version_info(env, pyodide_version, pyodide_build_version)
validate_pyodide_build_version(
xbuildenv_info=xbuildenv_info,
pyodide_build_version=pyodide_build_version,
)
validate_pyodide_target_python(
xbuildenv_info=xbuildenv_info,
python_configuration=python_configuration,
)
xbuildenv_cache_path = CIBW_CACHE_PATH / f"pyodide-build-{pyodide_build_version}"
log.step(f"Installing Pyodide xbuildenv version: {pyodide_version} ...")
env["PYODIDE_ROOT"] = install_xbuildenv(env, xbuildenv_cache_path, pyodide_version)
emscripten_version = xbuildenv_info["emscripten"]
log.step(
f"Installing Emscripten {emscripten_version} and applying Pyodide-specific patches ..."
)
emscripten_dir = install_emscripten(env, emscripten_version, xbuildenv_cache_path)
env["PATH"] = os.pathsep.join([str(emscripten_dir), env["PATH"]])
return env
def all_python_configurations() -> list[PythonConfiguration]:
full_python_configs = resources.read_python_configs("pyodide")
return [PythonConfiguration(**item) for item in full_python_configs]
def get_python_configurations(
build_selector: BuildSelector,
architectures: Set[Architecture], # noqa: ARG001
) -> list[PythonConfiguration]:
return [c for c in all_python_configurations() if build_selector(c.identifier)]
def build(options: Options, tmp_path: Path) -> None:
python_configurations = get_python_configurations(
options.globals.build_selector, options.globals.architectures
)
if not python_configurations:
return
try:
before_all_options_identifier = python_configurations[0].identifier
before_all_options = options.build_options(before_all_options_identifier)
if before_all_options.before_all:
log.step("Running before_all...")
env = before_all_options.environment.as_dictionary(prev_environment=os.environ)
before_all_prepared = prepare_command(
before_all_options.before_all, project=".", package=before_all_options.package_dir
)
shell(before_all_prepared, env=env)
built_wheels: list[Path] = []
for config in python_configurations:
build_options = options.build_options(config.identifier)
build_frontend = build_options.build_frontend
if build_frontend.name == "pip":
msg = "The pyodide platform doesn't support pip frontend"
raise errors.FatalError(msg)
log.build_start(config.identifier)
identifier_tmp_dir = tmp_path / config.identifier
built_wheel_dir = identifier_tmp_dir / "built_wheel"
repaired_wheel_dir = identifier_tmp_dir / "repaired_wheel"
identifier_tmp_dir.mkdir()
built_wheel_dir.mkdir()
repaired_wheel_dir.mkdir()
constraints_path = build_options.dependency_constraints.get_for_python_version(
version=config.version, variant="pyodide", tmp_dir=identifier_tmp_dir
)
env = setup_python(
tmp=identifier_tmp_dir / "build",
python_configuration=config,
constraints_path=constraints_path,
environment=build_options.environment,
user_pyodide_version=build_options.pyodide_version,
)
env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
pip_version = get_pip_version(env)
# The Pyodide command line runner mounts all directories in the host
# filesystem into the Pyodide file system, except for the custom
# file systems /dev, /lib, /proc, and /tmp. Mounting the mount
# points for alternate file systems causes some mysterious failure
# of the process (it just quits without any clear error).
#
# Because of this, by default Pyodide can't see anything under /tmp.
# This environment variable tells it also to mount our temp
# directory.
oldmounts = ""
extra_mounts = [str(identifier_tmp_dir)]
if Path.cwd().is_relative_to("/tmp"):
extra_mounts.append(str(Path.cwd()))
if "_PYODIDE_EXTRA_MOUNTS" in env:
oldmounts = env["_PYODIDE_EXTRA_MOUNTS"] + ":"
env["_PYODIDE_EXTRA_MOUNTS"] = oldmounts + ":".join(extra_mounts)
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
if compatible_wheel:
log.step_end()
print(
f"\nFound previously built wheel {compatible_wheel.name}, that's compatible with {config.identifier}. Skipping build step..."
)
built_wheel = compatible_wheel
else:
if build_options.before_build:
log.step("Running before_build...")
before_build_prepared = prepare_command(
build_options.before_build, project=".", package=build_options.package_dir
)
shell(before_build_prepared, env=env)
log.step("Building wheel...")
extra_flags = get_build_frontend_extra_flags(
build_frontend,
build_options.build_verbosity,
prepare_config_settings(
build_options.config_settings,
project=".",
package=build_options.package_dir,
),
)
call(
"pyodide",
"build",
build_options.package_dir,
f"--outdir={built_wheel_dir}",
*extra_flags,
env=env,
)
try:
built_wheel = next(built_wheel_dir.glob("*.whl"))
except StopIteration:
raise errors.BuildProducedNoWheelError() from None
if built_wheel.name.endswith("none-any.whl"):
raise errors.NonPlatformWheelError()
if build_options.repair_command:
log.step("Repairing wheel...")
repair_command_prepared = prepare_command(
build_options.repair_command,
wheel=built_wheel,
dest_dir=repaired_wheel_dir,
package=build_options.package_dir,
project=".",
)
shell(repair_command_prepared, env=env)
log.step_end()
else:
shutil.move(str(built_wheel), repaired_wheel_dir)
try:
repaired_wheel = next(repaired_wheel_dir.glob("*.whl"))
except StopIteration:
raise errors.RepairStepProducedNoWheelError() from None
if repaired_wheel.name in {wheel.name for wheel in built_wheels}:
raise errors.AlreadyBuiltWheelError(repaired_wheel.name)
run_audit(tmp_dir=tmp_path, build_options=build_options, wheel=repaired_wheel)
if build_options.test_command and build_options.test_selector(config.identifier):
log.step("Testing wheel...")
venv_dir = identifier_tmp_dir / "venv-test"
# set up a virtual environment to install and test from, to make sure
# there are no dependencies that were pulled in at build time.
virtualenv_env = env.copy()
virtualenv_env["PATH"] = os.pathsep.join(
[
str(ensure_node(config.node_version)),
virtualenv_env["PATH"],
]
)
# pyodide venv uses virtualenv under the hood
# use the pip embedded with virtualenv & disable network updates
virtualenv_create_env = virtualenv_env.copy()
virtualenv_create_env["VIRTUALENV_PIP"] = pip_version
virtualenv_create_env["VIRTUALENV_NO_PERIODIC_UPDATE"] = "1"
call("pyodide", "venv", venv_dir, env=virtualenv_create_env)
virtualenv_env["PATH"] = os.pathsep.join(
[
str(venv_dir / "bin"),
virtualenv_env["PATH"],
]
)
virtualenv_env["VIRTUAL_ENV"] = str(venv_dir)
virtualenv_env = build_options.test_environment.as_dictionary(
prev_environment=virtualenv_env
)
# check that we are using the Python from the virtual environment
call("which", "python", env=virtualenv_env)
if build_options.before_test:
before_test_prepared = prepare_command(
build_options.before_test,
project=".",
package=build_options.package_dir,
wheel=repaired_wheel,
)
shell(before_test_prepared, env=virtualenv_env)
# install the wheel
call(
"pip",
"install",
f"{repaired_wheel}{build_options.test_extras}",
env=virtualenv_env,
)
# test the wheel
if build_options.test_requires:
call("pip", "install", *build_options.test_requires, env=virtualenv_env)
# run the tests from a temp dir, with an absolute path in the command
# (this ensures that Python runs the tests against the installed wheel
# and not the repo code)
test_command_prepared = prepare_command(
build_options.test_command,
project=Path.cwd(),
package=build_options.package_dir.resolve(),
)
test_cwd = identifier_tmp_dir / "test_cwd"
test_cwd.mkdir(exist_ok=True)
if build_options.test_sources:
copy_test_sources(
build_options.test_sources,
Path.cwd(),
test_cwd,
)
else:
# Use the test_fail.py file to raise a nice error if the user
# tries to run tests in the cwd
(test_cwd / "test_fail.py").write_text(resources.TEST_FAIL_CWD_FILE.read_text())
shell(test_command_prepared, cwd=test_cwd, env=virtualenv_env)
# we're all done here; move it to output (overwrite existing)
output_wheel: Path | None = None
if compatible_wheel is None:
output_wheel = build_options.output_dir.joinpath(repaired_wheel.name)
moved_wheel = move_file(repaired_wheel, output_wheel)
if moved_wheel != output_wheel.resolve():
log.warning(
f"{repaired_wheel} was moved to {moved_wheel} instead of {output_wheel}"
)
built_wheels.append(output_wheel)
log.build_end(output_wheel)
except subprocess.CalledProcessError as error:
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
raise errors.FatalError(msg) from error
-684
View File
@@ -1,684 +0,0 @@
from __future__ import annotations
__lazy_modules__ = {
"cibuildwheel.architecture",
"cibuildwheel.audit",
"cibuildwheel.frontend",
"cibuildwheel.logger",
"cibuildwheel.util",
"cibuildwheel.util.cmd",
"cibuildwheel.util.file",
"cibuildwheel.util.helpers",
"cibuildwheel.util.packaging",
"cibuildwheel.venv",
"filelock",
"pathlib",
"platform",
"shutil",
"subprocess",
"textwrap",
"typing",
}
import dataclasses
import os
import platform as platform_module
import shutil
import subprocess
import textwrap
from functools import cache
from pathlib import Path
from typing import assert_never
from filelock import FileLock
from cibuildwheel import errors
from cibuildwheel.architecture import Architecture
from cibuildwheel.audit import run_audit
from cibuildwheel.frontend import (
BuildFrontendName,
get_build_frontend_extra_flags,
prepare_config_settings,
)
from cibuildwheel.logger import log
from cibuildwheel.util import resources
from cibuildwheel.util.cmd import call, shell
from cibuildwheel.util.file import (
CIBW_CACHE_PATH,
copy_test_sources,
download,
extract_zip,
move_file,
remove_on_error,
)
from cibuildwheel.util.helpers import prepare_command, unwrap
from cibuildwheel.util.packaging import find_compatible_wheel, get_pip_version
from cibuildwheel.venv import constraint_flags, find_uv, target_marker_env, virtualenv
TYPE_CHECKING = False
if TYPE_CHECKING:
from collections.abc import MutableMapping, Sequence, Set
from cibuildwheel.environment import ParsedEnvironment
from cibuildwheel.options import Options
from cibuildwheel.selector import BuildSelector
def get_nuget_args(
version: str, arch: str, free_threaded: bool, output_directory: Path
) -> list[str]:
package_name = {
"32": "pythonx86",
"64": "python",
"ARM64": "pythonarm64",
# Aliases for platform.machine() return values
"x86": "pythonx86",
"AMD64": "python",
}[arch]
if free_threaded:
package_name = f"{package_name}-freethreaded"
return [
package_name,
"-Version",
version,
"-FallbackSource",
"https://api.nuget.org/v3/index.json",
"-OutputDirectory",
str(output_directory),
]
@dataclasses.dataclass(frozen=True, kw_only=True)
class PythonConfiguration:
version: str
identifier: str
url: str | None = None
sha256: str = ""
@property
def arch(self) -> str:
return {
"win32": "32",
"win_amd64": "64",
"win_arm64": "ARM64",
}[self.identifier.split("-")[-1]]
def all_python_configurations() -> list[PythonConfiguration]:
config_dicts = resources.read_python_configs("windows")
return [PythonConfiguration(**item) for item in config_dicts]
def get_python_configurations(
build_selector: BuildSelector,
architectures: Set[Architecture],
) -> list[PythonConfiguration]:
python_configurations = all_python_configurations()
map_arch = {"32": Architecture.x86, "64": Architecture.AMD64, "ARM64": Architecture.ARM64}
# skip builds as required
python_configurations = [
c
for c in python_configurations
if build_selector(c.identifier) and map_arch[c.arch] in architectures
]
return python_configurations
@cache
def _ensure_nuget() -> Path:
nuget = CIBW_CACHE_PATH / "nuget.exe"
with FileLock(str(nuget) + ".lock"):
if not nuget.exists():
with remove_on_error(nuget):
download("https://dist.nuget.org/win-x86-commandline/latest/nuget.exe", nuget)
return nuget
def install_cpython(configuration: PythonConfiguration, arch: str | None = None) -> Path:
version = configuration.version
free_threaded = "t-" in configuration.identifier
if arch is None:
arch = configuration.arch
base_output_dir = CIBW_CACHE_PATH / "nuget-cpython"
nuget_args = get_nuget_args(version, arch, free_threaded, base_output_dir)
installation_path = base_output_dir / (nuget_args[0] + "." + version) / "tools"
free_threaded_str = "-freethreaded" if free_threaded else ""
with FileLock(str(base_output_dir) + f"-{version}{free_threaded_str}-{arch}.lock"):
if not installation_path.exists():
nuget = _ensure_nuget()
with remove_on_error(installation_path.parent):
call(nuget, "install", *nuget_args)
return installation_path / "python.exe"
def install_pypy(tmp: Path, arch: str, url: str, sha256: str) -> Path:
assert arch == "64"
assert "win64" in url
# Inside the PyPy zip file is a directory with the same name
zip_filename = url.rsplit("/", 1)[-1]
extension = ".zip"
assert zip_filename.endswith(extension)
installation_path = CIBW_CACHE_PATH / zip_filename[: -len(extension)]
with FileLock(str(installation_path) + ".lock"):
if not installation_path.exists():
pypy_zip = tmp / zip_filename
download(url, pypy_zip, sha256=sha256)
with remove_on_error(installation_path):
# Extract to the parent directory because the zip file still contains a directory
extract_zip(pypy_zip, installation_path.parent)
return installation_path / "python.exe"
def install_graalpy(tmp: Path, url: str, sha256: str) -> Path:
zip_filename = url.rsplit("/", 1)[-1]
extension = ".zip"
assert zip_filename.endswith(extension)
installation_path = CIBW_CACHE_PATH / zip_filename[: -len(extension)]
with FileLock(str(installation_path) + ".lock"):
if not installation_path.exists():
graalpy_zip = tmp / zip_filename
download(url, graalpy_zip, sha256=sha256)
with remove_on_error(installation_path):
# Extract to the parent directory because the zip file still contains a directory
extract_zip(graalpy_zip, installation_path.parent)
return installation_path / "bin" / "graalpy.exe"
def setup_setuptools_cross_compile(
tmp: Path,
python_configuration: PythonConfiguration,
python_libs_base: Path,
env: MutableMapping[str, str],
) -> None:
distutils_cfg = tmp / "extra-setup.cfg"
env["DIST_EXTRA_CONFIG"] = str(distutils_cfg)
log.notice(f"Setting DIST_EXTRA_CONFIG={distutils_cfg} for cross-compilation")
# Ensure our additional import libraries are made available, and explicitly
# set the platform name
map_plat = {"32": "win32", "64": "win-amd64", "ARM64": "win-arm64"}
plat_name = map_plat[python_configuration.arch]
# Set environment variable so that setuptools._distutils.get_platform()
# identifies the target, not the host
vscmd_arg_tgt_arch = {"32": "x86", "64": "x64", "ARM64": "arm64"}
current_tgt_arch = vscmd_arg_tgt_arch[python_configuration.arch]
if (env.get("VSCMD_ARG_TGT_ARCH") or current_tgt_arch) != current_tgt_arch:
msg = f"VSCMD_ARG_TGT_ARCH must be set to {current_tgt_arch!r}, got {env['VSCMD_ARG_TGT_ARCH']!r}. If you're setting up MSVC yourself (e.g. using vcvarsall.bat or msvc-dev-cmd), make sure to target the right architecture. Alternatively, run cibuildwheel without configuring MSVC, and let the build backend handle it."
raise errors.FatalError(msg)
env["VSCMD_ARG_TGT_ARCH"] = current_tgt_arch
# (This file must be default/locale encoding, so we can't pass 'encoding')
distutils_cfg.write_text(
textwrap.dedent(
f"""\
[build]
plat_name={plat_name}
[build_ext]
library_dirs={python_libs_base}
plat_name={plat_name}
[bdist_wheel]
plat_name={plat_name}
"""
)
)
# setuptools builds require explicit override of PYD extension
# This is because it always gets the extension from the running
# interpreter, and has no logic to construct it. Currently, CPython's
# extensions follow our identifiers, but if they ever diverge in the
# future, we will need to store new data
log.notice(
f"Setting SETUPTOOLS_EXT_SUFFIX=.{python_configuration.identifier}.pyd for cross-compilation"
)
env["SETUPTOOLS_EXT_SUFFIX"] = f".{python_configuration.identifier}.pyd"
# Cross-compilation requires fixes that only exist in setuptools's copy of
# distutils, so ensure that it is activated
# Since not all projects can handle the newer distutils, display a warning
# to help them figure out what may have gone wrong if this breaks for them
log.notice("Setting SETUPTOOLS_USE_DISTUTILS=local as it is required for cross-compilation")
env["SETUPTOOLS_USE_DISTUTILS"] = "local"
# These cross-compile setup functions have the same signature by design
def setup_rust_cross_compile(
tmp: Path, # noqa: ARG001
python_configuration: PythonConfiguration,
python_libs_base: Path, # noqa: ARG001
env: MutableMapping[str, str],
) -> None:
# Assume that MSVC will be used, because we already know that we are
# cross-compiling. MinGW users can set CARGO_BUILD_TARGET themselves
# and we will respect the existing value.
cargo_target = {
"64": "x86_64-pc-windows-msvc",
"32": "i686-pc-windows-msvc",
"ARM64": "aarch64-pc-windows-msvc",
}.get(python_configuration.arch)
# CARGO_BUILD_TARGET is the variable used by Cargo and setuptools_rust
if env.get("CARGO_BUILD_TARGET"):
if env["CARGO_BUILD_TARGET"] != cargo_target:
log.notice("Not overriding CARGO_BUILD_TARGET as it has already been set")
# No message if it was set to what we were planning to set it to
elif cargo_target:
log.notice(f"Setting CARGO_BUILD_TARGET={cargo_target} for cross-compilation")
env["CARGO_BUILD_TARGET"] = cargo_target
else:
log.warning(
f"Unable to configure Rust cross-compilation for architecture {python_configuration.arch}"
)
def setup_python(
tmp: Path,
python_configuration: PythonConfiguration,
dependency_constraint: Path | None,
environment: ParsedEnvironment,
build_frontend: BuildFrontendName,
) -> tuple[Path, dict[str, str]]:
tmp.mkdir()
implementation_id = python_configuration.identifier.split("-")[0]
python_libs_base = None
log.step(f"Installing Python {implementation_id}...")
if implementation_id.startswith("cp"):
native_arch = platform_module.machine()
base_python = install_cpython(python_configuration)
if python_configuration.arch == "ARM64" != native_arch:
# To cross-compile for ARM64, we need a native CPython to run the
# build, and a copy of the ARM64 import libraries ('.\libs\*.lib')
# for any extension modules.
python_libs_base = base_python.parent / "libs"
log.step(f"Installing native Python {native_arch} for cross-compilation...")
base_python = install_cpython(python_configuration, arch=native_arch)
elif implementation_id.startswith("pp"):
assert python_configuration.url is not None
assert python_configuration.sha256
base_python = install_pypy(
tmp, python_configuration.arch, python_configuration.url, python_configuration.sha256
)
elif implementation_id.startswith("gp"):
assert python_configuration.url is not None
assert python_configuration.sha256
base_python = install_graalpy(tmp, python_configuration.url, python_configuration.sha256)
else:
msg = "Unknown Python implementation"
raise ValueError(msg)
assert base_python.exists()
use_uv = build_frontend in {"build[uv]", "uv"}
uv_path = find_uv()
log.step("Setting up build environment...")
venv_path = tmp / "venv"
env = virtualenv(
python_configuration.version,
base_python,
venv_path,
dependency_constraint,
use_uv=use_uv,
marker_env=target_marker_env(implementation_id=implementation_id),
)
# set up environment variables for run_with_env
env["PYTHON_VERSION"] = python_configuration.version
env["PYTHON_ARCH"] = python_configuration.arch
env.pop("UV_PYTHON", None)
if not use_uv:
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
# update env with results from CIBW_ENVIRONMENT
env = environment.as_dictionary(prev_environment=env)
# check what Python version we're on
where_python = call("where", "python", env=env, capture_stdout=True).splitlines()[0].strip()
print(where_python)
if where_python != str(venv_path / "Scripts" / "python.exe"):
msg = "python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it."
raise errors.FatalError(msg)
call("python", "-V", "-V", env=env)
call("python", "-c", "\"import struct; print(struct.calcsize('P') * 8)\"", env=env)
# check what pip version we're on
if not use_uv:
assert (venv_path / "Scripts" / "pip.exe").exists()
where_pip = call("where", "pip", env=env, capture_stdout=True).splitlines()[0].strip()
print(where_pip)
if where_pip != str(venv_path / "Scripts" / "pip.exe"):
msg = "pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it."
raise errors.FatalError(msg)
call("pip", "--version", env=env)
log.step("Installing build tools...")
match build_frontend:
case "pip":
call(
"pip",
"install",
"--upgrade",
"delvewheel",
*constraint_flags(dependency_constraint),
env=env,
)
case "build":
call(
"pip",
"install",
"--upgrade",
"build[virtualenv]",
"delvewheel",
*constraint_flags(dependency_constraint),
env=env,
)
case "build[uv]":
assert uv_path is not None
call(
uv_path,
"pip",
"install",
"--python",
where_python,
"--upgrade",
"build[virtualenv]",
"delvewheel",
*constraint_flags(dependency_constraint),
env=env,
)
case "uv":
assert uv_path is not None
call(
uv_path,
"pip",
"install",
"--python",
where_python,
"--upgrade",
"delvewheel",
*constraint_flags(dependency_constraint),
env=env,
)
case _:
assert_never(build_frontend)
if python_libs_base:
# Set up the environment for various backends to enable cross-compilation
setup_setuptools_cross_compile(tmp, python_configuration, python_libs_base, env)
setup_rust_cross_compile(tmp, python_configuration, python_libs_base, env)
return base_python, env
def build(options: Options, tmp_path: Path) -> None:
python_configurations = get_python_configurations(
options.globals.build_selector, options.globals.architectures
)
if not python_configurations:
return
uv_path = find_uv()
try:
before_all_options_identifier = python_configurations[0].identifier
before_all_options = options.build_options(before_all_options_identifier)
if before_all_options.before_all:
log.step("Running before_all...")
env = before_all_options.environment.as_dictionary(prev_environment=os.environ)
before_all_prepared = prepare_command(
before_all_options.before_all, project=".", package=options.globals.package_dir
)
shell(before_all_prepared, env=env)
built_wheels: list[Path] = []
for config in python_configurations:
build_options = options.build_options(config.identifier)
build_frontend = build_options.build_frontend
use_uv = build_frontend.name in {"build[uv]", "uv"}
log.build_start(config.identifier)
identifier_tmp_dir = tmp_path / config.identifier
identifier_tmp_dir.mkdir()
built_wheel_dir = identifier_tmp_dir / "built_wheel"
repaired_wheel_dir = identifier_tmp_dir / "repaired_wheel"
constraints_path = build_options.dependency_constraints.get_for_python_version(
version=config.version,
tmp_dir=identifier_tmp_dir,
)
# install Python
base_python, env = setup_python(
identifier_tmp_dir / "build",
config,
constraints_path,
build_options.environment,
build_frontend.name,
)
env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
pip_version = None if use_uv else get_pip_version(env)
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
if compatible_wheel:
log.step_end()
print(
f"\nFound previously built wheel {compatible_wheel.name}, that's compatible with {config.identifier}. Skipping build step..."
)
repaired_wheel = compatible_wheel
else:
# run the before_build command
if build_options.before_build:
log.step("Running before_build...")
before_build_prepared = prepare_command(
build_options.before_build,
project=".",
package=options.globals.package_dir,
)
shell(before_build_prepared, env=env)
log.step("Building wheel...")
built_wheel_dir.mkdir()
extra_flags = get_build_frontend_extra_flags(
build_frontend,
build_options.build_verbosity,
prepare_config_settings(
build_options.config_settings,
project=".",
package=options.globals.package_dir,
),
)
match build_frontend.name:
case "pip":
# Path.resolve() is needed. Without it pip wheel may try to fetch package from pypi.org
# see https://github.com/pypa/cibuildwheel/pull/369
call(
"python",
"-m",
"pip",
"wheel",
options.globals.package_dir.resolve(),
f"--wheel-dir={built_wheel_dir}",
"--no-deps",
*extra_flags,
env=env,
)
case "build" | "build[uv]":
if (
use_uv
and "--no-isolation" not in extra_flags
and "-n" not in extra_flags
):
extra_flags.append("--installer=uv")
call(
"python",
"-m",
"build",
build_options.package_dir,
"--wheel",
f"--outdir={built_wheel_dir}",
*extra_flags,
env=env,
)
case "uv":
assert uv_path is not None
call(
uv_path,
"build",
f"--python={base_python}",
build_options.package_dir,
"--wheel",
f"--out-dir={built_wheel_dir}",
*extra_flags,
env=env,
)
case _:
assert_never(build_frontend)
try:
built_wheel = next(built_wheel_dir.glob("*.whl"))
except StopIteration:
raise errors.BuildProducedNoWheelError() from None
# repair the wheel
repaired_wheel_dir.mkdir()
if built_wheel.name.endswith("none-any.whl"):
raise errors.NonPlatformWheelError()
if build_options.repair_command:
log.step("Repairing wheel...")
repair_command_prepared = prepare_command(
build_options.repair_command,
wheel=built_wheel,
dest_dir=repaired_wheel_dir,
package=build_options.package_dir,
project=".",
)
shell(repair_command_prepared, env=env)
else:
shutil.move(str(built_wheel), repaired_wheel_dir)
try:
repaired_wheel = next(repaired_wheel_dir.glob("*.whl"))
except StopIteration:
raise errors.RepairStepProducedNoWheelError() from None
if repaired_wheel.name in {wheel.name for wheel in built_wheels}:
raise errors.AlreadyBuiltWheelError(repaired_wheel.name)
run_audit(tmp_dir=tmp_path, build_options=build_options, wheel=repaired_wheel)
test_selected = options.globals.test_selector(config.identifier)
if test_selected and config.arch == "ARM64" != platform_module.machine():
log.warning(
unwrap(
"""
While arm64 wheels can be built on other platforms, they cannot
be tested. An arm64 runner is required. To silence this warning,
set `CIBW_TEST_SKIP: "*-win_arm64"`.
"""
)
)
# skip this test
elif test_selected and build_options.test_command:
log.step("Testing wheel...")
# set up a virtual environment to install and test from, to make sure
# there are no dependencies that were pulled in at build time.
venv_dir = identifier_tmp_dir / "venv-test"
virtualenv_env = virtualenv(
config.version,
base_python,
venv_dir,
None,
use_uv=use_uv,
env=env,
pip_version=pip_version,
)
virtualenv_env = build_options.test_environment.as_dictionary(
prev_environment=virtualenv_env
)
# check that we are using the Python from the virtual environment
call("where", "python", env=virtualenv_env)
if build_options.before_test:
before_test_prepared = prepare_command(
build_options.before_test,
project=".",
package=build_options.package_dir,
)
shell(before_test_prepared, env=virtualenv_env)
pip: Sequence[Path | str]
if use_uv:
assert uv_path is not None
pip = [uv_path, "pip"]
else:
pip = ["pip"]
# install the wheel
call(
*pip,
"install",
str(repaired_wheel) + build_options.test_extras,
env=virtualenv_env,
)
# test the wheel
if build_options.test_requires:
call(*pip, "install", *build_options.test_requires, env=virtualenv_env)
# run the tests from a temp dir, with an absolute path in the command
# (this ensures that Python runs the tests against the installed wheel
# and not the repo code)
test_cwd = identifier_tmp_dir / "test_cwd"
test_cwd.mkdir()
if build_options.test_sources:
copy_test_sources(
build_options.test_sources,
Path.cwd(),
test_cwd,
)
else:
# Use the test_fail.py file to raise a nice error if the user
# tries to run tests in the cwd
(test_cwd / "test_fail.py").write_text(resources.TEST_FAIL_CWD_FILE.read_text())
test_command_prepared = prepare_command(
build_options.test_command,
project=Path.cwd(),
package=options.globals.package_dir.resolve(),
wheel=repaired_wheel,
)
shell(test_command_prepared, cwd=test_cwd, env=virtualenv_env)
# we're all done here; move it to output (remove if already exists)
output_wheel = None
if compatible_wheel is None:
output_wheel = build_options.output_dir.joinpath(repaired_wheel.name)
moved_wheel = move_file(repaired_wheel, output_wheel)
if moved_wheel != output_wheel.resolve():
log.warning(
f"{repaired_wheel} was moved to {moved_wheel} instead of {output_wheel}"
)
built_wheels.append(output_wheel)
# clean up
# (we ignore errors because occasionally Windows fails to unlink a file and we
# don't want to abort a build because of that)
shutil.rmtree(identifier_tmp_dir, ignore_errors=True)
log.build_end(output_wheel)
except subprocess.CalledProcessError as error:
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
raise errors.FatalError(msg) from error
+46 -84
View File
@@ -1,116 +1,78 @@
from __future__ import annotations
__lazy_modules__ = {"configparser", "contextlib", "dependency_groups"}
import ast
import configparser
import contextlib
import sys
from configparser import ConfigParser
from pathlib import Path
from typing import Any, Optional
import dependency_groups
import toml
TYPE_CHECKING = False
if TYPE_CHECKING:
from pathlib import Path
from typing import Any
if sys.version_info < (3, 8):
Constant = ast.Str
def get_constant(x: ast.Str) -> str:
return x.s
def get_parent(node: ast.AST | None, depth: int = 1) -> ast.AST | None:
for _ in range(depth):
node = getattr(node, "parent", None)
return node
else:
Constant = ast.Constant
def is_main(parent: ast.AST | None) -> bool:
match parent:
case ast.If(test=ast.Compare(left=left, ops=[ast.Eq()], comparators=[comp])):
values = {left, comp}
mains = {x for x in values if isinstance(x, ast.Constant) and x.value == "__main__"}
if len(mains) != 1:
return False
consts = {x for x in values if isinstance(x, ast.Name) and x.id == "__name__"}
return len(consts) == 1
case _:
return False
def get_constant(x: ast.Constant) -> Any:
return x.value
class Analyzer(ast.NodeVisitor):
def __init__(self) -> None:
self.requires_python: str | None = None
self.requires_python: Optional[str] = None
def visit(self, node: ast.AST) -> None:
for inner_node in ast.walk(node):
for child in ast.iter_child_nodes(inner_node):
child.parent = inner_node # type: ignore[attr-defined]
super().visit(node)
def visit(self, content: ast.AST) -> None:
for node in ast.walk(content):
for child in ast.iter_child_nodes(node):
child.parent = node # type: ignore
super().visit(content)
def visit_keyword(self, node: ast.keyword) -> None:
# Must not be nested except for if __name__ == "__main__"
self.generic_visit(node)
# This will be Module -> Expr -> Call -> keyword
parent = get_parent(node, 4)
unnested = parent is None
# This will be Module -> If -> Expr -> Call -> keyword
name_main_unnested = (
parent is not None and get_parent(parent) is None and is_main(get_parent(node, 3))
)
match node:
case ast.keyword(arg="python_requires", value=ast.Constant(value=str() as version)):
if unnested or name_main_unnested:
self.requires_python = version
if node.arg == "python_requires":
# Must not be nested in an if or other structure
# This will be Module -> Expr -> Call -> keyword
if not hasattr(node.parent.parent.parent, "parent") and isinstance( # type: ignore
node.value, Constant
):
self.requires_python = get_constant(node.value)
def setup_py_python_requires(content: str) -> str | None:
def setup_py_python_requires(content: str) -> Optional[str]:
try:
tree = ast.parse(content)
analyzer = Analyzer()
analyzer.visit(tree)
return analyzer.requires_python or None
except Exception: # pylint: disable=broad-except
except Exception:
return None
def get_requires_python_str(package_dir: Path, pyproject_toml: dict[str, Any] | None) -> str | None:
"""Return the python requires string from the most canonical source available, or None"""
def get_requires_python_str(package_dir: Path) -> Optional[str]:
"Return the python requires string from the most canonical source available, or None"
# Read in from pyproject.toml:project.requires-python
with contextlib.suppress(KeyError, IndexError, TypeError):
return str((pyproject_toml or {})["project"]["requires-python"])
try:
info = toml.load(package_dir / "pyproject.toml")
return str(info["project"]["requires-python"])
except (FileNotFoundError, KeyError, IndexError, TypeError):
pass
# Read in from setup.cfg:options.python_requires
config = configparser.ConfigParser()
with contextlib.suppress(FileNotFoundError):
try:
config = ConfigParser()
config.read(package_dir / "setup.cfg")
with contextlib.suppress(KeyError, IndexError, TypeError):
return str(config["options"]["python_requires"])
setup_py = package_dir / "setup.py"
with contextlib.suppress(FileNotFoundError), setup_py.open(encoding="utf8") as f2:
return setup_py_python_requires(f2.read())
return None
def resolve_dependency_groups(
pyproject_toml: dict[str, Any] | None, *groups: str
) -> tuple[str, ...]:
"""
Get the packages in dependency-groups for a package.
"""
if not groups:
return ()
if pyproject_toml is None:
msg = f"Didn't find a pyproject.toml, so can't read [dependency-groups] {groups!r} from it!"
raise FileNotFoundError(msg)
return str(config["options"]["python_requires"])
except (FileNotFoundError, KeyError, IndexError, TypeError):
pass
try:
dependency_groups_toml = pyproject_toml["dependency-groups"]
except KeyError:
msg = f"Didn't find [dependency-groups] in pyproject.toml, which is needed to resolve {groups!r}."
raise KeyError(msg) from None
with open(package_dir / "setup.py") as f:
return setup_py_python_requires(f.read())
except FileNotFoundError:
pass
return dependency_groups.resolve(dependency_groups_toml, *groups)
return None
@@ -1 +0,0 @@
import _cross_venv; _cross_venv.initialize()
@@ -1,90 +0,0 @@
# This module is copied into the site-packages directory of an Android build environment, and
# activated via a .pth file when we want the environment to simulate Android.
import os
import platform
import re
import sys
import sysconfig
from pathlib import Path
from typing import Any
def initialize() -> None:
if not (host_triplet := os.environ.get("CIBW_HOST_TRIPLET")):
return
# Pre-import any modules which would fail to import after the monkey-patching.
import ctypes # noqa: F401, PLC0415 - uses get_config_var("LDLIBRARY")
# os ######################################################################
def cross_os_uname() -> os.uname_result:
return os.uname_result(
(
"Linux",
"localhost",
# The Linux kernel version and release are unlikely to be significant, but return
# realistic values anyway (from an API level 24 emulator).
"3.18.91+",
"#1 SMP PREEMPT Tue Jan 9 20:35:43 UTC 2018",
host_triplet.split("-")[0],
)
)
os.name = "posix"
os.uname = cross_os_uname
# platform ################################################################
#
# We can't determine the user-visible Android version number from the API level, so return a
# string which will work fine for display, but will fail to parse as a version number.
def cross_android_ver(*args: Any, **kwargs: Any) -> platform.AndroidVer:
return platform.AndroidVer(
release=f"API level {cross_getandroidapilevel()}",
api_level=cross_getandroidapilevel(),
manufacturer="Google",
model="sdk_gphone64",
device="emu64",
is_emulator=True,
)
# platform.uname, platform.system etc. are all implemented in terms of platform.android_ver.
platform.android_ver = cross_android_ver
# sys #####################################################################
def cross_getandroidapilevel() -> int:
api_level = sysconfig.get_config_var("ANDROID_API_LEVEL")
assert isinstance(api_level, int)
return api_level
# Some packages may recognize sys.cross_compiling from the crossenv tool.
sys.cross_compiling = True # type: ignore[attr-defined]
sys.getandroidapilevel = cross_getandroidapilevel # type: ignore[attr-defined]
sys.implementation._multiarch = host_triplet # type: ignore[attr-defined]
sys.platform = "android"
# Determine the abiflags from the sysconfigdata filename.
sysconfigdata_path = next(Path(__file__).parent.glob("_sysconfigdata_*.py"))
abiflags_match = re.match(r"_sysconfigdata_(.*?)_", sysconfigdata_path.name)
assert abiflags_match is not None
sys.abiflags = abiflags_match[1]
# sysconfig ###############################################################
#
# Load the sysconfigdata file, generating its name from sys.abiflags,
# sys.platform, and sys.implementation._multiarch.
sysconfig._init_config_vars() # type: ignore[attr-defined]
# We don't change the actual sys.base_prefix and base_exec_prefix, because that
# could have unpredictable effects. Instead, we change the sysconfig variables
# used by sysconfig.get_paths().
vars = sysconfig.get_config_vars()
try:
host_prefix = vars["host_prefix"] # This variable was added in Python 3.14.
except KeyError:
host_prefix = vars["exec_prefix"]
vars["installed_base"] = vars["installed_platbase"] = host_prefix
# sysconfig.get_platform, which determines the wheel tag, is implemented in terms of
# sys.platform, sysconfig.get_config_var("ANDROID_API_LEVEL") (see localized_vars in
# android.py), and os.uname.
@@ -1,11 +0,0 @@
--- a/android.py
+++ b/android.py
@@ -1021,7 +1021,7 @@ def main():
context = parse_args()
# Set the CROSS_BUILD_DIR if an argument was provided
- if context.cross_build_dir:
+ if getattr(context, "cross_build_dir", None):
global CROSS_BUILD_DIR
CROSS_BUILD_DIR = context.cross_build_dir.resolve()
@@ -1,121 +0,0 @@
# This file intentionally has no hashbang line in the source: cibuildwheel will add it
# above this comment when the file is deployed.
import os
import re
import shutil
import sys
from pathlib import Path
from tempfile import gettempdir
from filelock import FileLock
from cibuildwheel.util.file import CIBW_CACHE_PATH, download
# In the future we might pick a different Flang release depending on the NDK version,
# but so far all Python versions use the same NDK version, so there's no need.
RELEASE_URL = "https://github.com/termux/ndk-toolchain-clang-with-flang/releases/download"
RELEASE_VERSION = "r27c"
ARCHS = ["aarch64", "x86_64"]
# The compiler is built for Linux x86_64, so we use Docker on macOS.
DOCKER_IMAGE = "debian:trixie-slim"
def main() -> None:
cache_dir = CIBW_CACHE_PATH / f"flang-android-{RELEASE_VERSION}"
with FileLock(f"{cache_dir}.lock"):
if not cache_dir.exists():
download_flang(cache_dir)
run_flang(cache_dir)
def download_flang(cache_dir: Path) -> None:
tmp_dir = Path(f"{cache_dir}.tmp")
if tmp_dir.exists():
shutil.rmtree(tmp_dir)
tmp_dir.mkdir(parents=True)
for archive_name in [f"package-flang-{arch}.tar.bz2" for arch in ARCHS] + [
"package-flang-host.tar.bz2",
"package-install.tar.bz2",
]:
archive_path = tmp_dir / archive_name
download(f"{RELEASE_URL}/{RELEASE_VERSION}/{archive_name}", archive_path)
shutil.unpack_archive(archive_path, tmp_dir)
archive_path.unlink()
# Merge the extracted trees together, along with the necessary parts of the NDK. Based on
# https://github.com/kivy/python-for-android/blob/develop/pythonforandroid/recipes/fortran/__init__.py.
flang_toolchain = tmp_dir / "toolchain"
(tmp_dir / "out/install/linux-x86/clang-dev").rename(flang_toolchain)
ndk_toolchain = Path(os.environ["CC"]).parents[1]
if (clang_ver_flang := clang_ver(flang_toolchain)) != (
clang_ver_ndk := clang_ver(ndk_toolchain)
):
msg = f"Flang uses Clang {clang_ver_flang}, but NDK uses Clang {clang_ver_ndk}"
raise ValueError(msg)
clang_lib_path = f"lib/clang/{clang_ver_ndk}/lib"
shutil.rmtree(flang_toolchain / clang_lib_path)
for src, dst in [
(f"{tmp_dir}/build-{arch}-install", f"sysroot/usr/lib/{arch}-linux-android")
for arch in ARCHS
] + [
(f"{tmp_dir}/build-host-install", ""),
(f"{ndk_toolchain}/{clang_lib_path}", clang_lib_path),
(f"{ndk_toolchain}/sysroot", "sysroot"),
]:
shutil.copytree(src, flang_toolchain / dst, symlinks=True, dirs_exist_ok=True)
flang_toolchain.rename(cache_dir)
shutil.rmtree(tmp_dir)
def clang_ver(toolchain: Path) -> str:
versions = [p.name for p in (toolchain / "lib/clang").iterdir()]
assert len(versions) == 1
return versions[0]
def run_flang(cache_dir: Path) -> None:
match = re.fullmatch(r".+/(.+)-clang", os.environ["CC"])
assert match is not None
target = match[1]
# In a future Flang version the executable name will change to "flang"
# (https://blog.llvm.org/posts/2025-03-11-flang-new/).
flang_args = [f"{cache_dir}/bin/flang-new", f"--target={target}", *sys.argv[1:]]
if sys.platform == "linux":
args = flang_args
elif sys.platform == "darwin":
args = ["docker", "run", "--rm", "--platform", "linux/amd64"]
# Docker on macOS only allows certain directories to be mounted as volumes
# by default, but they include all the locations we're likely to need.
for path in [
# Location of CIBW_CACHE_PATH (containing the Fortran compiler), and
# probably the project directory.
Path.home(),
# Temporary directories created by build backends and cibuildwheel itself.
# This is controlled by the TMPDIR environment variable, which is set to
# an unpredictable subdirectory of /var/folders. /var is itself a symlink
# to /private/var.
Path(gettempdir()),
Path(gettempdir()).resolve(),
]:
args += ["-v", f"{path}:{path}"]
args += ["--workdir", str(Path.cwd())]
args += ["--entrypoint", flang_args[0], DOCKER_IMAGE, *flang_args[1:]]
else:
msg = f"unknown platform: {sys.platform}"
raise ValueError(msg)
os.execvp(args[0], args)
if __name__ == "__main__":
main()
@@ -1,57 +0,0 @@
#!/usr/bin/env python3
import os
import shutil
import subprocess
import sys
from pathlib import Path
def main() -> None:
# CIBW_HOST_TRIPLET is set in the android_env to the Android target triplet.
target = os.environ.get("CIBW_HOST_TRIPLET")
cmd_name = Path(sys.argv[0]).name
# Find the real command in PATH, excluding the current script's directory
path_env = os.environ.get("PATH", "")
script_dir = Path(__file__).resolve().parent
paths = path_env.split(os.pathsep)
# Filter out the script directory to avoid recursion
filtered_paths = [p for p in paths if Path(p).resolve() != script_dir]
filtered_path_env = os.pathsep.join(filtered_paths)
real_cmd = shutil.which(cmd_name, path=filtered_path_env)
if not real_cmd:
sys.stderr.write(f"cibuildwheel: Error: Could not find system {cmd_name}\n")
sys.exit(1)
# If we have a target (i.e. we are in the android_env), try to install it.
if target:
# Check if rustup is available to install the target
rustup_path = shutil.which("rustup", path=filtered_path_env)
if rustup_path:
try:
# We call rustup to ensure the target is installed.
subprocess.run(
[rustup_path, "target", "add", target],
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
text=True,
)
except subprocess.CalledProcessError as e:
sys.stderr.write(
f"cibuildwheel: Error: Failed to install Rust target {target}: {e.stderr}\n"
)
sys.exit(1)
# Execute the real command
os.execv(real_cmd, [real_cmd, *sys.argv[1:]])
if __name__ == "__main__":
main()
+55 -228
View File
@@ -1,244 +1,71 @@
[linux]
python_configurations = [
{ identifier = "cp27-manylinux_x86_64", version = "2.7", path_str = "/opt/python/cp27-cp27m" },
{ identifier = "cp27-manylinux_x86_64", version = "2.7", path_str = "/opt/python/cp27-cp27mu" },
{ identifier = "cp35-manylinux_x86_64", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
{ identifier = "cp36-manylinux_x86_64", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
{ identifier = "cp37-manylinux_x86_64", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
{ identifier = "cp38-manylinux_x86_64", version = "3.8", path_str = "/opt/python/cp38-cp38" },
{ identifier = "cp39-manylinux_x86_64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-manylinux_x86_64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-manylinux_x86_64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-manylinux_x86_64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-manylinux_x86_64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-manylinux_x86_64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-manylinux_x86_64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-manylinux_x86_64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-manylinux_x86_64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "cp27-manylinux_i686", version = "2.7", path_str = "/opt/python/cp27-cp27m" },
{ identifier = "cp27-manylinux_i686", version = "2.7", path_str = "/opt/python/cp27-cp27mu" },
{ identifier = "cp35-manylinux_i686", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
{ identifier = "cp36-manylinux_i686", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
{ identifier = "cp37-manylinux_i686", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
{ identifier = "cp38-manylinux_i686", version = "3.8", path_str = "/opt/python/cp38-cp38" },
{ identifier = "cp39-manylinux_i686", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-manylinux_i686", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-manylinux_i686", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-manylinux_i686", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-manylinux_i686", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-manylinux_i686", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-manylinux_i686", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-manylinux_i686", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-manylinux_i686", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "pp39-manylinux_x86_64", version = "3.9", path_str = "/opt/python/pp39-pypy39_pp73" },
{ identifier = "pp310-manylinux_x86_64", version = "3.10", path_str = "/opt/python/pp310-pypy310_pp73" },
{ identifier = "pp311-manylinux_x86_64", version = "3.11", path_str = "/opt/python/pp311-pypy311_pp73" },
{ identifier = "gp312_250-manylinux_x86_64", version = "3.12", path_str = "/opt/python/graalpy312-graalpy250_312_native" },
{ identifier = "pp27-manylinux_x86_64", version = "2.7", path_str = "/opt/python/pp27-pypy_73" },
{ identifier = "pp36-manylinux_x86_64", version = "3.6", path_str = "/opt/python/pp36-pypy36_pp73" },
{ identifier = "pp37-manylinux_x86_64", version = "3.7", path_str = "/opt/python/pp37-pypy37_pp73" },
{ identifier = "cp35-manylinux_aarch64", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
{ identifier = "cp36-manylinux_aarch64", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
{ identifier = "cp37-manylinux_aarch64", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
{ identifier = "cp38-manylinux_aarch64", version = "3.8", path_str = "/opt/python/cp38-cp38" },
{ identifier = "cp39-manylinux_aarch64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-manylinux_aarch64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-manylinux_aarch64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-manylinux_aarch64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-manylinux_aarch64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-manylinux_aarch64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-manylinux_aarch64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-manylinux_aarch64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-manylinux_aarch64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "cp35-manylinux_ppc64le", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
{ identifier = "cp36-manylinux_ppc64le", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
{ identifier = "cp37-manylinux_ppc64le", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
{ identifier = "cp38-manylinux_ppc64le", version = "3.8", path_str = "/opt/python/cp38-cp38" },
{ identifier = "cp39-manylinux_ppc64le", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-manylinux_ppc64le", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-manylinux_ppc64le", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-manylinux_ppc64le", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-manylinux_ppc64le", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-manylinux_ppc64le", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-manylinux_ppc64le", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-manylinux_ppc64le", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-manylinux_ppc64le", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "cp35-manylinux_s390x", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
{ identifier = "cp36-manylinux_s390x", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
{ identifier = "cp37-manylinux_s390x", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
{ identifier = "cp38-manylinux_s390x", version = "3.8", path_str = "/opt/python/cp38-cp38" },
{ identifier = "cp39-manylinux_s390x", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-manylinux_s390x", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-manylinux_s390x", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-manylinux_s390x", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-manylinux_s390x", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-manylinux_s390x", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-manylinux_s390x", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-manylinux_s390x", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-manylinux_s390x", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "cp39-manylinux_armv7l", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-manylinux_armv7l", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-manylinux_armv7l", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-manylinux_armv7l", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-manylinux_armv7l", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-manylinux_armv7l", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-manylinux_armv7l", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-manylinux_armv7l", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-manylinux_armv7l", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "cp39-manylinux_riscv64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-manylinux_riscv64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-manylinux_riscv64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-manylinux_riscv64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-manylinux_riscv64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-manylinux_riscv64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-manylinux_riscv64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-manylinux_riscv64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-manylinux_riscv64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "pp39-manylinux_aarch64", version = "3.9", path_str = "/opt/python/pp39-pypy39_pp73" },
{ identifier = "pp310-manylinux_aarch64", version = "3.10", path_str = "/opt/python/pp310-pypy310_pp73" },
{ identifier = "pp311-manylinux_aarch64", version = "3.11", path_str = "/opt/python/pp311-pypy311_pp73" },
{ identifier = "gp312_250-manylinux_aarch64", version = "3.12", path_str = "/opt/python/graalpy312-graalpy250_312_native" },
{ identifier = "pp39-manylinux_i686", version = "3.9", path_str = "/opt/python/pp39-pypy39_pp73" },
{ identifier = "pp310-manylinux_i686", version = "3.10", path_str = "/opt/python/pp310-pypy310_pp73" },
{ identifier = "pp311-manylinux_i686", version = "3.11", path_str = "/opt/python/pp311-pypy311_pp73" },
{ identifier = "cp39-musllinux_x86_64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-musllinux_x86_64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-musllinux_x86_64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-musllinux_x86_64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-musllinux_x86_64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-musllinux_x86_64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-musllinux_x86_64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-musllinux_x86_64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-musllinux_x86_64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "cp39-musllinux_i686", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-musllinux_i686", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-musllinux_i686", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-musllinux_i686", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-musllinux_i686", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-musllinux_i686", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-musllinux_i686", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-musllinux_i686", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-musllinux_i686", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "cp39-musllinux_aarch64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-musllinux_aarch64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-musllinux_aarch64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-musllinux_aarch64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-musllinux_aarch64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-musllinux_aarch64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-musllinux_aarch64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-musllinux_aarch64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-musllinux_aarch64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "cp39-musllinux_ppc64le", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-musllinux_ppc64le", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-musllinux_ppc64le", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-musllinux_ppc64le", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-musllinux_ppc64le", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-musllinux_ppc64le", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-musllinux_ppc64le", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-musllinux_ppc64le", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-musllinux_ppc64le", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "cp39-musllinux_s390x", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-musllinux_s390x", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-musllinux_s390x", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-musllinux_s390x", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-musllinux_s390x", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-musllinux_s390x", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-musllinux_s390x", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-musllinux_s390x", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-musllinux_s390x", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "cp39-musllinux_armv7l", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-musllinux_armv7l", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-musllinux_armv7l", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-musllinux_armv7l", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-musllinux_armv7l", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-musllinux_armv7l", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-musllinux_armv7l", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-musllinux_armv7l", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-musllinux_armv7l", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
{ identifier = "cp39-musllinux_riscv64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
{ identifier = "cp310-musllinux_riscv64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
{ identifier = "cp311-musllinux_riscv64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
{ identifier = "cp312-musllinux_riscv64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
{ identifier = "cp313-musllinux_riscv64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
{ identifier = "cp314-musllinux_riscv64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
{ identifier = "cp314t-musllinux_riscv64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
{ identifier = "cp315-musllinux_riscv64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
{ identifier = "cp315t-musllinux_riscv64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
]
[macos]
python_configurations = [
{ identifier = "cp39-macosx_x86_64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.13/python-3.9.13-macos11.pkg", sha256 = "351fe18f4fb03be7afac5e4012fc0a51345f43202af43ef620cf1eee5ee36578" },
{ identifier = "cp39-macosx_arm64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.13/python-3.9.13-macos11.pkg", sha256 = "351fe18f4fb03be7afac5e4012fc0a51345f43202af43ef620cf1eee5ee36578" },
{ identifier = "cp39-macosx_universal2", version = "3.9", url = "https://www.python.org/ftp/python/3.9.13/python-3.9.13-macos11.pkg", sha256 = "351fe18f4fb03be7afac5e4012fc0a51345f43202af43ef620cf1eee5ee36578" },
{ identifier = "cp310-macosx_x86_64", version = "3.10", url = "https://www.python.org/ftp/python/3.10.11/python-3.10.11-macos11.pkg", sha256 = "767ed35ad688d28ea4494081ae96408a0318d0d5bb9ca0139d74d6247b231cfc" },
{ identifier = "cp310-macosx_arm64", version = "3.10", url = "https://www.python.org/ftp/python/3.10.11/python-3.10.11-macos11.pkg", sha256 = "767ed35ad688d28ea4494081ae96408a0318d0d5bb9ca0139d74d6247b231cfc" },
{ identifier = "cp310-macosx_universal2", version = "3.10", url = "https://www.python.org/ftp/python/3.10.11/python-3.10.11-macos11.pkg", sha256 = "767ed35ad688d28ea4494081ae96408a0318d0d5bb9ca0139d74d6247b231cfc" },
{ identifier = "cp311-macosx_x86_64", version = "3.11", url = "https://www.python.org/ftp/python/3.11.9/python-3.11.9-macos11.pkg", sha256 = "b6cfdee2571ca56ee895043ca1e7110fb78a878cee3eb0c21accb2de34d24b55" },
{ identifier = "cp311-macosx_arm64", version = "3.11", url = "https://www.python.org/ftp/python/3.11.9/python-3.11.9-macos11.pkg", sha256 = "b6cfdee2571ca56ee895043ca1e7110fb78a878cee3eb0c21accb2de34d24b55" },
{ identifier = "cp311-macosx_universal2", version = "3.11", url = "https://www.python.org/ftp/python/3.11.9/python-3.11.9-macos11.pkg", sha256 = "b6cfdee2571ca56ee895043ca1e7110fb78a878cee3eb0c21accb2de34d24b55" },
{ identifier = "cp312-macosx_x86_64", version = "3.12", url = "https://www.python.org/ftp/python/3.12.10/python-3.12.10-macos11.pkg", sha256 = "8373e58da4ea146b3eb1c1f9834f19a319440b6b679b06050b1f9ee3237aa8e4" },
{ identifier = "cp312-macosx_arm64", version = "3.12", url = "https://www.python.org/ftp/python/3.12.10/python-3.12.10-macos11.pkg", sha256 = "8373e58da4ea146b3eb1c1f9834f19a319440b6b679b06050b1f9ee3237aa8e4" },
{ identifier = "cp312-macosx_universal2", version = "3.12", url = "https://www.python.org/ftp/python/3.12.10/python-3.12.10-macos11.pkg", sha256 = "8373e58da4ea146b3eb1c1f9834f19a319440b6b679b06050b1f9ee3237aa8e4" },
{ identifier = "cp313-macosx_x86_64", version = "3.13", url = "https://www.python.org/ftp/python/3.13.13/python-3.13.13-macos11.pkg", sha256 = "a909cb655af5db67d5a90b3603437a1d58bec3446d624e4034e278ac62023cc9" },
{ identifier = "cp313-macosx_arm64", version = "3.13", url = "https://www.python.org/ftp/python/3.13.13/python-3.13.13-macos11.pkg", sha256 = "a909cb655af5db67d5a90b3603437a1d58bec3446d624e4034e278ac62023cc9" },
{ identifier = "cp313-macosx_universal2", version = "3.13", url = "https://www.python.org/ftp/python/3.13.13/python-3.13.13-macos11.pkg", sha256 = "a909cb655af5db67d5a90b3603437a1d58bec3446d624e4034e278ac62023cc9" },
{ identifier = "cp314-macosx_x86_64", version = "3.14", url = "https://www.python.org/ftp/python/3.14.5/python-3.14.5-macos11.pkg", sha256 = "b28a8dc33c456dd06c97024697d63ca916cfb494594c06fa3e4ef4d41fa82335" },
{ identifier = "cp314-macosx_arm64", version = "3.14", url = "https://www.python.org/ftp/python/3.14.5/python-3.14.5-macos11.pkg", sha256 = "b28a8dc33c456dd06c97024697d63ca916cfb494594c06fa3e4ef4d41fa82335" },
{ identifier = "cp314-macosx_universal2", version = "3.14", url = "https://www.python.org/ftp/python/3.14.5/python-3.14.5-macos11.pkg", sha256 = "b28a8dc33c456dd06c97024697d63ca916cfb494594c06fa3e4ef4d41fa82335" },
{ identifier = "cp314t-macosx_x86_64", version = "3.14", url = "https://www.python.org/ftp/python/3.14.5/python-3.14.5-macos11.pkg", sha256 = "b28a8dc33c456dd06c97024697d63ca916cfb494594c06fa3e4ef4d41fa82335" },
{ identifier = "cp314t-macosx_arm64", version = "3.14", url = "https://www.python.org/ftp/python/3.14.5/python-3.14.5-macos11.pkg", sha256 = "b28a8dc33c456dd06c97024697d63ca916cfb494594c06fa3e4ef4d41fa82335" },
{ identifier = "cp314t-macosx_universal2", version = "3.14", url = "https://www.python.org/ftp/python/3.14.5/python-3.14.5-macos11.pkg", sha256 = "b28a8dc33c456dd06c97024697d63ca916cfb494594c06fa3e4ef4d41fa82335" },
{ identifier = "cp315-macosx_x86_64", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-macos11.pkg", sha256 = "29b5c95f255f846f77e1c119146836445f929de1392a3ae9ef24f559e1345418" },
{ identifier = "cp315-macosx_arm64", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-macos11.pkg", sha256 = "29b5c95f255f846f77e1c119146836445f929de1392a3ae9ef24f559e1345418" },
{ identifier = "cp315-macosx_universal2", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-macos11.pkg", sha256 = "29b5c95f255f846f77e1c119146836445f929de1392a3ae9ef24f559e1345418" },
{ identifier = "cp315t-macosx_x86_64", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-macos11.pkg", sha256 = "29b5c95f255f846f77e1c119146836445f929de1392a3ae9ef24f559e1345418" },
{ identifier = "cp315t-macosx_arm64", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-macos11.pkg", sha256 = "29b5c95f255f846f77e1c119146836445f929de1392a3ae9ef24f559e1345418" },
{ identifier = "cp315t-macosx_universal2", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-macos11.pkg", sha256 = "29b5c95f255f846f77e1c119146836445f929de1392a3ae9ef24f559e1345418" },
{ identifier = "pp39-macosx_x86_64", version = "3.9", url = "https://downloads.python.org/pypy/pypy3.9-v7.3.16-macos_x86_64.tar.bz2", sha256 = "fda015431621e7e5aa16359d114f2c45a77ed936992c1efff86302e768a6b21c" },
{ identifier = "pp39-macosx_arm64", version = "3.9", url = "https://downloads.python.org/pypy/pypy3.9-v7.3.16-macos_arm64.tar.bz2", sha256 = "88f824e7a2d676440d09bc90fc959ae0fd3557d7e2f14bfbbe53d41d159a47fe" },
{ identifier = "pp310-macosx_x86_64", version = "3.10", url = "https://downloads.python.org/pypy/pypy3.10-v7.3.19-macos_x86_64.tar.bz2", sha256 = "2c5e5c2a33ac882551d7f28b98d19d486b8995aa50824a73b4edcc6aeec35c63" },
{ identifier = "pp310-macosx_arm64", version = "3.10", url = "https://downloads.python.org/pypy/pypy3.10-v7.3.19-macos_arm64.tar.bz2", sha256 = "3db8a03fc496164801646844d7f3b12baa0adb3ad9a0b7cb124521bc2e168a36" },
{ identifier = "pp311-macosx_x86_64", version = "3.11", url = "https://downloads.python.org/pypy/pypy3.11-v7.3.23-macos_x86_64.tar.bz2", sha256 = "c95363c4e87235d11a6cec8128239c291b1eb67a752778fbcfe029a71da82b5e" },
{ identifier = "pp311-macosx_arm64", version = "3.11", url = "https://downloads.python.org/pypy/pypy3.11-v7.3.23-macos_arm64.tar.bz2", sha256 = "4747b3aceba4c1c6104cddc0fe5ea302101d32955f0957347b9ecc4fbd7aed05" },
{ identifier = "gp312_250-macosx_x86_64", version = "3.12", url = "https://github.com/oracle/graalpython/releases/download/graal-25.0.1/graalpy-25.0.1-macos-amd64.tar.gz", sha256 = "10b0721d52397f0cc85f038900318da2203711cbcfae7899e3faed49d3dc6221" },
{ identifier = "gp312_250-macosx_arm64", version = "3.12", url = "https://github.com/oracle/graalpython/releases/download/graal-25.0.3/graalpy-25.0.3-macos-aarch64.tar.gz", sha256 = "387d9f5b376860842bea4d55aae5820974f3f3b68fc36c77ae863a061a888857" },
{ identifier = "cp27-macosx_x86_64", version = "2.7", url = "https://www.python.org/ftp/python/2.7.18/python-2.7.18-macosx10.9.pkg" },
{ identifier = "cp35-macosx_x86_64", version = "3.5", url = "https://www.python.org/ftp/python/3.5.4/python-3.5.4-macosx10.6.pkg" },
{ identifier = "cp36-macosx_x86_64", version = "3.6", url = "https://www.python.org/ftp/python/3.6.8/python-3.6.8-macosx10.9.pkg" },
{ identifier = "cp37-macosx_x86_64", version = "3.7", url = "https://www.python.org/ftp/python/3.7.9/python-3.7.9-macosx10.9.pkg" },
{ identifier = "cp38-macosx_x86_64", version = "3.8", url = "https://www.python.org/ftp/python/3.8.10/python-3.8.10-macosx10.9.pkg" },
{ identifier = "cp38-macosx_arm64", version = "3.8", url = "https://www.python.org/ftp/python/3.8.10/python-3.8.10-macosx10.9.pkg" },
{ identifier = "cp38-macosx_universal2", version = "3.8", url = "https://www.python.org/ftp/python/3.8.10/python-3.8.10-macosx10.9.pkg" },
{ identifier = "cp39-macosx_x86_64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.5/python-3.9.5-macos11.pkg" },
{ identifier = "cp39-macosx_arm64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.5/python-3.9.5-macos11.pkg" },
{ identifier = "cp39-macosx_universal2", version = "3.9", url = "https://www.python.org/ftp/python/3.9.5/python-3.9.5-macos11.pkg" },
{ identifier = "pp27-macosx_x86_64", version = "2.7", url = "https://downloads.python.org/pypy/pypy2.7-v7.3.3-osx64.tar.bz2" },
{ identifier = "pp36-macosx_x86_64", version = "3.6", url = "https://downloads.python.org/pypy/pypy3.6-v7.3.3-osx64.tar.bz2" },
{ identifier = "pp37-macosx_x86_64", version = "3.7", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.3-osx64.tar.bz2" },
]
[windows]
python_configurations = [
{ identifier = "cp39-win32", version = "3.9.13" },
{ identifier = "cp39-win_amd64", version = "3.9.13" },
{ identifier = "cp310-win32", version = "3.10.11" },
{ identifier = "cp310-win_amd64", version = "3.10.11" },
{ identifier = "cp311-win32", version = "3.11.9" },
{ identifier = "cp311-win_amd64", version = "3.11.9" },
{ identifier = "cp312-win32", version = "3.12.10" },
{ identifier = "cp312-win_amd64", version = "3.12.10" },
{ identifier = "cp313-win32", version = "3.13.13" },
{ identifier = "cp313-win_amd64", version = "3.13.13" },
{ identifier = "cp314-win32", version = "3.14.5" },
{ identifier = "cp314t-win32", version = "3.14.5" },
{ identifier = "cp314-win_amd64", version = "3.14.5" },
{ identifier = "cp314t-win_amd64", version = "3.14.5" },
{ identifier = "cp315-win32", version = "3.15.0-b2" },
{ identifier = "cp315t-win32", version = "3.15.0-b2" },
{ identifier = "cp315-win_amd64", version = "3.15.0-b2" },
{ identifier = "cp315t-win_amd64", version = "3.15.0-b2" },
{ identifier = "cp39-win_arm64", version = "3.9.10" },
{ identifier = "cp310-win_arm64", version = "3.10.11" },
{ identifier = "cp311-win_arm64", version = "3.11.9" },
{ identifier = "cp312-win_arm64", version = "3.12.10" },
{ identifier = "cp313-win_arm64", version = "3.13.13" },
{ identifier = "cp314-win_arm64", version = "3.14.5" },
{ identifier = "cp314t-win_arm64", version = "3.14.5" },
{ identifier = "cp315-win_arm64", version = "3.15.0-b2" },
{ identifier = "cp315t-win_arm64", version = "3.15.0-b2" },
{ identifier = "pp39-win_amd64", version = "3.9", url = "https://downloads.python.org/pypy/pypy3.9-v7.3.16-win64.zip", sha256 = "06ec12a5e964dc0ad33e6f380185a4d295178dce6d6df512f508e7aee00a1323" },
{ identifier = "pp310-win_amd64", version = "3.10", url = "https://downloads.python.org/pypy/pypy3.10-v7.3.19-win64.zip", sha256 = "c0d07bba6c8fb4e5804f4a8b3f8ef07cc3d89f6ad1db42a45ffb9be60bbb7cc2" },
{ identifier = "pp311-win_amd64", version = "3.11", url = "https://downloads.python.org/pypy/pypy3.11-v7.3.23-win64.zip", sha256 = "948b8ea58dea5b9917210fe4afd242c788fbfaba1c3f1a25e696a404f703389a" },
{ identifier = "gp312_250-win_amd64", version = "3.12", url = "https://github.com/oracle/graalpython/releases/download/graal-25.0.3/graalpy-25.0.3-windows-amd64.zip", sha256 = "2ae5c42b93e08d9e106a017634a3442b272ccae6d08ace157244de0449e937d4" },
]
[pyodide]
python_configurations = [
{ identifier = "cp312-pyodide_wasm32", version = "3.12", default_pyodide_version = "0.27.7", node_version = "v22", sha256 = "0c2e7db42efa5d1dac38b50f8b3d659a1e3885d0a233494831b8206281307d52" },
{ identifier = "cp313-pyodide_wasm32", version = "3.13", default_pyodide_version = "0.29.4", node_version = "v22", sha256 = "a29fc4a076408a18fc29eb4b280f80c6dddc95c19514c874c29e590d0931c02a" },
{ identifier = "cp314-pyodide_wasm32", version = "3.14", default_pyodide_version = "314.0.0", node_version = "v24", sha256 = "6e76cf18251bd7d046bdb7ef35c9f938c8e995f10d88397497ca4b7020f27ba6" },
]
[android]
python_configurations = [
{ identifier = "cp313-android_arm64_v8a", version = "3.13", url = "https://repo.maven.apache.org/maven2/com/chaquo/python/python/3.13.13/python-3.13.13-aarch64-linux-android.tar.gz", sha256 = "a21250c246b36eb704be096be51e40ab24cafa3ae1d7ca4c396e6bf780cf87cf" },
{ identifier = "cp313-android_x86_64", version = "3.13", url = "https://repo.maven.apache.org/maven2/com/chaquo/python/python/3.13.13/python-3.13.13-x86_64-linux-android.tar.gz", sha256 = "b021b76df2c8d64e41c76d18a2d91f27652356bd932c06f9076bac1b664c0e37" },
{ identifier = "cp314-android_arm64_v8a", version = "3.14", url = "https://www.python.org/ftp/python/3.14.5/python-3.14.5-aarch64-linux-android.tar.gz", sha256 = "f008321abf837fcaec569df143283ece0e764b18d8c75763200160553f906af1" },
{ identifier = "cp314-android_x86_64", version = "3.14", url = "https://www.python.org/ftp/python/3.14.5/python-3.14.5-x86_64-linux-android.tar.gz", sha256 = "36184e31464b2b959d92c81ded8d025834342cc080623bced282a2a9a57bd47c" },
{ identifier = "cp315-android_arm64_v8a", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-aarch64-linux-android.tar.gz", sha256 = "de5e36ffc3c33b63a1ed53fe83db4dee16b003a3180ac35e5c4c203d3750e076" },
{ identifier = "cp315-android_x86_64", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-x86_64-linux-android.tar.gz", sha256 = "e05ef9c142ea83b41b194b16014ffe2249d27783313c3b00b97f6c93d9a56b92" },
]
[ios]
python_configurations = [
{ identifier = "cp313-ios_arm64_iphoneos", version = "3.13", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.13-b13/Python-3.13-iOS-support.b13.tar.gz", sha256 = "d1f95f95137a4b91dc0cbe9b99ddfc0a78918dde883c7fbe7147074a5e715274" },
{ identifier = "cp313-ios_x86_64_iphonesimulator", version = "3.13", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.13-b13/Python-3.13-iOS-support.b13.tar.gz", sha256 = "d1f95f95137a4b91dc0cbe9b99ddfc0a78918dde883c7fbe7147074a5e715274" },
{ identifier = "cp313-ios_arm64_iphonesimulator", version = "3.13", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.13-b13/Python-3.13-iOS-support.b13.tar.gz", sha256 = "d1f95f95137a4b91dc0cbe9b99ddfc0a78918dde883c7fbe7147074a5e715274" },
{ identifier = "cp314-ios_arm64_iphoneos", version = "3.14", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.14-b9/Python-3.14-iOS-support.b9.tar.gz", sha256 = "8e4123b543bf17fdae2e2c6c033434487752438431014eb12e6d833aa35927a8" },
{ identifier = "cp314-ios_x86_64_iphonesimulator", version = "3.14", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.14-b9/Python-3.14-iOS-support.b9.tar.gz", sha256 = "8e4123b543bf17fdae2e2c6c033434487752438431014eb12e6d833aa35927a8" },
{ identifier = "cp314-ios_arm64_iphonesimulator", version = "3.14", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.14-b9/Python-3.14-iOS-support.b9.tar.gz", sha256 = "8e4123b543bf17fdae2e2c6c033434487752438431014eb12e6d833aa35927a8" },
{ identifier = "cp315-ios_arm64_iphoneos", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-iOS-XCframework.tar.gz", sha256 = "fd6b6fb8f82cf44622d28ea53b8197c89fb06505557d29d4d7a47f9386c1c896" },
{ identifier = "cp315-ios_x86_64_iphonesimulator", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-iOS-XCframework.tar.gz", sha256 = "fd6b6fb8f82cf44622d28ea53b8197c89fb06505557d29d4d7a47f9386c1c896" },
{ identifier = "cp315-ios_arm64_iphonesimulator", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-iOS-XCframework.tar.gz", sha256 = "fd6b6fb8f82cf44622d28ea53b8197c89fb06505557d29d4d7a47f9386c1c896" },
{ identifier = "cp27-win32", version = "2.7.18", arch = "32" },
{ identifier = "cp27-win_amd64", version = "2.7.18", arch = "64" },
{ identifier = "cp35-win32", version = "3.5.4", arch = "32" },
{ identifier = "cp35-win_amd64", version = "3.5.4", arch = "64" },
{ identifier = "cp36-win32", version = "3.6.8", arch = "32" },
{ identifier = "cp36-win_amd64", version = "3.6.8", arch = "64" },
{ identifier = "cp37-win32", version = "3.7.9", arch = "32" },
{ identifier = "cp37-win_amd64", version = "3.7.9", arch = "64" },
{ identifier = "cp38-win32", version = "3.8.9", arch = "32" },
{ identifier = "cp38-win_amd64", version = "3.8.9", arch = "64" },
{ identifier = "cp39-win32", version = "3.9.4", arch = "32" },
{ identifier = "cp39-win_amd64", version = "3.9.4", arch = "64" },
{ identifier = "pp27-win32", version = "2.7", arch = "32", url = "https://downloads.python.org/pypy/pypy2.7-v7.3.3-win32.zip" },
{ identifier = "pp36-win32", version = "3.6", arch = "32", url = "https://downloads.python.org/pypy/pypy3.6-v7.3.3-win32.zip" },
{ identifier = "pp37-win32", version = "3.7", arch = "32", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.3-win32.zip" },
]
File diff suppressed because it is too large Load Diff
@@ -1,91 +0,0 @@
# This file was autogenerated by uv via the following command:
# nox -s update_constraints
annotated-types==0.7.0
# via pydantic
auditwheel-emscripten==0.2.5
# via pyodide-build
build==1.5.0
# via
# -r .nox/update_constraints/tmp/constraints-pyodide.in
# pyodide-build
certifi==2026.5.20
# via requests
charset-normalizer==3.4.7
# via requests
click==8.1.8
# via
# -r .nox/update_constraints/tmp/constraints-pyodide.in
# pyodide-build
# pyodide-cli
distlib==0.4.1
# via virtualenv
filelock==3.29.1
# via
# python-discovery
# virtualenv
idna==3.18
# via requests
leb128==1.0.9
# via auditwheel-emscripten
markdown-it-py==4.2.0
# via rich
mdurl==0.1.2
# via markdown-it-py
packaging==26.2
# via
# auditwheel-emscripten
# build
# pyodide-build
# wheel
pip==26.1.2
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
platformdirs==4.10.0
# via
# pyodide-build
# python-discovery
# virtualenv
pydantic==2.13.4
# via
# pyodide-build
# pyodide-lock
pydantic-core==2.46.4
# via pydantic
pygments==2.20.0
# via rich
pyodide-build==0.35.0
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
pyodide-cli==0.5.0
# via
# auditwheel-emscripten
# pyodide-build
pyodide-lock==0.1.3
# via pyodide-build
pyproject-hooks==1.2.0
# via build
python-discovery==1.4.0
# via virtualenv
requests==2.34.2
# via pyodide-build
rich==15.0.0
# via
# pyodide-build
# pyodide-cli
ruamel-yaml==0.19.1
# via pyodide-build
typing-extensions==4.15.0
# via
# pydantic
# pydantic-core
# typing-inspection
typing-inspection==0.4.2
# via pydantic
urllib3==2.7.0
# via requests
virtualenv==21.4.2
# via
# build
# pyodide-build
wheel==0.47.0
# via
# auditwheel-emscripten
# pyodide-build
@@ -1,91 +0,0 @@
# This file was autogenerated by uv via the following command:
# nox -s update_constraints
annotated-types==0.7.0
# via pydantic
auditwheel-emscripten==0.2.5
# via pyodide-build
build==1.5.0
# via
# -r .nox/update_constraints/tmp/constraints-pyodide.in
# pyodide-build
certifi==2026.5.20
# via requests
charset-normalizer==3.4.7
# via requests
click==8.1.8
# via
# -r .nox/update_constraints/tmp/constraints-pyodide.in
# pyodide-build
# pyodide-cli
distlib==0.4.1
# via virtualenv
filelock==3.29.1
# via
# python-discovery
# virtualenv
idna==3.18
# via requests
leb128==1.0.9
# via auditwheel-emscripten
markdown-it-py==4.2.0
# via rich
mdurl==0.1.2
# via markdown-it-py
packaging==26.2
# via
# auditwheel-emscripten
# build
# pyodide-build
# wheel
pip==26.1.2
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
platformdirs==4.10.0
# via
# pyodide-build
# python-discovery
# virtualenv
pydantic==2.13.4
# via
# pyodide-build
# pyodide-lock
pydantic-core==2.46.4
# via pydantic
pygments==2.20.0
# via rich
pyodide-build==0.35.0
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
pyodide-cli==0.5.0
# via
# auditwheel-emscripten
# pyodide-build
pyodide-lock==0.1.3
# via pyodide-build
pyproject-hooks==1.2.0
# via build
python-discovery==1.4.0
# via virtualenv
requests==2.34.2
# via pyodide-build
rich==15.0.0
# via
# pyodide-build
# pyodide-cli
ruamel-yaml==0.19.1
# via pyodide-build
typing-extensions==4.15.0
# via
# pydantic
# pydantic-core
# typing-inspection
typing-inspection==0.4.2
# via pydantic
urllib3==2.7.0
# via requests
virtualenv==21.4.2
# via
# build
# pyodide-build
wheel==0.47.0
# via
# auditwheel-emscripten
# pyodide-build
@@ -1,91 +0,0 @@
# This file was autogenerated by uv via the following command:
# nox -s update_constraints
annotated-types==0.7.0
# via pydantic
auditwheel-emscripten==0.2.5
# via pyodide-build
build==1.5.0
# via
# -r .nox/update_constraints/tmp/constraints-pyodide.in
# pyodide-build
certifi==2026.5.20
# via requests
charset-normalizer==3.4.7
# via requests
click==8.1.8
# via
# -r .nox/update_constraints/tmp/constraints-pyodide.in
# pyodide-build
# pyodide-cli
distlib==0.4.1
# via virtualenv
filelock==3.29.1
# via
# python-discovery
# virtualenv
idna==3.18
# via requests
leb128==1.0.9
# via auditwheel-emscripten
markdown-it-py==4.2.0
# via rich
mdurl==0.1.2
# via markdown-it-py
packaging==26.2
# via
# auditwheel-emscripten
# build
# pyodide-build
# wheel
pip==26.1.2
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
platformdirs==4.10.0
# via
# pyodide-build
# python-discovery
# virtualenv
pydantic==2.13.4
# via
# pyodide-build
# pyodide-lock
pydantic-core==2.46.4
# via pydantic
pygments==2.20.0
# via rich
pyodide-build==0.35.0
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
pyodide-cli==0.5.0
# via
# auditwheel-emscripten
# pyodide-build
pyodide-lock==0.1.3
# via pyodide-build
pyproject-hooks==1.2.0
# via build
python-discovery==1.4.0
# via virtualenv
requests==2.34.2
# via pyodide-build
rich==15.0.0
# via
# pyodide-build
# pyodide-cli
ruamel-yaml==0.19.1
# via pyodide-build
typing-extensions==4.15.0
# via
# pydantic
# pydantic-core
# typing-inspection
typing-inspection==0.4.2
# via pydantic
urllib3==2.7.0
# via requests
virtualenv==21.4.2
# via
# build
# pyodide-build
wheel==0.47.0
# via
# auditwheel-emscripten
# pyodide-build
@@ -0,0 +1,55 @@
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bin/update_dependencies.py
#
appdirs==1.4.4
# via virtualenv
configparser==4.0.2
# via importlib-metadata
contextlib2==0.6.0.post1
# via
# importlib-metadata
# importlib-resources
delocate==0.8.2
# via -r cibuildwheel/resources/constraints.in
distlib==0.3.1
# via virtualenv
filelock==3.0.12
# via virtualenv
importlib-metadata==2.1.1
# via virtualenv
importlib-resources==3.3.1
# via virtualenv
pathlib2==2.3.5
# via
# importlib-metadata
# importlib-resources
# virtualenv
scandir==1.10.0
# via pathlib2
singledispatch==3.6.1
# via importlib-resources
six==1.15.0
# via
# pathlib2
# virtualenv
typing==3.7.4.3
# via importlib-resources
virtualenv==20.4.3
# via -r cibuildwheel/resources/constraints.in
wheel==0.36.2
# via
# -r cibuildwheel/resources/constraints.in
# delocate
zipp==1.2.0
# via
# importlib-metadata
# importlib-resources
# The following packages are considered to be unsafe in a requirements file:
pip==20.3.4
# via -r cibuildwheel/resources/constraints.in
setuptools==44.1.1
# via -r cibuildwheel/resources/constraints.in
@@ -1,105 +0,0 @@
# This file was autogenerated by uv via the following command:
# nox -s update_constraints
abi3audit==0.0.26
# via -r cibuildwheel/resources/constraints.in
abi3info==2025.11.29
# via abi3audit
altgraph==0.17.5
# via macholib
attrs==26.1.0
# via
# cattrs
# requests-cache
auditwheel==6.7.0
# via -r cibuildwheel/resources/constraints.in
build==1.5.0
# via -r cibuildwheel/resources/constraints.in
cattrs==26.1.0
# via requests-cache
certifi==2026.5.20
# via requests
charset-normalizer==3.4.7
# via requests
delocate==0.13.0
# via -r cibuildwheel/resources/constraints.in
delvewheel==1.13.0
# via -r cibuildwheel/resources/constraints.in
distlib==0.4.1
# via virtualenv
exceptiongroup==1.3.1
# via cattrs
filelock==3.29.1
# via
# python-discovery
# virtualenv
idna==3.18
# via
# requests
# url-normalize
importlib-metadata==9.0.0
# via build
kaitaistruct==0.11
# via abi3audit
macholib==1.16.4
# via delocate
markdown-it-py==4.2.0
# via rich
mdurl==0.1.2
# via markdown-it-py
packaging==26.2
# via
# abi3audit
# auditwheel
# build
# delocate
patchelf==0.17.2.4
# via -r cibuildwheel/resources/constraints.in
pefile==2024.8.26
# via
# abi3audit
# delvewheel
pip==26.1.2
# via -r cibuildwheel/resources/constraints.in
pkgconf==2.5.1.post2
# via -r cibuildwheel/resources/constraints.in
platformdirs==4.10.0
# via
# python-discovery
# requests-cache
# virtualenv
pyelftools==0.33
# via
# abi3audit
# auditwheel
pygments==2.20.0
# via rich
pyproject-hooks==1.2.0
# via build
python-discovery==1.4.0
# via virtualenv
requests==2.34.2
# via
# abi3audit
# requests-cache
requests-cache==1.3.2
# via abi3audit
rich==15.0.0
# via abi3audit
tomli==2.4.1
# via build
typing-extensions==4.15.0
# via
# cattrs
# delocate
# exceptiongroup
# virtualenv
url-normalize==3.0.0
# via requests-cache
urllib3==2.7.0
# via
# requests
# requests-cache
virtualenv==21.4.2
# via -r cibuildwheel/resources/constraints.in
zipp==4.1.0
# via importlib-metadata
@@ -1,96 +0,0 @@
# This file was autogenerated by uv via the following command:
# nox -s update_constraints
abi3audit==0.0.26
# via -r cibuildwheel/resources/constraints.in
abi3info==2025.11.29
# via abi3audit
altgraph==0.17.5
# via macholib
attrs==26.1.0
# via
# cattrs
# requests-cache
auditwheel==6.7.0
# via -r cibuildwheel/resources/constraints.in
build==1.5.0
# via -r cibuildwheel/resources/constraints.in
cattrs==26.1.0
# via requests-cache
certifi==2026.5.20
# via requests
charset-normalizer==3.4.7
# via requests
delocate==0.13.0
# via -r cibuildwheel/resources/constraints.in
delvewheel==1.13.0
# via -r cibuildwheel/resources/constraints.in
distlib==0.4.1
# via virtualenv
filelock==3.29.1
# via
# python-discovery
# virtualenv
idna==3.18
# via
# requests
# url-normalize
kaitaistruct==0.11
# via abi3audit
macholib==1.16.4
# via delocate
markdown-it-py==4.2.0
# via rich
mdurl==0.1.2
# via markdown-it-py
packaging==26.2
# via
# abi3audit
# auditwheel
# build
# delocate
patchelf==0.17.2.4
# via -r cibuildwheel/resources/constraints.in
pefile==2024.8.26
# via
# abi3audit
# delvewheel
pip==26.1.2; implementation_name != "graalpy" or platform_system != "Windows"
pip==26.0.1; implementation_name == "graalpy" and platform_system == "Windows"
# via -r cibuildwheel/resources/constraints.in
pkgconf==2.5.1.post2
# via -r cibuildwheel/resources/constraints.in
platformdirs==4.10.0
# via
# python-discovery
# requests-cache
# virtualenv
pyelftools==0.33
# via
# abi3audit
# auditwheel
pygments==2.20.0
# via rich
pyproject-hooks==1.2.0
# via build
python-discovery==1.4.0
# via virtualenv
requests==2.34.2
# via
# abi3audit
# requests-cache
requests-cache==1.3.2
# via abi3audit
rich==15.0.0
# via abi3audit
typing-extensions==4.15.0
# via
# cattrs
# delocate
url-normalize==3.0.0
# via requests-cache
urllib3==2.7.0
# via
# requests
# requests-cache
virtualenv==21.4.2
# via -r cibuildwheel/resources/constraints.in
@@ -1,96 +0,0 @@
# This file was autogenerated by uv via the following command:
# nox -s update_constraints
abi3audit==0.0.26
# via -r cibuildwheel/resources/constraints.in
abi3info==2025.11.29
# via abi3audit
altgraph==0.17.5
# via macholib
attrs==26.1.0
# via
# cattrs
# requests-cache
auditwheel==6.7.0
# via -r cibuildwheel/resources/constraints.in
build==1.5.0
# via -r cibuildwheel/resources/constraints.in
cattrs==26.1.0
# via requests-cache
certifi==2026.5.20
# via requests
charset-normalizer==3.4.7
# via requests
delocate==0.13.0
# via -r cibuildwheel/resources/constraints.in
delvewheel==1.13.0
# via -r cibuildwheel/resources/constraints.in
distlib==0.4.1
# via virtualenv
filelock==3.29.1
# via
# python-discovery
# virtualenv
idna==3.18
# via
# requests
# url-normalize
kaitaistruct==0.11
# via abi3audit
macholib==1.16.4
# via delocate
markdown-it-py==4.2.0
# via rich
mdurl==0.1.2
# via markdown-it-py
packaging==26.2
# via
# abi3audit
# auditwheel
# build
# delocate
patchelf==0.17.2.4
# via -r cibuildwheel/resources/constraints.in
pefile==2024.8.26
# via
# abi3audit
# delvewheel
pip==26.1.2; implementation_name != "graalpy" or platform_system != "Windows"
pip==26.0.1; implementation_name == "graalpy" and platform_system == "Windows"
# via -r cibuildwheel/resources/constraints.in
pkgconf==2.5.1.post2
# via -r cibuildwheel/resources/constraints.in
platformdirs==4.10.0
# via
# python-discovery
# requests-cache
# virtualenv
pyelftools==0.33
# via
# abi3audit
# auditwheel
pygments==2.20.0
# via rich
pyproject-hooks==1.2.0
# via build
python-discovery==1.4.0
# via virtualenv
requests==2.34.2
# via
# abi3audit
# requests-cache
requests-cache==1.3.2
# via abi3audit
rich==15.0.0
# via abi3audit
typing-extensions==4.15.0
# via
# cattrs
# delocate
url-normalize==3.0.0
# via requests-cache
urllib3==2.7.0
# via
# requests
# requests-cache
virtualenv==21.4.2
# via -r cibuildwheel/resources/constraints.in
@@ -1,95 +0,0 @@
# This file was autogenerated by uv via the following command:
# nox -s update_constraints
abi3audit==0.0.26
# via -r cibuildwheel/resources/constraints.in
abi3info==2025.11.29
# via abi3audit
altgraph==0.17.5
# via macholib
attrs==26.1.0
# via
# cattrs
# requests-cache
auditwheel==6.7.0
# via -r cibuildwheel/resources/constraints.in
build==1.5.0
# via -r cibuildwheel/resources/constraints.in
cattrs==26.1.0
# via requests-cache
certifi==2026.5.20
# via requests
charset-normalizer==3.4.7
# via requests
delocate==0.13.0
# via -r cibuildwheel/resources/constraints.in
delvewheel==1.13.0
# via -r cibuildwheel/resources/constraints.in
distlib==0.4.1
# via virtualenv
filelock==3.29.1
# via
# python-discovery
# virtualenv
idna==3.18
# via
# requests
# url-normalize
kaitaistruct==0.11
# via abi3audit
macholib==1.16.4
# via delocate
markdown-it-py==4.2.0
# via rich
mdurl==0.1.2
# via markdown-it-py
packaging==26.2
# via
# abi3audit
# auditwheel
# build
# delocate
patchelf==0.17.2.4
# via -r cibuildwheel/resources/constraints.in
pefile==2024.8.26
# via
# abi3audit
# delvewheel
pip==26.1.2
# via -r cibuildwheel/resources/constraints.in
pkgconf==2.5.1.post2
# via -r cibuildwheel/resources/constraints.in
platformdirs==4.10.0
# via
# python-discovery
# requests-cache
# virtualenv
pyelftools==0.33
# via
# abi3audit
# auditwheel
pygments==2.20.0
# via rich
pyproject-hooks==1.2.0
# via build
python-discovery==1.4.0
# via virtualenv
requests==2.34.2
# via
# abi3audit
# requests-cache
requests-cache==1.3.2
# via abi3audit
rich==15.0.0
# via abi3audit
typing-extensions==4.15.0
# via
# cattrs
# delocate
url-normalize==3.0.0
# via requests-cache
urllib3==2.7.0
# via
# requests
# requests-cache
virtualenv==21.4.2
# via -r cibuildwheel/resources/constraints.in
@@ -1,95 +0,0 @@
# This file was autogenerated by uv via the following command:
# nox -s update_constraints
abi3audit==0.0.26
# via -r cibuildwheel/resources/constraints.in
abi3info==2025.11.29
# via abi3audit
altgraph==0.17.5
# via macholib
attrs==26.1.0
# via
# cattrs
# requests-cache
auditwheel==6.7.0
# via -r cibuildwheel/resources/constraints.in
build==1.5.0
# via -r cibuildwheel/resources/constraints.in
cattrs==26.1.0
# via requests-cache
certifi==2026.5.20
# via requests
charset-normalizer==3.4.7
# via requests
delocate==0.13.0
# via -r cibuildwheel/resources/constraints.in
delvewheel==1.13.0
# via -r cibuildwheel/resources/constraints.in
distlib==0.4.1
# via virtualenv
filelock==3.29.1
# via
# python-discovery
# virtualenv
idna==3.18
# via
# requests
# url-normalize
kaitaistruct==0.11
# via abi3audit
macholib==1.16.4
# via delocate
markdown-it-py==4.2.0
# via rich
mdurl==0.1.2
# via markdown-it-py
packaging==26.2
# via
# abi3audit
# auditwheel
# build
# delocate
patchelf==0.17.2.4
# via -r cibuildwheel/resources/constraints.in
pefile==2024.8.26
# via
# abi3audit
# delvewheel
pip==26.1.2
# via -r cibuildwheel/resources/constraints.in
pkgconf==2.5.1.post2
# via -r cibuildwheel/resources/constraints.in
platformdirs==4.10.0
# via
# python-discovery
# requests-cache
# virtualenv
pyelftools==0.33
# via
# abi3audit
# auditwheel
pygments==2.20.0
# via rich
pyproject-hooks==1.2.0
# via build
python-discovery==1.4.0
# via virtualenv
requests==2.34.2
# via
# abi3audit
# requests-cache
requests-cache==1.3.2
# via abi3audit
rich==15.0.0
# via abi3audit
typing-extensions==4.15.0
# via
# cattrs
# delocate
url-normalize==3.0.0
# via requests-cache
urllib3==2.7.0
# via
# requests
# requests-cache
virtualenv==21.4.2
# via -r cibuildwheel/resources/constraints.in
@@ -1,95 +0,0 @@
# This file was autogenerated by uv via the following command:
# nox -s update_constraints
abi3audit==0.0.26
# via -r cibuildwheel/resources/constraints.in
abi3info==2025.11.29
# via abi3audit
altgraph==0.17.5
# via macholib
attrs==26.1.0
# via
# cattrs
# requests-cache
auditwheel==6.7.0
# via -r cibuildwheel/resources/constraints.in
build==1.5.0
# via -r cibuildwheel/resources/constraints.in
cattrs==26.1.0
# via requests-cache
certifi==2026.5.20
# via requests
charset-normalizer==3.4.7
# via requests
delocate==0.13.0
# via -r cibuildwheel/resources/constraints.in
delvewheel==1.13.0
# via -r cibuildwheel/resources/constraints.in
distlib==0.4.1
# via virtualenv
filelock==3.29.1
# via
# python-discovery
# virtualenv
idna==3.18
# via
# requests
# url-normalize
kaitaistruct==0.11
# via abi3audit
macholib==1.16.4
# via delocate
markdown-it-py==4.2.0
# via rich
mdurl==0.1.2
# via markdown-it-py
packaging==26.2
# via
# abi3audit
# auditwheel
# build
# delocate
patchelf==0.17.2.4
# via -r cibuildwheel/resources/constraints.in
pefile==2024.8.26
# via
# abi3audit
# delvewheel
pip==26.1.2
# via -r cibuildwheel/resources/constraints.in
pkgconf==2.5.1.post2
# via -r cibuildwheel/resources/constraints.in
platformdirs==4.10.0
# via
# python-discovery
# requests-cache
# virtualenv
pyelftools==0.33
# via
# abi3audit
# auditwheel
pygments==2.20.0
# via rich
pyproject-hooks==1.2.0
# via build
python-discovery==1.4.0
# via virtualenv
requests==2.34.2
# via
# abi3audit
# requests-cache
requests-cache==1.3.2
# via abi3audit
rich==15.0.0
# via abi3audit
typing-extensions==4.15.0
# via
# cattrs
# delocate
url-normalize==3.0.0
# via requests-cache
urllib3==2.7.0
# via
# requests
# requests-cache
virtualenv==21.4.2
# via -r cibuildwheel/resources/constraints.in
@@ -0,0 +1,36 @@
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bin/update_dependencies.py
#
appdirs==1.4.4
# via virtualenv
delocate==0.8.2
# via -r cibuildwheel/resources/constraints.in
distlib==0.3.1
# via virtualenv
filelock==3.0.12
# via virtualenv
importlib-metadata==2.1.1
# via virtualenv
importlib-resources==3.2.1
# via virtualenv
six==1.15.0
# via virtualenv
virtualenv==20.4.3
# via -r cibuildwheel/resources/constraints.in
wheel==0.36.2
# via
# -r cibuildwheel/resources/constraints.in
# delocate
zipp==1.2.0
# via
# importlib-metadata
# importlib-resources
# The following packages are considered to be unsafe in a requirements file:
pip==20.3.4
# via -r cibuildwheel/resources/constraints.in
setuptools==50.3.2
# via -r cibuildwheel/resources/constraints.in
@@ -0,0 +1,38 @@
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bin/update_dependencies.py
#
appdirs==1.4.4
# via virtualenv
delocate==0.8.2
# via -r cibuildwheel/resources/constraints.in
distlib==0.3.1
# via virtualenv
filelock==3.0.12
# via virtualenv
importlib-metadata==3.10.0
# via virtualenv
importlib-resources==5.1.2
# via virtualenv
six==1.15.0
# via virtualenv
typing-extensions==3.7.4.3
# via importlib-metadata
virtualenv==20.4.3
# via -r cibuildwheel/resources/constraints.in
wheel==0.36.2
# via
# -r cibuildwheel/resources/constraints.in
# delocate
zipp==3.4.1
# via
# importlib-metadata
# importlib-resources
# The following packages are considered to be unsafe in a requirements file:
pip==21.0.1
# via -r cibuildwheel/resources/constraints.in
setuptools==54.2.0
# via -r cibuildwheel/resources/constraints.in
@@ -0,0 +1,34 @@
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bin/update_dependencies.py
#
appdirs==1.4.4
# via virtualenv
delocate==0.8.2
# via -r cibuildwheel/resources/constraints.in
distlib==0.3.1
# via virtualenv
filelock==3.0.12
# via virtualenv
importlib-metadata==3.10.0
# via virtualenv
six==1.15.0
# via virtualenv
typing-extensions==3.7.4.3
# via importlib-metadata
virtualenv==20.4.3
# via -r cibuildwheel/resources/constraints.in
wheel==0.36.2
# via
# -r cibuildwheel/resources/constraints.in
# delocate
zipp==3.4.1
# via importlib-metadata
# The following packages are considered to be unsafe in a requirements file:
pip==21.0.1
# via -r cibuildwheel/resources/constraints.in
setuptools==54.2.0
# via -r cibuildwheel/resources/constraints.in
@@ -0,0 +1,28 @@
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bin/update_dependencies.py
#
appdirs==1.4.4
# via virtualenv
delocate==0.8.2
# via -r cibuildwheel/resources/constraints.in
distlib==0.3.1
# via virtualenv
filelock==3.0.12
# via virtualenv
six==1.15.0
# via virtualenv
virtualenv==20.4.3
# via -r cibuildwheel/resources/constraints.in
wheel==0.36.2
# via
# -r cibuildwheel/resources/constraints.in
# delocate
# The following packages are considered to be unsafe in a requirements file:
pip==21.0.1
# via -r cibuildwheel/resources/constraints.in
setuptools==54.2.0
# via -r cibuildwheel/resources/constraints.in
+22 -101
View File
@@ -1,107 +1,28 @@
# This file was autogenerated by uv via the following command:
# nox -s update_constraints
abi3audit==0.0.25
# via -r cibuildwheel/resources/constraints.in
abi3info==2025.4.29
# via abi3audit
altgraph==0.17.5
# via macholib
attrs==26.1.0
# via
# cattrs
# requests-cache
auditwheel==6.4.2
# via -r cibuildwheel/resources/constraints.in
build==1.4.4
# via -r cibuildwheel/resources/constraints.in
cattrs==25.3.0
# via requests-cache
certifi==2026.5.20
# via requests
charset-normalizer==3.4.7
# via requests
delocate==0.13.0
# via -r cibuildwheel/resources/constraints.in
delvewheel==1.13.0
# via -r cibuildwheel/resources/constraints.in
distlib==0.4.1
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bin/update_dependencies.py
#
appdirs==1.4.4
# via virtualenv
exceptiongroup==1.3.1
# via cattrs
filelock==3.19.1
# via
# python-discovery
# virtualenv
idna==3.18
# via
# requests
# url-normalize
importlib-metadata==8.7.1
# via
# build
# pkgconf
kaitaistruct==0.11
# via abi3audit
macholib==1.16.4
# via delocate
markdown-it-py==3.0.0
# via rich
mdurl==0.1.2
# via markdown-it-py
packaging==25.0
# via
# abi3audit
# auditwheel
# build
# delocate
patchelf==0.17.2.4
delocate==0.8.2
# via -r cibuildwheel/resources/constraints.in
pefile==2024.8.26
# via
# abi3audit
# delvewheel
pip==26.0.1
# via -r cibuildwheel/resources/constraints.in
pkgconf==2.4.3.post2
# via -r cibuildwheel/resources/constraints.in
platformdirs==4.4.0
# via
# python-discovery
# requests-cache
# virtualenv
pyelftools==0.32
# via
# abi3audit
# auditwheel
pygments==2.20.0
# via rich
pyproject-hooks==1.2.0
# via build
python-discovery==1.4.0
distlib==0.3.1
# via virtualenv
requests==2.32.5
filelock==3.0.12
# via virtualenv
six==1.15.0
# via virtualenv
virtualenv==20.4.3
# via -r cibuildwheel/resources/constraints.in
wheel==0.36.2
# via
# abi3audit
# requests-cache
requests-cache==1.2.1
# via abi3audit
rich==14.2.0
# via abi3audit
tomli==2.4.1
# via build
typing-extensions==4.15.0
# via
# cattrs
# -r cibuildwheel/resources/constraints.in
# delocate
# exceptiongroup
# virtualenv
url-normalize==2.2.1
# via requests-cache
urllib3==2.6.3
# via
# requests
# requests-cache
virtualenv==21.4.2
# The following packages are considered to be unsafe in a requirements file:
pip==21.0.1
# via -r cibuildwheel/resources/constraints.in
setuptools==54.2.0
# via -r cibuildwheel/resources/constraints.in
zipp==3.23.1
# via importlib-metadata
+2 -8
View File
@@ -1,11 +1,5 @@
pip
build
setuptools
wheel
delocate
delvewheel
virtualenv
abi3audit
# Android
auditwheel
patchelf
pkgconf
+22 -89
View File
@@ -1,95 +1,28 @@
# This file was autogenerated by uv via the following command:
# nox -s update_constraints
abi3audit==0.0.26
# via -r cibuildwheel/resources/constraints.in
abi3info==2025.11.29
# via abi3audit
altgraph==0.17.5
# via macholib
attrs==26.1.0
# via
# cattrs
# requests-cache
auditwheel==6.7.0
# via -r cibuildwheel/resources/constraints.in
build==1.5.0
# via -r cibuildwheel/resources/constraints.in
cattrs==26.1.0
# via requests-cache
certifi==2026.5.20
# via requests
charset-normalizer==3.4.7
# via requests
delocate==0.13.0
# via -r cibuildwheel/resources/constraints.in
delvewheel==1.13.0
# via -r cibuildwheel/resources/constraints.in
distlib==0.4.1
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bin/update_dependencies.py
#
appdirs==1.4.4
# via virtualenv
filelock==3.29.1
# via
# python-discovery
# virtualenv
idna==3.18
# via
# requests
# url-normalize
kaitaistruct==0.11
# via abi3audit
macholib==1.16.4
# via delocate
markdown-it-py==4.2.0
# via rich
mdurl==0.1.2
# via markdown-it-py
packaging==26.2
# via
# abi3audit
# auditwheel
# build
# delocate
patchelf==0.17.2.4
delocate==0.8.2
# via -r cibuildwheel/resources/constraints.in
pefile==2024.8.26
# via
# abi3audit
# delvewheel
pip==26.1.2
# via -r cibuildwheel/resources/constraints.in
pkgconf==2.5.1.post2
# via -r cibuildwheel/resources/constraints.in
platformdirs==4.10.0
# via
# python-discovery
# requests-cache
# virtualenv
pyelftools==0.33
# via
# abi3audit
# auditwheel
pygments==2.20.0
# via rich
pyproject-hooks==1.2.0
# via build
python-discovery==1.4.0
distlib==0.3.1
# via virtualenv
requests==2.34.2
filelock==3.0.12
# via virtualenv
six==1.15.0
# via virtualenv
virtualenv==20.4.3
# via -r cibuildwheel/resources/constraints.in
wheel==0.36.2
# via
# abi3audit
# requests-cache
requests-cache==1.3.2
# via abi3audit
rich==15.0.0
# via abi3audit
typing-extensions==4.15.0
# via
# cattrs
# -r cibuildwheel/resources/constraints.in
# delocate
url-normalize==3.0.0
# via requests-cache
urllib3==2.7.0
# via
# requests
# requests-cache
virtualenv==21.4.2
# The following packages are considered to be unsafe in a requirements file:
pip==21.0.1
# via -r cibuildwheel/resources/constraints.in
setuptools==54.2.0
# via -r cibuildwheel/resources/constraints.in
-81
View File
@@ -1,81 +0,0 @@
# These are the defaults used by cibuildwheel itself. They should match the values in
# generate_schema.py, which are used by external tools for validation and IDE support.
[tool.cibuildwheel]
build = "*"
skip = ""
test-skip = ""
enable = []
archs = ["auto"]
audit-requires = ["abi3audit"]
audit-command = "abi3audit --strict --report {abi3_wheel}"
build-frontend = "default"
config-settings = {}
dependency-versions = "pinned"
environment = {}
environment-pass = []
build-verbosity = 0
before-all = ""
before-build = ""
# TOML doesn't support explicit NULLs; use ["\u0000"] as a sentinel value.
xbuild-tools = ["\u0000"]
repair-wheel-command = ""
test-command = ""
before-test = ""
test-sources = []
test-requires = []
test-extras = []
test-groups = []
test-environment = {}
test-runtime = {}
container-engine = "docker"
pyodide-version = ""
manylinux-x86_64-image = "manylinux_2_28"
manylinux-i686-image = "manylinux_2_28"
manylinux-aarch64-image = "manylinux_2_28"
manylinux-ppc64le-image = "manylinux_2_28"
manylinux-s390x-image = "manylinux_2_28"
manylinux-armv7l-image = "manylinux_2_31"
manylinux-riscv64-image = "manylinux_2_39"
manylinux-pypy_x86_64-image = "manylinux_2_28"
manylinux-pypy_i686-image = "manylinux_2_28"
manylinux-pypy_aarch64-image = "manylinux_2_28"
musllinux-x86_64-image = "musllinux_1_2"
musllinux-i686-image = "musllinux_1_2"
musllinux-aarch64-image = "musllinux_1_2"
musllinux-ppc64le-image = "musllinux_1_2"
musllinux-s390x-image = "musllinux_1_2"
musllinux-armv7l-image = "musllinux_1_2"
musllinux-riscv64-image = "musllinux_1_2"
[tool.cibuildwheel.xbuild-files]
numpy = [
"numpy/_core/include/numpy/_numpyconfig.h",
"numpy/_core/include/numpy/numpyconfig.h",
"numpy/_core/lib/libnpymath.a",
"numpy/random/lib/libnpyrandom.a",
]
[tool.cibuildwheel.linux]
repair-wheel-command = "auditwheel repair -w {dest_dir} {wheel}"
[tool.cibuildwheel.macos]
repair-wheel-command = "delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}"
[tool.cibuildwheel.windows]
repair-wheel-command = "delvewheel repair -w {dest_dir} -v {wheel}"
[tool.cibuildwheel.android]
repair-wheel-command = "auditwheel repair --ldpaths {ldpaths} -w {dest_dir} {wheel}"
[tool.cibuildwheel.ios]
[tool.cibuildwheel.pyodide]
audit-command = ""
@@ -1,14 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<array>
<dict>
<key>attributeSetting</key>
<integer>1</integer>
<key>choiceAttribute</key>
<string>selected</string>
<key>choiceIdentifier</key>
<string>org.python.Python.PythonTFramework-3.14</string>
</dict>
</array>
</plist>
@@ -1,14 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<array>
<dict>
<key>attributeSetting</key>
<integer>1</integer>
<key>choiceAttribute</key>
<string>selected</string>
<key>choiceIdentifier</key>
<string>org.python.Python.PythonTFramework-3.15</string>
</dict>
</array>
</plist>
File diff suppressed because it is too large Load Diff
+8 -4
View File
@@ -6,7 +6,6 @@
# for the ssl module. Uses the certificates provided by the certifi package:
# https://pypi.org/project/certifi/
import contextlib
import os
import os.path
import ssl
@@ -25,23 +24,28 @@ STAT_0o775 = (
| stat.S_IXOTH
)
if sys.version_info[0] == 2:
FileNotFoundError = OSError
def main() -> None:
def main():
openssl_dir, openssl_cafile = os.path.split(ssl.get_default_verify_paths().openssl_cafile)
print(" -- pip install --upgrade certifi")
subprocess.check_call(
[sys.executable, "-E", "-s", "-m", "pip", "install", "--upgrade", "certifi"]
)
import certifi # noqa: PLC0415
import certifi
# change working directory to the default SSL directory
os.chdir(openssl_dir)
relpath_to_certifi_cafile = os.path.relpath(certifi.where())
print(" -- removing any existing file or link")
with contextlib.suppress(FileNotFoundError):
try:
os.remove(openssl_cafile)
except FileNotFoundError:
pass
print(" -- creating symlink to certifi certificate bundle")
os.symlink(relpath_to_certifi_cafile, openssl_cafile)

Some files were not shown because too many files have changed in this diff Show More