Compare commits
16
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6c62f9ee62 | ||
|
|
9a608b0b94 | ||
|
|
5ea360607d | ||
|
|
dc127cd69d | ||
|
|
d1898152f7 | ||
|
|
e4a917f717 | ||
|
|
6d5c80c624 | ||
|
|
6ab9906ff0 | ||
|
|
5150122a95 | ||
|
|
e5556f67f4 | ||
|
|
f6fc96400b | ||
|
|
7e3bbe6e04 | ||
|
|
13f875a04a | ||
|
|
339beb061c | ||
|
|
fee39c2f35 | ||
|
|
73adde0b9f |
@@ -1,122 +0,0 @@
|
||||
---
|
||||
name: changelog-entry
|
||||
description: Generate a new changelog entry for cibuildwheel based on all changes since the last tag. Use when updating the changelog or preparing a release.
|
||||
---
|
||||
|
||||
# Generate Changelog Entry
|
||||
|
||||
Produce a new version section at the top of `docs/changelog.md` summarizing all changes since the last git tag.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Find the last tag** — run `git tag --sort=-version:refname | head -1` to get the most recent version tag.
|
||||
2. **Gather changes** — run `git log <last-tag>..HEAD --oneline` and `git log <last-tag>..HEAD --format="%H %s"` to see all commits since that tag.
|
||||
3. **Read each merge/commit** — for substantive commits, read the full message and any linked PRs to understand the change. Use `git log <last-tag>..HEAD --format="---%n%B"` for full messages. If a commit message is ambiguous, use `gh pr view <number>` to get richer context from GitHub.
|
||||
4. **Classify and draft entries** — assign each change an emoji and write a one-line description following the style rules below. Combine related small PRs of the same type (bot dependency updates, dependabot CI action bumps, pre-commit autoupdates) into a single entry listing all PR numbers. Skip meta-changelog PRs (e.g., "Add missing CHANGELOG entries") — they don't describe user-facing changes. If a bot/dependency PR contains a substantive fix mixed in (e.g., a pip revert inside a dependency update), split it: mention the fix separately under its own category.
|
||||
5. **Determine the new version number** — inspect commits for breaking changes or new features to decide patch/minor/major bump. Ask the user if unclear.
|
||||
6. **Determine the date** — use today's date.
|
||||
7. **Insert the new section** — add it at the top of `docs/changelog.md`, right after the `# Changelog` heading and a blank line, before any existing version sections.
|
||||
|
||||
## Style Rules
|
||||
|
||||
These are non-negotiable formatting conventions derived from the existing changelog.
|
||||
|
||||
### Version heading
|
||||
|
||||
```markdown
|
||||
### v3.4.2
|
||||
```
|
||||
|
||||
`###` heading, `v` prefix, full semver.
|
||||
|
||||
### Date line
|
||||
|
||||
```markdown
|
||||
_14 May 2026_
|
||||
```
|
||||
|
||||
Italic (underscore-wrapped), day without leading zero, full month name, 4-digit year. One blank line after the date.
|
||||
|
||||
### Entry format
|
||||
|
||||
```markdown
|
||||
- <emoji> <Description> (#<PR number>)
|
||||
```
|
||||
|
||||
- Each entry is a single bullet starting with `- `.
|
||||
- Emoji immediately after the dash-space.
|
||||
- Space between emoji and description text.
|
||||
- PR number(s) in parens at end: `(#1234)` or `(#1234, #5678)`.
|
||||
- No trailing period for single-sentence entries.
|
||||
- Period at end of multi-sentence entries only.
|
||||
|
||||
### Emoji categories
|
||||
|
||||
Use exactly one emoji per entry, chosen by category:
|
||||
|
||||
| Emoji | Category | Used for |
|
||||
|-------|----------|----------|
|
||||
| 🌟 | Major feature | New platforms, significant new capabilities |
|
||||
| ✨ | Feature | New features, additions, user-visible enhancements |
|
||||
| 🐛 | Bug fix | Bug fixes |
|
||||
| 🛠 | Maintenance | Dep updates, internal improvements, behavior tweaks |
|
||||
| ⚠️ | Warning | Deprecations, breaking changes, dropped support |
|
||||
| 📚 | Docs | Documentation changes |
|
||||
| 💼 | Internal | Non-user-facing infra/tooling changes |
|
||||
| 🧪 | Tests | Test changes |
|
||||
| 🔐 | Security | Security-related changes (used in past changelogs) |
|
||||
|
||||
### Category disambiguation
|
||||
|
||||
When a change could fit multiple categories, use these tiebreakers:
|
||||
|
||||
- **CI/workflow fixes** → 💼 (not 🧪) — they fix infra, not test logic.
|
||||
- **Test suite changes** → 🧪 — only for changes to the test code itself.
|
||||
- **Diagnostic output changes** (e.g., printing more info during builds) → 🛠 (not 🐛) — they're improvements, not bug fixes.
|
||||
- **A bug fix that also changes test code** → 🐛 — the user-facing fix takes priority; test changes are implicit.
|
||||
|
||||
### Writing style
|
||||
|
||||
- **Present tense**: "Adds", "Fixes", "Updates", not "Added", "Fixed".
|
||||
- **Sentence case**: capitalize only the first word after the emoji.
|
||||
- **Link option names to docs**: `[`option-name`](https://cibuildwheel.pypa.io/en/stable/options/#option-name)`. Option anchors match the option name — verify in `docs/options.md` by searching for `{: #option-name }`.
|
||||
- Be specific about what changed and why, not just that something changed.
|
||||
|
||||
### Ordering
|
||||
|
||||
Within a version section, order entries by importance:
|
||||
|
||||
1. 🌟 entries first
|
||||
2. ⚠️ entries next
|
||||
3. ✨ entries
|
||||
4. 🐛 entries
|
||||
5. 🛠 entries
|
||||
6. 📚 entries
|
||||
7. 💼 entries
|
||||
8. 🧪 entries last
|
||||
|
||||
### Multi-line entries
|
||||
|
||||
For complex features needing explanation, use an indented italic paragraph:
|
||||
|
||||
```markdown
|
||||
- ✨ Short summary here. (#1234)
|
||||
|
||||
_Longer explanation with details and caveats._ (#1234)
|
||||
```
|
||||
|
||||
Adding a new Python beta version always has a specific longer explanation, check for a previous addition (like 3.14) for the note to use.
|
||||
|
||||
### Blank lines
|
||||
|
||||
- One blank line between version sections.
|
||||
- No blank lines between bullets within a version.
|
||||
|
||||
## Validation
|
||||
|
||||
After inserting the new section:
|
||||
|
||||
1. Check that the new section follows all style rules above.
|
||||
2. Verify PR numbers match actual PRs in the commit log.
|
||||
3. Ensure no duplicate entries — multiple commits to the same PR should produce one entry.
|
||||
4. Run a final review of the file to confirm formatting is consistent with surrounding entries.
|
||||
+22
-52
@@ -1,72 +1,42 @@
|
||||
version: 2.1
|
||||
version: 2
|
||||
|
||||
commands:
|
||||
cibw_prepare_environment:
|
||||
description: "Prepare the environment for testing."
|
||||
jobs:
|
||||
osx-python3.6:
|
||||
macos:
|
||||
xcode: 9.4.1
|
||||
environment:
|
||||
PYTHON: python3
|
||||
steps:
|
||||
- checkout
|
||||
|
||||
- run:
|
||||
name: Prepare the environment.
|
||||
command: bash .circleci/prepare.sh
|
||||
cibw_run_tests:
|
||||
description: "Runs tests, with CIBW_ENABLE=all on the main branch"
|
||||
steps:
|
||||
- run:
|
||||
name: Test
|
||||
command: |
|
||||
if [ "${CIRCLE_BRANCH}" == "main" ]; then
|
||||
echo "INFO: Exporting CIBW_ENABLE=all for main branch test run."
|
||||
export CIBW_ENABLE=all
|
||||
else
|
||||
echo "INFO: CIBW_ENABLE not set for this branch test run."
|
||||
fi
|
||||
|
||||
venv/bin/python ./bin/run_tests.py
|
||||
name: Test.
|
||||
command: venv/bin/python ./bin/run_tests.py
|
||||
no_output_timeout: 30m
|
||||
|
||||
jobs:
|
||||
osx-python312:
|
||||
macos:
|
||||
xcode: 15.4.0
|
||||
resource_class: macos.m1.medium.gen1
|
||||
environment:
|
||||
PYTHON: python3
|
||||
steps:
|
||||
- checkout
|
||||
- cibw_prepare_environment
|
||||
- cibw_run_tests
|
||||
|
||||
linux-python312:
|
||||
linux-python3.6:
|
||||
docker:
|
||||
- image: cimg/python:3.12
|
||||
- image: circleci/python:3.6
|
||||
environment:
|
||||
PYTHON: python3
|
||||
# Temporarily restrict the tests that are run on CircleCI to prevent
|
||||
# test timeouts.
|
||||
PYTEST_ADDOPTS: -k "unit_test or main_tests or test_0_basic or test_docker_images"
|
||||
steps:
|
||||
- checkout
|
||||
- setup_remote_docker
|
||||
- cibw_prepare_environment
|
||||
- cibw_run_tests
|
||||
|
||||
linux-aarch64:
|
||||
machine:
|
||||
image: default
|
||||
resource_class: arm.medium
|
||||
environment:
|
||||
PYTHON: python3
|
||||
# Temporarily restrict the tests that are run on CircleCI to prevent
|
||||
# test timeouts.
|
||||
PYTEST_ADDOPTS: -k "unit_test or main_tests or test_0_basic or test_docker_images"
|
||||
steps:
|
||||
- checkout
|
||||
- cibw_prepare_environment
|
||||
- cibw_run_tests
|
||||
- run:
|
||||
name: Prepare the environment.
|
||||
command: bash .circleci/prepare.sh
|
||||
- run:
|
||||
name: Test.
|
||||
command: venv/bin/python ./bin/run_tests.py
|
||||
no_output_timeout: 30m
|
||||
|
||||
workflows:
|
||||
version: 2
|
||||
all-tests:
|
||||
jobs:
|
||||
- osx-python312
|
||||
- linux-python312
|
||||
- linux-aarch64
|
||||
- osx-python3.6
|
||||
- linux-python3.6
|
||||
|
||||
+3
-10
@@ -1,16 +1,9 @@
|
||||
#!/usr/bin/env bash
|
||||
set -o errexit
|
||||
set -o xtrace
|
||||
|
||||
if [ "$(uname -s)" == "Darwin" ]; then
|
||||
sudo softwareupdate --install-rosetta --agree-to-license
|
||||
else
|
||||
docker run --rm --privileged docker.io/tonistiigi/binfmt:latest --install all
|
||||
fi
|
||||
|
||||
$PYTHON --version
|
||||
$PYTHON -m venv venv
|
||||
venv/bin/python -m pip install -U pip
|
||||
venv/bin/python -m pip install -e. --group test
|
||||
$PYTHON -m pip --version
|
||||
$PYTHON -m virtualenv -p $PYTHON venv
|
||||
venv/bin/python -m pip install -e ".[dev]"
|
||||
venv/bin/python -m pip freeze
|
||||
venv/bin/python --version
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
* text=auto
|
||||
|
||||
*.py diff=python
|
||||
*.md diff=markdown
|
||||
|
||||
*.svg -diff
|
||||
@@ -1,7 +0,0 @@
|
||||
# Specification files for platform maintainers
|
||||
cibuildwheel/platforms/ios.py @freakboy3742
|
||||
cibuildwheel/platforms/pyodide.py @hoodmane @ryanking13 @agriyakhetarpal
|
||||
cibuildwheel/platforms/android.py @mhsmith
|
||||
|
||||
# Constraints files
|
||||
cibuildwheel/resources/constraints-pyodide*.txt @hoodmane @ryanking13 @agriyakhetarpal
|
||||
@@ -1,31 +0,0 @@
|
||||
name: General issue
|
||||
description: Open a general issue
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Hi! If you're not sure whether your problem is a bug in cibuildwheel or something else, please consider asking a question in the Discussions tab first.
|
||||
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: Description
|
||||
|
||||
- type: input
|
||||
id: log
|
||||
attributes:
|
||||
label: Build log
|
||||
description: If relevant, please provide a link to a build log that shows the problem
|
||||
placeholder: https://...
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: input
|
||||
id: ci-config
|
||||
attributes:
|
||||
label: CI config
|
||||
description: Link to the CI config that produced the build log above. e.g. the Github Actions workflow file
|
||||
placeholder: https://...
|
||||
validations:
|
||||
required: false
|
||||
@@ -0,0 +1,7 @@
|
||||
# config file for https://github.com/bobvanderlinden/probot-auto-merge
|
||||
|
||||
# no approvals required, just the label is enough
|
||||
minApprovals: {}
|
||||
|
||||
requiredLabels:
|
||||
- merge when green
|
||||
@@ -1,13 +0,0 @@
|
||||
version: 2
|
||||
updates:
|
||||
# Maintain dependencies for GitHub Actions
|
||||
- package-ecosystem: "github-actions"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
groups:
|
||||
actions:
|
||||
patterns:
|
||||
- "*"
|
||||
cooldown:
|
||||
default-days: 7
|
||||
@@ -1,40 +0,0 @@
|
||||
name: "Copilot Setup Steps"
|
||||
|
||||
permissions: {}
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
paths:
|
||||
- .github/workflows/copilot-setup-steps.yml
|
||||
pull_request:
|
||||
paths:
|
||||
- .github/workflows/copilot-setup-steps.yml
|
||||
|
||||
jobs:
|
||||
copilot-setup-steps:
|
||||
name: Copilot Setup Steps
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
|
||||
with:
|
||||
python-version: "3.x"
|
||||
allow-prereleases: true
|
||||
|
||||
- uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
|
||||
|
||||
- name: Install tooling
|
||||
run: |
|
||||
uv tool install nox
|
||||
uv tool install prek
|
||||
|
||||
- name: Pre-install checks
|
||||
run: |
|
||||
prek prepare-hooks
|
||||
@@ -1,46 +0,0 @@
|
||||
name: Release
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
release:
|
||||
types:
|
||||
- published
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
dist:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- uses: hynek/build-and-inspect-python-package@d44ca7d91762de7a7d5436ddae667c6da6d1c3df # v2.18.0
|
||||
|
||||
publish:
|
||||
needs: [dist]
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'release' && github.event.action == 'published'
|
||||
environment:
|
||||
name: pypi
|
||||
url: https://pypi.org/p/cibuildwheel
|
||||
permissions:
|
||||
id-token: write
|
||||
attestations: write
|
||||
|
||||
steps:
|
||||
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
||||
with:
|
||||
name: Packages
|
||||
path: dist
|
||||
|
||||
- name: Generate artifact attestation for sdist and wheel
|
||||
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1
|
||||
with:
|
||||
subject-path: "dist/cibuildwheel-*"
|
||||
|
||||
- uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0
|
||||
with:
|
||||
attestations: true
|
||||
+39
-327
@@ -3,370 +3,82 @@ name: Test
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
- 2.x
|
||||
- master
|
||||
- 1.x
|
||||
pull_request:
|
||||
types:
|
||||
- opened
|
||||
- synchronize
|
||||
- reopened
|
||||
- labeled
|
||||
paths-ignore:
|
||||
- .ci*
|
||||
- bin/*
|
||||
- docs/**
|
||||
- examples/azure-pipelines-*
|
||||
- examples/ci*
|
||||
- examples/travis-ci-*
|
||||
- .pre-commit-config.yaml
|
||||
- .readthedocs.yml
|
||||
- .travis.yml
|
||||
- README.md
|
||||
- azure-pipelines.yml
|
||||
- properdocs.yml
|
||||
- noxfile.py
|
||||
- 'docs/**'
|
||||
workflow_dispatch:
|
||||
# allow manual runs on branches without a PR
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: test-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
lint:
|
||||
name: Linters (mypy, ruff, etc.)
|
||||
pre-commit:
|
||||
name: Pre-commit checks (mypy, flake8, etc.)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
|
||||
with:
|
||||
python-version: "3.x"
|
||||
- uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
|
||||
- uses: j178/prek-action@e98a699c41eb69ab013a45817a0406469a748f8d # v2.0.5
|
||||
- name: PyLint checks
|
||||
run: uvx nox -s pylint -- --output-format=github
|
||||
|
||||
sample:
|
||||
name: Generate a sample project
|
||||
needs: lint
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
|
||||
name: Install Python
|
||||
with:
|
||||
python-version: "3.x"
|
||||
- uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
|
||||
- name: Install dependencies
|
||||
run: uv sync --no-dev --group test
|
||||
- name: Generate a sample project
|
||||
run: uv run --no-sync -m test.test_projects test.test_0_basic.basic_project sample_proj
|
||||
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: sample_proj
|
||||
path: sample_proj
|
||||
- uses: actions/checkout@v2
|
||||
- uses: actions/setup-python@v2
|
||||
- uses: pre-commit/action@v2.0.0
|
||||
|
||||
test:
|
||||
name: Test on ${{ matrix.os }} (${{ matrix.python_version }}) ${{ matrix.test_select }}
|
||||
needs: sample
|
||||
name: Test cibuildwheel on ${{ matrix.os }}
|
||||
needs: pre-commit
|
||||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
# Min Python
|
||||
- os: ubuntu-latest
|
||||
python_version: '3.11'
|
||||
# Max Python
|
||||
- os: ubuntu-latest
|
||||
python_version: '3.15'
|
||||
- os: ubuntu-latest
|
||||
python_version: '3.14'
|
||||
test_select: android
|
||||
- os: ubuntu-24.04-arm
|
||||
python_version: '3.13'
|
||||
- os: windows-latest
|
||||
python_version: '3.13'
|
||||
- os: windows-11-arm
|
||||
python_version: '3.13'
|
||||
- os: macos-15-intel
|
||||
python_version: '3.13'
|
||||
- os: macos-15
|
||||
python_version: '3.13'
|
||||
- os: macos-15-intel
|
||||
python_version: '3.13'
|
||||
test_select: ios
|
||||
- os: macos-15
|
||||
python_version: '3.13'
|
||||
test_select: ios
|
||||
# Exercise iOS on a non-default simulator.
|
||||
test_runtime: 'args: --simulator "iPhone 16e,OS=18.5"'
|
||||
- os: macos-15-intel
|
||||
python_version: '3.13'
|
||||
test_select: android
|
||||
# Exercise Android on a non-default simulator
|
||||
test_runtime: 'args: --managed minVersion'
|
||||
- os: macos-15
|
||||
python_version: '3.13'
|
||||
test_select: android
|
||||
os: [ubuntu-latest, windows-latest, macos-latest]
|
||||
python_version: ['3.7']
|
||||
timeout-minutes: 180
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
|
||||
- uses: actions/checkout@v2
|
||||
- uses: actions/setup-python@v2
|
||||
name: Install Python ${{ matrix.python_version }}
|
||||
with:
|
||||
python-version: ${{ matrix.python_version }}
|
||||
allow-prereleases: true
|
||||
|
||||
- uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
|
||||
|
||||
- name: Free up disk space
|
||||
if: runner.os == 'Linux' && matrix.test_select != 'android'
|
||||
run: |
|
||||
sudo rm -rf $ANDROID_HOME/ndk /opt/hostedtoolcache/CodeQL \
|
||||
/usr/local/lib/node_modules /usr/local/share/chromium \
|
||||
/usr/local/share/powershell
|
||||
df -m
|
||||
|
||||
# for oci_container unit tests
|
||||
- name: Set up QEMU
|
||||
if: runner.os == 'Linux'
|
||||
uses: docker/setup-qemu-action@96fe6ef7f33517b61c61be40b68a1882f3264fb8 # v4.2.0
|
||||
|
||||
- name: Set up Xcode
|
||||
if: ${{ startsWith(matrix.os, 'macos-15') }}
|
||||
run: |
|
||||
# GitHub recommends explicitly selecting the desired Xcode version
|
||||
sudo xcode-select --switch /Applications/Xcode_16.4.app
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --no-dev --group test
|
||||
|
||||
- uses: joerick/pr-labels-action@0543b277721e852d821c6738d449f2f4dea03d5f # v1.0.9
|
||||
|
||||
- name: Set CIBW_ENABLE
|
||||
shell: bash
|
||||
run: |
|
||||
if [[ "$GITHUB_REF_NAME" == "main" ]]; then
|
||||
CIBW_ENABLE=all
|
||||
else
|
||||
# get the default CIBW_ENABLE value from the test module
|
||||
CIBW_ENABLE=$(uv run --no-sync python -c 'import sys, test.utils as c; sys.stdout.write(c.DEFAULT_CIBW_ENABLE)')
|
||||
python -m pip install -e ".[dev]"
|
||||
|
||||
# if this is a PR, check for labels
|
||||
if [[ -n "$GITHUB_PR_LABEL_CI_PYPY" ]]; then
|
||||
CIBW_ENABLE+=" pypy"
|
||||
fi
|
||||
if [[ -n "$GITHUB_PR_LABEL_CI_GRAALPY" ]]; then
|
||||
CIBW_ENABLE+=" graalpy"
|
||||
fi
|
||||
if [[ -n "$GITHUB_PR_LABEL_CI_PYPY_EOL" ]]; then
|
||||
CIBW_ENABLE+=" pypy-eol"
|
||||
fi
|
||||
fi
|
||||
echo "CIBW_ENABLE=${CIBW_ENABLE}" >> $GITHUB_ENV
|
||||
- name: Sample build
|
||||
if: "contains(github.event.pull_request.labels.*.name, 'CI: Sample build')"
|
||||
run: |
|
||||
python bin/sample_build.py
|
||||
|
||||
- name: Download a sample project
|
||||
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
||||
with:
|
||||
name: sample_proj
|
||||
path: sample_proj
|
||||
|
||||
- name: Run a sample build (GitHub Action)
|
||||
uses: ./
|
||||
with:
|
||||
package-dir: sample_proj
|
||||
output-dir: wheelhouse
|
||||
- name: Get some sample wheels
|
||||
run: |
|
||||
python -m test.test_projects test.test_0_basic.basic_project sample_proj
|
||||
cibuildwheel --output-dir wheelhouse sample_proj
|
||||
env:
|
||||
CIBW_ARCHS_MACOS: x86_64 universal2 arm64
|
||||
CIBW_BUILD_FRONTEND: ${{ matrix.test_select && 'build' || 'build[uv]' }}
|
||||
CIBW_PLATFORM: ${{ matrix.test_select }}
|
||||
CIBW_TEST_RUNTIME: ${{ matrix.test_runtime }}
|
||||
|
||||
- name: Run a sample build (GitHub Action, only)
|
||||
uses: ./
|
||||
if: matrix.test_select == ''
|
||||
- uses: actions/upload-artifact@v2
|
||||
with:
|
||||
package-dir: sample_proj
|
||||
output-dir: wheelhouse_only
|
||||
only: cp313-${{ runner.os == 'Linux' && (runner.arch == 'ARM64' && 'manylinux_aarch64' || 'manylinux_x86_64') || (runner.os == 'Windows' && 'win_amd64' || 'macosx_x86_64') }}
|
||||
|
||||
- name: Create custom configuration file
|
||||
shell: bash
|
||||
run: |
|
||||
cat > sample_proj/cibw.toml <<EOF
|
||||
[tool.cibuildwheel]
|
||||
# Only build on CPython 3.13 on native arch
|
||||
archs = ["native"]
|
||||
build = "cp313-*"
|
||||
# Skip musllinux
|
||||
skip = "*-musllinux*"
|
||||
EOF
|
||||
|
||||
- name: Run a sample build (GitHub Action, config-file)
|
||||
uses: ./
|
||||
env:
|
||||
CIBW_PLATFORM: ${{ matrix.test_select }}
|
||||
CIBW_TEST_RUNTIME: ${{ matrix.test_runtime }}
|
||||
with:
|
||||
package-dir: sample_proj
|
||||
output-dir: wheelhouse_config_file
|
||||
config-file: sample_proj/cibw.toml
|
||||
|
||||
- name: Check Action artifacts
|
||||
shell: bash
|
||||
run: |
|
||||
test $(find wheelhouse -name '*.whl' | wc -l) -ge 1
|
||||
test $(find wheelhouse_config_file -name '*.whl' | wc -l) -eq 1
|
||||
|
||||
- name: Check Action artifacts (native build only)
|
||||
if: matrix.test_select == ''
|
||||
shell: bash
|
||||
run: |
|
||||
test $(find wheelhouse_only -name '*.whl' | wc -l) -eq 1
|
||||
|
||||
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
|
||||
path: wheelhouse/*.whl
|
||||
name: sample_wheels
|
||||
path: wheelhouse
|
||||
|
||||
- name: Test cibuildwheel
|
||||
env:
|
||||
CIBW_TEST_RUNTIME: ${{ matrix.test_runtime }}
|
||||
run: uv run --no-sync bin/run_tests.py --test-select=${{ matrix.test_select || 'native' }} ${{ (runner.os == 'Linux' && runner.arch == 'X64') && '--run-podman' || '' }}
|
||||
|
||||
emulated-archs:
|
||||
name: Get qemu emulated architectures
|
||||
needs: lint
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
archs: ${{ steps.archs.outputs.archs }}
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
|
||||
with:
|
||||
python-version: "3.x"
|
||||
- uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
|
||||
- name: Install dependencies
|
||||
run: uv sync --no-dev --group test
|
||||
- name: Get qemu emulated architectures
|
||||
id: archs
|
||||
run: |
|
||||
OUTPUT=$(uv run --no-sync python -c "from json import dumps; from test.utils import EMULATED_ARCHS; print(dumps(EMULATED_ARCHS))")
|
||||
echo "${OUTPUT}"
|
||||
echo "archs=${OUTPUT}" >> "$GITHUB_OUTPUT"
|
||||
python ./bin/run_tests.py
|
||||
|
||||
test-emulated:
|
||||
name: Test Linux ${{ matrix.arch }} using qemu
|
||||
needs: emulated-archs
|
||||
name: Test emulated cibuildwheel using qemu
|
||||
needs: pre-commit
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 180
|
||||
strategy:
|
||||
matrix:
|
||||
arch: ${{ fromJSON(needs.emulated-archs.outputs.archs) }}
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
|
||||
with:
|
||||
python-version: "3.x"
|
||||
- uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
|
||||
- uses: actions/checkout@v2
|
||||
- uses: actions/setup-python@v2
|
||||
- name: Install dependencies
|
||||
run: uv sync --no-dev --group test
|
||||
run: |
|
||||
python -m pip install -r requirements-dev.txt
|
||||
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@96fe6ef7f33517b61c61be40b68a1882f3264fb8 # v4.2.0
|
||||
id: qemu
|
||||
uses: docker/setup-qemu-action@v1
|
||||
with:
|
||||
platforms: all
|
||||
|
||||
- name: Run the emulation tests
|
||||
env:
|
||||
MATRIX_ARCH: ${{ matrix.arch }}
|
||||
run: uv run --no-sync pytest --run-emulation "$MATRIX_ARCH" test/test_emulation.py
|
||||
|
||||
test-pyodide:
|
||||
name: Test pyodide
|
||||
needs: sample
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 180
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
|
||||
name: Install Python 3.12
|
||||
with:
|
||||
python-version: '3.12'
|
||||
- uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --no-dev --group test
|
||||
|
||||
- name: Download a sample project
|
||||
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
||||
with:
|
||||
name: sample_proj
|
||||
path: sample_proj
|
||||
|
||||
- name: Run a sample build (GitHub Action)
|
||||
uses: ./
|
||||
with:
|
||||
package-dir: sample_proj
|
||||
output-dir: wheelhouse
|
||||
env:
|
||||
CIBW_PLATFORM: pyodide
|
||||
CIBW_ENABLE: pyodide-prerelease
|
||||
|
||||
- name: Run tests with 'CIBW_PLATFORM' set to 'pyodide'
|
||||
run: uv run --no-sync ./bin/run_tests.py
|
||||
env:
|
||||
CIBW_PLATFORM: pyodide
|
||||
CIBW_ENABLE: pyodide-prerelease
|
||||
|
||||
test-uv-extras:
|
||||
name: Test uv extra on ${{ matrix.os }} ${{ matrix.test_select }}
|
||||
needs: sample
|
||||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
test_select: android
|
||||
- os: windows-latest
|
||||
- os: windows-11-arm
|
||||
- os: macos-15-intel
|
||||
- os: macos-15
|
||||
- os: macos-15-intel
|
||||
test_select: android
|
||||
- os: macos-15
|
||||
test_select: android
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Download a sample project
|
||||
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
||||
with:
|
||||
name: sample_proj
|
||||
path: sample_proj
|
||||
- name: Run a sample build (GitHub Action)
|
||||
uses: ./
|
||||
with:
|
||||
extras: uv
|
||||
package-dir: sample_proj
|
||||
output-dir: wheelhouse
|
||||
env:
|
||||
CIBW_ARCHS_MACOS: x86_64 universal2 arm64
|
||||
CIBW_BUILD: cp314-*
|
||||
CIBW_BUILD_FRONTEND: 'build[uv]'
|
||||
CIBW_PLATFORM: ${{ matrix.test_select }}
|
||||
run: |
|
||||
pytest --run-emulation test/test_emulation.py
|
||||
|
||||
@@ -1,60 +1,30 @@
|
||||
name: Update dependencies
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- '.github/workflows/update-dependencies.yml'
|
||||
- 'bin/update_pythons.py'
|
||||
- 'bin/update_docker.py'
|
||||
- 'bin/update_virtualenv.py'
|
||||
- 'bin/projects.py'
|
||||
- 'docs/data/projects.yml'
|
||||
- 'noxfile.py'
|
||||
workflow_dispatch:
|
||||
schedule:
|
||||
- cron: '0 6 * * 1' # "At 06:00 on Monday."
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
update-dependencies:
|
||||
name: Update dependencies
|
||||
if: github.repository_owner == 'pypa' || github.event_name != 'schedule'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
environment: ${{ github.ref == 'refs/heads/main' && github.repository == 'pypa/cibuildwheel' && 'update-dependencies-workflow' || '' }}
|
||||
steps:
|
||||
|
||||
# we use this step to grab a Github App auth token, so that PRs generated by this workflow
|
||||
# run the GHA tests.
|
||||
- uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
|
||||
id: generate-token
|
||||
if: github.ref == 'refs/heads/main' && github.repository == 'pypa/cibuildwheel'
|
||||
- uses: actions/checkout@v2
|
||||
- uses: actions/setup-python@v2
|
||||
name: Install Python 3.9
|
||||
with:
|
||||
client-id: ${{ secrets.CIBUILDWHEEL_BOT_APP_ID }}
|
||||
private-key: ${{ secrets.CIBUILDWHEEL_BOT_APP_PRIVATE_KEY }}
|
||||
permission-contents: write
|
||||
permission-pull-requests: write
|
||||
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
|
||||
with:
|
||||
python-version: "3.14"
|
||||
|
||||
python-version: 3.9
|
||||
architecture: x64
|
||||
- name: Install dependencies
|
||||
run: python -m pip install ".[dev]"
|
||||
- name: "Run update: dependencies"
|
||||
run: uvx nox --force-color -s update_constraints
|
||||
run: python ./bin/update_dependencies.py
|
||||
- name: "Run update: python configs"
|
||||
run: uvx nox --force-color -s update_pins
|
||||
- name: "Run update: docs user projects"
|
||||
run: uvx nox --force-color -s update_proj -- --auth=${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
run: python ./bin/update_pythons.py --force
|
||||
- name: Create Pull Request
|
||||
if: github.ref == 'refs/heads/main' && github.repository == 'pypa/cibuildwheel'
|
||||
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
|
||||
if: github.ref == 'refs/heads/master' && github.repository == 'pypa/cibuildwheel'
|
||||
uses: peter-evans/create-pull-request@v3
|
||||
with:
|
||||
commit-message: Update dependencies
|
||||
title: '[Bot] Update dependencies'
|
||||
@@ -63,11 +33,4 @@ jobs:
|
||||
|
||||
PR generated by "Update dependencies" [workflow](https://github.com/${{github.repository}}/actions/runs/${{github.run_id}}).
|
||||
branch: update-dependencies-pr
|
||||
sign-commits: true
|
||||
token: ${{ steps.generate-token.outputs.token }}
|
||||
delete-branch: true
|
||||
labels: |
|
||||
CI: GraalPy
|
||||
CI: PyPy
|
||||
CI: PyPy EoL
|
||||
dependencies
|
||||
|
||||
@@ -1,32 +0,0 @@
|
||||
name: Update the vX.Y tag
|
||||
|
||||
on:
|
||||
release:
|
||||
types: [released]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
TAG_NAME:
|
||||
description: 'Tag name that the major.minor tag will point to'
|
||||
required: true
|
||||
|
||||
env:
|
||||
TAG_NAME: ${{ github.event.inputs.TAG_NAME || github.event.release.tag_name }}
|
||||
|
||||
jobs:
|
||||
update_tag:
|
||||
name: Update the major.minor tag to include the ${{ github.event.inputs.TAG_NAME || github.event.release.tag_name }} changes
|
||||
environment:
|
||||
name: releaseNewActionVersion
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Update the ${{ env.TAG_NAME }} tag
|
||||
id: update-major-minor-tag
|
||||
uses: joerick/update-vX.Y-tag-action@c4cefb60c33b82e4aa73a469e4acf30ee48d5812 # v1.0.2
|
||||
with:
|
||||
source-tag: ${{ env.TAG_NAME }}
|
||||
@@ -1,24 +0,0 @@
|
||||
name: zizmor - GitHub Actions Security Analysis
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ["main"]
|
||||
pull_request:
|
||||
branches: ["**"]
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
zizmor:
|
||||
name: Run zizmor 🌈
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
security-events: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Run zizmor 🌈
|
||||
uses: zizmorcore/zizmor-action@192e21d79ab29983730a13d1382995c2307fbcaa # v0.5.7
|
||||
+5
-13
@@ -84,7 +84,9 @@ celerybeat-schedule
|
||||
|
||||
# virtualenv
|
||||
.venv
|
||||
venv*/
|
||||
venv/
|
||||
venv3/
|
||||
venv2/
|
||||
ENV/
|
||||
env/
|
||||
env2/
|
||||
@@ -110,15 +112,5 @@ all_known_setup.yaml
|
||||
# mkdocs
|
||||
site/
|
||||
|
||||
# PyCharm
|
||||
.idea/
|
||||
|
||||
# Lockfiles
|
||||
*.lock
|
||||
*.pylock
|
||||
|
||||
# OS files
|
||||
.DS_Store
|
||||
|
||||
# This file should be a symlink or contain "See @AGENTS.md"
|
||||
CLAUDE.md
|
||||
# Virtual environments
|
||||
venv*
|
||||
|
||||
+2
-45
@@ -1,5 +1,5 @@
|
||||
linux:
|
||||
image: python:3.13
|
||||
image: python:3.8
|
||||
services:
|
||||
- name: docker:dind
|
||||
entrypoint: ["env", "-u", "DOCKER_HOST"]
|
||||
@@ -9,50 +9,7 @@ linux:
|
||||
DOCKER_DRIVER: overlay2
|
||||
# See https://github.com/docker-library/docker/pull/166
|
||||
DOCKER_TLS_CERTDIR: ""
|
||||
|
||||
# skip all but the basic tests
|
||||
# (comment the below line in a PR to debug a Gitlab-specific issue)
|
||||
PYTEST_ADDOPTS: -k "unit_test or test_0_basic" --suppress-no-test-exit-code
|
||||
rules:
|
||||
- if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH =~ /^gitlab/'
|
||||
variables:
|
||||
CIBW_ENABLE: "all"
|
||||
script:
|
||||
- curl -sSL https://get.docker.com/ | sh
|
||||
- docker run --rm --privileged docker.io/tonistiigi/binfmt:latest --install all
|
||||
- python -m pip install -U pip
|
||||
- python -m pip install -e. pytest-custom-exit-code --group test
|
||||
- python -m pip install -e ".[dev]"
|
||||
- python ./bin/run_tests.py
|
||||
|
||||
windows:
|
||||
variables:
|
||||
PYTEST_ADDOPTS: -k "unit_test or test_0_basic" --suppress-no-test-exit-code
|
||||
rules:
|
||||
- if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH =~ /^gitlab/'
|
||||
variables:
|
||||
# Everything except graalpy. GraalPy is JVM-based and very slow to
|
||||
# start; on the small (2-core) GitLab SaaS Windows runners,
|
||||
# virtualenv's interpreter query for graalpy.exe times out, failing
|
||||
# the build. (Other platforms run the full "all" group.)
|
||||
CIBW_ENABLE: "cpython-prerelease pypy pypy-eol pyodide-eol pyodide-prerelease"
|
||||
script:
|
||||
- python -m pip install -U pip
|
||||
- python -m pip install -e. pytest-custom-exit-code --group test
|
||||
- python bin\run_tests.py
|
||||
tags:
|
||||
- saas-windows-medium-amd64
|
||||
|
||||
macos:
|
||||
image: macos-15-xcode-16
|
||||
variables:
|
||||
PYTEST_ADDOPTS: -k "unit_test or test_0_basic" --suppress-no-test-exit-code
|
||||
rules:
|
||||
- if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH =~ /^gitlab/'
|
||||
variables:
|
||||
CIBW_ENABLE: "all"
|
||||
script:
|
||||
- python3 -m pip install -U pip
|
||||
- python3 -m pip install -e. pytest-custom-exit-code --group test
|
||||
- python3 ./bin/run_tests.py
|
||||
tags:
|
||||
- saas-macos-medium-m1
|
||||
|
||||
+44
-81
@@ -1,10 +1,7 @@
|
||||
ci:
|
||||
autoupdate_schedule: monthly
|
||||
exclude: "^cibuildwheel/resources/ios-support/(?!make_cross_venv\\.py)"
|
||||
|
||||
repos:
|
||||
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: 3e8a8703264a2f4a69428a0aa4dcb512790b2c8c # frozen: v6.0.0
|
||||
rev: v3.4.0
|
||||
hooks:
|
||||
- id: check-case-conflict
|
||||
- id: check-merge-conflict
|
||||
@@ -15,94 +12,60 @@ repos:
|
||||
exclude: (cibuildwheel/resources/pinned_docker_images.cfg)|(.svg$)
|
||||
- id: mixed-line-ending
|
||||
- id: trailing-whitespace
|
||||
exclude: ^cibuildwheel/resources/android/android.patch$
|
||||
|
||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||
rev: c59bba8fb259db0fec2bbb77ad8ba51ea7341b56 # frozen: v0.15.20
|
||||
- repo: https://github.com/PyCQA/isort
|
||||
rev: 5.8.0
|
||||
hooks:
|
||||
- id: ruff-check
|
||||
args: ["--fix"]
|
||||
- id: ruff-format
|
||||
- id: isort
|
||||
|
||||
- repo: https://github.com/psf/black
|
||||
rev: 21.4b2
|
||||
hooks:
|
||||
- id: black
|
||||
|
||||
- repo: https://github.com/asottile/setup-cfg-fmt
|
||||
rev: v1.17.0
|
||||
hooks:
|
||||
- id: setup-cfg-fmt
|
||||
stages: [manual]
|
||||
|
||||
- repo: https://github.com/pre-commit/mirrors-mypy
|
||||
rev: d2823d321df3af8f878f7ee3414dc94d037145b9 # frozen: v2.1.0
|
||||
rev: v0.812
|
||||
hooks:
|
||||
- id: mypy
|
||||
name: mypy 3.11 on cibuildwheel/
|
||||
args: ["--python-version=3.11"]
|
||||
exclude: ^cibuildwheel/resources/android/_cross_venv.py$ # Requires Python 3.13 or later
|
||||
additional_dependencies: &mypy-dependencies
|
||||
- bracex
|
||||
- build
|
||||
- dependency-groups>=1.2
|
||||
- humanize
|
||||
- nox>=2025.2.9
|
||||
- packaging
|
||||
- pygithub
|
||||
- pytest
|
||||
- rich
|
||||
- tomli_w
|
||||
- types-certifi
|
||||
- types-click
|
||||
- types-jinja2
|
||||
- types-pyyaml
|
||||
- types-requests
|
||||
- types-setuptools
|
||||
- uv
|
||||
- validate-pyproject
|
||||
exclude: ^(bin/|cibuildwheel/resources/).*py$
|
||||
additional_dependencies: [packaging]
|
||||
args: ["--python-version=3.6", "--ignore-missing-imports", "--scripts-are-modules"]
|
||||
- id: mypy
|
||||
name: mypy 3.14
|
||||
args: ["--python-version=3.14"]
|
||||
additional_dependencies: *mypy-dependencies
|
||||
name: mypy 3.7+ on bin/
|
||||
files: ^bin/.*py$
|
||||
additional_dependencies: [packaging]
|
||||
args: ["--python-version=3.7", "--ignore-missing-imports", "--scripts-are-modules"]
|
||||
|
||||
- repo: https://github.com/shellcheck-py/shellcheck-py
|
||||
rev: 745eface02aef23e168a8afb6b5737818efbea95 # frozen: v0.11.0.1
|
||||
- repo: https://github.com/asottile/pyupgrade
|
||||
rev: v2.14.0
|
||||
hooks:
|
||||
- id: shellcheck
|
||||
- id: pyupgrade
|
||||
name: PyUpgrade 3.6+
|
||||
args: ["--py36-plus"]
|
||||
exclude: ^bin/
|
||||
- id: pyupgrade
|
||||
name: PyUpgrade 3.7+ on bin
|
||||
exclude: ^(cibuildwheel|unit_test|test)/
|
||||
args: ["--py37-plus"]
|
||||
|
||||
- repo: https://github.com/PyCQA/flake8
|
||||
rev: 3.9.1
|
||||
hooks:
|
||||
- id: flake8
|
||||
exclude: cibuildwheel/resources/
|
||||
additional_dependencies: [flake8-bugbear]
|
||||
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: disallow-caps
|
||||
name: Disallow improper capitalization
|
||||
language: pygrep
|
||||
entry: PyBind|Numpy|Cmake|Github|PyTest
|
||||
types: [markdown]
|
||||
exclude: ^docs/working-examples\.md$ # Autogenerated
|
||||
- id: cog
|
||||
name: Cog the README
|
||||
language: python
|
||||
pass_filenames: false
|
||||
entry: cog -c -P -r -I ./bin README.md
|
||||
files: '^(README\.md|docs/changelog\.md|docs/options\.md|bin/readme.*)$'
|
||||
additional_dependencies: [cogapp>=3.5]
|
||||
|
||||
- repo: https://github.com/henryiii/flake8-lazy
|
||||
rev: 5fed9a818f2583f0403aed779bf2b6c9aa8d25f4 # frozen: v0.8.5
|
||||
hooks:
|
||||
- id: flake8-lazy
|
||||
args: ["--apply=set"]
|
||||
files: '^cibuildwheel'
|
||||
exclude: "^cibuildwheel/resources/"
|
||||
|
||||
- repo: https://github.com/codespell-project/codespell
|
||||
rev: 2ccb47ff45ad361a21071a7eedda4c37e6ae8c5a # frozen: v2.4.2
|
||||
hooks:
|
||||
- id: codespell
|
||||
args: ["-w"]
|
||||
exclude: ^docs/working-examples\.md$ # Autogenerated
|
||||
|
||||
|
||||
- repo: https://github.com/python-jsonschema/check-jsonschema
|
||||
rev: 1a4bb160cab6417b3045e1b37b6b72449243e658 # frozen: 0.37.4
|
||||
hooks:
|
||||
- id: check-dependabot
|
||||
- id: check-github-actions
|
||||
- id: check-github-workflows
|
||||
- id: check-gitlab-ci
|
||||
- id: check-readthedocs
|
||||
- id: check-jsonschema
|
||||
name: Check projects
|
||||
args: [--schemafile, docs/data/projects.schema.json]
|
||||
files: '^docs/data/projects.yml$'
|
||||
- id: check-metaschema
|
||||
files: '^docs/data/projects.schema.json$'
|
||||
entry: PyBind|Numpy|Cmake|Github
|
||||
types:
|
||||
- markdown
|
||||
|
||||
+10
-7
@@ -2,10 +2,13 @@
|
||||
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
|
||||
version: 2
|
||||
|
||||
build:
|
||||
os: ubuntu-24.04
|
||||
commands:
|
||||
- asdf plugin add uv
|
||||
- asdf install uv latest
|
||||
- asdf global uv latest
|
||||
- NO_COLOR=1 uv run --python 3.14 --managed-python --no-dev --group docs properdocs build --strict --site-dir $READTHEDOCS_OUTPUT/html
|
||||
mkdocs:
|
||||
configuration: mkdocs.yml
|
||||
|
||||
python:
|
||||
version: 3.8
|
||||
install:
|
||||
- method: pip
|
||||
path: .
|
||||
extra_requirements:
|
||||
- docs
|
||||
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
os: linux
|
||||
dist: focal
|
||||
language: python
|
||||
|
||||
branches:
|
||||
only:
|
||||
- master
|
||||
|
||||
jobs:
|
||||
include:
|
||||
- name: Linux | x86_64 + i686 | Python 3.6
|
||||
python: 3.6
|
||||
services: docker
|
||||
env: PYTHON=python
|
||||
|
||||
- name: Linux | arm64 | Python 3.6
|
||||
python: 3.6
|
||||
services: docker
|
||||
arch: arm64
|
||||
env: PYTHON=python
|
||||
|
||||
- name: Linux | ppc64le | Python 3.6
|
||||
python: 3.6
|
||||
services: docker
|
||||
arch: ppc64le
|
||||
env:
|
||||
- PYTHON=python
|
||||
# skip test_manylinuxXXXX_only, it uses too much disk space
|
||||
# c.f. https://travis-ci.community/t/running-out-of-disk-space-quota-when-using-docker-on-ppc64le/11634
|
||||
- PYTEST_ADDOPTS='-k "not test_manylinuxXXXX_only"'
|
||||
|
||||
- name: Windows | x86_64 | Python 3.6
|
||||
os: windows
|
||||
language: shell
|
||||
before_install:
|
||||
- choco upgrade python3 -y --version 3.6.8 --limit-output
|
||||
# Update root certificates to fix SSL error; see http://www.chawn.com/RootCerts.htm
|
||||
- powershell "md C:\temp\certs; CertUtil -generateSSTFromWU C:\temp\certs\RootStore.sst; Get-ChildItem -Path C:\\temp\certs\Rootstore.sst | Import-Certificate -CertStoreLocation Cert:\\LocalMachine\\Root\\ | out-null"
|
||||
env:
|
||||
- PYTHON=C:\\Python36\\python
|
||||
|
||||
- name: Linux | s390x | Python 3.6
|
||||
python: 3.6
|
||||
services: docker
|
||||
arch: s390x
|
||||
env: PYTHON=python
|
||||
|
||||
install: $PYTHON -m pip install -e ".[dev]" pytest-custom-exit-code
|
||||
|
||||
script: $PYTHON ./bin/run_tests.py
|
||||
@@ -1,57 +0,0 @@
|
||||
# cibuildwheel — Agent Notes
|
||||
|
||||
## Always run
|
||||
|
||||
- `prek -a` should be run after changes to reformat, lint, and type check
|
||||
|
||||
## Developer commands
|
||||
- `uv run pytest unit_test` — Quick run for unit tests
|
||||
- `nox -s tests` — run doctests + unit tests + integration tests (slow, >30 min, requires system Python installs on macOS).
|
||||
- `nox -s tests -- unit_test` — fast unit tests only.
|
||||
- `nox -s tests -- test -k before_build` — single integration test/file via pytest `-k`.
|
||||
- `nox -s lint` — run all linters (pre-commit/prek).
|
||||
- `nox -s pylint` — run pylint separately (not in pre-commit).
|
||||
- `nox -s docs` — properdocs serve (interactive) or build (non-interactive).
|
||||
- Set up local dev env at `.venv`: `uv sync` (dependency groups used).
|
||||
|
||||
## Project layout
|
||||
- `cibuildwheel/` — main package. Entry point: `cibuildwheel.__main__:main`.
|
||||
- `test/` — **integration tests** (expensive, run actual wheel builds).
|
||||
- `unit_test/` — **unit tests** (fast, no wheel builds).
|
||||
- `bin/` — maintainer scripts (update pins, generate README tables, schema, etc.).
|
||||
- `docs/` — properdocs (MkDocs fork) source.
|
||||
|
||||
## Testing specifics
|
||||
- Three test suites exist, run in this order by `bin/run_tests.py`:
|
||||
1. `pytest cibuildwheel` — doctests.
|
||||
2. `pytest unit_test [...]` — unit tests.
|
||||
3. `pytest test [...]` — integration tests (split into `serial` and `not serial` runs).
|
||||
- Serial integration tests **must not** run in parallel; non-serial use pytest-xdist by default.
|
||||
- Custom pytest options:
|
||||
- `--run-docker` (unit_test + test): run OCI container tests. Linux only.
|
||||
- `--run-podman`: run podman tests (Linux).
|
||||
- `--run-emulation` (test): run QEMU emulation tests (e.g., `--run-emulation aarch64`).
|
||||
- `--platform linux` (test): force integration tests to target Linux container builds even on macOS/Windows.
|
||||
- `--enable` (test): sets `CIBW_ENABLE` env var (e.g., `pypy`, `graalpy`).
|
||||
- Integration tests auto-set a default `CIBW_ENABLE` if the env var is absent.
|
||||
- The `build_frontend_env` fixture parameterizes over `pip`, `build`, `build[uv]`, `uv` and skips unsupported combos per platform.
|
||||
- Some integration tests require system Python.org installs on macOS; missing them prints a download URL in the error.
|
||||
- iOS/Android/pyodide tests have dedicated pytest marks (`ios`, `android`, `pyodide`) and need platform-specific runners/simulators.
|
||||
|
||||
## Lint / typecheck
|
||||
- Ruff (lint + format) and mypy run via pre-commit. Pylint runs separately via `nox -s pylint`.
|
||||
- Mypy is strict (`strict = true`) and targets Python 3.11 for the package, 3.14 for a second check in pre-commit.
|
||||
- Ruff config in `pyproject.toml` (`line-length = 100`).
|
||||
- Python 3.11 is the minimum supported version for the package itself.
|
||||
|
||||
## Generated / maintained files
|
||||
- `README.md` contains two **cog-generated** tables (options table, changelog preview). Pre-commit runs `cog -c -P -r -I ./bin README.md`. Edit the source scripts (`bin/readme_*.py`) or the upstream files (`docs/options.md`, `docs/changelog.md`) — do not hand-edit the generated blocks. Note the identifier is _not_ cog generated, and can be edited.
|
||||
- `cibuildwheel/resources/cibuildwheel.schema.json` is generated by `bin/generate_schema.py` (run via `nox -s generate_schema`).
|
||||
- `cibuildwheel/resources/constraints-*.txt` are generated via `nox -s update_constraints`.
|
||||
- `cibuildwheel/resources/pinned_docker_images.cfg` and other resource files are updated via `nox -s update_pins`.
|
||||
|
||||
## CI / release quirks
|
||||
- CI uses `uv sync --no-dev --group test` for test installs, then `uv run --no-sync` to execute.
|
||||
- The release workflow uses `hynek/build-and-inspect-python-package` for dist building.
|
||||
- `test.yml` skips unrelated paths to avoid burning CI time on docs-only changes.
|
||||
- A sample project artifact is built once and downloaded by downstream test jobs to avoid redundant work.
|
||||
@@ -1,9 +1,12 @@
|
||||
This is a summary of the host Python versions and platforms covered by the different CI platforms:
|
||||
This is a summary of the Python versions and platforms covered by the different CI platforms:
|
||||
|
||||
| | 3.11 | 3.12 | 3.13 | 3.14 |
|
||||
|---------|----------------------------------|-----------------------|----------------|----------------|
|
||||
| Linux | Azure Pipelines / GitHub Actions | CircleCI¹ / GitLab¹ | GitHub Actions | GitHub Actions |
|
||||
| macOS | Azure Pipelines | CircleCI¹ / GitLab¹ | GitHub Actions | |
|
||||
| Windows | Azure Pipelines | GitLab¹ | GitHub Actions | |
|
||||
| | 3.6 | 3.7 | 3.8 |
|
||||
|----------|------------------------------|------------------------------------------|------------------|
|
||||
| Linux | Travis CI / CircleCI | AppVeyor² / GitHub Actions | Azure Pipelines |
|
||||
| macOS | CircleCI | AppVeyor² / Travis CI¹ / GitHub Actions | Azure Pipelines |
|
||||
| Windows | Travis CI / Azure Pipelines | AppVeyor² / GitHub Actions | Azure Pipelines |
|
||||
|
||||
> ¹ Runs a reduced set of tests to reduce CI load
|
||||
> ¹ Python version not really pinned, but dependent on the (default) version of image used.
|
||||
> ² AppVeyor only runs the "basic" test to reduce load.
|
||||
|
||||
Non-x86 architectures are covered on Travis CI using Python 3.6.
|
||||
|
||||
@@ -1,9 +1,28 @@
|
||||
Copyright 2017, Joe Rickerby and contributors. All rights reserved.
|
||||
This project is licensed under the 'BSD 2-clause license'.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
||||
Copyright (c) 2017-2019, Joe Rickerby and contributors. All rights reserved.
|
||||
|
||||
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are met:
|
||||
|
||||
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
|
||||
1. Redistributions of source code must retain the above copyright notice, this
|
||||
list of conditions and the following disclaimer.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
2. Redistributions in binary form must reproduce the above copyright notice,
|
||||
this list of conditions and the following disclaimer in the documentation
|
||||
and/or other materials provided with the distribution.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
||||
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
||||
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
|
||||
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
||||
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
|
||||
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
|
||||
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
||||
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
|
||||
Get-pip and Pip (bundled in cibuildwheel/resources) are licensed under the MIT
|
||||
license. See https://github.com/pypa/get-pip/blob/master/LICENSE.txt
|
||||
|
||||
+8
-127
@@ -9,18 +9,6 @@ inputs:
|
||||
description: 'Folder to place the outputs in, defaults to "wheelhouse"'
|
||||
required: false
|
||||
default: wheelhouse
|
||||
config-file:
|
||||
description: 'File containing the config, defaults to {package}/pyproject.toml'
|
||||
required: false
|
||||
default: ''
|
||||
only:
|
||||
description: 'Build a specific wheel only. No need for arch/platform if this is set'
|
||||
required: false
|
||||
default: ''
|
||||
extras:
|
||||
description: 'Comma-separated list of extras to install'
|
||||
required: false
|
||||
default: ''
|
||||
branding:
|
||||
icon: package
|
||||
color: yellow
|
||||
@@ -28,120 +16,13 @@ branding:
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
|
||||
id: python
|
||||
with:
|
||||
python-version: "3.11 - 3.14"
|
||||
update-environment: false
|
||||
|
||||
- id: cibw
|
||||
run: |
|
||||
# Install cibuildwheel and build the command line
|
||||
"$PYTHON" -u << "EOF"
|
||||
import os
|
||||
import shlex
|
||||
import shutil
|
||||
import sys
|
||||
import venv
|
||||
|
||||
from pathlib import Path
|
||||
from subprocess import run
|
||||
|
||||
EXTRAS = set(e.strip() for e in os.environ.get("INPUT_EXTRAS", "").split(",") if e.strip())
|
||||
|
||||
|
||||
class EnvBuilder(venv.EnvBuilder):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
|
||||
def setup_scripts(self, context):
|
||||
pass
|
||||
|
||||
def post_setup(self, context):
|
||||
super().post_setup(context)
|
||||
self.bin_path = Path(context.env_exe).parent
|
||||
install_spec = os.environ["GITHUB_ACTION_PATH"]
|
||||
if EXTRAS:
|
||||
install_spec += f"[{','.join(sorted(EXTRAS))}]"
|
||||
run([sys.executable, "-m", "pip", "--python", context.env_exe, "install", install_spec], check=True)
|
||||
|
||||
|
||||
print("::group::Install cibuildwheel")
|
||||
venv_path = Path(os.environ["RUNNER_TEMP"]) / "cibw"
|
||||
if venv_path.exists():
|
||||
shutil.rmtree(venv_path)
|
||||
|
||||
builder = EnvBuilder()
|
||||
builder.create(venv_path)
|
||||
exposed_binaries = {"cibuildwheel"}
|
||||
|
||||
if "uv" in EXTRAS:
|
||||
exposed_binaries.add("uv")
|
||||
clean_bin_path = builder.bin_path.parent / f"{builder.bin_path.name}.clean"
|
||||
clean_bin_path.mkdir()
|
||||
|
||||
for path in list(builder.bin_path.iterdir()):
|
||||
if path.stem in exposed_binaries:
|
||||
try:
|
||||
os.symlink(path, clean_bin_path / path.name)
|
||||
except OSError:
|
||||
shutil.copy2(path, clean_bin_path / path.name)
|
||||
|
||||
cibw_bin = [p for p in builder.bin_path.glob("cibuildwheel*") if p.stem == "cibuildwheel"][0]
|
||||
|
||||
# Build the command line
|
||||
cmd_args = [str(cibw_bin), os.environ["INPUT_PACKAGE_DIR"]]
|
||||
|
||||
if output_dir := os.environ.get("INPUT_OUTPUT_DIR"):
|
||||
cmd_args += ["--output-dir", output_dir]
|
||||
|
||||
if config_file := os.environ.get("INPUT_CONFIG_FILE"):
|
||||
cmd_args += ["--config-file", config_file]
|
||||
|
||||
if only := os.environ.get("INPUT_ONLY"):
|
||||
cmd_args += ["--only", only]
|
||||
|
||||
cmd_bash = shlex.join(cmd_args)
|
||||
|
||||
def pwsh_quote(text):
|
||||
# Wrap in single quotes and double-up any existing single quotes
|
||||
return "'" + str(text).replace("'", "''") + "'"
|
||||
|
||||
# Prepend '& ' so PowerShell executes the quoted binary path
|
||||
cmd_pwsh = "& " + " ".join(pwsh_quote(arg) for arg in cmd_args)
|
||||
|
||||
with open(os.environ["GITHUB_OUTPUT"], "at") as f:
|
||||
f.write(f"prepend-path={clean_bin_path}\n")
|
||||
f.write(f"cmd-bash={cmd_bash}\n")
|
||||
f.write(f"cmd-pwsh={cmd_pwsh}\n")
|
||||
|
||||
print("::endgroup::")
|
||||
EOF
|
||||
shell: bash
|
||||
env:
|
||||
PYTHON: ${{ steps.python.outputs.python-path }}
|
||||
INPUT_PACKAGE_DIR: ${{ inputs.package-dir }}
|
||||
INPUT_OUTPUT_DIR: ${{ inputs.output-dir }}
|
||||
INPUT_CONFIG_FILE: ${{ inputs.config-file }}
|
||||
INPUT_ONLY: ${{ inputs.only }}
|
||||
INPUT_EXTRAS: ${{ inputs.extras }}
|
||||
|
||||
# Redirecting stderr to stdout to fix interleaving issue in Actions.
|
||||
- run: |
|
||||
export PATH="$CIBW_PREPEND_PATH:$PATH"
|
||||
eval "$CIBW_CMD_BASH" 2>&1
|
||||
shell: bash
|
||||
if: runner.os != 'Windows'
|
||||
env:
|
||||
CIBW_PREPEND_PATH: ${{ steps.cibw.outputs.prepend-path }}
|
||||
CIBW_CMD_BASH: ${{ steps.cibw.outputs.cmd-bash }}
|
||||
|
||||
# Windows needs powershell to interact nicely with Meson
|
||||
- run: |
|
||||
$env:PATH = "$env:CIBW_PREPEND_PATH;$env:PATH"
|
||||
Invoke-Expression $env:CIBW_CMD_PWSH
|
||||
# This needs powershell-core because github.action path is a Windows-style
|
||||
# path on Windows. Powershell-core understands both types of paths.
|
||||
- run: >
|
||||
pipx run
|
||||
--spec ${{ github.action_path }}
|
||||
cibuildwheel
|
||||
${{ inputs.package-dir }}
|
||||
--output-dir ${{ inputs.output-dir }}
|
||||
shell: pwsh
|
||||
if: runner.os == 'Windows'
|
||||
env:
|
||||
CIBW_PREPEND_PATH: ${{ steps.cibw.outputs.prepend-path }}
|
||||
CIBW_CMD_PWSH: ${{ steps.cibw.outputs.cmd-pwsh }}
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
environment:
|
||||
matrix:
|
||||
- APPVEYOR_BUILD_WORKER_IMAGE: Ubuntu
|
||||
APPVEYOR_JOB_NAME: "python37-x64-ubuntu"
|
||||
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
|
||||
APPVEYOR_JOB_NAME: "python37-x64-vs2015"
|
||||
- APPVEYOR_BUILD_WORKER_IMAGE: macos-mojave
|
||||
APPVEYOR_JOB_NAME: "python37-x64-macos-mojave"
|
||||
|
||||
stack: python 3.7
|
||||
|
||||
build: off
|
||||
|
||||
init:
|
||||
- cmd: set PATH=C:\Python37;C:\Python37\Scripts;%PATH%
|
||||
- ps: |
|
||||
$BRANCH = if ($env:APPVEYOR_PULL_REQUEST_HEAD_REPO_BRANCH) { $env:APPVEYOR_PULL_REQUEST_HEAD_REPO_BRANCH } else { $env:APPVEYOR_REPO_BRANCH }
|
||||
if (-not ($BRANCH -eq 'master' -or $BRANCH.ToLower().StartsWith('appveyor-'))) {
|
||||
$env:PYTEST_ADDOPTS='-k "unit_test or test_0_basic" --suppress-no-test-exit-code'
|
||||
}
|
||||
|
||||
install: python -m pip install -e ".[dev]" pytest-custom-exit-code
|
||||
|
||||
# the '-u' flag is required so the output is in the correct order.
|
||||
# See https://github.com/pypa/cibuildwheel/pull/24 for more info.
|
||||
test_script: python -u ./bin/run_tests.py
|
||||
|
||||
branches:
|
||||
only:
|
||||
- master
|
||||
|
||||
skip_commits:
|
||||
files:
|
||||
- docs/*
|
||||
+37
-86
@@ -1,91 +1,42 @@
|
||||
pr:
|
||||
paths:
|
||||
exclude:
|
||||
- .github/**
|
||||
- bin/*
|
||||
- docs/**
|
||||
- examples/github-*
|
||||
- examples/ci*
|
||||
- examples/travis-ci-*
|
||||
- .ci*
|
||||
- .pre-commit-config.yaml
|
||||
- .readthedocs.yml
|
||||
- .travis.yml
|
||||
- README.md
|
||||
- properdocs.yml
|
||||
- noxfile.py
|
||||
|
||||
jobs:
|
||||
- job: tests
|
||||
strategy:
|
||||
matrix:
|
||||
linux_313:
|
||||
imageName: "ubuntu-latest"
|
||||
pythonVersion: "3.13"
|
||||
testSelect: "native"
|
||||
android_313:
|
||||
imageName: "ubuntu-latest"
|
||||
pythonVersion: "3.13"
|
||||
testSelect: "android"
|
||||
macos_313:
|
||||
imageName: "macos-latest"
|
||||
pythonVersion: "3.13"
|
||||
testSelect: "native"
|
||||
ios_313:
|
||||
imageName: "macos-14" # See https://github.com/actions/runner-images/issues/12777
|
||||
pythonVersion: "3.13"
|
||||
testSelect: "ios"
|
||||
android_macos_313:
|
||||
imageName: "macos-latest"
|
||||
pythonVersion: "3.13"
|
||||
testSelect: "android"
|
||||
windows_313:
|
||||
imageName: "windows-latest"
|
||||
pythonVersion: "3.13"
|
||||
testSelect: "native"
|
||||
timeoutInMinutes: 180
|
||||
pool:
|
||||
vmImage: $(imageName)
|
||||
variables:
|
||||
# Relocate cibuildwheel's download cache (interpreters) and the per-worker
|
||||
# test pip caches (which live under it) to a stable, cacheable path.
|
||||
CIBW_CACHE_PATH: $(Pipeline.Workspace)/.cibw-cache
|
||||
|
||||
- job: linux_38
|
||||
pool: {vmImage: 'Ubuntu-18.04'}
|
||||
steps:
|
||||
- task: UsePythonVersion@0
|
||||
inputs:
|
||||
versionSpec: $(pythonVersion)
|
||||
|
||||
- task: Cache@2
|
||||
displayName: 'Cache cibuildwheel downloads'
|
||||
inputs:
|
||||
# build-platforms.toml carries the interpreter pins; bump the version
|
||||
# prefix to force a cold cache. restoreKeys lets a stale cache still seed
|
||||
# a new run (cibuildwheel re-downloads only what's missing).
|
||||
key: 'cibw-cache | v1 | "$(imageName)" | "$(testSelect)" | cibuildwheel/resources/build-platforms.toml'
|
||||
restoreKeys: |
|
||||
cibw-cache | v1 | "$(imageName)" | "$(testSelect)"
|
||||
path: $(CIBW_CACHE_PATH)
|
||||
|
||||
- task: JavaToolInstaller@0
|
||||
condition: and(eq(variables['testSelect'], 'android'), eq(variables['Agent.OS'], 'Linux'))
|
||||
inputs:
|
||||
versionSpec: '17'
|
||||
jdkArchitectureOption: 'x64'
|
||||
jdkSourceOption: 'PreInstalled'
|
||||
|
||||
- bash: docker run --rm --privileged docker.io/tonistiigi/binfmt:latest --install all
|
||||
condition: and(eq(variables['imageName'], 'ubuntu-latest'), eq(variables['testSelect'], 'native'))
|
||||
displayName: 'Install binfmt on Linux'
|
||||
|
||||
- bash: python -m pip install -U pip && python -m pip install -e. --group test
|
||||
displayName: 'Update pip and install cibuildwheel --group test'
|
||||
|
||||
- bash: echo "##vso[task.setvariable variable=CIBW_ENABLE;]all"
|
||||
condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')
|
||||
displayName: Set CIBW_ENABLE to all (main branch)
|
||||
|
||||
versionSpec: '3.8'
|
||||
- bash: |
|
||||
echo "CIBW_ENABLE = $CIBW_ENABLE"
|
||||
python ./bin/run_tests.py --test-select $(testSelect)
|
||||
displayName: 'Run tests'
|
||||
python -m pip install -e ".[dev]" pytest-azurepipelines
|
||||
python ./bin/run_tests.py
|
||||
|
||||
- job: macos_38
|
||||
pool: {vmImage: 'macOS-10.15'}
|
||||
steps:
|
||||
- task: UsePythonVersion@0
|
||||
inputs:
|
||||
versionSpec: '3.8'
|
||||
- bash: |
|
||||
python -m pip install -e ".[dev]" pytest-azurepipelines
|
||||
python ./bin/run_tests.py
|
||||
|
||||
- job: windows_36
|
||||
pool: {vmImage: 'vs2017-win2016'}
|
||||
timeoutInMinutes: 180
|
||||
steps:
|
||||
- task: UsePythonVersion@0
|
||||
inputs:
|
||||
versionSpec: '3.6'
|
||||
- bash: |
|
||||
python -m pip install -e ".[dev]" pytest-azurepipelines
|
||||
python ./bin/run_tests.py
|
||||
|
||||
- job: windows_38
|
||||
pool: {vmImage: 'vs2017-win2016'}
|
||||
timeoutInMinutes: 180
|
||||
steps:
|
||||
- task: UsePythonVersion@0
|
||||
inputs:
|
||||
versionSpec: '3.8'
|
||||
- bash: |
|
||||
python -m pip install -e ".[dev]" pytest-azurepipelines
|
||||
python ./bin/run_tests.py
|
||||
|
||||
+15
-46
@@ -1,29 +1,30 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = ["click", "packaging", "prek"]
|
||||
# ///
|
||||
#!/usr/bin/env python3
|
||||
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import glob
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tomllib
|
||||
import urllib.parse
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
from packaging.version import InvalidVersion, Version
|
||||
|
||||
import cibuildwheel
|
||||
|
||||
config = [
|
||||
# file path, version find/replace format
|
||||
("pyproject.toml", 'version = "{}"'),
|
||||
("README.md", "cibuildwheel=={}"),
|
||||
("cibuildwheel/__init__.py", '__version__ = "{}"'),
|
||||
("docs/faq.md", "cibuildwheel=={}"),
|
||||
("docs/faq.md", "cibuildwheel@v{}"),
|
||||
("docs/setup.md", "cibuildwheel=={}"),
|
||||
("examples/*", "cibuildwheel=={}"),
|
||||
("examples/*", "cibuildwheel@v{}"),
|
||||
("setup.cfg", "version = {}"),
|
||||
]
|
||||
|
||||
RED = "\u001b[31m"
|
||||
@@ -33,8 +34,7 @@ OFF = "\u001b[0m"
|
||||
|
||||
@click.command()
|
||||
def bump_version() -> None:
|
||||
with open("pyproject.toml", "rb") as f:
|
||||
current_version = tomllib.load(f)["project"]["version"]
|
||||
current_version = cibuildwheel.__version__
|
||||
|
||||
try:
|
||||
commit_date_str = subprocess.run(
|
||||
@@ -60,7 +60,7 @@ def bump_version() -> None:
|
||||
print(e)
|
||||
print("Failed to get previous version tag information.")
|
||||
|
||||
git_changes_result = subprocess.run(["git diff-index --quiet HEAD --"], shell=True, check=False)
|
||||
git_changes_result = subprocess.run(["git diff-index --quiet HEAD --"], shell=True)
|
||||
repo_has_uncommitted_changes = git_changes_result.returncode != 0
|
||||
|
||||
if repo_has_uncommitted_changes:
|
||||
@@ -68,8 +68,8 @@ def bump_version() -> None:
|
||||
sys.exit(1)
|
||||
|
||||
# fmt: off
|
||||
print( "Current version:", current_version)
|
||||
new_version = input(" New version: ").strip()
|
||||
print( 'Current version:', current_version) # noqa
|
||||
new_version = input(' New version: ').strip()
|
||||
# fmt: on
|
||||
|
||||
try:
|
||||
@@ -82,7 +82,7 @@ def bump_version() -> None:
|
||||
actions = []
|
||||
|
||||
for path_pattern, version_pattern in config:
|
||||
paths = list(Path().glob(path_pattern))
|
||||
paths = [Path(p) for p in glob.glob(path_pattern)]
|
||||
|
||||
if not paths:
|
||||
print(f"error: Pattern {path_pattern} didn't match any files")
|
||||
@@ -129,33 +129,13 @@ def bump_version() -> None:
|
||||
contents = contents.replace(find, replace)
|
||||
path.write_text(contents, encoding="utf8")
|
||||
|
||||
print("Files updated. If you want to update docs/changelog.md as part of")
|
||||
print("this commit, do that now.")
|
||||
print("Files updated. If you want to update the changelog as part of this")
|
||||
print("commit, do that now.")
|
||||
print()
|
||||
|
||||
while input('Type "done" to continue: ').strip().lower() != "done":
|
||||
pass
|
||||
|
||||
# run pre-commit to update the README changelog
|
||||
subprocess.run(
|
||||
[
|
||||
"prek",
|
||||
"run",
|
||||
"--files=docs/changelog.md",
|
||||
],
|
||||
check=False,
|
||||
)
|
||||
|
||||
# run pre-commit to check that no errors occurred on the second run
|
||||
subprocess.run(
|
||||
[
|
||||
"prek",
|
||||
"run",
|
||||
"--files=docs/changelog.md",
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
|
||||
subprocess.run(
|
||||
[
|
||||
"git",
|
||||
@@ -178,17 +158,6 @@ def bump_version() -> None:
|
||||
)
|
||||
|
||||
print("Done.")
|
||||
print()
|
||||
|
||||
print("Push the new version to GitHub with:")
|
||||
print(f" git push && git push origin v{new_version}")
|
||||
print()
|
||||
|
||||
release_url = "https://github.com/pypa/cibuildwheel/releases/new?" + urllib.parse.urlencode(
|
||||
{"tag": f"v{new_version}"}
|
||||
)
|
||||
print("Then create a release at the URL:")
|
||||
print(f" {release_url}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Executable
+3
@@ -0,0 +1,3 @@
|
||||
#!/bin/bash
|
||||
|
||||
CIBW_PLATFORM=linux pytest "$@"
|
||||
@@ -1,79 +0,0 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "cibuildwheel",
|
||||
# ]
|
||||
#
|
||||
# [tool.uv.sources]
|
||||
# cibuildwheel = { path = ".." }
|
||||
# ///
|
||||
|
||||
import sys
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
|
||||
from cibuildwheel.extra import get_pyodide_xbuildenv_info
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument(
|
||||
"pyodide-version",
|
||||
type=str,
|
||||
)
|
||||
@click.option(
|
||||
"--output-file",
|
||||
type=click.Path(),
|
||||
default=None,
|
||||
help="Output file to write the constraints to. If not provided, the constraints will be printed to stdout.",
|
||||
)
|
||||
def generate_pyodide_constraints(pyodide_version: str, output_file: str | None = None) -> None:
|
||||
"""
|
||||
Generate constraints for a specific Pyodide version. The constraints are
|
||||
generated based on the Pyodide version's xbuildenv info, which is retrieved
|
||||
from the Pyodide repository.
|
||||
|
||||
These constraints should then be 'pinned' using `uv pip compile`.
|
||||
|
||||
Example usage:
|
||||
|
||||
bin/generate_pyodide_constraints.py 0.27.0
|
||||
"""
|
||||
xbuildenv_info = get_pyodide_xbuildenv_info()
|
||||
try:
|
||||
pyodide_version_xbuildenv_info = xbuildenv_info["releases"][pyodide_version]
|
||||
except KeyError as e:
|
||||
msg = f"Pyodide version {pyodide_version} not found in xbuildenv info. Versions available: {', '.join(xbuildenv_info['releases'].keys())}"
|
||||
raise click.BadParameter(msg) from e
|
||||
|
||||
pyodide_build_min_version = pyodide_version_xbuildenv_info.get("min_pyodide_build_version")
|
||||
pyodide_build_max_version = pyodide_version_xbuildenv_info.get("max_pyodide_build_version")
|
||||
|
||||
pyodide_build_specifier_parts: list[str] = []
|
||||
|
||||
if pyodide_build_min_version:
|
||||
pyodide_build_specifier_parts.append(f">={pyodide_build_min_version}")
|
||||
if pyodide_build_max_version:
|
||||
pyodide_build_specifier_parts.append(f"<={pyodide_build_max_version}")
|
||||
|
||||
pyodide_build_specifier = ",".join(pyodide_build_specifier_parts)
|
||||
|
||||
constraints_txt = textwrap.dedent(f"""
|
||||
pip
|
||||
build[virtualenv]
|
||||
pyodide-build{pyodide_build_specifier}
|
||||
click<8.2
|
||||
""")
|
||||
|
||||
if output_file is None:
|
||||
print(constraints_txt)
|
||||
else:
|
||||
Path(output_file).write_text(constraints_txt)
|
||||
print(f"Constraints written to {output_file}", file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
generate_pyodide_constraints()
|
||||
@@ -1,424 +0,0 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = ["pyyaml"]
|
||||
# ///
|
||||
|
||||
import argparse
|
||||
import copy
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
parser = argparse.ArgumentParser(allow_abbrev=False)
|
||||
parser.add_argument("--schemastore", action="store_true", help="Generate schema_store version")
|
||||
args = parser.parse_args()
|
||||
|
||||
# The defaults in the schema are used by external tools for validation and IDE support. They
|
||||
# should match the values in defaults.toml, which are used by cibuildwheel itself.
|
||||
starter = """
|
||||
$schema: http://json-schema.org/draft-07/schema#
|
||||
$id: https://github.com/pypa/cibuildwheel/blob/main/cibuildwheel/resources/cibuildwheel.schema.json
|
||||
$defs:
|
||||
inherit:
|
||||
enum:
|
||||
- none
|
||||
- prepend
|
||||
- append
|
||||
default: none
|
||||
description: How to inherit the parent's value.
|
||||
enable:
|
||||
enum:
|
||||
- cpython-prerelease
|
||||
- graalpy
|
||||
- pyodide-prerelease
|
||||
- pypy
|
||||
- pypy-eol
|
||||
description: A Python version or flavor to enable.
|
||||
additionalProperties: false
|
||||
description: cibuildwheel's settings.
|
||||
type: object
|
||||
properties:
|
||||
audit-command:
|
||||
description: Execute a shell command to audit each wheel after it is repaired. Use {wheel} for each wheel path, or {abi3_wheel} to only audit abi3 wheels.
|
||||
type: string_array
|
||||
audit-requires:
|
||||
description: Install Python dependencies for the audit step.
|
||||
type: string_array
|
||||
archs:
|
||||
description: Change the architectures built on your machine by default.
|
||||
type: string_array
|
||||
before-all:
|
||||
description: Execute a shell command on the build system before any wheels are built.
|
||||
type: string_array
|
||||
before-build:
|
||||
description: Execute a shell command preparing each wheel's build.
|
||||
type: string_array
|
||||
before-test:
|
||||
description: Execute a shell command before testing each wheel.
|
||||
type: string_array
|
||||
build:
|
||||
default: ['*']
|
||||
description: Choose the Python versions to build.
|
||||
type: string_array
|
||||
build-frontend:
|
||||
default: default
|
||||
description: Set the tool to use to build, either "build" (default), "build[uv]", "uv", or "pip"
|
||||
oneOf:
|
||||
- enum: [pip, build, "build[uv]", uv, default]
|
||||
- type: string
|
||||
pattern: '^pip; ?args:'
|
||||
- type: string
|
||||
pattern: '^build; ?args:'
|
||||
- type: string
|
||||
pattern: '^build\\[uv\\]; ?args:'
|
||||
- type: string
|
||||
pattern: '^uv; ?args:'
|
||||
- type: object
|
||||
additionalProperties: false
|
||||
required: [name]
|
||||
properties:
|
||||
name:
|
||||
enum: [pip, build, "build[uv]", uv]
|
||||
args:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
build-verbosity:
|
||||
type: integer
|
||||
minimum: -3
|
||||
maximum: 3
|
||||
default: 0
|
||||
description: Increase/decrease the output of pip wheel.
|
||||
config-settings:
|
||||
description: Specify config-settings for the build backend.
|
||||
type: string_table_array
|
||||
container-engine:
|
||||
oneOf:
|
||||
- enum: [docker, podman]
|
||||
- type: string
|
||||
pattern: '^docker; ?(create_args|disable_host_mount):'
|
||||
- type: string
|
||||
pattern: '^podman; ?(create_args|disable_host_mount):'
|
||||
- type: object
|
||||
additionalProperties: false
|
||||
required: [name]
|
||||
properties:
|
||||
name:
|
||||
enum: [docker, podman]
|
||||
create-args:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
disable-host-mount:
|
||||
type: boolean
|
||||
dependency-versions:
|
||||
default: pinned
|
||||
description: Specify how cibuildwheel controls the versions of the tools it uses
|
||||
oneOf:
|
||||
- enum: [pinned, latest]
|
||||
- type: string
|
||||
description: Path to a file containing dependency versions, or inline package specifications, starting with "packages:"
|
||||
not:
|
||||
enum: [pinned, latest]
|
||||
- type: object
|
||||
additionalProperties: false
|
||||
properties:
|
||||
file:
|
||||
type: string
|
||||
- type: object
|
||||
additionalProperties: false
|
||||
properties:
|
||||
packages:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
enable:
|
||||
description: Enable or disable certain builds.
|
||||
oneOf:
|
||||
- $ref: "#/$defs/enable"
|
||||
- type: array
|
||||
items:
|
||||
$ref: "#/$defs/enable"
|
||||
environment:
|
||||
description: Set environment variables needed during the build.
|
||||
type: string_table
|
||||
environment-pass:
|
||||
description: Set environment variables on the host to pass-through to the container
|
||||
during the build.
|
||||
type: string_array
|
||||
manylinux-aarch64-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
manylinux-armv7l-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
manylinux-i686-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
manylinux-ppc64le-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
manylinux-pypy_aarch64-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
manylinux-pypy_i686-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
manylinux-pypy_x86_64-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
manylinux-riscv64-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
manylinux-s390x-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
manylinux-x86_64-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
musllinux-aarch64-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
musllinux-armv7l-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
musllinux-i686-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
musllinux-ppc64le-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
musllinux-riscv64-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
musllinux-s390x-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
musllinux-x86_64-image:
|
||||
type: string
|
||||
description: Specify alternative manylinux / musllinux container images
|
||||
xbuild-tools:
|
||||
description: Binaries on the path that should be included in an isolated cross-build environment
|
||||
type: string_array
|
||||
xbuild-files:
|
||||
description: Platform-specific files in the build environment
|
||||
type: string_table_array
|
||||
pyodide-version:
|
||||
type: string
|
||||
description: Specify the version of Pyodide to use
|
||||
repair-wheel-command:
|
||||
description: Execute a shell command to repair each built wheel.
|
||||
type: string_array
|
||||
skip:
|
||||
description: Choose the Python versions to skip.
|
||||
type: string_array
|
||||
test-command:
|
||||
description: Execute a shell command to test each built wheel.
|
||||
type: string_array
|
||||
test-extras:
|
||||
description: Install your wheel for testing using `extras_require`
|
||||
type: string_array
|
||||
test-sources:
|
||||
description: Test files that are required by the test environment
|
||||
type: string_array
|
||||
test-groups:
|
||||
description: Install extra groups when testing
|
||||
type: string_array
|
||||
test-requires:
|
||||
description: Install Python dependencies before running the tests
|
||||
type: string_array
|
||||
test-skip:
|
||||
description: Skip running tests on some builds.
|
||||
type: string_array
|
||||
test-environment:
|
||||
description: Set environment variables for the test environment
|
||||
type: string_table
|
||||
test-runtime:
|
||||
description: Additional configuration for the test runner
|
||||
oneOf:
|
||||
- type: string
|
||||
pattern: '^$'
|
||||
- type: object
|
||||
additionalProperties: false
|
||||
- type: string
|
||||
pattern: 'args:'
|
||||
- type: object
|
||||
additionalProperties: false
|
||||
required: [args]
|
||||
properties:
|
||||
args:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
|
||||
"""
|
||||
|
||||
schema = yaml.safe_load(starter)
|
||||
|
||||
string_array = yaml.safe_load(
|
||||
"""
|
||||
- type: string
|
||||
- type: array
|
||||
items:
|
||||
type: string
|
||||
"""
|
||||
)
|
||||
|
||||
string_table_array = yaml.safe_load(
|
||||
"""
|
||||
- type: string
|
||||
- type: object
|
||||
additionalProperties: false
|
||||
patternProperties:
|
||||
.+:
|
||||
oneOf:
|
||||
- type: string
|
||||
- type: array
|
||||
items:
|
||||
type: string
|
||||
"""
|
||||
)
|
||||
|
||||
string_table = yaml.safe_load(
|
||||
"""
|
||||
- type: string
|
||||
- type: object
|
||||
additionalProperties: false
|
||||
patternProperties:
|
||||
.+:
|
||||
type: string
|
||||
"""
|
||||
)
|
||||
|
||||
for value in schema["properties"].values():
|
||||
match value:
|
||||
case {"type": "string_array"}:
|
||||
del value["type"]
|
||||
value["oneOf"] = string_array
|
||||
case {"type": "string_table"}:
|
||||
del value["type"]
|
||||
value["oneOf"] = string_table
|
||||
case {"type": "string_table_array"}:
|
||||
del value["type"]
|
||||
value["oneOf"] = string_table_array
|
||||
|
||||
overrides = yaml.safe_load(
|
||||
"""
|
||||
type: array
|
||||
description: An overrides array
|
||||
items:
|
||||
type: object
|
||||
required: ["select"]
|
||||
minProperties: 2
|
||||
additionalProperties: false
|
||||
properties:
|
||||
select: {}
|
||||
inherit:
|
||||
type: object
|
||||
additionalProperties: false
|
||||
properties:
|
||||
audit-command: {"$ref": "#/$defs/inherit"}
|
||||
audit-requires: {"$ref": "#/$defs/inherit"}
|
||||
before-all: {"$ref": "#/$defs/inherit"}
|
||||
before-build: {"$ref": "#/$defs/inherit"}
|
||||
xbuild-tools: {"$ref": "#/$defs/inherit"}
|
||||
xbuild-files: {"$ref": "#/$defs/inherit"}
|
||||
before-test: {"$ref": "#/$defs/inherit"}
|
||||
config-settings: {"$ref": "#/$defs/inherit"}
|
||||
container-engine: {"$ref": "#/$defs/inherit"}
|
||||
environment: {"$ref": "#/$defs/inherit"}
|
||||
environment-pass: {"$ref": "#/$defs/inherit"}
|
||||
repair-wheel-command: {"$ref": "#/$defs/inherit"}
|
||||
test-command: {"$ref": "#/$defs/inherit"}
|
||||
test-extras: {"$ref": "#/$defs/inherit"}
|
||||
test-sources: {"$ref": "#/$defs/inherit"}
|
||||
test-requires: {"$ref": "#/$defs/inherit"}
|
||||
test-environment: {"$ref": "#/$defs/inherit"}
|
||||
test-runtime: {"$ref": "#/$defs/inherit"}
|
||||
"""
|
||||
)
|
||||
|
||||
for key, value in schema["properties"].items():
|
||||
value["title"] = f"CIBW_{key.replace('-', '_').upper()}"
|
||||
|
||||
non_global_options = {k: {"$ref": f"#/properties/{k}"} for k in schema["properties"]}
|
||||
del non_global_options["build"]
|
||||
del non_global_options["skip"]
|
||||
del non_global_options["test-skip"]
|
||||
del non_global_options["enable"]
|
||||
|
||||
overrides["items"]["properties"]["select"]["oneOf"] = string_array
|
||||
overrides["items"]["properties"] |= non_global_options.copy()
|
||||
|
||||
del overrides["items"]["properties"]["archs"]
|
||||
|
||||
not_linux = non_global_options.copy()
|
||||
|
||||
del not_linux["environment-pass"]
|
||||
del not_linux["container-engine"]
|
||||
for key in list(not_linux):
|
||||
if "linux-" in key:
|
||||
del not_linux[key]
|
||||
|
||||
|
||||
def as_object(d: dict[str, Any]) -> dict[str, Any]:
|
||||
return {
|
||||
"type": "object",
|
||||
"additionalProperties": False,
|
||||
"properties": copy.deepcopy(d),
|
||||
}
|
||||
|
||||
|
||||
oses = {
|
||||
"linux": as_object(non_global_options),
|
||||
"windows": as_object(not_linux),
|
||||
"macos": as_object(not_linux),
|
||||
"pyodide": as_object(not_linux),
|
||||
"android": as_object(not_linux),
|
||||
"ios": as_object(not_linux),
|
||||
}
|
||||
|
||||
for os_name, command in [
|
||||
("linux", "auditwheel repair -w {dest_dir} {wheel}"),
|
||||
("windows", "delvewheel repair -w {dest_dir} -v {wheel}"),
|
||||
("macos", "delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}"),
|
||||
("android", "auditwheel repair --ldpaths {ldpaths} -w {dest_dir} {wheel}"),
|
||||
]:
|
||||
oses[os_name]["properties"]["repair-wheel-command"] = {
|
||||
**schema["properties"]["repair-wheel-command"],
|
||||
"default": command,
|
||||
}
|
||||
|
||||
del oses["linux"]["properties"]["dependency-versions"]
|
||||
|
||||
oses["pyodide"]["properties"]["build-frontend"] = {
|
||||
**schema["properties"]["build-frontend"],
|
||||
"default": "pyodide-build",
|
||||
"description": 'On the pyodide platform, the build frontend must be "pyodide-build"',
|
||||
"oneOf": [
|
||||
{"enum": ["pyodide-build"]},
|
||||
{"type": "string", "pattern": "^pyodide-build; ?args:"},
|
||||
{
|
||||
"type": "object",
|
||||
"additionalProperties": False,
|
||||
"required": ["name"],
|
||||
"properties": {
|
||||
"name": {"enum": ["pyodide-build"]},
|
||||
"args": {"type": "array", "items": {"type": "string"}},
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
schema["properties"]["overrides"] = overrides
|
||||
schema["properties"] |= oses
|
||||
|
||||
if args.schemastore:
|
||||
schema["$id"] = "https://json.schemastore.org/partial-cibuildwheel.json"
|
||||
schema["description"] = (
|
||||
"cibuildwheel's settings. Generated with ./bin/generate_schema.py --schemastore from cibuildwheel."
|
||||
)
|
||||
|
||||
print(json.dumps(schema, indent=2))
|
||||
@@ -1,45 +1,17 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "pygithub",
|
||||
# "pyyaml",
|
||||
# "rich",
|
||||
# "cibuildwheel",
|
||||
# ]
|
||||
#
|
||||
# [tool.uv.sources]
|
||||
# cibuildwheel = { path = ".." }
|
||||
# ///
|
||||
|
||||
"""
|
||||
Check known projects for usage of requires-python.
|
||||
|
||||
Usage:
|
||||
|
||||
./bin/inspect_all_known_projects.py --online=$GITHUB_TOKEN
|
||||
|
||||
This will cache the results to all_known_setup.yaml; you can reprint
|
||||
the results without the `--online` setting.
|
||||
"""
|
||||
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
from pathlib import Path
|
||||
from typing import Iterator
|
||||
|
||||
import click
|
||||
import yaml
|
||||
from github import Github, GithubException
|
||||
from ghapi.core import GhApi, HTTP404NotFoundError # type: ignore
|
||||
from rich import print
|
||||
|
||||
from cibuildwheel.projectfiles import Analyzer
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterable, Iterator
|
||||
|
||||
DIR = Path(__file__).parent.resolve()
|
||||
|
||||
|
||||
@@ -75,44 +47,39 @@ def check_repo(name: str, contents: str) -> str:
|
||||
|
||||
|
||||
class MaybeRemote:
|
||||
github: Github | None
|
||||
contents: dict[str, dict[str, str | None]]
|
||||
|
||||
def __init__(self, cached_file: Path | str, *, online: str | None) -> None:
|
||||
if online is not None:
|
||||
self.github = Github(online)
|
||||
self.contents = {
|
||||
def __init__(self, cached_file: Path | str, *, online: bool) -> None:
|
||||
self.online = online
|
||||
if self.online:
|
||||
self.contents: dict[str, dict[str, str | None]] = {
|
||||
"setup.py": {},
|
||||
"setup.cfg": {},
|
||||
"pyproject.toml": {},
|
||||
}
|
||||
else:
|
||||
self.github = None
|
||||
with open(cached_file) as f:
|
||||
self.contents = yaml.safe_load(f)
|
||||
|
||||
def get(self, repo: str, filename: str) -> str | None:
|
||||
if self.github:
|
||||
if self.online:
|
||||
try:
|
||||
gh_file = self.github.get_repo(repo).get_contents(filename)
|
||||
except GithubException:
|
||||
self.contents[filename][repo] = (
|
||||
GhApi(*repo.split("/")).get_content(filename).decode()
|
||||
)
|
||||
except HTTP404NotFoundError:
|
||||
self.contents[filename][repo] = None
|
||||
else:
|
||||
assert not isinstance(gh_file, list)
|
||||
self.contents[filename][repo] = gh_file.decoded_content.decode(encoding="utf-8")
|
||||
|
||||
return self.contents[filename][repo]
|
||||
elif repo in self.contents[filename]:
|
||||
return self.contents[filename][repo]
|
||||
else:
|
||||
msg = f"Trying to access {repo}:{filename} and not in cache, rebuild cache"
|
||||
raise RuntimeError(msg)
|
||||
raise RuntimeError(
|
||||
f"Trying to access {repo}:{filename} and not in cache, rebuild cache"
|
||||
)
|
||||
|
||||
def save(self, filename: Path | str) -> None:
|
||||
with open(filename, "w") as f:
|
||||
yaml.safe_dump(self.contents, f, default_flow_style=False)
|
||||
|
||||
def on_each(self, repos: Iterable[str]) -> Iterator[tuple[str, str, str | None]]:
|
||||
def on_each(self, repos: list[str]) -> Iterator[tuple[str, str, str | None]]:
|
||||
for repo in repos:
|
||||
print(f"[bold]{repo}:")
|
||||
for filename in sorted(self.contents, reverse=True):
|
||||
@@ -120,8 +87,8 @@ class MaybeRemote:
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option("--online", help="Set to $GITHUB_TOKEN")
|
||||
def main(online: str | None) -> None:
|
||||
@click.option("--online", is_flag=True, help="Remember to set GITHUB_TOKEN")
|
||||
def main(online: bool) -> None:
|
||||
with open(DIR / "../docs/data/projects.yml") as f:
|
||||
known = yaml.safe_load(f)
|
||||
|
||||
|
||||
@@ -1,36 +1,29 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# ]
|
||||
# ///
|
||||
#!/usr/bin/env python3
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import textwrap
|
||||
import time
|
||||
from pathlib import Path
|
||||
from subprocess import run
|
||||
|
||||
import click
|
||||
|
||||
|
||||
def shell(cmd: str, *, check: bool, **kwargs: object) -> subprocess.CompletedProcess[str]:
|
||||
return subprocess.run([cmd], shell=True, check=check, **kwargs) # type: ignore[call-overload, no-any-return]
|
||||
def shell(cmd, **kwargs):
|
||||
return run([cmd], shell=True, **kwargs)
|
||||
|
||||
|
||||
def git_repo_has_changes() -> bool:
|
||||
unstaged_changes: bool = shell("git diff-index --quiet HEAD --", check=False).returncode != 0
|
||||
staged_changes: bool = (
|
||||
shell("git diff-index --quiet --cached HEAD --", check=False).returncode != 0
|
||||
)
|
||||
def git_repo_has_changes():
|
||||
unstaged_changes = shell("git diff-index --quiet HEAD --").returncode != 0
|
||||
staged_changes = shell("git diff-index --quiet --cached HEAD --").returncode != 0
|
||||
return unstaged_changes or staged_changes
|
||||
|
||||
|
||||
@click.command()
|
||||
def main() -> None:
|
||||
def main():
|
||||
project_root = Path(__file__).parent / ".."
|
||||
os.chdir(project_root)
|
||||
|
||||
@@ -47,7 +40,7 @@ def main() -> None:
|
||||
timestamp = time.strftime("%Y-%m-%dT%H-%M-%S", time.gmtime())
|
||||
branch_name = f"update-constraints-{timestamp}"
|
||||
|
||||
shell(f"git checkout -b {branch_name} origin/main", check=True)
|
||||
shell(f"git checkout -b {branch_name} origin/master", check=True)
|
||||
|
||||
try:
|
||||
shell("bin/update_dependencies.py", check=True)
|
||||
@@ -61,16 +54,16 @@ def main() -> None:
|
||||
f"""
|
||||
Update the versions of our dependencies.
|
||||
|
||||
PR generated by `{Path(__file__).name}`.
|
||||
PR generated by `{os.path.basename(__file__)}`.
|
||||
"""
|
||||
)
|
||||
subprocess.run(
|
||||
run(
|
||||
[
|
||||
"gh",
|
||||
"pr",
|
||||
"create",
|
||||
"--repo=pypa/cibuildwheel",
|
||||
"--base=main",
|
||||
"--base=master",
|
||||
"--title=Update dependencies",
|
||||
f"--body='{body}'",
|
||||
],
|
||||
@@ -80,7 +73,7 @@ def main() -> None:
|
||||
print("Done.")
|
||||
finally:
|
||||
# remove any local changes
|
||||
shell("git checkout -- .", check=True)
|
||||
shell("git checkout -- .")
|
||||
shell(f"git checkout {previous_branch}", check=True)
|
||||
shell(f"git branch -D --force {branch_name}", check=True)
|
||||
|
||||
|
||||
+46
-107
@@ -1,19 +1,11 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "pyyaml",
|
||||
# "pygithub",
|
||||
# ]
|
||||
# ///
|
||||
#!/usr/bin/env python3
|
||||
|
||||
"""
|
||||
Convert a yaml project list into a nice table.
|
||||
|
||||
Suggested usage:
|
||||
|
||||
./bin/projects.py docs/data/projects.yml --online --auth $GITHUB_API_TOKEN
|
||||
./bin/projects.py docs/data/projects.yml --online --auth $GITHUB_API_TOKEN --readme README.md
|
||||
git diff
|
||||
"""
|
||||
|
||||
@@ -22,23 +14,19 @@ from __future__ import annotations
|
||||
import builtins
|
||||
import functools
|
||||
import textwrap
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import xml.dom.minidom
|
||||
from datetime import UTC, datetime
|
||||
from datetime import datetime
|
||||
from io import StringIO
|
||||
from pathlib import Path
|
||||
from typing import Any, Self, TextIO
|
||||
from typing import Any, TextIO
|
||||
|
||||
import click
|
||||
import yaml
|
||||
from github import Auth, Github, GithubException
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterable, Mapping, Sequence
|
||||
from github import Github, GithubException
|
||||
|
||||
ICONS = (
|
||||
"appveyor",
|
||||
"github",
|
||||
"azurepipelines",
|
||||
"circleci",
|
||||
@@ -47,16 +35,13 @@ ICONS = (
|
||||
"windows",
|
||||
"apple",
|
||||
"linux",
|
||||
"android",
|
||||
"ios",
|
||||
"pyodide",
|
||||
)
|
||||
|
||||
|
||||
class Project:
|
||||
NAME: int = 0
|
||||
|
||||
def __init__(self, config: Mapping[str, Any], github: Github | None = None):
|
||||
def __init__(self, config: dict[str, Any], github: Github | None = None):
|
||||
try:
|
||||
self.name: str = config["name"]
|
||||
self.gh: str = config["gh"]
|
||||
@@ -77,7 +62,7 @@ class Project:
|
||||
print(f"Broken: {self.stars_repo}")
|
||||
raise
|
||||
|
||||
self.num_stars: int = repo.stargazers_count
|
||||
self.num_stars = repo.stargazers_count
|
||||
self.pushed_at = repo.pushed_at
|
||||
if not self.notes:
|
||||
notes = repo.description
|
||||
@@ -85,12 +70,12 @@ class Project:
|
||||
self.notes = notes
|
||||
else:
|
||||
self.num_stars = 0
|
||||
self.pushed_at = datetime.now(UTC)
|
||||
self.pushed_at = datetime.utcnow()
|
||||
|
||||
name_len = len(self.name) + 4
|
||||
self.__class__.NAME = max(self.__class__.NAME, name_len)
|
||||
|
||||
def __lt__(self, other: Self) -> bool:
|
||||
def __lt__(self, other: Project) -> bool:
|
||||
if self.online:
|
||||
return self.num_stars < other.num_stars
|
||||
else:
|
||||
@@ -100,8 +85,8 @@ class Project:
|
||||
def header(cls) -> str:
|
||||
return textwrap.dedent(
|
||||
f"""\
|
||||
| {"Name":{cls.NAME}} | CI | OS | Notes |
|
||||
|{"":-^{cls.NAME + 2}}|----|----|:------|"""
|
||||
| {'Name':{cls.NAME}} | CI | OS | Notes |
|
||||
|{'':-^{cls.NAME+2 }}|----|----|:------|"""
|
||||
)
|
||||
|
||||
@property
|
||||
@@ -132,23 +117,17 @@ class Project:
|
||||
return f"[{self.name}]: {self.url}"
|
||||
|
||||
def info(self) -> str:
|
||||
days = (datetime.now(UTC) - self.pushed_at).days
|
||||
days = (datetime.utcnow() - self.pushed_at).days
|
||||
return f"<!-- {self.name}: {self.num_stars}, last pushed {days} days ago -->"
|
||||
|
||||
|
||||
def fetch_icon(icon_name: str) -> None:
|
||||
url = f"https://cdn.jsdelivr.net/npm/simple-icons@v4/icons/{icon_name}.svg"
|
||||
try:
|
||||
with urllib.request.urlopen(url) as f:
|
||||
original_svg_data = f.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 404 and path_for_icon(icon_name).exists():
|
||||
return
|
||||
raise
|
||||
with urllib.request.urlopen(url) as f:
|
||||
original_svg_data = f.read()
|
||||
|
||||
document = xml.dom.minidom.parseString(original_svg_data)
|
||||
svgElement = document.documentElement
|
||||
assert svgElement is not None
|
||||
assert svgElement.nodeName == "svg"
|
||||
svgElement.setAttribute("width", "16px")
|
||||
svgElement.setAttribute("fill", "#606060")
|
||||
@@ -160,37 +139,28 @@ def fetch_icon(icon_name: str) -> None:
|
||||
f.write(svgElement.toxml())
|
||||
|
||||
|
||||
def path_for_icon(icon_name: str, relative_to: Path | None = None) -> Path:
|
||||
from_path = Path.cwd() if relative_to is None else relative_to.parent
|
||||
|
||||
absolute_path = PROJECT_ROOT / "docs" / "data" / "readme_icons" / f"{icon_name}.svg"
|
||||
|
||||
return absolute_path.resolve().relative_to(from_path.resolve())
|
||||
def path_for_icon(icon_name: str) -> Path:
|
||||
return Path(".") / "docs" / "data" / "readme_icons" / f"{icon_name}.svg"
|
||||
|
||||
|
||||
def get_projects(
|
||||
config: Iterable[Mapping[str, Any]],
|
||||
def str_projects(
|
||||
config: list[dict[str, Any]],
|
||||
*,
|
||||
online: bool = True,
|
||||
auth: str | None = None,
|
||||
) -> list[Project]:
|
||||
if online:
|
||||
for icon in ICONS:
|
||||
fetch_icon(icon)
|
||||
|
||||
github = Github(auth=Auth.Token(auth)) if (online and auth) else None
|
||||
|
||||
return sorted((Project(item, github) for item in config), reverse=online)
|
||||
|
||||
|
||||
def render_projects(
|
||||
projects: Sequence[Project], *, dest_path: Path, include_info: bool = True
|
||||
) -> str:
|
||||
io = StringIO()
|
||||
print = functools.partial(builtins.print, file=io)
|
||||
|
||||
print(Project.header())
|
||||
if online:
|
||||
for icon in ICONS:
|
||||
fetch_icon(icon)
|
||||
|
||||
github = Github(auth) if online else None
|
||||
|
||||
projects = sorted((Project(item, github) for item in config), reverse=online)
|
||||
|
||||
print(Project.header())
|
||||
for project in projects:
|
||||
print(project.table_row())
|
||||
|
||||
@@ -200,73 +170,42 @@ def render_projects(
|
||||
|
||||
print()
|
||||
for icon in ICONS:
|
||||
print(f"[{icon} icon]: {path_for_icon(icon, relative_to=dest_path).as_posix()}")
|
||||
print(f"[{icon} icon]: {path_for_icon(icon).as_posix()}")
|
||||
|
||||
if include_info:
|
||||
print()
|
||||
for project in projects:
|
||||
print(project.info())
|
||||
print()
|
||||
for project in projects:
|
||||
print(project.info())
|
||||
|
||||
return io.getvalue()
|
||||
|
||||
|
||||
def insert_projects_table(
|
||||
file: Path,
|
||||
*,
|
||||
projects: Sequence[Project],
|
||||
input_filename: str,
|
||||
include_info: bool = True,
|
||||
) -> None:
|
||||
text = file.read_text()
|
||||
projects_table = render_projects(projects, include_info=include_info, dest_path=file)
|
||||
|
||||
start_str = "<!-- START bin/projects.py -->\n"
|
||||
|
||||
start = text.find(start_str)
|
||||
assert start != -1
|
||||
|
||||
end = text.find("<!-- END bin/projects.py -->\n")
|
||||
assert end != -1
|
||||
|
||||
generated_note = f"<!-- this section is generated by bin/projects.py. Don't edit it directly, instead, edit {input_filename} -->"
|
||||
new_text = (
|
||||
f"{text[: start + len(start_str)]}\n{generated_note}\n\n{projects_table}\n{text[end:]}"
|
||||
)
|
||||
|
||||
file.write_text(new_text)
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).parent / ".."
|
||||
README_FILE = PROJECT_ROOT / "README.md"
|
||||
DOCS_PAGE = PROJECT_ROOT / "docs" / "working-examples.md"
|
||||
|
||||
|
||||
@click.command(help="Try ./bin/projects.py docs/data/projects.yml")
|
||||
@click.command(help="Try ./bin/projects.py docs/data/projects.yml --readme README.md")
|
||||
@click.argument("input", type=click.File("r"))
|
||||
@click.option("--online/--no-online", default=True, help="Get info from GitHub")
|
||||
@click.option("--auth", help="GitHub authentication token")
|
||||
@click.option(
|
||||
"--dry-run", default=False, help="Print the output, rather than writing it to files in the repo"
|
||||
)
|
||||
@click.option("--readme", type=click.File("r+"), help="Modify a readme file if given")
|
||||
def projects(
|
||||
input: TextIO,
|
||||
online: bool,
|
||||
auth: str | None,
|
||||
dry_run: bool,
|
||||
readme: TextIO | None,
|
||||
) -> None:
|
||||
config = yaml.safe_load(input)
|
||||
projects = get_projects(config, online=online, auth=auth)
|
||||
output = str_projects(config, online=online, auth=auth)
|
||||
|
||||
if dry_run:
|
||||
output = render_projects(projects, dest_path=README_FILE)
|
||||
if readme is None:
|
||||
print(output)
|
||||
else:
|
||||
insert_projects_table(
|
||||
README_FILE, projects=projects[:10], input_filename=input.name, include_info=False
|
||||
)
|
||||
insert_projects_table(
|
||||
DOCS_PAGE, projects=projects, input_filename=input.name, include_info=False
|
||||
)
|
||||
text = readme.read()
|
||||
start_str = "<!-- START bin/projects.py -->\n"
|
||||
start = text.find(start_str)
|
||||
end = text.find("<!-- END bin/projects.py -->\n")
|
||||
generated_note = f"<!-- this section is generated by bin/projects.py. Don't edit it directly, instead, edit {input.name} -->"
|
||||
new_text = f"{text[:start + len(start_str)]}\n{generated_note}\n\n{output}\n{text[end:]}"
|
||||
|
||||
readme.seek(0)
|
||||
readme.write(new_text)
|
||||
readme.truncate()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT = Path(__file__).parent / ".."
|
||||
CHANGELOG_FILE = PROJECT_ROOT / "docs" / "changelog.md"
|
||||
|
||||
# https://regexr.com/622ds
|
||||
FIRST_5_CHANGELOG_ENTRIES_REGEX = re.compile(r"""(^###.*?(?=###)){5}""", re.DOTALL | re.MULTILINE)
|
||||
|
||||
|
||||
def mini_changelog() -> str:
|
||||
changelog_text = CHANGELOG_FILE.read_text()
|
||||
|
||||
mini_changelog_match = FIRST_5_CHANGELOG_ENTRIES_REGEX.search(changelog_text)
|
||||
assert mini_changelog_match, "Failed to find the first few changelog entries"
|
||||
|
||||
return f"\n{mini_changelog_match.group(0).strip()}\n"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(mini_changelog())
|
||||
@@ -1,68 +0,0 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
import dataclasses
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
DIR: Final[Path] = Path(__file__).parent.parent.resolve()
|
||||
OPTIONS_MD: Final[Path] = DIR / "docs" / "options.md"
|
||||
|
||||
SECTION_HEADER_REGEX = re.compile(r"^## (?P<name>.*?)$", re.MULTILINE)
|
||||
|
||||
# https://regexr.com/8f1ff
|
||||
OPTION_HEADER_REGEX = re.compile(
|
||||
r"^### (?P<name>.*?){.*#(?P<id>\S+).*}\n+> ?(?P<desc>.*)$", re.MULTILINE
|
||||
)
|
||||
|
||||
|
||||
@dataclasses.dataclass(kw_only=True)
|
||||
class Option:
|
||||
name: str
|
||||
id: str
|
||||
desc: str
|
||||
section: str
|
||||
|
||||
|
||||
def get_table() -> str:
|
||||
options_md = OPTIONS_MD.read_text(encoding="utf-8")
|
||||
|
||||
sections = SECTION_HEADER_REGEX.split(options_md)[1:]
|
||||
|
||||
options = []
|
||||
|
||||
for section_name, section_content in zip(sections[0::2], sections[1::2], strict=True):
|
||||
for match in OPTION_HEADER_REGEX.finditer(section_content):
|
||||
option = Option(
|
||||
name=match.group("name").strip(),
|
||||
id=match.group("id").strip(),
|
||||
desc=match.group("desc").strip(),
|
||||
section=section_name.strip(),
|
||||
)
|
||||
options.append(option)
|
||||
|
||||
table_md = "\n<!-- This table is auto-generated from docs/options.md by bin/readme_options_table.py -->\n\n"
|
||||
table_md += "| | Option | Description |\n"
|
||||
table_md += "|---|---|---|\n"
|
||||
last_section: str | None = None
|
||||
|
||||
for option in options:
|
||||
cells: list[str] = []
|
||||
|
||||
cells.append(f"**{option.section}**" if option.section != last_section else "")
|
||||
last_section = option.section
|
||||
|
||||
url = f"https://cibuildwheel.pypa.io/en/stable/options/#{option.id}"
|
||||
name = option.name.replace(", ", "<br>") # Replace commas with line breaks
|
||||
cells.append(f"[{name}]({url})")
|
||||
|
||||
cells.append(option.desc)
|
||||
|
||||
table_md += "| " + " | ".join(cells) + " |\n"
|
||||
table_md += "\n"
|
||||
|
||||
return table_md
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(get_table())
|
||||
+50
-118
@@ -1,108 +1,50 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "jinja2",
|
||||
# ]
|
||||
# ///
|
||||
#!/usr/bin/env python3
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import textwrap
|
||||
import time
|
||||
import typing
|
||||
from collections import namedtuple
|
||||
from glob import glob
|
||||
from pathlib import Path
|
||||
from subprocess import run
|
||||
from urllib.parse import quote
|
||||
|
||||
import click
|
||||
|
||||
DIR = Path(__file__).parent.parent.resolve()
|
||||
|
||||
BuildBackend = typing.Literal["setuptools", "meson"]
|
||||
def shell(cmd, **kwargs):
|
||||
return run([cmd], shell=True, **kwargs)
|
||||
|
||||
|
||||
def shell(cmd: str, *, check: bool, **kwargs: object) -> subprocess.CompletedProcess[str]:
|
||||
return subprocess.run([cmd], shell=True, check=check, **kwargs) # type: ignore[call-overload, no-any-return]
|
||||
|
||||
|
||||
def git_repo_has_changes() -> bool:
|
||||
unstaged_changes = shell("git diff-index --quiet HEAD --", check=False).returncode != 0
|
||||
staged_changes = shell("git diff-index --quiet --cached HEAD --", check=False).returncode != 0
|
||||
def git_repo_has_changes():
|
||||
unstaged_changes = shell("git diff-index --quiet HEAD --").returncode != 0
|
||||
staged_changes = shell("git diff-index --quiet --cached HEAD --").returncode != 0
|
||||
return unstaged_changes or staged_changes
|
||||
|
||||
|
||||
def generate_project(path: Path, build_backend: BuildBackend) -> None:
|
||||
def generate_basic_project(path):
|
||||
sys.path.insert(0, "")
|
||||
match build_backend:
|
||||
case "meson":
|
||||
from test.test_projects.meson import new_meson_project # noqa: PLC0415
|
||||
|
||||
project = new_meson_project()
|
||||
case "setuptools":
|
||||
from test.test_projects.setuptools import new_c_project # noqa: PLC0415
|
||||
|
||||
project = new_c_project()
|
||||
case _:
|
||||
typing.assert_never(build_backend)
|
||||
from test.test_projects.c import new_c_project
|
||||
|
||||
project = new_c_project()
|
||||
project.generate(path)
|
||||
|
||||
|
||||
class CIService(typing.NamedTuple):
|
||||
name: str
|
||||
dst_config_path: str
|
||||
badge_md: str
|
||||
config_file_transform: typing.Callable[[str, str], str] = lambda x, _: x # identity by default
|
||||
|
||||
|
||||
def github_config_file_transform(content: str, git_ref: str) -> str:
|
||||
# one of the the github configs only builds on main, so we need to remove that restriction
|
||||
# so our example build will run on the test branch.
|
||||
#
|
||||
# replace:
|
||||
# """
|
||||
# push:
|
||||
# branches:
|
||||
# - main
|
||||
# """
|
||||
# with:
|
||||
# """
|
||||
# push:
|
||||
# """"
|
||||
content = re.sub(
|
||||
r"push:\n\s+branches:\n\s+- main",
|
||||
"push:",
|
||||
content,
|
||||
)
|
||||
|
||||
# use the version of cibuildwheel from the current commit, not the latest
|
||||
# release
|
||||
# replace:
|
||||
# """
|
||||
# uses: pypa/cibuildwheel@v3.3.1
|
||||
# """
|
||||
# with:
|
||||
# """
|
||||
# uses: pypa/cibuildwheel@<latest commit hash>
|
||||
# """
|
||||
content = re.sub(
|
||||
r"uses: pypa/cibuildwheel@v.*",
|
||||
f"uses: pypa/cibuildwheel@{git_ref}",
|
||||
content,
|
||||
)
|
||||
return content
|
||||
|
||||
|
||||
CIService = namedtuple("CIService", "name dst_config_path badge_md")
|
||||
services = [
|
||||
CIService(
|
||||
name="appveyor",
|
||||
dst_config_path="appveyor.yml",
|
||||
badge_md="[](https://ci.appveyor.com/project/pypa/cibuildwheel/branch/{branch})",
|
||||
),
|
||||
CIService(
|
||||
name="azure-pipelines",
|
||||
dst_config_path="azure-pipelines.yml",
|
||||
badge_md="[](https://dev.azure.com/joerick0429/cibuildwheel/_build/latest?definitionId=2&branchName={branch})",
|
||||
badge_md="[](https://dev.azure.com/joerick0429/cibuildwheel/_build/latest?definitionId=2&branchName={branch})",
|
||||
),
|
||||
CIService(
|
||||
name="circleci",
|
||||
@@ -112,55 +54,49 @@ services = [
|
||||
CIService(
|
||||
name="github",
|
||||
dst_config_path=".github/workflows/example.yml",
|
||||
badge_md="[](https://github.com/pypa/cibuildwheel/actions?query=branch%3A{branch})",
|
||||
config_file_transform=github_config_file_transform,
|
||||
badge_md="[](https://github.com/pypa/cibuildwheel/actions)",
|
||||
),
|
||||
CIService(
|
||||
name="travis-ci",
|
||||
dst_config_path=".travis.yml",
|
||||
badge_md="[](https://app.travis-ci.com/pypa/cibuildwheel)",
|
||||
badge_md="[](https://travis-ci.org/pypa/cibuildwheel)",
|
||||
),
|
||||
CIService(
|
||||
name="gitlab",
|
||||
dst_config_path=".gitlab-ci.yml",
|
||||
badge_md="[](https://gitlab.com/joerick/cibuildwheel/-/commits/{branch})",
|
||||
badge_md="[](https://gitlab.com/pypa/cibuildwheel/-/commits/{branch})",
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def ci_service_for_config_file(config_file: Path) -> CIService:
|
||||
filename = config_file.name
|
||||
try:
|
||||
return next(s for s in services if filename.startswith(s.name))
|
||||
except StopIteration:
|
||||
msg = f"unknown ci service for config file {config_file}"
|
||||
raise ValueError(msg) from None
|
||||
def ci_service_for_config_file(config_file):
|
||||
service_name = Path(config_file).name.rsplit("-", 1)[0]
|
||||
|
||||
for service in services:
|
||||
if service.name == service_name:
|
||||
return service
|
||||
|
||||
raise ValueError(f"unknown ci service for config file {config_file}")
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("config_files", nargs=-1, type=click.Path())
|
||||
@click.option("--build-backend", type=click.Choice(["setuptools", "meson"]), default="setuptools")
|
||||
def run_example_ci_configs(
|
||||
config_files: list[str], build_backend: BuildBackend = "setuptools"
|
||||
) -> None:
|
||||
def run_example_ci_configs(config_files=None):
|
||||
"""
|
||||
Test the example configs. If no files are specified, will test
|
||||
examples/*-minimal.yml
|
||||
"""
|
||||
|
||||
if len(config_files) == 0:
|
||||
config_file_paths = list(Path("examples").glob("*-minimal.yml"))
|
||||
else:
|
||||
config_file_paths = [Path(f) for f in config_files]
|
||||
config_files = glob("examples/*-minimal.yml")
|
||||
|
||||
# check each CI service has at most 1 config file
|
||||
configs_by_service = set()
|
||||
for config_file in config_file_paths:
|
||||
configs_by_service = {}
|
||||
for config_file in config_files:
|
||||
service = ci_service_for_config_file(config_file)
|
||||
if service.name in configs_by_service:
|
||||
msg = "You cannot specify more than one config per CI service"
|
||||
raise Exception(msg)
|
||||
configs_by_service.add(service.name)
|
||||
raise Exception("You cannot specify more than one config per CI service")
|
||||
configs_by_service[service.name] = config_file
|
||||
|
||||
if git_repo_has_changes():
|
||||
print("Your git repo has uncommitted changes. Commit or stash before continuing.")
|
||||
@@ -169,9 +105,6 @@ def run_example_ci_configs(
|
||||
previous_branch = shell(
|
||||
"git rev-parse --abbrev-ref HEAD", check=True, capture_output=True, encoding="utf8"
|
||||
).stdout.strip()
|
||||
git_ref = shell(
|
||||
"git rev-parse HEAD", check=True, capture_output=True, encoding="utf8"
|
||||
).stdout.strip()
|
||||
|
||||
timestamp = time.strftime("%Y-%m-%dT%H-%M-%S", time.gmtime())
|
||||
branch_name = f"example-config-test---{previous_branch}-{timestamp}"
|
||||
@@ -180,29 +113,27 @@ def run_example_ci_configs(
|
||||
shell(f"git checkout --orphan {branch_name}", check=True)
|
||||
|
||||
example_project = Path("example_root")
|
||||
generate_project(example_project, build_backend=build_backend)
|
||||
generate_basic_project(example_project)
|
||||
|
||||
for config_file in config_file_paths:
|
||||
for config_file in config_files:
|
||||
service = ci_service_for_config_file(config_file)
|
||||
src_config_file = Path(config_file)
|
||||
dst_config_file = example_project / service.dst_config_path
|
||||
|
||||
dst_config_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copyfile(src_config_file, dst_config_file)
|
||||
|
||||
contents = config_file.read_text(encoding="utf8")
|
||||
contents = service.config_file_transform(contents, git_ref)
|
||||
dst_config_file.write_text(contents, encoding="utf8")
|
||||
|
||||
subprocess.run(["git", "add", example_project], check=True)
|
||||
run(["git", "add", example_project], check=True)
|
||||
message = textwrap.dedent(
|
||||
f"""\
|
||||
Test example CI configs
|
||||
Test example minimal configs
|
||||
|
||||
Testing files: {[str(f) for f in config_files]}
|
||||
Testing files: {config_files}
|
||||
Generated from branch: {previous_branch}
|
||||
Time: {timestamp}
|
||||
"""
|
||||
)
|
||||
subprocess.run(["git", "commit", "--no-verify", "--message", message], check=True)
|
||||
run(["git", "commit", "--no-verify", "--message", message], check=True)
|
||||
shell(f"git subtree --prefix={example_project} push origin {branch_name}", check=True)
|
||||
|
||||
print("---")
|
||||
@@ -213,24 +144,25 @@ def run_example_ci_configs(
|
||||
print("> ")
|
||||
print("> | Service | Config | Status |")
|
||||
print("> |---|---|---|")
|
||||
for config_file in config_file_paths:
|
||||
for config_file in config_files:
|
||||
service = ci_service_for_config_file(config_file)
|
||||
badge = service.badge_md.format(
|
||||
branch=branch_name, branch_escaped=quote(branch_name, safe="")
|
||||
)
|
||||
print(f"> | {service.name} | `{config_file}` | {badge} |")
|
||||
print("> ")
|
||||
print(f"> Generated by `{' '.join(sys.argv)}`")
|
||||
print("> Generated by `bin/run_example_ci_config.py`")
|
||||
print()
|
||||
print("---")
|
||||
finally:
|
||||
# remove any local changes
|
||||
shutil.rmtree(example_project, ignore_errors=True)
|
||||
shell("git checkout -- .", check=True)
|
||||
shell("git checkout -- .")
|
||||
shell(f"git checkout {previous_branch}", check=True)
|
||||
shell(f"git branch -D --force {branch_name}", check=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
os.chdir(DIR)
|
||||
os.chdir(os.path.dirname(__file__))
|
||||
os.chdir("..")
|
||||
run_example_ci_configs(standalone_mode=True)
|
||||
|
||||
+8
-111
@@ -1,126 +1,23 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
#!/usr/bin/env python3
|
||||
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
if __name__ == "__main__":
|
||||
if sys.version_info < (3, 13):
|
||||
default_cpu_count = os.cpu_count() or 2
|
||||
else:
|
||||
default_cpu_count = os.process_cpu_count() or 2
|
||||
|
||||
parser = argparse.ArgumentParser(allow_abbrev=False)
|
||||
parser.add_argument(
|
||||
"--run-podman", action="store_true", default=False, help="run podman tests (linux only)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--num-processes",
|
||||
action="store",
|
||||
default=default_cpu_count,
|
||||
help="number of processes to use for testing",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--test-select",
|
||||
choices={"all", "native", "android", "ios", "pyodide"},
|
||||
default="all",
|
||||
help="Either 'native' or 'android'/'ios'/'pyodide'",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
# move cwd to the project root
|
||||
os.chdir(Path(__file__).resolve().parents[1])
|
||||
|
||||
# doc tests
|
||||
doc_test_args = [sys.executable, "-m", "pytest", "cibuildwheel"]
|
||||
|
||||
print(
|
||||
"\n\n================================== DOC TESTS ==================================",
|
||||
flush=True,
|
||||
)
|
||||
result = subprocess.run(doc_test_args, check=False)
|
||||
if result.returncode not in (0, 5):
|
||||
# Allow case where no doctests are collected (returncode 5) because
|
||||
# circleci sets an explicit "-k" filter that disables doctests. There
|
||||
# isn't a pattern that will only select doctests. This can be removed
|
||||
# and have check=True if the circleci PYTEST_ADDOPTS is removed.
|
||||
raise subprocess.CalledProcessError(
|
||||
result.returncode, result.args, output=result.stdout, stderr=result.stderr
|
||||
)
|
||||
|
||||
# unit tests
|
||||
print(
|
||||
"\n\n================================== UNIT TESTS ==================================",
|
||||
flush=True,
|
||||
)
|
||||
# run the unit tests
|
||||
unit_test_args = [sys.executable, "-m", "pytest", "unit_test"]
|
||||
|
||||
if (
|
||||
sys.platform.startswith("linux")
|
||||
and os.environ.get("CIBW_PLATFORM", "linux") == "linux"
|
||||
and args.test_select in ["all", "native"]
|
||||
):
|
||||
# run the docker unit tests only on Linux
|
||||
# run the docker unit tests only on Linux
|
||||
if sys.platform.startswith("linux"):
|
||||
unit_test_args += ["--run-docker"]
|
||||
|
||||
if args.run_podman:
|
||||
unit_test_args += ["--run-podman"]
|
||||
|
||||
subprocess.run(unit_test_args, check=True)
|
||||
|
||||
print(
|
||||
"\n\n=========================== SERIAL INTEGRATION TESTS ===========================",
|
||||
flush=True,
|
||||
# run the integration tests
|
||||
subprocess.run(
|
||||
[sys.executable, "-m", "pytest", "-x", "--durations", "0", "--timeout=2400", "test"],
|
||||
check=True,
|
||||
)
|
||||
|
||||
match args.test_select:
|
||||
case "all":
|
||||
marks = []
|
||||
case "native":
|
||||
marks = ["not pyodide", "not android", "not ios"]
|
||||
case mark:
|
||||
marks = [f"{mark}"]
|
||||
|
||||
# Run the serial integration tests without multiple processes
|
||||
serial_integration_test_args = [
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pytest",
|
||||
"-m",
|
||||
f"{' and '.join(['serial', *marks])}",
|
||||
"-x",
|
||||
"--durations",
|
||||
"0",
|
||||
"--timeout=2400",
|
||||
"test",
|
||||
"-vv",
|
||||
]
|
||||
|
||||
subprocess.run(serial_integration_test_args, check=True)
|
||||
|
||||
print(
|
||||
"\n\n========================= NON-SERIAL INTEGRATION TESTS =========================",
|
||||
flush=True,
|
||||
)
|
||||
integration_test_args = [
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pytest",
|
||||
"-m",
|
||||
f"{' and '.join(['not serial', *marks])}",
|
||||
f"--numprocesses={args.num_processes}",
|
||||
"-x",
|
||||
"--durations",
|
||||
"0",
|
||||
"--timeout=2400",
|
||||
"test",
|
||||
"-vv",
|
||||
]
|
||||
|
||||
if sys.platform.startswith("linux") and args.run_podman:
|
||||
integration_test_args += ["--run-podman"]
|
||||
|
||||
subprocess.run(integration_test_args, check=True)
|
||||
|
||||
+4
-7
@@ -1,5 +1,6 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
#!/usr/bin/env python3
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
@@ -12,7 +13,7 @@ if __name__ == "__main__":
|
||||
# move cwd to the project root
|
||||
os.chdir(Path(__file__).resolve().parents[1])
|
||||
|
||||
parser = argparse.ArgumentParser(description="Runs a sample build", allow_abbrev=False)
|
||||
parser = argparse.ArgumentParser(description="Runs a sample build")
|
||||
parser.add_argument("project_python_path", nargs="?", default="test.test_0_basic.basic_project")
|
||||
|
||||
options = parser.parse_args()
|
||||
@@ -23,8 +24,4 @@ if __name__ == "__main__":
|
||||
check=True,
|
||||
)
|
||||
|
||||
sys.exit(
|
||||
subprocess.run(
|
||||
[sys.executable, "-m", "cibuildwheel"], cwd=project_dir, check=False
|
||||
).returncode
|
||||
)
|
||||
sys.exit(subprocess.run([sys.executable, "-m", "cibuildwheel"], cwd=project_dir).returncode)
|
||||
|
||||
Executable
+142
@@ -0,0 +1,142 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import configparser
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from typing import NamedTuple
|
||||
|
||||
import requests
|
||||
|
||||
os.chdir(os.path.dirname(__file__))
|
||||
os.chdir("..")
|
||||
|
||||
# CUSTOM_COMPILE_COMMAND is a pip-compile option that tells users how to
|
||||
# regenerate the constraints files
|
||||
os.environ["CUSTOM_COMPILE_COMMAND"] = "bin/update_dependencies.py"
|
||||
|
||||
PYTHON_VERSIONS = ["27", "35", "36", "37", "38", "39"]
|
||||
|
||||
if "--no-docker" in sys.argv:
|
||||
for python_version in PYTHON_VERSIONS:
|
||||
subprocess.run(
|
||||
[
|
||||
f"./env{python_version}/bin/pip-compile",
|
||||
"--allow-unsafe",
|
||||
"--upgrade",
|
||||
"cibuildwheel/resources/constraints.in",
|
||||
f"--output-file=cibuildwheel/resources/constraints-python{python_version}.txt",
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
else:
|
||||
# latest manylinux2010 image with cpython 2.7 support
|
||||
image_runner = "quay.io/pypa/manylinux2010_x86_64:2021-02-06-3d322a5"
|
||||
subprocess.run(["docker", "pull", image_runner], check=True)
|
||||
for python_version in PYTHON_VERSIONS:
|
||||
abi_flags = "" if int(python_version) >= 38 else "m"
|
||||
python_path = f"/opt/python/cp{python_version}-cp{python_version}{abi_flags}/bin/"
|
||||
command = (
|
||||
f"{python_path}pip install pip-tools && "
|
||||
f"{python_path}pip-compile --allow-unsafe --upgrade "
|
||||
"cibuildwheel/resources/constraints.in "
|
||||
f"--output-file cibuildwheel/resources/constraints-python{python_version}.txt"
|
||||
)
|
||||
subprocess.run(
|
||||
[
|
||||
"docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--env=CUSTOM_COMPILE_COMMAND",
|
||||
f"--volume={os.getcwd()}:/volume",
|
||||
"--workdir=/volume",
|
||||
image_runner,
|
||||
"bash",
|
||||
"-c",
|
||||
command,
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
|
||||
# default constraints.txt
|
||||
shutil.copyfile(
|
||||
f"cibuildwheel/resources/constraints-python{PYTHON_VERSIONS[-1]}.txt",
|
||||
"cibuildwheel/resources/constraints.txt",
|
||||
)
|
||||
|
||||
|
||||
class Image(NamedTuple):
|
||||
manylinux_version: str
|
||||
platform: str
|
||||
image_name: str
|
||||
tag: str | None
|
||||
|
||||
|
||||
images = [
|
||||
Image("manylinux1", "x86_64", "quay.io/pypa/manylinux1_x86_64", None),
|
||||
Image("manylinux1", "i686", "quay.io/pypa/manylinux1_i686", None),
|
||||
# Images for manylinux2010 are pinned to the latest tag supporting cp27
|
||||
Image("manylinux2010", "x86_64", "quay.io/pypa/manylinux2010_x86_64", "2021-02-06-3d322a5"),
|
||||
Image("manylinux2010", "i686", "quay.io/pypa/manylinux2010_i686", "2021-02-06-3d322a5"),
|
||||
Image("manylinux2010", "pypy_x86_64", "pypywheels/manylinux2010-pypy_x86_64", None),
|
||||
# 2014 images
|
||||
Image("manylinux2014", "x86_64", "quay.io/pypa/manylinux2014_x86_64", None),
|
||||
Image("manylinux2014", "i686", "quay.io/pypa/manylinux2014_i686", None),
|
||||
Image("manylinux2014", "aarch64", "quay.io/pypa/manylinux2014_aarch64", None),
|
||||
Image("manylinux2014", "ppc64le", "quay.io/pypa/manylinux2014_ppc64le", None),
|
||||
Image("manylinux2014", "s390x", "quay.io/pypa/manylinux2014_s390x", None),
|
||||
# 2_24 images
|
||||
Image("manylinux_2_24", "x86_64", "quay.io/pypa/manylinux_2_24_x86_64", None),
|
||||
Image("manylinux_2_24", "i686", "quay.io/pypa/manylinux_2_24_i686", None),
|
||||
Image("manylinux_2_24", "aarch64", "quay.io/pypa/manylinux_2_24_aarch64", None),
|
||||
Image("manylinux_2_24", "ppc64le", "quay.io/pypa/manylinux_2_24_ppc64le", None),
|
||||
Image("manylinux_2_24", "s390x", "quay.io/pypa/manylinux_2_24_s390x", None),
|
||||
]
|
||||
|
||||
config = configparser.ConfigParser()
|
||||
|
||||
for image in images:
|
||||
# get the tag name whose digest matches 'latest'
|
||||
if image.tag is not None:
|
||||
# image has been pinned, do not update
|
||||
tag_name = image.tag
|
||||
elif image.image_name.startswith("quay.io/"):
|
||||
_, _, repository_name = image.image_name.partition("/")
|
||||
response = requests.get(
|
||||
f"https://quay.io/api/v1/repository/{repository_name}?includeTags=true"
|
||||
)
|
||||
response.raise_for_status()
|
||||
repo_info = response.json()
|
||||
tags_dict = repo_info["tags"]
|
||||
|
||||
latest_tag = tags_dict.pop("latest")
|
||||
# find the tag whose manifest matches 'latest'
|
||||
tag_name = next(
|
||||
name
|
||||
for (name, info) in tags_dict.items()
|
||||
if info["manifest_digest"] == latest_tag["manifest_digest"]
|
||||
)
|
||||
else:
|
||||
response = requests.get(f"https://hub.docker.com/v2/repositories/{image.image_name}/tags")
|
||||
response.raise_for_status()
|
||||
tags = response.json()["results"]
|
||||
|
||||
latest_tag = next(tag for tag in tags if tag["name"] == "latest")
|
||||
# i don't know what it would mean to have multiple images per tag
|
||||
assert len(latest_tag["images"]) == 1
|
||||
digest = latest_tag["images"][0]["digest"]
|
||||
|
||||
pinned_tag = next(
|
||||
tag for tag in tags if tag != latest_tag and tag["images"][0]["digest"] == digest
|
||||
)
|
||||
tag_name = pinned_tag["name"]
|
||||
|
||||
if not config.has_section(image.platform):
|
||||
config[image.platform] = {}
|
||||
|
||||
config[image.platform][image.manylinux_version] = f"{image.image_name}:{tag_name}"
|
||||
|
||||
with open("cibuildwheel/resources/pinned_docker_images.cfg", "w") as f:
|
||||
config.write(f)
|
||||
@@ -1,201 +0,0 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "packaging",
|
||||
# "requests",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import configparser
|
||||
import dataclasses
|
||||
from functools import cache
|
||||
from pathlib import Path
|
||||
|
||||
import requests
|
||||
from packaging.version import Version
|
||||
|
||||
DIR = Path(__file__).parent.resolve()
|
||||
RESOURCES = DIR.parent / "cibuildwheel/resources"
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Image:
|
||||
manylinux_version: str
|
||||
platforms: list[str]
|
||||
image_name: str
|
||||
tag: str | None = None # Set this to pin the image
|
||||
use_platform_suffix: bool = False
|
||||
|
||||
|
||||
class PyPAImage(Image):
|
||||
def __init__(self, manylinux_version: str, platforms: list[str], tag: str | None = None):
|
||||
image_name = f"quay.io/pypa/{manylinux_version}"
|
||||
super().__init__(manylinux_version, platforms, image_name, tag, True)
|
||||
|
||||
|
||||
IMAGES = [
|
||||
# manylinux2014 images
|
||||
PyPAImage(
|
||||
"manylinux2014",
|
||||
[
|
||||
"x86_64",
|
||||
"i686",
|
||||
"aarch64",
|
||||
"ppc64le",
|
||||
"s390x",
|
||||
"pypy_x86_64",
|
||||
"pypy_i686",
|
||||
"pypy_aarch64",
|
||||
],
|
||||
),
|
||||
# manylinux_2_28 images
|
||||
PyPAImage(
|
||||
"manylinux_2_28",
|
||||
[
|
||||
"x86_64",
|
||||
"i686",
|
||||
"aarch64",
|
||||
"ppc64le",
|
||||
"s390x",
|
||||
"pypy_x86_64",
|
||||
"pypy_i686",
|
||||
"pypy_aarch64",
|
||||
],
|
||||
),
|
||||
# manylinux_2_31 images
|
||||
PyPAImage("manylinux_2_31", ["armv7l"]),
|
||||
# manylinux_2_34 images
|
||||
PyPAImage(
|
||||
"manylinux_2_34",
|
||||
[
|
||||
"x86_64",
|
||||
"i686",
|
||||
"aarch64",
|
||||
"ppc64le",
|
||||
"s390x",
|
||||
"pypy_x86_64",
|
||||
"pypy_i686",
|
||||
"pypy_aarch64",
|
||||
],
|
||||
),
|
||||
# manylinux_2_35 images
|
||||
PyPAImage("manylinux_2_35", ["armv7l"]),
|
||||
# manylinux_2_39 images
|
||||
PyPAImage("manylinux_2_39", ["riscv64"]),
|
||||
# musllinux_1_2 images
|
||||
PyPAImage(
|
||||
"musllinux_1_2", ["x86_64", "i686", "aarch64", "ppc64le", "s390x", "armv7l", "riscv64"]
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
@cache
|
||||
def quay_lookup(image_name: str, tag_name: str) -> tuple[str, str]:
|
||||
_, _, repository_name = image_name.partition("/")
|
||||
if tag_name == "latest":
|
||||
url = f"https://quay.io/api/v1/repository/{repository_name}?includeTags=true"
|
||||
else:
|
||||
url = f"https://quay.io/api/v1/repository/{repository_name}/tag?specificTag={tag_name}"
|
||||
response = requests.get(url)
|
||||
response.raise_for_status()
|
||||
info = response.json()
|
||||
if tag_name == "latest":
|
||||
tags_dict = info["tags"]
|
||||
tag_info = tags_dict.pop(tag_name)
|
||||
# find the tag whose manifest matches 'latest'
|
||||
tag_name, digest = next(
|
||||
(name, info["manifest_digest"])
|
||||
for (name, info) in tags_dict.items()
|
||||
if info["manifest_digest"] == tag_info["manifest_digest"]
|
||||
)
|
||||
else:
|
||||
tags_list = info["tags"]
|
||||
tag_info = next(tag for tag in tags_list if tag["name"] == tag_name)
|
||||
digest = tag_info["manifest_digest"]
|
||||
|
||||
return tag_name, digest
|
||||
|
||||
|
||||
@cache
|
||||
def ghcr_lookup(image_name: str, tag_name: str) -> tuple[str, str]:
|
||||
repository = image_name[8:]
|
||||
response = requests.get(
|
||||
"https://ghcr.io/token", params={"scope": f"repository:{repository}:pull"}
|
||||
)
|
||||
response.raise_for_status()
|
||||
token = response.json()["token"]
|
||||
if tag_name == "latest":
|
||||
response = requests.get(
|
||||
f"https://ghcr.io/v2/{repository}/tags/list",
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
)
|
||||
response.raise_for_status()
|
||||
info = response.json()
|
||||
ghcr_tags = [(Version(tag), tag) for tag in info["tags"] if tag != "latest"]
|
||||
ghcr_tags.sort(reverse=True)
|
||||
tag_name = ghcr_tags[0][1]
|
||||
|
||||
response = requests.head(
|
||||
f"https://ghcr.io/v2/{repository}/manifests/{tag_name}",
|
||||
headers={
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Accept": "application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.oci.artifact.manifest.v1+json",
|
||||
},
|
||||
)
|
||||
response.raise_for_status()
|
||||
digest = response.headers["Docker-Content-Digest"]
|
||||
return tag_name, digest
|
||||
|
||||
|
||||
@cache
|
||||
def dockerhub_lookup(image_name: str, tag_name: str) -> tuple[str, str]:
|
||||
response = requests.get(f"https://hub.docker.com/v2/repositories/{image_name}/tags")
|
||||
response.raise_for_status()
|
||||
tags = response.json()["results"]
|
||||
if tag_name == "latest":
|
||||
latest_tag = next(tag for tag in tags if tag["name"] == "latest")
|
||||
# i don't know what it would mean to have multiple images per tag
|
||||
assert len(latest_tag["images"]) == 1
|
||||
digest = latest_tag["images"][0]["digest"]
|
||||
|
||||
pinned_tag = next(
|
||||
tag for tag in tags if tag != latest_tag and tag["images"][0]["digest"] == digest
|
||||
)
|
||||
else:
|
||||
pinned_tag = next(tag for tag in tags if tag["name"] == tag_name)
|
||||
digest = pinned_tag["images"][0]["digest"]
|
||||
tag_name = pinned_tag["name"]
|
||||
return tag_name, digest
|
||||
|
||||
|
||||
def main() -> None:
|
||||
config = configparser.ConfigParser()
|
||||
for image in IMAGES:
|
||||
# get the tag name whose digest matches 'latest'
|
||||
# if image has been pinned, do not update
|
||||
search_tag = image.tag or "latest"
|
||||
if image.image_name.startswith("quay.io/"):
|
||||
lookup = quay_lookup
|
||||
elif image.image_name.startswith("ghcr.io/"):
|
||||
lookup = ghcr_lookup
|
||||
else:
|
||||
lookup = dockerhub_lookup
|
||||
|
||||
tag_name, digest = lookup(image.image_name, search_tag)
|
||||
for platform in image.platforms:
|
||||
if not config.has_section(platform):
|
||||
config[platform] = {}
|
||||
image_name = image.image_name
|
||||
if image.use_platform_suffix:
|
||||
image_name = f"{image_name}_{platform.removeprefix('pypy_')}"
|
||||
_, digest = lookup(image_name, tag_name)
|
||||
assert digest.startswith("sha256:")
|
||||
config[platform][image.manylinux_version] = f"{image_name}@{digest} # {tag_name}"
|
||||
|
||||
with open(RESOURCES / "pinned_docker_images.cfg", "w") as f:
|
||||
config.write(f)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,61 +0,0 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "playwright",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from playwright.sync_api import sync_playwright # type: ignore[import-not-found]
|
||||
|
||||
CSS = """
|
||||
<link href="https://fonts.googleapis.com/css2?family=Lato&display=swap" rel="stylesheet">
|
||||
<style>
|
||||
html, body {
|
||||
font-family: Lato, "Helvetica Neue", Helvetica, Arial, sans-serif;
|
||||
font-weight: 400;
|
||||
font-size: 16px;
|
||||
color: #404040;
|
||||
background: white;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
}
|
||||
</style>
|
||||
"""
|
||||
|
||||
|
||||
def main() -> None:
|
||||
subprocess.run([sys.executable, "-m", "playwright", "install", "chromium"], check=True)
|
||||
|
||||
html_str = Path("docs/diagram.html").read_text()
|
||||
html_str = f"<html><head>{CSS}</head><body>{html_str}</body></html>"
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp_dir_str:
|
||||
html_path = Path(tmp_dir_str) / "diagram_screenshot.html"
|
||||
html_path.write_text(html_str)
|
||||
|
||||
dest_path = Path("docs/data/how-it-works.png")
|
||||
|
||||
with sync_playwright() as p:
|
||||
browser = p.chromium.launch()
|
||||
page = browser.new_page(device_scale_factor=2, viewport={"width": 830, "height": 600})
|
||||
page.goto(html_path.as_uri())
|
||||
page.wait_for_load_state("networkidle")
|
||||
|
||||
height = page.evaluate("document.body.scrollHeight")
|
||||
page.set_viewport_size({"width": 830, "height": height})
|
||||
|
||||
page.screenshot(path=str(dest_path), full_page=True)
|
||||
browser.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,155 +0,0 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "packaging",
|
||||
# "requests",
|
||||
# "rich",
|
||||
# ]
|
||||
# ///
|
||||
|
||||
|
||||
import dataclasses
|
||||
import difflib
|
||||
import logging
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
import click
|
||||
import packaging.specifiers
|
||||
import requests
|
||||
import rich
|
||||
from packaging.version import InvalidVersion, Version
|
||||
from rich.logging import RichHandler
|
||||
from rich.syntax import Syntax
|
||||
|
||||
log = logging.getLogger("cibw")
|
||||
|
||||
# Looking up the dir instead of using utils.resources_dir
|
||||
# since we want to write to it.
|
||||
DIR: Final[Path] = Path(__file__).parent.parent.resolve()
|
||||
RESOURCES_DIR: Final[Path] = DIR / "cibuildwheel/resources"
|
||||
|
||||
NODEJS_DIST: Final[str] = "https://nodejs.org/dist/"
|
||||
NODEJS_INDEX: Final[str] = f"{NODEJS_DIST}index.json"
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, order=True)
|
||||
class VersionTuple:
|
||||
version: Version
|
||||
version_string: str
|
||||
|
||||
|
||||
def parse_nodejs_index() -> list[VersionTuple]:
|
||||
versions: list[VersionTuple] = []
|
||||
response = requests.get(NODEJS_INDEX)
|
||||
response.raise_for_status()
|
||||
versions_info = response.json()
|
||||
for version_info in versions_info:
|
||||
version_string = version_info.get("version", "???")
|
||||
if not version_info.get("lts", False):
|
||||
log.debug("Ignoring non LTS release %r", version_string)
|
||||
continue
|
||||
if "linux-x64" not in version_info.get("files", []):
|
||||
log.warning(
|
||||
"Ignoring release %r which does not include a linux-x64 binary", version_string
|
||||
)
|
||||
continue
|
||||
try:
|
||||
version = Version(version_string)
|
||||
if version.is_devrelease:
|
||||
log.info("Ignoring development release %r", str(version))
|
||||
continue
|
||||
if version.is_prerelease:
|
||||
log.info("Ignoring pre-release %r", str(version))
|
||||
continue
|
||||
versions.append(VersionTuple(version, version_string))
|
||||
except InvalidVersion:
|
||||
log.warning("Ignoring release %r", version_string)
|
||||
versions.sort(reverse=True)
|
||||
return versions
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option("--force", is_flag=True)
|
||||
@click.option(
|
||||
"--level", default="INFO", type=click.Choice(["WARNING", "INFO", "DEBUG"], case_sensitive=False)
|
||||
)
|
||||
def update_nodejs(force: bool, level: str) -> None:
|
||||
logging.basicConfig(
|
||||
level="INFO",
|
||||
format="%(message)s",
|
||||
datefmt="[%X]",
|
||||
handlers=[RichHandler(rich_tracebacks=True, markup=True)],
|
||||
)
|
||||
log.setLevel(level)
|
||||
|
||||
toml_file_path = RESOURCES_DIR / "nodejs.toml"
|
||||
|
||||
original_toml = toml_file_path.read_text()
|
||||
with toml_file_path.open("rb") as f:
|
||||
nodejs_data = tomllib.load(f)
|
||||
|
||||
nodejs_data.pop("url")
|
||||
|
||||
major_versions = [VersionTuple(Version(key), key) for key in nodejs_data]
|
||||
major_versions.sort(reverse=True)
|
||||
|
||||
versions = parse_nodejs_index()
|
||||
|
||||
# update existing versions, 1 per LTS
|
||||
for major_version in major_versions:
|
||||
current = Version(nodejs_data[major_version.version_string])
|
||||
specifier = packaging.specifiers.SpecifierSet(
|
||||
specifiers=f"=={major_version.version.major}.*"
|
||||
)
|
||||
for version in versions:
|
||||
if specifier.contains(version.version) and version.version > current:
|
||||
nodejs_data[major_version.version_string] = version.version_string
|
||||
break
|
||||
|
||||
# check for a new major LTS to insert
|
||||
if versions and versions[0].version.major > major_versions[0].version.major:
|
||||
major_versions.insert(
|
||||
0,
|
||||
VersionTuple(Version(str(versions[0].version.major)), f"v{versions[0].version.major}"),
|
||||
)
|
||||
nodejs_data[major_versions[0].version_string] = versions[0].version_string
|
||||
|
||||
versions_toml = "\n".join(
|
||||
f'{major_version.version_string} = "{nodejs_data[major_version.version_string]}"'
|
||||
for major_version in major_versions
|
||||
)
|
||||
result_toml = f'url = "{NODEJS_DIST}"\n{versions_toml}\n'
|
||||
|
||||
rich.print() # spacer
|
||||
|
||||
if original_toml == result_toml:
|
||||
rich.print("[green]Check complete, nodejs version unchanged.")
|
||||
return
|
||||
|
||||
rich.print("nodejs version updated.")
|
||||
rich.print("Changes:")
|
||||
rich.print()
|
||||
|
||||
toml_relpath = toml_file_path.relative_to(DIR).as_posix()
|
||||
diff_lines = difflib.unified_diff(
|
||||
original_toml.splitlines(keepends=True),
|
||||
result_toml.splitlines(keepends=True),
|
||||
fromfile=toml_relpath,
|
||||
tofile=toml_relpath,
|
||||
)
|
||||
rich.print(Syntax("".join(diff_lines), "diff", theme="ansi_light"))
|
||||
rich.print()
|
||||
|
||||
if force:
|
||||
toml_file_path.write_text(result_toml)
|
||||
rich.print("[green]TOML file updated.")
|
||||
else:
|
||||
rich.print("[yellow]File left unchanged. Use --force flag to update.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
update_nodejs()
|
||||
@@ -1,98 +0,0 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "cibuildwheel",
|
||||
# "requests",
|
||||
# ]
|
||||
#
|
||||
# [tool.uv.sources]
|
||||
# cibuildwheel = { path = ".." }
|
||||
# ///
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
import requests
|
||||
|
||||
from cibuildwheel.extra import github_api_request
|
||||
from cibuildwheel.util.python_build_standalone import (
|
||||
PythonBuildStandaloneAsset,
|
||||
PythonBuildStandaloneReleaseData,
|
||||
)
|
||||
|
||||
# Resolve path relative to this script so writes go to the source checkout,
|
||||
# not the uv-installed copy of the package.
|
||||
DIR: Final[Path] = Path(__file__).parent.parent.resolve()
|
||||
PYTHON_BUILD_STANDALONE_RELEASES: Final[Path] = (
|
||||
DIR / "cibuildwheel/resources/python-build-standalone-releases.json"
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""
|
||||
This script updates the vendored list of release assets to the latest
|
||||
version of astral-sh/python-build-standalone.
|
||||
"""
|
||||
|
||||
# Get the latest release tag from the GitHub API
|
||||
latest_release = github_api_request("repos/astral-sh/python-build-standalone/releases/latest")
|
||||
latest_tag = latest_release["tag_name"]
|
||||
|
||||
# Get the list of assets for the latest release
|
||||
github_assets = github_api_request(
|
||||
f"repos/astral-sh/python-build-standalone/releases/tags/{latest_tag}"
|
||||
)["assets"]
|
||||
|
||||
# Build a sha256 map from the SHA256SUMS file in the release
|
||||
sha256_sums_urls = [
|
||||
ga["browser_download_url"] for ga in github_assets if ga["name"] == "SHA256SUMS"
|
||||
]
|
||||
name_to_sha256: dict[str, str] = {}
|
||||
if sha256_sums_urls:
|
||||
response = requests.get(sha256_sums_urls[0])
|
||||
response.raise_for_status()
|
||||
for line in response.text.splitlines():
|
||||
parts = line.split()
|
||||
if len(parts) == 2:
|
||||
sha256_hex, filename = parts
|
||||
# The filename may have a leading "./" or spaces - strip it
|
||||
name_to_sha256[filename.lstrip("./")] = sha256_hex
|
||||
|
||||
assets = [
|
||||
PythonBuildStandaloneAsset(
|
||||
name=ga["name"],
|
||||
url=ga["browser_download_url"],
|
||||
sha256=name_to_sha256.get(ga["name"], ""),
|
||||
)
|
||||
for ga in github_assets
|
||||
if ga["name"].endswith("install_only.tar.gz")
|
||||
]
|
||||
|
||||
# Try to keep output order stable
|
||||
assets = sorted(assets, key=lambda x: x["name"])
|
||||
|
||||
# Write the assets to the JSON file. One day, we might need to support
|
||||
# multiple releases, but for now, we only support the latest one
|
||||
json_file_contents = PythonBuildStandaloneReleaseData(
|
||||
releases=[
|
||||
{
|
||||
"tag": latest_tag,
|
||||
"assets": assets,
|
||||
}
|
||||
]
|
||||
)
|
||||
|
||||
with PYTHON_BUILD_STANDALONE_RELEASES.open("w", encoding="utf-8") as f:
|
||||
json.dump(json_file_contents, f, indent=2)
|
||||
# Add a trailing newline, our pre-commit hook requires it
|
||||
f.write("\n")
|
||||
|
||||
print(
|
||||
f"Updated {PYTHON_BUILD_STANDALONE_RELEASES.name} with {len(assets)} assets for tag {latest_tag}"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+117
-420
@@ -1,43 +1,24 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
#!/usr/bin/env python3
|
||||
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "packaging",
|
||||
# "requests",
|
||||
# "rich",
|
||||
# "cibuildwheel",
|
||||
# ]
|
||||
#
|
||||
# [tool.uv.sources]
|
||||
# cibuildwheel = { path = ".." }
|
||||
# ///
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import difflib
|
||||
import hashlib
|
||||
import logging
|
||||
import operator
|
||||
import re
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
from typing import Any, Final, Literal, NotRequired, TypedDict
|
||||
from xml.etree import ElementTree as ET
|
||||
from typing import Union
|
||||
|
||||
import click
|
||||
import requests
|
||||
import rich
|
||||
import toml
|
||||
from packaging.specifiers import Specifier
|
||||
from packaging.version import Version
|
||||
from rich.logging import RichHandler
|
||||
from rich.syntax import Syntax
|
||||
|
||||
from cibuildwheel.extra import dump_python_configurations, get_pyodide_xbuildenv_info
|
||||
from cibuildwheel.platforms.android import android_triplet
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Mapping, MutableMapping
|
||||
from cibuildwheel.extra import InlineArrayDictEncoder
|
||||
from cibuildwheel.typing import Final, Literal, TypedDict
|
||||
|
||||
log = logging.getLogger("cibw")
|
||||
|
||||
@@ -47,23 +28,29 @@ DIR: Final[Path] = Path(__file__).parent.parent.resolve()
|
||||
RESOURCES_DIR: Final[Path] = DIR / "cibuildwheel/resources"
|
||||
|
||||
|
||||
ArchStr = Literal["32", "64", "ARM64"]
|
||||
ArchStr = Literal["32", "64"]
|
||||
|
||||
|
||||
class Config(TypedDict):
|
||||
class ConfigWinCP(TypedDict):
|
||||
identifier: str
|
||||
version: str
|
||||
arch: str
|
||||
|
||||
|
||||
class ConfigUrl(Config):
|
||||
class ConfigWinPP(TypedDict):
|
||||
identifier: str
|
||||
version: str
|
||||
arch: str
|
||||
url: str
|
||||
sha256: NotRequired[str]
|
||||
|
||||
|
||||
class ConfigPyodide(Config):
|
||||
default_pyodide_version: str
|
||||
node_version: str
|
||||
sha256: str
|
||||
class ConfigMacOS(TypedDict):
|
||||
identifier: str
|
||||
version: str
|
||||
url: str
|
||||
|
||||
|
||||
AnyConfig = Union[ConfigWinCP, ConfigWinPP, ConfigMacOS]
|
||||
|
||||
|
||||
# The following set of "Versions" classes allow the initial call to the APIs to
|
||||
@@ -71,7 +58,8 @@ class ConfigPyodide(Config):
|
||||
|
||||
|
||||
class WindowsVersions:
|
||||
def __init__(self, arch_str: ArchStr, free_threaded: bool) -> None:
|
||||
def __init__(self, arch_str: ArchStr) -> None:
|
||||
|
||||
response = requests.get("https://api.nuget.org/v3/index.json")
|
||||
response.raise_for_status()
|
||||
api_info = response.json()
|
||||
@@ -80,140 +68,41 @@ class WindowsVersions:
|
||||
if resource["@type"] == "PackageBaseAddress/3.0.0":
|
||||
endpoint = resource["@id"]
|
||||
|
||||
ARCH_DICT = {"32": "win32", "64": "win_amd64", "ARM64": "win_arm64"}
|
||||
PACKAGE_DICT = {"32": "pythonx86", "64": "python", "ARM64": "pythonarm64"}
|
||||
ARCH_DICT = {"32": "win32", "64": "win_amd64"}
|
||||
PACKAGE_DICT = {"32": "pythonx86", "64": "python"}
|
||||
|
||||
self.arch_str = arch_str
|
||||
self.arch = ARCH_DICT[arch_str]
|
||||
self.free_threaded = free_threaded
|
||||
|
||||
package = PACKAGE_DICT[arch_str]
|
||||
if free_threaded:
|
||||
package = f"{package}-freethreaded"
|
||||
|
||||
response = requests.get(f"{endpoint}{package}/index.json")
|
||||
response.raise_for_status()
|
||||
cp_info = response.json()
|
||||
|
||||
self.version_dict = {Version(v): v for v in cp_info["versions"]}
|
||||
versions = (Version(v) for v in cp_info["versions"])
|
||||
self.versions = sorted(v for v in versions if not v.is_devrelease)
|
||||
|
||||
def update_version_windows(self, spec: Specifier) -> Config | None:
|
||||
# Specifier.filter selects all non pre-releases that match the spec,
|
||||
# unless there are only pre-releases, then it selects pre-releases
|
||||
# instead (like pip)
|
||||
unsorted_versions = spec.filter(self.version_dict)
|
||||
versions = sorted(unsorted_versions, reverse=True)
|
||||
|
||||
log.debug("Windows %s %s has %s", self.arch, spec, ", ".join(str(v) for v in versions))
|
||||
def update_version_windows(self, spec: Specifier) -> ConfigWinCP | None:
|
||||
versions = sorted(v for v in self.versions if spec.contains(v))
|
||||
if not all(v.is_prerelease for v in versions):
|
||||
versions = [v for v in versions if not v.is_prerelease]
|
||||
log.debug(f"Windows {self.arch} {spec} has {', '.join(str(v) for v in versions)}")
|
||||
|
||||
if not versions:
|
||||
return None
|
||||
|
||||
flags = "t" if self.free_threaded else ""
|
||||
version = versions[0]
|
||||
identifier = f"cp{version.major}{version.minor}{flags}-{self.arch}"
|
||||
return Config(
|
||||
version = versions[-1]
|
||||
identifier = f"cp{version.major}{version.minor}-{self.arch}"
|
||||
return ConfigWinCP(
|
||||
identifier=identifier,
|
||||
version=self.version_dict[version],
|
||||
version=str(version),
|
||||
arch=self.arch_str,
|
||||
)
|
||||
|
||||
|
||||
class GraalPyVersions:
|
||||
def __init__(self) -> None:
|
||||
response = requests.get("https://api.github.com/repos/oracle/graalpython/releases")
|
||||
response.raise_for_status()
|
||||
|
||||
releases = response.json()
|
||||
gp_asset_re = re.compile(
|
||||
r"^(?P<prefix>graalpy(?P<cpython>\d+\.\d+)?-(?P<graalpy>\d+\.\d+\.\d+))-"
|
||||
)
|
||||
cp_version_re = re.compile(r"Python (\d+\.\d+(?:\.\d+)?)")
|
||||
for release in releases:
|
||||
for asset in release["assets"]:
|
||||
m = gp_asset_re.match(asset["name"])
|
||||
if m:
|
||||
release["asset_prefix"] = m.group("prefix")
|
||||
release["graalpy_version"] = Version(m.group("graalpy"))
|
||||
if m.group("cpython"):
|
||||
release["python_version"] = Version(m.group("cpython"))
|
||||
break
|
||||
if "python_version" not in release and (
|
||||
m := cp_version_re.search(release["body"] or "")
|
||||
):
|
||||
release["python_version"] = Version(m.group(1))
|
||||
|
||||
self.releases = [r for r in releases if "graalpy_version" in r and "python_version" in r]
|
||||
|
||||
def update_version(self, identifier: str, spec: Specifier) -> ConfigUrl | None:
|
||||
if "x86_64" in identifier or "amd64" in identifier:
|
||||
arch = "x86_64"
|
||||
elif "arm64" in identifier or "aarch64" in identifier:
|
||||
arch = "aarch64"
|
||||
else:
|
||||
msg = f"{identifier} not supported yet on GraalPy"
|
||||
raise RuntimeError(msg)
|
||||
|
||||
gpspec_str = identifier.split("-", maxsplit=1)[0].split("_")[1]
|
||||
if "." not in gpspec_str and len(gpspec_str) == 3:
|
||||
gpspec_str = gpspec_str[:2] + "." + gpspec_str[-1]
|
||||
gpspec = Specifier(f"=={gpspec_str}.*")
|
||||
|
||||
releases_tmp = (r for r in self.releases if spec.contains(r["python_version"]))
|
||||
releases_tmp = (r for r in releases_tmp if gpspec.contains(r["graalpy_version"]))
|
||||
releases = sorted(releases_tmp, key=lambda r: r["graalpy_version"])
|
||||
|
||||
if not releases:
|
||||
msg = f"GraalPy {arch} not found for {spec}!"
|
||||
raise RuntimeError(msg)
|
||||
|
||||
if "macosx" in identifier:
|
||||
arch = "x86_64" if "x86_64" in identifier else "arm64"
|
||||
platform = "macos"
|
||||
elif "win" in identifier:
|
||||
arch = "aarch64" if "arm64" in identifier else "x86_64"
|
||||
platform = "windows"
|
||||
else:
|
||||
msg = "GraalPy provides downloads for macOS and Windows and is included for manylinux"
|
||||
raise RuntimeError(msg)
|
||||
|
||||
arch = "amd64" if arch == "x86_64" else "aarch64"
|
||||
ext = "zip" if "win" in identifier else "tar.gz"
|
||||
for release in reversed(releases):
|
||||
version = release["python_version"]
|
||||
urls = [
|
||||
rf["browser_download_url"]
|
||||
for rf in release["assets"]
|
||||
if rf["name"].endswith(f"{platform}-{arch}.{ext}")
|
||||
and rf["name"].startswith(release["asset_prefix"])
|
||||
]
|
||||
if not urls:
|
||||
continue
|
||||
|
||||
(url,) = urls
|
||||
# Fetch sha256 from the ".sha256" sidecar asset in the same release.
|
||||
sha256 = ""
|
||||
sha256_asset_name = url.rsplit("/", 1)[-1] + ".sha256"
|
||||
sha256_urls = [
|
||||
rf["browser_download_url"]
|
||||
for rf in release["assets"]
|
||||
if rf["name"] == sha256_asset_name
|
||||
]
|
||||
if sha256_urls:
|
||||
sha256_response = requests.get(sha256_urls[0])
|
||||
sha256_response.raise_for_status()
|
||||
sha256 = sha256_response.text.strip().split()[0]
|
||||
return ConfigUrl(
|
||||
identifier=identifier,
|
||||
version=f"{version.major}.{version.minor}",
|
||||
url=url,
|
||||
sha256=sha256,
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
class PyPyVersions:
|
||||
def __init__(self, arch_str: ArchStr):
|
||||
|
||||
response = requests.get("https://downloads.python.org/pypy/versions.json")
|
||||
response.raise_for_status()
|
||||
|
||||
@@ -229,61 +118,50 @@ class PyPyVersions:
|
||||
]
|
||||
self.arch = arch_str
|
||||
|
||||
def get_arch_file(self, release: Mapping[str, Any]) -> str:
|
||||
urls: list[str] = [
|
||||
rf["download_url"]
|
||||
for rf in release["files"]
|
||||
if "" in rf["platform"] == f"win{self.arch}"
|
||||
]
|
||||
return urls[0] if urls else ""
|
||||
def update_version_windows(self, spec: Specifier) -> ConfigWinCP:
|
||||
if self.arch != "32":
|
||||
raise RuntimeError("64 bit releases not supported yet on Windows")
|
||||
|
||||
def update_version_windows(self, spec: Specifier) -> ConfigUrl:
|
||||
releases = [r for r in self.releases if spec.contains(r["python_version"])]
|
||||
releases = sorted(releases, key=operator.itemgetter("pypy_version"))
|
||||
releases = [r for r in releases if self.get_arch_file(r)]
|
||||
releases = sorted(releases, key=lambda r: r["pypy_version"])
|
||||
|
||||
if not releases:
|
||||
msg = f"PyPy Win {self.arch} not found for {spec}! {self.releases}"
|
||||
raise RuntimeError(msg)
|
||||
|
||||
version_arch = "win32" if self.arch == "32" else "win_amd64"
|
||||
raise RuntimeError(f"PyPy Win {self.arch} not found for {spec}! {self.releases}")
|
||||
|
||||
release = releases[-1]
|
||||
version = release["python_version"]
|
||||
identifier = f"pp{version.major}{version.minor}-{version_arch}"
|
||||
url = self.get_arch_file(release)
|
||||
identifier = f"pp{version.major}{version.minor}-win32"
|
||||
|
||||
return ConfigUrl(
|
||||
(url,) = [rf["download_url"] for rf in release["files"] if "" in rf["platform"] == "win32"]
|
||||
|
||||
return ConfigWinPP(
|
||||
identifier=identifier,
|
||||
version=f"{version.major}.{version.minor}",
|
||||
arch="32",
|
||||
url=url,
|
||||
)
|
||||
|
||||
def update_version_macos(self, spec: Specifier) -> ConfigUrl:
|
||||
if self.arch not in {"64", "ARM64"}:
|
||||
msg = f"'{self.arch}' arch not supported yet on macOS"
|
||||
raise RuntimeError(msg)
|
||||
def update_version_macos(self, spec: Specifier) -> ConfigMacOS:
|
||||
if self.arch != "64":
|
||||
raise RuntimeError("Other archs not supported yet on macOS")
|
||||
|
||||
releases = [r for r in self.releases if spec.contains(r["python_version"])]
|
||||
releases = sorted(releases, key=operator.itemgetter("pypy_version"))
|
||||
releases = sorted(releases, key=lambda r: r["pypy_version"])
|
||||
|
||||
if not releases:
|
||||
msg = f"PyPy macOS {self.arch} not found for {spec}!"
|
||||
raise RuntimeError(msg)
|
||||
raise RuntimeError(f"PyPy macOS {self.arch} not found for {spec}!")
|
||||
|
||||
release = releases[-1]
|
||||
version = release["python_version"]
|
||||
arch = "x86_64" if self.arch == "64" else self.arch.lower()
|
||||
identifier = f"pp{version.major}{version.minor}-macosx_{arch}"
|
||||
identifier = f"pp{version.major}{version.minor}-macosx_x86_64"
|
||||
|
||||
arch = "x64" if self.arch == "64" else self.arch.lower()
|
||||
(url,) = (
|
||||
(url,) = [
|
||||
rf["download_url"]
|
||||
for rf in release["files"]
|
||||
if "" in rf["platform"] == "darwin" and rf["arch"] == arch
|
||||
)
|
||||
if "" in rf["platform"] == "darwin" and rf["arch"] == "x64"
|
||||
]
|
||||
|
||||
return ConfigUrl(
|
||||
return ConfigMacOS(
|
||||
identifier=identifier,
|
||||
version=f"{version.major}.{version.minor}",
|
||||
url=url,
|
||||
@@ -292,6 +170,7 @@ class PyPyVersions:
|
||||
|
||||
class CPythonVersions:
|
||||
def __init__(self) -> None:
|
||||
|
||||
response = requests.get(
|
||||
"https://www.python.org/api/v2/downloads/release/?is_published=true"
|
||||
)
|
||||
@@ -301,155 +180,37 @@ class CPythonVersions:
|
||||
|
||||
self.versions_dict: dict[Version, int] = {}
|
||||
for release in releases_info:
|
||||
# Skip the pymanager releases
|
||||
if not release["slug"].startswith("python"):
|
||||
continue
|
||||
# Removing the prefix, Python 3.9 would use: release["name"].removeprefix("Python ")
|
||||
version = Version(release["name"][7:])
|
||||
|
||||
# Removing the prefix
|
||||
version = Version(release["name"].removeprefix("Python "))
|
||||
self.versions_dict[version] = release["resource_uri"]
|
||||
if not version.is_prerelease and not version.is_devrelease:
|
||||
uri = int(release["resource_uri"].rstrip("/").split("/")[-1])
|
||||
self.versions_dict[version] = uri
|
||||
|
||||
files_response = requests.get("https://www.python.org/api/v2/downloads/release_file/")
|
||||
files_response.raise_for_status()
|
||||
self.files_info = files_response.json()
|
||||
def update_version_macos(self, identifier: str, spec: Specifier) -> ConfigMacOS | None:
|
||||
file_idents = ("macos11.pkg", "macosx10.9.pkg", "macosx10.6.pkg")
|
||||
sorted_versions = sorted(v for v in self.versions_dict if spec.contains(v))
|
||||
|
||||
def update_version(self, identifier: str, spec: Specifier, file_ident: str) -> ConfigUrl | None:
|
||||
# see note above on Specifier.filter
|
||||
unsorted_versions = spec.filter(self.versions_dict)
|
||||
sorted_versions = sorted(unsorted_versions, reverse=True)
|
||||
|
||||
for new_version in sorted_versions:
|
||||
for version in reversed(sorted_versions):
|
||||
# Find the first patch version that contains the requested file
|
||||
uri = self.versions_dict[new_version]
|
||||
files = [rf for rf in self.files_info if rf["release"] == uri]
|
||||
uri = self.versions_dict[version]
|
||||
response = requests.get(
|
||||
f"https://www.python.org/api/v2/downloads/release_file/?release={uri}"
|
||||
)
|
||||
response.raise_for_status()
|
||||
file_info = response.json()
|
||||
|
||||
matching = [rf for rf in files if file_ident in rf["url"]]
|
||||
if matching:
|
||||
rf = matching[0]
|
||||
return ConfigUrl(
|
||||
identifier=identifier,
|
||||
version=f"{new_version.major}.{new_version.minor}",
|
||||
url=rf["url"],
|
||||
sha256=rf.get("sha256_sum", ""),
|
||||
)
|
||||
for file_ident in file_idents:
|
||||
urls = [rf["url"] for rf in file_info if file_ident in rf["url"]]
|
||||
if urls:
|
||||
return ConfigMacOS(
|
||||
identifier=identifier,
|
||||
version=f"{version.major}.{version.minor}",
|
||||
url=urls[0],
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
def update_version_macos(
|
||||
self, identifier: str, version: Version, spec: Specifier
|
||||
) -> ConfigUrl | None:
|
||||
macver = "x10.9" if version <= Version("3.8.9999") else "11"
|
||||
return self.update_version(identifier, spec, f"macos{macver}.pkg")
|
||||
|
||||
def update_version_android(self, identifier: str, spec: Specifier) -> ConfigUrl | None:
|
||||
return self.update_version(identifier, spec, android_triplet(identifier))
|
||||
|
||||
|
||||
class MavenVersions:
|
||||
MAVEN_URL = "https://repo.maven.apache.org/maven2/com/chaquo/python/python"
|
||||
|
||||
def __init__(self) -> None:
|
||||
response = requests.get(f"{self.MAVEN_URL}/maven-metadata.xml")
|
||||
response.raise_for_status()
|
||||
root = ET.fromstring(response.text)
|
||||
|
||||
self.versions: list[Version] = []
|
||||
for version_elem in root.findall("./versioning/versions/version"):
|
||||
version_str = version_elem.text
|
||||
assert isinstance(version_str, str), version_str
|
||||
self.versions.append(Version(version_str))
|
||||
|
||||
def update_version_android(self, identifier: str, spec: Specifier) -> ConfigUrl | None:
|
||||
sorted_versions = sorted(spec.filter(self.versions), reverse=True)
|
||||
|
||||
# Return a config using the highest version for the given specifier.
|
||||
if sorted_versions:
|
||||
max_version = sorted_versions[0]
|
||||
triplet = android_triplet(identifier)
|
||||
return ConfigUrl(
|
||||
identifier=identifier,
|
||||
version=f"{max_version.major}.{max_version.minor}",
|
||||
url=f"{self.MAVEN_URL}/{max_version}/python-{max_version}-{triplet}.tar.gz",
|
||||
)
|
||||
else:
|
||||
return None
|
||||
|
||||
|
||||
class CPythonIOSVersions:
|
||||
def __init__(self) -> None:
|
||||
response = requests.get(
|
||||
"https://api.github.com/repos/beeware/Python-Apple-support/releases",
|
||||
headers={
|
||||
"Accept": "application/vnd.github+json",
|
||||
"X-Github-Api-Version": "2022-11-28",
|
||||
},
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
releases_info = response.json()
|
||||
self.versions_dict: dict[Version, dict[int, str]] = {}
|
||||
|
||||
# Each release has a name like "3.13-b4"
|
||||
for release in releases_info:
|
||||
py_version, build = release["name"].split("-")
|
||||
version = Version(py_version)
|
||||
self.versions_dict.setdefault(version, {})
|
||||
|
||||
# There are several release assets associated with each release;
|
||||
# The name of the asset will be something like
|
||||
# "Python-3.11-iOS-support.b4.tar.gz". Store all builds that are
|
||||
# "-iOS-support" builds, retaining the download URL.
|
||||
for asset in release["assets"]:
|
||||
filename, build, _, _ = asset["name"].rsplit(".", 3)
|
||||
if filename.endswith("-iOS-support"):
|
||||
self.versions_dict[version][int(build[1:])] = asset["browser_download_url"]
|
||||
|
||||
def update_version_ios(self, identifier: str, version: Version) -> ConfigUrl | None:
|
||||
# Return a config using the highest build number for the given version.
|
||||
urls = [url for _, url in sorted(self.versions_dict.get(version, {}).items())]
|
||||
if urls:
|
||||
return ConfigUrl(
|
||||
identifier=identifier,
|
||||
version=str(version),
|
||||
url=urls[-1],
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
class PyodideVersions:
|
||||
def __init__(self) -> None:
|
||||
xbuildenv_info = get_pyodide_xbuildenv_info()
|
||||
self.releases = xbuildenv_info["releases"]
|
||||
|
||||
def update_version_pyodide(
|
||||
self, identifier: str, version: Version, spec: Specifier, node_version: str
|
||||
) -> ConfigPyodide | None:
|
||||
# get releases that match the python version
|
||||
releases = [
|
||||
r for r in self.releases.values() if spec.contains(Version(r["python_version"]))
|
||||
]
|
||||
# sort by version, latest first
|
||||
releases.sort(key=lambda r: Version(r["version"]), reverse=True)
|
||||
|
||||
if not releases:
|
||||
msg = f"Pyodide not found for {spec}!"
|
||||
raise ValueError(msg)
|
||||
|
||||
final_releases = [r for r in releases if not Version(r["version"]).is_prerelease]
|
||||
|
||||
# prefer a final release if available, otherwise use the latest
|
||||
# pre-release
|
||||
release = final_releases[0] if final_releases else releases[0]
|
||||
|
||||
return ConfigPyodide(
|
||||
identifier=identifier,
|
||||
version=str(version),
|
||||
default_pyodide_version=release["version"],
|
||||
node_version=node_version,
|
||||
sha256=release["sha256"],
|
||||
)
|
||||
|
||||
|
||||
# This is a universal interface to all the above Versions classes. Given an
|
||||
# identifier, it updates a config dict.
|
||||
@@ -457,118 +218,53 @@ class PyodideVersions:
|
||||
|
||||
class AllVersions:
|
||||
def __init__(self) -> None:
|
||||
self.windows_32 = WindowsVersions("32", False)
|
||||
self.windows_t_32 = WindowsVersions("32", True)
|
||||
self.windows_64 = WindowsVersions("64", False)
|
||||
self.windows_t_64 = WindowsVersions("64", True)
|
||||
self.windows_arm64 = WindowsVersions("ARM64", False)
|
||||
self.windows_t_arm64 = WindowsVersions("ARM64", True)
|
||||
self.windows_pypy_64 = PyPyVersions("64")
|
||||
self.windows_32 = WindowsVersions("32")
|
||||
self.windows_64 = WindowsVersions("64")
|
||||
self.windows_pypy = PyPyVersions("32")
|
||||
|
||||
self.cpython = CPythonVersions()
|
||||
self.macos_cpython = CPythonVersions()
|
||||
self.macos_pypy = PyPyVersions("64")
|
||||
self.macos_pypy_arm64 = PyPyVersions("ARM64")
|
||||
|
||||
self.maven = MavenVersions()
|
||||
self.ios_cpython = CPythonIOSVersions()
|
||||
|
||||
self.graalpy = GraalPyVersions()
|
||||
|
||||
self.pyodide = PyodideVersions()
|
||||
|
||||
def _stream_sha256(self, url: str) -> str:
|
||||
"""Download a file (streaming) and return its SHA256 hex digest."""
|
||||
log.debug("Computing sha256 for %s by streaming download...", url)
|
||||
response = requests.get(url, stream=True)
|
||||
response.raise_for_status()
|
||||
hasher = hashlib.sha256()
|
||||
for chunk in response.iter_content(65536):
|
||||
hasher.update(chunk)
|
||||
return hasher.hexdigest()
|
||||
|
||||
def update_config(self, config: MutableMapping[str, str]) -> None:
|
||||
def update_config(self, config: dict[str, str]) -> None:
|
||||
identifier = config["identifier"]
|
||||
version = Version(config["version"])
|
||||
spec = Specifier(f"=={version.major}.{version.minor}.*")
|
||||
log.info("Reading in %r -> %s @ %s", str(identifier), spec, version)
|
||||
config_update: Config | None = None
|
||||
log.info(f"Reading in '{identifier}' -> {spec} @ {version}")
|
||||
orig_config = copy.copy(config)
|
||||
config_update: AnyConfig | None
|
||||
|
||||
# We need to use ** in update due to MyPy (probably a bug)
|
||||
if "macosx" in identifier:
|
||||
if identifier.startswith("cp"):
|
||||
config_update = self.cpython.update_version_macos(identifier, version, spec)
|
||||
elif identifier.startswith("pp"):
|
||||
if "macosx_x86_64" in identifier:
|
||||
config_update = self.macos_pypy.update_version_macos(spec)
|
||||
elif "macosx_arm64" in identifier:
|
||||
config_update = self.macos_pypy_arm64.update_version_macos(spec)
|
||||
elif identifier.startswith("gp"):
|
||||
config_update = self.graalpy.update_version(identifier, spec)
|
||||
elif "t-win32" in identifier and identifier.startswith("cp"):
|
||||
config_update = self.windows_t_32.update_version_windows(spec)
|
||||
elif "win32" in identifier and identifier.startswith("cp"):
|
||||
config_update = self.windows_32.update_version_windows(spec)
|
||||
elif "t-win_amd64" in identifier and identifier.startswith("cp"):
|
||||
config_update = self.windows_t_64.update_version_windows(spec)
|
||||
elif "win_amd64" in identifier:
|
||||
if identifier.startswith("cp"):
|
||||
config_update = self.windows_64.update_version_windows(spec)
|
||||
elif identifier.startswith("pp"):
|
||||
config_update = self.windows_pypy_64.update_version_windows(spec)
|
||||
elif identifier.startswith("gp"):
|
||||
config_update = self.graalpy.update_version(identifier, spec)
|
||||
elif "t-win_arm64" in identifier and identifier.startswith("cp"):
|
||||
config_update = self.windows_t_arm64.update_version_windows(spec)
|
||||
elif "win_arm64" in identifier and identifier.startswith("cp"):
|
||||
config_update = self.windows_arm64.update_version_windows(spec)
|
||||
elif "android" in identifier:
|
||||
# Python 3.13 is released by Chaquopy on Maven Central.
|
||||
# Python 3.14 and newer have official releases on python.org.
|
||||
versions = self.maven if identifier.startswith("cp313") else self.cpython
|
||||
config_update = versions.update_version_android(identifier, spec)
|
||||
elif "ios" in identifier:
|
||||
# Python 3.13 and 3.14 are released by BeeWare on GitHub.
|
||||
# Python 3.15 and newer have official releases on python.org.
|
||||
if identifier.startswith(("cp313", "cp314")):
|
||||
config_update = self.ios_cpython.update_version_ios(identifier, version)
|
||||
if "macos" in identifier:
|
||||
if identifier.startswith("pp"):
|
||||
config_update = self.macos_pypy.update_version_macos(spec)
|
||||
else:
|
||||
config_update = self.cpython.update_version(
|
||||
identifier, spec, "iOS-XCframework.tar.gz"
|
||||
)
|
||||
elif "pyodide" in identifier:
|
||||
config_update = self.pyodide.update_version_pyodide(
|
||||
identifier, version, spec, config["node_version"]
|
||||
)
|
||||
config_update = self.macos_cpython.update_version_macos(identifier, spec)
|
||||
|
||||
assert config_update is not None, f"{identifier} not found!"
|
||||
|
||||
# Fill in sha256 for URL-based configs if not already provided by the
|
||||
# update_version_* method (e.g. PyPy, BeeWare iOS, Maven have no sidecar).
|
||||
# Also fills in sha256 when the CPython API doesn't return a sha256_sum
|
||||
# (e.g. for older releases).
|
||||
# Widen the type to allow arbitrary key access on the underlying dict.
|
||||
config_update_dict: dict[str, str] = config_update # type: ignore[assignment]
|
||||
if "url" in config_update_dict and not config_update_dict.get("sha256"):
|
||||
url = config_update_dict["url"]
|
||||
existing_sha256 = config.get("sha256", "")
|
||||
if url == config.get("url") and existing_sha256:
|
||||
# URL unchanged — preserve the existing sha256
|
||||
config_update_dict["sha256"] = existing_sha256
|
||||
else:
|
||||
config_update_dict["sha256"] = self._stream_sha256(url)
|
||||
|
||||
if config_update != config:
|
||||
log.info(" Updated %s to %s", config, config_update)
|
||||
config.clear()
|
||||
assert config_update is not None, f"MacOS {spec} not found!"
|
||||
config.update(**config_update)
|
||||
elif "win32" in identifier:
|
||||
if identifier.startswith("pp"):
|
||||
config.update(**self.windows_pypy.update_version_windows(spec))
|
||||
else:
|
||||
config_update = self.windows_32.update_version_windows(spec)
|
||||
if config_update:
|
||||
config.update(**config_update)
|
||||
elif "win_amd64" in identifier:
|
||||
config_update = self.windows_64.update_version_windows(spec)
|
||||
if config_update:
|
||||
config.update(**config_update)
|
||||
|
||||
if config != orig_config:
|
||||
log.info(f" Updated {orig_config} to {config}")
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option("--force", is_flag=True)
|
||||
@click.option(
|
||||
"--level", default="INFO", type=click.Choice(["WARNING", "INFO", "DEBUG"], case_sensitive=False)
|
||||
"--level", default="INFO", type=click.Choice(["INFO", "DEBUG", "TRACE"], case_sensitive=False)
|
||||
)
|
||||
def update_pythons(force: bool, level: str) -> None:
|
||||
|
||||
logging.basicConfig(
|
||||
level="INFO",
|
||||
format="%(message)s",
|
||||
@@ -581,14 +277,15 @@ def update_pythons(force: bool, level: str) -> None:
|
||||
toml_file_path = RESOURCES_DIR / "build-platforms.toml"
|
||||
|
||||
original_toml = toml_file_path.read_text()
|
||||
with toml_file_path.open("rb") as f:
|
||||
configs = tomllib.load(f)
|
||||
configs = toml.loads(original_toml)
|
||||
|
||||
for platform in ["windows", "macos", "android", "ios", "pyodide"]:
|
||||
for config in configs[platform]["python_configurations"]:
|
||||
all_versions.update_config(config)
|
||||
for config in configs["windows"]["python_configurations"]:
|
||||
all_versions.update_config(config)
|
||||
|
||||
result_toml = dump_python_configurations(configs)
|
||||
for config in configs["macos"]["python_configurations"]:
|
||||
all_versions.update_config(config)
|
||||
|
||||
result_toml = toml.dumps(configs, encoder=InlineArrayDictEncoder()) # type: ignore
|
||||
|
||||
rich.print() # spacer
|
||||
|
||||
|
||||
@@ -1,152 +0,0 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
|
||||
# /// script
|
||||
# dependencies = [
|
||||
# "click",
|
||||
# "packaging",
|
||||
# "requests",
|
||||
# "rich",
|
||||
# "cibuildwheel",
|
||||
# ]
|
||||
#
|
||||
# [tool.uv.sources]
|
||||
# cibuildwheel = { path = ".." }
|
||||
# ///
|
||||
|
||||
|
||||
import dataclasses
|
||||
import difflib
|
||||
import hashlib
|
||||
import logging
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
import click
|
||||
import requests
|
||||
import rich
|
||||
from packaging.version import Version
|
||||
from rich.logging import RichHandler
|
||||
from rich.syntax import Syntax
|
||||
|
||||
from cibuildwheel.extra import github_api_request
|
||||
|
||||
log = logging.getLogger("cibw")
|
||||
|
||||
# Looking up the dir instead of using utils.resources_dir
|
||||
# since we want to write to it.
|
||||
DIR: Final[Path] = Path(__file__).parent.parent.resolve()
|
||||
RESOURCES_DIR: Final[Path] = DIR / "cibuildwheel/resources"
|
||||
|
||||
GET_VIRTUALENV_GITHUB: Final[str] = "https://github.com/pypa/get-virtualenv"
|
||||
GET_VIRTUALENV_URL_TEMPLATE: Final[str] = (
|
||||
f"{GET_VIRTUALENV_GITHUB}/blob/{{version}}/public/virtualenv.pyz?raw=true"
|
||||
)
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, order=True)
|
||||
class VersionTuple:
|
||||
name: str
|
||||
download_url: str
|
||||
version: Version
|
||||
|
||||
|
||||
def get_latest_virtualenv_release() -> VersionTuple:
|
||||
response = github_api_request("repos/pypa/get-virtualenv/releases/latest")
|
||||
tag_name = response["tag_name"]
|
||||
|
||||
asset = next(
|
||||
(asset for asset in response["assets"] if asset["name"] == "virtualenv.pyz"),
|
||||
None,
|
||||
)
|
||||
if not asset:
|
||||
msg = "No asset named 'virtualenv.pyz' found in the latest release of get-virtualenv."
|
||||
raise RuntimeError(msg)
|
||||
|
||||
return VersionTuple(
|
||||
version=Version(tag_name), name=tag_name, download_url=asset["browser_download_url"]
|
||||
)
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option("--force", is_flag=True)
|
||||
@click.option(
|
||||
"--level", default="INFO", type=click.Choice(["WARNING", "INFO", "DEBUG"], case_sensitive=False)
|
||||
)
|
||||
def update_virtualenv(force: bool, level: str) -> None:
|
||||
logging.basicConfig(
|
||||
level="INFO",
|
||||
format="%(message)s",
|
||||
datefmt="[%X]",
|
||||
handlers=[RichHandler(rich_tracebacks=True, markup=True)],
|
||||
)
|
||||
log.setLevel(level)
|
||||
|
||||
toml_file_path = RESOURCES_DIR / "virtualenv.toml"
|
||||
|
||||
original_toml = toml_file_path.read_text()
|
||||
with toml_file_path.open("rb") as f:
|
||||
configurations = tomllib.load(f)
|
||||
default = configurations.pop("default")
|
||||
local_version = str(default["version"])
|
||||
|
||||
latest_release = get_latest_virtualenv_release()
|
||||
|
||||
if latest_release.version > Version(local_version):
|
||||
version = latest_release.name
|
||||
url = latest_release.download_url
|
||||
sha256 = "" # recomputed below
|
||||
else:
|
||||
version = local_version
|
||||
url = default["url"]
|
||||
sha256 = default["sha256"]
|
||||
|
||||
# Compute sha256 if not already stored (new version or first-time population)
|
||||
if not sha256:
|
||||
log.info("Computing sha256 for %s...", url)
|
||||
response = requests.get(url, stream=True)
|
||||
response.raise_for_status()
|
||||
hasher = hashlib.sha256()
|
||||
for chunk in response.iter_content(65536):
|
||||
hasher.update(chunk)
|
||||
sha256 = hasher.hexdigest()
|
||||
|
||||
configurations["default"] = {
|
||||
"version": version,
|
||||
"url": url,
|
||||
"sha256": sha256,
|
||||
}
|
||||
result_toml = "".join(
|
||||
f'{key} = {{ version = "{value["version"]}", url = "{value["url"]}", sha256 = "{value["sha256"]}" }}\n'
|
||||
for key, value in configurations.items()
|
||||
)
|
||||
|
||||
rich.print() # spacer
|
||||
|
||||
if original_toml == result_toml:
|
||||
rich.print("[green]Check complete, virtualenv version unchanged.")
|
||||
return
|
||||
|
||||
rich.print("virtualenv version updated.")
|
||||
rich.print("Changes:")
|
||||
rich.print()
|
||||
|
||||
toml_relpath = toml_file_path.relative_to(DIR).as_posix()
|
||||
diff_lines = difflib.unified_diff(
|
||||
original_toml.splitlines(keepends=True),
|
||||
result_toml.splitlines(keepends=True),
|
||||
fromfile=toml_relpath,
|
||||
tofile=toml_relpath,
|
||||
)
|
||||
rich.print(Syntax("".join(diff_lines), "diff", theme="ansi_light"))
|
||||
rich.print()
|
||||
|
||||
if force:
|
||||
toml_file_path.write_text(result_toml)
|
||||
rich.print("[green]TOML file updated.")
|
||||
else:
|
||||
rich.print("[yellow]File left unchanged. Use --force flag to update.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
update_virtualenv()
|
||||
@@ -1 +1 @@
|
||||
__version__ = "4.1.1"
|
||||
__version__ = "1.11.1.post1"
|
||||
|
||||
+337
-451
@@ -1,123 +1,90 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {
|
||||
"argparse",
|
||||
"cibuildwheel._compat",
|
||||
"cibuildwheel._compat.tarfile",
|
||||
"cibuildwheel.architecture",
|
||||
"cibuildwheel.ci",
|
||||
"cibuildwheel.logger",
|
||||
"cibuildwheel.options",
|
||||
"cibuildwheel.platforms",
|
||||
"cibuildwheel.selector",
|
||||
"cibuildwheel.typing",
|
||||
"cibuildwheel.util",
|
||||
"cibuildwheel.util.file",
|
||||
"cibuildwheel.util.helpers",
|
||||
"cibuildwheel.util.resources",
|
||||
"contextlib",
|
||||
"functools",
|
||||
"io",
|
||||
"pathlib",
|
||||
"shutil",
|
||||
"tempfile",
|
||||
"textwrap",
|
||||
"traceback",
|
||||
}
|
||||
|
||||
import argparse
|
||||
import contextlib
|
||||
import dataclasses
|
||||
import functools
|
||||
import io
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import textwrap
|
||||
import traceback
|
||||
import typing
|
||||
from configparser import ConfigParser
|
||||
from pathlib import Path
|
||||
from tempfile import mkdtemp
|
||||
from typing import Dict, List, Optional, Set, Union, overload
|
||||
|
||||
from packaging.specifiers import SpecifierSet
|
||||
|
||||
import cibuildwheel
|
||||
from cibuildwheel import errors
|
||||
from cibuildwheel._compat.tarfile import TarFile, safe_extractall
|
||||
import cibuildwheel.linux
|
||||
import cibuildwheel.macos
|
||||
import cibuildwheel.util
|
||||
import cibuildwheel.windows
|
||||
from cibuildwheel.architecture import Architecture, allowed_architectures_check
|
||||
from cibuildwheel.ci import CIProvider, detect_ci_provider, fix_ansi_codes_for_github_actions
|
||||
from cibuildwheel.logger import log
|
||||
from cibuildwheel.options import CommandLineArguments, Options, compute_options
|
||||
from cibuildwheel.platforms import ALL_PLATFORM_MODULES, get_build_identifiers, native_platform
|
||||
from cibuildwheel.selector import BuildSelector, EnableGroup, selector_matches
|
||||
from cibuildwheel.typing import PLATFORMS, PlatformName
|
||||
from cibuildwheel.util.file import CIBW_CACHE_PATH, ensure_cache_sentinel
|
||||
from cibuildwheel.util.helpers import strtobool
|
||||
from cibuildwheel.util.resources import read_all_configs
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Generator, Iterable, Sequence
|
||||
from typing import Literal
|
||||
from cibuildwheel.environment import EnvironmentParseError, parse_environment
|
||||
from cibuildwheel.projectfiles import get_requires_python_str
|
||||
from cibuildwheel.typing import PLATFORMS, PlatformName, assert_never
|
||||
from cibuildwheel.util import (
|
||||
BuildOptions,
|
||||
BuildSelector,
|
||||
DependencyConstraints,
|
||||
TestSelector,
|
||||
Unbuffered,
|
||||
detect_ci_provider,
|
||||
resources_dir,
|
||||
)
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class GlobalOptions:
|
||||
print_traceback_on_error: bool = True # decides what happens when errors are hit.
|
||||
@overload
|
||||
def get_option_from_environment(
|
||||
option_name: str, *, platform: Optional[str] = None, default: str
|
||||
) -> str:
|
||||
... # noqa: E704
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class FileReport:
|
||||
name: str
|
||||
size: str
|
||||
@overload
|
||||
def get_option_from_environment(
|
||||
option_name: str, *, platform: Optional[str] = None, default: None = None
|
||||
) -> Optional[str]:
|
||||
... # noqa: E704 E302
|
||||
|
||||
|
||||
def get_option_from_environment(
|
||||
option_name: str, *, platform: Optional[str] = None, default: Optional[str] = None
|
||||
) -> Optional[str]: # noqa: E302
|
||||
"""
|
||||
Returns an option from the environment, optionally scoped by the platform.
|
||||
|
||||
Example:
|
||||
get_option_from_environment('CIBW_COLOR', platform='macos')
|
||||
|
||||
This will return the value of CIBW_COLOR_MACOS if it exists, otherwise the value of
|
||||
CIBW_COLOR.
|
||||
"""
|
||||
if platform:
|
||||
option = os.environ.get(f"{option_name}_{platform.upper()}")
|
||||
if option is not None:
|
||||
return option
|
||||
|
||||
return os.environ.get(option_name, default)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
global_options = GlobalOptions()
|
||||
try:
|
||||
main_inner(global_options)
|
||||
except errors.FatalError as e:
|
||||
message = e.args[0]
|
||||
if log.step_active:
|
||||
log.step_end_with_error(message)
|
||||
else:
|
||||
log.error(message)
|
||||
platform: PlatformName
|
||||
|
||||
if global_options.print_traceback_on_error:
|
||||
traceback.print_exc(file=sys.stderr)
|
||||
|
||||
sys.exit(e.return_code)
|
||||
|
||||
|
||||
def main_inner(global_options: GlobalOptions) -> None:
|
||||
"""
|
||||
`main_inner` is the same as `main`, but it raises FatalError exceptions
|
||||
rather than exiting directly.
|
||||
"""
|
||||
# Default in 3.15+, only needed on 3.14
|
||||
if sys.version_info >= (3, 14):
|
||||
arg_parser = functools.partial(argparse.ArgumentParser, suggest_on_error=True)
|
||||
else:
|
||||
arg_parser = argparse.ArgumentParser
|
||||
|
||||
parser = arg_parser(
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Build wheels for all the platforms.",
|
||||
epilog="""
|
||||
Most options are supplied via environment variables or in
|
||||
--config-file (pyproject.toml usually). See
|
||||
https://github.com/pypa/cibuildwheel#options for info.
|
||||
Most options are supplied via environment variables.
|
||||
See https://github.com/pypa/cibuildwheel#options for info.
|
||||
""",
|
||||
allow_abbrev=False,
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--platform",
|
||||
choices=["auto", "linux", "macos", "windows", "pyodide", "android", "ios"],
|
||||
default=None,
|
||||
choices=["auto", "linux", "macos", "windows"],
|
||||
default=os.environ.get("CIBW_PLATFORM", "auto"),
|
||||
help="""
|
||||
Platform to build for. Use this option to override the auto-detected
|
||||
platform. Specifying "macos" or "windows" only works on that
|
||||
operating system. "linux" works on any desktop OS, as long as
|
||||
Docker/Podman is installed. "pyodide" and "android" only work on
|
||||
Linux and macOS. "ios" only works on macOS. Default: auto.
|
||||
Platform to build for. For "linux" you need docker running, on Mac
|
||||
or Linux. For "macos", you need a Mac machine, and note that this
|
||||
script is going to automatically install MacPython on your system,
|
||||
so don't run on your development machine. For "windows", you need to
|
||||
run in Windows, and it will build and test for all versions of
|
||||
Python. Default: auto.
|
||||
""",
|
||||
)
|
||||
|
||||
@@ -135,60 +102,21 @@ def main_inner(global_options: GlobalOptions) -> None:
|
||||
""",
|
||||
)
|
||||
|
||||
enable_groups_str = ", ".join(g.value for g in EnableGroup)
|
||||
parser.add_argument(
|
||||
"--enable",
|
||||
action="append",
|
||||
default=[],
|
||||
metavar="GROUP",
|
||||
help=f"""
|
||||
Enable an additional category of builds. Use multiple times to select multiple groups. Choices: {enable_groups_str}.
|
||||
""",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--only",
|
||||
default=None,
|
||||
choices=[v["identifier"] for vv in read_all_configs().values() for v in vv],
|
||||
metavar="IDENTIFIER",
|
||||
help="""
|
||||
Force a single wheel build when given an identifier. Overrides
|
||||
CIBW_BUILD/CIBW_SKIP. --platform and --arch cannot be specified
|
||||
if this is given.
|
||||
""",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
type=Path,
|
||||
default=Path(os.environ.get("CIBW_OUTPUT_DIR", "wheelhouse")),
|
||||
help="Destination folder for the wheels. Default: wheelhouse.",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--config-file",
|
||||
default="",
|
||||
help="""
|
||||
TOML config file. Default: "", meaning {package}/pyproject.toml, if
|
||||
it exists. To refer to a project inside your project, use {package};
|
||||
this matters if you build from an SDist.
|
||||
""",
|
||||
default=os.environ.get("CIBW_OUTPUT_DIR", "wheelhouse"),
|
||||
help="Destination folder for the wheels.",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"package_dir",
|
||||
metavar="PACKAGE",
|
||||
default=Path(),
|
||||
type=Path,
|
||||
default=".",
|
||||
nargs="?",
|
||||
help="""
|
||||
Path to the package that you want wheels for. Default: the working
|
||||
directory. Can be a directory inside the working directory, or an
|
||||
sdist. When set to a directory, the working directory is still
|
||||
considered the 'project' and is copied into the build container
|
||||
on Linux. When set to a tar.gz sdist file, --config-file
|
||||
and --output-dir are relative to the current directory, and other
|
||||
paths are relative to the expanded SDist directory.
|
||||
Path to the package that you want wheels for. Must be a subdirectory of
|
||||
the working directory. When set, the working directory is still
|
||||
considered the 'project' and is copied into the Docker container on
|
||||
Linux. Default: the working directory.
|
||||
""",
|
||||
)
|
||||
|
||||
@@ -198,219 +126,264 @@ def main_inner(global_options: GlobalOptions) -> None:
|
||||
help="Print the build identifiers matched by the current invocation and exit.",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--clean-cache",
|
||||
action="store_true",
|
||||
help="Clear the cibuildwheel cache and exit.",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--allow-empty",
|
||||
action="store_true",
|
||||
help="Do not report an error code if the build does not match any wheels.",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--debug-traceback",
|
||||
action="store_true",
|
||||
default=strtobool(os.environ.get("CIBW_DEBUG_TRACEBACK", "0")),
|
||||
help="Print a full traceback for all errors",
|
||||
args = parser.parse_args()
|
||||
|
||||
detect_obsolete_options()
|
||||
|
||||
if args.platform != "auto":
|
||||
platform = args.platform
|
||||
else:
|
||||
ci_provider = detect_ci_provider()
|
||||
if ci_provider is None:
|
||||
print(
|
||||
textwrap.dedent(
|
||||
"""
|
||||
cibuildwheel: Unable to detect platform. cibuildwheel should run on your CI server;
|
||||
Travis CI, AppVeyor, Azure Pipelines, GitHub Actions, CircleCI, and Gitlab are
|
||||
supported. You can run on your development machine or other CI providers using the
|
||||
--platform argument. Check --help output for more information.
|
||||
"""
|
||||
),
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(2)
|
||||
if sys.platform.startswith("linux"):
|
||||
platform = "linux"
|
||||
elif sys.platform == "darwin":
|
||||
platform = "macos"
|
||||
elif sys.platform == "win32":
|
||||
platform = "windows"
|
||||
else:
|
||||
print(
|
||||
'cibuildwheel: Unable to detect platform from "sys.platform" in a CI environment. You can run '
|
||||
"cibuildwheel using the --platform argument. Check --help output for more information.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(2)
|
||||
|
||||
if platform not in PLATFORMS:
|
||||
print(f"cibuildwheel: Unsupported platform: {platform}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
package_dir = Path(args.package_dir)
|
||||
output_dir = Path(args.output_dir)
|
||||
|
||||
if platform == "linux":
|
||||
repair_command_default = "auditwheel repair -w {dest_dir} {wheel}"
|
||||
elif platform == "macos":
|
||||
repair_command_default = "delocate-listdeps {wheel} && delocate-wheel --require-archs {delocate_archs} -w {dest_dir} {wheel}"
|
||||
elif platform == "windows":
|
||||
repair_command_default = ""
|
||||
else:
|
||||
assert_never(platform)
|
||||
|
||||
build_config = os.environ.get("CIBW_BUILD") or "*"
|
||||
skip_config = os.environ.get("CIBW_SKIP", "")
|
||||
test_skip = os.environ.get("CIBW_TEST_SKIP", "")
|
||||
environment_config = get_option_from_environment(
|
||||
"CIBW_ENVIRONMENT", platform=platform, default=""
|
||||
)
|
||||
before_all = get_option_from_environment("CIBW_BEFORE_ALL", platform=platform, default="")
|
||||
before_build = get_option_from_environment("CIBW_BEFORE_BUILD", platform=platform)
|
||||
repair_command = get_option_from_environment(
|
||||
"CIBW_REPAIR_WHEEL_COMMAND", platform=platform, default=repair_command_default
|
||||
)
|
||||
dependency_versions = get_option_from_environment(
|
||||
"CIBW_DEPENDENCY_VERSIONS", platform=platform, default="pinned"
|
||||
)
|
||||
test_command = get_option_from_environment("CIBW_TEST_COMMAND", platform=platform)
|
||||
before_test = get_option_from_environment("CIBW_BEFORE_TEST", platform=platform)
|
||||
test_requires = get_option_from_environment(
|
||||
"CIBW_TEST_REQUIRES", platform=platform, default=""
|
||||
).split()
|
||||
test_extras = get_option_from_environment("CIBW_TEST_EXTRAS", platform=platform, default="")
|
||||
build_verbosity_str = get_option_from_environment(
|
||||
"CIBW_BUILD_VERBOSITY", platform=platform, default=""
|
||||
)
|
||||
|
||||
args = CommandLineArguments(**vars(parser.parse_args()))
|
||||
|
||||
global_options.print_traceback_on_error = args.debug_traceback
|
||||
|
||||
if args.clean_cache:
|
||||
if not CIBW_CACHE_PATH.exists():
|
||||
print(f"Cache directory does not exist: {CIBW_CACHE_PATH}")
|
||||
sys.exit(0)
|
||||
|
||||
sentinel_file = CIBW_CACHE_PATH / "CACHEDIR.TAG"
|
||||
if not sentinel_file.exists():
|
||||
print(
|
||||
f"Error: {CIBW_CACHE_PATH} does not appear to be a cibuildwheel cache directory.",
|
||||
"Only directories with a CACHEDIR.TAG sentinel file can be cleaned.",
|
||||
sep="\n",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
# Verify signature to ensure it's a proper cache dir
|
||||
# See https://bford.info/cachedir/ for more
|
||||
try:
|
||||
sentinel_content = sentinel_file.read_text(encoding="utf-8")
|
||||
except OSError as e:
|
||||
print(f"Error reading cache directory tag: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if not sentinel_content.startswith("Signature: 8a477f597d28d172789f06886806bc55"):
|
||||
print(
|
||||
f"Error: {sentinel_file} does not contain a valid cache directory signature.",
|
||||
"For safety, only properly signed cache directories can be cleaned.",
|
||||
sep="\n",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
print(f"Clearing cache directory: {CIBW_CACHE_PATH}")
|
||||
try:
|
||||
shutil.rmtree(CIBW_CACHE_PATH)
|
||||
print("Cache cleared successfully.")
|
||||
except OSError as e:
|
||||
print(f"Error clearing cache: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
sys.exit(0)
|
||||
|
||||
args.package_dir = args.package_dir.resolve()
|
||||
|
||||
# This are always relative to the base directory, even in SDist builds
|
||||
args.output_dir = args.output_dir.resolve()
|
||||
|
||||
# Standard builds if a directory or non-existent path is given
|
||||
if not args.package_dir.is_file() and not args.package_dir.name.endswith("tar.gz"):
|
||||
build_in_directory(args)
|
||||
return
|
||||
|
||||
# Tarfile builds require extraction and changing the directory
|
||||
temp_dir = Path(mkdtemp(prefix="cibw-sdist-")).resolve(strict=True)
|
||||
try:
|
||||
with TarFile.open(args.package_dir) as tar:
|
||||
safe_extractall(tar, temp_dir)
|
||||
|
||||
# The extract directory is now the project dir
|
||||
try:
|
||||
(project_dir,) = temp_dir.iterdir()
|
||||
except ValueError:
|
||||
msg = "invalid sdist: didn't contain a single dir"
|
||||
raise SystemExit(msg) from None
|
||||
|
||||
# This is now the new package dir
|
||||
args.package_dir = project_dir.resolve()
|
||||
|
||||
with contextlib.chdir(project_dir):
|
||||
build_in_directory(args)
|
||||
finally:
|
||||
# avoid https://github.com/python/cpython/issues/86962 by performing
|
||||
# cleanup manually
|
||||
shutil.rmtree(temp_dir, ignore_errors=sys.platform.startswith("win"))
|
||||
if temp_dir.exists():
|
||||
log.warning(f"Can't delete temporary folder '{temp_dir}'")
|
||||
|
||||
|
||||
def _compute_platform_only(only: str) -> PlatformName:
|
||||
if "linux_" in only:
|
||||
return "linux"
|
||||
if "macosx_" in only:
|
||||
return "macos"
|
||||
if "win_" in only or "win32" in only:
|
||||
return "windows"
|
||||
if "pyodide_" in only:
|
||||
return "pyodide"
|
||||
if "android_" in only:
|
||||
return "android"
|
||||
if "ios_" in only:
|
||||
return "ios"
|
||||
msg = f"Invalid --only='{only}', must be a build selector with a known platform"
|
||||
raise errors.ConfigurationError(msg)
|
||||
|
||||
|
||||
def _compute_platform(args: CommandLineArguments) -> PlatformName:
|
||||
platform_option_value = args.platform or os.environ.get("CIBW_PLATFORM", "") or "auto"
|
||||
|
||||
if args.only and args.platform is not None:
|
||||
msg = "--platform cannot be specified with --only, it is computed from --only"
|
||||
raise errors.ConfigurationError(msg)
|
||||
if args.only and args.archs is not None:
|
||||
msg = "--arch cannot be specified with --only, it is computed from --only"
|
||||
raise errors.ConfigurationError(msg)
|
||||
|
||||
if platform_option_value not in PLATFORMS | {"auto"}:
|
||||
msg = f"Unsupported platform: {platform_option_value}"
|
||||
raise errors.ConfigurationError(msg)
|
||||
|
||||
if args.only:
|
||||
return _compute_platform_only(args.only)
|
||||
elif platform_option_value != "auto":
|
||||
return typing.cast("PlatformName", platform_option_value)
|
||||
|
||||
return native_platform()
|
||||
|
||||
|
||||
def build_in_directory(args: CommandLineArguments) -> None:
|
||||
platform: PlatformName = _compute_platform(args)
|
||||
if platform == "pyodide" and sys.platform == "win32":
|
||||
msg = "Building for pyodide is not supported on Windows"
|
||||
raise errors.ConfigurationError(msg)
|
||||
|
||||
options = compute_options(platform=platform, command_line_arguments=args, env=os.environ)
|
||||
|
||||
package_dir = options.globals.package_dir
|
||||
package_files = {"setup.py", "setup.cfg", "pyproject.toml"}
|
||||
|
||||
if not any(package_dir.joinpath(name).exists() for name in package_files):
|
||||
names = ", ".join(sorted(package_files, reverse=True))
|
||||
msg = f"Could not find any of {{{names}}} at root of package"
|
||||
raise errors.ConfigurationError(msg)
|
||||
print(
|
||||
f"cibuildwheel: Could not find any of {{{names}}} at root of package", file=sys.stderr
|
||||
)
|
||||
sys.exit(2)
|
||||
|
||||
platform_module = ALL_PLATFORM_MODULES[platform]
|
||||
identifiers = get_build_identifiers(
|
||||
platform_module=platform_module,
|
||||
build_selector=options.globals.build_selector,
|
||||
architectures=options.globals.architectures,
|
||||
# Passing this in as an environment variable will override pyproject.toml, setup.cfg, or setup.py
|
||||
requires_python_str: Optional[str] = os.environ.get(
|
||||
"CIBW_PROJECT_REQUIRES_PYTHON"
|
||||
) or get_requires_python_str(package_dir)
|
||||
requires_python = None if requires_python_str is None else SpecifierSet(requires_python_str)
|
||||
|
||||
build_selector = BuildSelector(
|
||||
build_config=build_config, skip_config=skip_config, requires_python=requires_python
|
||||
)
|
||||
test_selector = TestSelector(skip_config=test_skip)
|
||||
|
||||
try:
|
||||
environment = parse_environment(environment_config)
|
||||
except (EnvironmentParseError, ValueError):
|
||||
print(f'cibuildwheel: Malformed environment option "{environment_config}"', file=sys.stderr)
|
||||
traceback.print_exc(None, sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
if dependency_versions == "pinned":
|
||||
dependency_constraints: Optional[
|
||||
DependencyConstraints
|
||||
] = DependencyConstraints.with_defaults()
|
||||
elif dependency_versions == "latest":
|
||||
dependency_constraints = None
|
||||
else:
|
||||
dependency_versions_path = Path(dependency_versions)
|
||||
dependency_constraints = DependencyConstraints(dependency_versions_path)
|
||||
|
||||
if test_extras:
|
||||
test_extras = f"[{test_extras}]"
|
||||
|
||||
try:
|
||||
build_verbosity = min(3, max(-3, int(build_verbosity_str)))
|
||||
except ValueError:
|
||||
build_verbosity = 0
|
||||
|
||||
# Add CIBUILDWHEEL environment variable
|
||||
# This needs to be passed on to the docker container in linux.py
|
||||
os.environ["CIBUILDWHEEL"] = "1"
|
||||
|
||||
if args.archs is not None:
|
||||
archs_config_str = args.archs
|
||||
else:
|
||||
archs_config_str = get_option_from_environment(
|
||||
"CIBW_ARCHS", platform=platform, default="auto"
|
||||
)
|
||||
|
||||
archs = Architecture.parse_config(archs_config_str, platform=platform)
|
||||
|
||||
identifiers = get_build_identifiers(platform, build_selector, archs)
|
||||
|
||||
if args.print_build_identifiers:
|
||||
for identifier in identifiers:
|
||||
print(identifier)
|
||||
sys.exit(0)
|
||||
|
||||
# Add CIBUILDWHEEL environment variable
|
||||
os.environ["CIBUILDWHEEL"] = "1"
|
||||
manylinux_images: Optional[Dict[str, str]] = None
|
||||
if platform == "linux":
|
||||
pinned_docker_images_file = resources_dir / "pinned_docker_images.cfg"
|
||||
all_pinned_docker_images = ConfigParser()
|
||||
all_pinned_docker_images.read(pinned_docker_images_file)
|
||||
# all_pinned_docker_images looks like a dict of dicts, e.g.
|
||||
# { 'x86_64': {'manylinux1': '...', 'manylinux2010': '...', 'manylinux2014': '...'},
|
||||
# 'i686': {'manylinux1': '...', 'manylinux2010': '...', 'manylinux2014': '...'},
|
||||
# 'pypy_x86_64': {'manylinux2010': '...' }
|
||||
# ... }
|
||||
|
||||
# Python block-buffers stdout when it isn't a tty, which de-interleaves
|
||||
# `print` output and direct fd-1 writes from subprocesses sharing this
|
||||
# stdout. line_buffering allows each newline to flush all the way to fd-1.
|
||||
if isinstance(sys.stdout, io.TextIOWrapper):
|
||||
sys.stdout.reconfigure(line_buffering=True)
|
||||
manylinux_images = {}
|
||||
|
||||
# create the cache dir before it gets printed & builds performed
|
||||
CIBW_CACHE_PATH.mkdir(parents=True, exist_ok=True)
|
||||
ensure_cache_sentinel(CIBW_CACHE_PATH)
|
||||
for build_platform in ["x86_64", "i686", "pypy_x86_64", "aarch64", "ppc64le", "s390x"]:
|
||||
pinned_images = all_pinned_docker_images[build_platform]
|
||||
|
||||
print_preamble(platform=platform, options=options, identifiers=identifiers)
|
||||
config_name = f"CIBW_MANYLINUX_{build_platform.upper()}_IMAGE"
|
||||
config_value = os.environ.get(config_name)
|
||||
|
||||
if config_value is None:
|
||||
# default to manylinux2010 if it's available, otherwise manylinux2014
|
||||
image = pinned_images.get("manylinux2010") or pinned_images.get("manylinux2014")
|
||||
elif config_value in pinned_images:
|
||||
image = pinned_images[config_value]
|
||||
else:
|
||||
image = config_value
|
||||
|
||||
manylinux_images[build_platform] = image
|
||||
|
||||
build_options = BuildOptions(
|
||||
architectures=archs,
|
||||
package_dir=package_dir,
|
||||
output_dir=output_dir,
|
||||
test_command=test_command,
|
||||
test_requires=test_requires,
|
||||
test_extras=test_extras,
|
||||
before_test=before_test,
|
||||
before_build=before_build,
|
||||
before_all=before_all,
|
||||
build_verbosity=build_verbosity,
|
||||
build_selector=build_selector,
|
||||
test_selector=test_selector,
|
||||
repair_command=repair_command,
|
||||
environment=environment,
|
||||
dependency_constraints=dependency_constraints,
|
||||
manylinux_images=manylinux_images,
|
||||
)
|
||||
|
||||
# Python is buffering by default when running on the CI platforms, giving problems interleaving subprocess call output with unflushed calls to 'print'
|
||||
sys.stdout = Unbuffered(sys.stdout) # type: ignore
|
||||
|
||||
print_preamble(platform, build_options)
|
||||
|
||||
try:
|
||||
options.check_for_invalid_configuration(identifiers)
|
||||
allowed_architectures_check(platform, options.globals.architectures)
|
||||
allowed_architectures_check(platform, build_options.architectures)
|
||||
except ValueError as err:
|
||||
raise errors.DeprecationError(*err.args) from err
|
||||
print("cibuildwheel:", *err.args, file=sys.stderr)
|
||||
sys.exit(4)
|
||||
|
||||
if not identifiers:
|
||||
message = f"No build identifiers selected: {options.globals.build_selector}"
|
||||
if options.globals.allow_empty:
|
||||
print(f"cibuildwheel: {message}", file=sys.stderr)
|
||||
print(f"cibuildwheel: No build identifiers selected: {build_selector}", file=sys.stderr)
|
||||
if not args.allow_empty:
|
||||
sys.exit(3)
|
||||
|
||||
if not output_dir.exists():
|
||||
output_dir.mkdir(parents=True)
|
||||
|
||||
with cibuildwheel.util.print_new_wheels(
|
||||
"\n{n} wheels produced in {m:.0f} minutes:", output_dir
|
||||
):
|
||||
if platform == "linux":
|
||||
cibuildwheel.linux.build(build_options)
|
||||
elif platform == "windows":
|
||||
cibuildwheel.windows.build(build_options)
|
||||
elif platform == "macos":
|
||||
cibuildwheel.macos.build(build_options)
|
||||
else:
|
||||
raise errors.NothingToDoError(message)
|
||||
|
||||
output_dir = options.globals.output_dir
|
||||
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
tmp_path = Path(mkdtemp(prefix="cibw-run-")).resolve(strict=True)
|
||||
try:
|
||||
with log.print_summary(options=options):
|
||||
platform_module.build(options, tmp_path)
|
||||
finally:
|
||||
# avoid https://github.com/python/cpython/issues/86962 by performing
|
||||
# cleanup manually
|
||||
shutil.rmtree(tmp_path, ignore_errors=sys.platform.startswith("win"))
|
||||
if tmp_path.exists():
|
||||
log.warning(f"Can't delete temporary folder '{tmp_path}'")
|
||||
assert_never(platform)
|
||||
|
||||
|
||||
def print_preamble(platform: str, options: Options, identifiers: Sequence[str]) -> None:
|
||||
def detect_obsolete_options() -> None:
|
||||
# Check the old 'MANYLINUX1_*_IMAGE' options
|
||||
for (deprecated, alternative) in [
|
||||
("CIBW_MANYLINUX1_X86_64_IMAGE", "CIBW_MANYLINUX_X86_64_IMAGE"),
|
||||
("CIBW_MANYLINUX1_I686_IMAGE", "CIBW_MANYLINUX_I686_IMAGE"),
|
||||
]:
|
||||
if deprecated in os.environ:
|
||||
print(
|
||||
f"'{deprecated}' has been deprecated, and will be removed in a future release. Use the option '{alternative}' instead."
|
||||
)
|
||||
if alternative not in os.environ:
|
||||
print(f"Using value of option '{deprecated}' as replacement for '{alternative}'")
|
||||
os.environ[alternative] = os.environ[deprecated]
|
||||
else:
|
||||
print(f"Option '{alternative}' is not empty. Please unset '{deprecated}'")
|
||||
sys.exit(2)
|
||||
|
||||
# Check for deprecated identifiers in 'CIBW_BUILD' and 'CIBW_SKIP' options
|
||||
for option in ["CIBW_BUILD", "CIBW_SKIP"]:
|
||||
for deprecated, alternative in [
|
||||
("manylinux1", "manylinux"),
|
||||
("macosx_10_6_intel", "macosx_x86_64"),
|
||||
("macosx_10_9_x86_64", "macosx_x86_64"),
|
||||
]:
|
||||
if option in os.environ and deprecated in os.environ[option]:
|
||||
print(
|
||||
f"Build identifiers with '{deprecated}' have been deprecated. Replacing all occurences of '{deprecated}' with '{alternative}' in the option '{option}'"
|
||||
)
|
||||
os.environ[option] = os.environ[option].replace(deprecated, alternative)
|
||||
|
||||
|
||||
def print_preamble(platform: str, build_options: BuildOptions) -> None:
|
||||
print(
|
||||
textwrap.dedent(
|
||||
"""
|
||||
@@ -425,146 +398,59 @@ def print_preamble(platform: str, options: Options, identifiers: Sequence[str])
|
||||
print(f"cibuildwheel version {cibuildwheel.__version__}\n")
|
||||
|
||||
print("Build options:")
|
||||
print(f" platform: {platform}")
|
||||
options_summary = textwrap.indent(options.summary(identifiers), " ")
|
||||
if detect_ci_provider() == CIProvider.github_actions:
|
||||
options_summary = fix_ansi_codes_for_github_actions(options_summary)
|
||||
print(options_summary)
|
||||
print(f" platform: {platform!r}")
|
||||
for option, value in sorted(build_options._asdict().items()):
|
||||
print(f" {option}: {value!r}")
|
||||
|
||||
print()
|
||||
print(f"Cache folder: {CIBW_CACHE_PATH}")
|
||||
print()
|
||||
warnings = detect_warnings(platform, build_options)
|
||||
if warnings:
|
||||
print("\nWarnings:")
|
||||
for warning in warnings:
|
||||
print(" " + warning)
|
||||
|
||||
warnings = detect_warnings(options=options)
|
||||
for warning in warnings:
|
||||
log.warning(warning)
|
||||
|
||||
error_list = list(detect_errors(options=options, identifiers=identifiers))
|
||||
if error_list:
|
||||
for error in error_list:
|
||||
log.error(error)
|
||||
msg = "\n".join(error_list)
|
||||
raise errors.ConfigurationError(msg)
|
||||
|
||||
n = len(identifiers)
|
||||
print(f"{n} build{'s' if n != 1 else ''} selected:")
|
||||
print(", ".join(identifiers))
|
||||
print()
|
||||
print("Here we go!\n")
|
||||
print("\nHere we go!\n")
|
||||
|
||||
|
||||
def detect_errors(*, options: Options, identifiers: Iterable[str]) -> Generator[str, None, None]:
|
||||
# Check for deprecated CIBW_FREE_THREADED_SUPPORT environment variable
|
||||
if "CIBW_FREE_THREADED_SUPPORT" in os.environ:
|
||||
yield "CIBW_FREE_THREADED_SUPPORT environment variable is no longer supported."
|
||||
def get_build_identifiers(
|
||||
platform: PlatformName, build_selector: BuildSelector, architectures: Set[Architecture]
|
||||
) -> List[str]:
|
||||
python_configurations: Union[
|
||||
List[cibuildwheel.linux.PythonConfiguration],
|
||||
List[cibuildwheel.windows.PythonConfiguration],
|
||||
List[cibuildwheel.macos.PythonConfiguration],
|
||||
]
|
||||
|
||||
# Deprecated {python} and {pip}
|
||||
for option_name in ["test_command", "before_build"]:
|
||||
option_values = [getattr(options.build_options(i), option_name) for i in identifiers]
|
||||
|
||||
if any(o and ("{python}" in o or "{pip}" in o) for o in option_values):
|
||||
# Reminder: in an f-string, double braces means literal single brace
|
||||
yield (
|
||||
f"{option_name}: '{{python}}' and '{{pip}}' are no longer supported "
|
||||
"and have been removed in cibuildwheel 3. Simply use 'python' or 'pip' instead."
|
||||
)
|
||||
|
||||
|
||||
def detect_warnings(*, options: Options) -> Generator[str, None, None]:
|
||||
python_version_deprecation = ((3, 11), 3)
|
||||
if sys.version_info[:2] < python_version_deprecation[0]:
|
||||
python_version = ".".join(map(str, python_version_deprecation[0]))
|
||||
yield (
|
||||
f"cibuildwheel {python_version_deprecation[1]} will require Python {python_version}+, "
|
||||
"please upgrade the Python version used to run cibuildwheel. "
|
||||
"This does not affect the versions you can target when building wheels. See: https://cibuildwheel.pypa.io/en/stable/#what-does-it-do"
|
||||
if platform == "linux":
|
||||
python_configurations = cibuildwheel.linux.get_python_configurations(
|
||||
build_selector, architectures
|
||||
)
|
||||
elif platform == "windows":
|
||||
python_configurations = cibuildwheel.windows.get_python_configurations(
|
||||
build_selector, architectures
|
||||
)
|
||||
elif platform == "macos":
|
||||
python_configurations = cibuildwheel.macos.get_python_configurations(
|
||||
build_selector, architectures
|
||||
)
|
||||
else:
|
||||
assert_never(platform)
|
||||
|
||||
build_selector = options.globals.build_selector
|
||||
test_selector = options.globals.test_selector
|
||||
|
||||
all_valid_identifiers = [
|
||||
config.identifier
|
||||
for module in ALL_PLATFORM_MODULES.values()
|
||||
for config in module.all_python_configurations()
|
||||
]
|
||||
|
||||
enabled_selector = BuildSelector(
|
||||
build_config="*", skip_config="", enable=options.globals.build_selector.enable
|
||||
)
|
||||
all_enabled_identifiers = [
|
||||
identifier for identifier in all_valid_identifiers if enabled_selector(identifier)
|
||||
]
|
||||
|
||||
yield from check_for_invalid_selectors(
|
||||
selector_name="build",
|
||||
selector_value=build_selector.build_config,
|
||||
all_valid_identifiers=all_valid_identifiers,
|
||||
all_enabled_identifiers=all_enabled_identifiers,
|
||||
)
|
||||
yield from check_for_invalid_selectors(
|
||||
selector_name="skip",
|
||||
selector_value=build_selector.skip_config,
|
||||
all_valid_identifiers=all_valid_identifiers,
|
||||
all_enabled_identifiers=all_enabled_identifiers,
|
||||
)
|
||||
yield from check_for_invalid_selectors(
|
||||
selector_name="test_skip",
|
||||
selector_value=test_selector.skip_config,
|
||||
all_valid_identifiers=all_valid_identifiers,
|
||||
all_enabled_identifiers=all_enabled_identifiers,
|
||||
)
|
||||
return [config.identifier for config in python_configurations]
|
||||
|
||||
|
||||
def check_for_invalid_selectors(
|
||||
*,
|
||||
selector_name: Literal["build", "skip", "test_skip"],
|
||||
selector_value: str,
|
||||
all_valid_identifiers: Sequence[str],
|
||||
all_enabled_identifiers: Sequence[str],
|
||||
) -> list[str]:
|
||||
def detect_warnings(platform: str, build_options: BuildOptions) -> List[str]:
|
||||
warnings = []
|
||||
|
||||
for selector in selector_value.split():
|
||||
selector_ = selector
|
||||
if selector_name == "test_skip":
|
||||
# macosx_universal2 uses an additional identifier for tests which ends with ":{arch}"
|
||||
values = selector.split(":")
|
||||
universal2_identifiers = filter(
|
||||
lambda x: x.endswith("-macosx_universal2"), all_valid_identifiers
|
||||
# warn about deprecated {python} and {pip}
|
||||
for option_name in ["test_command", "before_build"]:
|
||||
option_value = getattr(build_options, option_name)
|
||||
|
||||
if option_value and ("{python}" in option_value or "{pip}" in option_value):
|
||||
# Reminder: in an f-string, double braces means literal single brace
|
||||
msg = (
|
||||
f"{option_name}: '{{python}}' and '{{pip}}' are no longer needed, "
|
||||
"and will be removed in a future release. Simply use 'python' or 'pip' instead."
|
||||
)
|
||||
if len(values) == 2 and any(
|
||||
selector_matches(selector_, f"{i}:{arch}")
|
||||
for i in universal2_identifiers
|
||||
for arch in ["arm64", "x86_64"]
|
||||
):
|
||||
# just ignore the arch part in the rest of the check
|
||||
selector_ = values[0]
|
||||
if not any(selector_matches(selector_, i) for i in all_enabled_identifiers):
|
||||
msg = f"Invalid {selector_name} selector: {selector!r}. "
|
||||
error_type: type = errors.ConfigurationError
|
||||
|
||||
if any(selector_matches(selector_, i) for i in all_valid_identifiers):
|
||||
msg += "This selector matches a group that wasn't enabled. Enable it using the `enable` option or remove this selector. "
|
||||
|
||||
if "p2" in selector_ or "p35" in selector_:
|
||||
msg += f"cibuildwheel 4.x no longer supports Python < 3.9. Please use the 1.x series or update `{selector_name}`. "
|
||||
error_type = errors.DeprecationError
|
||||
if "p36" in selector_ or "p37" in selector_:
|
||||
msg += f"cibuildwheel 4.x no longer supports Python < 3.9. Please use the 2.x series or update `{selector_name}`. "
|
||||
error_type = errors.DeprecationError
|
||||
if "p38" in selector_:
|
||||
msg += f"cibuildwheel 4.x no longer supports Python < 3.9. Please use the 3.x series or update `{selector_name}`. "
|
||||
error_type = errors.DeprecationError
|
||||
if "cp313t" in selector_:
|
||||
msg += f"cibuildwheel 4.x no longer supports Python 3.13 free-threading. Please use the 3.x series or update `{selector_name}`. "
|
||||
error_type = errors.DeprecationError
|
||||
|
||||
if selector_name == "build":
|
||||
raise error_type(msg)
|
||||
|
||||
msg += "This selector will have no effect. "
|
||||
|
||||
warnings.append(msg)
|
||||
|
||||
return warnings
|
||||
|
||||
@@ -1,71 +0,0 @@
|
||||
# Based on https://github.com/pypa/build/blob/f4ebd495cc0c2c74155bd4fe48b76399fb7927ac/src/build/_compat/tarfile.py
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import tarfile
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
TarFile = tarfile.TarFile
|
||||
|
||||
# Per https://peps.python.org/pep-0706/, the "data" filter will become
|
||||
# the default in Python 3.14. The first series of releases with the filter
|
||||
# had a broken filter that could not process symlinks correctly.
|
||||
elif (3, 11, 5) <= sys.version_info < (3, 14):
|
||||
|
||||
class TarFile(tarfile.TarFile): # pragma: no cover
|
||||
extraction_filter = staticmethod(tarfile.data_filter)
|
||||
|
||||
else:
|
||||
TarFile = tarfile.TarFile # pragma: no cover
|
||||
|
||||
|
||||
# Same availability matrix as the TarFile subclass above. On runtimes that
|
||||
# ship the stdlib ``data`` filter we delegate to it; the fallback branch is
|
||||
# only reached on 3.10.0-3.10.12 / 3.11.0-3.11.4 and validates each member
|
||||
# manually before extraction.
|
||||
if sys.version_info >= (3, 11, 5):
|
||||
|
||||
def safe_extractall(tar: tarfile.TarFile, path: Path) -> None: # pragma: no cover
|
||||
"""Extract every member of ``tar`` into ``path`` via the PEP 706 ``data`` filter."""
|
||||
tar.extractall(path, filter="data")
|
||||
|
||||
else:
|
||||
|
||||
def safe_extractall(tar: tarfile.TarFile, path: Path) -> None: # pragma: no cover
|
||||
"""Validate every member of ``tar``, then extract into ``path``.
|
||||
|
||||
Reached on 3.10.0-3.10.12 / 3.11.0-3.11.4 where the stdlib ``data`` filter is missing. Device or special files,
|
||||
paths that escape ``path``, and symlinks/hardlinks whose targets resolve outside ``path`` are rejected before
|
||||
any write hits the disk.
|
||||
|
||||
"""
|
||||
base = path.resolve()
|
||||
for member in tar.getmembers():
|
||||
_validate_safe_member(member, base)
|
||||
tar.extractall(path)
|
||||
|
||||
|
||||
def _validate_safe_member(member: tarfile.TarInfo, base: Path) -> None:
|
||||
if member.ischr() or member.isblk() or member.isfifo():
|
||||
msg = f"refusing to extract special device file {member.name!r}"
|
||||
raise tarfile.TarError(msg)
|
||||
target = (base / member.name).resolve(strict=False)
|
||||
if not target.is_relative_to(base):
|
||||
msg = f"refusing to extract {member.name!r}: path escapes destination"
|
||||
raise tarfile.TarError(msg)
|
||||
if member.issym() or member.islnk():
|
||||
link_base = target.parent if member.issym() else base
|
||||
link_target = (link_base / member.linkname).resolve(strict=False)
|
||||
if not link_target.is_relative_to(base):
|
||||
msg = f"refusing to extract {member.name!r}: link target escapes destination"
|
||||
raise tarfile.TarError(msg)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"TarFile",
|
||||
"safe_extractall",
|
||||
]
|
||||
+73
-212
@@ -1,249 +1,110 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {"platform", "re", "shutil", "subprocess"}
|
||||
|
||||
import functools
|
||||
import platform as platform_module
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import typing
|
||||
from enum import StrEnum, auto
|
||||
from enum import Enum
|
||||
from typing import Set
|
||||
|
||||
from cibuildwheel import errors
|
||||
from .typing import Literal, PlatformName, assert_never
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Set
|
||||
from typing import Final, Literal, Self
|
||||
|
||||
from cibuildwheel.typing import PlatformName
|
||||
|
||||
PRETTY_NAMES: Final[dict[PlatformName, str]] = {
|
||||
"linux": "Linux",
|
||||
"macos": "macOS",
|
||||
"windows": "Windows",
|
||||
"pyodide": "Pyodide",
|
||||
"android": "Android",
|
||||
"ios": "iOS",
|
||||
}
|
||||
|
||||
ARCH_SYNONYMS: Final[list[dict[PlatformName, str | None]]] = [
|
||||
{"linux": "x86_64", "macos": "x86_64", "windows": "AMD64", "android": "x86_64"},
|
||||
{"linux": "i686", "macos": None, "windows": "x86"},
|
||||
{"linux": "aarch64", "macos": "arm64", "windows": "ARM64", "android": "arm64_v8a"},
|
||||
]
|
||||
PRETTY_NAMES = {"linux": "Linux", "macos": "macOS", "windows": "Windows"}
|
||||
|
||||
|
||||
def arch_synonym(arch: str, from_platform: PlatformName, to_platform: PlatformName) -> str | None:
|
||||
for arch_synonym_ in ARCH_SYNONYMS:
|
||||
if arch == arch_synonym_.get(from_platform):
|
||||
return arch_synonym_.get(to_platform, arch)
|
||||
@functools.total_ordering
|
||||
class Architecture(Enum):
|
||||
value: str
|
||||
|
||||
return arch
|
||||
|
||||
|
||||
def _check_aarch32_el0() -> bool:
|
||||
"""Check if running armv7l natively on aarch64 is supported"""
|
||||
if not sys.platform.startswith("linux"):
|
||||
return False
|
||||
if platform_module.machine() != "aarch64":
|
||||
return False
|
||||
executable = shutil.which("linux32")
|
||||
if executable is None:
|
||||
return False
|
||||
check = subprocess.run([executable, "uname", "-m"], check=False, capture_output=True, text=True)
|
||||
return check.returncode == 0 and check.stdout.startswith("armv")
|
||||
|
||||
|
||||
@typing.final
|
||||
class Architecture(StrEnum):
|
||||
# mac/linux/android archs
|
||||
x86_64 = auto()
|
||||
# mac/linux archs
|
||||
x86_64 = "x86_64"
|
||||
|
||||
# linux archs
|
||||
i686 = auto()
|
||||
aarch64 = auto()
|
||||
ppc64le = auto()
|
||||
s390x = auto()
|
||||
armv7l = auto()
|
||||
riscv64 = auto()
|
||||
i686 = "i686"
|
||||
aarch64 = "aarch64"
|
||||
ppc64le = "ppc64le"
|
||||
s390x = "s390x"
|
||||
|
||||
# mac archs
|
||||
universal2 = auto()
|
||||
arm64 = auto()
|
||||
universal2 = "universal2"
|
||||
arm64 = "arm64"
|
||||
|
||||
# windows archs
|
||||
x86 = auto()
|
||||
x86 = "x86"
|
||||
AMD64 = "AMD64"
|
||||
ARM64 = "ARM64"
|
||||
|
||||
# WebAssembly
|
||||
wasm32 = auto()
|
||||
# Allow this to be sorted
|
||||
def __lt__(self, other: "Architecture") -> bool:
|
||||
return self.value < other.value
|
||||
|
||||
# android archs
|
||||
arm64_v8a = auto()
|
||||
|
||||
# iOS "multiarch" architectures that include both
|
||||
# the CPU architecture and the ABI.
|
||||
arm64_iphoneos = auto()
|
||||
arm64_iphonesimulator = auto()
|
||||
x86_64_iphonesimulator = auto()
|
||||
|
||||
@classmethod
|
||||
def parse_config(cls, config: str, platform: PlatformName) -> set[Self]:
|
||||
@staticmethod
|
||||
def parse_config(config: str, platform: PlatformName) -> "Set[Architecture]":
|
||||
result = set()
|
||||
for arch_str in re.split(r"[\s,]+", config):
|
||||
match arch_str:
|
||||
case "auto":
|
||||
result |= cls.auto_archs(platform=platform)
|
||||
case "native":
|
||||
if native_arch := cls.native_arch(platform=platform):
|
||||
result.add(native_arch)
|
||||
case "all":
|
||||
result |= cls.all_archs(platform=platform)
|
||||
case "auto64":
|
||||
result |= cls.bitness_archs(platform=platform, bitness="64")
|
||||
case "auto32":
|
||||
result |= cls.bitness_archs(platform=platform, bitness="32")
|
||||
case _:
|
||||
if arch := cls._parse_arch_name(arch_str, platform=platform):
|
||||
result.add(arch)
|
||||
else:
|
||||
msg = f"Invalid architecture '{arch_str}'"
|
||||
raise errors.ConfigurationError(msg)
|
||||
return result
|
||||
|
||||
@classmethod
|
||||
def _parse_arch_name(cls, arch_str: str, platform: PlatformName) -> Self | None:
|
||||
"""Resolve an architecture name case-insensitively.
|
||||
|
||||
The same value (e.g. "arm64"/"ARM64") can map to different members
|
||||
depending on the platform, so prefer a member valid for ``platform``
|
||||
before falling back to any case-insensitive match.
|
||||
"""
|
||||
for arch in cls.all_archs(platform):
|
||||
if arch.value.lower() == arch_str.lower():
|
||||
return arch
|
||||
for arch in cls:
|
||||
if arch.value.lower() == arch_str.lower():
|
||||
return arch
|
||||
return None
|
||||
|
||||
@classmethod
|
||||
def native_arch(cls, platform: PlatformName) -> Self | None:
|
||||
native_machine = platform_module.machine()
|
||||
native_architecture = cls(native_machine)
|
||||
|
||||
# Cross-platform support. Used for --print-build-identifiers or docker builds.
|
||||
host_platform: PlatformName = (
|
||||
"windows"
|
||||
if sys.platform.startswith("win")
|
||||
else ("macos" if sys.platform.startswith("darwin") else "linux")
|
||||
)
|
||||
|
||||
if platform == "pyodide":
|
||||
return cls.wasm32
|
||||
elif platform == "ios":
|
||||
# Can only build for iOS on macOS. The "native" architecture is the
|
||||
# simulator for the macOS native platform.
|
||||
if host_platform == "macos":
|
||||
if native_architecture == cls.x86_64:
|
||||
return cls.x86_64_iphonesimulator
|
||||
else:
|
||||
return cls.arm64_iphonesimulator
|
||||
if arch_str == "auto":
|
||||
result |= Architecture.auto_archs(platform=platform)
|
||||
elif arch_str == "native":
|
||||
result.add(Architecture(platform_module.machine()))
|
||||
elif arch_str == "all":
|
||||
result |= Architecture.all_archs(platform=platform)
|
||||
elif arch_str == "auto64":
|
||||
result |= Architecture.bitness_archs(platform=platform, bitness="64")
|
||||
elif arch_str == "auto32":
|
||||
result |= Architecture.bitness_archs(platform=platform, bitness="32")
|
||||
else:
|
||||
return None
|
||||
result.add(Architecture(arch_str))
|
||||
return result
|
||||
|
||||
# we might need to rename the native arch to the machine we're running
|
||||
# on, as the same arch can have different names on different platforms
|
||||
if host_platform != platform:
|
||||
synonym = arch_synonym(native_machine, host_platform, platform)
|
||||
if synonym is None:
|
||||
# can't build anything on this platform
|
||||
return None
|
||||
@staticmethod
|
||||
def auto_archs(platform: PlatformName) -> "Set[Architecture]":
|
||||
native_architecture = Architecture(platform_module.machine())
|
||||
result = {native_architecture}
|
||||
|
||||
native_architecture = cls(synonym)
|
||||
if platform == "linux" and native_architecture == Architecture.x86_64:
|
||||
# x86_64 machines can run i686 docker containers
|
||||
result.add(Architecture.i686)
|
||||
|
||||
return native_architecture
|
||||
if platform == "windows" and native_architecture == Architecture.AMD64:
|
||||
result.add(Architecture.x86)
|
||||
|
||||
@classmethod
|
||||
def auto_archs(cls, platform: PlatformName) -> set[Self]:
|
||||
native_arch = cls.native_arch(platform)
|
||||
if native_arch is None:
|
||||
return set() # can't build anything on this platform
|
||||
result = {native_arch}
|
||||
|
||||
match platform:
|
||||
case "windows" if cls.AMD64 in result:
|
||||
result.add(cls.x86)
|
||||
case "ios" if native_arch == cls.arm64_iphonesimulator:
|
||||
# Also build the device wheel if we're on ARM64.
|
||||
result.add(cls.arm64_iphoneos)
|
||||
if platform == "macos" and native_architecture == Architecture.arm64:
|
||||
# arm64 can build and test both archs of a universal2 wheel.
|
||||
result.add(Architecture.universal2)
|
||||
|
||||
return result
|
||||
|
||||
@classmethod
|
||||
def all_archs(cls, platform: PlatformName) -> set[Self]:
|
||||
all_archs_map = {
|
||||
"linux": {
|
||||
cls.x86_64,
|
||||
cls.i686,
|
||||
cls.aarch64,
|
||||
cls.ppc64le,
|
||||
cls.s390x,
|
||||
cls.armv7l,
|
||||
cls.riscv64,
|
||||
},
|
||||
"macos": {cls.x86_64, cls.arm64, cls.universal2},
|
||||
"windows": {cls.x86, cls.AMD64, cls.ARM64},
|
||||
"pyodide": {cls.wasm32},
|
||||
"android": {cls.x86_64, cls.arm64_v8a},
|
||||
"ios": {
|
||||
cls.x86_64_iphonesimulator,
|
||||
cls.arm64_iphonesimulator,
|
||||
cls.arm64_iphoneos,
|
||||
},
|
||||
}
|
||||
return all_archs_map[platform]
|
||||
@staticmethod
|
||||
def all_archs(platform: PlatformName) -> "Set[Architecture]":
|
||||
if platform == "linux":
|
||||
return {
|
||||
Architecture.x86_64,
|
||||
Architecture.i686,
|
||||
Architecture.aarch64,
|
||||
Architecture.ppc64le,
|
||||
Architecture.s390x,
|
||||
}
|
||||
elif platform == "macos":
|
||||
return {Architecture.x86_64, Architecture.arm64, Architecture.universal2}
|
||||
elif platform == "windows":
|
||||
return {Architecture.x86, Architecture.AMD64}
|
||||
else:
|
||||
assert_never(platform)
|
||||
|
||||
@classmethod
|
||||
def bitness_archs(cls, platform: PlatformName, bitness: Literal["64", "32"]) -> set[Self]:
|
||||
# This map maps 64-bit architectures to their 32-bit equivalents.
|
||||
archs_map = {
|
||||
cls.x86_64: cls.i686,
|
||||
cls.AMD64: cls.x86,
|
||||
cls.aarch64: cls.armv7l,
|
||||
}
|
||||
native_arch = cls.native_arch(platform)
|
||||
@staticmethod
|
||||
def bitness_archs(platform: PlatformName, bitness: Literal["64", "32"]) -> "Set[Architecture]":
|
||||
archs_32 = {Architecture.i686, Architecture.x86}
|
||||
auto_archs = Architecture.auto_archs(platform)
|
||||
|
||||
if native_arch is None:
|
||||
return set() # can't build anything on this platform
|
||||
|
||||
if native_arch == cls.wasm32:
|
||||
return {native_arch} if bitness == "32" else set()
|
||||
|
||||
match bitness:
|
||||
case "64":
|
||||
return {native_arch} if native_arch not in archs_map.values() else set()
|
||||
case "32":
|
||||
if native_arch in archs_map.values():
|
||||
return {native_arch}
|
||||
elif native_arch in archs_map and platform in {"linux", "windows"}:
|
||||
if native_arch == cls.aarch64 and not _check_aarch32_el0():
|
||||
# If we're on aarch64, skip if we cannot build armv7l wheels.
|
||||
return set()
|
||||
return {archs_map[native_arch]}
|
||||
else:
|
||||
return set()
|
||||
case _:
|
||||
typing.assert_never(bitness)
|
||||
if bitness == "64":
|
||||
return auto_archs - archs_32
|
||||
elif bitness == "32":
|
||||
return auto_archs & archs_32
|
||||
else:
|
||||
assert_never(bitness)
|
||||
|
||||
|
||||
def allowed_architectures_check(
|
||||
platform: PlatformName,
|
||||
architectures: Set[Architecture],
|
||||
) -> None:
|
||||
|
||||
allowed_architectures = Architecture.all_archs(platform)
|
||||
|
||||
msg = f"{PRETTY_NAMES[platform]} only supports {sorted(allowed_architectures)} at the moment."
|
||||
|
||||
@@ -1,148 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {
|
||||
"cibuildwheel.logger",
|
||||
"cibuildwheel.util",
|
||||
"cibuildwheel.util.cmd",
|
||||
"cibuildwheel.util.helpers",
|
||||
"cibuildwheel.util.packaging",
|
||||
"cibuildwheel.venv",
|
||||
"pathlib",
|
||||
"subprocess",
|
||||
}
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from cibuildwheel import errors
|
||||
from cibuildwheel.logger import log
|
||||
from cibuildwheel.util.cmd import call, shell
|
||||
from cibuildwheel.util.helpers import prepare_command
|
||||
from cibuildwheel.util.packaging import is_abi3_wheel
|
||||
from cibuildwheel.venv import activate_virtualenv, find_uv, virtualenv
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from cibuildwheel.options import BuildOptions
|
||||
|
||||
|
||||
def run_audit(
|
||||
*,
|
||||
tmp_dir: Path,
|
||||
build_options: BuildOptions,
|
||||
wheel: Path,
|
||||
) -> None:
|
||||
"""
|
||||
Run the audit commands on a single wheel.
|
||||
|
||||
Creates a virtualenv (or reuses an existing one) and installs any
|
||||
audit requirements, then runs each audit command template against
|
||||
the wheel. Commands containing {abi3_wheel} are skipped for
|
||||
non-abi3 wheels.
|
||||
"""
|
||||
|
||||
if not needs_audit(build_options.audit_command, wheel.name):
|
||||
return
|
||||
|
||||
log.step("Auditing wheel...")
|
||||
|
||||
use_uv = build_options.build_frontend.name in {"build[uv]", "uv"}
|
||||
version = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}"
|
||||
dependency_constraint = build_options.dependency_constraints.get_for_python_version(
|
||||
version=version, tmp_dir=tmp_dir
|
||||
)
|
||||
|
||||
# Use the base interpreter, not the venv python, to avoid nested-venv
|
||||
# issues where pip can't be found (seen on Windows + Python 3.13).
|
||||
host_python = Path(getattr(sys, "_base_executable", sys.executable))
|
||||
|
||||
audit_venv_dir = tmp_dir / "audit_venv"
|
||||
if not (audit_venv_dir / "pyvenv.cfg").exists():
|
||||
env = virtualenv(
|
||||
version,
|
||||
host_python,
|
||||
audit_venv_dir,
|
||||
dependency_constraint=dependency_constraint,
|
||||
use_uv=use_uv,
|
||||
)
|
||||
else:
|
||||
env = activate_virtualenv(audit_venv_dir)
|
||||
|
||||
# install audit requirements. This is run every time in case the user has
|
||||
# defined overrides.
|
||||
audit_requires = build_options.audit_requires
|
||||
if audit_requires:
|
||||
print(f"Installing audit dependencies: {', '.join(audit_requires)}")
|
||||
|
||||
pip: list[str]
|
||||
if use_uv:
|
||||
uv_path = find_uv()
|
||||
assert uv_path is not None
|
||||
pip = [str(uv_path), "pip"]
|
||||
else:
|
||||
pip = ["pip"]
|
||||
# we pin if the audit-requires is left as the default "abi3audit"
|
||||
should_pin = audit_requires == ["abi3audit"] and dependency_constraint
|
||||
|
||||
call(
|
||||
*pip,
|
||||
"install",
|
||||
*(["--constraint", str(dependency_constraint)] if should_pin else []),
|
||||
*audit_requires,
|
||||
env=env,
|
||||
)
|
||||
|
||||
audit_command = build_options.audit_command
|
||||
|
||||
for command_template in audit_command:
|
||||
if "{abi3_wheel}" in command_template and "{wheel}" in command_template:
|
||||
msg = (
|
||||
f"Invalid audit command {command_template!r}: cannot contain both {{abi3_wheel}} "
|
||||
"and {{wheel}} placeholders"
|
||||
)
|
||||
raise errors.ConfigurationError(msg)
|
||||
|
||||
if "{abi3_wheel}" in command_template and not is_abi3_wheel(wheel.name):
|
||||
continue
|
||||
|
||||
prepared_command = prepare_command(
|
||||
command_template,
|
||||
abi3_wheel=wheel,
|
||||
wheel=wheel,
|
||||
project=".",
|
||||
package=build_options.package_dir,
|
||||
)
|
||||
|
||||
print(f"Running audit command: {prepared_command}")
|
||||
try:
|
||||
shell(prepared_command, env=env)
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"Audit command failed with exit code {e.returncode}")
|
||||
msg = f"Audit command failed: {prepared_command}"
|
||||
raise errors.AuditCommandFailedError(msg) from e
|
||||
|
||||
|
||||
def needs_audit(audit_commands: list[str], wheel_name: str) -> bool:
|
||||
saw_abi3_placeholder = False
|
||||
for audit_command in audit_commands:
|
||||
if "{abi3_wheel}" not in audit_command and "{wheel}" not in audit_command:
|
||||
msg = (
|
||||
f"Invalid audit command {audit_command!r}: must contain either "
|
||||
"{{abi3_wheel}} or {{wheel}} placeholder"
|
||||
)
|
||||
raise errors.ConfigurationError(msg)
|
||||
|
||||
if "{abi3_wheel}" in audit_command:
|
||||
saw_abi3_placeholder = True
|
||||
if is_abi3_wheel(wheel_name):
|
||||
return True
|
||||
elif "{wheel}" in audit_command:
|
||||
return True
|
||||
|
||||
if saw_abi3_placeholder:
|
||||
print("No audit required for this wheel, as it is not abi3")
|
||||
else:
|
||||
print("No audit configured")
|
||||
|
||||
return False
|
||||
@@ -1,38 +1,26 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {"bashlex", "subprocess"}
|
||||
|
||||
import dataclasses
|
||||
import subprocess
|
||||
from typing import Callable, Dict, List, NamedTuple, Optional, Sequence
|
||||
|
||||
import bashlex
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import (
|
||||
Callable,
|
||||
Iterable,
|
||||
Mapping,
|
||||
Sequence,
|
||||
)
|
||||
|
||||
# a function that takes a command and the environment, and returns the result
|
||||
EnvironmentExecutor = Callable[[list[str], dict[str, str]], str]
|
||||
# a function that takes a command and the environment, and returns the result
|
||||
EnvironmentExecutor = Callable[[List[str], Dict[str, str]], str]
|
||||
|
||||
|
||||
def local_environment_executor(command: Sequence[str], env: Mapping[str, str]) -> str:
|
||||
return subprocess.run(command, env=env, text=True, stdout=subprocess.PIPE, check=True).stdout
|
||||
def local_environment_executor(command: List[str], env: Dict[str, str]) -> str:
|
||||
return subprocess.run(
|
||||
command, env=env, universal_newlines=True, stdout=subprocess.PIPE, check=True
|
||||
).stdout
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, kw_only=True)
|
||||
class NodeExecutionContext:
|
||||
environment: dict[str, str]
|
||||
class NodeExecutionContext(NamedTuple):
|
||||
environment: Dict[str, str]
|
||||
input: str
|
||||
executor: EnvironmentExecutor
|
||||
|
||||
|
||||
def evaluate(
|
||||
value: str, environment: Mapping[str, str], executor: EnvironmentExecutor | None = None
|
||||
value: str, environment: Dict[str, str], executor: Optional[EnvironmentExecutor] = None
|
||||
) -> str:
|
||||
if not value:
|
||||
# empty string evaluates to empty string
|
||||
@@ -42,17 +30,14 @@ def evaluate(
|
||||
command_node = bashlex.parsesingle(value)
|
||||
|
||||
if len(command_node.parts) != 1:
|
||||
msg = f"{value!r} has too many parts"
|
||||
raise ValueError(msg)
|
||||
raise ValueError(f'"{value}" has too many parts')
|
||||
|
||||
value_word_node = command_node.parts[0]
|
||||
|
||||
return evaluate_node(
|
||||
value_word_node,
|
||||
context=NodeExecutionContext(
|
||||
environment=dict(environment),
|
||||
input=value,
|
||||
executor=executor or local_environment_executor,
|
||||
environment=environment, input=value, executor=executor or local_environment_executor
|
||||
),
|
||||
)
|
||||
|
||||
@@ -67,8 +52,7 @@ def evaluate_node(node: bashlex.ast.node, context: NodeExecutionContext) -> str:
|
||||
elif node.kind == "parameter":
|
||||
return evaluate_parameter_node(node, context=context)
|
||||
else:
|
||||
msg = f"Unsupported bash construct: {node.kind!r}"
|
||||
raise ValueError(msg)
|
||||
raise ValueError(f'Unsupported bash construct: "{node.kind}"')
|
||||
|
||||
|
||||
def evaluate_word_node(node: bashlex.ast.node, context: NodeExecutionContext) -> str:
|
||||
@@ -79,8 +63,10 @@ def evaluate_word_node(node: bashlex.ast.node, context: NodeExecutionContext) ->
|
||||
part_value = evaluate_node(part, context=context)
|
||||
|
||||
if part_string not in value:
|
||||
msg = f"bash parse failed. part {part_string!r} not found in {value!r}. Word was {node.word!r}. Full input was {context.input!r}"
|
||||
raise RuntimeError(msg)
|
||||
raise RuntimeError(
|
||||
f'bash parse failed. part "{part_string}" not found in "{value}". '
|
||||
f'Word was "{node.word}". Full input was "{context.input}"'
|
||||
)
|
||||
|
||||
value = value.replace(part_string, part_value, 1)
|
||||
|
||||
@@ -107,17 +93,15 @@ def evaluate_nodes_as_compound_command(
|
||||
result += evaluate_command_node(node, context=context)
|
||||
elif node.kind == "operator":
|
||||
if node.op != ";":
|
||||
msg = f"Unsupported bash operator: {node.op!r}"
|
||||
raise ValueError(msg)
|
||||
raise ValueError(f'Unsupported bash operator: "{node.op}"')
|
||||
else:
|
||||
msg = f"Unsupported bash node in compound command: {node.kind!r}"
|
||||
raise ValueError(msg)
|
||||
raise ValueError(f'Unsupported bash node in compound command: "{node.kind}"')
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def evaluate_nodes_as_simple_command(
|
||||
nodes: Iterable[bashlex.ast.node], context: NodeExecutionContext
|
||||
nodes: List[bashlex.ast.node], context: NodeExecutionContext
|
||||
) -> str:
|
||||
command = [evaluate_node(part, context=context) for part in nodes]
|
||||
return context.executor(command, context.environment)
|
||||
|
||||
@@ -1,78 +0,0 @@
|
||||
__lazy_modules__ = {"cibuildwheel.util", "cibuildwheel.util.helpers"}
|
||||
|
||||
import os
|
||||
import re
|
||||
from enum import Enum
|
||||
|
||||
from cibuildwheel.util.helpers import strtobool
|
||||
|
||||
ANSI_CODE_REGEX = re.compile(r"(\033\[[0-9;]*m)")
|
||||
|
||||
|
||||
class CIProvider(Enum):
|
||||
# official support
|
||||
travis_ci = "travis"
|
||||
circle_ci = "circle_ci"
|
||||
azure_pipelines = "azure_pipelines"
|
||||
github_actions = "github_actions"
|
||||
gitlab = "gitlab"
|
||||
|
||||
# unofficial support
|
||||
appveyor = "appveyor"
|
||||
|
||||
other = "other"
|
||||
|
||||
|
||||
def detect_ci_provider() -> CIProvider | None:
|
||||
if "TRAVIS" in os.environ:
|
||||
return CIProvider.travis_ci
|
||||
elif "APPVEYOR" in os.environ:
|
||||
return CIProvider.appveyor
|
||||
elif "CIRCLECI" in os.environ:
|
||||
return CIProvider.circle_ci
|
||||
elif "AZURE_HTTP_USER_AGENT" in os.environ:
|
||||
return CIProvider.azure_pipelines
|
||||
elif "GITHUB_ACTIONS" in os.environ:
|
||||
return CIProvider.github_actions
|
||||
elif "GITLAB_CI" in os.environ:
|
||||
return CIProvider.gitlab
|
||||
elif strtobool(os.environ.get("CI", "false")):
|
||||
return CIProvider.other
|
||||
else:
|
||||
return None
|
||||
|
||||
|
||||
def fix_ansi_codes_for_github_actions(text: str) -> str:
|
||||
"""
|
||||
Github Actions forgets the current ANSI style on every new line. This
|
||||
function repeats the current ANSI style on every new line.
|
||||
"""
|
||||
|
||||
ansi_codes: list[str] = []
|
||||
output = ""
|
||||
|
||||
for line in text.splitlines(keepends=True):
|
||||
# add the current ANSI codes to the beginning of the line
|
||||
output += "".join(ansi_codes) + line
|
||||
|
||||
# split the line at each ANSI code
|
||||
parts = ANSI_CODE_REGEX.split(line)
|
||||
# if there are any ANSI codes, save them
|
||||
if len(parts) > 1:
|
||||
# iterate over the ANSI codes in this line
|
||||
for code in parts[1::2]:
|
||||
if code == "\033[0m":
|
||||
# reset the list of ANSI codes when the clear code is found
|
||||
ansi_codes = []
|
||||
else:
|
||||
ansi_codes.append(code)
|
||||
|
||||
return output
|
||||
|
||||
|
||||
def filter_ansi_codes(text: str, /) -> str:
|
||||
"""
|
||||
Remove ANSI codes from text.
|
||||
"""
|
||||
|
||||
return ANSI_CODE_REGEX.sub("", text)
|
||||
@@ -0,0 +1,242 @@
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
import uuid
|
||||
from pathlib import Path, PurePath
|
||||
from types import TracebackType
|
||||
from typing import IO, Dict, List, Optional, Sequence, Type, cast
|
||||
|
||||
from .typing import PathOrStr, PopenBytes
|
||||
|
||||
|
||||
class DockerContainer:
|
||||
"""
|
||||
An object that represents a running Docker container.
|
||||
|
||||
Intended for use as a context manager e.g.
|
||||
`with DockerContainer('ubuntu') as docker:`
|
||||
|
||||
A bash shell is running in the remote container. When `call()` is invoked,
|
||||
the command is relayed to the remote shell, and the results are streamed
|
||||
back to cibuildwheel.
|
||||
"""
|
||||
|
||||
UTILITY_PYTHON = "/opt/python/cp38-cp38/bin/python"
|
||||
|
||||
process: PopenBytes
|
||||
bash_stdin: IO[bytes]
|
||||
bash_stdout: IO[bytes]
|
||||
|
||||
def __init__(
|
||||
self, docker_image: str, simulate_32_bit: bool = False, cwd: Optional[PathOrStr] = None
|
||||
):
|
||||
if not docker_image:
|
||||
raise ValueError("Must have a non-empty docker image to run.")
|
||||
|
||||
self.docker_image = docker_image
|
||||
self.simulate_32_bit = simulate_32_bit
|
||||
self.cwd = cwd
|
||||
self.name: Optional[str] = None
|
||||
|
||||
def __enter__(self) -> "DockerContainer":
|
||||
self.name = f"cibuildwheel-{uuid.uuid4()}"
|
||||
cwd_args = ["-w", str(self.cwd)] if self.cwd else []
|
||||
shell_args = ["linux32", "/bin/bash"] if self.simulate_32_bit else ["/bin/bash"]
|
||||
subprocess.run(
|
||||
[
|
||||
"docker",
|
||||
"create",
|
||||
"--env=CIBUILDWHEEL",
|
||||
f"--name={self.name}",
|
||||
"--interactive",
|
||||
"--volume=/:/host", # ignored on CircleCI
|
||||
*cwd_args,
|
||||
self.docker_image,
|
||||
*shell_args,
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
self.process = subprocess.Popen(
|
||||
[
|
||||
"docker",
|
||||
"start",
|
||||
"--attach",
|
||||
"--interactive",
|
||||
self.name,
|
||||
],
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
)
|
||||
|
||||
assert self.process.stdin and self.process.stdout
|
||||
self.bash_stdin = self.process.stdin
|
||||
self.bash_stdout = self.process.stdout
|
||||
|
||||
# run a noop command to block until the container is responding
|
||||
self.call(["/bin/true"])
|
||||
|
||||
return self
|
||||
|
||||
def __exit__(
|
||||
self,
|
||||
exc_type: Optional[Type[BaseException]],
|
||||
exc_val: Optional[BaseException],
|
||||
exc_tb: Optional[TracebackType],
|
||||
) -> None:
|
||||
|
||||
self.bash_stdin.close()
|
||||
self.process.terminate()
|
||||
self.process.wait()
|
||||
|
||||
assert isinstance(self.name, str)
|
||||
|
||||
subprocess.run(["docker", "rm", "--force", "-v", self.name], stdout=subprocess.DEVNULL)
|
||||
self.name = None
|
||||
|
||||
def copy_into(self, from_path: Path, to_path: PurePath) -> None:
|
||||
# `docker cp` causes 'no space left on device' error when
|
||||
# a container is running and the host filesystem is
|
||||
# mounted. https://github.com/moby/moby/issues/38995
|
||||
# Use `docker exec` instead.
|
||||
|
||||
if from_path.is_dir():
|
||||
self.call(["mkdir", "-p", to_path])
|
||||
subprocess.run(
|
||||
f"tar cf - . | docker exec -i {self.name} tar -xC {shell_quote(to_path)} -f -",
|
||||
shell=True,
|
||||
check=True,
|
||||
cwd=from_path,
|
||||
)
|
||||
else:
|
||||
subprocess.run(
|
||||
f'cat {shell_quote(from_path)} | docker exec -i {self.name} sh -c "cat > {shell_quote(to_path)}"',
|
||||
shell=True,
|
||||
check=True,
|
||||
)
|
||||
|
||||
def copy_out(self, from_path: PurePath, to_path: Path) -> None:
|
||||
# note: we assume from_path is a dir
|
||||
to_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
subprocess.run(
|
||||
f"docker exec -i {self.name} tar -cC {shell_quote(from_path)} -f - . | tar -xf -",
|
||||
shell=True,
|
||||
check=True,
|
||||
cwd=to_path,
|
||||
)
|
||||
|
||||
def glob(self, path: PurePath, pattern: str) -> List[PurePath]:
|
||||
glob_pattern = os.path.join(str(path), pattern)
|
||||
|
||||
path_strs = json.loads(
|
||||
self.call(
|
||||
[
|
||||
self.UTILITY_PYTHON,
|
||||
"-c",
|
||||
f"import sys, json, glob; json.dump(glob.glob({glob_pattern!r}), sys.stdout)",
|
||||
],
|
||||
capture_output=True,
|
||||
)
|
||||
)
|
||||
|
||||
return [PurePath(p) for p in path_strs]
|
||||
|
||||
def call(
|
||||
self,
|
||||
args: Sequence[PathOrStr],
|
||||
env: Optional[Dict[str, str]] = None,
|
||||
capture_output: bool = False,
|
||||
cwd: Optional[PathOrStr] = None,
|
||||
) -> str:
|
||||
|
||||
chdir = f"cd {cwd}" if cwd else ""
|
||||
env_assignments = (
|
||||
" ".join(f"{shlex.quote(k)}={shlex.quote(v)}" for k, v in env.items())
|
||||
if env is not None
|
||||
else ""
|
||||
)
|
||||
command = " ".join(shlex.quote(str(a)) for a in args)
|
||||
end_of_message = str(uuid.uuid4())
|
||||
|
||||
# log the command we're executing
|
||||
print(f" + {command}")
|
||||
|
||||
# Write a command to the remote shell. First we change the
|
||||
# cwd, if that's required. Then, we use the `env` utility to run
|
||||
# `command` inside the specified environment. We use `env` because it
|
||||
# can cope with spaces and strange characters in the name or value.
|
||||
# Finally, the remote shell is told to write a footer - this will show
|
||||
# up in the output so we know when to stop reading, and will include
|
||||
# the returncode of `command`.
|
||||
self.bash_stdin.write(
|
||||
bytes(
|
||||
f"""(
|
||||
{chdir}
|
||||
env {env_assignments} {command}
|
||||
printf "%04d%s\n" $? {end_of_message}
|
||||
)
|
||||
""",
|
||||
encoding="utf8",
|
||||
errors="surrogateescape",
|
||||
)
|
||||
)
|
||||
self.bash_stdin.flush()
|
||||
|
||||
if capture_output:
|
||||
output_io: IO[bytes] = io.BytesIO()
|
||||
else:
|
||||
output_io = sys.stdout.buffer
|
||||
|
||||
while True:
|
||||
line = self.bash_stdout.readline()
|
||||
|
||||
if line.endswith(bytes(end_of_message, encoding="utf8") + b"\n"):
|
||||
# fmt: off
|
||||
footer_offset = (
|
||||
len(line)
|
||||
- 1 # newline character
|
||||
- len(end_of_message) # delimiter
|
||||
- 4 # 4 returncode decimals
|
||||
)
|
||||
# fmt: on
|
||||
returncode_str = line[footer_offset : footer_offset + 4]
|
||||
returncode = int(returncode_str)
|
||||
# add the last line to output, without the footer
|
||||
output_io.write(line[0:footer_offset])
|
||||
break
|
||||
else:
|
||||
output_io.write(line)
|
||||
|
||||
if isinstance(output_io, io.BytesIO):
|
||||
output = str(output_io.getvalue(), encoding="utf8", errors="surrogateescape")
|
||||
else:
|
||||
output = ""
|
||||
|
||||
if returncode != 0:
|
||||
raise subprocess.CalledProcessError(returncode, args, output)
|
||||
|
||||
return output
|
||||
|
||||
def get_environment(self) -> Dict[str, str]:
|
||||
env = json.loads(
|
||||
self.call(
|
||||
[
|
||||
self.UTILITY_PYTHON,
|
||||
"-c",
|
||||
"import sys, json, os; json.dump(os.environ.copy(), sys.stdout)",
|
||||
],
|
||||
capture_output=True,
|
||||
)
|
||||
)
|
||||
return cast(Dict[str, str], env)
|
||||
|
||||
def environment_executor(self, command: List[str], environment: Dict[str, str]) -> str:
|
||||
# used as an EnvironmentExecutor to evaluate commands and capture output
|
||||
return self.call(command, env=environment, capture_output=True)
|
||||
|
||||
|
||||
def shell_quote(path: PurePath) -> str:
|
||||
return shlex.quote(str(path))
|
||||
+20
-78
@@ -1,26 +1,15 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {"bashlex", "bashlex.errors"}
|
||||
|
||||
import dataclasses
|
||||
from typing import Protocol
|
||||
from typing import Dict, List, Mapping, Optional
|
||||
|
||||
import bashlex
|
||||
import bashlex.errors
|
||||
|
||||
from cibuildwheel import bashlex_eval
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Mapping, Sequence
|
||||
from typing import Any
|
||||
from . import bashlex_eval
|
||||
|
||||
|
||||
class EnvironmentParseError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def split_env_items(env_string: str) -> list[str]:
|
||||
def split_env_items(env_string: str) -> List[str]:
|
||||
"""Splits space-separated variable assignments into a list of individual assignments.
|
||||
|
||||
>>> split_env_items('VAR=abc')
|
||||
@@ -39,11 +28,7 @@ def split_env_items(env_string: str) -> list[str]:
|
||||
if not env_string:
|
||||
return []
|
||||
|
||||
try:
|
||||
command_node = bashlex.parsesingle(env_string)
|
||||
except bashlex.errors.ParsingError as e:
|
||||
raise EnvironmentParseError(env_string) from e
|
||||
|
||||
command_node = bashlex.parsesingle(env_string)
|
||||
result = []
|
||||
|
||||
for word_node in command_node.parts:
|
||||
@@ -53,44 +38,7 @@ def split_env_items(env_string: str) -> list[str]:
|
||||
return result
|
||||
|
||||
|
||||
class EnvironmentAssignment(Protocol):
|
||||
name: str
|
||||
|
||||
def evaluated_value(
|
||||
self,
|
||||
*,
|
||||
environment: Mapping[str, str],
|
||||
executor: bashlex_eval.EnvironmentExecutor | None = None,
|
||||
) -> str:
|
||||
"""Returns the value of this assignment, as evaluated in the environment"""
|
||||
|
||||
|
||||
class EnvironmentAssignmentRaw:
|
||||
"""
|
||||
An environment variable - a simple name/value pair
|
||||
"""
|
||||
|
||||
def __init__(self, name: str, value: str):
|
||||
self.name = name
|
||||
self.value = value
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"{self.name}={self.value}"
|
||||
|
||||
def evaluated_value(self, **_: Any) -> str:
|
||||
return self.value
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class EnvironmentAssignmentBash:
|
||||
"""
|
||||
An environment variable, in bash syntax. The value can use bash constructs
|
||||
like "$OTHER_VAR" and "$(command arg1 arg2)".
|
||||
"""
|
||||
|
||||
name: str
|
||||
value: str
|
||||
|
||||
class EnvironmentAssignment:
|
||||
def __init__(self, assignment: str):
|
||||
name, equals, value = assignment.partition("=")
|
||||
if not equals:
|
||||
@@ -100,28 +48,29 @@ class EnvironmentAssignmentBash:
|
||||
|
||||
def evaluated_value(
|
||||
self,
|
||||
environment: Mapping[str, str],
|
||||
executor: bashlex_eval.EnvironmentExecutor | None = None,
|
||||
environment: Dict[str, str],
|
||||
executor: Optional[bashlex_eval.EnvironmentExecutor] = None,
|
||||
) -> str:
|
||||
"""Returns the value of this assignment, as evaluated in the environment"""
|
||||
return bashlex_eval.evaluate(self.value, environment=environment, executor=executor)
|
||||
|
||||
def as_shell_assignment(self) -> str:
|
||||
return f"export {self.name}={self.value}"
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"{self.name}={self.value}"
|
||||
|
||||
|
||||
@dataclasses.dataclass(kw_only=True)
|
||||
class ParsedEnvironment:
|
||||
assignments: list[EnvironmentAssignment]
|
||||
|
||||
def __init__(self, assignments: Sequence[EnvironmentAssignment]) -> None:
|
||||
self.assignments = list(assignments)
|
||||
def __init__(self, assignments: List[EnvironmentAssignment]):
|
||||
self.assignments = assignments
|
||||
|
||||
def as_dictionary(
|
||||
self,
|
||||
prev_environment: Mapping[str, str],
|
||||
executor: bashlex_eval.EnvironmentExecutor | None = None,
|
||||
) -> dict[str, str]:
|
||||
environment = {**prev_environment}
|
||||
executor: Optional[bashlex_eval.EnvironmentExecutor] = None,
|
||||
) -> Dict[str, str]:
|
||||
environment = dict(**prev_environment)
|
||||
|
||||
for assignment in self.assignments:
|
||||
value = assignment.evaluated_value(environment=environment, executor=executor)
|
||||
@@ -129,21 +78,14 @@ class ParsedEnvironment:
|
||||
|
||||
return environment
|
||||
|
||||
def add(self, name: str, value: str, prepend: bool = False) -> None:
|
||||
assignment = EnvironmentAssignmentRaw(name=name, value=value)
|
||||
if prepend:
|
||||
self.assignments.insert(0, assignment)
|
||||
else:
|
||||
self.assignments.append(assignment)
|
||||
def as_shell_commands(self) -> List[str]:
|
||||
return [a.as_shell_assignment() for a in self.assignments]
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"{self.__class__.__name__}({[repr(a) for a in self.assignments]!r})"
|
||||
|
||||
def options_summary(self) -> list[EnvironmentAssignment]:
|
||||
return self.assignments
|
||||
return f"ParsedEnvironment({[repr(a) for a in self.assignments]!r})"
|
||||
|
||||
|
||||
def parse_environment(env_string: str) -> ParsedEnvironment:
|
||||
env_items = split_env_items(env_string)
|
||||
assignments = [EnvironmentAssignmentBash(item) for item in env_items]
|
||||
assignments = [EnvironmentAssignment(item) for item in env_items]
|
||||
return ParsedEnvironment(assignments=assignments)
|
||||
|
||||
@@ -1,129 +0,0 @@
|
||||
"""
|
||||
Errors that can cause the build to fail. Each subclass of FatalError has
|
||||
a different return code, by defining them all here, we can ensure that they're
|
||||
semantically clear and unique.
|
||||
"""
|
||||
|
||||
__lazy_modules__ = {"textwrap"}
|
||||
|
||||
import textwrap
|
||||
|
||||
|
||||
class FatalError(BaseException):
|
||||
"""
|
||||
Raising an error of this type will cause the message to be printed to
|
||||
stderr and the process to be terminated. Within cibuildwheel, raising this
|
||||
exception produces a better error message, and optional traceback.
|
||||
"""
|
||||
|
||||
return_code: int = 1
|
||||
|
||||
|
||||
class ConfigurationError(FatalError):
|
||||
return_code = 2
|
||||
|
||||
|
||||
class NothingToDoError(FatalError):
|
||||
return_code = 3
|
||||
|
||||
|
||||
class DeprecationError(FatalError):
|
||||
return_code = 4
|
||||
|
||||
|
||||
class NonPlatformWheelError(FatalError):
|
||||
def __init__(self) -> None:
|
||||
message = textwrap.dedent(
|
||||
"""
|
||||
Build failed because a pure Python wheel was generated.
|
||||
|
||||
If you intend to build a pure-Python wheel, you don't need
|
||||
cibuildwheel - use `pip wheel .`, `pipx run build --wheel`, `uv
|
||||
build --wheel`, etc. instead. You only need cibuildwheel if you
|
||||
have compiled (not Python) code in your wheels making them depend
|
||||
on the platform.
|
||||
|
||||
If you expected a platform wheel, check your project configuration,
|
||||
or run cibuildwheel with CIBW_BUILD_VERBOSITY=1 to view build logs.
|
||||
"""
|
||||
)
|
||||
super().__init__(message)
|
||||
self.return_code = 5
|
||||
|
||||
|
||||
class AlreadyBuiltWheelError(FatalError):
|
||||
def __init__(self, wheel_name: str) -> None:
|
||||
message = textwrap.dedent(
|
||||
f"""
|
||||
Build failed because a wheel named {wheel_name} was already generated in the current run.
|
||||
|
||||
If you expected another wheel to be generated, check your project configuration, or run
|
||||
cibuildwheel with CIBW_BUILD_VERBOSITY=1 to view build logs.
|
||||
"""
|
||||
)
|
||||
super().__init__(message)
|
||||
self.return_code = 6
|
||||
|
||||
|
||||
class OCIEngineTooOldError(FatalError):
|
||||
def __init__(self, message: str) -> None:
|
||||
super().__init__(message)
|
||||
self.return_code = 7
|
||||
|
||||
|
||||
class RepairStepProducedNoWheelError(FatalError):
|
||||
def __init__(self) -> None:
|
||||
message = textwrap.dedent(
|
||||
"""
|
||||
Build failed because the repair step completed successfully but
|
||||
did not produce a wheel.
|
||||
|
||||
Your `repair-wheel-command` must place the repaired
|
||||
wheel in the {dest_dir} directory. See the documentation for
|
||||
example configurations:
|
||||
|
||||
https://cibuildwheel.pypa.io/en/stable/options/#repair-wheel-command
|
||||
"""
|
||||
)
|
||||
super().__init__(message)
|
||||
self.return_code = 8
|
||||
|
||||
|
||||
class RepairStepProducedMultipleWheelsError(FatalError):
|
||||
def __init__(self, wheels: list[str]) -> None:
|
||||
message = textwrap.dedent(
|
||||
f"""
|
||||
Build failed because the repair step completed successfully but
|
||||
produced multiple wheels: {wheels}
|
||||
|
||||
Your `repair-wheel-command` must place exactly one repaired
|
||||
wheel in the {{dest_dir}} directory. See the documentation for
|
||||
example configurations:
|
||||
|
||||
https://cibuildwheel.pypa.io/en/stable/options/#repair-wheel-command
|
||||
"""
|
||||
)
|
||||
super().__init__(message)
|
||||
self.return_code = 8
|
||||
|
||||
|
||||
class AuditCommandFailedError(FatalError):
|
||||
def __init__(self, message: str) -> None:
|
||||
super().__init__(message)
|
||||
self.return_code = 9
|
||||
|
||||
|
||||
class BuildProducedNoWheelError(FatalError):
|
||||
def __init__(self) -> None:
|
||||
message = textwrap.dedent(
|
||||
"""
|
||||
Build failed because the build frontend completed successfully but
|
||||
did not produce a wheel.
|
||||
|
||||
This usually indicates a problem with your project configuration or
|
||||
build backend. Check your project configuration, or run cibuildwheel
|
||||
with CIBW_BUILD_VERBOSITY=1 to view build logs.
|
||||
"""
|
||||
)
|
||||
super().__init__(message)
|
||||
self.return_code = 10
|
||||
+15
-103
@@ -2,110 +2,22 @@
|
||||
These are utilities for the `/bin` scripts, not for the `cibuildwheel` program.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
from typing import Any, Dict
|
||||
|
||||
__lazy_modules__ = {"io", "json", "urllib", "urllib.error", "urllib.request"}
|
||||
|
||||
import json
|
||||
import time
|
||||
import typing
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from io import StringIO
|
||||
from typing import NotRequired, Protocol
|
||||
|
||||
from cibuildwheel import __version__ as cibw_version
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Mapping, Sequence
|
||||
from typing import Any
|
||||
import toml.encoder
|
||||
from packaging.version import Version
|
||||
|
||||
|
||||
__all__ = ("Printable", "dump_python_configurations")
|
||||
class InlineArrayDictEncoder(toml.encoder.TomlEncoder): # type: ignore
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.dump_funcs[Version] = lambda v: f'"{v}"'
|
||||
|
||||
|
||||
class Printable(Protocol):
|
||||
def __str__(self) -> str: ...
|
||||
|
||||
|
||||
def dump_python_configurations(
|
||||
inp: Mapping[str, Mapping[str, Sequence[Mapping[str, Printable]]]],
|
||||
) -> str:
|
||||
output = StringIO()
|
||||
for header, values in inp.items():
|
||||
output.write(f"[{header}]\n")
|
||||
for inner_header, listing in values.items():
|
||||
output.write(f"{inner_header} = [\n")
|
||||
for item in listing:
|
||||
output.write(" { ")
|
||||
dict_contents = (f'{key} = "{value}"' for key, value in item.items())
|
||||
output.write(", ".join(dict_contents))
|
||||
output.write(" },\n")
|
||||
output.write("]\n")
|
||||
output.write("\n")
|
||||
# Strip the final newline, to avoid two blank lines at the end.
|
||||
return output.getvalue()[:-1]
|
||||
|
||||
|
||||
def _json_request(request: urllib.request.Request, timeout: int = 30) -> dict[str, Any]:
|
||||
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||
return typing.cast("dict[str, Any]", json.load(response))
|
||||
|
||||
|
||||
def github_api_request(path: str, *, max_retries: int = 3) -> dict[str, Any]:
|
||||
"""
|
||||
Makes a GitHub API request to the given path and returns the JSON response.
|
||||
"""
|
||||
api_url = f"https://api.github.com/{path}"
|
||||
headers = {
|
||||
"Accept": "application/vnd.github.v3+json",
|
||||
"User-Agent": f"cibuildwheel/{cibw_version}",
|
||||
}
|
||||
request = urllib.request.Request(api_url, headers=headers)
|
||||
|
||||
for retry_count in range(max_retries):
|
||||
try:
|
||||
return _json_request(request)
|
||||
except (urllib.error.URLError, TimeoutError) as e:
|
||||
# pylint: disable=E1101
|
||||
if (
|
||||
isinstance(e, urllib.error.HTTPError)
|
||||
and (e.code in {403, 429})
|
||||
and e.headers.get("x-ratelimit-remaining") == "0"
|
||||
):
|
||||
reset_time = int(e.headers.get("x-ratelimit-reset", 0))
|
||||
wait_time = max(0, reset_time - int(e.headers.get("date", 0)))
|
||||
print(f"Github rate limit exceeded. Waiting for {wait_time} seconds.")
|
||||
time.sleep(wait_time)
|
||||
else:
|
||||
print(f"Retrying GitHub API request due to error: {e}")
|
||||
|
||||
if retry_count == max_retries - 1:
|
||||
print(f"GitHub API request failed (Network error: {e}). Check network connection.")
|
||||
raise e
|
||||
|
||||
# Should never be reached but to keep the type checker happy
|
||||
msg = "Unexpected execution path in github_api_request"
|
||||
raise RuntimeError(msg)
|
||||
|
||||
|
||||
class PyodideXBuildEnvRelease(typing.TypedDict):
|
||||
version: str
|
||||
sha256: str
|
||||
python_version: str
|
||||
emscripten_version: str
|
||||
min_pyodide_build_version: NotRequired[str]
|
||||
max_pyodide_build_version: NotRequired[str]
|
||||
|
||||
|
||||
class PyodideXBuildEnvInfo(typing.TypedDict):
|
||||
releases: dict[str, PyodideXBuildEnvRelease]
|
||||
|
||||
|
||||
def get_pyodide_xbuildenv_info() -> PyodideXBuildEnvInfo:
|
||||
xbuildenv_info_url = (
|
||||
"https://pyodide.github.io/pyodide/api/v2/pyodide-cross-build-environments.json"
|
||||
)
|
||||
with urllib.request.urlopen(xbuildenv_info_url) as response:
|
||||
return typing.cast("PyodideXBuildEnvInfo", json.loads(response.read().decode("utf-8")))
|
||||
def dump_sections(self, o: Dict[str, Any], sup: str) -> Any:
|
||||
if not all(isinstance(a, list) for a in o.values()):
|
||||
return super().dump_sections(o, sup)
|
||||
val = ""
|
||||
for k, v in o.items():
|
||||
inner = ",\n ".join(self.dump_inline_table(d_i).strip() for d_i in v)
|
||||
val += f"{k} = [\n {inner},\n]\n"
|
||||
return val, self._dict()
|
||||
|
||||
@@ -1,108 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {"cibuildwheel.util", "cibuildwheel.util.helpers", "shlex"}
|
||||
|
||||
import dataclasses
|
||||
import shlex
|
||||
import typing
|
||||
from typing import Literal, get_args
|
||||
|
||||
from cibuildwheel.util.helpers import parse_key_value_string, prepare_command
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence
|
||||
from typing import Self
|
||||
|
||||
from cibuildwheel.typing import PathOrStr
|
||||
|
||||
BuildFrontendName = Literal["pip", "build", "build[uv]", "uv", "pyodide-build"]
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class BuildFrontendConfig:
|
||||
name: BuildFrontendName
|
||||
args: Sequence[str] = ()
|
||||
|
||||
@classmethod
|
||||
def from_config_string(cls, config_string: str) -> Self:
|
||||
config_dict = parse_key_value_string(config_string, ["name"], ["args"])
|
||||
name = " ".join(config_dict["name"])
|
||||
if name not in get_args(BuildFrontendName):
|
||||
names = ", ".join(repr(n) for n in get_args(BuildFrontendName))
|
||||
msg = f"Unrecognised build frontend {name!r}, must be one of {names}"
|
||||
raise ValueError(msg)
|
||||
|
||||
name = typing.cast("BuildFrontendName", name)
|
||||
|
||||
args = config_dict.get("args") or []
|
||||
return cls(name=name, args=args)
|
||||
|
||||
def options_summary(self) -> str | dict[str, str]:
|
||||
if not self.args:
|
||||
return self.name
|
||||
else:
|
||||
return {"name": self.name, "args": repr(self.args)}
|
||||
|
||||
|
||||
def _get_verbosity_flags(level: int, frontend: BuildFrontendName) -> list[str]:
|
||||
if frontend == "pyodide-build":
|
||||
if level > 0:
|
||||
return ["-" + min(level, 2) * "v"]
|
||||
return []
|
||||
|
||||
if level < 0:
|
||||
return ["-" + -level * "q"]
|
||||
|
||||
if level > 0:
|
||||
if frontend == "pip":
|
||||
return ["-" + level * "v"]
|
||||
if level > 1:
|
||||
return ["-" + (level - 1) * "v"]
|
||||
|
||||
return []
|
||||
|
||||
|
||||
def _split_config_settings(config_settings: str) -> list[str]:
|
||||
config_settings_list = shlex.split(config_settings)
|
||||
return [f"-C{setting}" for setting in config_settings_list]
|
||||
|
||||
|
||||
def prepare_config_settings(config_settings: str, *, project: PathOrStr, package: PathOrStr) -> str:
|
||||
# Substitute the {project}/{package} placeholders on each already-split
|
||||
# token rather than on the raw string. A substituted path may contain
|
||||
# spaces or backslashes (e.g. a Windows `{package}` path), and the result
|
||||
# is later re-parsed with shlex.split (in _split_config_settings /
|
||||
# parse_config_settings) — substituting on the whole string would let
|
||||
# those characters be reinterpreted, splitting one setting into several or
|
||||
# eating backslashes. shlex.join re-quotes each token so the round-trip is
|
||||
# lossless.
|
||||
settings = shlex.split(config_settings)
|
||||
prepared = [prepare_command(setting, project=project, package=package) for setting in settings]
|
||||
return shlex.join(prepared)
|
||||
|
||||
|
||||
# Based on build.__main__.main.
|
||||
def parse_config_settings(config_settings_str: str) -> dict[str, str | list[str]]:
|
||||
config_settings: dict[str, str | list[str]] = {}
|
||||
for arg in shlex.split(config_settings_str):
|
||||
setting, _, value = arg.partition("=")
|
||||
existing_value = config_settings.get(setting)
|
||||
if existing_value is None:
|
||||
config_settings[setting] = value
|
||||
elif isinstance(existing_value, str):
|
||||
config_settings[setting] = [existing_value, value]
|
||||
else:
|
||||
existing_value.append(value)
|
||||
|
||||
return config_settings
|
||||
|
||||
|
||||
def get_build_frontend_extra_flags(
|
||||
build_frontend: BuildFrontendConfig, verbosity_level: int, config_settings: str
|
||||
) -> list[str]:
|
||||
return [
|
||||
*_split_config_settings(config_settings),
|
||||
*build_frontend.args,
|
||||
*_get_verbosity_flags(verbosity_level, build_frontend.name),
|
||||
]
|
||||
@@ -0,0 +1,336 @@
|
||||
import subprocess
|
||||
import sys
|
||||
import textwrap
|
||||
from pathlib import Path, PurePath
|
||||
from typing import List, NamedTuple, Set
|
||||
|
||||
from .architecture import Architecture
|
||||
from .docker_container import DockerContainer
|
||||
from .logger import log
|
||||
from .typing import PathOrStr
|
||||
from .util import (
|
||||
BuildOptions,
|
||||
BuildSelector,
|
||||
NonPlatformWheelError,
|
||||
get_build_verbosity_extra_flags,
|
||||
prepare_command,
|
||||
read_python_configs,
|
||||
resources_dir,
|
||||
)
|
||||
|
||||
|
||||
class PythonConfiguration(NamedTuple):
|
||||
version: str
|
||||
identifier: str
|
||||
path_str: str
|
||||
|
||||
@property
|
||||
def path(self) -> PurePath:
|
||||
return PurePath(self.path_str)
|
||||
|
||||
|
||||
def get_python_configurations(
|
||||
build_selector: BuildSelector,
|
||||
architectures: Set[Architecture],
|
||||
) -> List[PythonConfiguration]:
|
||||
|
||||
full_python_configs = read_python_configs("linux")
|
||||
|
||||
python_configurations = [PythonConfiguration(**item) for item in full_python_configs]
|
||||
|
||||
# return all configurations whose arch is in our `architectures` set,
|
||||
# and match the build/skip rules
|
||||
return [
|
||||
c
|
||||
for c in python_configurations
|
||||
if any(c.identifier.endswith(arch.value) for arch in architectures)
|
||||
and build_selector(c.identifier)
|
||||
]
|
||||
|
||||
|
||||
def build(options: BuildOptions) -> None:
|
||||
try:
|
||||
# check docker is installed
|
||||
subprocess.run(["docker", "--version"], check=True, stdout=subprocess.DEVNULL)
|
||||
except Exception:
|
||||
print(
|
||||
"cibuildwheel: Docker not found. Docker is required to run Linux builds. "
|
||||
"If you're building on Travis CI, add `services: [docker]` to your .travis.yml."
|
||||
"If you're building on Circle CI in Linux, add a `setup_remote_docker` step to your .circleci/config.yml",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(2)
|
||||
|
||||
assert options.manylinux_images is not None
|
||||
python_configurations = get_python_configurations(options.build_selector, options.architectures)
|
||||
platforms = [
|
||||
("cp", "manylinux_x86_64", options.manylinux_images["x86_64"]),
|
||||
("cp", "manylinux_i686", options.manylinux_images["i686"]),
|
||||
("cp", "manylinux_aarch64", options.manylinux_images["aarch64"]),
|
||||
("cp", "manylinux_ppc64le", options.manylinux_images["ppc64le"]),
|
||||
("cp", "manylinux_s390x", options.manylinux_images["s390x"]),
|
||||
("pp", "manylinux_x86_64", options.manylinux_images["pypy_x86_64"]),
|
||||
]
|
||||
|
||||
cwd = Path.cwd()
|
||||
abs_package_dir = options.package_dir.resolve()
|
||||
if cwd != abs_package_dir and cwd not in abs_package_dir.parents:
|
||||
raise Exception("package_dir must be inside the working directory")
|
||||
|
||||
container_project_path = PurePath("/project")
|
||||
container_package_dir = container_project_path / abs_package_dir.relative_to(cwd)
|
||||
container_output_dir = PurePath("/output")
|
||||
|
||||
for implementation, platform_tag, docker_image in platforms:
|
||||
platform_configs = [
|
||||
c
|
||||
for c in python_configurations
|
||||
if c.identifier.startswith(implementation) and c.identifier.endswith(platform_tag)
|
||||
]
|
||||
if not platform_configs:
|
||||
continue
|
||||
|
||||
try:
|
||||
log.step(f"Starting Docker image {docker_image}...")
|
||||
with DockerContainer(
|
||||
docker_image,
|
||||
simulate_32_bit=platform_tag.endswith("i686"),
|
||||
cwd=container_project_path,
|
||||
) as docker:
|
||||
|
||||
log.step("Copying project into Docker...")
|
||||
docker.copy_into(Path.cwd(), container_project_path)
|
||||
|
||||
if options.before_all:
|
||||
log.step("Running before_all...")
|
||||
|
||||
env = docker.get_environment()
|
||||
env["PATH"] = f'/opt/python/cp38-cp38/bin:{env["PATH"]}'
|
||||
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
|
||||
env = options.environment.as_dictionary(
|
||||
env, executor=docker.environment_executor
|
||||
)
|
||||
|
||||
before_all_prepared = prepare_command(
|
||||
options.before_all,
|
||||
project=container_project_path,
|
||||
package=container_package_dir,
|
||||
)
|
||||
docker.call(["sh", "-c", before_all_prepared], env=env)
|
||||
|
||||
for config in platform_configs:
|
||||
log.build_start(config.identifier)
|
||||
|
||||
dependency_constraint_flags: List[PathOrStr] = []
|
||||
if config.identifier.startswith("pp"):
|
||||
# Patch PyPy to make sure headers get installed into a venv
|
||||
patch_version = "_27" if config.version == "2.7" else ""
|
||||
patch_path = resources_dir / f"pypy_venv{patch_version}.patch"
|
||||
patch_docker_path = PurePath("/pypy_venv.patch")
|
||||
docker.copy_into(patch_path, patch_docker_path)
|
||||
try:
|
||||
docker.call(
|
||||
[
|
||||
"patch",
|
||||
"--force",
|
||||
"-p1",
|
||||
"-d",
|
||||
config.path,
|
||||
"-i",
|
||||
patch_docker_path,
|
||||
]
|
||||
)
|
||||
except subprocess.CalledProcessError:
|
||||
print("PyPy patch not applied", file=sys.stderr)
|
||||
|
||||
if options.dependency_constraints:
|
||||
constraints_file = options.dependency_constraints.get_for_python_version(
|
||||
config.version
|
||||
)
|
||||
container_constraints_file = PurePath("/constraints.txt")
|
||||
|
||||
docker.copy_into(constraints_file, container_constraints_file)
|
||||
dependency_constraint_flags = ["-c", container_constraints_file]
|
||||
|
||||
log.step("Setting up build environment...")
|
||||
|
||||
env = docker.get_environment()
|
||||
|
||||
# put this config's python top of the list
|
||||
python_bin = config.path / "bin"
|
||||
env["PATH"] = f'{python_bin}:{env["PATH"]}'
|
||||
|
||||
env = options.environment.as_dictionary(
|
||||
env, executor=docker.environment_executor
|
||||
)
|
||||
|
||||
# check config python and pip are still on PATH
|
||||
which_python = docker.call(
|
||||
["which", "python"], env=env, capture_output=True
|
||||
).strip()
|
||||
if PurePath(which_python) != python_bin / "python":
|
||||
print(
|
||||
"cibuildwheel: python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
which_pip = docker.call(["which", "pip"], env=env, capture_output=True).strip()
|
||||
if PurePath(which_pip) != python_bin / "pip":
|
||||
print(
|
||||
"cibuildwheel: pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
if options.before_build:
|
||||
log.step("Running before_build...")
|
||||
before_build_prepared = prepare_command(
|
||||
options.before_build,
|
||||
project=container_project_path,
|
||||
package=container_package_dir,
|
||||
)
|
||||
docker.call(["sh", "-c", before_build_prepared], env=env)
|
||||
|
||||
log.step("Building wheel...")
|
||||
|
||||
temp_dir = PurePath("/tmp/cibuildwheel")
|
||||
built_wheel_dir = temp_dir / "built_wheel"
|
||||
docker.call(["rm", "-rf", built_wheel_dir])
|
||||
docker.call(["mkdir", "-p", built_wheel_dir])
|
||||
|
||||
docker.call(
|
||||
[
|
||||
"pip",
|
||||
"wheel",
|
||||
container_package_dir,
|
||||
"--wheel-dir",
|
||||
built_wheel_dir,
|
||||
"--no-deps",
|
||||
*get_build_verbosity_extra_flags(options.build_verbosity),
|
||||
],
|
||||
env=env,
|
||||
)
|
||||
|
||||
built_wheel = docker.glob(built_wheel_dir, "*.whl")[0]
|
||||
|
||||
repaired_wheel_dir = temp_dir / "repaired_wheel"
|
||||
docker.call(["rm", "-rf", repaired_wheel_dir])
|
||||
docker.call(["mkdir", "-p", repaired_wheel_dir])
|
||||
|
||||
if built_wheel.name.endswith("none-any.whl"):
|
||||
raise NonPlatformWheelError()
|
||||
|
||||
if options.repair_command:
|
||||
log.step("Repairing wheel...")
|
||||
repair_command_prepared = prepare_command(
|
||||
options.repair_command, wheel=built_wheel, dest_dir=repaired_wheel_dir
|
||||
)
|
||||
docker.call(["sh", "-c", repair_command_prepared], env=env)
|
||||
else:
|
||||
docker.call(["mv", built_wheel, repaired_wheel_dir])
|
||||
|
||||
repaired_wheels = docker.glob(repaired_wheel_dir, "*.whl")
|
||||
|
||||
if options.test_command and options.test_selector(config.identifier):
|
||||
log.step("Testing wheel...")
|
||||
|
||||
# set up a virtual environment to install and test from, to make sure
|
||||
# there are no dependencies that were pulled in at build time.
|
||||
docker.call(
|
||||
["pip", "install", "virtualenv", *dependency_constraint_flags], env=env
|
||||
)
|
||||
venv_dir = (
|
||||
PurePath(docker.call(["mktemp", "-d"], capture_output=True).strip())
|
||||
/ "venv"
|
||||
)
|
||||
|
||||
docker.call(
|
||||
["python", "-m", "virtualenv", "--no-download", venv_dir], env=env
|
||||
)
|
||||
|
||||
virtualenv_env = env.copy()
|
||||
virtualenv_env["PATH"] = f"{venv_dir / 'bin'}:{virtualenv_env['PATH']}"
|
||||
|
||||
if options.before_test:
|
||||
before_test_prepared = prepare_command(
|
||||
options.before_test,
|
||||
project=container_project_path,
|
||||
package=container_package_dir,
|
||||
)
|
||||
docker.call(["sh", "-c", before_test_prepared], env=virtualenv_env)
|
||||
|
||||
# Install the wheel we just built
|
||||
# Note: If auditwheel produced two wheels, it's because the earlier produced wheel
|
||||
# conforms to multiple manylinux standards. These multiple versions of the wheel are
|
||||
# functionally the same, differing only in name, wheel metadata, and possibly include
|
||||
# different external shared libraries. so it doesn't matter which one we run the tests on.
|
||||
# Let's just pick the first one.
|
||||
wheel_to_test = repaired_wheels[0]
|
||||
docker.call(
|
||||
["pip", "install", str(wheel_to_test) + options.test_extras],
|
||||
env=virtualenv_env,
|
||||
)
|
||||
|
||||
# Install any requirements to run the tests
|
||||
if options.test_requires:
|
||||
docker.call(
|
||||
["pip", "install", *options.test_requires], env=virtualenv_env
|
||||
)
|
||||
|
||||
# Run the tests from a different directory
|
||||
test_command_prepared = prepare_command(
|
||||
options.test_command,
|
||||
project=container_project_path,
|
||||
package=container_package_dir,
|
||||
)
|
||||
docker.call(
|
||||
["sh", "-c", test_command_prepared], cwd="/root", env=virtualenv_env
|
||||
)
|
||||
|
||||
# clean up test environment
|
||||
docker.call(["rm", "-rf", venv_dir])
|
||||
|
||||
# move repaired wheels to output
|
||||
docker.call(["mkdir", "-p", container_output_dir])
|
||||
docker.call(["mv", *repaired_wheels, container_output_dir])
|
||||
|
||||
log.build_end()
|
||||
|
||||
log.step("Copying wheels back to host...")
|
||||
# copy the output back into the host
|
||||
docker.copy_out(container_output_dir, options.output_dir)
|
||||
log.step_end()
|
||||
except subprocess.CalledProcessError as error:
|
||||
log.step_end_with_error(
|
||||
f"Command {error.cmd} failed with code {error.returncode}. {error.stdout}"
|
||||
)
|
||||
troubleshoot(options.package_dir, error)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def troubleshoot(package_dir: Path, error: Exception) -> None:
|
||||
if isinstance(error, subprocess.CalledProcessError) and error.cmd[0:2] == ["pip", "wheel"]:
|
||||
# the 'pip wheel' step failed.
|
||||
print("Checking for common errors...")
|
||||
so_files = list(package_dir.glob("**/*.so"))
|
||||
|
||||
if so_files:
|
||||
print(
|
||||
textwrap.dedent(
|
||||
"""
|
||||
NOTE: Shared object (.so) files found in this project.
|
||||
|
||||
These files might be built against the wrong OS, causing problems with
|
||||
auditwheel.
|
||||
|
||||
If you're using Cython and have previously done an in-place build,
|
||||
remove those build files (*.so and *.c) before starting cibuildwheel.
|
||||
"""
|
||||
),
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
print(" Files detected:")
|
||||
print("\n".join(f" {f}" for f in so_files))
|
||||
print("")
|
||||
+80
-264
@@ -1,142 +1,41 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {
|
||||
"cibuildwheel.ci",
|
||||
"contextlib",
|
||||
"functools",
|
||||
"hashlib",
|
||||
"humanize",
|
||||
"io",
|
||||
"pathlib",
|
||||
"re",
|
||||
"textwrap",
|
||||
}
|
||||
|
||||
import codecs
|
||||
import contextlib
|
||||
import dataclasses
|
||||
import functools
|
||||
import hashlib
|
||||
import io
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import textwrap
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import IO, AnyStr, Optional, Union
|
||||
|
||||
import humanize
|
||||
from cibuildwheel.util import CIProvider, detect_ci_provider
|
||||
|
||||
from cibuildwheel.ci import CIProvider, detect_ci_provider, filter_ansi_codes
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Generator
|
||||
from typing import IO, AnyStr, Final, Literal
|
||||
|
||||
from cibuildwheel.options import Options
|
||||
|
||||
FoldPattern = tuple[str, str]
|
||||
|
||||
DEFAULT_FOLD_PATTERN: Final[FoldPattern] = ("{name}", "")
|
||||
FOLD_PATTERNS: Final[dict[str, FoldPattern]] = {
|
||||
DEFAULT_FOLD_PATTERN = ("{name}", "")
|
||||
FOLD_PATTERNS = {
|
||||
"azure": ("##[group]{name}", "##[endgroup]"),
|
||||
"travis": ("travis_fold:start:{identifier}\n{name}", "travis_fold:end:{identifier}"),
|
||||
"github": ("::group::{name}", "::endgroup::{name}"),
|
||||
}
|
||||
|
||||
PLATFORM_IDENTIFIER_DESCRIPTIONS: Final[dict[str, str]] = {
|
||||
PLATFORM_IDENTIFIER_DESCIPTIONS = {
|
||||
"manylinux_x86_64": "manylinux x86_64",
|
||||
"manylinux_i686": "manylinux i686",
|
||||
"manylinux_aarch64": "manylinux aarch64",
|
||||
"manylinux_ppc64le": "manylinux ppc64le",
|
||||
"manylinux_s390x": "manylinux s390x",
|
||||
"manylinux_armv7l": "manylinux armv7l",
|
||||
"manylinux_riscv64": "manylinux riscv64",
|
||||
"musllinux_x86_64": "musllinux x86_64",
|
||||
"musllinux_i686": "musllinux i686",
|
||||
"musllinux_aarch64": "musllinux aarch64",
|
||||
"musllinux_ppc64le": "musllinux ppc64le",
|
||||
"musllinux_s390x": "musllinux s390x",
|
||||
"musllinux_armv7l": "musllinux armv7l",
|
||||
"musllinux_riscv64": "musllinux riscv64",
|
||||
"win32": "Windows 32bit",
|
||||
"win_amd64": "Windows 64bit",
|
||||
"win_arm64": "Windows on ARM 64bit",
|
||||
"macosx_x86_64": "macOS x86_64",
|
||||
"macosx_universal2": "macOS Universal 2 - x86_64 and arm64",
|
||||
"macosx_arm64": "macOS arm64 - Apple Silicon",
|
||||
"pyodide_wasm32": "Pyodide",
|
||||
"android_arm64_v8a": "Android arm64_v8a",
|
||||
"android_x86_64": "Android x86_64",
|
||||
"ios_arm64_iphoneos": "iOS Device (ARM64)",
|
||||
"ios_arm64_iphonesimulator": "iOS Simulator (ARM64)",
|
||||
"ios_x86_64_iphonesimulator": "iOS Simulator (x86_64)",
|
||||
}
|
||||
|
||||
|
||||
class Colors:
|
||||
def __init__(self, *, enabled: bool) -> None:
|
||||
self.red = "\033[31m" if enabled else ""
|
||||
self.green = "\033[32m" if enabled else ""
|
||||
self.yellow = "\033[33m" if enabled else ""
|
||||
self.blue = "\033[34m" if enabled else ""
|
||||
self.cyan = "\033[36m" if enabled else ""
|
||||
self.bright_red = "\033[91m" if enabled else ""
|
||||
self.bright_green = "\033[92m" if enabled else ""
|
||||
self.white = "\033[37m\033[97m" if enabled else ""
|
||||
self.gray = "\033[38;5;244m" if enabled else ""
|
||||
|
||||
self.bg_grey = "\033[48;5;235m" if enabled else ""
|
||||
|
||||
self.bold = "\033[1m" if enabled else ""
|
||||
self.faint = "\033[2m" if enabled else ""
|
||||
|
||||
self.end = "\033[0m" if enabled else ""
|
||||
|
||||
|
||||
class Symbols:
|
||||
def __init__(self, *, unicode: bool) -> None:
|
||||
self.done = "✓" if unicode else "done"
|
||||
self.error = "✕" if unicode else "failed"
|
||||
|
||||
|
||||
@dataclasses.dataclass(kw_only=True, frozen=True)
|
||||
class BuildInfo:
|
||||
identifier: str
|
||||
filename: Path | None
|
||||
duration: float
|
||||
|
||||
@functools.cached_property
|
||||
def size(self) -> str | None:
|
||||
if self.filename is None:
|
||||
return None
|
||||
return humanize.naturalsize(self.filename.stat().st_size)
|
||||
|
||||
@functools.cached_property
|
||||
def sha256(self) -> str | None:
|
||||
if self.filename is None:
|
||||
return None
|
||||
with self.filename.open("rb") as f:
|
||||
digest = hashlib.file_digest(f, "sha256")
|
||||
return digest.hexdigest()
|
||||
|
||||
def __str__(self) -> str:
|
||||
duration = humanize.naturaldelta(self.duration)
|
||||
if self.filename:
|
||||
return f"{self.identifier}: {self.filename.name} {self.size} in {duration}, SHA256={self.sha256}"
|
||||
return f"{self.identifier}: {duration} (test only)"
|
||||
|
||||
|
||||
class Logger:
|
||||
fold_mode: Literal["azure", "github", "travis", "disabled"]
|
||||
fold_mode: str
|
||||
colors_enabled: bool
|
||||
unicode_enabled: bool
|
||||
active_build_identifier: str | None = None
|
||||
build_start_time: float | None = None
|
||||
step_start_time: float | None = None
|
||||
active_fold_group_name: str | None = None
|
||||
summary: list[BuildInfo]
|
||||
active_build_identifier: Optional[str] = None
|
||||
build_start_time: Optional[float] = None
|
||||
step_start_time: Optional[float] = None
|
||||
active_fold_group_name: Optional[str] = None
|
||||
|
||||
def __init__(self) -> None:
|
||||
if sys.platform == "win32" and hasattr(sys.stdout, "reconfigure"):
|
||||
@@ -148,28 +47,25 @@ class Logger:
|
||||
|
||||
ci_provider = detect_ci_provider()
|
||||
|
||||
match ci_provider:
|
||||
case CIProvider.azure_pipelines:
|
||||
self.fold_mode = "azure"
|
||||
self.colors_enabled = True
|
||||
if ci_provider == CIProvider.azure_pipelines:
|
||||
self.fold_mode = "azure"
|
||||
self.colors_enabled = True
|
||||
|
||||
case CIProvider.github_actions:
|
||||
self.fold_mode = "github"
|
||||
self.colors_enabled = True
|
||||
elif ci_provider == CIProvider.github_actions:
|
||||
self.fold_mode = "github"
|
||||
self.colors_enabled = True
|
||||
|
||||
case CIProvider.travis_ci:
|
||||
self.fold_mode = "travis"
|
||||
self.colors_enabled = True
|
||||
elif ci_provider == CIProvider.travis_ci:
|
||||
self.fold_mode = "travis"
|
||||
self.colors_enabled = True
|
||||
|
||||
case CIProvider.appveyor:
|
||||
self.fold_mode = "disabled"
|
||||
self.colors_enabled = True
|
||||
elif ci_provider == CIProvider.appveyor:
|
||||
self.fold_mode = "disabled"
|
||||
self.colors_enabled = True
|
||||
|
||||
case _:
|
||||
self.fold_mode = "disabled"
|
||||
self.colors_enabled = file_supports_color(sys.stdout)
|
||||
|
||||
self.summary = []
|
||||
else:
|
||||
self.fold_mode = "disabled"
|
||||
self.colors_enabled = file_supports_color(sys.stdout)
|
||||
|
||||
def build_start(self, identifier: str) -> None:
|
||||
self.step_end()
|
||||
@@ -183,7 +79,7 @@ class Logger:
|
||||
self.build_start_time = time.time()
|
||||
self.active_build_identifier = identifier
|
||||
|
||||
def build_end(self, filename: Path | None) -> None:
|
||||
def build_end(self) -> None:
|
||||
assert self.build_start_time is not None
|
||||
assert self.active_build_identifier is not None
|
||||
self.step_end()
|
||||
@@ -191,14 +87,11 @@ class Logger:
|
||||
c = self.colors
|
||||
s = self.symbols
|
||||
duration = time.time() - self.build_start_time
|
||||
duration_str = humanize.naturaldelta(duration, minimum_unit="milliseconds")
|
||||
|
||||
print()
|
||||
print(f"{c.green}{s.done} {c.end}{self.active_build_identifier} finished in {duration_str}")
|
||||
self.summary.append(
|
||||
BuildInfo(identifier=self.active_build_identifier, filename=filename, duration=duration)
|
||||
print(
|
||||
f"{c.green}{s.done} {c.end}{self.active_build_identifier} finished in {duration:.2f}s"
|
||||
)
|
||||
|
||||
self.build_start_time = None
|
||||
self.active_build_identifier = None
|
||||
|
||||
@@ -213,7 +106,6 @@ class Logger:
|
||||
c = self.colors
|
||||
s = self.symbols
|
||||
duration = time.time() - self.step_start_time
|
||||
|
||||
if success:
|
||||
print(f"{c.green}{s.done} {c.end}{duration:.2f}s".rjust(78))
|
||||
else:
|
||||
@@ -221,58 +113,23 @@ class Logger:
|
||||
|
||||
self.step_start_time = None
|
||||
|
||||
def step_end_with_error(self, error: BaseException | str) -> None:
|
||||
def step_end_with_error(self, error: Union[BaseException, str]) -> None:
|
||||
self.step_end(success=False)
|
||||
self.error(error)
|
||||
|
||||
def quiet(self, message: str) -> None:
|
||||
c = self.colors
|
||||
print(f"{c.gray}{message}{c.end}", file=sys.stderr)
|
||||
|
||||
def notice(self, message: str) -> None:
|
||||
if self.fold_mode == "github":
|
||||
print(f"::notice::cibuildwheel: {message}\n", file=sys.stderr)
|
||||
else:
|
||||
c = self.colors
|
||||
print(f"cibuildwheel: {c.bold}note{c.end}: {message}\n", file=sys.stderr)
|
||||
|
||||
def warning(self, message: str) -> None:
|
||||
if self.fold_mode == "github":
|
||||
print(f"::warning::cibuildwheel: {message}\n", file=sys.stderr)
|
||||
print(f"::warning::{message}\n", file=sys.stderr)
|
||||
else:
|
||||
c = self.colors
|
||||
print(f"cibuildwheel: {c.yellow}warning{c.end}: {message}\n", file=sys.stderr)
|
||||
print(f"{c.yellow}Warning{c.end}: {message}\n", file=sys.stderr)
|
||||
|
||||
def error(self, error: BaseException | str) -> None:
|
||||
def error(self, error: Union[BaseException, str]) -> None:
|
||||
if self.fold_mode == "github":
|
||||
print(f"::error::cibuildwheel: {error}\n", file=sys.stderr)
|
||||
print(f"::error::{error}\n", file=sys.stderr)
|
||||
else:
|
||||
c = self.colors
|
||||
print(f"cibuildwheel: {c.bright_red}error{c.end}: {error}\n", file=sys.stderr)
|
||||
|
||||
@contextlib.contextmanager
|
||||
def print_summary(self, *, options: Options) -> Generator[None, None, None]:
|
||||
start = time.time()
|
||||
yield
|
||||
duration = time.time() - start
|
||||
if summary_path := os.environ.get("GITHUB_STEP_SUMMARY"):
|
||||
github_summary = self._github_step_summary(duration=duration, options=options)
|
||||
Path(summary_path).write_text(filter_ansi_codes(github_summary), encoding="utf-8")
|
||||
|
||||
n_wheels = len([info for info in self.summary if info.filename])
|
||||
s = "s" if n_wheels > 1 else ""
|
||||
duration_str = humanize.naturaldelta(duration)
|
||||
print()
|
||||
self._start_fold_group(f"{n_wheels} wheel{s} produced in {duration_str}")
|
||||
for build_info in self.summary:
|
||||
print(" ", build_info)
|
||||
self._end_fold_group()
|
||||
|
||||
self.summary = []
|
||||
|
||||
@property
|
||||
def step_active(self) -> bool:
|
||||
return self.step_start_time is not None
|
||||
print(f"{c.bright_red}Error{c.end}: {error}\n", file=sys.stderr)
|
||||
|
||||
def _start_fold_group(self, name: str) -> None:
|
||||
self._end_fold_group()
|
||||
@@ -294,8 +151,7 @@ class Logger:
|
||||
sys.stdout.flush()
|
||||
self.active_fold_group_name = None
|
||||
|
||||
@staticmethod
|
||||
def _fold_group_identifier(name: str) -> str:
|
||||
def _fold_group_identifier(self, name: str) -> str:
|
||||
"""
|
||||
Travis doesn't like fold groups identifiers that have spaces in. This
|
||||
method converts them to ascii identifiers
|
||||
@@ -309,79 +165,19 @@ class Logger:
|
||||
# lowercase, shorten
|
||||
return identifier.lower()[:20]
|
||||
|
||||
def _github_step_summary(self, duration: float, options: Options) -> str:
|
||||
"""
|
||||
Returns the GitHub step summary, in markdown format.
|
||||
"""
|
||||
out = io.StringIO()
|
||||
options_summary = options.summary(
|
||||
identifiers=[bi.identifier for bi in self.summary], skip_unset=True
|
||||
)
|
||||
out.write(
|
||||
textwrap.dedent("""\
|
||||
### 🎡 cibuildwheel
|
||||
@property
|
||||
def colors(self) -> "Colors":
|
||||
if self.colors_enabled:
|
||||
return Colors(enabled=True)
|
||||
else:
|
||||
return Colors(enabled=False)
|
||||
|
||||
<details>
|
||||
<summary>
|
||||
Build options
|
||||
</summary>
|
||||
|
||||
```yaml
|
||||
{options_summary}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
""").format(options_summary=options_summary)
|
||||
)
|
||||
n_wheels = len([b for b in self.summary if b.filename])
|
||||
wheel_rows = "\n".join(
|
||||
"<tr>"
|
||||
f"<td nowrap>{'<samp>' + b.filename.name + '</samp>' if b.filename else '*Test only*'}</td>"
|
||||
f"<td nowrap>{b.size or 'N/A'}</td>"
|
||||
f"<td nowrap><samp>{b.identifier}</samp></td>"
|
||||
f"<td nowrap>{humanize.naturaldelta(b.duration)}</td>"
|
||||
f"<td nowrap><samp>{b.sha256 or 'N/A'}</samp></td>"
|
||||
"</tr>"
|
||||
for b in self.summary
|
||||
)
|
||||
out.write(
|
||||
textwrap.dedent("""\
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th align="left">Wheel</th>
|
||||
<th align="left">Size</th>
|
||||
<th align="left">Build identifier</th>
|
||||
<th align="left">Time</th>
|
||||
<th align="left">SHA256</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{wheel_rows}
|
||||
</tbody>
|
||||
</table>
|
||||
<div align="right"><sup>{n} wheel{s} created in {duration_str}</sup></div>
|
||||
""").format(
|
||||
wheel_rows=wheel_rows,
|
||||
n=n_wheels,
|
||||
duration_str=humanize.naturaldelta(duration),
|
||||
s="s" if n_wheels > 1 else "",
|
||||
)
|
||||
)
|
||||
|
||||
out.write("\n")
|
||||
out.write("---")
|
||||
out.write("\n")
|
||||
return out.getvalue()
|
||||
|
||||
@functools.cached_property
|
||||
def colors(self) -> Colors:
|
||||
return Colors(enabled=self.colors_enabled)
|
||||
|
||||
@functools.cached_property
|
||||
def symbols(self) -> Symbols:
|
||||
return Symbols(unicode=self.unicode_enabled)
|
||||
@property
|
||||
def symbols(self) -> "Symbols":
|
||||
if self.unicode_enabled:
|
||||
return Symbols(unicode=True)
|
||||
else:
|
||||
return Symbols(unicode=False)
|
||||
|
||||
|
||||
def build_description_from_identifier(identifier: str) -> str:
|
||||
@@ -390,32 +186,50 @@ def build_description_from_identifier(identifier: str) -> str:
|
||||
build_description = ""
|
||||
|
||||
python_interpreter = python_identifier[0:2]
|
||||
version_parts = python_identifier[2:].split("_")
|
||||
python_version = version_parts[0]
|
||||
python_version = python_identifier[2:4]
|
||||
|
||||
if python_interpreter == "cp":
|
||||
build_description += "CPython"
|
||||
elif python_interpreter == "pp":
|
||||
build_description += "PyPy"
|
||||
elif python_interpreter == "gp":
|
||||
build_description += "GraalPy"
|
||||
else:
|
||||
msg = f"unknown python {python_interpreter!r}"
|
||||
raise Exception(msg)
|
||||
raise Exception("unknown python")
|
||||
|
||||
build_description += f" {python_version[0]}.{python_version[1:]} "
|
||||
if len(version_parts) > 1:
|
||||
build_description += f"(ABI {version_parts[1]}) "
|
||||
build_description += f" {python_version[0]}.{python_version[1]} "
|
||||
|
||||
try:
|
||||
build_description += PLATFORM_IDENTIFIER_DESCRIPTIONS[platform_identifier]
|
||||
build_description += PLATFORM_IDENTIFIER_DESCIPTIONS[platform_identifier]
|
||||
except KeyError as e:
|
||||
msg = f"unknown platform {platform_identifier!r}"
|
||||
raise Exception(msg) from e
|
||||
raise Exception("unknown platform") from e
|
||||
|
||||
return build_description
|
||||
|
||||
|
||||
class Colors:
|
||||
def __init__(self, *, enabled: bool) -> None:
|
||||
self.red = "\033[31m" if enabled else ""
|
||||
self.green = "\033[32m" if enabled else ""
|
||||
self.yellow = "\033[33m" if enabled else ""
|
||||
self.blue = "\033[34m" if enabled else ""
|
||||
self.cyan = "\033[36m" if enabled else ""
|
||||
self.bright_red = "\033[91m" if enabled else ""
|
||||
self.bright_green = "\033[92m" if enabled else ""
|
||||
self.white = "\033[37m\033[97m" if enabled else ""
|
||||
|
||||
self.bg_grey = "\033[48;5;235m" if enabled else ""
|
||||
|
||||
self.bold = "\033[1m" if enabled else ""
|
||||
self.faint = "\033[2m" if enabled else ""
|
||||
|
||||
self.end = "\033[0m" if enabled else ""
|
||||
|
||||
|
||||
class Symbols:
|
||||
def __init__(self, *, unicode: bool) -> None:
|
||||
self.done = "✓" if unicode else "done"
|
||||
self.error = "✕" if unicode else "failed"
|
||||
|
||||
|
||||
def file_supports_color(file_obj: IO[AnyStr]) -> bool:
|
||||
"""
|
||||
Returns True if the running system's terminal supports color.
|
||||
@@ -442,6 +256,8 @@ def file_supports_unicode(file_obj: IO[AnyStr]) -> bool:
|
||||
return "utf" in codec_info.name
|
||||
|
||||
|
||||
# Global instance of the Logger.
|
||||
"""
|
||||
Global instance of the Logger.
|
||||
"""
|
||||
# (there's only one stdout per-process, so a global instance is justified)
|
||||
log = Logger()
|
||||
|
||||
@@ -0,0 +1,584 @@
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import shlex
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, NamedTuple, Optional, Sequence, Set, Tuple, cast
|
||||
|
||||
from .architecture import Architecture
|
||||
from .environment import ParsedEnvironment
|
||||
from .logger import log
|
||||
from .typing import Literal, PathOrStr
|
||||
from .util import (
|
||||
BuildOptions,
|
||||
BuildSelector,
|
||||
NonPlatformWheelError,
|
||||
download,
|
||||
get_build_verbosity_extra_flags,
|
||||
get_pip_script,
|
||||
install_certifi_script,
|
||||
prepare_command,
|
||||
read_python_configs,
|
||||
resources_dir,
|
||||
unwrap,
|
||||
)
|
||||
|
||||
|
||||
def call(
|
||||
args: Sequence[PathOrStr],
|
||||
env: Optional[Dict[str, str]] = None,
|
||||
cwd: Optional[str] = None,
|
||||
shell: bool = False,
|
||||
) -> None:
|
||||
# print the command executing for the logs
|
||||
if shell:
|
||||
print(f"+ {args}")
|
||||
else:
|
||||
print("+ " + " ".join(shlex.quote(str(a)) for a in args))
|
||||
|
||||
subprocess.run(args, env=env, cwd=cwd, shell=shell, check=True)
|
||||
|
||||
|
||||
def get_macos_version() -> Tuple[int, int]:
|
||||
"""
|
||||
Returns the macOS major/minor version, as a tuple, e.g. (10, 15) or (11, 0)
|
||||
|
||||
These tuples can be used in comparisons, e.g.
|
||||
(10, 14) <= (11, 0) == True
|
||||
(10, 14) <= (10, 16) == True
|
||||
(11, 2) <= (11, 0) != True
|
||||
"""
|
||||
version_str, _, _ = platform.mac_ver()
|
||||
version = tuple(map(int, version_str.split(".")[:2]))
|
||||
return cast(Tuple[int, int], version)
|
||||
|
||||
|
||||
def get_macos_sdks() -> List[str]:
|
||||
output = subprocess.run(
|
||||
["xcodebuild", "-showsdks"],
|
||||
universal_newlines=True,
|
||||
check=True,
|
||||
stdout=subprocess.PIPE,
|
||||
).stdout
|
||||
|
||||
return [m.group(1) for m in re.finditer(r"-sdk (macosx\S+)", output)]
|
||||
|
||||
|
||||
class PythonConfiguration(NamedTuple):
|
||||
version: str
|
||||
identifier: str
|
||||
url: str
|
||||
|
||||
|
||||
def get_python_configurations(
|
||||
build_selector: BuildSelector, architectures: Set[Architecture]
|
||||
) -> List[PythonConfiguration]:
|
||||
|
||||
full_python_configs = read_python_configs("macos")
|
||||
|
||||
python_configurations = [PythonConfiguration(**item) for item in full_python_configs]
|
||||
|
||||
# filter out configs that don't match any of the selected architectures
|
||||
python_configurations = [
|
||||
c
|
||||
for c in python_configurations
|
||||
if any(c.identifier.endswith(a.value) for a in architectures)
|
||||
]
|
||||
|
||||
# skip builds as required by BUILD/SKIP
|
||||
python_configurations = [c for c in python_configurations if build_selector(c.identifier)]
|
||||
|
||||
# When running on macOS 11 and x86_64, the reported OS is '10.16', but
|
||||
# there is no such OS - it really means macOS 11.
|
||||
if get_macos_version() >= (10, 16):
|
||||
if any(c.identifier.startswith("pp") for c in python_configurations):
|
||||
# pypy doesn't work on macOS 11 yet
|
||||
# See https://foss.heptapod.net/pypy/pypy/-/issues/3314
|
||||
log.warning(
|
||||
unwrap(
|
||||
"""
|
||||
PyPy is currently unsupported when building on macOS 11. To build macOS PyPy
|
||||
wheels, build on an older OS, such as macOS 10.15. To silence this warning,
|
||||
deselect PyPy by adding "pp*-macosx*" to your CIBW_SKIP option.
|
||||
"""
|
||||
)
|
||||
)
|
||||
python_configurations = [
|
||||
c for c in python_configurations if not c.identifier.startswith("pp")
|
||||
]
|
||||
|
||||
if any(c.identifier.startswith("cp35") for c in python_configurations):
|
||||
# CPython 3.5 doesn't work on macOS 11
|
||||
log.warning(
|
||||
unwrap(
|
||||
"""
|
||||
CPython 3.5 is unsupported when building on macOS 11. To build CPython 3.5
|
||||
wheels, build on an older OS, such as macOS 10.15. To silence this warning,
|
||||
deselect CPython 3.5 by adding "cp35-macosx_x86_64" to your CIBW_SKIP option.
|
||||
"""
|
||||
)
|
||||
)
|
||||
python_configurations = [
|
||||
c for c in python_configurations if not c.identifier.startswith("cp35")
|
||||
]
|
||||
|
||||
return python_configurations
|
||||
|
||||
|
||||
SYMLINKS_DIR = Path("/tmp/cibw_bin")
|
||||
|
||||
|
||||
def make_symlinks(installation_bin_path: Path, python_executable: str, pip_executable: str) -> None:
|
||||
assert (installation_bin_path / python_executable).exists()
|
||||
|
||||
# Python bin folders on Mac don't symlink `python3` to `python`, and neither
|
||||
# does PyPy for `pypy` or `pypy3`, so we do that so `python` and `pip` always
|
||||
# point to the active configuration.
|
||||
if SYMLINKS_DIR.exists():
|
||||
shutil.rmtree(SYMLINKS_DIR)
|
||||
SYMLINKS_DIR.mkdir(parents=True)
|
||||
|
||||
(SYMLINKS_DIR / "python").symlink_to(installation_bin_path / python_executable)
|
||||
(SYMLINKS_DIR / "python-config").symlink_to(
|
||||
installation_bin_path / (python_executable + "-config")
|
||||
)
|
||||
(SYMLINKS_DIR / "pip").symlink_to(installation_bin_path / pip_executable)
|
||||
|
||||
|
||||
def install_cpython(version: str, url: str) -> Path:
|
||||
installed_system_packages = subprocess.run(
|
||||
["pkgutil", "--pkgs"], universal_newlines=True, check=True, stdout=subprocess.PIPE
|
||||
).stdout.splitlines()
|
||||
|
||||
# if this version of python isn't installed, get it from python.org and install
|
||||
python_package_identifier = f"org.python.Python.PythonFramework-{version}"
|
||||
python_executable = "python3" if version[0] == "3" else "python"
|
||||
installation_bin_path = Path(f"/Library/Frameworks/Python.framework/Versions/{version}/bin")
|
||||
|
||||
if python_package_identifier not in installed_system_packages:
|
||||
# download the pkg
|
||||
download(url, Path("/tmp/Python.pkg"))
|
||||
# install
|
||||
call(["sudo", "installer", "-pkg", "/tmp/Python.pkg", "-target", "/"])
|
||||
# patch open ssl
|
||||
if version == "3.5":
|
||||
open_ssl_patch_url = f"https://github.com/mayeut/patch-macos-python-openssl/releases/download/v1.1.1h/patch-macos-python-{version}-openssl-v1.1.1h.tar.gz"
|
||||
download(open_ssl_patch_url, Path("/tmp/python-patch.tar.gz"))
|
||||
call(
|
||||
[
|
||||
"sudo",
|
||||
"tar",
|
||||
"-C",
|
||||
f"/Library/Frameworks/Python.framework/Versions/{version}/",
|
||||
"-xmf",
|
||||
"/tmp/python-patch.tar.gz",
|
||||
]
|
||||
)
|
||||
|
||||
call(["sudo", str(installation_bin_path / python_executable), str(install_certifi_script)])
|
||||
|
||||
pip_executable = "pip3" if version[0] == "3" else "pip"
|
||||
make_symlinks(installation_bin_path, python_executable, pip_executable)
|
||||
|
||||
return installation_bin_path
|
||||
|
||||
|
||||
def install_pypy(version: str, url: str) -> Path:
|
||||
pypy_tar_bz2 = url.rsplit("/", 1)[-1]
|
||||
extension = ".tar.bz2"
|
||||
assert pypy_tar_bz2.endswith(extension)
|
||||
pypy_base_filename = pypy_tar_bz2[: -len(extension)]
|
||||
installation_path = Path("/tmp") / pypy_base_filename
|
||||
if not installation_path.exists():
|
||||
downloaded_tar_bz2 = Path("/tmp") / pypy_tar_bz2
|
||||
download(url, downloaded_tar_bz2)
|
||||
call(["tar", "-C", "/tmp", "-xf", downloaded_tar_bz2])
|
||||
# Patch PyPy to make sure headers get installed into a venv
|
||||
patch_version = "_27" if version == "2.7" else ""
|
||||
patch_path = resources_dir / f"pypy_venv{patch_version}.patch"
|
||||
call(["patch", "--force", "-p1", "-d", installation_path, "-i", patch_path])
|
||||
|
||||
installation_bin_path = installation_path / "bin"
|
||||
python_executable = "pypy3" if version[0] == "3" else "pypy"
|
||||
pip_executable = "pip3" if version[0] == "3" else "pip"
|
||||
make_symlinks(installation_bin_path, python_executable, pip_executable)
|
||||
|
||||
return installation_bin_path
|
||||
|
||||
|
||||
def setup_python(
|
||||
python_configuration: PythonConfiguration,
|
||||
dependency_constraint_flags: Sequence[PathOrStr],
|
||||
environment: ParsedEnvironment,
|
||||
) -> Dict[str, str]:
|
||||
implementation_id = python_configuration.identifier.split("-")[0]
|
||||
log.step(f"Installing Python {implementation_id}...")
|
||||
|
||||
if implementation_id.startswith("cp"):
|
||||
installation_bin_path = install_cpython(
|
||||
python_configuration.version, python_configuration.url
|
||||
)
|
||||
elif implementation_id.startswith("pp"):
|
||||
installation_bin_path = install_pypy(python_configuration.version, python_configuration.url)
|
||||
else:
|
||||
raise ValueError("Unknown Python implementation")
|
||||
|
||||
log.step("Setting up build environment...")
|
||||
|
||||
env = os.environ.copy()
|
||||
env["PATH"] = os.pathsep.join(
|
||||
[
|
||||
str(SYMLINKS_DIR),
|
||||
str(installation_bin_path),
|
||||
env["PATH"],
|
||||
]
|
||||
)
|
||||
|
||||
# Fix issue with site.py setting the wrong `sys.prefix`, `sys.exec_prefix`,
|
||||
# `sys.path`, ... for PyPy: https://foss.heptapod.net/pypy/pypy/issues/3175
|
||||
# Also fix an issue with the shebang of installed scripts inside the
|
||||
# testing virtualenv- see https://github.com/theacodes/nox/issues/44 and
|
||||
# https://github.com/pypa/virtualenv/issues/620
|
||||
# Also see https://github.com/python/cpython/pull/9516
|
||||
env.pop("__PYVENV_LAUNCHER__", None)
|
||||
env = environment.as_dictionary(prev_environment=env)
|
||||
|
||||
# we version pip ourselves, so we don't care about pip version checking
|
||||
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
|
||||
|
||||
# check what version we're on
|
||||
call(["which", "python"], env=env)
|
||||
call(["python", "--version"], env=env)
|
||||
which_python = subprocess.run(
|
||||
["which", "python"], env=env, universal_newlines=True, check=True, stdout=subprocess.PIPE
|
||||
).stdout.strip()
|
||||
if which_python != "/tmp/cibw_bin/python":
|
||||
print(
|
||||
"cibuildwheel: python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
# install pip & wheel
|
||||
call(["python", get_pip_script, *dependency_constraint_flags], env=env, cwd="/tmp")
|
||||
assert (installation_bin_path / "pip").exists()
|
||||
call(["which", "pip"], env=env)
|
||||
call(["pip", "--version"], env=env)
|
||||
which_pip = subprocess.run(
|
||||
["which", "pip"], env=env, universal_newlines=True, check=True, stdout=subprocess.PIPE
|
||||
).stdout.strip()
|
||||
if which_pip != "/tmp/cibw_bin/pip":
|
||||
print(
|
||||
"cibuildwheel: pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
# Set MACOSX_DEPLOYMENT_TARGET to 10.9, if the user didn't set it.
|
||||
# CPython 3.5 defaults to 10.6, and pypy defaults to 10.7, causing
|
||||
# inconsistencies if it's left unset.
|
||||
env.setdefault("MACOSX_DEPLOYMENT_TARGET", "10.9")
|
||||
|
||||
config_is_arm64 = python_configuration.identifier.endswith("arm64")
|
||||
config_is_universal2 = python_configuration.identifier.endswith("universal2")
|
||||
|
||||
if python_configuration.version == "3.5":
|
||||
# Cross-compilation platform override - CPython 3.5 has an
|
||||
# i386/x86_64 version of Python, but we only want a x64_64 build
|
||||
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-10.9-x86_64")
|
||||
# https://github.com/python/cpython/blob/a5ed2fe0eedefa1649aa93ee74a0bafc8e628a10/Lib/_osx_support.py#L260
|
||||
env.setdefault("ARCHFLAGS", "-arch x86_64")
|
||||
|
||||
if python_configuration.version == "3.9":
|
||||
if python_configuration.identifier.endswith("x86_64"):
|
||||
# even on the macos11.0 Python installer, on the x86_64 side it's
|
||||
# compatible back to 10.9.
|
||||
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-10.9-x86_64")
|
||||
env.setdefault("ARCHFLAGS", "-arch x86_64")
|
||||
elif config_is_arm64:
|
||||
# macOS 11 is the first OS with arm64 support, so the wheels
|
||||
# have that as a minimum.
|
||||
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-11.0-arm64")
|
||||
env.setdefault("ARCHFLAGS", "-arch arm64")
|
||||
elif config_is_universal2:
|
||||
if get_macos_version() < (10, 16):
|
||||
# we can do universal2 builds on macos 10.15, but we need to
|
||||
# set ARCHFLAGS otherwise CPython sets it to `-arch x86_64`
|
||||
env.setdefault("ARCHFLAGS", "-arch arm64 -arch x86_64")
|
||||
|
||||
building_arm64 = config_is_arm64 or config_is_universal2
|
||||
if building_arm64 and get_macos_version() < (10, 16) and "SDKROOT" not in env:
|
||||
# xcode 12.2 or higher can build arm64 on macos 10.15 or below, but
|
||||
# needs the correct SDK selected.
|
||||
sdks = get_macos_sdks()
|
||||
|
||||
# Different versions of Xcode contain different SDK versions...
|
||||
# we're happy with anything newer than macOS 11.0
|
||||
arm64_compatible_sdks = [s for s in sdks if not s.startswith("macosx10.")]
|
||||
|
||||
if not arm64_compatible_sdks:
|
||||
log.warning(
|
||||
unwrap(
|
||||
"""
|
||||
SDK for building arm64-compatible wheels not found. You need Xcode 12.2 or later
|
||||
to build universal2 or arm64 wheels.
|
||||
"""
|
||||
)
|
||||
)
|
||||
else:
|
||||
env.setdefault("SDKROOT", arm64_compatible_sdks[0])
|
||||
|
||||
log.step("Installing build tools...")
|
||||
call(
|
||||
[
|
||||
"pip",
|
||||
"install",
|
||||
"--upgrade",
|
||||
"setuptools",
|
||||
"wheel",
|
||||
"delocate",
|
||||
*dependency_constraint_flags,
|
||||
],
|
||||
env=env,
|
||||
)
|
||||
|
||||
return env
|
||||
|
||||
|
||||
def build(options: BuildOptions) -> None:
|
||||
temp_dir = Path(tempfile.mkdtemp(prefix="cibuildwheel"))
|
||||
built_wheel_dir = temp_dir / "built_wheel"
|
||||
repaired_wheel_dir = temp_dir / "repaired_wheel"
|
||||
|
||||
try:
|
||||
if options.before_all:
|
||||
log.step("Running before_all...")
|
||||
env = options.environment.as_dictionary(prev_environment=os.environ)
|
||||
env.setdefault("MACOSX_DEPLOYMENT_TARGET", "10.9")
|
||||
before_all_prepared = prepare_command(
|
||||
options.before_all, project=".", package=options.package_dir
|
||||
)
|
||||
call([before_all_prepared], shell=True, env=env)
|
||||
|
||||
python_configurations = get_python_configurations(
|
||||
options.build_selector, options.architectures
|
||||
)
|
||||
|
||||
for config in python_configurations:
|
||||
log.build_start(config.identifier)
|
||||
|
||||
config_is_arm64 = config.identifier.endswith("arm64")
|
||||
config_is_universal2 = config.identifier.endswith("universal2")
|
||||
|
||||
dependency_constraint_flags: Sequence[PathOrStr] = []
|
||||
if options.dependency_constraints:
|
||||
dependency_constraint_flags = [
|
||||
"-c",
|
||||
options.dependency_constraints.get_for_python_version(config.version),
|
||||
]
|
||||
|
||||
env = setup_python(config, dependency_constraint_flags, options.environment)
|
||||
|
||||
if options.before_build:
|
||||
log.step("Running before_build...")
|
||||
before_build_prepared = prepare_command(
|
||||
options.before_build, project=".", package=options.package_dir
|
||||
)
|
||||
call(before_build_prepared, env=env, shell=True)
|
||||
|
||||
log.step("Building wheel...")
|
||||
if built_wheel_dir.exists():
|
||||
shutil.rmtree(built_wheel_dir)
|
||||
built_wheel_dir.mkdir(parents=True)
|
||||
|
||||
# Path.resolve() is needed. Without it pip wheel may try to fetch package from pypi.org
|
||||
# see https://github.com/pypa/cibuildwheel/pull/369
|
||||
call(
|
||||
[
|
||||
"pip",
|
||||
"wheel",
|
||||
options.package_dir.resolve(),
|
||||
"--wheel-dir",
|
||||
built_wheel_dir,
|
||||
"--no-deps",
|
||||
*get_build_verbosity_extra_flags(options.build_verbosity),
|
||||
],
|
||||
env=env,
|
||||
)
|
||||
|
||||
built_wheel = next(built_wheel_dir.glob("*.whl"))
|
||||
|
||||
if repaired_wheel_dir.exists():
|
||||
shutil.rmtree(repaired_wheel_dir)
|
||||
repaired_wheel_dir.mkdir(parents=True)
|
||||
|
||||
if built_wheel.name.endswith("none-any.whl"):
|
||||
raise NonPlatformWheelError()
|
||||
|
||||
if options.repair_command:
|
||||
log.step("Repairing wheel...")
|
||||
|
||||
if config_is_universal2:
|
||||
delocate_archs = "x86_64,arm64"
|
||||
elif config_is_arm64:
|
||||
delocate_archs = "arm64"
|
||||
else:
|
||||
delocate_archs = "x86_64"
|
||||
|
||||
repair_command_prepared = prepare_command(
|
||||
options.repair_command,
|
||||
wheel=built_wheel,
|
||||
dest_dir=repaired_wheel_dir,
|
||||
delocate_archs=delocate_archs,
|
||||
)
|
||||
call(repair_command_prepared, env=env, shell=True)
|
||||
else:
|
||||
shutil.move(str(built_wheel), repaired_wheel_dir)
|
||||
|
||||
repaired_wheel = next(repaired_wheel_dir.glob("*.whl"))
|
||||
|
||||
log.step_end()
|
||||
|
||||
if options.test_command and options.test_selector(config.identifier):
|
||||
machine_arch = platform.machine()
|
||||
testing_archs: List[Literal["x86_64", "arm64"]] = []
|
||||
|
||||
if config_is_arm64:
|
||||
testing_archs = ["arm64"]
|
||||
elif config_is_universal2:
|
||||
testing_archs = ["x86_64", "arm64"]
|
||||
else:
|
||||
testing_archs = ["x86_64"]
|
||||
|
||||
for testing_arch in testing_archs:
|
||||
if config_is_universal2:
|
||||
arch_specific_identifier = f"{config.identifier}:{testing_arch}"
|
||||
if not options.test_selector(arch_specific_identifier):
|
||||
continue
|
||||
|
||||
if machine_arch == "x86_64" and testing_arch == "arm64":
|
||||
if config_is_arm64:
|
||||
log.warning(
|
||||
unwrap(
|
||||
"""
|
||||
While arm64 wheels can be built on x86_64, they cannot be
|
||||
tested. The ability to test the arm64 wheels will be added in a
|
||||
future release of cibuildwheel, once Apple Silicon CI runners
|
||||
are widely available. To silence this warning, set
|
||||
`CIBW_TEST_SKIP: *-macosx_arm64`.
|
||||
"""
|
||||
)
|
||||
)
|
||||
elif config_is_universal2:
|
||||
log.warning(
|
||||
unwrap(
|
||||
"""
|
||||
While universal2 wheels can be built on x86_64, the arm64 part
|
||||
of them cannot currently be tested. The ability to test the
|
||||
arm64 part of a universal2 wheel will be added in a future
|
||||
release of cibuildwheel, once Apple Silicon CI runners are
|
||||
widely available. To silence this warning, set
|
||||
`CIBW_TEST_SKIP: *-macosx_universal2:arm64`.
|
||||
"""
|
||||
)
|
||||
)
|
||||
else:
|
||||
raise RuntimeError("unreachable")
|
||||
|
||||
# skip this test
|
||||
continue
|
||||
|
||||
log.step(
|
||||
"Testing wheel..."
|
||||
if testing_arch == machine_arch
|
||||
else f"Testing wheel on {testing_arch}..."
|
||||
)
|
||||
|
||||
# set up a virtual environment to install and test from, to make sure
|
||||
# there are no dependencies that were pulled in at build time.
|
||||
call(["pip", "install", "virtualenv", *dependency_constraint_flags], env=env)
|
||||
venv_dir = Path(tempfile.mkdtemp())
|
||||
|
||||
arch_prefix = []
|
||||
if testing_arch != machine_arch:
|
||||
if machine_arch == "arm64" and testing_arch == "x86_64":
|
||||
# rosetta2 will provide the emulation with just the arch prefix.
|
||||
arch_prefix = ["arch", "-x86_64"]
|
||||
else:
|
||||
raise RuntimeError(
|
||||
"don't know how to emulate {testing_arch} on {machine_arch}"
|
||||
)
|
||||
|
||||
# define a custom 'call' function that adds the arch prefix each time
|
||||
def call_with_arch(args: Sequence[PathOrStr], **kwargs: Any) -> None:
|
||||
if isinstance(args, str):
|
||||
args = " ".join(arch_prefix) + " " + args
|
||||
else:
|
||||
args = [*arch_prefix, *args]
|
||||
call(args, **kwargs)
|
||||
|
||||
# Use --no-download to ensure determinism by using seed libraries
|
||||
# built into virtualenv
|
||||
call_with_arch(
|
||||
["python", "-m", "virtualenv", "--no-download", venv_dir], env=env
|
||||
)
|
||||
|
||||
virtualenv_env = env.copy()
|
||||
virtualenv_env["PATH"] = os.pathsep.join(
|
||||
[
|
||||
str(venv_dir / "bin"),
|
||||
virtualenv_env["PATH"],
|
||||
]
|
||||
)
|
||||
|
||||
# check that we are using the Python from the virtual environment
|
||||
call_with_arch(["which", "python"], env=virtualenv_env)
|
||||
|
||||
if options.before_test:
|
||||
before_test_prepared = prepare_command(
|
||||
options.before_test, project=".", package=options.package_dir
|
||||
)
|
||||
call_with_arch(before_test_prepared, env=virtualenv_env, shell=True)
|
||||
|
||||
# install the wheel
|
||||
call_with_arch(
|
||||
["pip", "install", f"{repaired_wheel}{options.test_extras}"],
|
||||
env=virtualenv_env,
|
||||
)
|
||||
|
||||
# test the wheel
|
||||
if options.test_requires:
|
||||
call_with_arch(
|
||||
["pip", "install"] + options.test_requires, env=virtualenv_env
|
||||
)
|
||||
|
||||
# run the tests from $HOME, with an absolute path in the command
|
||||
# (this ensures that Python runs the tests against the installed wheel
|
||||
# and not the repo code)
|
||||
test_command_prepared = prepare_command(
|
||||
options.test_command,
|
||||
project=Path(".").resolve(),
|
||||
package=options.package_dir.resolve(),
|
||||
)
|
||||
call_with_arch(
|
||||
test_command_prepared,
|
||||
cwd=os.environ["HOME"],
|
||||
env=virtualenv_env,
|
||||
shell=True,
|
||||
)
|
||||
|
||||
# clean up
|
||||
shutil.rmtree(venv_dir)
|
||||
|
||||
# we're all done here; move it to output (overwrite existing)
|
||||
shutil.move(str(repaired_wheel), options.output_dir)
|
||||
log.build_end()
|
||||
except subprocess.CalledProcessError as error:
|
||||
log.step_end_with_error(
|
||||
f"Command {error.cmd} failed with code {error.returncode}. {error.stdout}"
|
||||
)
|
||||
sys.exit(1)
|
||||
@@ -1,615 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {
|
||||
"cibuildwheel.ci",
|
||||
"cibuildwheel.errors",
|
||||
"cibuildwheel.logger",
|
||||
"cibuildwheel.util",
|
||||
"cibuildwheel.util.cmd",
|
||||
"cibuildwheel.util.helpers",
|
||||
"contextlib",
|
||||
"io",
|
||||
"json",
|
||||
"platform",
|
||||
"shlex",
|
||||
"shutil",
|
||||
"subprocess",
|
||||
"textwrap",
|
||||
"uuid",
|
||||
}
|
||||
|
||||
import contextlib
|
||||
import dataclasses
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import shlex
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import textwrap
|
||||
import typing
|
||||
import uuid
|
||||
from enum import Enum
|
||||
from pathlib import PurePosixPath
|
||||
from typing import Literal, assert_never
|
||||
|
||||
from cibuildwheel.ci import CIProvider, detect_ci_provider
|
||||
from cibuildwheel.errors import OCIEngineTooOldError
|
||||
from cibuildwheel.logger import log
|
||||
from cibuildwheel.util.cmd import call
|
||||
from cibuildwheel.util.helpers import FlexibleVersion, parse_key_value_string, strtobool
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Mapping, Sequence
|
||||
from pathlib import Path, PurePath
|
||||
from types import TracebackType
|
||||
from typing import IO, Self
|
||||
|
||||
from cibuildwheel.typing import PathOrStr
|
||||
|
||||
ContainerEngineName = Literal["docker", "podman"]
|
||||
|
||||
|
||||
# Order of the enum matters for tests. 386 shall appear before amd64.
|
||||
class OCIPlatform(Enum):
|
||||
i386 = "linux/386"
|
||||
AMD64 = "linux/amd64"
|
||||
ARMV7 = "linux/arm/v7"
|
||||
ARM64 = "linux/arm64"
|
||||
PPC64LE = "linux/ppc64le"
|
||||
RISCV64 = "linux/riscv64"
|
||||
S390X = "linux/s390x"
|
||||
|
||||
@classmethod
|
||||
def native(cls) -> Self:
|
||||
"""Return the current OCI platform, or raise ValueError if unknown."""
|
||||
arch = platform.machine().lower()
|
||||
mapping = {
|
||||
"i386": cls.i386,
|
||||
"i686": cls.i386,
|
||||
"x86_64": cls.AMD64,
|
||||
"amd64": cls.AMD64,
|
||||
"armv7l": cls.ARMV7,
|
||||
"aarch64": cls.ARM64,
|
||||
"arm64": cls.ARM64,
|
||||
"ppc64le": cls.PPC64LE,
|
||||
"riscv64": cls.RISCV64,
|
||||
"s390x": cls.S390X,
|
||||
}
|
||||
try:
|
||||
return mapping[arch]
|
||||
except KeyError as ex:
|
||||
msg = f"Unsupported platform architecture: {arch}"
|
||||
raise OSError(msg) from ex
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class OCIContainerEngineConfig:
|
||||
name: ContainerEngineName
|
||||
_: dataclasses.KW_ONLY
|
||||
create_args: tuple[str, ...] = dataclasses.field(default_factory=tuple)
|
||||
disable_host_mount: bool = False
|
||||
|
||||
@classmethod
|
||||
def from_config_string(cls, config_string: str) -> Self:
|
||||
config_dict = parse_key_value_string(
|
||||
config_string,
|
||||
["name"],
|
||||
["create_args", "create-args", "disable_host_mount", "disable-host-mount"],
|
||||
)
|
||||
name = " ".join(config_dict["name"])
|
||||
if name not in {"docker", "podman"}:
|
||||
msg = f"unknown container engine {name}"
|
||||
raise ValueError(msg)
|
||||
|
||||
name = typing.cast("ContainerEngineName", name)
|
||||
# some flexibility in the option names to cope with TOML conventions
|
||||
create_args = config_dict.get("create_args") or config_dict.get("create-args") or []
|
||||
disable_host_mount_options = (
|
||||
config_dict.get("disable_host_mount") or config_dict.get("disable-host-mount") or []
|
||||
)
|
||||
disable_host_mount = (
|
||||
strtobool(disable_host_mount_options[-1]) if disable_host_mount_options else False
|
||||
)
|
||||
if "--platform" in create_args or any(arg.startswith("--platform=") for arg in create_args):
|
||||
msg = "Using '--platform' in 'container-engine::create_args' is deprecated. It will be ignored."
|
||||
log.warning(msg)
|
||||
if "--platform" in create_args:
|
||||
index = create_args.index("--platform")
|
||||
create_args.pop(index)
|
||||
create_args.pop(index)
|
||||
else:
|
||||
create_args = [arg for arg in create_args if not arg.startswith("--platform=")]
|
||||
|
||||
return cls(name=name, create_args=tuple(create_args), disable_host_mount=disable_host_mount)
|
||||
|
||||
def options_summary(self) -> str | dict[str, str]:
|
||||
if not self.create_args:
|
||||
return self.name
|
||||
else:
|
||||
return {
|
||||
"name": self.name,
|
||||
"create_args": repr(self.create_args),
|
||||
"disable_host_mount": str(self.disable_host_mount),
|
||||
}
|
||||
|
||||
|
||||
DEFAULT_ENGINE = OCIContainerEngineConfig("docker")
|
||||
|
||||
|
||||
def _check_engine_version(engine: OCIContainerEngineConfig) -> None:
|
||||
try:
|
||||
version_string = call(engine.name, "version", "-f", "{{json .}}", capture_stdout=True)
|
||||
# We are using lowercase keys for all dicts so we are not affected by casing of keys
|
||||
version_info = json.loads(
|
||||
version_string.strip(), object_pairs_hook=lambda inp: {k.lower(): v for k, v in inp}
|
||||
)
|
||||
match engine.name:
|
||||
case "docker":
|
||||
client_api_version = FlexibleVersion(version_info["client"]["apiversion"])
|
||||
server_api_version = FlexibleVersion(version_info["server"]["apiversion"])
|
||||
# --platform support was introduced in 1.32 as experimental, 1.41 removed the experimental flag
|
||||
version = min(client_api_version, server_api_version)
|
||||
minimum_version = FlexibleVersion("1.41")
|
||||
minimum_version_str = "20.10.0" # docker version
|
||||
error_msg = textwrap.dedent(
|
||||
f"""
|
||||
Build failed because {engine.name} is too old.
|
||||
|
||||
cibuildwheel requires {engine.name}>={minimum_version_str} running API version {minimum_version}.
|
||||
The API version found by cibuildwheel is {version}.
|
||||
"""
|
||||
)
|
||||
case "podman":
|
||||
# podman uses the same version string for "Version" & "ApiVersion"
|
||||
client_version = FlexibleVersion(version_info["client"]["version"])
|
||||
if "server" in version_info:
|
||||
server_version = FlexibleVersion(version_info["server"]["version"])
|
||||
else:
|
||||
server_version = client_version
|
||||
# --platform support was introduced in v3
|
||||
version = min(client_version, server_version)
|
||||
minimum_version = FlexibleVersion("3")
|
||||
error_msg = textwrap.dedent(
|
||||
f"""
|
||||
Build failed because {engine.name} is too old.
|
||||
|
||||
cibuildwheel requires {engine.name}>={minimum_version}.
|
||||
The version found by cibuildwheel is {version}.
|
||||
"""
|
||||
)
|
||||
case _:
|
||||
assert_never(engine.name)
|
||||
|
||||
if version < minimum_version:
|
||||
raise OCIEngineTooOldError(error_msg) from None
|
||||
|
||||
except (subprocess.CalledProcessError, KeyError, ValueError) as e:
|
||||
msg = f"Build failed because {engine.name} is too old or is not working properly."
|
||||
raise OCIEngineTooOldError(msg) from e
|
||||
|
||||
|
||||
class OCIContainer:
|
||||
"""
|
||||
An object that represents a running OCI (e.g. Docker) container.
|
||||
|
||||
Intended for use as a context manager e.g.
|
||||
`with OCIContainer(image = 'ubuntu') as docker:`
|
||||
|
||||
A bash shell is running in the remote container. When `call()` is invoked,
|
||||
the command is relayed to the remote shell, and the results are streamed
|
||||
back to cibuildwheel.
|
||||
|
||||
Example:
|
||||
>>> # xdoctest: +REQUIRES(LINUX)
|
||||
>>> from cibuildwheel.oci_container import * # NOQA
|
||||
>>> from cibuildwheel.options import _get_pinned_container_images
|
||||
>>> import pytest
|
||||
>>> try:
|
||||
... oci_platform = OCIPlatform.native()
|
||||
... except OSError as ex:
|
||||
... pytest.skip(str(ex))
|
||||
>>> if oci_platform != OCIPlatform.AMD64:
|
||||
... pytest.skip('only runs on amd64')
|
||||
>>> image = _get_pinned_container_images()['x86_64']['manylinux2014']
|
||||
>>> # Test the default container
|
||||
>>> with OCIContainer(image=image, oci_platform=oci_platform) as self:
|
||||
... self.call(["echo", "hello world"])
|
||||
... self.call(["cat", "/proc/1/cgroup"])
|
||||
... print(self.get_environment())
|
||||
... print(self.debug_info())
|
||||
"""
|
||||
|
||||
UTILITY_PYTHON = "/opt/python/cp39-cp39/bin/python"
|
||||
|
||||
bash_stdin: IO[bytes]
|
||||
bash_stdout: IO[bytes]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
image: str,
|
||||
oci_platform: OCIPlatform,
|
||||
cwd: PathOrStr | None = None,
|
||||
engine: OCIContainerEngineConfig = DEFAULT_ENGINE,
|
||||
):
|
||||
if not image:
|
||||
msg = "Must have a non-empty image to run."
|
||||
raise ValueError(msg)
|
||||
|
||||
self.image = image
|
||||
self.oci_platform = oci_platform
|
||||
self.cwd = cwd
|
||||
self.name: str | None = None
|
||||
self.process: subprocess.Popen[bytes] | None = None
|
||||
self.engine = engine
|
||||
self.host_tar_format = ""
|
||||
if sys.platform.startswith("darwin"):
|
||||
self.host_tar_format = "--format gnutar"
|
||||
|
||||
def _get_platform_args(self, *, oci_platform: OCIPlatform | None = None) -> tuple[str, str]:
|
||||
if oci_platform is None:
|
||||
oci_platform = self.oci_platform
|
||||
|
||||
# we need '--pull=always' otherwise some images with the wrong platform get reused (e.g. 386 image for amd64)
|
||||
# c.f. https://github.com/moby/moby/issues/48197#issuecomment-2282802313
|
||||
pull = "always"
|
||||
try:
|
||||
image_platform = call(
|
||||
self.engine.name,
|
||||
"image",
|
||||
"inspect",
|
||||
self.image,
|
||||
"--format",
|
||||
(
|
||||
"{{.Os}}/{{.Architecture}}/{{.Variant}}"
|
||||
if len(oci_platform.value.split("/")) == 3
|
||||
else "{{.Os}}/{{.Architecture}}"
|
||||
),
|
||||
capture_stdout=True,
|
||||
).strip()
|
||||
if image_platform == oci_platform.value:
|
||||
# in case the correct image is already present, don't pull
|
||||
# this allows to run local only images
|
||||
pull = "never"
|
||||
except subprocess.CalledProcessError:
|
||||
# silently fallback to "--pull=always"
|
||||
pass
|
||||
return f"--platform={oci_platform.value}", f"--pull={pull}"
|
||||
|
||||
def __enter__(self) -> Self:
|
||||
assert self.process is None
|
||||
self.name = f"cibuildwheel-{uuid.uuid4()}"
|
||||
|
||||
_check_engine_version(self.engine)
|
||||
|
||||
# work-around for Travis-CI PPC64le Docker runs since 2021:
|
||||
# this avoids network splits
|
||||
# https://github.com/pypa/cibuildwheel/issues/904
|
||||
# https://github.com/conda-forge/conda-smithy/pull/1520
|
||||
network_args = []
|
||||
if detect_ci_provider() == CIProvider.travis_ci and platform.machine() == "ppc64le":
|
||||
network_args = ["--network=host"]
|
||||
|
||||
platform_args = self._get_platform_args()
|
||||
|
||||
simulate_32_bit = False
|
||||
if self.oci_platform in {OCIPlatform.i386, OCIPlatform.ARMV7}:
|
||||
# If the architecture running the image is already the right one
|
||||
# or the image entrypoint takes care of enforcing this, then we don't need to
|
||||
# simulate this
|
||||
run_cmd = [self.engine.name, "run", "--rm"]
|
||||
ctr_cmd = ["uname", "-m"]
|
||||
try:
|
||||
container_machine = call(
|
||||
*run_cmd, *platform_args, self.image, *ctr_cmd, capture_stdout=True
|
||||
).strip()
|
||||
except subprocess.CalledProcessError:
|
||||
if self.oci_platform == OCIPlatform.i386:
|
||||
# The image might have been built with amd64 architecture
|
||||
# Let's try that
|
||||
platform_args = self._get_platform_args(oci_platform=OCIPlatform.AMD64)
|
||||
container_machine = call(
|
||||
*run_cmd, *platform_args, self.image, *ctr_cmd, capture_stdout=True
|
||||
).strip()
|
||||
else:
|
||||
raise
|
||||
if container_machine not in {"i686", "armv7l", "armv8l"}:
|
||||
simulate_32_bit = True
|
||||
# sanity check to ensure no deadlock waiting for container to start
|
||||
call(
|
||||
*run_cmd,
|
||||
*platform_args,
|
||||
self.image,
|
||||
"linux32",
|
||||
"/bin/true",
|
||||
capture_stdout=True,
|
||||
)
|
||||
|
||||
shell_args = ["linux32", "/bin/bash"] if simulate_32_bit else ["/bin/bash"]
|
||||
|
||||
subprocess.run(
|
||||
[
|
||||
self.engine.name,
|
||||
"create",
|
||||
"--env=CIBUILDWHEEL",
|
||||
"--env=SOURCE_DATE_EPOCH",
|
||||
f"--name={self.name}",
|
||||
"--interactive",
|
||||
*(["--volume=/:/host"] if not self.engine.disable_host_mount else []),
|
||||
*network_args,
|
||||
*platform_args,
|
||||
*self.engine.create_args,
|
||||
self.image,
|
||||
*shell_args,
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
|
||||
try:
|
||||
self.process = subprocess.Popen(
|
||||
[
|
||||
self.engine.name,
|
||||
"start",
|
||||
"--attach",
|
||||
"--interactive",
|
||||
self.name,
|
||||
],
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
)
|
||||
|
||||
assert self.process.stdin
|
||||
assert self.process.stdout
|
||||
self.bash_stdin = self.process.stdin
|
||||
self.bash_stdout = self.process.stdout
|
||||
|
||||
# run a noop command to block until the container is responding
|
||||
self.call(["/bin/true"], cwd="/")
|
||||
|
||||
if self.cwd:
|
||||
# Although `docker create -w` does create the working dir if it
|
||||
# does not exist, podman does not. There does not seem to be a way
|
||||
# to setup a workdir for a container running in podman.
|
||||
self.call(["mkdir", "-p", os.fspath(self.cwd)], cwd="/")
|
||||
except BaseException:
|
||||
# clean-up
|
||||
if self.process is not None:
|
||||
if self.process.poll() is None:
|
||||
self.process.kill()
|
||||
self.process.communicate()
|
||||
self.process = None
|
||||
self._remove_container()
|
||||
raise
|
||||
return self
|
||||
|
||||
def __exit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None,
|
||||
exc_val: BaseException | None,
|
||||
exc_tb: TracebackType | None,
|
||||
) -> None:
|
||||
assert self.process is not None
|
||||
try:
|
||||
# Ask bash to exit cleanly and wait for the `start` process to finish.
|
||||
# If the container/bash has already died, the write raises
|
||||
# BrokenPipeError; if bash refuses to exit, `wait` raises
|
||||
# TimeoutExpired. Both are handled below so we always reach the
|
||||
# cleanup in `finally` rather than leaking the process or container.
|
||||
self.bash_stdin.write(b"exit 0\n")
|
||||
self.bash_stdin.flush()
|
||||
self.process.wait(timeout=30)
|
||||
|
||||
if self.engine.name == "podman":
|
||||
# This works around what seems to be a race condition in the
|
||||
# podman backend. The full reason is not understood. See PR #966
|
||||
# for a discussion on possible causes and attempts to remove this
|
||||
# line. For now, this seems to work "well enough".
|
||||
self.process.wait()
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
# bash didn't shut down cleanly; force the process down so it isn't
|
||||
# leaked, then continue to cleanup.
|
||||
if self.process.poll() is None:
|
||||
self.process.kill()
|
||||
self.process.wait()
|
||||
finally:
|
||||
with contextlib.suppress(OSError):
|
||||
self.bash_stdin.close()
|
||||
with contextlib.suppress(OSError):
|
||||
self.bash_stdout.close()
|
||||
self.process = None
|
||||
|
||||
keep_container = strtobool(os.environ.get("CIBW_DEBUG_KEEP_CONTAINER", ""))
|
||||
if not keep_container:
|
||||
self._remove_container()
|
||||
|
||||
def _remove_container(self) -> None:
|
||||
assert self.name is not None
|
||||
result = subprocess.run(
|
||||
[self.engine.name, "rm", "--force", "-v", self.name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
# only warn when not running in CI
|
||||
if result.returncode != 0 and detect_ci_provider() is None:
|
||||
msg = f"Failed to remove {self.name!r} container."
|
||||
log.warning(msg)
|
||||
self.name = None
|
||||
|
||||
def copy_into(self, from_path: Path, to_path: PurePath) -> None:
|
||||
if from_path.is_dir():
|
||||
self.call(["mkdir", "-p", to_path])
|
||||
subprocess.run(
|
||||
f"tar -c {self.host_tar_format} -f - . | {self.engine.name} exec -i {self.name} tar --no-same-owner -xC {shell_quote(to_path)} -f -",
|
||||
shell=True,
|
||||
check=True,
|
||||
cwd=from_path,
|
||||
)
|
||||
else:
|
||||
exec_process: subprocess.Popen[bytes]
|
||||
self.call(["mkdir", "-p", to_path.parent])
|
||||
with subprocess.Popen(
|
||||
[
|
||||
self.engine.name,
|
||||
"exec",
|
||||
"-i",
|
||||
str(self.name),
|
||||
"sh",
|
||||
"-c",
|
||||
f"cat > {shell_quote(to_path)}",
|
||||
],
|
||||
stdin=subprocess.PIPE,
|
||||
) as exec_process:
|
||||
assert exec_process.stdin
|
||||
with open(from_path, "rb") as from_file:
|
||||
shutil.copyfileobj(from_file, exec_process.stdin)
|
||||
|
||||
exec_process.stdin.close()
|
||||
exec_process.wait()
|
||||
|
||||
if exec_process.returncode:
|
||||
raise subprocess.CalledProcessError(
|
||||
exec_process.returncode, exec_process.args, None, None
|
||||
)
|
||||
|
||||
def copy_out(self, from_path: PurePath, to_path: Path) -> None:
|
||||
# note: we assume from_path is a dir
|
||||
to_path.mkdir(parents=True, exist_ok=True)
|
||||
call(self.engine.name, "cp", f"{self.name}:{from_path}/.", to_path)
|
||||
|
||||
def glob(self, path: PurePosixPath, pattern: str) -> list[PurePosixPath]:
|
||||
glob_pattern = path.joinpath(pattern)
|
||||
|
||||
path_strings = json.loads(
|
||||
self.call(
|
||||
[
|
||||
self.UTILITY_PYTHON,
|
||||
"-c",
|
||||
f"import sys, json, glob; json.dump(glob.glob({str(glob_pattern)!r}), sys.stdout)",
|
||||
],
|
||||
capture_output=True,
|
||||
)
|
||||
)
|
||||
|
||||
return [PurePosixPath(p) for p in path_strings]
|
||||
|
||||
def call(
|
||||
self,
|
||||
args: Sequence[PathOrStr],
|
||||
env: Mapping[str, str] | None = None,
|
||||
capture_output: bool = False,
|
||||
cwd: PathOrStr | None = None,
|
||||
) -> str:
|
||||
if cwd is None:
|
||||
# Podman does not start the a container in a specific working dir
|
||||
# so we always need to specify it when making calls.
|
||||
cwd = self.cwd
|
||||
|
||||
chdir = f"cd {cwd}" if cwd else ""
|
||||
env_assignments = (
|
||||
" ".join(f"{shlex.quote(k)}={shlex.quote(v)}" for k, v in env.items())
|
||||
if env is not None
|
||||
else ""
|
||||
)
|
||||
command = " ".join(shlex.quote(str(a)) for a in args)
|
||||
end_of_message = str(uuid.uuid4())
|
||||
|
||||
# log the command we're executing
|
||||
print(f" + {command}")
|
||||
|
||||
# Write a command to the remote shell. First we change the
|
||||
# cwd, if that's required. Then, we use the `env` utility to run
|
||||
# `command` inside the specified environment. We use `env` because it
|
||||
# can cope with spaces and strange characters in the name or value.
|
||||
# Finally, the remote shell is told to write a footer - this will show
|
||||
# up in the output so we know when to stop reading, and will include
|
||||
# the return code of `command`.
|
||||
self.bash_stdin.write(
|
||||
bytes(
|
||||
f"""(
|
||||
{chdir}
|
||||
env {env_assignments} {command}
|
||||
printf "%04d%s\n" $? {end_of_message}
|
||||
)
|
||||
""",
|
||||
encoding="utf8",
|
||||
errors="surrogateescape",
|
||||
)
|
||||
)
|
||||
self.bash_stdin.flush()
|
||||
|
||||
if capture_output:
|
||||
output_io: IO[bytes] = io.BytesIO()
|
||||
else:
|
||||
output_io = sys.stdout.buffer
|
||||
|
||||
while True:
|
||||
line = self.bash_stdout.readline()
|
||||
|
||||
if line.endswith(bytes(end_of_message, encoding="utf8") + b"\n"):
|
||||
# fmt: off
|
||||
footer_offset = (
|
||||
len(line)
|
||||
- 1 # newline character
|
||||
- len(end_of_message) # delimiter
|
||||
- 4 # 4 return code decimals
|
||||
)
|
||||
# fmt: on
|
||||
return_code_str = line[footer_offset : footer_offset + 4]
|
||||
return_code = int(return_code_str)
|
||||
# add the last line to output, without the footer
|
||||
output_io.write(line[0:footer_offset])
|
||||
output_io.flush()
|
||||
break
|
||||
else:
|
||||
output_io.write(line)
|
||||
output_io.flush()
|
||||
|
||||
if isinstance(output_io, io.BytesIO):
|
||||
output = str(output_io.getvalue(), encoding="utf8", errors="surrogateescape")
|
||||
else:
|
||||
output = ""
|
||||
|
||||
if return_code != 0:
|
||||
raise subprocess.CalledProcessError(return_code, args, output)
|
||||
|
||||
return output
|
||||
|
||||
def get_environment(self) -> dict[str, str]:
|
||||
env = json.loads(
|
||||
self.call(
|
||||
[
|
||||
self.UTILITY_PYTHON,
|
||||
"-c",
|
||||
"import sys, json, os; json.dump(os.environ.copy(), sys.stdout)",
|
||||
],
|
||||
capture_output=True,
|
||||
)
|
||||
)
|
||||
return typing.cast("dict[str, str]", env)
|
||||
|
||||
def environment_executor(self, command: Sequence[str], environment: dict[str, str]) -> str:
|
||||
# used as an EnvironmentExecutor to evaluate commands and capture output
|
||||
return self.call(command, env=environment, capture_output=True)
|
||||
|
||||
def debug_info(self) -> str:
|
||||
command = [self.engine.name, "info"]
|
||||
if self.engine.name == "podman":
|
||||
command.append("--debug")
|
||||
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
check=True,
|
||||
cwd=self.cwd,
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
)
|
||||
output = str(completed.stdout, encoding="utf8", errors="surrogateescape")
|
||||
return output
|
||||
|
||||
|
||||
def shell_quote(path: PurePath) -> str:
|
||||
return shlex.quote(os.fspath(path))
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,65 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from typing import Protocol
|
||||
|
||||
from cibuildwheel import errors
|
||||
from cibuildwheel.platforms import android, ios, linux, macos, pyodide, windows
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
from cibuildwheel.architecture import Architecture
|
||||
from cibuildwheel.options import Options
|
||||
from cibuildwheel.selector import BuildSelector
|
||||
from cibuildwheel.typing import GenericPythonConfiguration, PlatformName
|
||||
|
||||
|
||||
class PlatformModule(Protocol):
|
||||
# note that as per PEP544, the self argument is ignored when the protocol
|
||||
# is applied to a module
|
||||
def all_python_configurations(self) -> Sequence[GenericPythonConfiguration]: ...
|
||||
|
||||
def get_python_configurations(
|
||||
self, build_selector: BuildSelector, architectures: set[Architecture]
|
||||
) -> Sequence[GenericPythonConfiguration]: ...
|
||||
|
||||
def build(self, options: Options, tmp_path: Path) -> None: ...
|
||||
|
||||
|
||||
ALL_PLATFORM_MODULES: Final[dict[PlatformName, PlatformModule]] = {
|
||||
"linux": linux,
|
||||
"windows": windows,
|
||||
"macos": macos,
|
||||
"pyodide": pyodide,
|
||||
"android": android,
|
||||
"ios": ios,
|
||||
}
|
||||
|
||||
|
||||
def native_platform() -> PlatformName:
|
||||
if sys.platform.startswith("linux"):
|
||||
return "linux"
|
||||
elif sys.platform == "darwin":
|
||||
return "macos"
|
||||
elif sys.platform == "win32":
|
||||
return "windows"
|
||||
else:
|
||||
msg = (
|
||||
'Unable to detect platform from "sys.platform". cibuildwheel doesn\'t '
|
||||
"support building wheels for this platform. You might be able to build for a different "
|
||||
"platform using the --platform argument. Check --help output for more information."
|
||||
)
|
||||
raise errors.ConfigurationError(msg)
|
||||
|
||||
|
||||
def get_build_identifiers(
|
||||
platform_module: PlatformModule,
|
||||
build_selector: BuildSelector,
|
||||
architectures: set[Architecture],
|
||||
) -> list[str]:
|
||||
python_configurations = platform_module.get_python_configurations(build_selector, architectures)
|
||||
return [config.identifier for config in python_configurations]
|
||||
@@ -1,783 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {
|
||||
"build",
|
||||
"build.env",
|
||||
"cibuildwheel.architecture",
|
||||
"cibuildwheel.audit",
|
||||
"cibuildwheel.frontend",
|
||||
"cibuildwheel.logger",
|
||||
"cibuildwheel.util.cmd",
|
||||
"cibuildwheel.util.file",
|
||||
"cibuildwheel.util.helpers",
|
||||
"cibuildwheel.util.packaging",
|
||||
"cibuildwheel.util.python_build_standalone",
|
||||
"cibuildwheel.venv",
|
||||
"filelock",
|
||||
"packaging",
|
||||
"packaging.utils",
|
||||
"pathlib",
|
||||
"platform",
|
||||
"pprint",
|
||||
"re",
|
||||
"runpy",
|
||||
"shlex",
|
||||
"shutil",
|
||||
"subprocess",
|
||||
"textwrap",
|
||||
"typing",
|
||||
}
|
||||
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import shlex
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from pprint import pprint
|
||||
from runpy import run_path
|
||||
from textwrap import dedent
|
||||
from typing import Any
|
||||
|
||||
from build import ProjectBuilder
|
||||
from build.env import IsolatedEnv
|
||||
from filelock import FileLock
|
||||
from packaging.utils import canonicalize_name
|
||||
|
||||
from cibuildwheel import errors, platforms # pylint: disable=cyclic-import
|
||||
from cibuildwheel.architecture import Architecture, arch_synonym
|
||||
from cibuildwheel.audit import run_audit
|
||||
from cibuildwheel.frontend import (
|
||||
get_build_frontend_extra_flags,
|
||||
parse_config_settings,
|
||||
prepare_config_settings,
|
||||
)
|
||||
from cibuildwheel.logger import log
|
||||
from cibuildwheel.util import resources
|
||||
from cibuildwheel.util.cmd import call, shell
|
||||
from cibuildwheel.util.file import (
|
||||
CIBW_CACHE_PATH,
|
||||
copy_test_sources,
|
||||
download,
|
||||
move_file,
|
||||
remove_on_error,
|
||||
)
|
||||
from cibuildwheel.util.helpers import prepare_command
|
||||
from cibuildwheel.util.packaging import find_compatible_wheel
|
||||
from cibuildwheel.util.python_build_standalone import create_python_build_standalone_environment
|
||||
from cibuildwheel.venv import constraint_flags, find_uv, virtualenv
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from cibuildwheel.options import BuildOptions, Options
|
||||
from cibuildwheel.selector import BuildSelector
|
||||
from cibuildwheel.typing import PathOrStr
|
||||
|
||||
RESOURCES_ANDROID = resources.PATH / "android"
|
||||
ANDROID_TRIPLET = {
|
||||
"arm64_v8a": "aarch64-linux-android",
|
||||
"x86_64": "x86_64-linux-android",
|
||||
}
|
||||
|
||||
|
||||
def parse_identifier(identifier: str) -> tuple[str, str]:
|
||||
match = re.fullmatch(r"cp(\d)(\d+)-android_(.+)", identifier)
|
||||
if not match:
|
||||
msg = f"invalid Android identifier: '{identifier}'"
|
||||
raise ValueError(msg)
|
||||
major, minor, arch = match.groups()
|
||||
return (f"{major}.{minor}", arch)
|
||||
|
||||
|
||||
def android_triplet(identifier: str) -> str:
|
||||
return ANDROID_TRIPLET[parse_identifier(identifier)[1]]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PythonConfiguration:
|
||||
version: str
|
||||
identifier: str
|
||||
url: str
|
||||
sha256: str
|
||||
|
||||
@property
|
||||
def arch(self) -> str:
|
||||
return parse_identifier(self.identifier)[1]
|
||||
|
||||
|
||||
def all_python_configurations() -> list[PythonConfiguration]:
|
||||
return [PythonConfiguration(**item) for item in resources.read_python_configs("android")]
|
||||
|
||||
|
||||
def get_python_configurations(
|
||||
build_selector: BuildSelector, architectures: set[Architecture]
|
||||
) -> list[PythonConfiguration]:
|
||||
return [
|
||||
c
|
||||
for c in all_python_configurations()
|
||||
if c.arch in architectures and build_selector(c.identifier)
|
||||
]
|
||||
|
||||
|
||||
def shell_prepared(command: str, *, build_options: BuildOptions, env: dict[str, str]) -> None:
|
||||
shell(
|
||||
prepare_command(command, project=".", package=build_options.package_dir),
|
||||
env=env,
|
||||
)
|
||||
|
||||
|
||||
def before_all(options: Options, python_configurations: list[PythonConfiguration]) -> None:
|
||||
before_all_options = options.build_options(python_configurations[0].identifier)
|
||||
if before_all_options.before_all:
|
||||
log.step("Running before_all...")
|
||||
shell_prepared(
|
||||
before_all_options.before_all,
|
||||
build_options=before_all_options,
|
||||
env=before_all_options.environment.as_dictionary(os.environ),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BuildState:
|
||||
config: PythonConfiguration
|
||||
options: BuildOptions
|
||||
build_path: Path
|
||||
python_dir: Path
|
||||
build_env: dict[str, str]
|
||||
android_env: dict[str, str]
|
||||
|
||||
|
||||
def build(options: Options, tmp_path: Path) -> None:
|
||||
if "ANDROID_HOME" not in os.environ:
|
||||
msg = (
|
||||
"ANDROID_HOME environment variable is not set. For instructions, see "
|
||||
"https://cibuildwheel.pypa.io/en/stable/platforms/#android"
|
||||
)
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
configs = get_python_configurations(
|
||||
options.globals.build_selector, options.globals.architectures
|
||||
)
|
||||
if not configs:
|
||||
return
|
||||
|
||||
try:
|
||||
before_all(options, configs)
|
||||
|
||||
built_wheels: list[Path] = []
|
||||
for config in configs:
|
||||
log.build_start(config.identifier)
|
||||
build_options = options.build_options(config.identifier)
|
||||
build_path = tmp_path / config.identifier
|
||||
build_path.mkdir()
|
||||
python_dir = setup_target_python(config, build_path)
|
||||
build_env, android_env = setup_env(config, build_options, build_path, python_dir)
|
||||
|
||||
state = BuildState(
|
||||
config, build_options, build_path, python_dir, build_env, android_env
|
||||
)
|
||||
setup_xbuild_files(state)
|
||||
|
||||
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
|
||||
if compatible_wheel:
|
||||
print(
|
||||
f"\nFound previously built wheel {compatible_wheel.name} that is "
|
||||
f"compatible with {config.identifier}. Skipping build step..."
|
||||
)
|
||||
repaired_wheel = compatible_wheel
|
||||
else:
|
||||
before_build(state)
|
||||
built_wheel = build_wheel(state)
|
||||
repaired_wheel = repair_wheel(state, built_wheel)
|
||||
run_audit(tmp_dir=tmp_path, build_options=build_options, wheel=repaired_wheel)
|
||||
|
||||
test_wheel(state, repaired_wheel)
|
||||
|
||||
output_wheel: Path | None = None
|
||||
if compatible_wheel is None:
|
||||
output_wheel = move_file(
|
||||
repaired_wheel, build_options.output_dir / repaired_wheel.name
|
||||
)
|
||||
built_wheels.append(output_wheel)
|
||||
|
||||
shutil.rmtree(build_path)
|
||||
log.build_end(output_wheel)
|
||||
|
||||
except subprocess.CalledProcessError as error:
|
||||
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
|
||||
raise errors.FatalError(msg) from error
|
||||
|
||||
|
||||
def setup_target_python(config: PythonConfiguration, build_path: Path) -> Path:
|
||||
log.step("Installing target Python...")
|
||||
python_tgz = CIBW_CACHE_PATH / config.url.rpartition("/")[-1]
|
||||
with FileLock(f"{python_tgz}.lock"):
|
||||
if not python_tgz.exists():
|
||||
with remove_on_error(python_tgz):
|
||||
download(config.url, python_tgz, sha256=config.sha256)
|
||||
|
||||
python_dir = build_path / "python"
|
||||
python_dir.mkdir()
|
||||
shutil.unpack_archive(python_tgz, python_dir)
|
||||
|
||||
# Work around https://github.com/python/cpython/issues/138800. This can be removed
|
||||
# once we've updated to Python versions that include the fix.
|
||||
pc_path = python_dir / f"prefix/lib/pkgconfig/python-{config.version}.pc"
|
||||
pc_path.write_text(pc_path.read_text().replace("$(BLDLIBRARY)", f"-lpython{config.version}"))
|
||||
|
||||
return python_dir
|
||||
|
||||
|
||||
def setup_env(
|
||||
config: PythonConfiguration, build_options: BuildOptions, build_path: Path, python_dir: Path
|
||||
) -> tuple[dict[str, str], dict[str, str]]:
|
||||
"""
|
||||
Returns two environment dicts, both pointing at the same virtual environment:
|
||||
|
||||
* build_env, which uses the environment normally.
|
||||
* android_env, which uses the environment while simulating running on Android.
|
||||
"""
|
||||
log.step("Setting up build environment...")
|
||||
use_uv, pip = find_pip(build_options)
|
||||
|
||||
# Create virtual environment
|
||||
python_exe = create_python_build_standalone_environment(
|
||||
config.version, build_path, CIBW_CACHE_PATH
|
||||
)
|
||||
venv_dir = build_path / "venv"
|
||||
dependency_constraint = build_options.dependency_constraints.get_for_python_version(
|
||||
version=config.version, tmp_dir=build_path
|
||||
)
|
||||
build_env = virtualenv(
|
||||
config.version, python_exe, venv_dir, dependency_constraint, use_uv=use_uv
|
||||
)
|
||||
create_cmake_toolchain(config, build_path, python_dir, build_env)
|
||||
|
||||
# See platforms.md for the reason why we use this default API level.
|
||||
build_env.setdefault("ANDROID_API_LEVEL", "24")
|
||||
|
||||
# Apply custom environment variables, and check environment is still valid
|
||||
build_env = build_options.environment.as_dictionary(build_env)
|
||||
build_env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
|
||||
build_env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
|
||||
for command in ["python"] if use_uv else ["python", "pip"]:
|
||||
command_path = call("which", command, env=build_env, capture_stdout=True).strip()
|
||||
if command_path != f"{venv_dir}/bin/{command}":
|
||||
msg = (
|
||||
f"{command} available on PATH doesn't match our installed instance. If you "
|
||||
f"have modified PATH, ensure that you don't overwrite cibuildwheel's entry "
|
||||
f"or insert {command} above it."
|
||||
)
|
||||
raise errors.FatalError(msg)
|
||||
if command == "python":
|
||||
call(command, "-V", "-V", env=build_env)
|
||||
else:
|
||||
call(command, "--version", env=build_env)
|
||||
|
||||
# Install build tools
|
||||
tools = ["auditwheel", "patchelf", "pkgconf"]
|
||||
if build_options.build_frontend.name in {"build", "build[uv]"}:
|
||||
tools.append("build")
|
||||
call(*pip, "install", *tools, *constraint_flags(dependency_constraint), env=build_env)
|
||||
|
||||
# Construct an altered environment which simulates running on Android.
|
||||
android_env = setup_android_env(config, python_dir, build_env)
|
||||
|
||||
# Build-time requirements must be queried within android_env, because
|
||||
# `get_requires_for_build` can run arbitrary code in setup.py scripts, which may be
|
||||
# affected by the target platform. However, the requirements must be installed
|
||||
# within build_env, because they're going to run on the build machine.
|
||||
#
|
||||
# The `build` CLI doesn't support this combination, so we use its API to query the
|
||||
# requirements, and then install them ourselves with pip. We'll later run `build` in
|
||||
# the same environment, passing the `--no-isolation` option.
|
||||
class AndroidEnv(IsolatedEnv):
|
||||
@property
|
||||
def python_executable(self) -> str:
|
||||
return f"{venv_dir}/bin/python"
|
||||
|
||||
def make_extra_environ(self) -> dict[str, str]:
|
||||
return android_env
|
||||
|
||||
pb = ProjectBuilder.from_isolated_env(AndroidEnv(), build_options.package_dir)
|
||||
if pb.build_system_requires:
|
||||
call(*pip, "install", *pb.build_system_requires, env=build_env)
|
||||
|
||||
requires_for_build = pb.get_requires_for_build(
|
||||
"wheel",
|
||||
parse_config_settings(
|
||||
prepare_config_settings(
|
||||
build_options.config_settings,
|
||||
project=Path.cwd(),
|
||||
package=build_options.package_dir,
|
||||
)
|
||||
),
|
||||
)
|
||||
if requires_for_build:
|
||||
call(*pip, "install", *requires_for_build, env=build_env)
|
||||
|
||||
return build_env, android_env
|
||||
|
||||
|
||||
def create_cmake_toolchain(
|
||||
config: PythonConfiguration, build_path: Path, python_dir: Path, build_env: dict[str, str]
|
||||
) -> None:
|
||||
toolchain_path = build_path / "toolchain.cmake"
|
||||
build_env["CMAKE_TOOLCHAIN_FILE"] = str(toolchain_path)
|
||||
with open(toolchain_path, "w", encoding="UTF-8") as toolchain_file:
|
||||
print(
|
||||
dedent(
|
||||
f"""\
|
||||
# To support as many build systems as possible, we use environment
|
||||
# variables as the single source of truth for compiler flags and paths,
|
||||
# so they don't need to be specified here.
|
||||
|
||||
set(CMAKE_SYSTEM_NAME Android)
|
||||
set(CMAKE_SYSTEM_PROCESSOR {android_triplet(config.identifier).split("-")[0]})
|
||||
|
||||
# Inhibit all of CMake's own NDK handling code.
|
||||
set(CMAKE_SYSTEM_VERSION 1)
|
||||
|
||||
# Tell CMake where to look for headers and libraries.
|
||||
set(CMAKE_FIND_ROOT_PATH "{python_dir}/prefix")
|
||||
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
|
||||
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
|
||||
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
|
||||
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE BOTH)
|
||||
|
||||
# Allow CMake to run Python in the simulated Android environment when
|
||||
# policy CMP0190 is active.
|
||||
set(CMAKE_CROSSCOMPILING_EMULATOR /bin/sh -c [["$0" "$@"]])
|
||||
"""
|
||||
),
|
||||
file=toolchain_file,
|
||||
)
|
||||
|
||||
|
||||
def localize_sysconfigdata(
|
||||
python_dir: Path, build_env: dict[str, str], sysconfigdata_path: Path
|
||||
) -> dict[str, Any]:
|
||||
sysconfigdata: dict[str, Any] = run_path(str(sysconfigdata_path))["build_time_vars"]
|
||||
with sysconfigdata_path.open("w", encoding="UTF-8") as f:
|
||||
f.write("# Generated by cibuildwheel\n")
|
||||
f.write("build_time_vars = ")
|
||||
sysconfigdata = localized_vars(build_env, sysconfigdata, python_dir / "prefix")
|
||||
pprint(sysconfigdata, stream=f, compact=True)
|
||||
return sysconfigdata
|
||||
|
||||
|
||||
def localized_vars(
|
||||
build_env: dict[str, str], orig_vars: dict[str, Any], prefix: Path
|
||||
) -> dict[str, Any]:
|
||||
orig_prefix = orig_vars["prefix"]
|
||||
localized_vars_ = {}
|
||||
for key, value in orig_vars.items():
|
||||
# The host's sysconfigdata will include references to build-time paths.
|
||||
# Update these to refer to the current prefix.
|
||||
final = value
|
||||
if isinstance(final, str):
|
||||
final = final.replace(orig_prefix, str(prefix))
|
||||
|
||||
if key == "ANDROID_API_LEVEL":
|
||||
try:
|
||||
final = int(build_env[key])
|
||||
except ValueError as e:
|
||||
msg = f"ANDROID_API_LEVEL: {e}. This variable must be an integer."
|
||||
raise errors.FatalError(msg) from e
|
||||
|
||||
# Build systems vary in whether FLAGS variables are read from sysconfig, and if so,
|
||||
# whether they're replaced by environment variables or combined with them. Even
|
||||
# setuptools has changed its behavior here
|
||||
# (https://github.com/pypa/setuptools/issues/4836).
|
||||
#
|
||||
# Ensure consistency by clearing the sysconfig variables and letting the environment
|
||||
# variables take effect alone. This will also work for any non-Python build systems
|
||||
# which the build script may call.
|
||||
elif key in ["CFLAGS", "CXXFLAGS", "LDFLAGS"]:
|
||||
final = ""
|
||||
|
||||
# These variables contain an embedded copy of LDFLAGS.
|
||||
elif key in ["LDSHARED", "LDCXXSHARED"]:
|
||||
final = final.removesuffix(" " + orig_vars["LDFLAGS"])
|
||||
|
||||
localized_vars_[key] = final
|
||||
|
||||
return localized_vars_
|
||||
|
||||
|
||||
def setup_android_env(
|
||||
config: PythonConfiguration, python_dir: Path, build_env: dict[str, str]
|
||||
) -> dict[str, str]:
|
||||
site_packages = find_site_packages(build_env)
|
||||
for suffix in ["pth", "py"]:
|
||||
shutil.copy(RESOURCES_ANDROID / f"_cross_venv.{suffix}", site_packages)
|
||||
|
||||
sysconfigdata_path = Path(
|
||||
shutil.copy(
|
||||
glob1(python_dir, "prefix/lib/python*/_sysconfigdata_*.py"),
|
||||
site_packages,
|
||||
)
|
||||
)
|
||||
sysconfigdata = localize_sysconfigdata(python_dir, build_env, sysconfigdata_path)
|
||||
|
||||
# Activate the code in _cross_venv.py.
|
||||
android_env = build_env.copy()
|
||||
android_env["CIBW_HOST_TRIPLET"] = android_triplet(config.identifier)
|
||||
|
||||
# Get the environment variables needed to build for Android (CC, CFLAGS, etc). These are
|
||||
# generated by https://github.com/python/cpython/blob/main/Android/android-env.sh.
|
||||
env_output = call(python_dir / "android.py", "env", env=build_env, capture_stdout=True)
|
||||
|
||||
# shlex.split should produce a sequence alternating between:
|
||||
# * the word "export"
|
||||
# * a key=value string, without quotes
|
||||
for i, token in enumerate(shlex.split(env_output)):
|
||||
if i % 2 == 0:
|
||||
if token != "export":
|
||||
msg = f"Unexpected output from android.py env: expected 'export', got {token!r}"
|
||||
raise errors.FatalError(msg)
|
||||
else:
|
||||
key, sep, value = token.partition("=")
|
||||
if sep != "=":
|
||||
msg = f"Unexpected output from android.py env: expected 'key=value', got {token!r}"
|
||||
raise errors.FatalError(msg)
|
||||
android_env[key] = value
|
||||
|
||||
# localized_vars cleared the CFLAGS and CXXFLAGS in the sysconfigdata, but most
|
||||
# packages take their optimization flags from these variables. Pass these flags via
|
||||
# environment variables instead.
|
||||
#
|
||||
# We don't enable debug information, because it significantly increases binary size,
|
||||
# and most Android app developers don't have the NDK installed, so they would have no
|
||||
# way to strip it.
|
||||
opt = " ".join(word for word in sysconfigdata["OPT"].split() if not word.startswith("-g"))
|
||||
for key in ["CFLAGS", "CXXFLAGS"]:
|
||||
android_env[key] += " " + opt
|
||||
|
||||
# Cargo target linker needs to be specified after CC is set
|
||||
setup_rust(config, python_dir, android_env)
|
||||
|
||||
# Create shims which install additional build tools on first use.
|
||||
setup_fortran(android_env)
|
||||
|
||||
# `android.py env` returns PKG_CONFIG="pkg-config --define-prefix", but some build
|
||||
# systems can't handle arguments in that variable. Since we have a known version
|
||||
# of pkgconf, it's safe to use PKG_CONFIG_RELOCATE_PATHS instead.
|
||||
android_env["PKG_CONFIG"] = call(
|
||||
"which", "pkgconf-pypi", env=build_env, capture_stdout=True
|
||||
).strip()
|
||||
android_env["PKG_CONFIG_RELOCATE_PATHS"] = "1"
|
||||
|
||||
# Format the environment so it can be pasted into a shell when debugging.
|
||||
for key, value in sorted(android_env.items()):
|
||||
if os.environ.get(key) != value:
|
||||
print(f"export {key}={shlex.quote(value)}")
|
||||
|
||||
return android_env
|
||||
|
||||
|
||||
def setup_rust(config: PythonConfiguration, python_dir: Path, env: dict[str, str]) -> None:
|
||||
cargo_target = android_triplet(config.identifier)
|
||||
|
||||
# CARGO_BUILD_TARGET is the variable used by Cargo and setuptools_rust
|
||||
env["CARGO_BUILD_TARGET"] = cargo_target
|
||||
|
||||
# The linker needs to be specified after CC is set by android-env.sh
|
||||
cargo_target_linker_env_name = f"CARGO_TARGET_{cargo_target.upper().replace('-', '_')}_LINKER"
|
||||
# CC has already been set by calling android.py (it calls android-env.sh)
|
||||
env[cargo_target_linker_env_name] = env["CC"]
|
||||
|
||||
# All Python extension modules must be explicitly linked against libpython3.x.so when building for Android.
|
||||
# See: https://peps.python.org/pep-0738/#linkage
|
||||
# For projects using PyO3, this requires setting PYO3_CROSS_LIB_DIR to the directory containing libpython3.x.so.
|
||||
# See: https://pyo3.rs/v0.27.1/building-and-distribution.html#cross-compiling
|
||||
env["PYO3_CROSS_LIB_DIR"] = str(python_dir / "prefix" / "lib")
|
||||
|
||||
venv_bin = Path(env["VIRTUAL_ENV"]) / "bin"
|
||||
for tool in ["cargo", "rustup"]:
|
||||
shim_path = venv_bin / tool
|
||||
shutil.copy(RESOURCES_ANDROID / "rust_shim.py", shim_path)
|
||||
shim_path.chmod(0o755)
|
||||
|
||||
|
||||
def setup_fortran(env: dict[str, str]) -> None:
|
||||
# In case there's any autodetection based on the executable name, use the same name
|
||||
# as the real executable (see fortran_shim.run_flang)
|
||||
shim_in = RESOURCES_ANDROID / "fortran_shim.py"
|
||||
shim_out = Path(env["VIRTUAL_ENV"]) / "bin/flang-new"
|
||||
|
||||
# The hashbang line runs the shim in cibuildwheel's own virtual environment, so it
|
||||
# has access to utility functions for downloading and caching files.
|
||||
shim_out.write_text(f"#!{sys.executable}\n\n" + shim_in.read_text())
|
||||
shim_out.chmod(0o755)
|
||||
env["FC"] = str(shim_out)
|
||||
|
||||
|
||||
def setup_xbuild_files(state: BuildState) -> None:
|
||||
_, pip = find_pip(state.options)
|
||||
xbf_dir = state.build_path / "xbuild_files"
|
||||
xbf_dir.mkdir()
|
||||
|
||||
for requirement in call(*pip, "freeze", env=state.build_env, capture_stdout=True).splitlines():
|
||||
name, _, _ = requirement.strip().partition("==")
|
||||
xbuild_files = state.options.xbuild_files.get(canonicalize_name(name), [])
|
||||
if xbuild_files:
|
||||
log.step(f"Installing xbuild-files for {name}...")
|
||||
pip_install_android(state, xbf_dir, "--no-deps", requirement)
|
||||
for xbf in xbuild_files:
|
||||
if (xbf_dir / xbf).exists():
|
||||
shutil.copy(
|
||||
xbf_dir / xbf,
|
||||
find_site_packages(state.build_env) / xbf,
|
||||
)
|
||||
else:
|
||||
log.warning(f"{xbf_dir / xbf} does not exist")
|
||||
|
||||
|
||||
def pip_install_android(state: BuildState, target: Path, *args: PathOrStr) -> None:
|
||||
use_uv, pip = find_pip(state.options)
|
||||
call(
|
||||
*pip,
|
||||
"install",
|
||||
"--only-binary=:all:",
|
||||
*(["--python-platform", android_triplet(state.config.identifier)] if use_uv else []),
|
||||
"--target",
|
||||
target,
|
||||
*args,
|
||||
env=state.android_env,
|
||||
)
|
||||
|
||||
|
||||
def find_site_packages(env: dict[str, str]) -> Path:
|
||||
return glob1(Path(env["VIRTUAL_ENV"]), "lib/python*/site-packages")
|
||||
|
||||
|
||||
def glob1(base: Path, pattern: str) -> Path:
|
||||
results = list(base.glob(pattern))
|
||||
if len(results) != 1:
|
||||
msg = f"{base} contains {len(results)} paths matching '{pattern}'; expected 1"
|
||||
raise errors.FatalError(msg)
|
||||
return results[0]
|
||||
|
||||
|
||||
def find_pip(build_options: BuildOptions) -> tuple[bool, list[str]]:
|
||||
use_uv = build_options.build_frontend.name in {"build[uv]", "uv"}
|
||||
uv_path = find_uv()
|
||||
if use_uv and uv_path is None:
|
||||
msg = "uv not found"
|
||||
raise AssertionError(msg)
|
||||
pip = ["pip"] if not use_uv else [str(uv_path), "pip"]
|
||||
return use_uv, pip
|
||||
|
||||
|
||||
def before_build(state: BuildState) -> None:
|
||||
if state.options.before_build:
|
||||
log.step("Running before_build...")
|
||||
shell_prepared(
|
||||
state.options.before_build,
|
||||
build_options=state.options,
|
||||
env=state.android_env,
|
||||
)
|
||||
|
||||
|
||||
def build_wheel(state: BuildState) -> Path:
|
||||
log.step("Building wheel...")
|
||||
built_wheel_dir = state.build_path / "built_wheel"
|
||||
match state.options.build_frontend.name:
|
||||
case "build" | "build[uv]":
|
||||
call(
|
||||
"python",
|
||||
"-m",
|
||||
"build",
|
||||
state.options.package_dir,
|
||||
"--wheel",
|
||||
"--no-isolation",
|
||||
"--skip-dependency-check",
|
||||
f"--outdir={built_wheel_dir}",
|
||||
*get_build_frontend_extra_flags(
|
||||
state.options.build_frontend,
|
||||
state.options.build_verbosity,
|
||||
prepare_config_settings(
|
||||
state.options.config_settings,
|
||||
project=Path.cwd(),
|
||||
package=state.options.package_dir,
|
||||
),
|
||||
),
|
||||
env=state.android_env,
|
||||
)
|
||||
case "uv":
|
||||
uv_path = find_uv()
|
||||
assert uv_path is not None
|
||||
call(
|
||||
uv_path,
|
||||
"build",
|
||||
state.options.package_dir,
|
||||
"--wheel",
|
||||
"--no-build-isolation",
|
||||
f"--out-dir={built_wheel_dir}",
|
||||
*get_build_frontend_extra_flags(
|
||||
state.options.build_frontend,
|
||||
state.options.build_verbosity,
|
||||
prepare_config_settings(
|
||||
state.options.config_settings,
|
||||
project=Path.cwd(),
|
||||
package=state.options.package_dir,
|
||||
),
|
||||
),
|
||||
env=state.android_env,
|
||||
)
|
||||
case x:
|
||||
msg = f"Android requires the build frontend to be 'build' or 'uv', not {x!r}"
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
built_wheel = glob1(built_wheel_dir, "*.whl")
|
||||
if built_wheel.name.endswith("none-any.whl"):
|
||||
raise errors.NonPlatformWheelError()
|
||||
return built_wheel
|
||||
|
||||
|
||||
def repair_wheel(state: BuildState, built_wheel: Path) -> Path:
|
||||
log.step("Repairing wheel...")
|
||||
repaired_wheel_dir = state.build_path / "repaired_wheel"
|
||||
repaired_wheel_dir.mkdir()
|
||||
|
||||
if state.options.repair_command:
|
||||
# Tell auditwheel the locations of compiler libraries.
|
||||
toolchain = Path(state.android_env["CC"]).parent.parent
|
||||
triplet = android_triplet(state.config.identifier)
|
||||
ldpaths = ":".join(
|
||||
str(glob1(toolchain, pattern))
|
||||
for pattern in [
|
||||
f"lib/clang/*/lib/linux/{triplet.split('-')[0]}", # libomp
|
||||
f"sysroot/usr/lib/{triplet}", # libc++_shared
|
||||
]
|
||||
)
|
||||
shell(
|
||||
prepare_command(
|
||||
state.options.repair_command,
|
||||
ldpaths=ldpaths,
|
||||
wheel=built_wheel,
|
||||
dest_dir=repaired_wheel_dir,
|
||||
package=state.options.package_dir,
|
||||
project=".",
|
||||
),
|
||||
env=state.build_env,
|
||||
)
|
||||
else:
|
||||
shutil.move(built_wheel, repaired_wheel_dir)
|
||||
|
||||
repaired_wheels = list(repaired_wheel_dir.glob("*.whl"))
|
||||
if len(repaired_wheels) == 0:
|
||||
raise errors.RepairStepProducedNoWheelError()
|
||||
if len(repaired_wheels) != 1:
|
||||
raise errors.RepairStepProducedMultipleWheelsError(
|
||||
[rw.name for rw in repaired_wheels],
|
||||
)
|
||||
repaired_wheel = repaired_wheels[0]
|
||||
|
||||
if repaired_wheel.name.endswith("none-any.whl"):
|
||||
raise errors.NonPlatformWheelError()
|
||||
return repaired_wheel
|
||||
|
||||
|
||||
def test_wheel(state: BuildState, wheel: Path) -> None:
|
||||
test_command = state.options.test_command
|
||||
if not (test_command and state.options.test_selector(state.config.identifier)):
|
||||
return
|
||||
|
||||
log.step("Testing wheel...")
|
||||
native_arch = arch_synonym(platform.machine(), platforms.native_platform(), "android")
|
||||
if state.config.arch != native_arch:
|
||||
log.warning(
|
||||
f"Skipping tests for {state.config.arch}, as the build machine only "
|
||||
f"supports {native_arch}"
|
||||
)
|
||||
return
|
||||
|
||||
if state.options.before_test:
|
||||
shell_prepared(
|
||||
state.options.before_test,
|
||||
build_options=state.options,
|
||||
env=state.android_env,
|
||||
)
|
||||
|
||||
# Install the wheel and test-requires.
|
||||
site_packages_dir = state.build_path / "site-packages"
|
||||
site_packages_dir.mkdir()
|
||||
pip_install_android(
|
||||
state,
|
||||
site_packages_dir,
|
||||
f"{wheel}{state.options.test_extras}",
|
||||
*state.options.test_requires,
|
||||
)
|
||||
|
||||
# Copy test-sources.
|
||||
cwd_dir = state.build_path / "cwd"
|
||||
cwd_dir.mkdir()
|
||||
if state.options.test_sources:
|
||||
copy_test_sources(state.options.test_sources, Path.cwd(), cwd_dir)
|
||||
else:
|
||||
(cwd_dir / "test_fail.py").write_text(
|
||||
resources.TEST_FAIL_CWD_FILE.read_text(),
|
||||
)
|
||||
|
||||
# Android doesn't support placeholders in the test command.
|
||||
if any(("{" + placeholder + "}") in test_command for placeholder in ("project", "package")):
|
||||
msg = (
|
||||
f"Test command {test_command!r} with a "
|
||||
"'{project}' or '{package}' placeholder is not supported on Android, "
|
||||
"because the source directory is not visible on the emulator."
|
||||
)
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
# Parse test-command.
|
||||
test_args = shlex.split(test_command)
|
||||
if test_args[0] in {"python", "python3"} and any(arg in test_args for arg in ("-c", "-m")):
|
||||
# Forward the args to the CPython testbed script. We require '-c' or '-m'
|
||||
# to be in the command, because without those flags, the testbed script
|
||||
# will prepend '-m test', which will run Python's own test suite.
|
||||
del test_args[0]
|
||||
elif test_args[0] == "pytest":
|
||||
# We transform some commands into the `python -m` form, but this is deprecated.
|
||||
msg = (
|
||||
f"Test command {test_command!r} is not supported on Android. "
|
||||
"cibuildwheel will try to execute it as if it started with 'python -m'. "
|
||||
"If this works, all you need to do is add that to your test command."
|
||||
)
|
||||
log.warning(msg)
|
||||
test_args.insert(0, "-m")
|
||||
else:
|
||||
msg = (
|
||||
f"Test command {test_command!r} is not supported on Android. "
|
||||
f"Command must begin with 'python' or 'python3', and contain '-m' or '-c'."
|
||||
)
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
# By default, run on a testbed managed emulator running the newest supported
|
||||
# Android version. However, if the user specifies a --managed or --connected
|
||||
# test execution argument, that argument takes precedence.
|
||||
test_runtime_args = state.options.test_runtime.args
|
||||
|
||||
if any(arg.startswith(("--managed", "--connected")) for arg in test_runtime_args):
|
||||
emulator_args = []
|
||||
else:
|
||||
emulator_args = ["--managed", "maxVersion"]
|
||||
|
||||
# Run the test app.
|
||||
call(
|
||||
state.python_dir / "android.py",
|
||||
"test",
|
||||
"--site-packages",
|
||||
site_packages_dir,
|
||||
"--cwd",
|
||||
cwd_dir,
|
||||
*emulator_args,
|
||||
*(["-v"] if state.options.build_verbosity > 0 else []),
|
||||
*test_runtime_args,
|
||||
"--",
|
||||
*test_args,
|
||||
env=state.build_env,
|
||||
)
|
||||
@@ -1,816 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {
|
||||
"cibuildwheel.audit",
|
||||
"cibuildwheel.frontend",
|
||||
"cibuildwheel.logger",
|
||||
"cibuildwheel.platforms.macos",
|
||||
"cibuildwheel.util",
|
||||
"cibuildwheel.util.cmd",
|
||||
"cibuildwheel.util.file",
|
||||
"cibuildwheel.util.helpers",
|
||||
"cibuildwheel.util.packaging",
|
||||
"cibuildwheel.venv",
|
||||
"filelock",
|
||||
"packaging",
|
||||
"packaging.version",
|
||||
"pathlib",
|
||||
"platform",
|
||||
"shlex",
|
||||
"shutil",
|
||||
"subprocess",
|
||||
"textwrap",
|
||||
"typing",
|
||||
}
|
||||
|
||||
import dataclasses
|
||||
import os
|
||||
import platform
|
||||
import shlex
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
from typing import assert_never
|
||||
|
||||
from filelock import FileLock
|
||||
from packaging.version import Version
|
||||
|
||||
from cibuildwheel import errors
|
||||
from cibuildwheel.audit import run_audit
|
||||
from cibuildwheel.frontend import (
|
||||
BuildFrontendName,
|
||||
get_build_frontend_extra_flags,
|
||||
prepare_config_settings,
|
||||
)
|
||||
from cibuildwheel.logger import log
|
||||
from cibuildwheel.platforms.macos import install_cpython as install_build_cpython
|
||||
from cibuildwheel.util import resources
|
||||
from cibuildwheel.util.cmd import call, shell, split_command
|
||||
from cibuildwheel.util.file import (
|
||||
CIBW_CACHE_PATH,
|
||||
copy_test_sources,
|
||||
download,
|
||||
move_file,
|
||||
remove_on_error,
|
||||
)
|
||||
from cibuildwheel.util.helpers import prepare_command, unwrap_preserving_paragraphs
|
||||
from cibuildwheel.util.packaging import find_compatible_wheel
|
||||
from cibuildwheel.venv import constraint_flags, virtualenv
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence, Set
|
||||
|
||||
from cibuildwheel.architecture import Architecture
|
||||
from cibuildwheel.environment import ParsedEnvironment
|
||||
from cibuildwheel.options import Options
|
||||
from cibuildwheel.selector import BuildSelector
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, kw_only=True)
|
||||
class PythonConfiguration:
|
||||
version: str
|
||||
identifier: str
|
||||
url: str
|
||||
build_url: str
|
||||
build_sha256: str
|
||||
sha256: str
|
||||
|
||||
@property
|
||||
def sdk(self) -> str:
|
||||
return self.multiarch.rsplit("-", 1)[1]
|
||||
|
||||
@property
|
||||
def arch(self) -> str:
|
||||
return self.multiarch.rsplit("-", 1)[0]
|
||||
|
||||
@property
|
||||
def multiarch(self) -> str:
|
||||
# The multiarch identifier, as reported by `sys.implementation._multiarch`
|
||||
return "-".join(self.identifier.split("-ios_")[1].rsplit("_", 1))
|
||||
|
||||
@property
|
||||
def is_simulator(self) -> bool:
|
||||
return self.identifier.endswith("_iphonesimulator")
|
||||
|
||||
@property
|
||||
def xcframework_slice(self) -> str:
|
||||
"XCframeworks include binaries for multiple ABIs; which ABI section should be used?"
|
||||
return "ios-arm64_x86_64-simulator" if self.is_simulator else "ios-arm64"
|
||||
|
||||
|
||||
def all_python_configurations() -> list[PythonConfiguration]:
|
||||
# iOS builds are always cross builds; we need to install a macOS Python as
|
||||
# well. Rather than duplicate the location of the URL of macOS installers,
|
||||
# load the macos configurations, determine the macOS configuration that
|
||||
# matches the platform we're building, and embed that URL in the parsed iOS
|
||||
# configuration.
|
||||
macos_python_configs = resources.read_python_configs("macos")
|
||||
|
||||
def build_python_config(config_dict: dict[str, str]) -> dict[str, str]:
|
||||
# The iOS identifier will be something like cp313-ios_arm64_iphoneos.
|
||||
# Drop the iphoneos suffix, then replace ios with macosx to yield
|
||||
# cp313-macosx_arm64, which will be a macOS build identifier.
|
||||
modified_ios_identifier = config_dict["identifier"].rsplit("_", 1)[0]
|
||||
macos_identifier = modified_ios_identifier.replace("ios", "macosx")
|
||||
matching = [
|
||||
config for config in macos_python_configs if config["identifier"] == macos_identifier
|
||||
]
|
||||
if not matching:
|
||||
msg = (
|
||||
f"Internal error: no macOS configuration found matching "
|
||||
f"{macos_identifier!r}, needed to build the iOS configuration "
|
||||
f"{config_dict['identifier']!r}. The bundled build-platforms.toml "
|
||||
f"resources are inconsistent."
|
||||
)
|
||||
raise errors.FatalError(msg)
|
||||
return matching[0]
|
||||
|
||||
# Load the platform configuration
|
||||
full_python_configs = resources.read_python_configs("ios")
|
||||
# Build the configurations, annotating with macOS URL details.
|
||||
python_configurations = []
|
||||
for item in full_python_configs:
|
||||
build_config = build_python_config(item)
|
||||
python_configurations.append(
|
||||
PythonConfiguration(
|
||||
**item,
|
||||
build_url=build_config["url"],
|
||||
build_sha256=build_config["sha256"],
|
||||
)
|
||||
)
|
||||
return python_configurations
|
||||
|
||||
|
||||
def get_python_configurations(
|
||||
build_selector: BuildSelector,
|
||||
architectures: Set[Architecture],
|
||||
) -> list[PythonConfiguration]:
|
||||
python_configurations = all_python_configurations()
|
||||
|
||||
# Filter out configs that don't match any of the selected architectures
|
||||
python_configurations = [
|
||||
c
|
||||
for c in python_configurations
|
||||
if any(c.identifier.endswith(f"-ios_{a.value}") for a in architectures)
|
||||
]
|
||||
|
||||
# Skip builds as required by BUILD/SKIP
|
||||
python_configurations = [c for c in python_configurations if build_selector(c.identifier)]
|
||||
|
||||
return python_configurations
|
||||
|
||||
|
||||
def install_target_cpython(tmp: Path, config: PythonConfiguration, free_threading: bool) -> Path:
|
||||
if free_threading:
|
||||
msg = "Free threading builds aren't available for iOS (yet)"
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
# Install an iOS build of CPython
|
||||
ios_python_tar_gz = config.url.rsplit("/", 1)[-1]
|
||||
extension = ".tar.gz"
|
||||
assert ios_python_tar_gz.endswith(extension)
|
||||
installation_path = CIBW_CACHE_PATH / ios_python_tar_gz[: -len(extension)]
|
||||
with FileLock(str(installation_path) + ".lock"):
|
||||
if not installation_path.exists():
|
||||
downloaded_tar_gz = tmp / ios_python_tar_gz
|
||||
download(config.url, downloaded_tar_gz, sha256=config.sha256)
|
||||
with remove_on_error(installation_path):
|
||||
installation_path.mkdir(parents=True)
|
||||
call("tar", "-C", installation_path, "-xf", downloaded_tar_gz)
|
||||
downloaded_tar_gz.unlink()
|
||||
|
||||
return installation_path
|
||||
|
||||
|
||||
def cross_virtualenv(
|
||||
*,
|
||||
py_version: str,
|
||||
target_python: Path,
|
||||
multiarch: str,
|
||||
build_python: Path,
|
||||
venv_path: Path,
|
||||
dependency_constraint: Path | None,
|
||||
xbuild_tools: Sequence[str] | None,
|
||||
) -> dict[str, str]:
|
||||
"""Create a cross-compilation virtual environment.
|
||||
|
||||
In a cross-compilation environment, the *target* is the platform where the
|
||||
code will ultimately run, and the *build* is the platform where you're
|
||||
running the compilation. When building iOS wheels, iOS is the target machine
|
||||
and macOS is the build machine. The terminology around these machines varies
|
||||
between build tools (configure uses "host" and "build"; cmake uses "target" and
|
||||
"build host").
|
||||
|
||||
A cross-compilation virtualenv is an environment that is based on the
|
||||
*build* python (so that binaries can execute); but it modifies the
|
||||
environment at startup so that any request for platform details (such as
|
||||
`sys.platform` or `sysconfig.get_platform()`) return details of the target
|
||||
platform. It also applies a loader patch so that any virtualenv created by
|
||||
the cross-compilation environment will also be a cross-compilation
|
||||
environment.
|
||||
|
||||
:param py_version: The Python version (major.minor) in use
|
||||
:param target_python: The path to the python binary for the target platform
|
||||
:param multiarch: The multiarch tag for the target platform (i.e., the value
|
||||
of `sys.implementation._multiarch`)
|
||||
:param build_python: The path to the python binary for the build platform
|
||||
:param venv_path: The path where the cross virtual environment should be
|
||||
created.
|
||||
:param dependency_constraint: A path to a constraint file that should be
|
||||
used when constraining dependencies in the environment.
|
||||
:param xbuild_tools: A list of executable names (without paths) that are
|
||||
on the path, but must be preserved in the cross environment.
|
||||
"""
|
||||
# Create an initial macOS virtual environment
|
||||
env = virtualenv(
|
||||
py_version,
|
||||
build_python,
|
||||
venv_path,
|
||||
dependency_constraint,
|
||||
use_uv=False,
|
||||
)
|
||||
|
||||
# Convert the macOS virtual environment into an iOS virtual environment
|
||||
# using make_cross_venv.py.
|
||||
|
||||
# target_python is the path to the Python binary;
|
||||
# determine the root of the XCframework slice that is being used.
|
||||
slice_path = target_python.parent.parent
|
||||
if Version(py_version) >= Version("3.15"):
|
||||
# python.org 3.15+ distributions: sysconfig data lives in the
|
||||
# stdlib directory (lib-<arch>/python<version>); cross-venv scripts
|
||||
# come from cibuildwheel resources.
|
||||
arch = multiarch.split("-", maxsplit=1)[0]
|
||||
stdlib_path = slice_path / f"lib-{arch}" / f"python{py_version}"
|
||||
call(
|
||||
"python",
|
||||
str(resources.IOS_SUPPORT_FILES / "make_cross_venv.py"),
|
||||
str(venv_path),
|
||||
str(stdlib_path),
|
||||
str(resources.IOS_SUPPORT_FILES),
|
||||
env=env,
|
||||
cwd=venv_path,
|
||||
)
|
||||
else:
|
||||
# BeeWare distributions: platform-config already contains both
|
||||
# sysconfig data and the make_cross_venv.py script.
|
||||
platform_config_path = slice_path / f"platform-config/{multiarch}"
|
||||
call(
|
||||
"python",
|
||||
str(platform_config_path / "make_cross_venv.py"),
|
||||
str(venv_path),
|
||||
env=env,
|
||||
cwd=venv_path,
|
||||
)
|
||||
|
||||
# When running on macOS, it's easy for the build environment to leak into
|
||||
# the target environment, especially when building for ARM64 (because the
|
||||
# build architecture is the same as the target architecture). The primary
|
||||
# culprit for this is Homebrew libraries leaking in as dependencies for iOS
|
||||
# libraries.
|
||||
#
|
||||
# To prevent problems, set the PATH to isolate the build environment from
|
||||
# sources that could introduce incompatible binaries.
|
||||
#
|
||||
# However, there may be some tools on the path that are needed for the
|
||||
# build. Find their location on the path, and link the underlying binaries
|
||||
# (fully resolving symlinks) to a "safe" location that will *only* contain
|
||||
# those tools. This avoids needing to add *all* of Homebrew to the path just
|
||||
# to get access to (for example) cmake for build purposes. A value of None
|
||||
# means the user hasn't provided a list of xbuild tools.
|
||||
xbuild_tools_path = venv_path / "cibw_xbuild_tools"
|
||||
xbuild_tools_path.mkdir()
|
||||
if xbuild_tools is None:
|
||||
log.warning(
|
||||
textwrap.dedent(
|
||||
"""
|
||||
Your project configuration does not define any cross-build tools.
|
||||
|
||||
iOS builds use an isolated build environment; if your build process requires any
|
||||
third-party tools (such as cmake, ninja, or rustc), you must explicitly declare
|
||||
that those tools are required using xbuild-tools/CIBW_XBUILD_TOOLS. This will
|
||||
likely manifest as a "somebuildtool: command not found" error.
|
||||
|
||||
If the build succeeds, you can silence this warning by setting adding
|
||||
`xbuild-tools = []` to your pyproject.toml configuration, or exporting
|
||||
CIBW_XBUILD_TOOLS as an empty string into your environment.
|
||||
"""
|
||||
)
|
||||
)
|
||||
else:
|
||||
for tool in xbuild_tools:
|
||||
tool_path = shutil.which(tool)
|
||||
if tool_path is None:
|
||||
msg = f"Could not find a {tool!r} executable on the path."
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
# Link the binary into the safe tools directory
|
||||
original = Path(tool_path).resolve()
|
||||
print(f"{tool!r} will be included in the cross-build environment (using {original})")
|
||||
(xbuild_tools_path / tool).symlink_to(original)
|
||||
|
||||
env["PATH"] = os.pathsep.join(
|
||||
[
|
||||
# The target python's binary directory
|
||||
str(target_python.parent),
|
||||
# The cross-platform environment's binary directory
|
||||
str(venv_path / "bin"),
|
||||
# The directory of cross-build tools
|
||||
str(xbuild_tools_path),
|
||||
# The bare minimum Apple system paths.
|
||||
"/usr/bin",
|
||||
"/bin",
|
||||
"/usr/sbin",
|
||||
"/sbin",
|
||||
"/Library/Apple/usr/bin",
|
||||
]
|
||||
)
|
||||
# Also unset DYLD_LIBRARY_PATH to ensure that no macOS libraries will be
|
||||
# found and linked.
|
||||
env.pop("DYLD_LIBRARY_PATH", None)
|
||||
|
||||
return env
|
||||
|
||||
|
||||
def setup_python(
|
||||
tmp: Path,
|
||||
*,
|
||||
python_configuration: PythonConfiguration,
|
||||
dependency_constraint: Path | None,
|
||||
environment: ParsedEnvironment,
|
||||
build_frontend: BuildFrontendName,
|
||||
xbuild_tools: Sequence[str] | None,
|
||||
) -> tuple[Path, dict[str, str]]:
|
||||
# Not using set because mypy can't narrow it
|
||||
if build_frontend == "build[uv]" or build_frontend == "uv": # noqa: PLR1714
|
||||
msg = "uv doesn't support iOS"
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
# An iOS environment requires 2 python installs - one for the build machine
|
||||
# (macOS), and one for the target (iOS). We'll only ever interact with the
|
||||
# *target* python, but the build Python needs to exist to act as the base
|
||||
# for a cross venv.
|
||||
tmp.mkdir()
|
||||
implementation_id = python_configuration.identifier.split("-")[0]
|
||||
log.step(f"Installing Build Python {implementation_id}...")
|
||||
if implementation_id.startswith("cp"):
|
||||
free_threading = "t-ios" in python_configuration.identifier
|
||||
build_python = install_build_cpython(
|
||||
tmp,
|
||||
python_configuration.version,
|
||||
python_configuration.build_url,
|
||||
free_threading,
|
||||
python_configuration.build_sha256,
|
||||
)
|
||||
else:
|
||||
msg = f"Unknown Python implementation: {implementation_id}"
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
assert build_python.exists(), (
|
||||
f"{build_python.name} not found, has {list(build_python.parent.iterdir())}"
|
||||
)
|
||||
|
||||
log.step(f"Installing Target Python {implementation_id}...")
|
||||
target_install_path = install_target_cpython(tmp, python_configuration, free_threading)
|
||||
target_python = (
|
||||
target_install_path
|
||||
/ "Python.xcframework"
|
||||
/ python_configuration.xcframework_slice
|
||||
/ "bin"
|
||||
/ f"python{python_configuration.version}"
|
||||
)
|
||||
|
||||
assert target_python.parent.exists(), (
|
||||
f"{target_python.parent} not found, has {list(target_install_path.iterdir())}"
|
||||
)
|
||||
|
||||
log.step("Creating cross build environment...")
|
||||
|
||||
venv_path = tmp / "venv"
|
||||
env = cross_virtualenv(
|
||||
py_version=python_configuration.version,
|
||||
target_python=target_python,
|
||||
multiarch=python_configuration.multiarch,
|
||||
build_python=build_python,
|
||||
venv_path=venv_path,
|
||||
dependency_constraint=dependency_constraint,
|
||||
xbuild_tools=xbuild_tools,
|
||||
)
|
||||
venv_bin_path = venv_path / "bin"
|
||||
assert venv_bin_path.exists()
|
||||
|
||||
# We version pip ourselves, so we don't care about pip version checking
|
||||
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
|
||||
|
||||
# upgrade pip to the version matching our constraints
|
||||
# if necessary, reinstall it to ensure that it's available on PATH as 'pip'
|
||||
pip = ["python", "-m", "pip"]
|
||||
call(
|
||||
*pip,
|
||||
"install",
|
||||
"--upgrade",
|
||||
"pip",
|
||||
*constraint_flags(dependency_constraint),
|
||||
env=env,
|
||||
cwd=venv_path,
|
||||
)
|
||||
|
||||
# Apply our environment after pip is ready
|
||||
env = environment.as_dictionary(prev_environment=env)
|
||||
|
||||
# Check what Python version we're on
|
||||
which_python = call("which", "python", env=env, capture_stdout=True).strip()
|
||||
print(which_python)
|
||||
if which_python != str(venv_bin_path / "python"):
|
||||
msg = (
|
||||
"cibuildwheel: python available on PATH doesn't match our installed instance. "
|
||||
"If you have modified PATH, ensure that you don't overwrite cibuildwheel's "
|
||||
"entry or insert python above it."
|
||||
)
|
||||
raise errors.FatalError(msg)
|
||||
call("python", "-V", "-V", env=env)
|
||||
|
||||
# Check what pip version we're on
|
||||
assert (venv_bin_path / "pip").exists()
|
||||
which_pip = call("which", "pip", env=env, capture_stdout=True).strip()
|
||||
print(which_pip)
|
||||
if which_pip != str(venv_bin_path / "pip"):
|
||||
msg = (
|
||||
"cibuildwheel: pip available on PATH doesn't match our installed instance. "
|
||||
"If you have modified PATH, ensure that you don't overwrite cibuildwheel's "
|
||||
"entry or insert pip above it."
|
||||
)
|
||||
raise errors.FatalError(msg)
|
||||
call("pip", "--version", env=env)
|
||||
|
||||
# Ensure that IPHONEOS_DEPLOYMENT_TARGET is set in the environment
|
||||
env.setdefault("IPHONEOS_DEPLOYMENT_TARGET", "13.0")
|
||||
|
||||
log.step("Installing build tools...")
|
||||
match build_frontend:
|
||||
case "pip":
|
||||
# No additional build tools required
|
||||
pass
|
||||
case "build":
|
||||
call(
|
||||
"pip",
|
||||
"install",
|
||||
"--upgrade",
|
||||
"build[virtualenv]",
|
||||
*constraint_flags(dependency_constraint),
|
||||
env=env,
|
||||
)
|
||||
case "pyodide-build":
|
||||
msg = "The 'pyodide-build' build frontend is not supported on this platform"
|
||||
raise errors.FatalError(msg)
|
||||
case _:
|
||||
assert_never(build_frontend)
|
||||
|
||||
return target_install_path, env
|
||||
|
||||
|
||||
def build(options: Options, tmp_path: Path) -> None:
|
||||
if sys.platform != "darwin":
|
||||
msg = "iOS binaries can only be built on macOS"
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
python_configurations = get_python_configurations(
|
||||
build_selector=options.globals.build_selector,
|
||||
architectures=options.globals.architectures,
|
||||
)
|
||||
|
||||
if not python_configurations:
|
||||
return
|
||||
|
||||
try:
|
||||
before_all_options_identifier = python_configurations[0].identifier
|
||||
before_all_options = options.build_options(before_all_options_identifier)
|
||||
|
||||
if before_all_options.before_all:
|
||||
log.step("Running before_all...")
|
||||
env = before_all_options.environment.as_dictionary(prev_environment=os.environ)
|
||||
env.setdefault("IPHONEOS_DEPLOYMENT_TARGET", "13.0")
|
||||
before_all_prepared = prepare_command(
|
||||
before_all_options.before_all,
|
||||
project=".",
|
||||
package=before_all_options.package_dir,
|
||||
)
|
||||
shell(before_all_prepared, env=env)
|
||||
|
||||
built_wheels: list[Path] = []
|
||||
|
||||
for config in python_configurations:
|
||||
build_options = options.build_options(config.identifier)
|
||||
build_frontend = build_options.build_frontend
|
||||
# uv doesn't support iOS
|
||||
# Not using set because mypy can't narrow it
|
||||
if build_frontend.name == "build[uv]" or build_frontend.name == "uv": # noqa: PLR1714
|
||||
msg = "uv doesn't support iOS"
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
log.build_start(config.identifier)
|
||||
|
||||
identifier_tmp_dir = tmp_path / config.identifier
|
||||
identifier_tmp_dir.mkdir()
|
||||
built_wheel_dir = identifier_tmp_dir / "built_wheel"
|
||||
repaired_wheel_dir = identifier_tmp_dir / "repaired_wheel"
|
||||
|
||||
constraints_path = build_options.dependency_constraints.get_for_python_version(
|
||||
version=config.version, tmp_dir=identifier_tmp_dir
|
||||
)
|
||||
|
||||
target_install_path, env = setup_python(
|
||||
identifier_tmp_dir / "build",
|
||||
python_configuration=config,
|
||||
dependency_constraint=constraints_path,
|
||||
environment=build_options.environment,
|
||||
build_frontend=build_frontend.name,
|
||||
xbuild_tools=build_options.xbuild_tools,
|
||||
)
|
||||
env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
|
||||
|
||||
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
|
||||
if compatible_wheel:
|
||||
log.step_end()
|
||||
print(
|
||||
f"\nFound previously built wheel {compatible_wheel.name} "
|
||||
f"that is compatible with {config.identifier}. "
|
||||
"Skipping build step..."
|
||||
)
|
||||
test_wheel = compatible_wheel
|
||||
else:
|
||||
if build_options.before_build:
|
||||
log.step("Running before_build...")
|
||||
before_build_prepared = prepare_command(
|
||||
build_options.before_build,
|
||||
project=".",
|
||||
package=build_options.package_dir,
|
||||
)
|
||||
shell(before_build_prepared, env=env)
|
||||
|
||||
log.step("Building wheel...")
|
||||
built_wheel_dir.mkdir()
|
||||
|
||||
extra_flags = get_build_frontend_extra_flags(
|
||||
build_frontend,
|
||||
build_options.build_verbosity,
|
||||
prepare_config_settings(
|
||||
build_options.config_settings,
|
||||
project=Path.cwd(),
|
||||
package=build_options.package_dir,
|
||||
),
|
||||
)
|
||||
|
||||
match build_frontend.name:
|
||||
case "pip":
|
||||
# Path.resolve() is needed. Without it pip wheel may try to
|
||||
# fetch package from pypi.org. See
|
||||
# https://github.com/pypa/cibuildwheel/pull/369
|
||||
call(
|
||||
"python",
|
||||
"-m",
|
||||
"pip",
|
||||
"wheel",
|
||||
build_options.package_dir.resolve(),
|
||||
f"--wheel-dir={built_wheel_dir}",
|
||||
"--no-deps",
|
||||
*extra_flags,
|
||||
env=env,
|
||||
)
|
||||
case "build":
|
||||
call(
|
||||
"python",
|
||||
"-m",
|
||||
"build",
|
||||
build_options.package_dir,
|
||||
"--wheel",
|
||||
f"--outdir={built_wheel_dir}",
|
||||
*extra_flags,
|
||||
env=env,
|
||||
)
|
||||
case "pyodide-build":
|
||||
msg = "The 'pyodide-build' build frontend is not supported on this platform"
|
||||
raise errors.FatalError(msg)
|
||||
case _:
|
||||
assert_never(build_frontend)
|
||||
|
||||
try:
|
||||
built_wheel = next(built_wheel_dir.glob("*.whl"))
|
||||
except StopIteration:
|
||||
raise errors.BuildProducedNoWheelError() from None
|
||||
|
||||
if built_wheel.name.endswith("none-any.whl"):
|
||||
raise errors.NonPlatformWheelError()
|
||||
|
||||
repaired_wheel_dir.mkdir()
|
||||
if build_options.repair_command:
|
||||
log.step("Repairing wheel...")
|
||||
|
||||
repair_command_prepared = prepare_command(
|
||||
build_options.repair_command,
|
||||
wheel=built_wheel,
|
||||
dest_dir=repaired_wheel_dir,
|
||||
package=build_options.package_dir,
|
||||
project=".",
|
||||
)
|
||||
shell(repair_command_prepared, env=env)
|
||||
else:
|
||||
shutil.move(str(built_wheel), repaired_wheel_dir)
|
||||
|
||||
try:
|
||||
repaired_wheel = next(repaired_wheel_dir.glob("*.whl"))
|
||||
except StopIteration:
|
||||
raise errors.RepairStepProducedNoWheelError() from None
|
||||
|
||||
if repaired_wheel.name in {wheel.name for wheel in built_wheels}:
|
||||
raise errors.AlreadyBuiltWheelError(repaired_wheel.name)
|
||||
|
||||
log.step_end()
|
||||
|
||||
run_audit(tmp_dir=tmp_path, build_options=build_options, wheel=repaired_wheel)
|
||||
|
||||
test_wheel = repaired_wheel
|
||||
|
||||
if build_options.test_command and build_options.test_selector(config.identifier):
|
||||
if not config.is_simulator:
|
||||
log.step("Skipping tests on non-simulator SDK")
|
||||
elif config.arch != os.uname().machine:
|
||||
log.step("Skipping tests on non-native simulator architecture")
|
||||
else:
|
||||
test_env = build_options.test_environment.as_dictionary(prev_environment=env)
|
||||
|
||||
if build_options.before_test:
|
||||
before_test_prepared = prepare_command(
|
||||
build_options.before_test,
|
||||
project=".",
|
||||
package=build_options.package_dir,
|
||||
)
|
||||
shell(before_test_prepared, env=test_env)
|
||||
|
||||
log.step("Setting up test harness...")
|
||||
# Clone the testbed project into the build directory
|
||||
testbed_path = identifier_tmp_dir / "testbed"
|
||||
call(
|
||||
"python",
|
||||
target_install_path / "testbed",
|
||||
"clone",
|
||||
testbed_path,
|
||||
env=test_env,
|
||||
)
|
||||
|
||||
testbed_app_path = testbed_path / "iOSTestbed" / "app"
|
||||
|
||||
# Copy the test sources to the testbed app
|
||||
if build_options.test_sources:
|
||||
copy_test_sources(
|
||||
build_options.test_sources,
|
||||
Path.cwd(),
|
||||
testbed_app_path,
|
||||
)
|
||||
else:
|
||||
(testbed_app_path / "test_fail.py").write_text(
|
||||
resources.TEST_FAIL_CWD_FILE.read_text()
|
||||
)
|
||||
|
||||
log.step("Installing test requirements...")
|
||||
# Install the compiled wheel (with any test extras), plus
|
||||
# the test requirements. Use the --platform tag to force
|
||||
# the installation of iOS wheels; this requires the use of
|
||||
# --only-binary=:all:
|
||||
ios_version = test_env["IPHONEOS_DEPLOYMENT_TARGET"]
|
||||
platform_tag = f"ios_{ios_version.replace('.', '_')}_{config.arch}_{config.sdk}"
|
||||
|
||||
call(
|
||||
"python",
|
||||
"-m",
|
||||
"pip",
|
||||
"install",
|
||||
"--only-binary=:all:",
|
||||
"--platform",
|
||||
platform_tag,
|
||||
"--target",
|
||||
testbed_path / "iOSTestbed" / "app_packages",
|
||||
f"{test_wheel}{build_options.test_extras}",
|
||||
*build_options.test_requires,
|
||||
env=test_env,
|
||||
)
|
||||
|
||||
log.step("Running test suite...")
|
||||
|
||||
# iOS doesn't support placeholders in the test command,
|
||||
# because the source dir isn't visible on the simulator.
|
||||
if (
|
||||
"{project}" in build_options.test_command
|
||||
or "{package}" in build_options.test_command
|
||||
):
|
||||
msg = unwrap_preserving_paragraphs(
|
||||
f"""
|
||||
iOS tests configured with a test command that uses the "{{project}}" or
|
||||
"{{package}}" placeholder. iOS tests cannot use placeholders, because the
|
||||
source directory is not visible on the simulator.
|
||||
|
||||
In addition, iOS tests must run as a Python module, so the test command
|
||||
must begin with 'python -m'.
|
||||
|
||||
Test command: {build_options.test_command!r}
|
||||
"""
|
||||
)
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
test_command_list = shlex.split(build_options.test_command)
|
||||
try:
|
||||
for test_command_parts in split_command(test_command_list):
|
||||
match test_command_parts:
|
||||
case ["python", "-m", *rest]:
|
||||
final_command = rest
|
||||
case ["pytest", *rest]:
|
||||
# pytest works exactly the same as a module, so we
|
||||
# can just run it as a module.
|
||||
msg = unwrap_preserving_paragraphs(f"""
|
||||
iOS tests configured with a test command which doesn't start
|
||||
with 'python -m'. iOS tests must execute python modules - other
|
||||
entrypoints are not supported.
|
||||
|
||||
cibuildwheel will try to execute it as if it started with
|
||||
'python -m'. If this works, all you need to do is add that to
|
||||
your test command.
|
||||
|
||||
Test command: {build_options.test_command!r}
|
||||
""")
|
||||
log.warning(msg)
|
||||
final_command = ["pytest", *rest]
|
||||
case _:
|
||||
msg = unwrap_preserving_paragraphs(
|
||||
f"""
|
||||
iOS tests configured with a test command which doesn't start
|
||||
with 'python -m'. iOS tests must execute python modules - other
|
||||
entrypoints are not supported.
|
||||
|
||||
Test command: {build_options.test_command!r}
|
||||
"""
|
||||
)
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
test_runtime_args = build_options.test_runtime.args
|
||||
|
||||
# 2025-10: The GitHub Actions macos-15 runner has a known issue where
|
||||
# the default simulator won't start due to a disk performance issue;
|
||||
# see https://github.com/actions/runner-images/issues/12777 for details.
|
||||
# In the meantime, if it looks like we're running on a GitHub Actions
|
||||
# macos-15 runner, use a simulator that is known to work, unless the
|
||||
# user explicitly specifies a simulator.
|
||||
os_version, _, arch = platform.mac_ver()
|
||||
if (
|
||||
"GITHUB_ACTIONS" in os.environ
|
||||
and os_version.startswith("15.")
|
||||
and arch == "arm64"
|
||||
and not any(
|
||||
arg.startswith("--simulator") for arg in test_runtime_args
|
||||
)
|
||||
):
|
||||
test_runtime_args = [
|
||||
"--simulator",
|
||||
"iPhone 16e,OS=18.5",
|
||||
*test_runtime_args,
|
||||
]
|
||||
|
||||
call(
|
||||
"python",
|
||||
testbed_path,
|
||||
"run",
|
||||
*(["--verbose"] if build_options.build_verbosity > 0 else []),
|
||||
*test_runtime_args,
|
||||
"--",
|
||||
*final_command,
|
||||
env=test_env,
|
||||
)
|
||||
except subprocess.CalledProcessError:
|
||||
# catches the first test command failure in the loop,
|
||||
# implementing short-circuiting
|
||||
log.step_end(success=False)
|
||||
log.error(f"Test suite failed on {config.identifier}")
|
||||
sys.exit(1)
|
||||
|
||||
log.step_end()
|
||||
|
||||
# We're all done here; move it to output (overwrite existing)
|
||||
output_wheel: Path | None = None
|
||||
if compatible_wheel is None:
|
||||
output_wheel = build_options.output_dir.joinpath(repaired_wheel.name)
|
||||
moved_wheel = move_file(repaired_wheel, output_wheel)
|
||||
if moved_wheel != output_wheel.resolve():
|
||||
log.warning(
|
||||
f"{repaired_wheel} was moved to {moved_wheel} instead of {output_wheel}"
|
||||
)
|
||||
built_wheels.append(output_wheel)
|
||||
|
||||
# Clean up
|
||||
shutil.rmtree(identifier_tmp_dir)
|
||||
|
||||
log.build_end(output_wheel)
|
||||
except subprocess.CalledProcessError as error:
|
||||
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
|
||||
raise errors.FatalError(msg) from error
|
||||
@@ -1,616 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {
|
||||
"cibuildwheel.audit",
|
||||
"cibuildwheel.frontend",
|
||||
"cibuildwheel.logger",
|
||||
"cibuildwheel.util",
|
||||
"cibuildwheel.util.file",
|
||||
"cibuildwheel.util.helpers",
|
||||
"cibuildwheel.util.packaging",
|
||||
"collections",
|
||||
"contextlib",
|
||||
"pathlib",
|
||||
"shutil",
|
||||
"subprocess",
|
||||
"textwrap",
|
||||
"typing",
|
||||
}
|
||||
|
||||
import contextlib
|
||||
import dataclasses
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import textwrap
|
||||
from collections import OrderedDict
|
||||
from pathlib import Path, PurePath, PurePosixPath
|
||||
from typing import assert_never
|
||||
|
||||
from cibuildwheel import errors
|
||||
from cibuildwheel.architecture import Architecture
|
||||
from cibuildwheel.audit import needs_audit, run_audit
|
||||
from cibuildwheel.frontend import get_build_frontend_extra_flags, prepare_config_settings
|
||||
from cibuildwheel.logger import log
|
||||
from cibuildwheel.oci_container import OCIContainer, OCIContainerEngineConfig, OCIPlatform
|
||||
from cibuildwheel.util import resources
|
||||
from cibuildwheel.util.file import copy_test_sources
|
||||
from cibuildwheel.util.helpers import prepare_command, unwrap
|
||||
from cibuildwheel.util.packaging import find_compatible_wheel
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterable, Iterator, Sequence, Set
|
||||
|
||||
from cibuildwheel.options import BuildOptions, Options
|
||||
from cibuildwheel.selector import BuildSelector
|
||||
from cibuildwheel.typing import PathOrStr
|
||||
|
||||
ARCHITECTURE_OCI_PLATFORM_MAP = {
|
||||
Architecture.x86_64: OCIPlatform.AMD64,
|
||||
Architecture.i686: OCIPlatform.i386,
|
||||
Architecture.aarch64: OCIPlatform.ARM64,
|
||||
Architecture.ppc64le: OCIPlatform.PPC64LE,
|
||||
Architecture.s390x: OCIPlatform.S390X,
|
||||
Architecture.armv7l: OCIPlatform.ARMV7,
|
||||
Architecture.riscv64: OCIPlatform.RISCV64,
|
||||
}
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, kw_only=True)
|
||||
class PythonConfiguration:
|
||||
version: str
|
||||
identifier: str
|
||||
path_str: str
|
||||
|
||||
@property
|
||||
def path(self) -> PurePosixPath:
|
||||
return PurePosixPath(self.path_str)
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, kw_only=True)
|
||||
class BuildStep:
|
||||
platform_configs: list[PythonConfiguration]
|
||||
platform_tag: str
|
||||
container_engine: OCIContainerEngineConfig
|
||||
container_image: str
|
||||
|
||||
|
||||
def all_python_configurations() -> list[PythonConfiguration]:
|
||||
config_dicts = resources.read_python_configs("linux")
|
||||
return [PythonConfiguration(**item) for item in config_dicts]
|
||||
|
||||
|
||||
def get_python_configurations(
|
||||
build_selector: BuildSelector,
|
||||
architectures: Set[Architecture],
|
||||
) -> list[PythonConfiguration]:
|
||||
python_configurations = all_python_configurations()
|
||||
|
||||
# return all configurations whose arch is in our `architectures` set,
|
||||
# and match the build/skip rules
|
||||
return [
|
||||
c
|
||||
for c in python_configurations
|
||||
if any(c.identifier.endswith(arch.value) for arch in architectures)
|
||||
and build_selector(c.identifier)
|
||||
]
|
||||
|
||||
|
||||
def container_image_for_python_configuration(
|
||||
config: PythonConfiguration, build_options: BuildOptions
|
||||
) -> str:
|
||||
# e.g
|
||||
# identifier is 'cp310-manylinux_x86_64'
|
||||
# platform_tag is 'manylinux_x86_64'
|
||||
# platform_arch is 'x86_64'
|
||||
_, platform_tag = config.identifier.split("-", 1)
|
||||
_, platform_arch = platform_tag.split("_", 1)
|
||||
|
||||
assert build_options.manylinux_images is not None
|
||||
assert build_options.musllinux_images is not None
|
||||
|
||||
return (
|
||||
build_options.manylinux_images[platform_arch]
|
||||
if platform_tag.startswith("manylinux")
|
||||
else build_options.musllinux_images[platform_arch]
|
||||
)
|
||||
|
||||
|
||||
def get_build_steps(
|
||||
options: Options, python_configurations: list[PythonConfiguration]
|
||||
) -> Iterator[BuildStep]:
|
||||
"""
|
||||
Groups PythonConfigurations into BuildSteps. Each BuildStep represents a
|
||||
separate container instance.
|
||||
"""
|
||||
steps = OrderedDict[tuple[str, str, str, OCIContainerEngineConfig], BuildStep]()
|
||||
|
||||
for config in python_configurations:
|
||||
_, platform_tag = config.identifier.split("-", 1)
|
||||
|
||||
build_options = options.build_options(config.identifier)
|
||||
|
||||
before_all = build_options.before_all
|
||||
container_image = container_image_for_python_configuration(config, build_options)
|
||||
container_engine = build_options.container_engine
|
||||
|
||||
step_key = (platform_tag, container_image, before_all, container_engine)
|
||||
|
||||
if step_key in steps:
|
||||
steps[step_key].platform_configs.append(config)
|
||||
else:
|
||||
steps[step_key] = BuildStep(
|
||||
platform_configs=[config],
|
||||
platform_tag=platform_tag,
|
||||
container_engine=container_engine,
|
||||
container_image=container_image,
|
||||
)
|
||||
|
||||
yield from steps.values()
|
||||
|
||||
|
||||
def check_all_python_exist(
|
||||
*, platform_configs: Iterable[PythonConfiguration], container: OCIContainer
|
||||
) -> None:
|
||||
exist = True
|
||||
has_manylinux_interpreters = False
|
||||
messages = []
|
||||
|
||||
with contextlib.suppress(subprocess.CalledProcessError):
|
||||
# use capture_output to keep quiet
|
||||
container.call(["manylinux-interpreters", "--help"], capture_output=True)
|
||||
has_manylinux_interpreters = True
|
||||
|
||||
for config in platform_configs:
|
||||
python_path = config.path / "bin" / "python"
|
||||
if has_manylinux_interpreters:
|
||||
try:
|
||||
container.call(["manylinux-interpreters", "ensure", config.path.name])
|
||||
except subprocess.CalledProcessError:
|
||||
messages.append(
|
||||
f" 'manylinux-interpreters ensure {config.path.name}' needed to build '{config.identifier}' failed in container running image '{container.image}'."
|
||||
" Either the installation failed or this interpreter is not available in that image. Please check the logs."
|
||||
)
|
||||
exist = False
|
||||
else:
|
||||
try:
|
||||
container.call(["test", "-x", python_path])
|
||||
except subprocess.CalledProcessError:
|
||||
messages.append(
|
||||
f" '{python_path}' executable doesn't exist in image '{container.image}' to build '{config.identifier}'."
|
||||
)
|
||||
exist = False
|
||||
if not exist:
|
||||
message = "\n".join(messages)
|
||||
raise errors.FatalError(message)
|
||||
|
||||
|
||||
def build_in_container(
|
||||
*,
|
||||
options: Options,
|
||||
platform_configs: Sequence[PythonConfiguration],
|
||||
container: OCIContainer,
|
||||
container_project_path: PurePath,
|
||||
container_package_dir: PurePath,
|
||||
local_tmp_dir: Path,
|
||||
) -> None:
|
||||
container_output_dir = PurePosixPath("/output")
|
||||
|
||||
check_all_python_exist(platform_configs=platform_configs, container=container)
|
||||
|
||||
log.step("Copying project into container...")
|
||||
container.copy_into(Path.cwd(), container_project_path)
|
||||
|
||||
before_all_options_identifier = platform_configs[0].identifier
|
||||
before_all_options = options.build_options(before_all_options_identifier)
|
||||
|
||||
if before_all_options.before_all:
|
||||
log.step("Running before_all...")
|
||||
|
||||
env = container.get_environment()
|
||||
env["PATH"] = f"/opt/python/cp39-cp39/bin:{env['PATH']}"
|
||||
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
|
||||
env["PIP_ROOT_USER_ACTION"] = "ignore"
|
||||
env = before_all_options.environment.as_dictionary(
|
||||
env, executor=container.environment_executor
|
||||
)
|
||||
|
||||
before_all_prepared = prepare_command(
|
||||
before_all_options.before_all,
|
||||
project=container_project_path,
|
||||
package=container_package_dir,
|
||||
)
|
||||
container.call(["sh", "-c", before_all_prepared], env=env)
|
||||
|
||||
built_wheels: list[PurePosixPath] = []
|
||||
|
||||
for config in platform_configs:
|
||||
log.build_start(config.identifier)
|
||||
local_identifier_tmp_dir = local_tmp_dir / config.identifier
|
||||
build_options = options.build_options(config.identifier)
|
||||
build_frontend = build_options.build_frontend
|
||||
use_uv = build_frontend.name in {"build[uv]", "uv"}
|
||||
pip = ["uv", "pip"] if use_uv else ["pip"]
|
||||
|
||||
log.step("Setting up build environment...")
|
||||
|
||||
dependency_constraint_flags: list[PathOrStr] = []
|
||||
local_constraints_file = build_options.dependency_constraints.get_for_python_version(
|
||||
version=config.version,
|
||||
tmp_dir=local_identifier_tmp_dir,
|
||||
)
|
||||
if local_constraints_file:
|
||||
container_constraints_file = PurePosixPath("/constraints.txt")
|
||||
container.copy_into(local_constraints_file, container_constraints_file)
|
||||
dependency_constraint_flags = ["-c", container_constraints_file]
|
||||
|
||||
env = container.get_environment()
|
||||
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
|
||||
env["PIP_ROOT_USER_ACTION"] = "ignore"
|
||||
|
||||
# put this config's python top of the list
|
||||
python_bin = config.path / "bin"
|
||||
env["PATH"] = f"{python_bin}:{env['PATH']}"
|
||||
|
||||
env = build_options.environment.as_dictionary(env, executor=container.environment_executor)
|
||||
env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
|
||||
|
||||
# check config python is still on PATH
|
||||
which_python = container.call(["which", "python"], env=env, capture_output=True).strip()
|
||||
if PurePosixPath(which_python) != python_bin / "python":
|
||||
msg = "python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it."
|
||||
raise errors.FatalError(msg)
|
||||
container.call(["python", "-V", "-V"], env=env)
|
||||
|
||||
if use_uv:
|
||||
which_uv = container.call(["which", "uv"], env=env, capture_output=True).strip()
|
||||
if not which_uv:
|
||||
msg = "uv not found on PATH. You must use a supported manylinux or musllinux environment with uv."
|
||||
raise errors.FatalError(msg)
|
||||
else:
|
||||
which_pip = container.call(["which", "pip"], env=env, capture_output=True).strip()
|
||||
if PurePosixPath(which_pip) != python_bin / "pip":
|
||||
msg = "pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it."
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
|
||||
if compatible_wheel:
|
||||
log.step_end()
|
||||
print(
|
||||
f"\nFound previously built wheel {compatible_wheel.name}, that's compatible with {config.identifier}. Skipping build step..."
|
||||
)
|
||||
repaired_wheel = compatible_wheel
|
||||
else:
|
||||
if build_options.before_build:
|
||||
log.step("Running before_build...")
|
||||
before_build_prepared = prepare_command(
|
||||
build_options.before_build,
|
||||
project=container_project_path,
|
||||
package=container_package_dir,
|
||||
)
|
||||
before_build_env = env.copy()
|
||||
if use_uv:
|
||||
# On Linux, no virtualenv is created for the build environment
|
||||
# (unlike macOS/Windows, where one is set up before before_build
|
||||
# runs). uv requires either an active venv or an explicit Python
|
||||
# target to install packages. Pin UV_PYTHON to the exact interpreter
|
||||
# for this build so that `uv pip install` works in before_build
|
||||
# without requiring users to pass --system.
|
||||
before_build_env["UV_PYTHON"] = str(python_bin / "python")
|
||||
container.call(["sh", "-c", before_build_prepared], env=before_build_env)
|
||||
|
||||
log.step("Building wheel...")
|
||||
|
||||
temp_dir = PurePosixPath("/tmp/cibuildwheel")
|
||||
built_wheel_dir = temp_dir / "built_wheel"
|
||||
container.call(["rm", "-rf", built_wheel_dir])
|
||||
container.call(["mkdir", "-p", built_wheel_dir])
|
||||
|
||||
extra_flags = get_build_frontend_extra_flags(
|
||||
build_frontend,
|
||||
build_options.build_verbosity,
|
||||
prepare_config_settings(
|
||||
build_options.config_settings,
|
||||
project=container_project_path,
|
||||
package=container_package_dir,
|
||||
),
|
||||
)
|
||||
|
||||
match build_frontend.name:
|
||||
case "pip":
|
||||
container.call(
|
||||
[
|
||||
"python",
|
||||
"-m",
|
||||
"pip",
|
||||
"wheel",
|
||||
container_package_dir,
|
||||
f"--wheel-dir={built_wheel_dir}",
|
||||
"--no-deps",
|
||||
*extra_flags,
|
||||
],
|
||||
env=env,
|
||||
)
|
||||
case "build" | "build[uv]":
|
||||
if use_uv and "--no-isolation" not in extra_flags and "-n" not in extra_flags:
|
||||
extra_flags += ["--installer=uv"]
|
||||
container.call(
|
||||
[
|
||||
"python",
|
||||
"-m",
|
||||
"build",
|
||||
container_package_dir,
|
||||
"--wheel",
|
||||
f"--outdir={built_wheel_dir}",
|
||||
*extra_flags,
|
||||
],
|
||||
env=env,
|
||||
)
|
||||
case "uv":
|
||||
container.call(
|
||||
[
|
||||
"uv",
|
||||
"build",
|
||||
f"--python={python_bin / 'python'}",
|
||||
container_package_dir,
|
||||
"--wheel",
|
||||
f"--out-dir={built_wheel_dir}",
|
||||
*extra_flags,
|
||||
],
|
||||
env=env,
|
||||
)
|
||||
case "pyodide-build":
|
||||
msg = "The 'pyodide-build' build frontend is not supported on this platform"
|
||||
raise errors.FatalError(msg)
|
||||
case _:
|
||||
assert_never(build_frontend)
|
||||
|
||||
try:
|
||||
built_wheel = container.glob(built_wheel_dir, "*.whl")[0]
|
||||
except IndexError:
|
||||
raise errors.BuildProducedNoWheelError() from None
|
||||
|
||||
repaired_wheel_dir = temp_dir / "repaired_wheel"
|
||||
container.call(["rm", "-rf", repaired_wheel_dir])
|
||||
container.call(["mkdir", "-p", repaired_wheel_dir])
|
||||
|
||||
if built_wheel.name.endswith("none-any.whl"):
|
||||
raise errors.NonPlatformWheelError()
|
||||
|
||||
if build_options.repair_command:
|
||||
log.step("Repairing wheel...")
|
||||
repair_command_prepared = prepare_command(
|
||||
build_options.repair_command,
|
||||
wheel=built_wheel,
|
||||
dest_dir=repaired_wheel_dir,
|
||||
package=container_package_dir,
|
||||
project=container_project_path,
|
||||
)
|
||||
container.call(["sh", "-c", repair_command_prepared], env=env)
|
||||
else:
|
||||
container.call(["mv", built_wheel, repaired_wheel_dir])
|
||||
|
||||
match container.glob(repaired_wheel_dir, "*.whl"):
|
||||
case []:
|
||||
raise errors.RepairStepProducedNoWheelError()
|
||||
case [repaired_wheel]:
|
||||
pass
|
||||
case too_many:
|
||||
raise errors.RepairStepProducedMultipleWheelsError([p.name for p in too_many])
|
||||
|
||||
if repaired_wheel.name in {wheel.name for wheel in built_wheels}:
|
||||
raise errors.AlreadyBuiltWheelError(repaired_wheel.name)
|
||||
|
||||
log.step_end()
|
||||
|
||||
if needs_audit(build_options.audit_command, repaired_wheel.name):
|
||||
local_abi3audit_dir = local_identifier_tmp_dir / "audit"
|
||||
local_abi3audit_dir.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
container.copy_out(repaired_wheel_dir, local_abi3audit_dir)
|
||||
local_wheel = local_abi3audit_dir / repaired_wheel.name
|
||||
run_audit(tmp_dir=local_tmp_dir, build_options=build_options, wheel=local_wheel)
|
||||
finally:
|
||||
shutil.rmtree(local_abi3audit_dir, ignore_errors=True)
|
||||
|
||||
if build_options.test_command and build_options.test_selector(config.identifier):
|
||||
log.step("Testing wheel...")
|
||||
|
||||
# set up a virtual environment to install and test from, to make sure
|
||||
# there are no dependencies that were pulled in at build time.
|
||||
if not use_uv:
|
||||
container.call(
|
||||
["pip", "install", "virtualenv", *dependency_constraint_flags], env=env
|
||||
)
|
||||
|
||||
testing_temp_dir = PurePosixPath(
|
||||
container.call(["mktemp", "-d"], capture_output=True).strip()
|
||||
)
|
||||
venv_dir = testing_temp_dir / "venv"
|
||||
|
||||
if use_uv:
|
||||
container.call(["uv", "venv", venv_dir, "--python", python_bin / "python"], env=env)
|
||||
else:
|
||||
# Use embedded dependencies from virtualenv to ensure determinism
|
||||
venv_args = ["--no-periodic-update", "--pip=embed", "--no-setuptools"]
|
||||
if "38" in config.identifier:
|
||||
venv_args.append("--no-wheel")
|
||||
container.call(["python", "-m", "virtualenv", *venv_args, venv_dir], env=env)
|
||||
|
||||
virtualenv_env = env.copy()
|
||||
virtualenv_env["PATH"] = f"{venv_dir / 'bin'}:{virtualenv_env['PATH']}"
|
||||
virtualenv_env["VIRTUAL_ENV"] = str(venv_dir)
|
||||
virtualenv_env = build_options.test_environment.as_dictionary(
|
||||
prev_environment=virtualenv_env
|
||||
)
|
||||
|
||||
if build_options.before_test:
|
||||
before_test_prepared = prepare_command(
|
||||
build_options.before_test,
|
||||
project=container_project_path,
|
||||
package=container_package_dir,
|
||||
)
|
||||
container.call(["sh", "-c", before_test_prepared], env=virtualenv_env)
|
||||
|
||||
# Install the wheel we just built
|
||||
container.call(
|
||||
[*pip, "install", str(repaired_wheel) + build_options.test_extras],
|
||||
env=virtualenv_env,
|
||||
)
|
||||
|
||||
# Install any requirements to run the tests
|
||||
if build_options.test_requires:
|
||||
container.call([*pip, "install", *build_options.test_requires], env=virtualenv_env)
|
||||
|
||||
# Run the tests from a different directory
|
||||
test_command_prepared = prepare_command(
|
||||
build_options.test_command,
|
||||
project=container_project_path,
|
||||
package=container_package_dir,
|
||||
wheel=repaired_wheel,
|
||||
)
|
||||
|
||||
test_cwd = testing_temp_dir / "test_cwd"
|
||||
container.call(["mkdir", "-p", test_cwd])
|
||||
|
||||
if build_options.test_sources:
|
||||
copy_test_sources(
|
||||
build_options.test_sources,
|
||||
Path.cwd(),
|
||||
test_cwd,
|
||||
copy_into=container.copy_into,
|
||||
)
|
||||
else:
|
||||
# Use the test_fail.py file to raise a nice error if the user
|
||||
# tries to run tests in the cwd
|
||||
container.copy_into(resources.TEST_FAIL_CWD_FILE, test_cwd / "test_fail.py")
|
||||
|
||||
container.call(["sh", "-c", test_command_prepared], cwd=test_cwd, env=virtualenv_env)
|
||||
|
||||
# clean up test environment
|
||||
container.call(["rm", "-rf", testing_temp_dir])
|
||||
|
||||
# move repaired wheel to output
|
||||
output_wheel: Path | None = None
|
||||
if compatible_wheel is None:
|
||||
container.call(["mkdir", "-p", container_output_dir])
|
||||
container.call(["mv", repaired_wheel, container_output_dir])
|
||||
built_wheels.append(container_output_dir / repaired_wheel.name)
|
||||
output_wheel = options.globals.output_dir / repaired_wheel.name
|
||||
|
||||
log.build_end(output_wheel)
|
||||
|
||||
log.step("Copying wheels back to host...")
|
||||
# copy the output back into the host
|
||||
container.copy_out(container_output_dir, options.globals.output_dir)
|
||||
log.step_end()
|
||||
|
||||
|
||||
def build(options: Options, tmp_path: Path) -> None:
|
||||
python_configurations = get_python_configurations(
|
||||
options.globals.build_selector, options.globals.architectures
|
||||
)
|
||||
|
||||
cwd = Path.cwd()
|
||||
abs_package_dir = options.globals.package_dir.resolve()
|
||||
if cwd != abs_package_dir and cwd not in abs_package_dir.parents:
|
||||
msg = "package_dir must be inside the working directory"
|
||||
raise errors.ConfigurationError(msg)
|
||||
|
||||
container_project_path = PurePosixPath("/project")
|
||||
container_package_dir = container_project_path / abs_package_dir.relative_to(cwd)
|
||||
|
||||
for build_step in get_build_steps(options, python_configurations):
|
||||
try:
|
||||
# check the container engine is installed
|
||||
subprocess.run(
|
||||
[build_step.container_engine.name, "--version"],
|
||||
check=True,
|
||||
stdout=subprocess.DEVNULL,
|
||||
)
|
||||
except subprocess.CalledProcessError as error:
|
||||
msg = unwrap(
|
||||
f"""
|
||||
{build_step.container_engine.name} not found. An OCI exe like
|
||||
Docker or Podman is required to run Linux builds. If you're
|
||||
building on Travis CI, add `services: [docker]` to your
|
||||
.travis.yml. If you're building on Circle CI in Linux, add a
|
||||
`setup_remote_docker` step to your .circleci/config.yml.
|
||||
"""
|
||||
)
|
||||
raise errors.ConfigurationError(msg) from error
|
||||
|
||||
try:
|
||||
ids_to_build = [x.identifier for x in build_step.platform_configs]
|
||||
log.step(f"Starting container image {build_step.container_image}...")
|
||||
|
||||
print(f"info: This container will host the build for {', '.join(ids_to_build)}...")
|
||||
architecture = Architecture(build_step.platform_tag.split("_", 1)[1])
|
||||
|
||||
with OCIContainer(
|
||||
image=build_step.container_image,
|
||||
oci_platform=ARCHITECTURE_OCI_PLATFORM_MAP[architecture],
|
||||
cwd=container_project_path,
|
||||
engine=build_step.container_engine,
|
||||
) as container:
|
||||
build_in_container(
|
||||
options=options,
|
||||
platform_configs=build_step.platform_configs,
|
||||
container=container,
|
||||
container_project_path=container_project_path,
|
||||
container_package_dir=container_package_dir,
|
||||
local_tmp_dir=tmp_path,
|
||||
)
|
||||
|
||||
except subprocess.CalledProcessError as error:
|
||||
troubleshoot(options, error)
|
||||
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
|
||||
raise errors.FatalError(msg) from error
|
||||
|
||||
|
||||
def _matches_prepared_command(error_cmd: Sequence[str], command_template: str) -> bool:
|
||||
if len(error_cmd) < 3 or error_cmd[0:2] != ["sh", "-c"]:
|
||||
return False
|
||||
command_prefix = command_template.split("{", maxsplit=1)[0].strip()
|
||||
return error_cmd[2].startswith(command_prefix)
|
||||
|
||||
|
||||
def troubleshoot(options: Options, error: Exception) -> None:
|
||||
if isinstance(error, subprocess.CalledProcessError) and (
|
||||
error.cmd[0:4] == ["python", "-m", "pip", "wheel"]
|
||||
or error.cmd[0:2] == ["uv", "build"]
|
||||
or error.cmd[0:3] == ["python", "-m", "build"]
|
||||
or _matches_prepared_command(
|
||||
error.cmd, options.build_options(None).repair_command
|
||||
) # TODO allow matching of overrides too?
|
||||
):
|
||||
# the wheel build step or the repair step failed
|
||||
so_files = list(options.globals.package_dir.glob("**/*.so"))
|
||||
|
||||
if so_files:
|
||||
print(
|
||||
textwrap.dedent(
|
||||
"""
|
||||
NOTE: Shared object (.so) files found in this project.
|
||||
|
||||
These files might be built against the wrong OS, causing problems with
|
||||
auditwheel. If possible, run cibuildwheel in a clean checkout.
|
||||
|
||||
If you're using Cython and have previously done an in-place build,
|
||||
remove those build files (*.so and *.c) before starting cibuildwheel.
|
||||
|
||||
setuptools uses the build/ folder to store its build cache. It
|
||||
may be necessary to remove those build files (*.so and *.o) before
|
||||
starting cibuildwheel.
|
||||
|
||||
Files that belong to a virtual environment are probably not an issue
|
||||
unless you used a custom command telling cibuildwheel to activate it.
|
||||
"""
|
||||
),
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
print(" Files detected:")
|
||||
print("\n".join(f" {f}" for f in so_files))
|
||||
print()
|
||||
@@ -1,808 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {
|
||||
"cibuildwheel.audit",
|
||||
"cibuildwheel.ci",
|
||||
"cibuildwheel.frontend",
|
||||
"cibuildwheel.logger",
|
||||
"cibuildwheel.util",
|
||||
"cibuildwheel.util.cmd",
|
||||
"cibuildwheel.util.file",
|
||||
"cibuildwheel.util.helpers",
|
||||
"cibuildwheel.util.packaging",
|
||||
"cibuildwheel.venv",
|
||||
"filelock",
|
||||
"inspect",
|
||||
"packaging",
|
||||
"packaging.version",
|
||||
"pathlib",
|
||||
"platform",
|
||||
"re",
|
||||
"shutil",
|
||||
"subprocess",
|
||||
}
|
||||
|
||||
import dataclasses
|
||||
import functools
|
||||
import inspect
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import typing
|
||||
from pathlib import Path
|
||||
from typing import assert_never
|
||||
|
||||
from filelock import FileLock
|
||||
from packaging.version import Version
|
||||
|
||||
from cibuildwheel import errors
|
||||
from cibuildwheel.audit import run_audit
|
||||
from cibuildwheel.ci import detect_ci_provider
|
||||
from cibuildwheel.frontend import (
|
||||
BuildFrontendName,
|
||||
get_build_frontend_extra_flags,
|
||||
prepare_config_settings,
|
||||
)
|
||||
from cibuildwheel.logger import log
|
||||
from cibuildwheel.util import resources
|
||||
from cibuildwheel.util.cmd import call, shell
|
||||
from cibuildwheel.util.file import (
|
||||
CIBW_CACHE_PATH,
|
||||
copy_test_sources,
|
||||
download,
|
||||
move_file,
|
||||
remove_on_error,
|
||||
)
|
||||
from cibuildwheel.util.helpers import prepare_command, unwrap
|
||||
from cibuildwheel.util.packaging import find_compatible_wheel, get_pip_version
|
||||
from cibuildwheel.venv import constraint_flags, find_uv, target_marker_env, virtualenv
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Set
|
||||
from typing import Literal
|
||||
|
||||
from cibuildwheel.architecture import Architecture
|
||||
from cibuildwheel.environment import ParsedEnvironment
|
||||
from cibuildwheel.options import Options
|
||||
from cibuildwheel.selector import BuildSelector
|
||||
|
||||
|
||||
@functools.cache
|
||||
def get_macos_version() -> tuple[int, int]:
|
||||
"""
|
||||
Returns the macOS major/minor version, as a tuple, e.g. (10, 15) or (11, 0)
|
||||
|
||||
These tuples can be used in comparisons, e.g.
|
||||
(10, 14) <= (11, 0) == True
|
||||
(10, 14) <= (10, 16) == True
|
||||
(11, 2) <= (11, 0) != True
|
||||
"""
|
||||
version_str, _, _ = platform.mac_ver()
|
||||
version = tuple(map(int, version_str.split(".")[:2]))
|
||||
if (10, 15) < version < (11, 0):
|
||||
# When built against an older macOS SDK, Python will report macOS 10.16
|
||||
# instead of the real version.
|
||||
version_str = call(
|
||||
sys.executable,
|
||||
"-sS",
|
||||
"-c",
|
||||
"import platform; print(platform.mac_ver()[0])",
|
||||
env={"SYSTEM_VERSION_COMPAT": "0"},
|
||||
capture_stdout=True,
|
||||
)
|
||||
version = tuple(map(int, version_str.split(".")[:2]))
|
||||
return typing.cast("tuple[int, int]", version)
|
||||
|
||||
|
||||
@functools.cache
|
||||
def get_test_macosx_deployment_target() -> str:
|
||||
version = get_macos_version()
|
||||
if version >= (11, 0):
|
||||
return f"{version[0]}.0"
|
||||
return f"{version[0]}.{version[1]}"
|
||||
|
||||
|
||||
def get_macos_sdks() -> list[str]:
|
||||
output = call("xcodebuild", "-showsdks", capture_stdout=True)
|
||||
return [m.group(1) for m in re.finditer(r"-sdk (macosx\S+)", output)]
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, kw_only=True)
|
||||
class PythonConfiguration:
|
||||
version: str
|
||||
identifier: str
|
||||
url: str
|
||||
sha256: str
|
||||
|
||||
|
||||
def all_python_configurations() -> list[PythonConfiguration]:
|
||||
config_dicts = resources.read_python_configs("macos")
|
||||
return [PythonConfiguration(**item) for item in config_dicts]
|
||||
|
||||
|
||||
def get_python_configurations(
|
||||
build_selector: BuildSelector, architectures: Set[Architecture]
|
||||
) -> list[PythonConfiguration]:
|
||||
python_configurations = all_python_configurations()
|
||||
|
||||
# filter out configs that don't match any of the selected architectures
|
||||
python_configurations = [
|
||||
c
|
||||
for c in python_configurations
|
||||
if any(c.identifier.endswith(a.value) for a in architectures)
|
||||
]
|
||||
|
||||
# skip builds as required by BUILD/SKIP
|
||||
python_configurations = [c for c in python_configurations if build_selector(c.identifier)]
|
||||
|
||||
# filter-out some cross-compilation configs with PyPy and GraalPy:
|
||||
# can't build arm64 on x86_64
|
||||
# rosetta allows to build x86_64 on arm64
|
||||
if platform.machine() == "x86_64":
|
||||
python_configurations_before = set(python_configurations)
|
||||
python_configurations = [
|
||||
c
|
||||
for c in python_configurations
|
||||
if not (c.identifier.startswith(("pp", "gp")) and c.identifier.endswith("arm64"))
|
||||
]
|
||||
removed_elements = python_configurations_before - set(python_configurations)
|
||||
if removed_elements:
|
||||
ids = ", ".join(c.identifier for c in removed_elements)
|
||||
log.quiet(
|
||||
unwrap(
|
||||
f"""
|
||||
Note: {ids} {"was" if len(removed_elements) == 1 else "were"}
|
||||
selected, but can't be built on x86_64 so will be skipped automatically.
|
||||
"""
|
||||
)
|
||||
)
|
||||
|
||||
return python_configurations
|
||||
|
||||
|
||||
def install_cpython(_tmp: Path, version: str, url: str, free_threading: bool, sha256: str) -> Path:
|
||||
ft = "T" if free_threading else ""
|
||||
installation_path = Path(f"/Library/Frameworks/Python{ft}.framework/Versions/{version}")
|
||||
with FileLock(CIBW_CACHE_PATH / f"cpython{version}.lock"):
|
||||
installed_system_packages = call("pkgutil", "--pkgs", capture_stdout=True).splitlines()
|
||||
# if this version of python isn't installed, get it from python.org and install
|
||||
python_package_identifier = f"org.python.Python.Python{ft}Framework-{version}"
|
||||
if python_package_identifier not in installed_system_packages:
|
||||
if detect_ci_provider() is None:
|
||||
# if running locally, we don't want to install CPython with sudo
|
||||
# let the user know & provide a link to the installer
|
||||
msg = inspect.cleandoc(
|
||||
f"""
|
||||
Framework CPython {version} not detected as installed in:
|
||||
{installation_path}
|
||||
cibuildwheel will not perform system-wide installs when running outside of CI.
|
||||
To build locally, install CPython {version} on this machine, or, disable this
|
||||
version of Python using CIBW_SKIP=cp{version.replace(".", "")}-macosx_*
|
||||
For portable builds, cibuildwheel needs the official builds from python.org.
|
||||
Download link: {url}
|
||||
"""
|
||||
)
|
||||
raise errors.FatalError(msg)
|
||||
python_filename = url.rsplit("/", maxsplit=1)[-1]
|
||||
pkg_path = CIBW_CACHE_PATH / "cpython-installer" / python_filename
|
||||
if not pkg_path.exists():
|
||||
download(url, pkg_path, sha256=sha256)
|
||||
args = []
|
||||
if version.startswith("3.14"):
|
||||
args += ["-applyChoiceChangesXML", str(resources.FREE_THREAD_ENABLE_314.resolve())]
|
||||
elif version.startswith("3.15"):
|
||||
args += ["-applyChoiceChangesXML", str(resources.FREE_THREAD_ENABLE_315.resolve())]
|
||||
call("sudo", "installer", "-pkg", pkg_path, *args, "-target", "/")
|
||||
pkg_path.unlink()
|
||||
env = os.environ.copy()
|
||||
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
|
||||
|
||||
if free_threading:
|
||||
call(installation_path / f"bin/python{version}t", "-m", "ensurepip", env=env)
|
||||
call(
|
||||
installation_path / f"bin/python{version}t",
|
||||
resources.INSTALL_CERTIFI_SCRIPT,
|
||||
env=env,
|
||||
)
|
||||
else:
|
||||
call(installation_path / "bin/python3", resources.INSTALL_CERTIFI_SCRIPT, env=env)
|
||||
|
||||
return installation_path / "bin" / (f"python{version}t" if free_threading else "python3")
|
||||
|
||||
|
||||
def install_pypy(tmp: Path, url: str, sha256: str) -> Path:
|
||||
pypy_archive = url.rsplit("/", 1)[-1]
|
||||
# PyPy ships .tar.bz2 for some platforms and .tar.gz for others.
|
||||
extension = ".tar.bz2" if pypy_archive.endswith(".tar.bz2") else ".tar.gz"
|
||||
assert pypy_archive.endswith(extension)
|
||||
installation_path = CIBW_CACHE_PATH / pypy_archive[: -len(extension)]
|
||||
with FileLock(str(installation_path) + ".lock"):
|
||||
if not installation_path.exists():
|
||||
downloaded_archive = tmp / pypy_archive
|
||||
download(url, downloaded_archive, sha256=sha256)
|
||||
installation_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with remove_on_error(installation_path):
|
||||
call("tar", "-C", installation_path.parent, "-xf", downloaded_archive)
|
||||
downloaded_archive.unlink()
|
||||
return installation_path / "bin" / "pypy3"
|
||||
|
||||
|
||||
def install_graalpy(tmp: Path, url: str, sha256: str) -> Path:
|
||||
graalpy_archive = url.rsplit("/", 1)[-1]
|
||||
extension = ".tar.gz"
|
||||
assert graalpy_archive.endswith(extension)
|
||||
installation_path = CIBW_CACHE_PATH / graalpy_archive[: -len(extension)]
|
||||
with FileLock(str(installation_path) + ".lock"):
|
||||
if not installation_path.exists():
|
||||
downloaded_archive = tmp / graalpy_archive
|
||||
download(url, downloaded_archive, sha256=sha256)
|
||||
with remove_on_error(installation_path):
|
||||
installation_path.mkdir(parents=True)
|
||||
# GraalPy top-folder name is inconsistent with archive name
|
||||
call(
|
||||
"tar",
|
||||
"-C",
|
||||
installation_path,
|
||||
"--strip-components=1",
|
||||
"-xzf",
|
||||
downloaded_archive,
|
||||
)
|
||||
downloaded_archive.unlink()
|
||||
return installation_path / "bin" / "graalpy"
|
||||
|
||||
|
||||
def setup_python(
|
||||
tmp: Path,
|
||||
python_configuration: PythonConfiguration,
|
||||
dependency_constraint: Path | None,
|
||||
environment: ParsedEnvironment,
|
||||
build_frontend: BuildFrontendName,
|
||||
) -> tuple[Path, dict[str, str]]:
|
||||
uv_path = find_uv()
|
||||
use_uv = build_frontend in {"build[uv]", "uv"}
|
||||
|
||||
tmp.mkdir()
|
||||
implementation_id = python_configuration.identifier.split("-")[0]
|
||||
log.step(f"Installing Python {implementation_id}...")
|
||||
if implementation_id.startswith("cp"):
|
||||
free_threading = "t-macos" in python_configuration.identifier
|
||||
base_python = install_cpython(
|
||||
tmp,
|
||||
python_configuration.version,
|
||||
python_configuration.url,
|
||||
free_threading,
|
||||
python_configuration.sha256,
|
||||
)
|
||||
|
||||
elif implementation_id.startswith("pp"):
|
||||
base_python = install_pypy(tmp, python_configuration.url, python_configuration.sha256)
|
||||
elif implementation_id.startswith("gp"):
|
||||
base_python = install_graalpy(tmp, python_configuration.url, python_configuration.sha256)
|
||||
else:
|
||||
msg = "Unknown Python implementation"
|
||||
raise ValueError(msg)
|
||||
assert base_python.exists(), (
|
||||
f"{base_python.name} not found, has {list(base_python.parent.iterdir())}"
|
||||
)
|
||||
|
||||
log.step("Setting up build environment...")
|
||||
venv_path = tmp / "venv"
|
||||
env = virtualenv(
|
||||
python_configuration.version,
|
||||
base_python,
|
||||
venv_path,
|
||||
dependency_constraint,
|
||||
use_uv=use_uv,
|
||||
marker_env=target_marker_env(implementation_id=implementation_id),
|
||||
)
|
||||
venv_bin_path = venv_path / "bin"
|
||||
assert venv_bin_path.exists()
|
||||
# Fix issue with site.py setting the wrong `sys.prefix`, `sys.exec_prefix`,
|
||||
# `sys.path`, ... for PyPy: https://foss.heptapod.net/pypy/pypy/issues/3175
|
||||
# Also fix an issue with the shebang of installed scripts inside the
|
||||
# testing virtualenv- see https://github.com/theacodes/nox/issues/44 and
|
||||
# https://github.com/pypa/virtualenv/issues/620
|
||||
# Also see https://github.com/python/cpython/pull/9516
|
||||
env.pop("__PYVENV_LAUNCHER__", None)
|
||||
# uv uses this over the current environment's python, so remove it to avoid confusion
|
||||
env.pop("PYTHON_VERSION", None)
|
||||
env.pop("PYTHON_ARCH", None)
|
||||
env.pop("UV_PYTHON", None)
|
||||
|
||||
# we version pip ourselves, so we don't care about pip version checking
|
||||
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
|
||||
|
||||
# Apply our environment after pip is ready
|
||||
env = environment.as_dictionary(prev_environment=env)
|
||||
|
||||
# check what Python version we're on
|
||||
which_python = call("which", "python", env=env, capture_stdout=True).strip()
|
||||
print(which_python)
|
||||
if which_python != str(venv_bin_path / "python"):
|
||||
msg = "python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it."
|
||||
raise errors.FatalError(msg)
|
||||
call("python", "-V", "-V", env=env)
|
||||
|
||||
# check what pip version we're on
|
||||
if not use_uv:
|
||||
assert (venv_bin_path / "pip").exists()
|
||||
which_pip = call("which", "pip", env=env, capture_stdout=True).strip()
|
||||
print(which_pip)
|
||||
if which_pip != str(venv_bin_path / "pip"):
|
||||
msg = "pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it."
|
||||
raise errors.FatalError(msg)
|
||||
call("pip", "--version", env=env)
|
||||
|
||||
config_is_arm64 = python_configuration.identifier.endswith("arm64")
|
||||
config_is_universal2 = python_configuration.identifier.endswith("universal2")
|
||||
|
||||
# Set MACOSX_DEPLOYMENT_TARGET, if the user didn't set it.
|
||||
# For arm64, the minimal deployment target is 11.0.
|
||||
# On x86_64 (or universal2), use 10.9 as a default.
|
||||
# CPython 3.12.6+ needs 10.13.
|
||||
# CPython 3.14.0 needs 10.15.
|
||||
if config_is_arm64:
|
||||
default_target = "11.0"
|
||||
elif Version(python_configuration.version) >= Version("3.14"):
|
||||
default_target = "10.15"
|
||||
elif Version(python_configuration.version) >= Version("3.12"):
|
||||
default_target = "10.13"
|
||||
elif python_configuration.identifier.startswith("pp") and Version(
|
||||
python_configuration.version
|
||||
) >= Version("3.9"):
|
||||
default_target = "10.15"
|
||||
else:
|
||||
default_target = "10.9"
|
||||
env.setdefault("MACOSX_DEPLOYMENT_TARGET", default_target)
|
||||
|
||||
# This is a floor, it can't be set lower than the default_target.
|
||||
if Version(env["MACOSX_DEPLOYMENT_TARGET"]) < Version(default_target):
|
||||
log.warning(
|
||||
f"Bumping MACOSX_DEPLOYMENT_TARGET ({env['MACOSX_DEPLOYMENT_TARGET']}) to the minimum required ({default_target})."
|
||||
)
|
||||
env["MACOSX_DEPLOYMENT_TARGET"] = default_target
|
||||
|
||||
if config_is_arm64:
|
||||
# macOS 11 is the first OS with arm64 support, so the wheels
|
||||
# have that as a minimum.
|
||||
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-11.0-arm64")
|
||||
env.setdefault("ARCHFLAGS", "-arch arm64")
|
||||
elif config_is_universal2:
|
||||
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-10.9-universal2")
|
||||
env.setdefault("ARCHFLAGS", "-arch arm64 -arch x86_64")
|
||||
elif python_configuration.identifier.endswith("x86_64"):
|
||||
# even on the macos11.0 Python installer, on the x86_64 side it's
|
||||
# compatible back to 10.9.
|
||||
env.setdefault("_PYTHON_HOST_PLATFORM", "macosx-10.9-x86_64")
|
||||
env.setdefault("ARCHFLAGS", "-arch x86_64")
|
||||
|
||||
building_arm64 = config_is_arm64 or config_is_universal2
|
||||
if building_arm64 and get_macos_version() < (10, 16) and "SDKROOT" not in env:
|
||||
# xcode 12.2 or higher can build arm64 on macos 10.15 or below, but
|
||||
# needs the correct SDK selected.
|
||||
sdks = get_macos_sdks()
|
||||
|
||||
# Different versions of Xcode contain different SDK versions...
|
||||
# we're happy with anything newer than macOS 11.0
|
||||
arm64_compatible_sdks = [s for s in sdks if not s.startswith("macosx10.")]
|
||||
|
||||
if not arm64_compatible_sdks:
|
||||
log.warning(
|
||||
unwrap(
|
||||
"""
|
||||
SDK for building arm64-compatible wheels not found. You need Xcode 12.2 or later
|
||||
to build universal2 or arm64 wheels.
|
||||
"""
|
||||
)
|
||||
)
|
||||
else:
|
||||
env.setdefault("SDKROOT", arm64_compatible_sdks[0])
|
||||
|
||||
log.step("Installing build tools...")
|
||||
match build_frontend:
|
||||
case "pip":
|
||||
call(
|
||||
"pip",
|
||||
"install",
|
||||
"--upgrade",
|
||||
"delocate",
|
||||
*constraint_flags(dependency_constraint),
|
||||
env=env,
|
||||
)
|
||||
case "build":
|
||||
call(
|
||||
"pip",
|
||||
"install",
|
||||
"--upgrade",
|
||||
"delocate",
|
||||
"build[virtualenv]",
|
||||
*constraint_flags(dependency_constraint),
|
||||
env=env,
|
||||
)
|
||||
case "build[uv]":
|
||||
assert uv_path is not None
|
||||
call(
|
||||
uv_path,
|
||||
"pip",
|
||||
"install",
|
||||
"--python",
|
||||
which_python,
|
||||
"--upgrade",
|
||||
"delocate",
|
||||
"build[virtualenv, uv]",
|
||||
*constraint_flags(dependency_constraint),
|
||||
env=env,
|
||||
)
|
||||
case "uv":
|
||||
assert uv_path is not None
|
||||
call(
|
||||
uv_path,
|
||||
"pip",
|
||||
"install",
|
||||
"--python",
|
||||
which_python,
|
||||
"--upgrade",
|
||||
"delocate",
|
||||
*constraint_flags(dependency_constraint),
|
||||
env=env,
|
||||
)
|
||||
case "pyodide-build":
|
||||
msg = "The 'pyodide-build' build frontend is not supported on this platform"
|
||||
raise errors.FatalError(msg)
|
||||
case _:
|
||||
assert_never(build_frontend)
|
||||
|
||||
return base_python, env
|
||||
|
||||
|
||||
def build(options: Options, tmp_path: Path) -> None:
|
||||
python_configurations = get_python_configurations(
|
||||
options.globals.build_selector, options.globals.architectures
|
||||
)
|
||||
|
||||
if not python_configurations:
|
||||
return
|
||||
|
||||
try:
|
||||
before_all_options_identifier = python_configurations[0].identifier
|
||||
before_all_options = options.build_options(before_all_options_identifier)
|
||||
|
||||
if before_all_options.before_all:
|
||||
log.step("Running before_all...")
|
||||
env = before_all_options.environment.as_dictionary(prev_environment=os.environ)
|
||||
env.setdefault("MACOSX_DEPLOYMENT_TARGET", "10.9")
|
||||
before_all_prepared = prepare_command(
|
||||
before_all_options.before_all, project=".", package=before_all_options.package_dir
|
||||
)
|
||||
shell(before_all_prepared, env=env)
|
||||
|
||||
built_wheels: list[Path] = []
|
||||
|
||||
for config in python_configurations:
|
||||
build_options = options.build_options(config.identifier)
|
||||
build_frontend = build_options.build_frontend
|
||||
use_uv = build_frontend.name in {"build[uv]", "uv"}
|
||||
uv_path = find_uv()
|
||||
if use_uv and uv_path is None:
|
||||
msg = "uv not found"
|
||||
raise AssertionError(msg)
|
||||
pip = ["pip"] if not use_uv else [str(uv_path), "pip"]
|
||||
log.build_start(config.identifier)
|
||||
|
||||
identifier_tmp_dir = tmp_path / config.identifier
|
||||
identifier_tmp_dir.mkdir()
|
||||
built_wheel_dir = identifier_tmp_dir / "built_wheel"
|
||||
repaired_wheel_dir = identifier_tmp_dir / "repaired_wheel"
|
||||
|
||||
config_is_arm64 = config.identifier.endswith("arm64")
|
||||
config_is_universal2 = config.identifier.endswith("universal2")
|
||||
|
||||
constraints_path = build_options.dependency_constraints.get_for_python_version(
|
||||
version=config.version, tmp_dir=identifier_tmp_dir
|
||||
)
|
||||
|
||||
base_python, env = setup_python(
|
||||
identifier_tmp_dir / "build",
|
||||
config,
|
||||
constraints_path,
|
||||
build_options.environment,
|
||||
build_frontend.name,
|
||||
)
|
||||
env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
|
||||
pip_version = None if use_uv else get_pip_version(env)
|
||||
|
||||
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
|
||||
if compatible_wheel:
|
||||
log.step_end()
|
||||
print(
|
||||
f"\nFound previously built wheel {compatible_wheel.name}, that's compatible with {config.identifier}. Skipping build step..."
|
||||
)
|
||||
repaired_wheel = compatible_wheel
|
||||
else:
|
||||
if build_options.before_build:
|
||||
log.step("Running before_build...")
|
||||
before_build_prepared = prepare_command(
|
||||
build_options.before_build, project=".", package=build_options.package_dir
|
||||
)
|
||||
shell(before_build_prepared, env=env)
|
||||
|
||||
log.step("Building wheel...")
|
||||
built_wheel_dir.mkdir()
|
||||
|
||||
extra_flags = get_build_frontend_extra_flags(
|
||||
build_frontend,
|
||||
build_options.build_verbosity,
|
||||
prepare_config_settings(
|
||||
build_options.config_settings,
|
||||
project=Path.cwd(),
|
||||
package=build_options.package_dir,
|
||||
),
|
||||
)
|
||||
|
||||
build_env = env.copy()
|
||||
|
||||
match build_frontend.name:
|
||||
case "pip":
|
||||
# Path.resolve() is needed. Without it pip wheel may try to fetch package from pypi.org
|
||||
# see https://github.com/pypa/cibuildwheel/pull/369
|
||||
call(
|
||||
"python",
|
||||
"-m",
|
||||
"pip",
|
||||
"wheel",
|
||||
build_options.package_dir.resolve(),
|
||||
f"--wheel-dir={built_wheel_dir}",
|
||||
"--no-deps",
|
||||
*extra_flags,
|
||||
env=build_env,
|
||||
)
|
||||
case "build" | "build[uv]":
|
||||
if (
|
||||
use_uv
|
||||
and "--no-isolation" not in extra_flags
|
||||
and "-n" not in extra_flags
|
||||
):
|
||||
extra_flags.append("--installer=uv")
|
||||
call(
|
||||
"python",
|
||||
"-m",
|
||||
"build",
|
||||
build_options.package_dir,
|
||||
"--wheel",
|
||||
f"--outdir={built_wheel_dir}",
|
||||
*extra_flags,
|
||||
env=build_env,
|
||||
)
|
||||
case "uv":
|
||||
assert uv_path is not None
|
||||
call(
|
||||
uv_path,
|
||||
"build",
|
||||
f"--python={base_python}",
|
||||
build_options.package_dir,
|
||||
"--wheel",
|
||||
f"--out-dir={built_wheel_dir}",
|
||||
*extra_flags,
|
||||
env=build_env,
|
||||
)
|
||||
case "pyodide-build":
|
||||
msg = "The 'pyodide-build' build frontend is not supported on this platform"
|
||||
raise errors.FatalError(msg)
|
||||
case _:
|
||||
assert_never(build_frontend)
|
||||
|
||||
try:
|
||||
built_wheel = next(built_wheel_dir.glob("*.whl"))
|
||||
except StopIteration:
|
||||
raise errors.BuildProducedNoWheelError() from None
|
||||
|
||||
repaired_wheel_dir.mkdir()
|
||||
|
||||
if built_wheel.name.endswith("none-any.whl"):
|
||||
raise errors.NonPlatformWheelError()
|
||||
|
||||
if build_options.repair_command:
|
||||
log.step("Repairing wheel...")
|
||||
|
||||
if config_is_universal2:
|
||||
delocate_archs = "x86_64,arm64"
|
||||
elif config_is_arm64:
|
||||
delocate_archs = "arm64"
|
||||
else:
|
||||
delocate_archs = "x86_64"
|
||||
|
||||
repair_command_prepared = prepare_command(
|
||||
build_options.repair_command,
|
||||
wheel=built_wheel,
|
||||
dest_dir=repaired_wheel_dir,
|
||||
delocate_archs=delocate_archs,
|
||||
package=build_options.package_dir,
|
||||
project=".",
|
||||
)
|
||||
shell(repair_command_prepared, env=env)
|
||||
else:
|
||||
shutil.move(str(built_wheel), repaired_wheel_dir)
|
||||
|
||||
try:
|
||||
repaired_wheel = next(repaired_wheel_dir.glob("*.whl"))
|
||||
except StopIteration:
|
||||
raise errors.RepairStepProducedNoWheelError() from None
|
||||
|
||||
if repaired_wheel.name in {wheel.name for wheel in built_wheels}:
|
||||
raise errors.AlreadyBuiltWheelError(repaired_wheel.name)
|
||||
|
||||
log.step_end()
|
||||
|
||||
run_audit(tmp_dir=tmp_path, build_options=build_options, wheel=repaired_wheel)
|
||||
|
||||
if build_options.test_command and build_options.test_selector(config.identifier):
|
||||
machine_arch = platform.machine()
|
||||
testing_archs: list[Literal["x86_64", "arm64"]]
|
||||
|
||||
if config_is_arm64:
|
||||
testing_archs = ["arm64"]
|
||||
elif config_is_universal2:
|
||||
testing_archs = ["x86_64", "arm64"]
|
||||
else:
|
||||
testing_archs = ["x86_64"]
|
||||
|
||||
for testing_arch in testing_archs:
|
||||
if config_is_universal2:
|
||||
arch_specific_identifier = f"{config.identifier}:{testing_arch}"
|
||||
if not build_options.test_selector(arch_specific_identifier):
|
||||
continue
|
||||
|
||||
if machine_arch == "x86_64" and testing_arch == "arm64":
|
||||
if config_is_arm64:
|
||||
log.warning(
|
||||
unwrap(
|
||||
"""
|
||||
While arm64 wheels can be built on x86_64, they cannot be
|
||||
tested. Consider building arm64 wheels natively, if your CI
|
||||
provider offers this. To silence this warning, set
|
||||
`CIBW_TEST_SKIP: "*-macosx_arm64"`.
|
||||
"""
|
||||
)
|
||||
)
|
||||
elif config_is_universal2:
|
||||
log.warning(
|
||||
unwrap(
|
||||
"""
|
||||
While universal2 wheels can be built on x86_64, the arm64 part
|
||||
of the wheel cannot be tested on x86_64. Consider building
|
||||
universal2 wheels on an arm64 runner, if your CI provider offers
|
||||
this. Notably, an arm64 runner can also test the x86_64 part of
|
||||
the wheel, through Rosetta emulation. To silence this warning,
|
||||
set `CIBW_TEST_SKIP: "*-macosx_universal2:arm64"`.
|
||||
"""
|
||||
)
|
||||
)
|
||||
else:
|
||||
msg = "unreachable"
|
||||
raise RuntimeError(msg)
|
||||
|
||||
# skip this test
|
||||
continue
|
||||
|
||||
log.step(
|
||||
"Testing wheel..."
|
||||
if testing_arch == machine_arch
|
||||
else f"Testing wheel on {testing_arch}..."
|
||||
)
|
||||
|
||||
arch_prefix = []
|
||||
uv_arch_args = []
|
||||
if testing_arch != machine_arch:
|
||||
if machine_arch == "arm64" and testing_arch == "x86_64":
|
||||
# rosetta2 will provide the emulation with just the arch prefix.
|
||||
arch_prefix = ["arch", "-x86_64"]
|
||||
uv_arch_args = ["--python-platform", "x86_64-apple-darwin"]
|
||||
else:
|
||||
msg = f"don't know how to emulate {testing_arch} on {machine_arch}"
|
||||
raise RuntimeError(msg)
|
||||
|
||||
# define a custom 'call' function that adds the arch prefix each time
|
||||
call_with_arch = functools.partial(call, *arch_prefix)
|
||||
shell_with_arch = functools.partial(call, *arch_prefix, "/bin/sh", "-c")
|
||||
|
||||
# set up a virtual environment to install and test from, to make sure
|
||||
# there are no dependencies that were pulled in at build time.
|
||||
venv_dir = identifier_tmp_dir / f"venv-test-{testing_arch}"
|
||||
virtualenv_env = virtualenv(
|
||||
config.version,
|
||||
base_python,
|
||||
venv_dir,
|
||||
None,
|
||||
use_uv=use_uv,
|
||||
env=env,
|
||||
pip_version=pip_version,
|
||||
)
|
||||
if use_uv:
|
||||
pip_install = functools.partial(call, *pip, "install", *uv_arch_args)
|
||||
else:
|
||||
pip_install = functools.partial(call_with_arch, *pip, "install")
|
||||
|
||||
virtualenv_env["MACOSX_DEPLOYMENT_TARGET"] = get_test_macosx_deployment_target()
|
||||
|
||||
virtualenv_env = build_options.test_environment.as_dictionary(
|
||||
prev_environment=virtualenv_env
|
||||
)
|
||||
|
||||
# check that we are using the Python from the virtual environment
|
||||
call_with_arch("which", "python", env=virtualenv_env)
|
||||
|
||||
if build_options.before_test:
|
||||
before_test_prepared = prepare_command(
|
||||
build_options.before_test,
|
||||
project=".",
|
||||
package=build_options.package_dir,
|
||||
)
|
||||
shell_with_arch(before_test_prepared, env=virtualenv_env)
|
||||
|
||||
# install the wheel
|
||||
pip_install(
|
||||
f"{repaired_wheel}{build_options.test_extras}",
|
||||
env=virtualenv_env,
|
||||
)
|
||||
|
||||
# test the wheel
|
||||
if build_options.test_requires:
|
||||
pip_install(
|
||||
*build_options.test_requires,
|
||||
env=virtualenv_env,
|
||||
)
|
||||
|
||||
# run the tests from a temp dir, with an absolute path in the command
|
||||
# (this ensures that Python runs the tests against the installed wheel
|
||||
# and not the repo code)
|
||||
test_command_prepared = prepare_command(
|
||||
build_options.test_command,
|
||||
project=Path.cwd(),
|
||||
package=build_options.package_dir.resolve(),
|
||||
wheel=repaired_wheel,
|
||||
)
|
||||
|
||||
test_cwd = identifier_tmp_dir / "test_cwd"
|
||||
|
||||
if build_options.test_sources:
|
||||
# only create test_cwd if it doesn't already exist - it
|
||||
# may have been created during a previous `testing_arch`
|
||||
if not test_cwd.exists():
|
||||
test_cwd.mkdir()
|
||||
copy_test_sources(
|
||||
build_options.test_sources,
|
||||
Path.cwd(),
|
||||
test_cwd,
|
||||
)
|
||||
else:
|
||||
# Use the test_fail.py file to raise a nice error if the user
|
||||
# tries to run tests in the cwd
|
||||
test_cwd.mkdir(exist_ok=True)
|
||||
(test_cwd / "test_fail.py").write_text(
|
||||
resources.TEST_FAIL_CWD_FILE.read_text()
|
||||
)
|
||||
|
||||
shell_with_arch(test_command_prepared, cwd=test_cwd, env=virtualenv_env)
|
||||
|
||||
# we're all done here; move it to output (overwrite existing)
|
||||
output_wheel = None
|
||||
if compatible_wheel is None:
|
||||
output_wheel = build_options.output_dir.joinpath(repaired_wheel.name)
|
||||
moved_wheel = move_file(repaired_wheel, output_wheel)
|
||||
if moved_wheel != output_wheel.resolve():
|
||||
log.warning(
|
||||
f"{repaired_wheel} was moved to {moved_wheel} instead of {output_wheel}"
|
||||
)
|
||||
built_wheels.append(output_wheel)
|
||||
|
||||
# clean up
|
||||
shutil.rmtree(identifier_tmp_dir)
|
||||
|
||||
log.build_end(output_wheel)
|
||||
except subprocess.CalledProcessError as error:
|
||||
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
|
||||
raise errors.FatalError(msg) from error
|
||||
@@ -1,636 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {
|
||||
"cibuildwheel.architecture",
|
||||
"cibuildwheel.audit",
|
||||
"cibuildwheel.frontend",
|
||||
"cibuildwheel.logger",
|
||||
"cibuildwheel.util",
|
||||
"cibuildwheel.util.cmd",
|
||||
"cibuildwheel.util.file",
|
||||
"cibuildwheel.util.helpers",
|
||||
"cibuildwheel.util.packaging",
|
||||
"cibuildwheel.util.python_build_standalone",
|
||||
"cibuildwheel.venv",
|
||||
"filelock",
|
||||
"json",
|
||||
"pathlib",
|
||||
"shutil",
|
||||
"subprocess",
|
||||
"tempfile",
|
||||
"tomllib",
|
||||
}
|
||||
|
||||
import dataclasses
|
||||
import functools
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tomllib
|
||||
import typing
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
from typing import Final, TypedDict
|
||||
|
||||
from filelock import FileLock
|
||||
|
||||
from cibuildwheel import errors
|
||||
from cibuildwheel.architecture import Architecture
|
||||
from cibuildwheel.audit import run_audit
|
||||
from cibuildwheel.frontend import get_build_frontend_extra_flags, prepare_config_settings
|
||||
from cibuildwheel.logger import log
|
||||
from cibuildwheel.util import resources
|
||||
from cibuildwheel.util.cmd import call, shell
|
||||
from cibuildwheel.util.file import (
|
||||
CIBW_CACHE_PATH,
|
||||
copy_test_sources,
|
||||
download,
|
||||
extract_tar,
|
||||
extract_zip,
|
||||
move_file,
|
||||
remove_on_error,
|
||||
)
|
||||
from cibuildwheel.util.helpers import prepare_command, unwrap, unwrap_preserving_paragraphs
|
||||
from cibuildwheel.util.packaging import find_compatible_wheel, get_pip_version
|
||||
from cibuildwheel.util.python_build_standalone import (
|
||||
PythonBuildStandaloneError,
|
||||
create_python_build_standalone_environment,
|
||||
)
|
||||
from cibuildwheel.venv import constraint_flags, virtualenv
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Set
|
||||
|
||||
from cibuildwheel.environment import ParsedEnvironment
|
||||
from cibuildwheel.options import Options
|
||||
from cibuildwheel.selector import BuildSelector
|
||||
|
||||
IS_WIN: Final[bool] = sys.platform.startswith("win")
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, kw_only=True)
|
||||
class PythonConfiguration:
|
||||
version: str
|
||||
identifier: str
|
||||
default_pyodide_version: str
|
||||
node_version: str
|
||||
sha256: str = ""
|
||||
|
||||
|
||||
class PyodideXBuildEnvInfoVersionRange(TypedDict):
|
||||
min: str | None
|
||||
max: str | None
|
||||
|
||||
|
||||
class PyodideXBuildEnvInfo(TypedDict):
|
||||
version: str
|
||||
python: str
|
||||
emscripten: str
|
||||
pyodide_build: PyodideXBuildEnvInfoVersionRange
|
||||
compatible: bool
|
||||
|
||||
|
||||
@functools.cache
|
||||
def ensure_node(major_version: str) -> Path:
|
||||
with resources.NODEJS.open("rb") as f:
|
||||
loaded_file = tomllib.load(f)
|
||||
version = str(loaded_file[major_version])
|
||||
base_url = str(loaded_file["url"])
|
||||
ext = "zip" if IS_WIN else "tar.xz"
|
||||
platform = "win" if IS_WIN else ("darwin" if sys.platform.startswith("darwin") else "linux")
|
||||
linux_arch = Architecture.native_arch("linux")
|
||||
assert linux_arch is not None
|
||||
arch = {"x86_64": "x64", "i686": "x86", "aarch64": "arm64"}.get(
|
||||
linux_arch.value, linux_arch.value
|
||||
)
|
||||
name = f"node-{version}-{platform}-{arch}"
|
||||
path = CIBW_CACHE_PATH / name
|
||||
with FileLock(str(path) + ".lock"):
|
||||
if not path.exists():
|
||||
url = f"{base_url}{version}/{name}.{ext}"
|
||||
with TemporaryDirectory() as tmp_path:
|
||||
archive = Path(tmp_path) / f"{name}.{ext}"
|
||||
download(url, archive)
|
||||
with remove_on_error(path):
|
||||
if ext == "zip":
|
||||
extract_zip(archive, path.parent)
|
||||
else:
|
||||
extract_tar(archive, path.parent)
|
||||
assert path.exists()
|
||||
if not IS_WIN:
|
||||
return path / "bin"
|
||||
return path
|
||||
|
||||
|
||||
def install_emscripten(env: dict[str, str], version: str, xbuildenv_cache_path: Path) -> Path:
|
||||
"""Install Emscripten via pyodide-build, which also applies Pyodide-specific patches."""
|
||||
emscripten_dir = Path(
|
||||
call("pyodide", "config", "get", "emscripten_dir", env=env, capture_stdout=True).strip()
|
||||
)
|
||||
with FileLock(CIBW_CACHE_PATH / "emscripten.lock"):
|
||||
if emscripten_dir.exists():
|
||||
return emscripten_dir
|
||||
with remove_on_error(emscripten_dir):
|
||||
call(
|
||||
"pyodide",
|
||||
"xbuildenv",
|
||||
"install-emscripten",
|
||||
"--force",
|
||||
"--version",
|
||||
version,
|
||||
"--path",
|
||||
str(xbuildenv_cache_path),
|
||||
env=env,
|
||||
cwd=CIBW_CACHE_PATH,
|
||||
)
|
||||
assert emscripten_dir.exists()
|
||||
return emscripten_dir
|
||||
|
||||
|
||||
def get_all_xbuildenv_version_info(env: dict[str, str]) -> list[PyodideXBuildEnvInfo]:
|
||||
xbuildenvs_info_str = call(
|
||||
"pyodide",
|
||||
"xbuildenv",
|
||||
"search",
|
||||
"--json",
|
||||
"--all",
|
||||
env=env,
|
||||
cwd=CIBW_CACHE_PATH,
|
||||
capture_stdout=True,
|
||||
).strip()
|
||||
|
||||
xbuildenvs_info = json.loads(xbuildenvs_info_str)
|
||||
|
||||
if "environments" not in xbuildenvs_info:
|
||||
msg = f"Invalid xbuildenvs info, got {xbuildenvs_info}"
|
||||
raise ValueError(msg)
|
||||
|
||||
return typing.cast("list[PyodideXBuildEnvInfo]", xbuildenvs_info["environments"])
|
||||
|
||||
|
||||
def get_xbuildenv_version_info(
|
||||
env: dict[str, str], version: str, pyodide_build_version: str
|
||||
) -> PyodideXBuildEnvInfo:
|
||||
xbuildenvs_info = get_all_xbuildenv_version_info(env)
|
||||
for xbuildenv_info in xbuildenvs_info:
|
||||
if xbuildenv_info["version"] == version:
|
||||
return xbuildenv_info
|
||||
|
||||
msg = unwrap(f"""
|
||||
Could not find Pyodide cross-build environment version {version} in the available
|
||||
versions as reported by pyodide-build v{pyodide_build_version}.
|
||||
Available pyodide xbuildenv versions are:
|
||||
{", ".join(e["version"] for e in xbuildenvs_info if e["compatible"])}
|
||||
""")
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
|
||||
# The default pyodide xbuildenv version that's specified in
|
||||
# build-platforms.toml is compatible with the pyodide-build version that's
|
||||
# pinned in the bundled constraints file. But if the user changes
|
||||
# pyodide-version and/or dependency-constraints in the cibuildwheel config, we
|
||||
# need to check if the xbuildenv version is compatible with the pyodide-build
|
||||
# version.
|
||||
def validate_pyodide_build_version(
|
||||
xbuildenv_info: PyodideXBuildEnvInfo, pyodide_build_version: str
|
||||
) -> None:
|
||||
"""
|
||||
Validate the Pyodide version is compatible with the installed
|
||||
pyodide-build version.
|
||||
"""
|
||||
|
||||
pyodide_version = xbuildenv_info["version"]
|
||||
|
||||
if not xbuildenv_info["compatible"]:
|
||||
msg = unwrap_preserving_paragraphs(f"""
|
||||
The Pyodide xbuildenv version {pyodide_version} is not compatible
|
||||
with the pyodide-build version {pyodide_build_version}. Please use
|
||||
the 'pyodide xbuildenv search --all' command to find a compatible
|
||||
version.
|
||||
|
||||
Set the pyodide-build version using the `dependency-constraints`
|
||||
option, or set the Pyodide xbuildenv version using the
|
||||
`pyodide-version` option.
|
||||
""")
|
||||
raise errors.FatalError(msg)
|
||||
|
||||
|
||||
def validate_pyodide_target_python(
|
||||
xbuildenv_info: PyodideXBuildEnvInfo, python_configuration: PythonConfiguration
|
||||
) -> None:
|
||||
"""
|
||||
Validate that the resolved Pyodide xbuildenv targets the same Python version
|
||||
as the identifier being built.
|
||||
|
||||
This is to catch the case where a global ``pyodide-version`` is applied to an
|
||||
identifier whose Python version does not match, such as setting
|
||||
``pyodide-version = "3Y.0.0"`` (a Python 3.Y environment) while
|
||||
building ``cp3X-pyodide_wasm32``.
|
||||
"""
|
||||
xbuildenv_version = xbuildenv_info["version"]
|
||||
identifier = python_configuration.identifier
|
||||
expected_version = python_configuration.version
|
||||
default_version = python_configuration.default_pyodide_version
|
||||
xbuildenv_python_minor = ".".join(xbuildenv_info["python"].split(".")[:2])
|
||||
if xbuildenv_python_minor != expected_version:
|
||||
msg = unwrap_preserving_paragraphs(f"""
|
||||
The `pyodide-version` option is set to {xbuildenv_version}, which
|
||||
provides Python {xbuildenv_python_minor}, but the {identifier} build
|
||||
needs Python {expected_version}.
|
||||
|
||||
Either remove `pyodide-version` so {identifier} uses its default
|
||||
({default_version}), or stop building {identifier} so that
|
||||
`pyodide-version` only applies to an identifier it matches.
|
||||
""")
|
||||
raise errors.ConfigurationError(msg)
|
||||
|
||||
|
||||
def install_xbuildenv(env: dict[str, str], xbuildenv_cache_path: Path, pyodide_version: str) -> str:
|
||||
"""Install a particular Pyodide xbuildenv version and set a path to the Pyodide root."""
|
||||
pyodide_root = xbuildenv_cache_path / pyodide_version / "xbuildenv" / "pyodide-root"
|
||||
|
||||
with FileLock(CIBW_CACHE_PATH / "xbuildenv.lock"):
|
||||
if pyodide_root.exists():
|
||||
return str(pyodide_root)
|
||||
|
||||
# We don't want to mutate env but we need to delete any existing
|
||||
# PYODIDE_ROOT so copy it first.
|
||||
env = dict(env)
|
||||
env.pop("PYODIDE_ROOT", None)
|
||||
|
||||
# Install the xbuildenv
|
||||
with remove_on_error(xbuildenv_cache_path / pyodide_version):
|
||||
call(
|
||||
"pyodide",
|
||||
"xbuildenv",
|
||||
"install",
|
||||
"--path",
|
||||
str(xbuildenv_cache_path),
|
||||
pyodide_version,
|
||||
env=env,
|
||||
cwd=CIBW_CACHE_PATH,
|
||||
)
|
||||
assert pyodide_root.exists()
|
||||
|
||||
return str(pyodide_root)
|
||||
|
||||
|
||||
def get_base_python(tmp: Path, python_configuration: PythonConfiguration) -> Path:
|
||||
try:
|
||||
return create_python_build_standalone_environment(
|
||||
python_version=python_configuration.version,
|
||||
temp_dir=tmp,
|
||||
cache_dir=CIBW_CACHE_PATH,
|
||||
)
|
||||
except PythonBuildStandaloneError as e:
|
||||
msg = unwrap(f"""
|
||||
Failed to create a Python build environment:
|
||||
{e}
|
||||
""")
|
||||
raise errors.FatalError(msg) from e
|
||||
|
||||
|
||||
def setup_python(
|
||||
tmp: Path,
|
||||
python_configuration: PythonConfiguration,
|
||||
constraints_path: Path | None,
|
||||
environment: ParsedEnvironment,
|
||||
user_pyodide_version: str | None,
|
||||
) -> dict[str, str]:
|
||||
log.step("Installing a base python environment...")
|
||||
base_python = get_base_python(tmp / "base", python_configuration)
|
||||
|
||||
log.step("Setting up build environment...")
|
||||
pyodide_version = user_pyodide_version or python_configuration.default_pyodide_version
|
||||
venv_path = tmp / "venv"
|
||||
env = virtualenv(python_configuration.version, base_python, venv_path, None, use_uv=False)
|
||||
venv_bin_path = venv_path / "bin"
|
||||
assert venv_bin_path.exists()
|
||||
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
|
||||
|
||||
# upgrade pip to the version matching our constraints
|
||||
# if necessary, reinstall it to ensure that it's available on PATH as 'pip'
|
||||
call(
|
||||
"python",
|
||||
"-m",
|
||||
"pip",
|
||||
"install",
|
||||
"--upgrade",
|
||||
"pip",
|
||||
*constraint_flags(constraints_path),
|
||||
env=env,
|
||||
cwd=venv_path,
|
||||
)
|
||||
|
||||
env = environment.as_dictionary(prev_environment=env)
|
||||
|
||||
# check what Python version we're on
|
||||
which_python = call("which", "python", env=env, capture_stdout=True).strip()
|
||||
print(which_python)
|
||||
if which_python != str(venv_bin_path / "python"):
|
||||
msg = "python available on PATH doesn't match our venv instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it."
|
||||
raise errors.FatalError(msg)
|
||||
call("python", "-V", "-V", env=env)
|
||||
|
||||
# check what pip version we're on
|
||||
assert (venv_bin_path / "pip").exists()
|
||||
which_pip = call("which", "pip", env=env, capture_stdout=True).strip()
|
||||
print(which_pip)
|
||||
if which_pip != str(venv_bin_path / "pip"):
|
||||
msg = "pip available on PATH doesn't match our venv instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it."
|
||||
raise errors.FatalError(msg)
|
||||
call("pip", "--version", env=env)
|
||||
|
||||
log.step("Installing build tools...")
|
||||
call(
|
||||
"pip",
|
||||
"install",
|
||||
"--upgrade",
|
||||
"pyodide-build",
|
||||
*constraint_flags(constraints_path),
|
||||
env=env,
|
||||
)
|
||||
|
||||
pyodide_build_version = call(
|
||||
"python",
|
||||
"-c",
|
||||
"from importlib.metadata import version; print(version('pyodide-build'))",
|
||||
env=env,
|
||||
capture_stdout=True,
|
||||
).strip()
|
||||
|
||||
xbuildenv_info = get_xbuildenv_version_info(env, pyodide_version, pyodide_build_version)
|
||||
validate_pyodide_build_version(
|
||||
xbuildenv_info=xbuildenv_info,
|
||||
pyodide_build_version=pyodide_build_version,
|
||||
)
|
||||
validate_pyodide_target_python(
|
||||
xbuildenv_info=xbuildenv_info,
|
||||
python_configuration=python_configuration,
|
||||
)
|
||||
|
||||
xbuildenv_cache_path = CIBW_CACHE_PATH / f"pyodide-build-{pyodide_build_version}"
|
||||
|
||||
log.step(f"Installing Pyodide xbuildenv version: {pyodide_version} ...")
|
||||
env["PYODIDE_ROOT"] = install_xbuildenv(env, xbuildenv_cache_path, pyodide_version)
|
||||
|
||||
emscripten_version = xbuildenv_info["emscripten"]
|
||||
log.step(
|
||||
f"Installing Emscripten {emscripten_version} and applying Pyodide-specific patches ..."
|
||||
)
|
||||
emscripten_dir = install_emscripten(env, emscripten_version, xbuildenv_cache_path)
|
||||
|
||||
env["PATH"] = os.pathsep.join([str(emscripten_dir), env["PATH"]])
|
||||
|
||||
return env
|
||||
|
||||
|
||||
def all_python_configurations() -> list[PythonConfiguration]:
|
||||
full_python_configs = resources.read_python_configs("pyodide")
|
||||
return [PythonConfiguration(**item) for item in full_python_configs]
|
||||
|
||||
|
||||
def get_python_configurations(
|
||||
build_selector: BuildSelector,
|
||||
architectures: Set[Architecture], # noqa: ARG001
|
||||
) -> list[PythonConfiguration]:
|
||||
return [c for c in all_python_configurations() if build_selector(c.identifier)]
|
||||
|
||||
|
||||
def build(options: Options, tmp_path: Path) -> None:
|
||||
python_configurations = get_python_configurations(
|
||||
options.globals.build_selector, options.globals.architectures
|
||||
)
|
||||
|
||||
if not python_configurations:
|
||||
return
|
||||
|
||||
try:
|
||||
before_all_options_identifier = python_configurations[0].identifier
|
||||
before_all_options = options.build_options(before_all_options_identifier)
|
||||
|
||||
if before_all_options.before_all:
|
||||
log.step("Running before_all...")
|
||||
env = before_all_options.environment.as_dictionary(prev_environment=os.environ)
|
||||
before_all_prepared = prepare_command(
|
||||
before_all_options.before_all, project=".", package=before_all_options.package_dir
|
||||
)
|
||||
shell(before_all_prepared, env=env)
|
||||
|
||||
built_wheels: list[Path] = []
|
||||
|
||||
for config in python_configurations:
|
||||
build_options = options.build_options(config.identifier)
|
||||
build_frontend = build_options.build_frontend
|
||||
|
||||
log.build_start(config.identifier)
|
||||
|
||||
identifier_tmp_dir = tmp_path / config.identifier
|
||||
|
||||
built_wheel_dir = identifier_tmp_dir / "built_wheel"
|
||||
repaired_wheel_dir = identifier_tmp_dir / "repaired_wheel"
|
||||
identifier_tmp_dir.mkdir()
|
||||
built_wheel_dir.mkdir()
|
||||
repaired_wheel_dir.mkdir()
|
||||
|
||||
constraints_path = build_options.dependency_constraints.get_for_python_version(
|
||||
version=config.version, variant="pyodide", tmp_dir=identifier_tmp_dir
|
||||
)
|
||||
|
||||
env = setup_python(
|
||||
tmp=identifier_tmp_dir / "build",
|
||||
python_configuration=config,
|
||||
constraints_path=constraints_path,
|
||||
environment=build_options.environment,
|
||||
user_pyodide_version=build_options.pyodide_version,
|
||||
)
|
||||
env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
|
||||
pip_version = get_pip_version(env)
|
||||
# The Pyodide command line runner mounts all directories in the host
|
||||
# filesystem into the Pyodide file system, except for the custom
|
||||
# file systems /dev, /lib, /proc, and /tmp. Mounting the mount
|
||||
# points for alternate file systems causes some mysterious failure
|
||||
# of the process (it just quits without any clear error).
|
||||
#
|
||||
# Because of this, by default Pyodide can't see anything under /tmp.
|
||||
# This environment variable tells it also to mount our temp
|
||||
# directory.
|
||||
oldmounts = ""
|
||||
extra_mounts = [str(identifier_tmp_dir)]
|
||||
if Path.cwd().is_relative_to("/tmp"):
|
||||
extra_mounts.append(str(Path.cwd()))
|
||||
|
||||
if "_PYODIDE_EXTRA_MOUNTS" in env:
|
||||
oldmounts = env["_PYODIDE_EXTRA_MOUNTS"] + ":"
|
||||
env["_PYODIDE_EXTRA_MOUNTS"] = oldmounts + ":".join(extra_mounts)
|
||||
|
||||
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
|
||||
if compatible_wheel:
|
||||
log.step_end()
|
||||
print(
|
||||
f"\nFound previously built wheel {compatible_wheel.name}, that's compatible with {config.identifier}. Skipping build step..."
|
||||
)
|
||||
built_wheel = compatible_wheel
|
||||
else:
|
||||
if build_options.before_build:
|
||||
log.step("Running before_build...")
|
||||
before_build_prepared = prepare_command(
|
||||
build_options.before_build, project=".", package=build_options.package_dir
|
||||
)
|
||||
shell(before_build_prepared, env=env)
|
||||
|
||||
log.step("Building wheel...")
|
||||
|
||||
extra_flags = get_build_frontend_extra_flags(
|
||||
build_frontend,
|
||||
build_options.build_verbosity,
|
||||
prepare_config_settings(
|
||||
build_options.config_settings,
|
||||
project=Path.cwd(),
|
||||
package=build_options.package_dir,
|
||||
),
|
||||
)
|
||||
|
||||
call(
|
||||
"pyodide",
|
||||
"build",
|
||||
build_options.package_dir,
|
||||
f"--outdir={built_wheel_dir}",
|
||||
*extra_flags,
|
||||
env=env,
|
||||
)
|
||||
try:
|
||||
built_wheel = next(built_wheel_dir.glob("*.whl"))
|
||||
except StopIteration:
|
||||
raise errors.BuildProducedNoWheelError() from None
|
||||
|
||||
if built_wheel.name.endswith("none-any.whl"):
|
||||
raise errors.NonPlatformWheelError()
|
||||
|
||||
if build_options.repair_command:
|
||||
log.step("Repairing wheel...")
|
||||
|
||||
repair_command_prepared = prepare_command(
|
||||
build_options.repair_command,
|
||||
wheel=built_wheel,
|
||||
dest_dir=repaired_wheel_dir,
|
||||
package=build_options.package_dir,
|
||||
project=".",
|
||||
)
|
||||
shell(repair_command_prepared, env=env)
|
||||
log.step_end()
|
||||
else:
|
||||
shutil.move(str(built_wheel), repaired_wheel_dir)
|
||||
|
||||
try:
|
||||
repaired_wheel = next(repaired_wheel_dir.glob("*.whl"))
|
||||
except StopIteration:
|
||||
raise errors.RepairStepProducedNoWheelError() from None
|
||||
|
||||
if repaired_wheel.name in {wheel.name for wheel in built_wheels}:
|
||||
raise errors.AlreadyBuiltWheelError(repaired_wheel.name)
|
||||
|
||||
run_audit(tmp_dir=tmp_path, build_options=build_options, wheel=repaired_wheel)
|
||||
|
||||
if build_options.test_command and build_options.test_selector(config.identifier):
|
||||
log.step("Testing wheel...")
|
||||
|
||||
venv_dir = identifier_tmp_dir / "venv-test"
|
||||
# set up a virtual environment to install and test from, to make sure
|
||||
# there are no dependencies that were pulled in at build time.
|
||||
|
||||
virtualenv_env = env.copy()
|
||||
virtualenv_env["PATH"] = os.pathsep.join(
|
||||
[
|
||||
str(ensure_node(config.node_version)),
|
||||
virtualenv_env["PATH"],
|
||||
]
|
||||
)
|
||||
|
||||
# pyodide venv uses virtualenv under the hood
|
||||
# use the pip embedded with virtualenv & disable network updates
|
||||
virtualenv_create_env = virtualenv_env.copy()
|
||||
virtualenv_create_env["VIRTUALENV_PIP"] = pip_version
|
||||
virtualenv_create_env["VIRTUALENV_NO_PERIODIC_UPDATE"] = "1"
|
||||
|
||||
call("pyodide", "venv", venv_dir, env=virtualenv_create_env)
|
||||
|
||||
virtualenv_env["PATH"] = os.pathsep.join(
|
||||
[
|
||||
str(venv_dir / "bin"),
|
||||
virtualenv_env["PATH"],
|
||||
]
|
||||
)
|
||||
virtualenv_env["VIRTUAL_ENV"] = str(venv_dir)
|
||||
|
||||
virtualenv_env = build_options.test_environment.as_dictionary(
|
||||
prev_environment=virtualenv_env
|
||||
)
|
||||
|
||||
# check that we are using the Python from the virtual environment
|
||||
call("which", "python", env=virtualenv_env)
|
||||
|
||||
if build_options.before_test:
|
||||
before_test_prepared = prepare_command(
|
||||
build_options.before_test,
|
||||
project=".",
|
||||
package=build_options.package_dir,
|
||||
wheel=repaired_wheel,
|
||||
)
|
||||
shell(before_test_prepared, env=virtualenv_env)
|
||||
|
||||
# install the wheel
|
||||
call(
|
||||
"pip",
|
||||
"install",
|
||||
f"{repaired_wheel}{build_options.test_extras}",
|
||||
env=virtualenv_env,
|
||||
)
|
||||
|
||||
# test the wheel
|
||||
if build_options.test_requires:
|
||||
call("pip", "install", *build_options.test_requires, env=virtualenv_env)
|
||||
|
||||
# run the tests from a temp dir, with an absolute path in the command
|
||||
# (this ensures that Python runs the tests against the installed wheel
|
||||
# and not the repo code)
|
||||
test_command_prepared = prepare_command(
|
||||
build_options.test_command,
|
||||
project=Path.cwd(),
|
||||
package=build_options.package_dir.resolve(),
|
||||
)
|
||||
|
||||
test_cwd = identifier_tmp_dir / "test_cwd"
|
||||
test_cwd.mkdir(exist_ok=True)
|
||||
|
||||
if build_options.test_sources:
|
||||
copy_test_sources(
|
||||
build_options.test_sources,
|
||||
Path.cwd(),
|
||||
test_cwd,
|
||||
)
|
||||
else:
|
||||
# Use the test_fail.py file to raise a nice error if the user
|
||||
# tries to run tests in the cwd
|
||||
(test_cwd / "test_fail.py").write_text(resources.TEST_FAIL_CWD_FILE.read_text())
|
||||
|
||||
shell(test_command_prepared, cwd=test_cwd, env=virtualenv_env)
|
||||
|
||||
# we're all done here; move it to output (overwrite existing)
|
||||
output_wheel: Path | None = None
|
||||
if compatible_wheel is None:
|
||||
output_wheel = build_options.output_dir.joinpath(repaired_wheel.name)
|
||||
moved_wheel = move_file(repaired_wheel, output_wheel)
|
||||
if moved_wheel != output_wheel.resolve():
|
||||
log.warning(
|
||||
f"{repaired_wheel} was moved to {moved_wheel} instead of {output_wheel}"
|
||||
)
|
||||
built_wheels.append(output_wheel)
|
||||
log.build_end(output_wheel)
|
||||
|
||||
except subprocess.CalledProcessError as error:
|
||||
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
|
||||
raise errors.FatalError(msg) from error
|
||||
@@ -1,690 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {
|
||||
"cibuildwheel.architecture",
|
||||
"cibuildwheel.audit",
|
||||
"cibuildwheel.frontend",
|
||||
"cibuildwheel.logger",
|
||||
"cibuildwheel.util",
|
||||
"cibuildwheel.util.cmd",
|
||||
"cibuildwheel.util.file",
|
||||
"cibuildwheel.util.helpers",
|
||||
"cibuildwheel.util.packaging",
|
||||
"cibuildwheel.venv",
|
||||
"filelock",
|
||||
"pathlib",
|
||||
"platform",
|
||||
"shutil",
|
||||
"subprocess",
|
||||
"textwrap",
|
||||
"typing",
|
||||
}
|
||||
|
||||
import dataclasses
|
||||
import os
|
||||
import platform as platform_module
|
||||
import shutil
|
||||
import subprocess
|
||||
import textwrap
|
||||
from functools import cache
|
||||
from pathlib import Path
|
||||
from typing import assert_never
|
||||
|
||||
from filelock import FileLock
|
||||
|
||||
from cibuildwheel import errors
|
||||
from cibuildwheel.architecture import Architecture
|
||||
from cibuildwheel.audit import run_audit
|
||||
from cibuildwheel.frontend import (
|
||||
BuildFrontendName,
|
||||
get_build_frontend_extra_flags,
|
||||
prepare_config_settings,
|
||||
)
|
||||
from cibuildwheel.logger import log
|
||||
from cibuildwheel.util import resources
|
||||
from cibuildwheel.util.cmd import call, shell
|
||||
from cibuildwheel.util.file import (
|
||||
CIBW_CACHE_PATH,
|
||||
copy_test_sources,
|
||||
download,
|
||||
extract_zip,
|
||||
move_file,
|
||||
remove_on_error,
|
||||
)
|
||||
from cibuildwheel.util.helpers import prepare_command, unwrap
|
||||
from cibuildwheel.util.packaging import find_compatible_wheel, get_pip_version
|
||||
from cibuildwheel.venv import constraint_flags, find_uv, target_marker_env, virtualenv
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import MutableMapping, Sequence, Set
|
||||
|
||||
from cibuildwheel.environment import ParsedEnvironment
|
||||
from cibuildwheel.options import Options
|
||||
from cibuildwheel.selector import BuildSelector
|
||||
|
||||
|
||||
def get_nuget_args(
|
||||
version: str, arch: str, free_threaded: bool, output_directory: Path
|
||||
) -> list[str]:
|
||||
package_name = {
|
||||
"32": "pythonx86",
|
||||
"64": "python",
|
||||
"ARM64": "pythonarm64",
|
||||
# Aliases for platform.machine() return values
|
||||
"x86": "pythonx86",
|
||||
"AMD64": "python",
|
||||
}[arch]
|
||||
if free_threaded:
|
||||
package_name = f"{package_name}-freethreaded"
|
||||
return [
|
||||
package_name,
|
||||
"-Version",
|
||||
version,
|
||||
"-FallbackSource",
|
||||
"https://api.nuget.org/v3/index.json",
|
||||
"-OutputDirectory",
|
||||
str(output_directory),
|
||||
]
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, kw_only=True)
|
||||
class PythonConfiguration:
|
||||
version: str
|
||||
identifier: str
|
||||
url: str | None = None
|
||||
sha256: str = ""
|
||||
|
||||
@property
|
||||
def arch(self) -> str:
|
||||
return {
|
||||
"win32": "32",
|
||||
"win_amd64": "64",
|
||||
"win_arm64": "ARM64",
|
||||
}[self.identifier.split("-")[-1]]
|
||||
|
||||
|
||||
def all_python_configurations() -> list[PythonConfiguration]:
|
||||
config_dicts = resources.read_python_configs("windows")
|
||||
return [PythonConfiguration(**item) for item in config_dicts]
|
||||
|
||||
|
||||
def get_python_configurations(
|
||||
build_selector: BuildSelector,
|
||||
architectures: Set[Architecture],
|
||||
) -> list[PythonConfiguration]:
|
||||
python_configurations = all_python_configurations()
|
||||
|
||||
map_arch = {"32": Architecture.x86, "64": Architecture.AMD64, "ARM64": Architecture.ARM64}
|
||||
|
||||
# skip builds as required
|
||||
python_configurations = [
|
||||
c
|
||||
for c in python_configurations
|
||||
if build_selector(c.identifier) and map_arch[c.arch] in architectures
|
||||
]
|
||||
|
||||
return python_configurations
|
||||
|
||||
|
||||
@cache
|
||||
def _ensure_nuget() -> Path:
|
||||
nuget = CIBW_CACHE_PATH / "nuget.exe"
|
||||
with FileLock(str(nuget) + ".lock"):
|
||||
if not nuget.exists():
|
||||
with remove_on_error(nuget):
|
||||
download("https://dist.nuget.org/win-x86-commandline/latest/nuget.exe", nuget)
|
||||
return nuget
|
||||
|
||||
|
||||
def install_cpython(configuration: PythonConfiguration, arch: str | None = None) -> Path:
|
||||
version = configuration.version
|
||||
free_threaded = "t-" in configuration.identifier
|
||||
if arch is None:
|
||||
arch = configuration.arch
|
||||
base_output_dir = CIBW_CACHE_PATH / "nuget-cpython"
|
||||
nuget_args = get_nuget_args(version, arch, free_threaded, base_output_dir)
|
||||
installation_path = base_output_dir / (nuget_args[0] + "." + version) / "tools"
|
||||
free_threaded_str = "-freethreaded" if free_threaded else ""
|
||||
with FileLock(str(base_output_dir) + f"-{version}{free_threaded_str}-{arch}.lock"):
|
||||
if not installation_path.exists():
|
||||
nuget = _ensure_nuget()
|
||||
with remove_on_error(installation_path.parent):
|
||||
call(nuget, "install", *nuget_args)
|
||||
return installation_path / "python.exe"
|
||||
|
||||
|
||||
def install_pypy(tmp: Path, arch: str, url: str, sha256: str) -> Path:
|
||||
assert arch == "64"
|
||||
assert "win64" in url
|
||||
# Inside the PyPy zip file is a directory with the same name
|
||||
zip_filename = url.rsplit("/", 1)[-1]
|
||||
extension = ".zip"
|
||||
assert zip_filename.endswith(extension)
|
||||
installation_path = CIBW_CACHE_PATH / zip_filename[: -len(extension)]
|
||||
with FileLock(str(installation_path) + ".lock"):
|
||||
if not installation_path.exists():
|
||||
pypy_zip = tmp / zip_filename
|
||||
download(url, pypy_zip, sha256=sha256)
|
||||
with remove_on_error(installation_path):
|
||||
# Extract to the parent directory because the zip file still contains a directory
|
||||
extract_zip(pypy_zip, installation_path.parent)
|
||||
return installation_path / "python.exe"
|
||||
|
||||
|
||||
def install_graalpy(tmp: Path, url: str, sha256: str) -> Path:
|
||||
zip_filename = url.rsplit("/", 1)[-1]
|
||||
extension = ".zip"
|
||||
assert zip_filename.endswith(extension)
|
||||
installation_path = CIBW_CACHE_PATH / zip_filename[: -len(extension)]
|
||||
with FileLock(str(installation_path) + ".lock"):
|
||||
if not installation_path.exists():
|
||||
graalpy_zip = tmp / zip_filename
|
||||
download(url, graalpy_zip, sha256=sha256)
|
||||
with remove_on_error(installation_path):
|
||||
# Extract to the parent directory because the zip file still contains a directory
|
||||
extract_zip(graalpy_zip, installation_path.parent)
|
||||
return installation_path / "bin" / "graalpy.exe"
|
||||
|
||||
|
||||
def setup_setuptools_cross_compile(
|
||||
tmp: Path,
|
||||
python_configuration: PythonConfiguration,
|
||||
python_libs_base: Path,
|
||||
env: MutableMapping[str, str],
|
||||
) -> None:
|
||||
distutils_cfg = tmp / "extra-setup.cfg"
|
||||
env["DIST_EXTRA_CONFIG"] = str(distutils_cfg)
|
||||
log.notice(f"Setting DIST_EXTRA_CONFIG={distutils_cfg} for cross-compilation")
|
||||
|
||||
# Ensure our additional import libraries are made available, and explicitly
|
||||
# set the platform name
|
||||
map_plat = {"32": "win32", "64": "win-amd64", "ARM64": "win-arm64"}
|
||||
plat_name = map_plat[python_configuration.arch]
|
||||
|
||||
# Set environment variable so that setuptools._distutils.get_platform()
|
||||
# identifies the target, not the host
|
||||
vscmd_arg_tgt_arch = {"32": "x86", "64": "x64", "ARM64": "arm64"}
|
||||
current_tgt_arch = vscmd_arg_tgt_arch[python_configuration.arch]
|
||||
if (env.get("VSCMD_ARG_TGT_ARCH") or current_tgt_arch) != current_tgt_arch:
|
||||
msg = f"VSCMD_ARG_TGT_ARCH must be set to {current_tgt_arch!r}, got {env['VSCMD_ARG_TGT_ARCH']!r}. If you're setting up MSVC yourself (e.g. using vcvarsall.bat or msvc-dev-cmd), make sure to target the right architecture. Alternatively, run cibuildwheel without configuring MSVC, and let the build backend handle it."
|
||||
raise errors.FatalError(msg)
|
||||
env["VSCMD_ARG_TGT_ARCH"] = current_tgt_arch
|
||||
|
||||
# (This file must be default/locale encoding, so we can't pass 'encoding')
|
||||
distutils_cfg.write_text(
|
||||
textwrap.dedent(
|
||||
f"""\
|
||||
[build]
|
||||
plat_name={plat_name}
|
||||
[build_ext]
|
||||
library_dirs={python_libs_base}
|
||||
plat_name={plat_name}
|
||||
[bdist_wheel]
|
||||
plat_name={plat_name}
|
||||
"""
|
||||
)
|
||||
)
|
||||
|
||||
# setuptools builds require explicit override of PYD extension
|
||||
# This is because it always gets the extension from the running
|
||||
# interpreter, and has no logic to construct it. Currently, CPython's
|
||||
# extensions follow our identifiers, but if they ever diverge in the
|
||||
# future, we will need to store new data
|
||||
log.notice(
|
||||
f"Setting SETUPTOOLS_EXT_SUFFIX=.{python_configuration.identifier}.pyd for cross-compilation"
|
||||
)
|
||||
env["SETUPTOOLS_EXT_SUFFIX"] = f".{python_configuration.identifier}.pyd"
|
||||
|
||||
# Cross-compilation requires fixes that only exist in setuptools's copy of
|
||||
# distutils, so ensure that it is activated
|
||||
# Since not all projects can handle the newer distutils, display a warning
|
||||
# to help them figure out what may have gone wrong if this breaks for them
|
||||
log.notice("Setting SETUPTOOLS_USE_DISTUTILS=local as it is required for cross-compilation")
|
||||
env["SETUPTOOLS_USE_DISTUTILS"] = "local"
|
||||
|
||||
|
||||
# These cross-compile setup functions have the same signature by design
|
||||
def setup_rust_cross_compile(
|
||||
tmp: Path, # noqa: ARG001
|
||||
python_configuration: PythonConfiguration,
|
||||
python_libs_base: Path, # noqa: ARG001
|
||||
env: MutableMapping[str, str],
|
||||
) -> None:
|
||||
# Assume that MSVC will be used, because we already know that we are
|
||||
# cross-compiling. MinGW users can set CARGO_BUILD_TARGET themselves
|
||||
# and we will respect the existing value.
|
||||
cargo_target = {
|
||||
"64": "x86_64-pc-windows-msvc",
|
||||
"32": "i686-pc-windows-msvc",
|
||||
"ARM64": "aarch64-pc-windows-msvc",
|
||||
}.get(python_configuration.arch)
|
||||
|
||||
# CARGO_BUILD_TARGET is the variable used by Cargo and setuptools_rust
|
||||
if env.get("CARGO_BUILD_TARGET"):
|
||||
if env["CARGO_BUILD_TARGET"] != cargo_target:
|
||||
log.notice("Not overriding CARGO_BUILD_TARGET as it has already been set")
|
||||
# No message if it was set to what we were planning to set it to
|
||||
elif cargo_target:
|
||||
log.notice(f"Setting CARGO_BUILD_TARGET={cargo_target} for cross-compilation")
|
||||
env["CARGO_BUILD_TARGET"] = cargo_target
|
||||
else:
|
||||
log.warning(
|
||||
f"Unable to configure Rust cross-compilation for architecture {python_configuration.arch}"
|
||||
)
|
||||
|
||||
|
||||
def setup_python(
|
||||
tmp: Path,
|
||||
python_configuration: PythonConfiguration,
|
||||
dependency_constraint: Path | None,
|
||||
environment: ParsedEnvironment,
|
||||
build_frontend: BuildFrontendName,
|
||||
) -> tuple[Path, dict[str, str]]:
|
||||
tmp.mkdir()
|
||||
implementation_id = python_configuration.identifier.split("-")[0]
|
||||
python_libs_base = None
|
||||
log.step(f"Installing Python {implementation_id}...")
|
||||
if implementation_id.startswith("cp"):
|
||||
native_arch = platform_module.machine()
|
||||
base_python = install_cpython(python_configuration)
|
||||
if python_configuration.arch == "ARM64" != native_arch:
|
||||
# To cross-compile for ARM64, we need a native CPython to run the
|
||||
# build, and a copy of the ARM64 import libraries ('.\libs\*.lib')
|
||||
# for any extension modules.
|
||||
python_libs_base = base_python.parent / "libs"
|
||||
log.step(f"Installing native Python {native_arch} for cross-compilation...")
|
||||
base_python = install_cpython(python_configuration, arch=native_arch)
|
||||
elif implementation_id.startswith("pp"):
|
||||
assert python_configuration.url is not None
|
||||
assert python_configuration.sha256
|
||||
base_python = install_pypy(
|
||||
tmp, python_configuration.arch, python_configuration.url, python_configuration.sha256
|
||||
)
|
||||
elif implementation_id.startswith("gp"):
|
||||
assert python_configuration.url is not None
|
||||
assert python_configuration.sha256
|
||||
base_python = install_graalpy(tmp, python_configuration.url, python_configuration.sha256)
|
||||
else:
|
||||
msg = "Unknown Python implementation"
|
||||
raise ValueError(msg)
|
||||
assert base_python.exists()
|
||||
|
||||
use_uv = build_frontend in {"build[uv]", "uv"}
|
||||
uv_path = find_uv()
|
||||
|
||||
log.step("Setting up build environment...")
|
||||
venv_path = tmp / "venv"
|
||||
env = virtualenv(
|
||||
python_configuration.version,
|
||||
base_python,
|
||||
venv_path,
|
||||
dependency_constraint,
|
||||
use_uv=use_uv,
|
||||
marker_env=target_marker_env(implementation_id=implementation_id),
|
||||
)
|
||||
|
||||
# set up environment variables for run_with_env
|
||||
env["PYTHON_VERSION"] = python_configuration.version
|
||||
env["PYTHON_ARCH"] = python_configuration.arch
|
||||
env.pop("UV_PYTHON", None)
|
||||
if not use_uv:
|
||||
env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
|
||||
|
||||
# update env with results from CIBW_ENVIRONMENT
|
||||
env = environment.as_dictionary(prev_environment=env)
|
||||
|
||||
# check what Python version we're on
|
||||
where_python = call("where", "python", env=env, capture_stdout=True).splitlines()[0].strip()
|
||||
print(where_python)
|
||||
if where_python != str(venv_path / "Scripts" / "python.exe"):
|
||||
msg = "python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it."
|
||||
raise errors.FatalError(msg)
|
||||
call("python", "-V", "-V", env=env)
|
||||
call("python", "-c", "\"import struct; print(struct.calcsize('P') * 8)\"", env=env)
|
||||
|
||||
# check what pip version we're on
|
||||
if not use_uv:
|
||||
assert (venv_path / "Scripts" / "pip.exe").exists()
|
||||
where_pip = call("where", "pip", env=env, capture_stdout=True).splitlines()[0].strip()
|
||||
print(where_pip)
|
||||
if where_pip != str(venv_path / "Scripts" / "pip.exe"):
|
||||
msg = "pip available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert pip above it."
|
||||
raise errors.FatalError(msg)
|
||||
call("pip", "--version", env=env)
|
||||
|
||||
log.step("Installing build tools...")
|
||||
match build_frontend:
|
||||
case "pip":
|
||||
call(
|
||||
"pip",
|
||||
"install",
|
||||
"--upgrade",
|
||||
"delvewheel",
|
||||
*constraint_flags(dependency_constraint),
|
||||
env=env,
|
||||
)
|
||||
case "build":
|
||||
call(
|
||||
"pip",
|
||||
"install",
|
||||
"--upgrade",
|
||||
"build[virtualenv]",
|
||||
"delvewheel",
|
||||
*constraint_flags(dependency_constraint),
|
||||
env=env,
|
||||
)
|
||||
case "build[uv]":
|
||||
assert uv_path is not None
|
||||
call(
|
||||
uv_path,
|
||||
"pip",
|
||||
"install",
|
||||
"--python",
|
||||
where_python,
|
||||
"--upgrade",
|
||||
"build[virtualenv]",
|
||||
"delvewheel",
|
||||
*constraint_flags(dependency_constraint),
|
||||
env=env,
|
||||
)
|
||||
case "uv":
|
||||
assert uv_path is not None
|
||||
call(
|
||||
uv_path,
|
||||
"pip",
|
||||
"install",
|
||||
"--python",
|
||||
where_python,
|
||||
"--upgrade",
|
||||
"delvewheel",
|
||||
*constraint_flags(dependency_constraint),
|
||||
env=env,
|
||||
)
|
||||
case "pyodide-build":
|
||||
msg = "The 'pyodide-build' build frontend is not supported on this platform"
|
||||
raise errors.FatalError(msg)
|
||||
case _:
|
||||
assert_never(build_frontend)
|
||||
|
||||
if python_libs_base:
|
||||
# Set up the environment for various backends to enable cross-compilation
|
||||
setup_setuptools_cross_compile(tmp, python_configuration, python_libs_base, env)
|
||||
setup_rust_cross_compile(tmp, python_configuration, python_libs_base, env)
|
||||
|
||||
return base_python, env
|
||||
|
||||
|
||||
def build(options: Options, tmp_path: Path) -> None:
|
||||
python_configurations = get_python_configurations(
|
||||
options.globals.build_selector, options.globals.architectures
|
||||
)
|
||||
|
||||
if not python_configurations:
|
||||
return
|
||||
|
||||
uv_path = find_uv()
|
||||
|
||||
try:
|
||||
before_all_options_identifier = python_configurations[0].identifier
|
||||
before_all_options = options.build_options(before_all_options_identifier)
|
||||
|
||||
if before_all_options.before_all:
|
||||
log.step("Running before_all...")
|
||||
env = before_all_options.environment.as_dictionary(prev_environment=os.environ)
|
||||
before_all_prepared = prepare_command(
|
||||
before_all_options.before_all, project=".", package=options.globals.package_dir
|
||||
)
|
||||
shell(before_all_prepared, env=env)
|
||||
|
||||
built_wheels: list[Path] = []
|
||||
|
||||
for config in python_configurations:
|
||||
build_options = options.build_options(config.identifier)
|
||||
build_frontend = build_options.build_frontend
|
||||
use_uv = build_frontend.name in {"build[uv]", "uv"}
|
||||
log.build_start(config.identifier)
|
||||
|
||||
identifier_tmp_dir = tmp_path / config.identifier
|
||||
identifier_tmp_dir.mkdir()
|
||||
built_wheel_dir = identifier_tmp_dir / "built_wheel"
|
||||
repaired_wheel_dir = identifier_tmp_dir / "repaired_wheel"
|
||||
|
||||
constraints_path = build_options.dependency_constraints.get_for_python_version(
|
||||
version=config.version,
|
||||
tmp_dir=identifier_tmp_dir,
|
||||
)
|
||||
|
||||
# install Python
|
||||
base_python, env = setup_python(
|
||||
identifier_tmp_dir / "build",
|
||||
config,
|
||||
constraints_path,
|
||||
build_options.environment,
|
||||
build_frontend.name,
|
||||
)
|
||||
env["CIBUILDWHEEL_BUILD_IDENTIFIER"] = config.identifier
|
||||
pip_version = None if use_uv else get_pip_version(env)
|
||||
|
||||
compatible_wheel = find_compatible_wheel(built_wheels, config.identifier)
|
||||
if compatible_wheel:
|
||||
log.step_end()
|
||||
print(
|
||||
f"\nFound previously built wheel {compatible_wheel.name}, that's compatible with {config.identifier}. Skipping build step..."
|
||||
)
|
||||
repaired_wheel = compatible_wheel
|
||||
else:
|
||||
# run the before_build command
|
||||
if build_options.before_build:
|
||||
log.step("Running before_build...")
|
||||
before_build_prepared = prepare_command(
|
||||
build_options.before_build,
|
||||
project=".",
|
||||
package=options.globals.package_dir,
|
||||
)
|
||||
shell(before_build_prepared, env=env)
|
||||
|
||||
log.step("Building wheel...")
|
||||
built_wheel_dir.mkdir()
|
||||
|
||||
extra_flags = get_build_frontend_extra_flags(
|
||||
build_frontend,
|
||||
build_options.build_verbosity,
|
||||
prepare_config_settings(
|
||||
build_options.config_settings,
|
||||
project=Path.cwd(),
|
||||
package=options.globals.package_dir,
|
||||
),
|
||||
)
|
||||
|
||||
match build_frontend.name:
|
||||
case "pip":
|
||||
# Path.resolve() is needed. Without it pip wheel may try to fetch package from pypi.org
|
||||
# see https://github.com/pypa/cibuildwheel/pull/369
|
||||
call(
|
||||
"python",
|
||||
"-m",
|
||||
"pip",
|
||||
"wheel",
|
||||
options.globals.package_dir.resolve(),
|
||||
f"--wheel-dir={built_wheel_dir}",
|
||||
"--no-deps",
|
||||
*extra_flags,
|
||||
env=env,
|
||||
)
|
||||
case "build" | "build[uv]":
|
||||
if (
|
||||
use_uv
|
||||
and "--no-isolation" not in extra_flags
|
||||
and "-n" not in extra_flags
|
||||
):
|
||||
extra_flags.append("--installer=uv")
|
||||
|
||||
call(
|
||||
"python",
|
||||
"-m",
|
||||
"build",
|
||||
build_options.package_dir,
|
||||
"--wheel",
|
||||
f"--outdir={built_wheel_dir}",
|
||||
*extra_flags,
|
||||
env=env,
|
||||
)
|
||||
case "uv":
|
||||
assert uv_path is not None
|
||||
call(
|
||||
uv_path,
|
||||
"build",
|
||||
f"--python={base_python}",
|
||||
build_options.package_dir,
|
||||
"--wheel",
|
||||
f"--out-dir={built_wheel_dir}",
|
||||
*extra_flags,
|
||||
env=env,
|
||||
)
|
||||
case "pyodide-build":
|
||||
msg = "The 'pyodide-build' build frontend is not supported on this platform"
|
||||
raise errors.FatalError(msg)
|
||||
case _:
|
||||
assert_never(build_frontend)
|
||||
|
||||
try:
|
||||
built_wheel = next(built_wheel_dir.glob("*.whl"))
|
||||
except StopIteration:
|
||||
raise errors.BuildProducedNoWheelError() from None
|
||||
|
||||
# repair the wheel
|
||||
repaired_wheel_dir.mkdir()
|
||||
|
||||
if built_wheel.name.endswith("none-any.whl"):
|
||||
raise errors.NonPlatformWheelError()
|
||||
|
||||
if build_options.repair_command:
|
||||
log.step("Repairing wheel...")
|
||||
repair_command_prepared = prepare_command(
|
||||
build_options.repair_command,
|
||||
wheel=built_wheel,
|
||||
dest_dir=repaired_wheel_dir,
|
||||
package=build_options.package_dir,
|
||||
project=".",
|
||||
)
|
||||
shell(repair_command_prepared, env=env)
|
||||
else:
|
||||
shutil.move(str(built_wheel), repaired_wheel_dir)
|
||||
|
||||
try:
|
||||
repaired_wheel = next(repaired_wheel_dir.glob("*.whl"))
|
||||
except StopIteration:
|
||||
raise errors.RepairStepProducedNoWheelError() from None
|
||||
|
||||
if repaired_wheel.name in {wheel.name for wheel in built_wheels}:
|
||||
raise errors.AlreadyBuiltWheelError(repaired_wheel.name)
|
||||
|
||||
run_audit(tmp_dir=tmp_path, build_options=build_options, wheel=repaired_wheel)
|
||||
|
||||
test_selected = options.globals.test_selector(config.identifier)
|
||||
if test_selected and config.arch == "ARM64" != platform_module.machine():
|
||||
log.warning(
|
||||
unwrap(
|
||||
"""
|
||||
While arm64 wheels can be built on other platforms, they cannot
|
||||
be tested. An arm64 runner is required. To silence this warning,
|
||||
set `CIBW_TEST_SKIP: "*-win_arm64"`.
|
||||
"""
|
||||
)
|
||||
)
|
||||
# skip this test
|
||||
elif test_selected and build_options.test_command:
|
||||
log.step("Testing wheel...")
|
||||
# set up a virtual environment to install and test from, to make sure
|
||||
# there are no dependencies that were pulled in at build time.
|
||||
venv_dir = identifier_tmp_dir / "venv-test"
|
||||
virtualenv_env = virtualenv(
|
||||
config.version,
|
||||
base_python,
|
||||
venv_dir,
|
||||
None,
|
||||
use_uv=use_uv,
|
||||
env=env,
|
||||
pip_version=pip_version,
|
||||
)
|
||||
|
||||
virtualenv_env = build_options.test_environment.as_dictionary(
|
||||
prev_environment=virtualenv_env
|
||||
)
|
||||
|
||||
# check that we are using the Python from the virtual environment
|
||||
call("where", "python", env=virtualenv_env)
|
||||
|
||||
if build_options.before_test:
|
||||
before_test_prepared = prepare_command(
|
||||
build_options.before_test,
|
||||
project=".",
|
||||
package=build_options.package_dir,
|
||||
)
|
||||
shell(before_test_prepared, env=virtualenv_env)
|
||||
|
||||
pip: Sequence[Path | str]
|
||||
if use_uv:
|
||||
assert uv_path is not None
|
||||
pip = [uv_path, "pip"]
|
||||
else:
|
||||
pip = ["pip"]
|
||||
|
||||
# install the wheel
|
||||
call(
|
||||
*pip,
|
||||
"install",
|
||||
str(repaired_wheel) + build_options.test_extras,
|
||||
env=virtualenv_env,
|
||||
)
|
||||
|
||||
# test the wheel
|
||||
if build_options.test_requires:
|
||||
call(*pip, "install", *build_options.test_requires, env=virtualenv_env)
|
||||
|
||||
# run the tests from a temp dir, with an absolute path in the command
|
||||
# (this ensures that Python runs the tests against the installed wheel
|
||||
# and not the repo code)
|
||||
test_cwd = identifier_tmp_dir / "test_cwd"
|
||||
test_cwd.mkdir()
|
||||
|
||||
if build_options.test_sources:
|
||||
copy_test_sources(
|
||||
build_options.test_sources,
|
||||
Path.cwd(),
|
||||
test_cwd,
|
||||
)
|
||||
else:
|
||||
# Use the test_fail.py file to raise a nice error if the user
|
||||
# tries to run tests in the cwd
|
||||
(test_cwd / "test_fail.py").write_text(resources.TEST_FAIL_CWD_FILE.read_text())
|
||||
|
||||
test_command_prepared = prepare_command(
|
||||
build_options.test_command,
|
||||
project=Path.cwd(),
|
||||
package=options.globals.package_dir.resolve(),
|
||||
wheel=repaired_wheel,
|
||||
)
|
||||
shell(test_command_prepared, cwd=test_cwd, env=virtualenv_env)
|
||||
|
||||
# we're all done here; move it to output (remove if already exists)
|
||||
output_wheel = None
|
||||
if compatible_wheel is None:
|
||||
output_wheel = build_options.output_dir.joinpath(repaired_wheel.name)
|
||||
moved_wheel = move_file(repaired_wheel, output_wheel)
|
||||
if moved_wheel != output_wheel.resolve():
|
||||
log.warning(
|
||||
f"{repaired_wheel} was moved to {moved_wheel} instead of {output_wheel}"
|
||||
)
|
||||
built_wheels.append(output_wheel)
|
||||
|
||||
# clean up
|
||||
# (we ignore errors because occasionally Windows fails to unlink a file and we
|
||||
# don't want to abort a build because of that)
|
||||
shutil.rmtree(identifier_tmp_dir, ignore_errors=True)
|
||||
|
||||
log.build_end(output_wheel)
|
||||
except subprocess.CalledProcessError as error:
|
||||
msg = f"Command {error.cmd} failed with code {error.returncode}. {error.stdout or ''}"
|
||||
raise errors.FatalError(msg) from error
|
||||
@@ -1,116 +1,78 @@
|
||||
from __future__ import annotations
|
||||
|
||||
__lazy_modules__ = {"configparser", "contextlib", "dependency_groups"}
|
||||
|
||||
import ast
|
||||
import configparser
|
||||
import contextlib
|
||||
import sys
|
||||
from configparser import ConfigParser
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
|
||||
import dependency_groups
|
||||
import toml
|
||||
|
||||
TYPE_CHECKING = False
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
if sys.version_info < (3, 8):
|
||||
Constant = ast.Str
|
||||
|
||||
def get_constant(x: ast.Str) -> str:
|
||||
return x.s
|
||||
|
||||
|
||||
def get_parent(node: ast.AST | None, depth: int = 1) -> ast.AST | None:
|
||||
for _ in range(depth):
|
||||
node = getattr(node, "parent", None)
|
||||
return node
|
||||
else:
|
||||
Constant = ast.Constant
|
||||
|
||||
|
||||
def is_main(parent: ast.AST | None) -> bool:
|
||||
match parent:
|
||||
case ast.If(test=ast.Compare(left=left, ops=[ast.Eq()], comparators=[comp])):
|
||||
values = {left, comp}
|
||||
mains = {x for x in values if isinstance(x, ast.Constant) and x.value == "__main__"}
|
||||
if len(mains) != 1:
|
||||
return False
|
||||
consts = {x for x in values if isinstance(x, ast.Name) and x.id == "__name__"}
|
||||
return len(consts) == 1
|
||||
case _:
|
||||
return False
|
||||
def get_constant(x: ast.Constant) -> Any:
|
||||
return x.value
|
||||
|
||||
|
||||
class Analyzer(ast.NodeVisitor):
|
||||
def __init__(self) -> None:
|
||||
self.requires_python: str | None = None
|
||||
self.requires_python: Optional[str] = None
|
||||
|
||||
def visit(self, node: ast.AST) -> None:
|
||||
for inner_node in ast.walk(node):
|
||||
for child in ast.iter_child_nodes(inner_node):
|
||||
child.parent = inner_node # type: ignore[attr-defined]
|
||||
super().visit(node)
|
||||
def visit(self, content: ast.AST) -> None:
|
||||
for node in ast.walk(content):
|
||||
for child in ast.iter_child_nodes(node):
|
||||
child.parent = node # type: ignore
|
||||
super().visit(content)
|
||||
|
||||
def visit_keyword(self, node: ast.keyword) -> None:
|
||||
# Must not be nested except for if __name__ == "__main__"
|
||||
|
||||
self.generic_visit(node)
|
||||
# This will be Module -> Expr -> Call -> keyword
|
||||
parent = get_parent(node, 4)
|
||||
unnested = parent is None
|
||||
|
||||
# This will be Module -> If -> Expr -> Call -> keyword
|
||||
name_main_unnested = (
|
||||
parent is not None and get_parent(parent) is None and is_main(get_parent(node, 3))
|
||||
)
|
||||
|
||||
match node:
|
||||
case ast.keyword(arg="python_requires", value=ast.Constant(value=str() as version)):
|
||||
if unnested or name_main_unnested:
|
||||
self.requires_python = version
|
||||
if node.arg == "python_requires":
|
||||
# Must not be nested in an if or other structure
|
||||
# This will be Module -> Expr -> Call -> keyword
|
||||
if not hasattr(node.parent.parent.parent, "parent") and isinstance( # type: ignore
|
||||
node.value, Constant
|
||||
):
|
||||
self.requires_python = get_constant(node.value)
|
||||
|
||||
|
||||
def setup_py_python_requires(content: str) -> str | None:
|
||||
def setup_py_python_requires(content: str) -> Optional[str]:
|
||||
try:
|
||||
tree = ast.parse(content)
|
||||
analyzer = Analyzer()
|
||||
analyzer.visit(tree)
|
||||
return analyzer.requires_python or None
|
||||
except Exception: # pylint: disable=broad-except
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def get_requires_python_str(package_dir: Path, pyproject_toml: dict[str, Any] | None) -> str | None:
|
||||
"""Return the python requires string from the most canonical source available, or None"""
|
||||
def get_requires_python_str(package_dir: Path) -> Optional[str]:
|
||||
"Return the python requires string from the most canonical source available, or None"
|
||||
|
||||
# Read in from pyproject.toml:project.requires-python
|
||||
with contextlib.suppress(KeyError, IndexError, TypeError):
|
||||
return str((pyproject_toml or {})["project"]["requires-python"])
|
||||
try:
|
||||
info = toml.load(package_dir / "pyproject.toml")
|
||||
return str(info["project"]["requires-python"])
|
||||
except (FileNotFoundError, KeyError, IndexError, TypeError):
|
||||
pass
|
||||
|
||||
# Read in from setup.cfg:options.python_requires
|
||||
config = configparser.ConfigParser()
|
||||
with contextlib.suppress(FileNotFoundError):
|
||||
try:
|
||||
config = ConfigParser()
|
||||
config.read(package_dir / "setup.cfg")
|
||||
with contextlib.suppress(KeyError, IndexError, TypeError):
|
||||
return str(config["options"]["python_requires"])
|
||||
|
||||
setup_py = package_dir / "setup.py"
|
||||
with contextlib.suppress(FileNotFoundError), setup_py.open(encoding="utf8") as f2:
|
||||
return setup_py_python_requires(f2.read())
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def resolve_dependency_groups(
|
||||
pyproject_toml: dict[str, Any] | None, *groups: str
|
||||
) -> tuple[str, ...]:
|
||||
"""
|
||||
Get the packages in dependency-groups for a package.
|
||||
"""
|
||||
|
||||
if not groups:
|
||||
return ()
|
||||
|
||||
if pyproject_toml is None:
|
||||
msg = f"Didn't find a pyproject.toml, so can't read [dependency-groups] {groups!r} from it!"
|
||||
raise FileNotFoundError(msg)
|
||||
return str(config["options"]["python_requires"])
|
||||
except (FileNotFoundError, KeyError, IndexError, TypeError):
|
||||
pass
|
||||
|
||||
try:
|
||||
dependency_groups_toml = pyproject_toml["dependency-groups"]
|
||||
except KeyError:
|
||||
msg = f"Didn't find [dependency-groups] in pyproject.toml, which is needed to resolve {groups!r}."
|
||||
raise KeyError(msg) from None
|
||||
with open(package_dir / "setup.py") as f:
|
||||
return setup_py_python_requires(f.read())
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
|
||||
return dependency_groups.resolve(dependency_groups_toml, *groups)
|
||||
return None
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
import _cross_venv; _cross_venv.initialize()
|
||||
@@ -1,90 +0,0 @@
|
||||
# This module is copied into the site-packages directory of an Android build environment, and
|
||||
# activated via a .pth file when we want the environment to simulate Android.
|
||||
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import sys
|
||||
import sysconfig
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
def initialize() -> None:
|
||||
if not (host_triplet := os.environ.get("CIBW_HOST_TRIPLET")):
|
||||
return
|
||||
|
||||
# Pre-import any modules which would fail to import after the monkey-patching.
|
||||
import ctypes # noqa: F401, PLC0415 - uses get_config_var("LDLIBRARY")
|
||||
|
||||
# os ######################################################################
|
||||
def cross_os_uname() -> os.uname_result:
|
||||
return os.uname_result(
|
||||
(
|
||||
"Linux",
|
||||
"localhost",
|
||||
# The Linux kernel version and release are unlikely to be significant, but return
|
||||
# realistic values anyway (from an API level 24 emulator).
|
||||
"3.18.91+",
|
||||
"#1 SMP PREEMPT Tue Jan 9 20:35:43 UTC 2018",
|
||||
host_triplet.split("-")[0],
|
||||
)
|
||||
)
|
||||
|
||||
os.name = "posix"
|
||||
os.uname = cross_os_uname
|
||||
|
||||
# platform ################################################################
|
||||
#
|
||||
# We can't determine the user-visible Android version number from the API level, so return a
|
||||
# string which will work fine for display, but will fail to parse as a version number.
|
||||
def cross_android_ver(*args: Any, **kwargs: Any) -> platform.AndroidVer:
|
||||
return platform.AndroidVer(
|
||||
release=f"API level {cross_getandroidapilevel()}",
|
||||
api_level=cross_getandroidapilevel(),
|
||||
manufacturer="Google",
|
||||
model="sdk_gphone64",
|
||||
device="emu64",
|
||||
is_emulator=True,
|
||||
)
|
||||
|
||||
# platform.uname, platform.system etc. are all implemented in terms of platform.android_ver.
|
||||
platform.android_ver = cross_android_ver
|
||||
|
||||
# sys #####################################################################
|
||||
def cross_getandroidapilevel() -> int:
|
||||
api_level = sysconfig.get_config_var("ANDROID_API_LEVEL")
|
||||
assert isinstance(api_level, int)
|
||||
return api_level
|
||||
|
||||
# Some packages may recognize sys.cross_compiling from the crossenv tool.
|
||||
sys.cross_compiling = True # type: ignore[attr-defined]
|
||||
sys.getandroidapilevel = cross_getandroidapilevel # type: ignore[attr-defined]
|
||||
sys.implementation._multiarch = host_triplet # type: ignore[attr-defined]
|
||||
sys.platform = "android"
|
||||
|
||||
# Determine the abiflags from the sysconfigdata filename.
|
||||
sysconfigdata_path = next(Path(__file__).parent.glob("_sysconfigdata_*.py"))
|
||||
abiflags_match = re.match(r"_sysconfigdata_(.*?)_", sysconfigdata_path.name)
|
||||
assert abiflags_match is not None
|
||||
sys.abiflags = abiflags_match[1]
|
||||
|
||||
# sysconfig ###############################################################
|
||||
#
|
||||
# Load the sysconfigdata file, generating its name from sys.abiflags,
|
||||
# sys.platform, and sys.implementation._multiarch.
|
||||
sysconfig._init_config_vars() # type: ignore[attr-defined]
|
||||
|
||||
# We don't change the actual sys.base_prefix and base_exec_prefix, because that
|
||||
# could have unpredictable effects. Instead, we change the sysconfig variables
|
||||
# used by sysconfig.get_paths().
|
||||
vars = sysconfig.get_config_vars()
|
||||
try:
|
||||
host_prefix = vars["host_prefix"] # This variable was added in Python 3.14.
|
||||
except KeyError:
|
||||
host_prefix = vars["exec_prefix"]
|
||||
vars["installed_base"] = vars["installed_platbase"] = host_prefix
|
||||
|
||||
# sysconfig.get_platform, which determines the wheel tag, is implemented in terms of
|
||||
# sys.platform, sysconfig.get_config_var("ANDROID_API_LEVEL") (see localized_vars in
|
||||
# android.py), and os.uname.
|
||||
@@ -1,121 +0,0 @@
|
||||
# This file intentionally has no hashbang line in the source: cibuildwheel will add it
|
||||
# above this comment when the file is deployed.
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from tempfile import gettempdir
|
||||
|
||||
from filelock import FileLock
|
||||
|
||||
from cibuildwheel.util.file import CIBW_CACHE_PATH, download
|
||||
|
||||
# In the future we might pick a different Flang release depending on the NDK version,
|
||||
# but so far all Python versions use the same NDK version, so there's no need.
|
||||
RELEASE_URL = "https://github.com/termux/ndk-toolchain-clang-with-flang/releases/download"
|
||||
RELEASE_VERSION = "r27c"
|
||||
ARCHS = ["aarch64", "x86_64"]
|
||||
|
||||
# The compiler is built for Linux x86_64, so we use Docker on macOS.
|
||||
DOCKER_IMAGE = "debian:trixie-slim"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
cache_dir = CIBW_CACHE_PATH / f"flang-android-{RELEASE_VERSION}"
|
||||
with FileLock(f"{cache_dir}.lock"):
|
||||
if not cache_dir.exists():
|
||||
download_flang(cache_dir)
|
||||
|
||||
run_flang(cache_dir)
|
||||
|
||||
|
||||
def download_flang(cache_dir: Path) -> None:
|
||||
tmp_dir = Path(f"{cache_dir}.tmp")
|
||||
if tmp_dir.exists():
|
||||
shutil.rmtree(tmp_dir)
|
||||
tmp_dir.mkdir(parents=True)
|
||||
|
||||
for archive_name in [f"package-flang-{arch}.tar.bz2" for arch in ARCHS] + [
|
||||
"package-flang-host.tar.bz2",
|
||||
"package-install.tar.bz2",
|
||||
]:
|
||||
archive_path = tmp_dir / archive_name
|
||||
download(f"{RELEASE_URL}/{RELEASE_VERSION}/{archive_name}", archive_path)
|
||||
shutil.unpack_archive(archive_path, tmp_dir)
|
||||
archive_path.unlink()
|
||||
|
||||
# Merge the extracted trees together, along with the necessary parts of the NDK. Based on
|
||||
# https://github.com/kivy/python-for-android/blob/develop/pythonforandroid/recipes/fortran/__init__.py.
|
||||
flang_toolchain = tmp_dir / "toolchain"
|
||||
(tmp_dir / "out/install/linux-x86/clang-dev").rename(flang_toolchain)
|
||||
|
||||
ndk_toolchain = Path(os.environ["CC"]).parents[1]
|
||||
if (clang_ver_flang := clang_ver(flang_toolchain)) != (
|
||||
clang_ver_ndk := clang_ver(ndk_toolchain)
|
||||
):
|
||||
msg = f"Flang uses Clang {clang_ver_flang}, but NDK uses Clang {clang_ver_ndk}"
|
||||
raise ValueError(msg)
|
||||
|
||||
clang_lib_path = f"lib/clang/{clang_ver_ndk}/lib"
|
||||
shutil.rmtree(flang_toolchain / clang_lib_path)
|
||||
|
||||
for src, dst in [
|
||||
(f"{tmp_dir}/build-{arch}-install", f"sysroot/usr/lib/{arch}-linux-android")
|
||||
for arch in ARCHS
|
||||
] + [
|
||||
(f"{tmp_dir}/build-host-install", ""),
|
||||
(f"{ndk_toolchain}/{clang_lib_path}", clang_lib_path),
|
||||
(f"{ndk_toolchain}/sysroot", "sysroot"),
|
||||
]:
|
||||
shutil.copytree(src, flang_toolchain / dst, symlinks=True, dirs_exist_ok=True)
|
||||
|
||||
flang_toolchain.rename(cache_dir)
|
||||
shutil.rmtree(tmp_dir)
|
||||
|
||||
|
||||
def clang_ver(toolchain: Path) -> str:
|
||||
versions = [p.name for p in (toolchain / "lib/clang").iterdir()]
|
||||
assert len(versions) == 1
|
||||
return versions[0]
|
||||
|
||||
|
||||
def run_flang(cache_dir: Path) -> None:
|
||||
match = re.fullmatch(r".+/(.+)-clang", os.environ["CC"])
|
||||
assert match is not None
|
||||
target = match[1]
|
||||
|
||||
# In a future Flang version the executable name will change to "flang"
|
||||
# (https://blog.llvm.org/posts/2025-03-11-flang-new/).
|
||||
flang_args = [f"{cache_dir}/bin/flang-new", f"--target={target}", *sys.argv[1:]]
|
||||
|
||||
if sys.platform == "linux":
|
||||
args = flang_args
|
||||
elif sys.platform == "darwin":
|
||||
args = ["docker", "run", "--rm", "--platform", "linux/amd64"]
|
||||
# Docker on macOS only allows certain directories to be mounted as volumes
|
||||
# by default, but they include all the locations we're likely to need.
|
||||
for path in [
|
||||
# Location of CIBW_CACHE_PATH (containing the Fortran compiler), and
|
||||
# probably the project directory.
|
||||
Path.home(),
|
||||
# Temporary directories created by build backends and cibuildwheel itself.
|
||||
# This is controlled by the TMPDIR environment variable, which is set to
|
||||
# an unpredictable subdirectory of /var/folders. /var is itself a symlink
|
||||
# to /private/var.
|
||||
Path(gettempdir()),
|
||||
Path(gettempdir()).resolve(),
|
||||
]:
|
||||
args += ["-v", f"{path}:{path}"]
|
||||
args += ["--workdir", str(Path.cwd())]
|
||||
args += ["--entrypoint", flang_args[0], DOCKER_IMAGE, *flang_args[1:]]
|
||||
else:
|
||||
msg = f"unknown platform: {sys.platform}"
|
||||
raise ValueError(msg)
|
||||
|
||||
os.execvp(args[0], args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,57 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main() -> None:
|
||||
# CIBW_HOST_TRIPLET is set in the android_env to the Android target triplet.
|
||||
target = os.environ.get("CIBW_HOST_TRIPLET")
|
||||
|
||||
cmd_name = Path(sys.argv[0]).name
|
||||
|
||||
# Find the real command in PATH, excluding the current script's directory
|
||||
path_env = os.environ.get("PATH", "")
|
||||
script_dir = Path(__file__).resolve().parent
|
||||
|
||||
paths = path_env.split(os.pathsep)
|
||||
# Filter out the script directory to avoid recursion
|
||||
filtered_paths = [p for p in paths if Path(p).resolve() != script_dir]
|
||||
filtered_path_env = os.pathsep.join(filtered_paths)
|
||||
|
||||
real_cmd = shutil.which(cmd_name, path=filtered_path_env)
|
||||
|
||||
if not real_cmd:
|
||||
sys.stderr.write(f"cibuildwheel: Error: Could not find system {cmd_name}\n")
|
||||
sys.exit(1)
|
||||
|
||||
# If we have a target (i.e. we are in the android_env), try to install it.
|
||||
if target:
|
||||
# Check if rustup is available to install the target
|
||||
rustup_path = shutil.which("rustup", path=filtered_path_env)
|
||||
|
||||
if rustup_path:
|
||||
try:
|
||||
# We call rustup to ensure the target is installed.
|
||||
subprocess.run(
|
||||
[rustup_path, "target", "add", target],
|
||||
check=True,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
)
|
||||
except subprocess.CalledProcessError as e:
|
||||
sys.stderr.write(
|
||||
f"cibuildwheel: Error: Failed to install Rust target {target}: {e.stderr}\n"
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
# Execute the real command
|
||||
os.execv(real_cmd, [real_cmd, *sys.argv[1:]])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,244 +1,69 @@
|
||||
[linux]
|
||||
python_configurations = [
|
||||
{ identifier = "cp27-manylinux_x86_64", version = "2.7", path_str = "/opt/python/cp27-cp27m" },
|
||||
{ identifier = "cp27-manylinux_x86_64", version = "2.7", path_str = "/opt/python/cp27-cp27mu" },
|
||||
{ identifier = "cp35-manylinux_x86_64", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
|
||||
{ identifier = "cp36-manylinux_x86_64", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
|
||||
{ identifier = "cp37-manylinux_x86_64", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
|
||||
{ identifier = "cp38-manylinux_x86_64", version = "3.8", path_str = "/opt/python/cp38-cp38" },
|
||||
{ identifier = "cp39-manylinux_x86_64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-manylinux_x86_64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-manylinux_x86_64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-manylinux_x86_64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-manylinux_x86_64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-manylinux_x86_64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-manylinux_x86_64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-manylinux_x86_64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-manylinux_x86_64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "cp27-manylinux_i686", version = "2.7", path_str = "/opt/python/cp27-cp27m" },
|
||||
{ identifier = "cp27-manylinux_i686", version = "2.7", path_str = "/opt/python/cp27-cp27mu" },
|
||||
{ identifier = "cp35-manylinux_i686", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
|
||||
{ identifier = "cp36-manylinux_i686", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
|
||||
{ identifier = "cp37-manylinux_i686", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
|
||||
{ identifier = "cp38-manylinux_i686", version = "3.8", path_str = "/opt/python/cp38-cp38" },
|
||||
{ identifier = "cp39-manylinux_i686", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-manylinux_i686", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-manylinux_i686", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-manylinux_i686", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-manylinux_i686", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-manylinux_i686", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-manylinux_i686", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-manylinux_i686", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-manylinux_i686", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "pp39-manylinux_x86_64", version = "3.9", path_str = "/opt/python/pp39-pypy39_pp73" },
|
||||
{ identifier = "pp310-manylinux_x86_64", version = "3.10", path_str = "/opt/python/pp310-pypy310_pp73" },
|
||||
{ identifier = "pp311-manylinux_x86_64", version = "3.11", path_str = "/opt/python/pp311-pypy311_pp73" },
|
||||
{ identifier = "gp312_250-manylinux_x86_64", version = "3.12", path_str = "/opt/python/graalpy312-graalpy250_312_native" },
|
||||
{ identifier = "pp27-manylinux_x86_64", version = "2.7", path_str = "/opt/python/pp27-pypy_73" },
|
||||
{ identifier = "pp36-manylinux_x86_64", version = "3.6", path_str = "/opt/python/pp36-pypy36_pp73" },
|
||||
{ identifier = "pp37-manylinux_x86_64", version = "3.7", path_str = "/opt/python/pp37-pypy37_pp73" },
|
||||
{ identifier = "cp35-manylinux_aarch64", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
|
||||
{ identifier = "cp36-manylinux_aarch64", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
|
||||
{ identifier = "cp37-manylinux_aarch64", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
|
||||
{ identifier = "cp38-manylinux_aarch64", version = "3.8", path_str = "/opt/python/cp38-cp38" },
|
||||
{ identifier = "cp39-manylinux_aarch64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-manylinux_aarch64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-manylinux_aarch64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-manylinux_aarch64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-manylinux_aarch64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-manylinux_aarch64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-manylinux_aarch64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-manylinux_aarch64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-manylinux_aarch64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "cp35-manylinux_ppc64le", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
|
||||
{ identifier = "cp36-manylinux_ppc64le", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
|
||||
{ identifier = "cp37-manylinux_ppc64le", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
|
||||
{ identifier = "cp38-manylinux_ppc64le", version = "3.8", path_str = "/opt/python/cp38-cp38" },
|
||||
{ identifier = "cp39-manylinux_ppc64le", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-manylinux_ppc64le", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-manylinux_ppc64le", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-manylinux_ppc64le", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-manylinux_ppc64le", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-manylinux_ppc64le", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-manylinux_ppc64le", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-manylinux_ppc64le", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-manylinux_ppc64le", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "cp35-manylinux_s390x", version = "3.5", path_str = "/opt/python/cp35-cp35m" },
|
||||
{ identifier = "cp36-manylinux_s390x", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
|
||||
{ identifier = "cp37-manylinux_s390x", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
|
||||
{ identifier = "cp38-manylinux_s390x", version = "3.8", path_str = "/opt/python/cp38-cp38" },
|
||||
{ identifier = "cp39-manylinux_s390x", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-manylinux_s390x", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-manylinux_s390x", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-manylinux_s390x", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-manylinux_s390x", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-manylinux_s390x", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-manylinux_s390x", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-manylinux_s390x", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-manylinux_s390x", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "cp39-manylinux_armv7l", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-manylinux_armv7l", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-manylinux_armv7l", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-manylinux_armv7l", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-manylinux_armv7l", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-manylinux_armv7l", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-manylinux_armv7l", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-manylinux_armv7l", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-manylinux_armv7l", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "cp39-manylinux_riscv64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-manylinux_riscv64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-manylinux_riscv64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-manylinux_riscv64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-manylinux_riscv64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-manylinux_riscv64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-manylinux_riscv64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-manylinux_riscv64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-manylinux_riscv64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "pp39-manylinux_aarch64", version = "3.9", path_str = "/opt/python/pp39-pypy39_pp73" },
|
||||
{ identifier = "pp310-manylinux_aarch64", version = "3.10", path_str = "/opt/python/pp310-pypy310_pp73" },
|
||||
{ identifier = "pp311-manylinux_aarch64", version = "3.11", path_str = "/opt/python/pp311-pypy311_pp73" },
|
||||
{ identifier = "gp312_250-manylinux_aarch64", version = "3.12", path_str = "/opt/python/graalpy312-graalpy250_312_native" },
|
||||
{ identifier = "pp39-manylinux_i686", version = "3.9", path_str = "/opt/python/pp39-pypy39_pp73" },
|
||||
{ identifier = "pp310-manylinux_i686", version = "3.10", path_str = "/opt/python/pp310-pypy310_pp73" },
|
||||
{ identifier = "pp311-manylinux_i686", version = "3.11", path_str = "/opt/python/pp311-pypy311_pp73" },
|
||||
{ identifier = "cp39-musllinux_x86_64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-musllinux_x86_64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-musllinux_x86_64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-musllinux_x86_64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-musllinux_x86_64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-musllinux_x86_64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-musllinux_x86_64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-musllinux_x86_64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-musllinux_x86_64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "cp39-musllinux_i686", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-musllinux_i686", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-musllinux_i686", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-musllinux_i686", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-musllinux_i686", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-musllinux_i686", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-musllinux_i686", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-musllinux_i686", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-musllinux_i686", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "cp39-musllinux_aarch64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-musllinux_aarch64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-musllinux_aarch64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-musllinux_aarch64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-musllinux_aarch64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-musllinux_aarch64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-musllinux_aarch64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-musllinux_aarch64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-musllinux_aarch64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "cp39-musllinux_ppc64le", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-musllinux_ppc64le", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-musllinux_ppc64le", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-musllinux_ppc64le", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-musllinux_ppc64le", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-musllinux_ppc64le", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-musllinux_ppc64le", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-musllinux_ppc64le", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-musllinux_ppc64le", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "cp39-musllinux_s390x", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-musllinux_s390x", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-musllinux_s390x", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-musllinux_s390x", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-musllinux_s390x", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-musllinux_s390x", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-musllinux_s390x", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-musllinux_s390x", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-musllinux_s390x", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "cp39-musllinux_armv7l", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-musllinux_armv7l", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-musllinux_armv7l", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-musllinux_armv7l", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-musllinux_armv7l", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-musllinux_armv7l", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-musllinux_armv7l", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-musllinux_armv7l", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-musllinux_armv7l", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
{ identifier = "cp39-musllinux_riscv64", version = "3.9", path_str = "/opt/python/cp39-cp39" },
|
||||
{ identifier = "cp310-musllinux_riscv64", version = "3.10", path_str = "/opt/python/cp310-cp310" },
|
||||
{ identifier = "cp311-musllinux_riscv64", version = "3.11", path_str = "/opt/python/cp311-cp311" },
|
||||
{ identifier = "cp312-musllinux_riscv64", version = "3.12", path_str = "/opt/python/cp312-cp312" },
|
||||
{ identifier = "cp313-musllinux_riscv64", version = "3.13", path_str = "/opt/python/cp313-cp313" },
|
||||
{ identifier = "cp314-musllinux_riscv64", version = "3.14", path_str = "/opt/python/cp314-cp314" },
|
||||
{ identifier = "cp314t-musllinux_riscv64", version = "3.14", path_str = "/opt/python/cp314-cp314t" },
|
||||
{ identifier = "cp315-musllinux_riscv64", version = "3.15", path_str = "/opt/python/cp315-cp315" },
|
||||
{ identifier = "cp315t-musllinux_riscv64", version = "3.15", path_str = "/opt/python/cp315-cp315t" },
|
||||
]
|
||||
|
||||
[macos]
|
||||
python_configurations = [
|
||||
{ identifier = "cp39-macosx_x86_64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.13/python-3.9.13-macos11.pkg", sha256 = "351fe18f4fb03be7afac5e4012fc0a51345f43202af43ef620cf1eee5ee36578" },
|
||||
{ identifier = "cp39-macosx_arm64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.13/python-3.9.13-macos11.pkg", sha256 = "351fe18f4fb03be7afac5e4012fc0a51345f43202af43ef620cf1eee5ee36578" },
|
||||
{ identifier = "cp39-macosx_universal2", version = "3.9", url = "https://www.python.org/ftp/python/3.9.13/python-3.9.13-macos11.pkg", sha256 = "351fe18f4fb03be7afac5e4012fc0a51345f43202af43ef620cf1eee5ee36578" },
|
||||
{ identifier = "cp310-macosx_x86_64", version = "3.10", url = "https://www.python.org/ftp/python/3.10.11/python-3.10.11-macos11.pkg", sha256 = "767ed35ad688d28ea4494081ae96408a0318d0d5bb9ca0139d74d6247b231cfc" },
|
||||
{ identifier = "cp310-macosx_arm64", version = "3.10", url = "https://www.python.org/ftp/python/3.10.11/python-3.10.11-macos11.pkg", sha256 = "767ed35ad688d28ea4494081ae96408a0318d0d5bb9ca0139d74d6247b231cfc" },
|
||||
{ identifier = "cp310-macosx_universal2", version = "3.10", url = "https://www.python.org/ftp/python/3.10.11/python-3.10.11-macos11.pkg", sha256 = "767ed35ad688d28ea4494081ae96408a0318d0d5bb9ca0139d74d6247b231cfc" },
|
||||
{ identifier = "cp311-macosx_x86_64", version = "3.11", url = "https://www.python.org/ftp/python/3.11.9/python-3.11.9-macos11.pkg", sha256 = "b6cfdee2571ca56ee895043ca1e7110fb78a878cee3eb0c21accb2de34d24b55" },
|
||||
{ identifier = "cp311-macosx_arm64", version = "3.11", url = "https://www.python.org/ftp/python/3.11.9/python-3.11.9-macos11.pkg", sha256 = "b6cfdee2571ca56ee895043ca1e7110fb78a878cee3eb0c21accb2de34d24b55" },
|
||||
{ identifier = "cp311-macosx_universal2", version = "3.11", url = "https://www.python.org/ftp/python/3.11.9/python-3.11.9-macos11.pkg", sha256 = "b6cfdee2571ca56ee895043ca1e7110fb78a878cee3eb0c21accb2de34d24b55" },
|
||||
{ identifier = "cp312-macosx_x86_64", version = "3.12", url = "https://www.python.org/ftp/python/3.12.10/python-3.12.10-macos11.pkg", sha256 = "8373e58da4ea146b3eb1c1f9834f19a319440b6b679b06050b1f9ee3237aa8e4" },
|
||||
{ identifier = "cp312-macosx_arm64", version = "3.12", url = "https://www.python.org/ftp/python/3.12.10/python-3.12.10-macos11.pkg", sha256 = "8373e58da4ea146b3eb1c1f9834f19a319440b6b679b06050b1f9ee3237aa8e4" },
|
||||
{ identifier = "cp312-macosx_universal2", version = "3.12", url = "https://www.python.org/ftp/python/3.12.10/python-3.12.10-macos11.pkg", sha256 = "8373e58da4ea146b3eb1c1f9834f19a319440b6b679b06050b1f9ee3237aa8e4" },
|
||||
{ identifier = "cp313-macosx_x86_64", version = "3.13", url = "https://www.python.org/ftp/python/3.13.14/python-3.13.14-macos11.pkg", sha256 = "8e58affb218c155a1dfdc27b291f817129669f8760e7a297adb2e4439ba5d2e8" },
|
||||
{ identifier = "cp313-macosx_arm64", version = "3.13", url = "https://www.python.org/ftp/python/3.13.14/python-3.13.14-macos11.pkg", sha256 = "8e58affb218c155a1dfdc27b291f817129669f8760e7a297adb2e4439ba5d2e8" },
|
||||
{ identifier = "cp313-macosx_universal2", version = "3.13", url = "https://www.python.org/ftp/python/3.13.14/python-3.13.14-macos11.pkg", sha256 = "8e58affb218c155a1dfdc27b291f817129669f8760e7a297adb2e4439ba5d2e8" },
|
||||
{ identifier = "cp314-macosx_x86_64", version = "3.14", url = "https://www.python.org/ftp/python/3.14.6/python-3.14.6-macos11.pkg", sha256 = "d3c9fff52214847e4fab03e9eaf53dd2a8e51e3534aa0b61f201b749f86bef28" },
|
||||
{ identifier = "cp314-macosx_arm64", version = "3.14", url = "https://www.python.org/ftp/python/3.14.6/python-3.14.6-macos11.pkg", sha256 = "d3c9fff52214847e4fab03e9eaf53dd2a8e51e3534aa0b61f201b749f86bef28" },
|
||||
{ identifier = "cp314-macosx_universal2", version = "3.14", url = "https://www.python.org/ftp/python/3.14.6/python-3.14.6-macos11.pkg", sha256 = "d3c9fff52214847e4fab03e9eaf53dd2a8e51e3534aa0b61f201b749f86bef28" },
|
||||
{ identifier = "cp314t-macosx_x86_64", version = "3.14", url = "https://www.python.org/ftp/python/3.14.6/python-3.14.6-macos11.pkg", sha256 = "d3c9fff52214847e4fab03e9eaf53dd2a8e51e3534aa0b61f201b749f86bef28" },
|
||||
{ identifier = "cp314t-macosx_arm64", version = "3.14", url = "https://www.python.org/ftp/python/3.14.6/python-3.14.6-macos11.pkg", sha256 = "d3c9fff52214847e4fab03e9eaf53dd2a8e51e3534aa0b61f201b749f86bef28" },
|
||||
{ identifier = "cp314t-macosx_universal2", version = "3.14", url = "https://www.python.org/ftp/python/3.14.6/python-3.14.6-macos11.pkg", sha256 = "d3c9fff52214847e4fab03e9eaf53dd2a8e51e3534aa0b61f201b749f86bef28" },
|
||||
{ identifier = "cp315-macosx_x86_64", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b4-macos11.pkg", sha256 = "6f1ed07cc92ed6e694b38b29d929a4e55190532fddb3828afe68d07cd4bf4aff" },
|
||||
{ identifier = "cp315-macosx_arm64", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b4-macos11.pkg", sha256 = "6f1ed07cc92ed6e694b38b29d929a4e55190532fddb3828afe68d07cd4bf4aff" },
|
||||
{ identifier = "cp315-macosx_universal2", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b4-macos11.pkg", sha256 = "6f1ed07cc92ed6e694b38b29d929a4e55190532fddb3828afe68d07cd4bf4aff" },
|
||||
{ identifier = "cp315t-macosx_x86_64", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b4-macos11.pkg", sha256 = "6f1ed07cc92ed6e694b38b29d929a4e55190532fddb3828afe68d07cd4bf4aff" },
|
||||
{ identifier = "cp315t-macosx_arm64", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b4-macos11.pkg", sha256 = "6f1ed07cc92ed6e694b38b29d929a4e55190532fddb3828afe68d07cd4bf4aff" },
|
||||
{ identifier = "cp315t-macosx_universal2", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b4-macos11.pkg", sha256 = "6f1ed07cc92ed6e694b38b29d929a4e55190532fddb3828afe68d07cd4bf4aff" },
|
||||
{ identifier = "pp39-macosx_x86_64", version = "3.9", url = "https://downloads.python.org/pypy/pypy3.9-v7.3.16-macos_x86_64.tar.gz", sha256 = "7ea762a1b0af7545a74cd4260bb3fd2812813deffef2ef8d2fddd5e03fa63ef1" },
|
||||
{ identifier = "pp39-macosx_arm64", version = "3.9", url = "https://downloads.python.org/pypy/pypy3.9-v7.3.16-macos_arm64.tar.gz", sha256 = "b09ce3a865c1c0f4e58c0ee4f70511f1b27dfc1f7148f4b6060cfdfd581c9e82" },
|
||||
{ identifier = "pp310-macosx_x86_64", version = "3.10", url = "https://downloads.python.org/pypy/pypy3.10-v7.3.19-macos_x86_64.tar.gz", sha256 = "63f9d30cdd9e47cfe279be941975ac3d24e46c79756cd1d1bd2c551c27d8b32c" },
|
||||
{ identifier = "pp310-macosx_arm64", version = "3.10", url = "https://downloads.python.org/pypy/pypy3.10-v7.3.19-macos_arm64.tar.gz", sha256 = "72a74cf645e43608f7d88831581fb0f8e1f963fb38f0793bacd37d5018a9aeb8" },
|
||||
{ identifier = "pp311-macosx_x86_64", version = "3.11", url = "https://downloads.python.org/pypy/pypy3.11-v7.3.23-macos_x86_64.tar.gz", sha256 = "798279b0f233893ae22754290aedf35a1aea1ec9013a8ad8110f166d85944c91" },
|
||||
{ identifier = "pp311-macosx_arm64", version = "3.11", url = "https://downloads.python.org/pypy/pypy3.11-v7.3.23-macos_arm64.tar.gz", sha256 = "f837b3c203d60d7e25a31e267671a07ba0c9b9ac8081ad225571fb06fccf1b7b" },
|
||||
{ identifier = "gp312_250-macosx_x86_64", version = "3.12", url = "https://github.com/oracle/graalpython/releases/download/graal-25.0.1/graalpy-25.0.1-macos-amd64.tar.gz", sha256 = "10b0721d52397f0cc85f038900318da2203711cbcfae7899e3faed49d3dc6221" },
|
||||
{ identifier = "gp312_250-macosx_arm64", version = "3.12", url = "https://github.com/oracle/graalpython/releases/download/graal-25.0.3/graalpy-25.0.3-macos-aarch64.tar.gz", sha256 = "387d9f5b376860842bea4d55aae5820974f3f3b68fc36c77ae863a061a888857" },
|
||||
{ identifier = "cp27-macosx_x86_64", version = "2.7", url = "https://www.python.org/ftp/python/2.7.18/python-2.7.18-macosx10.9.pkg" },
|
||||
{ identifier = "cp35-macosx_x86_64", version = "3.5", url = "https://www.python.org/ftp/python/3.5.4/python-3.5.4-macosx10.6.pkg" },
|
||||
{ identifier = "cp36-macosx_x86_64", version = "3.6", url = "https://www.python.org/ftp/python/3.6.8/python-3.6.8-macosx10.9.pkg" },
|
||||
{ identifier = "cp37-macosx_x86_64", version = "3.7", url = "https://www.python.org/ftp/python/3.7.9/python-3.7.9-macosx10.9.pkg" },
|
||||
{ identifier = "cp38-macosx_x86_64", version = "3.8", url = "https://www.python.org/ftp/python/3.8.9/python-3.8.9-macosx10.9.pkg" },
|
||||
{ identifier = "cp39-macosx_x86_64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.4/python-3.9.4-macos11.pkg" },
|
||||
{ identifier = "cp39-macosx_arm64", version = "3.9", url = "https://www.python.org/ftp/python/3.9.4/python-3.9.4-macos11.pkg" },
|
||||
{ identifier = "cp39-macosx_universal2", version = "3.9", url = "https://www.python.org/ftp/python/3.9.4/python-3.9.4-macos11.pkg" },
|
||||
{ identifier = "pp27-macosx_x86_64", version = "2.7", url = "https://downloads.python.org/pypy/pypy2.7-v7.3.3-osx64.tar.bz2" },
|
||||
{ identifier = "pp36-macosx_x86_64", version = "3.6", url = "https://downloads.python.org/pypy/pypy3.6-v7.3.3-osx64.tar.bz2" },
|
||||
{ identifier = "pp37-macosx_x86_64", version = "3.7", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.3-osx64.tar.bz2" },
|
||||
]
|
||||
|
||||
[windows]
|
||||
python_configurations = [
|
||||
{ identifier = "cp39-win32", version = "3.9.13" },
|
||||
{ identifier = "cp39-win_amd64", version = "3.9.13" },
|
||||
{ identifier = "cp310-win32", version = "3.10.11" },
|
||||
{ identifier = "cp310-win_amd64", version = "3.10.11" },
|
||||
{ identifier = "cp311-win32", version = "3.11.9" },
|
||||
{ identifier = "cp311-win_amd64", version = "3.11.9" },
|
||||
{ identifier = "cp312-win32", version = "3.12.10" },
|
||||
{ identifier = "cp312-win_amd64", version = "3.12.10" },
|
||||
{ identifier = "cp313-win32", version = "3.13.14" },
|
||||
{ identifier = "cp313-win_amd64", version = "3.13.14" },
|
||||
{ identifier = "cp314-win32", version = "3.14.6" },
|
||||
{ identifier = "cp314t-win32", version = "3.14.6" },
|
||||
{ identifier = "cp314-win_amd64", version = "3.14.6" },
|
||||
{ identifier = "cp314t-win_amd64", version = "3.14.6" },
|
||||
{ identifier = "cp315-win32", version = "3.15.0-b4" },
|
||||
{ identifier = "cp315t-win32", version = "3.15.0-b4" },
|
||||
{ identifier = "cp315-win_amd64", version = "3.15.0-b4" },
|
||||
{ identifier = "cp315t-win_amd64", version = "3.15.0-b4" },
|
||||
{ identifier = "cp39-win_arm64", version = "3.9.10" },
|
||||
{ identifier = "cp310-win_arm64", version = "3.10.11" },
|
||||
{ identifier = "cp311-win_arm64", version = "3.11.9" },
|
||||
{ identifier = "cp312-win_arm64", version = "3.12.10" },
|
||||
{ identifier = "cp313-win_arm64", version = "3.13.14" },
|
||||
{ identifier = "cp314-win_arm64", version = "3.14.6" },
|
||||
{ identifier = "cp314t-win_arm64", version = "3.14.6" },
|
||||
{ identifier = "cp315-win_arm64", version = "3.15.0-b4" },
|
||||
{ identifier = "cp315t-win_arm64", version = "3.15.0-b4" },
|
||||
{ identifier = "pp39-win_amd64", version = "3.9", url = "https://downloads.python.org/pypy/pypy3.9-v7.3.16-win64.zip", sha256 = "06ec12a5e964dc0ad33e6f380185a4d295178dce6d6df512f508e7aee00a1323" },
|
||||
{ identifier = "pp310-win_amd64", version = "3.10", url = "https://downloads.python.org/pypy/pypy3.10-v7.3.19-win64.zip", sha256 = "c0d07bba6c8fb4e5804f4a8b3f8ef07cc3d89f6ad1db42a45ffb9be60bbb7cc2" },
|
||||
{ identifier = "pp311-win_amd64", version = "3.11", url = "https://downloads.python.org/pypy/pypy3.11-v7.3.23-win64.zip", sha256 = "948b8ea58dea5b9917210fe4afd242c788fbfaba1c3f1a25e696a404f703389a" },
|
||||
{ identifier = "gp312_250-win_amd64", version = "3.12", url = "https://github.com/oracle/graalpython/releases/download/graal-25.0.3/graalpy-25.0.3-windows-amd64.zip", sha256 = "2ae5c42b93e08d9e106a017634a3442b272ccae6d08ace157244de0449e937d4" },
|
||||
]
|
||||
|
||||
[pyodide]
|
||||
python_configurations = [
|
||||
{ identifier = "cp312-pyodide_wasm32", version = "3.12", default_pyodide_version = "0.27.7", node_version = "v22", sha256 = "0c2e7db42efa5d1dac38b50f8b3d659a1e3885d0a233494831b8206281307d52" },
|
||||
{ identifier = "cp313-pyodide_wasm32", version = "3.13", default_pyodide_version = "0.29.4", node_version = "v22", sha256 = "a29fc4a076408a18fc29eb4b280f80c6dddc95c19514c874c29e590d0931c02a" },
|
||||
{ identifier = "cp314-pyodide_wasm32", version = "3.14", default_pyodide_version = "314.0.2", node_version = "v24", sha256 = "01ab1b229bf2031603bf20ad4ceb9c1059ba680b26df7bda2db4748906dfd78a" },
|
||||
]
|
||||
|
||||
[android]
|
||||
python_configurations = [
|
||||
{ identifier = "cp313-android_arm64_v8a", version = "3.13", url = "https://repo.maven.apache.org/maven2/com/chaquo/python/python/3.13.14/python-3.13.14-aarch64-linux-android.tar.gz", sha256 = "2541c4fa77b6543504c97fa90a71c745225b5800fe25bc8c73a6516b594d0cbc" },
|
||||
{ identifier = "cp313-android_x86_64", version = "3.13", url = "https://repo.maven.apache.org/maven2/com/chaquo/python/python/3.13.14/python-3.13.14-x86_64-linux-android.tar.gz", sha256 = "9d96f7e077677b6651b2e756034ed41ab4479f5f8bf3168a814f458bc2ac137e" },
|
||||
{ identifier = "cp314-android_arm64_v8a", version = "3.14", url = "https://www.python.org/ftp/python/3.14.6/python-3.14.6-aarch64-linux-android.tar.gz", sha256 = "38bbe77d3167b5cd554e03b1021324926f09f3825202b065951dd7638e9c37e5" },
|
||||
{ identifier = "cp314-android_x86_64", version = "3.14", url = "https://www.python.org/ftp/python/3.14.6/python-3.14.6-x86_64-linux-android.tar.gz", sha256 = "e04eb26607627e68d148f89de793372f54a345c91b13628567f24abcdd3bfa3e" },
|
||||
{ identifier = "cp315-android_arm64_v8a", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-aarch64-linux-android.tar.gz", sha256 = "de5e36ffc3c33b63a1ed53fe83db4dee16b003a3180ac35e5c4c203d3750e076" },
|
||||
{ identifier = "cp315-android_x86_64", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b2-x86_64-linux-android.tar.gz", sha256 = "e05ef9c142ea83b41b194b16014ffe2249d27783313c3b00b97f6c93d9a56b92" },
|
||||
]
|
||||
|
||||
[ios]
|
||||
python_configurations = [
|
||||
{ identifier = "cp313-ios_arm64_iphoneos", version = "3.13", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.13-b14/Python-3.13-iOS-support.b14.tar.gz", sha256 = "8b5cb76ef8d8a2946052479358eeec9d54b4496cb60920e175ec1489b5cf7963" },
|
||||
{ identifier = "cp313-ios_x86_64_iphonesimulator", version = "3.13", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.13-b14/Python-3.13-iOS-support.b14.tar.gz", sha256 = "8b5cb76ef8d8a2946052479358eeec9d54b4496cb60920e175ec1489b5cf7963" },
|
||||
{ identifier = "cp313-ios_arm64_iphonesimulator", version = "3.13", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.13-b14/Python-3.13-iOS-support.b14.tar.gz", sha256 = "8b5cb76ef8d8a2946052479358eeec9d54b4496cb60920e175ec1489b5cf7963" },
|
||||
{ identifier = "cp314-ios_arm64_iphoneos", version = "3.14", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.14-b10/Python-3.14-iOS-support.b10.tar.gz", sha256 = "200ef60eb67be0483ceb638daa9048f84f41a9a952707a5ad4c3198037c7b583" },
|
||||
{ identifier = "cp314-ios_x86_64_iphonesimulator", version = "3.14", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.14-b10/Python-3.14-iOS-support.b10.tar.gz", sha256 = "200ef60eb67be0483ceb638daa9048f84f41a9a952707a5ad4c3198037c7b583" },
|
||||
{ identifier = "cp314-ios_arm64_iphonesimulator", version = "3.14", url = "https://github.com/beeware/Python-Apple-support/releases/download/3.14-b10/Python-3.14-iOS-support.b10.tar.gz", sha256 = "200ef60eb67be0483ceb638daa9048f84f41a9a952707a5ad4c3198037c7b583" },
|
||||
{ identifier = "cp315-ios_arm64_iphoneos", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b4-iOS-XCframework.tar.gz", sha256 = "dbf1095502e0b4a0dc2a4d96ae1b1d92ee8c0b7bea773862f7124ede0017e3dc" },
|
||||
{ identifier = "cp315-ios_x86_64_iphonesimulator", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b4-iOS-XCframework.tar.gz", sha256 = "dbf1095502e0b4a0dc2a4d96ae1b1d92ee8c0b7bea773862f7124ede0017e3dc" },
|
||||
{ identifier = "cp315-ios_arm64_iphonesimulator", version = "3.15", url = "https://www.python.org/ftp/python/3.15.0/python-3.15.0b4-iOS-XCframework.tar.gz", sha256 = "dbf1095502e0b4a0dc2a4d96ae1b1d92ee8c0b7bea773862f7124ede0017e3dc" },
|
||||
{ identifier = "cp27-win32", version = "2.7.18", arch = "32" },
|
||||
{ identifier = "cp27-win_amd64", version = "2.7.18", arch = "64" },
|
||||
{ identifier = "cp35-win32", version = "3.5.4", arch = "32" },
|
||||
{ identifier = "cp35-win_amd64", version = "3.5.4", arch = "64" },
|
||||
{ identifier = "cp36-win32", version = "3.6.8", arch = "32" },
|
||||
{ identifier = "cp36-win_amd64", version = "3.6.8", arch = "64" },
|
||||
{ identifier = "cp37-win32", version = "3.7.9", arch = "32" },
|
||||
{ identifier = "cp37-win_amd64", version = "3.7.9", arch = "64" },
|
||||
{ identifier = "cp38-win32", version = "3.8.9", arch = "32" },
|
||||
{ identifier = "cp38-win_amd64", version = "3.8.9", arch = "64" },
|
||||
{ identifier = "cp39-win32", version = "3.9.4", arch = "32" },
|
||||
{ identifier = "cp39-win_amd64", version = "3.9.4", arch = "64" },
|
||||
{ identifier = "pp27-win32", version = "2.7", arch = "32", url = "https://downloads.python.org/pypy/pypy2.7-v7.3.3-win32.zip" },
|
||||
{ identifier = "pp36-win32", version = "3.6", arch = "32", url = "https://downloads.python.org/pypy/pypy3.6-v7.3.3-win32.zip" },
|
||||
{ identifier = "pp37-win32", version = "3.7", arch = "32", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.3-win32.zip" },
|
||||
]
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,91 +0,0 @@
|
||||
# This file was autogenerated by uv via the following command:
|
||||
# nox -s update_constraints
|
||||
annotated-types==0.7.0
|
||||
# via pydantic
|
||||
auditwheel-emscripten==0.2.5
|
||||
# via pyodide-build
|
||||
build==1.5.0
|
||||
# via
|
||||
# -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
# pyodide-build
|
||||
certifi==2026.6.17
|
||||
# via requests
|
||||
charset-normalizer==3.4.9
|
||||
# via requests
|
||||
click==8.1.8
|
||||
# via
|
||||
# -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
# pyodide-build
|
||||
# pyodide-cli
|
||||
distlib==0.4.3
|
||||
# via virtualenv
|
||||
filelock==3.31.1
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
idna==3.18
|
||||
# via requests
|
||||
leb128==1.0.9
|
||||
# via auditwheel-emscripten
|
||||
markdown-it-py==4.2.0
|
||||
# via rich
|
||||
mdurl==0.1.2
|
||||
# via markdown-it-py
|
||||
packaging==26.2
|
||||
# via
|
||||
# auditwheel-emscripten
|
||||
# build
|
||||
# pyodide-build
|
||||
# wheel
|
||||
pip==26.1.2
|
||||
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
platformdirs==4.10.1
|
||||
# via
|
||||
# pyodide-build
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
pydantic==2.13.4
|
||||
# via
|
||||
# pyodide-build
|
||||
# pyodide-lock
|
||||
pydantic-core==2.46.4
|
||||
# via pydantic
|
||||
pygments==2.20.0
|
||||
# via rich
|
||||
pyodide-build==0.36.0
|
||||
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
pyodide-cli==0.5.0
|
||||
# via
|
||||
# auditwheel-emscripten
|
||||
# pyodide-build
|
||||
pyodide-lock==0.1.3
|
||||
# via pyodide-build
|
||||
pyproject-hooks==1.2.0
|
||||
# via build
|
||||
python-discovery==1.4.4
|
||||
# via virtualenv
|
||||
requests==2.34.2
|
||||
# via pyodide-build
|
||||
rich==15.0.0
|
||||
# via
|
||||
# pyodide-build
|
||||
# pyodide-cli
|
||||
ruamel-yaml==0.19.1
|
||||
# via pyodide-build
|
||||
typing-extensions==4.16.0
|
||||
# via
|
||||
# pydantic
|
||||
# pydantic-core
|
||||
# typing-inspection
|
||||
typing-inspection==0.4.2
|
||||
# via pydantic
|
||||
urllib3==2.7.0
|
||||
# via requests
|
||||
virtualenv==21.6.1
|
||||
# via
|
||||
# build
|
||||
# pyodide-build
|
||||
wheel==0.47.0
|
||||
# via
|
||||
# auditwheel-emscripten
|
||||
# pyodide-build
|
||||
@@ -1,91 +0,0 @@
|
||||
# This file was autogenerated by uv via the following command:
|
||||
# nox -s update_constraints
|
||||
annotated-types==0.7.0
|
||||
# via pydantic
|
||||
auditwheel-emscripten==0.2.5
|
||||
# via pyodide-build
|
||||
build==1.5.0
|
||||
# via
|
||||
# -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
# pyodide-build
|
||||
certifi==2026.6.17
|
||||
# via requests
|
||||
charset-normalizer==3.4.9
|
||||
# via requests
|
||||
click==8.1.8
|
||||
# via
|
||||
# -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
# pyodide-build
|
||||
# pyodide-cli
|
||||
distlib==0.4.3
|
||||
# via virtualenv
|
||||
filelock==3.31.1
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
idna==3.18
|
||||
# via requests
|
||||
leb128==1.0.9
|
||||
# via auditwheel-emscripten
|
||||
markdown-it-py==4.2.0
|
||||
# via rich
|
||||
mdurl==0.1.2
|
||||
# via markdown-it-py
|
||||
packaging==26.2
|
||||
# via
|
||||
# auditwheel-emscripten
|
||||
# build
|
||||
# pyodide-build
|
||||
# wheel
|
||||
pip==26.1.2
|
||||
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
platformdirs==4.10.1
|
||||
# via
|
||||
# pyodide-build
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
pydantic==2.13.4
|
||||
# via
|
||||
# pyodide-build
|
||||
# pyodide-lock
|
||||
pydantic-core==2.46.4
|
||||
# via pydantic
|
||||
pygments==2.20.0
|
||||
# via rich
|
||||
pyodide-build==0.36.0
|
||||
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
pyodide-cli==0.5.0
|
||||
# via
|
||||
# auditwheel-emscripten
|
||||
# pyodide-build
|
||||
pyodide-lock==0.1.3
|
||||
# via pyodide-build
|
||||
pyproject-hooks==1.2.0
|
||||
# via build
|
||||
python-discovery==1.4.4
|
||||
# via virtualenv
|
||||
requests==2.34.2
|
||||
# via pyodide-build
|
||||
rich==15.0.0
|
||||
# via
|
||||
# pyodide-build
|
||||
# pyodide-cli
|
||||
ruamel-yaml==0.19.1
|
||||
# via pyodide-build
|
||||
typing-extensions==4.16.0
|
||||
# via
|
||||
# pydantic
|
||||
# pydantic-core
|
||||
# typing-inspection
|
||||
typing-inspection==0.4.2
|
||||
# via pydantic
|
||||
urllib3==2.7.0
|
||||
# via requests
|
||||
virtualenv==21.6.1
|
||||
# via
|
||||
# build
|
||||
# pyodide-build
|
||||
wheel==0.47.0
|
||||
# via
|
||||
# auditwheel-emscripten
|
||||
# pyodide-build
|
||||
@@ -1,91 +0,0 @@
|
||||
# This file was autogenerated by uv via the following command:
|
||||
# nox -s update_constraints
|
||||
annotated-types==0.7.0
|
||||
# via pydantic
|
||||
auditwheel-emscripten==0.2.5
|
||||
# via pyodide-build
|
||||
build==1.5.0
|
||||
# via
|
||||
# -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
# pyodide-build
|
||||
certifi==2026.6.17
|
||||
# via requests
|
||||
charset-normalizer==3.4.9
|
||||
# via requests
|
||||
click==8.1.8
|
||||
# via
|
||||
# -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
# pyodide-build
|
||||
# pyodide-cli
|
||||
distlib==0.4.3
|
||||
# via virtualenv
|
||||
filelock==3.31.1
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
idna==3.18
|
||||
# via requests
|
||||
leb128==1.0.9
|
||||
# via auditwheel-emscripten
|
||||
markdown-it-py==4.2.0
|
||||
# via rich
|
||||
mdurl==0.1.2
|
||||
# via markdown-it-py
|
||||
packaging==26.2
|
||||
# via
|
||||
# auditwheel-emscripten
|
||||
# build
|
||||
# pyodide-build
|
||||
# wheel
|
||||
pip==26.1.2
|
||||
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
platformdirs==4.10.1
|
||||
# via
|
||||
# pyodide-build
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
pydantic==2.13.4
|
||||
# via
|
||||
# pyodide-build
|
||||
# pyodide-lock
|
||||
pydantic-core==2.46.4
|
||||
# via pydantic
|
||||
pygments==2.20.0
|
||||
# via rich
|
||||
pyodide-build==0.36.0
|
||||
# via -r .nox/update_constraints/tmp/constraints-pyodide.in
|
||||
pyodide-cli==0.5.0
|
||||
# via
|
||||
# auditwheel-emscripten
|
||||
# pyodide-build
|
||||
pyodide-lock==0.1.3
|
||||
# via pyodide-build
|
||||
pyproject-hooks==1.2.0
|
||||
# via build
|
||||
python-discovery==1.4.4
|
||||
# via virtualenv
|
||||
requests==2.34.2
|
||||
# via pyodide-build
|
||||
rich==15.0.0
|
||||
# via
|
||||
# pyodide-build
|
||||
# pyodide-cli
|
||||
ruamel-yaml==0.19.1
|
||||
# via pyodide-build
|
||||
typing-extensions==4.16.0
|
||||
# via
|
||||
# pydantic
|
||||
# pydantic-core
|
||||
# typing-inspection
|
||||
typing-inspection==0.4.2
|
||||
# via pydantic
|
||||
urllib3==2.7.0
|
||||
# via requests
|
||||
virtualenv==21.6.1
|
||||
# via
|
||||
# build
|
||||
# pyodide-build
|
||||
wheel==0.47.0
|
||||
# via
|
||||
# auditwheel-emscripten
|
||||
# pyodide-build
|
||||
@@ -0,0 +1,55 @@
|
||||
#
|
||||
# This file is autogenerated by pip-compile
|
||||
# To update, run:
|
||||
#
|
||||
# bin/update_dependencies.py
|
||||
#
|
||||
appdirs==1.4.4
|
||||
# via virtualenv
|
||||
configparser==4.0.2
|
||||
# via importlib-metadata
|
||||
contextlib2==0.6.0.post1
|
||||
# via
|
||||
# importlib-metadata
|
||||
# importlib-resources
|
||||
delocate==0.8.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.3.1
|
||||
# via virtualenv
|
||||
filelock==3.0.12
|
||||
# via virtualenv
|
||||
importlib-metadata==2.1.1
|
||||
# via virtualenv
|
||||
importlib-resources==3.3.1
|
||||
# via virtualenv
|
||||
pathlib2==2.3.5
|
||||
# via
|
||||
# importlib-metadata
|
||||
# importlib-resources
|
||||
# virtualenv
|
||||
scandir==1.10.0
|
||||
# via pathlib2
|
||||
singledispatch==3.6.1
|
||||
# via importlib-resources
|
||||
six==1.15.0
|
||||
# via
|
||||
# pathlib2
|
||||
# virtualenv
|
||||
typing==3.7.4.3
|
||||
# via importlib-resources
|
||||
virtualenv==20.4.3
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
wheel==0.36.2
|
||||
# via
|
||||
# -r cibuildwheel/resources/constraints.in
|
||||
# delocate
|
||||
zipp==1.2.0
|
||||
# via
|
||||
# importlib-metadata
|
||||
# importlib-resources
|
||||
|
||||
# The following packages are considered to be unsafe in a requirements file:
|
||||
pip==20.3.4
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
setuptools==44.1.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
@@ -1,105 +0,0 @@
|
||||
# This file was autogenerated by uv via the following command:
|
||||
# nox -s update_constraints
|
||||
abi3audit==0.0.26
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
abi3info==2025.11.29
|
||||
# via abi3audit
|
||||
altgraph==0.17.5
|
||||
# via macholib
|
||||
attrs==26.1.0
|
||||
# via
|
||||
# cattrs
|
||||
# requests-cache
|
||||
auditwheel==6.7.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
build==1.5.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
cattrs==26.1.0
|
||||
# via requests-cache
|
||||
certifi==2026.6.17
|
||||
# via requests
|
||||
charset-normalizer==3.4.9
|
||||
# via requests
|
||||
delocate==0.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
delvewheel==1.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.4.3
|
||||
# via virtualenv
|
||||
exceptiongroup==1.3.1
|
||||
# via cattrs
|
||||
filelock==3.31.1
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
idna==3.18
|
||||
# via
|
||||
# requests
|
||||
# url-normalize
|
||||
importlib-metadata==9.0.0
|
||||
# via build
|
||||
kaitaistruct==0.11
|
||||
# via abi3audit
|
||||
macholib==1.16.4
|
||||
# via delocate
|
||||
markdown-it-py==4.2.0
|
||||
# via rich
|
||||
mdurl==0.1.2
|
||||
# via markdown-it-py
|
||||
packaging==26.2
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
# build
|
||||
# delocate
|
||||
patchelf==0.17.2.4
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pefile==2024.8.26
|
||||
# via
|
||||
# abi3audit
|
||||
# delvewheel
|
||||
pip==26.1.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pkgconf==3.0.1.post0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
platformdirs==4.10.1
|
||||
# via
|
||||
# python-discovery
|
||||
# requests-cache
|
||||
# virtualenv
|
||||
pyelftools==0.33
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
pygments==2.20.0
|
||||
# via rich
|
||||
pyproject-hooks==1.2.0
|
||||
# via build
|
||||
python-discovery==1.4.4
|
||||
# via virtualenv
|
||||
requests==2.34.2
|
||||
# via
|
||||
# abi3audit
|
||||
# requests-cache
|
||||
requests-cache==1.3.3
|
||||
# via abi3audit
|
||||
rich==15.0.0
|
||||
# via abi3audit
|
||||
tomli==2.4.1
|
||||
# via build
|
||||
typing-extensions==4.16.0
|
||||
# via
|
||||
# cattrs
|
||||
# delocate
|
||||
# exceptiongroup
|
||||
# virtualenv
|
||||
url-normalize==3.0.0
|
||||
# via requests-cache
|
||||
urllib3==2.7.0
|
||||
# via
|
||||
# requests
|
||||
# requests-cache
|
||||
virtualenv==21.6.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
zipp==4.1.0
|
||||
# via importlib-metadata
|
||||
@@ -1,95 +0,0 @@
|
||||
# This file was autogenerated by uv via the following command:
|
||||
# nox -s update_constraints
|
||||
abi3audit==0.0.26
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
abi3info==2025.11.29
|
||||
# via abi3audit
|
||||
altgraph==0.17.5
|
||||
# via macholib
|
||||
attrs==26.1.0
|
||||
# via
|
||||
# cattrs
|
||||
# requests-cache
|
||||
auditwheel==6.7.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
build==1.5.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
cattrs==26.1.0
|
||||
# via requests-cache
|
||||
certifi==2026.6.17
|
||||
# via requests
|
||||
charset-normalizer==3.4.9
|
||||
# via requests
|
||||
delocate==0.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
delvewheel==1.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.4.3
|
||||
# via virtualenv
|
||||
filelock==3.31.1
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
idna==3.18
|
||||
# via
|
||||
# requests
|
||||
# url-normalize
|
||||
kaitaistruct==0.11
|
||||
# via abi3audit
|
||||
macholib==1.16.4
|
||||
# via delocate
|
||||
markdown-it-py==4.2.0
|
||||
# via rich
|
||||
mdurl==0.1.2
|
||||
# via markdown-it-py
|
||||
packaging==26.2
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
# build
|
||||
# delocate
|
||||
patchelf==0.17.2.4
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pefile==2024.8.26
|
||||
# via
|
||||
# abi3audit
|
||||
# delvewheel
|
||||
pip==26.1.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pkgconf==3.0.1.post0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
platformdirs==4.10.1
|
||||
# via
|
||||
# python-discovery
|
||||
# requests-cache
|
||||
# virtualenv
|
||||
pyelftools==0.33
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
pygments==2.20.0
|
||||
# via rich
|
||||
pyproject-hooks==1.2.0
|
||||
# via build
|
||||
python-discovery==1.4.4
|
||||
# via virtualenv
|
||||
requests==2.34.2
|
||||
# via
|
||||
# abi3audit
|
||||
# requests-cache
|
||||
requests-cache==1.3.3
|
||||
# via abi3audit
|
||||
rich==15.0.0
|
||||
# via abi3audit
|
||||
typing-extensions==4.16.0
|
||||
# via
|
||||
# cattrs
|
||||
# delocate
|
||||
url-normalize==3.0.0
|
||||
# via requests-cache
|
||||
urllib3==2.7.0
|
||||
# via
|
||||
# requests
|
||||
# requests-cache
|
||||
virtualenv==21.6.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
@@ -1,97 +0,0 @@
|
||||
# This file was autogenerated by uv via the following command:
|
||||
# nox -s update_constraints
|
||||
abi3audit==0.0.26
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
abi3info==2025.11.29
|
||||
# via abi3audit
|
||||
altgraph==0.17.5
|
||||
# via macholib
|
||||
attrs==26.1.0
|
||||
# via
|
||||
# cattrs
|
||||
# requests-cache
|
||||
auditwheel==6.7.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
build==1.5.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
cattrs==26.1.0
|
||||
# via requests-cache
|
||||
certifi==2026.6.17
|
||||
# via requests
|
||||
charset-normalizer==3.4.9
|
||||
# via requests
|
||||
delocate==0.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
delvewheel==1.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.4.3
|
||||
# via virtualenv
|
||||
filelock==3.31.1; implementation_name != "graalpy"
|
||||
filelock==3.29.7; implementation_name == "graalpy"
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
idna==3.18
|
||||
# via
|
||||
# requests
|
||||
# url-normalize
|
||||
kaitaistruct==0.11
|
||||
# via abi3audit
|
||||
macholib==1.16.4
|
||||
# via delocate
|
||||
markdown-it-py==4.2.0
|
||||
# via rich
|
||||
mdurl==0.1.2
|
||||
# via markdown-it-py
|
||||
packaging==26.2
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
# build
|
||||
# delocate
|
||||
patchelf==0.17.2.4
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pefile==2024.8.26
|
||||
# via
|
||||
# abi3audit
|
||||
# delvewheel
|
||||
pip==26.1.2; implementation_name != "graalpy" or platform_system != "Windows"
|
||||
pip==26.0.1; implementation_name == "graalpy" and platform_system == "Windows"
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pkgconf==3.0.1.post0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
platformdirs==4.10.1
|
||||
# via
|
||||
# python-discovery
|
||||
# requests-cache
|
||||
# virtualenv
|
||||
pyelftools==0.33
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
pygments==2.20.0
|
||||
# via rich
|
||||
pyproject-hooks==1.2.0
|
||||
# via build
|
||||
python-discovery==1.4.4
|
||||
# via virtualenv
|
||||
requests==2.34.2
|
||||
# via
|
||||
# abi3audit
|
||||
# requests-cache
|
||||
requests-cache==1.3.3
|
||||
# via abi3audit
|
||||
rich==15.0.0
|
||||
# via abi3audit
|
||||
typing-extensions==4.16.0
|
||||
# via
|
||||
# cattrs
|
||||
# delocate
|
||||
url-normalize==3.0.0
|
||||
# via requests-cache
|
||||
urllib3==2.7.0
|
||||
# via
|
||||
# requests
|
||||
# requests-cache
|
||||
virtualenv==21.6.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
@@ -1,95 +0,0 @@
|
||||
# This file was autogenerated by uv via the following command:
|
||||
# nox -s update_constraints
|
||||
abi3audit==0.0.26
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
abi3info==2025.11.29
|
||||
# via abi3audit
|
||||
altgraph==0.17.5
|
||||
# via macholib
|
||||
attrs==26.1.0
|
||||
# via
|
||||
# cattrs
|
||||
# requests-cache
|
||||
auditwheel==6.7.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
build==1.5.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
cattrs==26.1.0
|
||||
# via requests-cache
|
||||
certifi==2026.6.17
|
||||
# via requests
|
||||
charset-normalizer==3.4.9
|
||||
# via requests
|
||||
delocate==0.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
delvewheel==1.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.4.3
|
||||
# via virtualenv
|
||||
filelock==3.31.1
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
idna==3.18
|
||||
# via
|
||||
# requests
|
||||
# url-normalize
|
||||
kaitaistruct==0.11
|
||||
# via abi3audit
|
||||
macholib==1.16.4
|
||||
# via delocate
|
||||
markdown-it-py==4.2.0
|
||||
# via rich
|
||||
mdurl==0.1.2
|
||||
# via markdown-it-py
|
||||
packaging==26.2
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
# build
|
||||
# delocate
|
||||
patchelf==0.17.2.4
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pefile==2024.8.26
|
||||
# via
|
||||
# abi3audit
|
||||
# delvewheel
|
||||
pip==26.1.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pkgconf==3.0.1.post0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
platformdirs==4.10.1
|
||||
# via
|
||||
# python-discovery
|
||||
# requests-cache
|
||||
# virtualenv
|
||||
pyelftools==0.33
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
pygments==2.20.0
|
||||
# via rich
|
||||
pyproject-hooks==1.2.0
|
||||
# via build
|
||||
python-discovery==1.4.4
|
||||
# via virtualenv
|
||||
requests==2.34.2
|
||||
# via
|
||||
# abi3audit
|
||||
# requests-cache
|
||||
requests-cache==1.3.3
|
||||
# via abi3audit
|
||||
rich==15.0.0
|
||||
# via abi3audit
|
||||
typing-extensions==4.16.0
|
||||
# via
|
||||
# cattrs
|
||||
# delocate
|
||||
url-normalize==3.0.0
|
||||
# via requests-cache
|
||||
urllib3==2.7.0
|
||||
# via
|
||||
# requests
|
||||
# requests-cache
|
||||
virtualenv==21.6.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
@@ -1,95 +0,0 @@
|
||||
# This file was autogenerated by uv via the following command:
|
||||
# nox -s update_constraints
|
||||
abi3audit==0.0.26
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
abi3info==2025.11.29
|
||||
# via abi3audit
|
||||
altgraph==0.17.5
|
||||
# via macholib
|
||||
attrs==26.1.0
|
||||
# via
|
||||
# cattrs
|
||||
# requests-cache
|
||||
auditwheel==6.7.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
build==1.5.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
cattrs==26.1.0
|
||||
# via requests-cache
|
||||
certifi==2026.6.17
|
||||
# via requests
|
||||
charset-normalizer==3.4.9
|
||||
# via requests
|
||||
delocate==0.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
delvewheel==1.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.4.3
|
||||
# via virtualenv
|
||||
filelock==3.31.1
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
idna==3.18
|
||||
# via
|
||||
# requests
|
||||
# url-normalize
|
||||
kaitaistruct==0.11
|
||||
# via abi3audit
|
||||
macholib==1.16.4
|
||||
# via delocate
|
||||
markdown-it-py==4.2.0
|
||||
# via rich
|
||||
mdurl==0.1.2
|
||||
# via markdown-it-py
|
||||
packaging==26.2
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
# build
|
||||
# delocate
|
||||
patchelf==0.17.2.4
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pefile==2024.8.26
|
||||
# via
|
||||
# abi3audit
|
||||
# delvewheel
|
||||
pip==26.1.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pkgconf==3.0.1.post0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
platformdirs==4.10.1
|
||||
# via
|
||||
# python-discovery
|
||||
# requests-cache
|
||||
# virtualenv
|
||||
pyelftools==0.33
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
pygments==2.20.0
|
||||
# via rich
|
||||
pyproject-hooks==1.2.0
|
||||
# via build
|
||||
python-discovery==1.4.4
|
||||
# via virtualenv
|
||||
requests==2.34.2
|
||||
# via
|
||||
# abi3audit
|
||||
# requests-cache
|
||||
requests-cache==1.3.3
|
||||
# via abi3audit
|
||||
rich==15.0.0
|
||||
# via abi3audit
|
||||
typing-extensions==4.16.0
|
||||
# via
|
||||
# cattrs
|
||||
# delocate
|
||||
url-normalize==3.0.0
|
||||
# via requests-cache
|
||||
urllib3==2.7.0
|
||||
# via
|
||||
# requests
|
||||
# requests-cache
|
||||
virtualenv==21.6.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
@@ -1,95 +0,0 @@
|
||||
# This file was autogenerated by uv via the following command:
|
||||
# nox -s update_constraints
|
||||
abi3audit==0.0.26
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
abi3info==2025.11.29
|
||||
# via abi3audit
|
||||
altgraph==0.17.5
|
||||
# via macholib
|
||||
attrs==26.1.0
|
||||
# via
|
||||
# cattrs
|
||||
# requests-cache
|
||||
auditwheel==6.7.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
build==1.5.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
cattrs==26.1.0
|
||||
# via requests-cache
|
||||
certifi==2026.6.17
|
||||
# via requests
|
||||
charset-normalizer==3.4.9
|
||||
# via requests
|
||||
delocate==0.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
delvewheel==1.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.4.3
|
||||
# via virtualenv
|
||||
filelock==3.31.1
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
idna==3.18
|
||||
# via
|
||||
# requests
|
||||
# url-normalize
|
||||
kaitaistruct==0.11
|
||||
# via abi3audit
|
||||
macholib==1.16.4
|
||||
# via delocate
|
||||
markdown-it-py==4.2.0
|
||||
# via rich
|
||||
mdurl==0.1.2
|
||||
# via markdown-it-py
|
||||
packaging==26.2
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
# build
|
||||
# delocate
|
||||
patchelf==0.17.2.4
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pefile==2024.8.26
|
||||
# via
|
||||
# abi3audit
|
||||
# delvewheel
|
||||
pip==26.1.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pkgconf==3.0.1.post0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
platformdirs==4.10.1
|
||||
# via
|
||||
# python-discovery
|
||||
# requests-cache
|
||||
# virtualenv
|
||||
pyelftools==0.33
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
pygments==2.20.0
|
||||
# via rich
|
||||
pyproject-hooks==1.2.0
|
||||
# via build
|
||||
python-discovery==1.4.4
|
||||
# via virtualenv
|
||||
requests==2.34.2
|
||||
# via
|
||||
# abi3audit
|
||||
# requests-cache
|
||||
requests-cache==1.3.3
|
||||
# via abi3audit
|
||||
rich==15.0.0
|
||||
# via abi3audit
|
||||
typing-extensions==4.16.0
|
||||
# via
|
||||
# cattrs
|
||||
# delocate
|
||||
url-normalize==3.0.0
|
||||
# via requests-cache
|
||||
urllib3==2.7.0
|
||||
# via
|
||||
# requests
|
||||
# requests-cache
|
||||
virtualenv==21.6.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
@@ -0,0 +1,36 @@
|
||||
#
|
||||
# This file is autogenerated by pip-compile
|
||||
# To update, run:
|
||||
#
|
||||
# bin/update_dependencies.py
|
||||
#
|
||||
appdirs==1.4.4
|
||||
# via virtualenv
|
||||
delocate==0.8.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.3.1
|
||||
# via virtualenv
|
||||
filelock==3.0.12
|
||||
# via virtualenv
|
||||
importlib-metadata==2.1.1
|
||||
# via virtualenv
|
||||
importlib-resources==3.2.1
|
||||
# via virtualenv
|
||||
six==1.15.0
|
||||
# via virtualenv
|
||||
virtualenv==20.4.3
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
wheel==0.36.2
|
||||
# via
|
||||
# -r cibuildwheel/resources/constraints.in
|
||||
# delocate
|
||||
zipp==1.2.0
|
||||
# via
|
||||
# importlib-metadata
|
||||
# importlib-resources
|
||||
|
||||
# The following packages are considered to be unsafe in a requirements file:
|
||||
pip==20.3.4
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
setuptools==50.3.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
@@ -0,0 +1,38 @@
|
||||
#
|
||||
# This file is autogenerated by pip-compile
|
||||
# To update, run:
|
||||
#
|
||||
# bin/update_dependencies.py
|
||||
#
|
||||
appdirs==1.4.4
|
||||
# via virtualenv
|
||||
delocate==0.8.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.3.1
|
||||
# via virtualenv
|
||||
filelock==3.0.12
|
||||
# via virtualenv
|
||||
importlib-metadata==3.10.0
|
||||
# via virtualenv
|
||||
importlib-resources==5.1.2
|
||||
# via virtualenv
|
||||
six==1.15.0
|
||||
# via virtualenv
|
||||
typing-extensions==3.7.4.3
|
||||
# via importlib-metadata
|
||||
virtualenv==20.4.3
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
wheel==0.36.2
|
||||
# via
|
||||
# -r cibuildwheel/resources/constraints.in
|
||||
# delocate
|
||||
zipp==3.4.1
|
||||
# via
|
||||
# importlib-metadata
|
||||
# importlib-resources
|
||||
|
||||
# The following packages are considered to be unsafe in a requirements file:
|
||||
pip==21.0.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
setuptools==54.2.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
@@ -0,0 +1,34 @@
|
||||
#
|
||||
# This file is autogenerated by pip-compile
|
||||
# To update, run:
|
||||
#
|
||||
# bin/update_dependencies.py
|
||||
#
|
||||
appdirs==1.4.4
|
||||
# via virtualenv
|
||||
delocate==0.8.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.3.1
|
||||
# via virtualenv
|
||||
filelock==3.0.12
|
||||
# via virtualenv
|
||||
importlib-metadata==3.10.0
|
||||
# via virtualenv
|
||||
six==1.15.0
|
||||
# via virtualenv
|
||||
typing-extensions==3.7.4.3
|
||||
# via importlib-metadata
|
||||
virtualenv==20.4.3
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
wheel==0.36.2
|
||||
# via
|
||||
# -r cibuildwheel/resources/constraints.in
|
||||
# delocate
|
||||
zipp==3.4.1
|
||||
# via importlib-metadata
|
||||
|
||||
# The following packages are considered to be unsafe in a requirements file:
|
||||
pip==21.0.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
setuptools==54.2.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
@@ -0,0 +1,28 @@
|
||||
#
|
||||
# This file is autogenerated by pip-compile
|
||||
# To update, run:
|
||||
#
|
||||
# bin/update_dependencies.py
|
||||
#
|
||||
appdirs==1.4.4
|
||||
# via virtualenv
|
||||
delocate==0.8.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.3.1
|
||||
# via virtualenv
|
||||
filelock==3.0.12
|
||||
# via virtualenv
|
||||
six==1.15.0
|
||||
# via virtualenv
|
||||
virtualenv==20.4.3
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
wheel==0.36.2
|
||||
# via
|
||||
# -r cibuildwheel/resources/constraints.in
|
||||
# delocate
|
||||
|
||||
# The following packages are considered to be unsafe in a requirements file:
|
||||
pip==21.0.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
setuptools==54.2.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
@@ -1,107 +1,28 @@
|
||||
# This file was autogenerated by uv via the following command:
|
||||
# nox -s update_constraints
|
||||
abi3audit==0.0.25
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
abi3info==2025.4.29
|
||||
# via abi3audit
|
||||
altgraph==0.17.5
|
||||
# via macholib
|
||||
attrs==26.1.0
|
||||
# via
|
||||
# cattrs
|
||||
# requests-cache
|
||||
auditwheel==6.4.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
build==1.4.4
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
cattrs==25.3.0
|
||||
# via requests-cache
|
||||
certifi==2026.6.17
|
||||
# via requests
|
||||
charset-normalizer==3.4.9
|
||||
# via requests
|
||||
delocate==0.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
delvewheel==1.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.4.3
|
||||
#
|
||||
# This file is autogenerated by pip-compile
|
||||
# To update, run:
|
||||
#
|
||||
# bin/update_dependencies.py
|
||||
#
|
||||
appdirs==1.4.4
|
||||
# via virtualenv
|
||||
exceptiongroup==1.3.1
|
||||
# via cattrs
|
||||
filelock==3.19.1
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
idna==3.18
|
||||
# via
|
||||
# requests
|
||||
# url-normalize
|
||||
importlib-metadata==8.7.1
|
||||
# via
|
||||
# build
|
||||
# pkgconf
|
||||
kaitaistruct==0.11
|
||||
# via abi3audit
|
||||
macholib==1.16.4
|
||||
# via delocate
|
||||
markdown-it-py==3.0.0
|
||||
# via rich
|
||||
mdurl==0.1.2
|
||||
# via markdown-it-py
|
||||
packaging==25.0
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
# build
|
||||
# delocate
|
||||
patchelf==0.17.2.4
|
||||
delocate==0.8.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pefile==2024.8.26
|
||||
# via
|
||||
# abi3audit
|
||||
# delvewheel
|
||||
pip==26.0.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pkgconf==2.4.3.post2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
platformdirs==4.4.0
|
||||
# via
|
||||
# python-discovery
|
||||
# requests-cache
|
||||
# virtualenv
|
||||
pyelftools==0.32
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
pygments==2.20.0
|
||||
# via rich
|
||||
pyproject-hooks==1.2.0
|
||||
# via build
|
||||
python-discovery==1.4.4
|
||||
distlib==0.3.1
|
||||
# via virtualenv
|
||||
requests==2.32.5
|
||||
filelock==3.0.12
|
||||
# via virtualenv
|
||||
six==1.15.0
|
||||
# via virtualenv
|
||||
virtualenv==20.4.3
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
wheel==0.36.2
|
||||
# via
|
||||
# abi3audit
|
||||
# requests-cache
|
||||
requests-cache==1.2.1
|
||||
# via abi3audit
|
||||
rich==14.2.0
|
||||
# via abi3audit
|
||||
tomli==2.4.1
|
||||
# via build
|
||||
typing-extensions==4.16.0
|
||||
# via
|
||||
# cattrs
|
||||
# -r cibuildwheel/resources/constraints.in
|
||||
# delocate
|
||||
# exceptiongroup
|
||||
# virtualenv
|
||||
url-normalize==2.2.1
|
||||
# via requests-cache
|
||||
urllib3==2.6.3
|
||||
# via
|
||||
# requests
|
||||
# requests-cache
|
||||
virtualenv==21.6.1
|
||||
|
||||
# The following packages are considered to be unsafe in a requirements file:
|
||||
pip==21.0.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
setuptools==54.2.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
zipp==3.23.1
|
||||
# via importlib-metadata
|
||||
|
||||
@@ -1,11 +1,5 @@
|
||||
pip
|
||||
build
|
||||
setuptools
|
||||
wheel
|
||||
delocate
|
||||
delvewheel
|
||||
virtualenv
|
||||
abi3audit
|
||||
|
||||
# Android
|
||||
auditwheel
|
||||
patchelf
|
||||
pkgconf
|
||||
|
||||
@@ -1,95 +1,28 @@
|
||||
# This file was autogenerated by uv via the following command:
|
||||
# nox -s update_constraints
|
||||
abi3audit==0.0.26
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
abi3info==2025.11.29
|
||||
# via abi3audit
|
||||
altgraph==0.17.5
|
||||
# via macholib
|
||||
attrs==26.1.0
|
||||
# via
|
||||
# cattrs
|
||||
# requests-cache
|
||||
auditwheel==6.7.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
build==1.5.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
cattrs==26.1.0
|
||||
# via requests-cache
|
||||
certifi==2026.6.17
|
||||
# via requests
|
||||
charset-normalizer==3.4.9
|
||||
# via requests
|
||||
delocate==0.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
delvewheel==1.13.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
distlib==0.4.3
|
||||
#
|
||||
# This file is autogenerated by pip-compile
|
||||
# To update, run:
|
||||
#
|
||||
# bin/update_dependencies.py
|
||||
#
|
||||
appdirs==1.4.4
|
||||
# via virtualenv
|
||||
filelock==3.31.1
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
idna==3.18
|
||||
# via
|
||||
# requests
|
||||
# url-normalize
|
||||
kaitaistruct==0.11
|
||||
# via abi3audit
|
||||
macholib==1.16.4
|
||||
# via delocate
|
||||
markdown-it-py==4.2.0
|
||||
# via rich
|
||||
mdurl==0.1.2
|
||||
# via markdown-it-py
|
||||
packaging==26.2
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
# build
|
||||
# delocate
|
||||
patchelf==0.17.2.4
|
||||
delocate==0.8.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pefile==2024.8.26
|
||||
# via
|
||||
# abi3audit
|
||||
# delvewheel
|
||||
pip==26.1.2
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
pkgconf==3.0.1.post0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
platformdirs==4.10.1
|
||||
# via
|
||||
# python-discovery
|
||||
# requests-cache
|
||||
# virtualenv
|
||||
pyelftools==0.33
|
||||
# via
|
||||
# abi3audit
|
||||
# auditwheel
|
||||
pygments==2.20.0
|
||||
# via rich
|
||||
pyproject-hooks==1.2.0
|
||||
# via build
|
||||
python-discovery==1.4.4
|
||||
distlib==0.3.1
|
||||
# via virtualenv
|
||||
requests==2.34.2
|
||||
filelock==3.0.12
|
||||
# via virtualenv
|
||||
six==1.15.0
|
||||
# via virtualenv
|
||||
virtualenv==20.4.3
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
wheel==0.36.2
|
||||
# via
|
||||
# abi3audit
|
||||
# requests-cache
|
||||
requests-cache==1.3.3
|
||||
# via abi3audit
|
||||
rich==15.0.0
|
||||
# via abi3audit
|
||||
typing-extensions==4.16.0
|
||||
# via
|
||||
# cattrs
|
||||
# -r cibuildwheel/resources/constraints.in
|
||||
# delocate
|
||||
url-normalize==3.0.0
|
||||
# via requests-cache
|
||||
urllib3==2.7.0
|
||||
# via
|
||||
# requests
|
||||
# requests-cache
|
||||
virtualenv==21.6.1
|
||||
|
||||
# The following packages are considered to be unsafe in a requirements file:
|
||||
pip==21.0.1
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
setuptools==54.2.0
|
||||
# via -r cibuildwheel/resources/constraints.in
|
||||
|
||||
@@ -1,82 +0,0 @@
|
||||
# These are the defaults used by cibuildwheel itself. They should match the values in
|
||||
# generate_schema.py, which are used by external tools for validation and IDE support.
|
||||
|
||||
[tool.cibuildwheel]
|
||||
build = "*"
|
||||
skip = ""
|
||||
test-skip = ""
|
||||
enable = []
|
||||
|
||||
archs = ["auto"]
|
||||
audit-requires = ["abi3audit"]
|
||||
audit-command = "abi3audit --strict --report {abi3_wheel}"
|
||||
build-frontend = "default"
|
||||
config-settings = {}
|
||||
dependency-versions = "pinned"
|
||||
environment = {}
|
||||
environment-pass = []
|
||||
build-verbosity = 0
|
||||
|
||||
before-all = ""
|
||||
before-build = ""
|
||||
# TOML doesn't support explicit NULLs; use ["\u0000"] as a sentinel value.
|
||||
xbuild-tools = ["\u0000"]
|
||||
repair-wheel-command = ""
|
||||
|
||||
test-command = ""
|
||||
before-test = ""
|
||||
test-sources = []
|
||||
test-requires = []
|
||||
test-extras = []
|
||||
test-groups = []
|
||||
test-environment = {}
|
||||
test-runtime = {}
|
||||
|
||||
container-engine = "docker"
|
||||
|
||||
pyodide-version = ""
|
||||
|
||||
manylinux-x86_64-image = "manylinux_2_28"
|
||||
manylinux-i686-image = "manylinux_2_28"
|
||||
manylinux-aarch64-image = "manylinux_2_28"
|
||||
manylinux-ppc64le-image = "manylinux_2_28"
|
||||
manylinux-s390x-image = "manylinux_2_28"
|
||||
manylinux-armv7l-image = "manylinux_2_31"
|
||||
manylinux-riscv64-image = "manylinux_2_39"
|
||||
manylinux-pypy_x86_64-image = "manylinux_2_28"
|
||||
manylinux-pypy_i686-image = "manylinux_2_28"
|
||||
manylinux-pypy_aarch64-image = "manylinux_2_28"
|
||||
|
||||
musllinux-x86_64-image = "musllinux_1_2"
|
||||
musllinux-i686-image = "musllinux_1_2"
|
||||
musllinux-aarch64-image = "musllinux_1_2"
|
||||
musllinux-ppc64le-image = "musllinux_1_2"
|
||||
musllinux-s390x-image = "musllinux_1_2"
|
||||
musllinux-armv7l-image = "musllinux_1_2"
|
||||
musllinux-riscv64-image = "musllinux_1_2"
|
||||
|
||||
[tool.cibuildwheel.xbuild-files]
|
||||
numpy = [
|
||||
"numpy/_core/include/numpy/_numpyconfig.h",
|
||||
"numpy/_core/include/numpy/numpyconfig.h",
|
||||
"numpy/_core/lib/libnpymath.a",
|
||||
"numpy/random/lib/libnpyrandom.a",
|
||||
]
|
||||
|
||||
[tool.cibuildwheel.linux]
|
||||
repair-wheel-command = "auditwheel repair -w {dest_dir} {wheel}"
|
||||
|
||||
[tool.cibuildwheel.macos]
|
||||
repair-wheel-command = "delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}"
|
||||
|
||||
[tool.cibuildwheel.windows]
|
||||
repair-wheel-command = "delvewheel repair -w {dest_dir} -v {wheel}"
|
||||
|
||||
[tool.cibuildwheel.android]
|
||||
repair-wheel-command = "auditwheel repair --ldpaths {ldpaths} -w {dest_dir} {wheel}"
|
||||
|
||||
[tool.cibuildwheel.ios]
|
||||
|
||||
[tool.cibuildwheel.pyodide]
|
||||
audit-command = ""
|
||||
build-frontend = "pyodide-build"
|
||||
@@ -1,14 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<array>
|
||||
<dict>
|
||||
<key>attributeSetting</key>
|
||||
<integer>1</integer>
|
||||
<key>choiceAttribute</key>
|
||||
<string>selected</string>
|
||||
<key>choiceIdentifier</key>
|
||||
<string>org.python.Python.PythonTFramework-3.14</string>
|
||||
</dict>
|
||||
</array>
|
||||
</plist>
|
||||
@@ -1,14 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<array>
|
||||
<dict>
|
||||
<key>attributeSetting</key>
|
||||
<integer>1</integer>
|
||||
<key>choiceAttribute</key>
|
||||
<string>selected</string>
|
||||
<key>choiceIdentifier</key>
|
||||
<string>org.python.Python.PythonTFramework-3.15</string>
|
||||
</dict>
|
||||
</array>
|
||||
</plist>
|
||||
File diff suppressed because it is too large
Load Diff
@@ -6,7 +6,6 @@
|
||||
# for the ssl module. Uses the certificates provided by the certifi package:
|
||||
# https://pypi.org/project/certifi/
|
||||
|
||||
import contextlib
|
||||
import os
|
||||
import os.path
|
||||
import ssl
|
||||
@@ -25,23 +24,28 @@ STAT_0o775 = (
|
||||
| stat.S_IXOTH
|
||||
)
|
||||
|
||||
if sys.version_info[0] == 2:
|
||||
FileNotFoundError = OSError
|
||||
|
||||
def main() -> None:
|
||||
|
||||
def main():
|
||||
openssl_dir, openssl_cafile = os.path.split(ssl.get_default_verify_paths().openssl_cafile)
|
||||
print(" -- pip install --upgrade certifi")
|
||||
subprocess.check_call(
|
||||
[sys.executable, "-E", "-s", "-m", "pip", "install", "--upgrade", "certifi"]
|
||||
)
|
||||
|
||||
import certifi # noqa: PLC0415
|
||||
import certifi
|
||||
|
||||
# change working directory to the default SSL directory
|
||||
os.chdir(openssl_dir)
|
||||
relpath_to_certifi_cafile = os.path.relpath(certifi.where())
|
||||
|
||||
print(" -- removing any existing file or link")
|
||||
with contextlib.suppress(FileNotFoundError):
|
||||
try:
|
||||
os.remove(openssl_cafile)
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
print(" -- creating symlink to certifi certificate bundle")
|
||||
os.symlink(relpath_to_certifi_cafile, openssl_cafile)
|
||||
|
||||
|
||||
@@ -1,103 +0,0 @@
|
||||
import shutil
|
||||
import sys
|
||||
import sysconfig
|
||||
from pathlib import Path
|
||||
|
||||
SITE_PACKAGE_PATH = Path(__file__).parent
|
||||
|
||||
###########################################################################
|
||||
# importlib module patches
|
||||
###########################################################################
|
||||
|
||||
|
||||
def patch_env_create(env):
|
||||
"""
|
||||
Patch the process of creating virtual environments to ensure that the cross
|
||||
environment modification files are also copied as part of environment
|
||||
creation.
|
||||
"""
|
||||
old_pip_env_create = env._PipBackend.create
|
||||
|
||||
def pip_env_create(self, path, *args, **kwargs):
|
||||
result = old_pip_env_create(self, path, *args, **kwargs)
|
||||
# Copy any _cross_*.pth or _cross_*.py file, plus the cross-platform
|
||||
# sysconfigdata module and sysconfig_vars JSON to the new environment.
|
||||
data_name = sysconfig._get_sysconfigdata_name()
|
||||
json_name = data_name.replace("_sysconfigdata", "_sysconfig_vars")
|
||||
for filename in [
|
||||
"_cross_venv.pth",
|
||||
"_cross_venv.py",
|
||||
f"_cross_{sys.implementation._multiarch.replace('-', '_')}.py",
|
||||
f"{data_name}.py",
|
||||
f"{json_name}.json",
|
||||
]:
|
||||
src = SITE_PACKAGE_PATH / filename
|
||||
target = Path(path) / src.relative_to(SITE_PACKAGE_PATH.parent.parent.parent)
|
||||
if not target.exists():
|
||||
shutil.copy(src, target)
|
||||
return result
|
||||
|
||||
env._PipBackend.create = pip_env_create
|
||||
|
||||
|
||||
# Import hook that patches the creation of virtual environments by `build`
|
||||
#
|
||||
# The approach used here is the same as the one used by virtualenv to patch
|
||||
# distutils (but without support for the older load_module API).
|
||||
# https://docs.python.org/3/library/importlib.html#setting-up-an-importer
|
||||
_BUILD_PATCH = ("build.env",)
|
||||
|
||||
|
||||
class _Finder:
|
||||
"""A meta path finder that allows patching the imported build modules."""
|
||||
|
||||
fullname = None
|
||||
|
||||
# lock[0] is threading.Lock(), but initialized lazily to avoid importing
|
||||
# threading very early at startup, because there are gevent-based
|
||||
# applications that need to be first to import threading by themselves.
|
||||
# See https://github.com/pypa/virtualenv/issues/1895 for details.
|
||||
lock = [] # noqa: RUF012
|
||||
|
||||
def find_spec(self, fullname, path, target=None):
|
||||
if fullname in _BUILD_PATCH and self.fullname is None:
|
||||
# initialize lock[0] lazily
|
||||
if len(self.lock) == 0:
|
||||
import threading
|
||||
|
||||
lock = threading.Lock()
|
||||
# there is possibility that two threads T1 and T2 are
|
||||
# simultaneously running into find_spec, observing .lock as
|
||||
# empty, and further going into hereby initialization. However
|
||||
# due to the GIL, list.append() operation is atomic and this
|
||||
# way only one of the threads will "win" to put the lock
|
||||
# - that every thread will use - into .lock[0].
|
||||
# https://docs.python.org/3/faq/library.html#what-kinds-of-global-value-mutation-are-thread-safe
|
||||
self.lock.append(lock)
|
||||
|
||||
from functools import partial
|
||||
from importlib.util import find_spec
|
||||
|
||||
with self.lock[0]:
|
||||
self.fullname = fullname
|
||||
try:
|
||||
spec = find_spec(fullname, path)
|
||||
if spec is not None:
|
||||
# https://www.python.org/dev/peps/pep-0451/#how-loading-will-work
|
||||
old = spec.loader.exec_module
|
||||
func = self.exec_module
|
||||
if old is not func:
|
||||
spec.loader.exec_module = partial(func, old)
|
||||
return spec
|
||||
finally:
|
||||
self.fullname = None
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def exec_module(old, module):
|
||||
old(module)
|
||||
if module.__name__ in _BUILD_PATCH:
|
||||
patch_env_create(module)
|
||||
|
||||
|
||||
sys.meta_path.insert(0, _Finder())
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user