Files
cibuildwheel/.github/workflows/test.yml
T
Agriya KhetarpalHood ChathamJoe RickerbyJoe RickerbyHenry Schreinerpre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
163864158d feat: Pyodide improvements: version setting, standalone environments (#2002)
* Fix a typo: pyoodide ➡️ pyodide

* Add `pyodide_build_version` attribute

* Add version to xbuildenv log step

* Add version to Emscripten log step

* Use `pyodide-build`'s version for updating constraints

* Bump Pyodide constraints by updating `pyodide-build`

* Add a schema for `pyodide-version`

* Update Pyodide constraints

* Bump `pyodide-build` to new 0.29.0

* Test out another Pyodide identifier

* Update outdated Pyodide constraints

* Add Pyodide version to temp directory name

* Remove Pyodide 0.26.1 from build configurations

* Retrieve + validate + install specific xbuildenvs

* Test wheel builds with Pyodide 0.26.2

* Add correct Pyodide version to identifier temp dir

* Don't pre-call Pyodide xbuildenv search

* Fetch just the stable Pyodide versions

* Refactor search + validation + install into one step

* Move all of it under a lock

* Reorder xbuildenv installation

* Add env and cwd to xbuildenv search call

* Temporarily lower to 0.26.2 target

* Separate out search, validate, install; again

* Run xbuildenv search in `CIBW_CACHE_PATH`

* Remove prior `PYODIDE_ROOT` env vars, copy envs

* Validate doesn't need to depend on searching

* Add file lock when searching xbuildenvs

* Test the original version: 0.26.1

* Update Pyodide constraints

* Update constraints for `pyodide-build` 0.29.0 again

* Bump Pyodide from version 0.26.1 ➡️ version 0.26.4

* Add note on compatibility for macOS + other archs

* Note Pyodide version for Pyodide identifier

* Docs about `CIBW_PYODIDE_VERSION`

* Don't fetch just the stable versions

* Discard a variable that's not used later

* Rename `search_xbuildenv` ➡️ `get_xbuildenv_versions`

* `validate_xbuildenv` ➡️ `validate_xbuildenv_version`

* Replace ordered comment, add newline

* Replace sentence on macOS support

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Capitalise: "pyodide" ➡️ "Pyodide"

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* "work" ➡️ "may succeed"

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Add another job to test a custom Pyodide version

* Handle "v"-prefixed + non-prefixed versions

* Convert to a proper toml-able option, and remove some hardcoded versions

This removes the enscripten and pyodide-build version specs from
pyproject.toml - pyodide-build is spec'd in the constraints file, and
the emscripten version can be read from the pyodide-build output.

* Add a schema entry

* Add docs for CIBW_PYODIDE_VERSION

* Rephrase

* Add tests for pyodide-version

* Apply suggestions from code review

* Add python_build_standalone util

* Hook up to python-build-standalone, removing dependency on host python

* Remove python hard-code in action.yml

* Add log step

* Add workaround for https://github.com/pyodide/pyodide-build/issues/143

* Add emscripten pytest test

* Remove unneeded checks

* Fix a pytest invoke for emscripten

* Generate pyodide-build constraints from the pinned pyodide version

* Remove pyodide python-build-standalone workaround

* Fixup paths from newer version of pyodide-build

* Fix/skip some failing tests

* Use `python -m pytest` on pyodide, even on Linux

* Docs fixes

* Don't call the github API at runtime, cache the release assets instead

* Ignore pylint false positive

* Add version auto-updating for pyodide

* Add support for pyodide 3.13.

* Fix tests for multiple pyodide wheels

* Remove workaround for unreleased pyodide-build

* Rename to "test_pyodide"

* Fix pathname confusion

* Remove extra github actions job

* Fix expectation for test_abi_none

* Fix the custom_repair_wheel test to actually have clashing names

* Fix pinned version test

* Document test-command limitation

* Remove pyodide 0.28.0a1 for now

* Update constraints files

* Docs/test fixes post removing pyodide cp313

* Fix ABI test expectation

* Docs improvements

* Improve some comments

* Remove logic duplication

* remove pyodide special casing

* Refactor constraints code to use a utility script, circumventing import issues

* chore: nicer nox env

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: typo in variable name found by copilot

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Apply suggestions from code review

* Some more Pyodide version updates in the docs section

* We haven't released Pyodide v0.27.6 yet

* Back to the Github URL for cross-build-environments

* `pyodide-build`, not `emsdk` for Windows skips

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Move to a separate `_json_request` function

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Rename "retries" ➡️ "retry_count"

* Add some type hints

* Copy env vars before `UV_CUSTOM_COMPILE_COMMAND`

* Use `HTTPError.headers.get` instead

* Remove extra end quote

* Change download tests URL to `https://badssl.com/`

Co-Authored-By: Joe Rickerby <1244307+joerick@users.noreply.github.com>

* Download size changes, too

* Use jsdelivr for Github asset mirroring

* Bump to Pyodide v0.27.6

* Fix unit tests

* Move to pyodide v0.27.6 again

* Bump to pyodide-build 0.30.4

* Use new URL for cross-build environments metadata

Co-authored-by: Joe Rickerby <joerick@mac.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
Co-authored-by: Joe Rickerby <joerick@mac.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Joe Rickerby <1244307+joerick@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-05-20 16:59:58 -04:00

243 lines
7.0 KiB
YAML

name: Test
on:
push:
branches:
- main
- 2.x
pull_request:
types:
- opened
- synchronize
- reopened
- labeled
paths-ignore:
- 'docs/**'
- .pre-commit-config.yaml
workflow_dispatch:
# allow manual runs on branches without a PR
concurrency:
group: test-${{ github.ref }}
cancel-in-progress: true
jobs:
lint:
name: Linters (mypy, flake8, etc.)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
id: python
with:
python-version: "3.x"
- uses: pre-commit/action@v3.0.1
- name: PyLint checks
run: pipx run --python "${{ steps.python.outputs.python-path }}" nox -s pylint -- --output-format=github
test:
name: Test on ${{ matrix.os }} (${{ matrix.python_version }})
needs: lint
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, ubuntu-24.04-arm, windows-latest, windows-11-arm, macos-13, macos-15]
python_version: ['3.13']
include:
- os: ubuntu-latest
python_version: '3.11'
- os: ubuntu-latest
python_version: '3.14'
timeout-minutes: 180
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
name: Install Python ${{ matrix.python_version }}
with:
python-version: ${{ matrix.python_version }}
allow-prereleases: true
- uses: astral-sh/setup-uv@v6
# free some space to prevent reaching GHA disk space limits
- name: Clean docker images
if: runner.os == 'Linux'
run: |
docker system prune -a -f
df -h
# for oci_container unit tests
- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v3
- name: Install dependencies
run: |
uv sync --no-dev --group test
- uses: joerick/pr-labels-action@v1.0.9
- name: Set CIBW_ENABLE
shell: bash
run: |
if [[ "${{ github.ref_name }}" == "main" ]]; then
CIBW_ENABLE=all
else
# get the default CIBW_ENABLE value from the test module
CIBW_ENABLE=$(uv run --no-sync python -c 'import sys, test.utils as c; sys.stdout.write(c.DEFAULT_CIBW_ENABLE)')
# if this is a PR, check for labels
if [[ -n "$GITHUB_PR_LABEL_CI_PYPY" ]]; then
CIBW_ENABLE+=" pypy"
fi
if [[ -n "$GITHUB_PR_LABEL_CI_GRAALPY" ]]; then
CIBW_ENABLE+=" graalpy"
fi
fi
echo "CIBW_ENABLE=${CIBW_ENABLE}" >> $GITHUB_ENV
- name: Generate a sample project
run: |
uv run --no-sync -m test.test_projects test.test_0_basic.basic_project sample_proj
- name: Run a sample build (GitHub Action)
uses: ./
with:
package-dir: sample_proj
output-dir: wheelhouse
env:
CIBW_ARCHS_MACOS: x86_64 universal2 arm64
CIBW_BUILD_FRONTEND: 'build[uv]'
- name: Run a sample build (GitHub Action, only)
uses: ./
with:
package-dir: sample_proj
output-dir: wheelhouse_only
only: cp312-${{ runner.os == 'Linux' && (runner.arch == 'ARM64' && 'manylinux_aarch64' || 'manylinux_x86_64') || (runner.os == 'Windows' && 'win_amd64' || 'macosx_x86_64') }}
- name: Create custom configuration file
shell: bash
run: |
cat > sample_proj/cibw.toml <<EOF
[tool.cibuildwheel]
# Only build on CPython 3.12 on native arch
archs = ["native"]
build = "cp312-*"
# Skip musllinux
skip = "*-musllinux*"
EOF
- name: Run a sample build (GitHub Action, config-file)
uses: ./
with:
package-dir: sample_proj
output-dir: wheelhouse_config_file
config-file: sample_proj/cibw.toml
- name: Check Action artifacts
shell: bash
run: |
test $(find wheelhouse -name '*.whl' | wc -l) -ge 1
test $(find wheelhouse_only -name '*.whl' | wc -l) -eq 1
test $(find wheelhouse_config_file -name '*.whl' | wc -l) -eq 1
- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: wheelhouse/*.whl
- name: Test cibuildwheel
run: |
uv run --no-sync bin/run_tests.py ${{ (runner.os == 'Linux' && runner.arch == 'X64') && '--run-podman' || '' }}
emulated-archs:
name: Get qemu emulated architectures
needs: lint
runs-on: ubuntu-latest
outputs:
archs: ${{ steps.archs.outputs.archs }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.x"
- uses: astral-sh/setup-uv@v6
- name: Install dependencies
run: uv sync --no-dev --group test
- name: Get qemu emulated architectures
id: archs
run: |
OUTPUT=$(uv run --no-sync python -c "from json import dumps; from test.utils import EMULATED_ARCHS; print(dumps(EMULATED_ARCHS))")
echo "${OUTPUT}"
echo "archs=${OUTPUT}" >> "$GITHUB_OUTPUT"
test-emulated:
name: Test Linux ${{ matrix.arch }} using qemu
needs: emulated-archs
runs-on: ubuntu-latest
timeout-minutes: 180
strategy:
matrix:
arch: ${{ fromJSON(needs.emulated-archs.outputs.archs) }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.x"
- uses: astral-sh/setup-uv@v6
- name: Install dependencies
run: uv sync --no-dev --group test
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Run the emulation tests
run: uv run --no-sync pytest --run-emulation ${{ matrix.arch }} test/test_emulation.py
test-pyodide:
name: Test pyodide
needs: lint
runs-on: ubuntu-24.04
timeout-minutes: 180
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
name: Install Python 3.12
with:
python-version: '3.12'
- uses: astral-sh/setup-uv@v6
- name: Install dependencies
run: uv sync --no-dev --group test
- name: Generate a sample project
run: |
uv run --no-sync -m test.test_projects test.test_0_basic.basic_project sample_proj
- name: Run a sample build (GitHub Action)
uses: ./
with:
package-dir: sample_proj
output-dir: wheelhouse
env:
CIBW_PLATFORM: pyodide
- name: Run a sample build (GitHub Action) for an overridden Pyodide version
uses: ./
with:
package-dir: sample_proj
output-dir: wheelhouse
# In case this breaks at any point in time, switch to using the latest version
# available or any other version that is not the same as the default one set
# in cibuildwheel/resources/build-platforms.toml.
env:
CIBW_PLATFORM: pyodide
CIBW_BUILD: "cp312*"
CIBW_PYODIDE_VERSION: "0.27.6"
- name: Run tests with 'CIBW_PLATFORM' set to 'pyodide'
run: |
uv run --no-sync ./bin/run_tests.py
env:
CIBW_PLATFORM: pyodide