Files
cibuildwheel/noxfile.py
T
Agriya KhetarpalHood ChathamJoe RickerbyJoe RickerbyHenry Schreinerpre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
163864158d feat: Pyodide improvements: version setting, standalone environments (#2002)
* Fix a typo: pyoodide ➡️ pyodide

* Add `pyodide_build_version` attribute

* Add version to xbuildenv log step

* Add version to Emscripten log step

* Use `pyodide-build`'s version for updating constraints

* Bump Pyodide constraints by updating `pyodide-build`

* Add a schema for `pyodide-version`

* Update Pyodide constraints

* Bump `pyodide-build` to new 0.29.0

* Test out another Pyodide identifier

* Update outdated Pyodide constraints

* Add Pyodide version to temp directory name

* Remove Pyodide 0.26.1 from build configurations

* Retrieve + validate + install specific xbuildenvs

* Test wheel builds with Pyodide 0.26.2

* Add correct Pyodide version to identifier temp dir

* Don't pre-call Pyodide xbuildenv search

* Fetch just the stable Pyodide versions

* Refactor search + validation + install into one step

* Move all of it under a lock

* Reorder xbuildenv installation

* Add env and cwd to xbuildenv search call

* Temporarily lower to 0.26.2 target

* Separate out search, validate, install; again

* Run xbuildenv search in `CIBW_CACHE_PATH`

* Remove prior `PYODIDE_ROOT` env vars, copy envs

* Validate doesn't need to depend on searching

* Add file lock when searching xbuildenvs

* Test the original version: 0.26.1

* Update Pyodide constraints

* Update constraints for `pyodide-build` 0.29.0 again

* Bump Pyodide from version 0.26.1 ➡️ version 0.26.4

* Add note on compatibility for macOS + other archs

* Note Pyodide version for Pyodide identifier

* Docs about `CIBW_PYODIDE_VERSION`

* Don't fetch just the stable versions

* Discard a variable that's not used later

* Rename `search_xbuildenv` ➡️ `get_xbuildenv_versions`

* `validate_xbuildenv` ➡️ `validate_xbuildenv_version`

* Replace ordered comment, add newline

* Replace sentence on macOS support

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Capitalise: "pyodide" ➡️ "Pyodide"

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* "work" ➡️ "may succeed"

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Add another job to test a custom Pyodide version

* Handle "v"-prefixed + non-prefixed versions

* Convert to a proper toml-able option, and remove some hardcoded versions

This removes the enscripten and pyodide-build version specs from
pyproject.toml - pyodide-build is spec'd in the constraints file, and
the emscripten version can be read from the pyodide-build output.

* Add a schema entry

* Add docs for CIBW_PYODIDE_VERSION

* Rephrase

* Add tests for pyodide-version

* Apply suggestions from code review

* Add python_build_standalone util

* Hook up to python-build-standalone, removing dependency on host python

* Remove python hard-code in action.yml

* Add log step

* Add workaround for https://github.com/pyodide/pyodide-build/issues/143

* Add emscripten pytest test

* Remove unneeded checks

* Fix a pytest invoke for emscripten

* Generate pyodide-build constraints from the pinned pyodide version

* Remove pyodide python-build-standalone workaround

* Fixup paths from newer version of pyodide-build

* Fix/skip some failing tests

* Use `python -m pytest` on pyodide, even on Linux

* Docs fixes

* Don't call the github API at runtime, cache the release assets instead

* Ignore pylint false positive

* Add version auto-updating for pyodide

* Add support for pyodide 3.13.

* Fix tests for multiple pyodide wheels

* Remove workaround for unreleased pyodide-build

* Rename to "test_pyodide"

* Fix pathname confusion

* Remove extra github actions job

* Fix expectation for test_abi_none

* Fix the custom_repair_wheel test to actually have clashing names

* Fix pinned version test

* Document test-command limitation

* Remove pyodide 0.28.0a1 for now

* Update constraints files

* Docs/test fixes post removing pyodide cp313

* Fix ABI test expectation

* Docs improvements

* Improve some comments

* Remove logic duplication

* remove pyodide special casing

* Refactor constraints code to use a utility script, circumventing import issues

* chore: nicer nox env

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: typo in variable name found by copilot

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Apply suggestions from code review

* Some more Pyodide version updates in the docs section

* We haven't released Pyodide v0.27.6 yet

* Back to the Github URL for cross-build-environments

* `pyodide-build`, not `emsdk` for Windows skips

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Move to a separate `_json_request` function

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Rename "retries" ➡️ "retry_count"

* Add some type hints

* Copy env vars before `UV_CUSTOM_COMPILE_COMMAND`

* Use `HTTPError.headers.get` instead

* Remove extra end quote

* Change download tests URL to `https://badssl.com/`

Co-Authored-By: Joe Rickerby <1244307+joerick@users.noreply.github.com>

* Download size changes, too

* Use jsdelivr for Github asset mirroring

* Bump to Pyodide v0.27.6

* Fix unit tests

* Move to pyodide v0.27.6 again

* Bump to pyodide-build 0.30.4

* Use new URL for cross-build environments metadata

Co-authored-by: Joe Rickerby <joerick@mac.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
Co-authored-by: Joe Rickerby <joerick@mac.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Joe Rickerby <1244307+joerick@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-05-20 16:59:58 -04:00

206 lines
5.5 KiB
Python
Executable File

#!/usr/bin/env -S uv run
# /// script
# dependencies = ["nox>=2025.2.9"]
# ///
"""
cibuildwheel's nox support
Tags:
lint: All linting jobs
update: All update jobs
See sessions with `nox -l`
"""
import os
import shutil
import sys
from pathlib import Path
import nox
nox.needs_version = ">=2025.2.9"
nox.options.default_venv_backend = "uv|virtualenv"
DIR = Path(__file__).parent.resolve()
@nox.session(tags=["lint"])
def lint(session: nox.Session) -> None:
"""
Run the linter.
"""
session.install("pre-commit")
session.run("pre-commit", "run", "--all-files", *session.posargs)
@nox.session(tags=["lint"])
def pylint(session: nox.Session) -> None:
"""
Run pylint.
"""
session.install("pylint>=3.2", "-e.")
session.run("pylint", "cibuildwheel", *session.posargs)
@nox.session
def tests(session: nox.Session) -> None:
"""
Run the unit and regular tests.
"""
pyproject = nox.project.load_toml()
session.install("-e.", *nox.project.dependency_groups(pyproject, "test"))
if session.posargs:
session.run("pytest", *session.posargs)
else:
unit_test_args = ["--run-docker"] if sys.platform.startswith("linux") else []
session.run("pytest", "unit_test", *unit_test_args)
session.run("pytest", "test", "-x", "--durations", "0", "--timeout=2400", "test")
@nox.session(default=False, tags=["update"])
def update_constraints(session: nox.Session) -> None:
"""
Update the dependencies inplace.
"""
session.install("-e.", "click")
resources = Path("cibuildwheel/resources")
if session.venv_backend != "uv":
session.install("uv>=0.1.23")
# CUSTOM_COMPILE_COMMAND is a pip-compile option that tells users how to
# regenerate the constraints files
env = os.environ.copy()
env["UV_CUSTOM_COMPILE_COMMAND"] = f"nox -s {session.name}"
for minor_version in range(8, 15):
python_version = f"3.{minor_version}"
output_file = resources / f"constraints-python{python_version.replace('.', '')}.txt"
session.run(
"uv",
"pip",
"compile",
f"--python-version={python_version}",
"--upgrade",
resources / "constraints.in",
f"--output-file={output_file}",
env=env,
)
shutil.copyfile(
resources / "constraints-python314.txt",
resources / "constraints.txt",
)
build_platforms = nox.project.load_toml(resources / "build-platforms.toml")
pyodides = build_platforms["pyodide"]["python_configurations"]
for pyodide in pyodides:
python_version = ".".join(pyodide["version"].split(".")[:2])
pyodide_version = pyodide["default_pyodide_version"]
tmp_file = Path(session.create_tmp()) / "constraints-pyodide.in"
session.run(
"python",
"bin/generate_pyodide_constraints.py",
"--output-file",
tmp_file,
pyodide_version,
)
output_file = resources / f"constraints-pyodide{python_version.replace('.', '')}.txt"
session.run(
"uv",
"pip",
"compile",
f"--python-version={python_version}",
"--upgrade",
tmp_file,
f"--output-file={output_file}",
env=env,
)
@nox.session(default=False, tags=["update"])
def update_pins(session: nox.Session) -> None:
"""
Update the python, docker, virtualenv, node, and python-build-standalone
version pins inplace.
"""
pyproject = nox.project.load_toml()
session.install("-e.", *nox.project.dependency_groups(pyproject, "bin"))
session.run("python", "bin/update_pythons.py", "--force")
session.run("python", "bin/update_docker.py")
session.run("python", "bin/update_virtualenv.py", "--force")
session.run("python", "bin/update_nodejs.py", "--force")
session.run("python", "bin/update_python_build_standalone.py")
@nox.session(default=False, reuse_venv=True, tags=["update"])
def update_proj(session: nox.Session) -> None:
"""
Update the README inplace.
"""
session.install_and_run_script(
"bin/projects.py",
"docs/data/projects.yml",
*session.posargs,
)
@nox.session(default=False, reuse_venv=True, tags=["update"])
def generate_schema(session: nox.Session) -> None:
"""
Generate the cibuildwheel.schema.json file.
"""
output = session.install_and_run_script("bin/generate_schema.py", silent=True)
assert isinstance(output, str)
DIR.joinpath("cibuildwheel/resources/cibuildwheel.schema.json").write_text(output)
@nox.session(default=False, reuse_venv=True)
def bump_version(session: nox.Session) -> None:
"""
Bump cibuildwheel's version. Interactive.
"""
session.install_and_run_script("bin/bump_version.py")
@nox.session(default=False, python="3.12")
def docs(session: nox.Session) -> None:
"""
Build the docs. Will serve unless --non-interactive
"""
pyproject = nox.project.load_toml()
session.install("-e.", *nox.project.dependency_groups(pyproject, "docs"))
session.run("mkdocs", "serve" if session.interactive else "build", "--strict", *session.posargs)
@nox.session(default=False)
def build(session: nox.Session) -> None:
"""
Build an SDist and wheel.
"""
build_p = DIR.joinpath("build")
if build_p.exists():
shutil.rmtree(build_p)
dist_p = DIR.joinpath("dist")
if dist_p.exists():
shutil.rmtree(dist_p)
session.install("build")
session.run("python", "-m", "build")
if __name__ == "__main__":
nox.main()