Files
cibuildwheel/test/test_environment.py
T
Agriya KhetarpalHood ChathamJoe RickerbyJoe RickerbyHenry Schreinerpre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
163864158d feat: Pyodide improvements: version setting, standalone environments (#2002)
* Fix a typo: pyoodide ➡️ pyodide

* Add `pyodide_build_version` attribute

* Add version to xbuildenv log step

* Add version to Emscripten log step

* Use `pyodide-build`'s version for updating constraints

* Bump Pyodide constraints by updating `pyodide-build`

* Add a schema for `pyodide-version`

* Update Pyodide constraints

* Bump `pyodide-build` to new 0.29.0

* Test out another Pyodide identifier

* Update outdated Pyodide constraints

* Add Pyodide version to temp directory name

* Remove Pyodide 0.26.1 from build configurations

* Retrieve + validate + install specific xbuildenvs

* Test wheel builds with Pyodide 0.26.2

* Add correct Pyodide version to identifier temp dir

* Don't pre-call Pyodide xbuildenv search

* Fetch just the stable Pyodide versions

* Refactor search + validation + install into one step

* Move all of it under a lock

* Reorder xbuildenv installation

* Add env and cwd to xbuildenv search call

* Temporarily lower to 0.26.2 target

* Separate out search, validate, install; again

* Run xbuildenv search in `CIBW_CACHE_PATH`

* Remove prior `PYODIDE_ROOT` env vars, copy envs

* Validate doesn't need to depend on searching

* Add file lock when searching xbuildenvs

* Test the original version: 0.26.1

* Update Pyodide constraints

* Update constraints for `pyodide-build` 0.29.0 again

* Bump Pyodide from version 0.26.1 ➡️ version 0.26.4

* Add note on compatibility for macOS + other archs

* Note Pyodide version for Pyodide identifier

* Docs about `CIBW_PYODIDE_VERSION`

* Don't fetch just the stable versions

* Discard a variable that's not used later

* Rename `search_xbuildenv` ➡️ `get_xbuildenv_versions`

* `validate_xbuildenv` ➡️ `validate_xbuildenv_version`

* Replace ordered comment, add newline

* Replace sentence on macOS support

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Capitalise: "pyodide" ➡️ "Pyodide"

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* "work" ➡️ "may succeed"

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Add another job to test a custom Pyodide version

* Handle "v"-prefixed + non-prefixed versions

* Convert to a proper toml-able option, and remove some hardcoded versions

This removes the enscripten and pyodide-build version specs from
pyproject.toml - pyodide-build is spec'd in the constraints file, and
the emscripten version can be read from the pyodide-build output.

* Add a schema entry

* Add docs for CIBW_PYODIDE_VERSION

* Rephrase

* Add tests for pyodide-version

* Apply suggestions from code review

* Add python_build_standalone util

* Hook up to python-build-standalone, removing dependency on host python

* Remove python hard-code in action.yml

* Add log step

* Add workaround for https://github.com/pyodide/pyodide-build/issues/143

* Add emscripten pytest test

* Remove unneeded checks

* Fix a pytest invoke for emscripten

* Generate pyodide-build constraints from the pinned pyodide version

* Remove pyodide python-build-standalone workaround

* Fixup paths from newer version of pyodide-build

* Fix/skip some failing tests

* Use `python -m pytest` on pyodide, even on Linux

* Docs fixes

* Don't call the github API at runtime, cache the release assets instead

* Ignore pylint false positive

* Add version auto-updating for pyodide

* Add support for pyodide 3.13.

* Fix tests for multiple pyodide wheels

* Remove workaround for unreleased pyodide-build

* Rename to "test_pyodide"

* Fix pathname confusion

* Remove extra github actions job

* Fix expectation for test_abi_none

* Fix the custom_repair_wheel test to actually have clashing names

* Fix pinned version test

* Document test-command limitation

* Remove pyodide 0.28.0a1 for now

* Update constraints files

* Docs/test fixes post removing pyodide cp313

* Fix ABI test expectation

* Docs improvements

* Improve some comments

* Remove logic duplication

* remove pyodide special casing

* Refactor constraints code to use a utility script, circumventing import issues

* chore: nicer nox env

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: typo in variable name found by copilot

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Apply suggestions from code review

* Some more Pyodide version updates in the docs section

* We haven't released Pyodide v0.27.6 yet

* Back to the Github URL for cross-build-environments

* `pyodide-build`, not `emsdk` for Windows skips

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Move to a separate `_json_request` function

Co-Authored-By: Hood Chatham <roberthoodchatham@gmail.com>

* Rename "retries" ➡️ "retry_count"

* Add some type hints

* Copy env vars before `UV_CUSTOM_COMPILE_COMMAND`

* Use `HTTPError.headers.get` instead

* Remove extra end quote

* Change download tests URL to `https://badssl.com/`

Co-Authored-By: Joe Rickerby <1244307+joerick@users.noreply.github.com>

* Download size changes, too

* Use jsdelivr for Github asset mirroring

* Bump to Pyodide v0.27.6

* Fix unit tests

* Move to pyodide v0.27.6 again

* Bump to pyodide-build 0.30.4

* Use new URL for cross-build environments metadata

Co-authored-by: Joe Rickerby <joerick@mac.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
Co-authored-by: Joe Rickerby <joerick@mac.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Joe Rickerby <1244307+joerick@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-05-20 16:59:58 -04:00

157 lines
5.3 KiB
Python

import os
import subprocess
import sys
import textwrap
import pytest
from . import test_projects, utils
project_with_environment_asserts = test_projects.new_c_project(
setup_py_add=textwrap.dedent(
r"""
import os
# explode if environment isn't correct, as set in CIBW_ENVIRONMENT
CIBW_TEST_VAR = os.environ.get("CIBW_TEST_VAR")
CIBW_TEST_VAR_2 = os.environ.get("CIBW_TEST_VAR_2")
CIBW_TEST_VAR_3 = os.environ.get("CIBW_TEST_VAR_3")
PATH = os.environ.get("PATH")
if CIBW_TEST_VAR != "a b c":
raise Exception('CIBW_TEST_VAR should equal "a b c". It was "%s"' % CIBW_TEST_VAR)
if CIBW_TEST_VAR_2 != "1":
raise Exception('CIBW_TEST_VAR_2 should equal "1". It was "%s"' % CIBW_TEST_VAR_2)
if CIBW_TEST_VAR_3 != "test string 3":
raise Exception('CIBW_TEST_VAR_3 should equal "test string 3". It was "%s"' % CIBW_TEST_VAR_3)
if "/opt/cibw_test_path" not in PATH:
raise Exception('PATH should contain "/opt/cibw_test_path". It was "%s"' % PATH)
if "$PATH" in PATH:
raise Exception('$PATH should be expanded in PATH. It was "%s"' % PATH)
"""
)
)
def test(tmp_path):
python_echo = f"'{sys.executable}' -c \"import sys; print(*sys.argv[1:])\""
project_dir = tmp_path / "project"
project_with_environment_asserts.generate(project_dir)
# write some information into the CIBW_ENVIRONMENT, for expansion and
# insertion into the environment by cibuildwheel. This is checked
# in setup.py
actual_wheels = utils.cibuildwheel_run(
project_dir,
add_env={
"CIBW_ENVIRONMENT": """CIBW_TEST_VAR="a b c" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3="$(echo 'test string 3')" PATH=$PATH:/opt/cibw_test_path""",
"CIBW_ENVIRONMENT_WINDOWS": f'''CIBW_TEST_VAR="a b c" CIBW_TEST_VAR_2=1 CIBW_TEST_VAR_3="$({python_echo} 'test string 3')" PATH="$PATH;/opt/cibw_test_path"''',
},
single_python=True,
)
# also check that we got the right wheels built
expected_wheels = utils.expected_wheels("spam", "0.1.0", single_python=True)
assert set(actual_wheels) == set(expected_wheels)
def test_overridden_path(tmp_path, capfd):
project_dir = tmp_path / "project"
output_dir = tmp_path / "output"
project = test_projects.new_c_project()
project.generate(project_dir)
output_dir.mkdir()
# mess up PATH, somehow
if utils.get_platform() == "linux":
with pytest.raises(subprocess.CalledProcessError):
utils.cibuildwheel_run(
project_dir,
output_dir=output_dir,
add_env={
"CIBW_BEFORE_ALL": "mkdir new_path && touch new_path/python && chmod +x new_path/python",
"CIBW_ENVIRONMENT": '''PATH="$(pwd)/new_path:$PATH"''',
},
)
else:
new_path = tmp_path / "another_bin"
new_path.mkdir()
(new_path / "python").touch(mode=0o777)
with pytest.raises(subprocess.CalledProcessError):
utils.cibuildwheel_run(
project_dir,
output_dir=output_dir,
add_env={
"NEW_PATH": str(new_path),
"CIBW_ENVIRONMENT": f'''PATH="$NEW_PATH{os.pathsep}$PATH"''',
},
)
assert len(list(output_dir.iterdir())) == 0
captured = capfd.readouterr()
assert "python available on PATH doesn't match our installed instance" in captured.err.replace(
"venv", "installed"
)
@pytest.mark.parametrize(
"build_frontend",
[
pytest.param("pip", marks=utils.skip_if_pyodide("No pip for pyodide")),
pytest.param(
"build",
marks=utils.skip_if_pyodide(
"pyodide doesn't support multiple values for PIP_CONSTRAINT"
),
),
],
)
def test_overridden_pip_constraint(tmp_path, build_frontend):
"""
Verify that users can use PIP_CONSTRAINT to specify a specific version of
a build-system.requires dependency, by asserting the version of pytz in the
setup.py.
"""
project_dir = tmp_path / "project"
project = test_projects.new_c_project(
setup_py_add=textwrap.dedent(
"""
import pytz
assert pytz.__version__ == "2022.4", f"{pytz.__version__!r} != '2022.4'"
"""
)
)
project.files["pyproject.toml"] = textwrap.dedent(
"""
[build-system]
requires = ["setuptools", "pytz"]
build-backend = "setuptools.build_meta"
"""
)
project.generate(project_dir)
if utils.get_platform() == "linux":
# put the constraints file in the project directory, so it's available
# in the docker container
constraints_file = project_dir / "constraints.txt"
else:
constraints_file = tmp_path / "constraints.txt"
constraints_file.write_text("pytz==2022.4")
actual_wheels = utils.cibuildwheel_run(
project_dir,
add_env={
"CIBW_BUILD_FRONTEND": build_frontend,
"PIP_CONSTRAINT": str(constraints_file),
"CIBW_ENVIRONMENT_LINUX": "PIP_CONSTRAINT=./constraints.txt",
},
single_python=True,
)
expected_wheels = utils.expected_wheels("spam", "0.1.0", single_python=True)
assert set(actual_wheels) == set(expected_wheels)