cd38ee1548
* feat: add SHA256 verification for direct downloads
Store SHA256 hashes when running update scripts and verify them
when downloading files at build time. This improves security by
detecting unexpected changes to downloaded artifacts.
Platforms covered: macOS (CPython, PyPy, GraalPy), iOS, Android,
virtualenv, and python-build-standalone. Windows (nuget) and
Linux (Docker) are excluded.
SHA256 sources per platform:
- macOS/iOS/Android CPython (python.org): sha256_sum from API
- GraalPy: .sha256 sidecar assets from GitHub releases
- python-build-standalone: SHA256SUMS file in release
- PyPy, BeeWare iOS, Maven (Chaquopy): stream-download and compute
Changes:
- cibuildwheel/util/file.py: add sha256 param to download()
- cibuildwheel/platforms/{macos,ios,android}.py: add sha256 to
PythonConfiguration and pass to download()
- cibuildwheel/venv.py: read sha256 from toml and pass to download()
- cibuildwheel/util/python_build_standalone.py: add sha256 to
PythonBuildStandaloneAsset and pass to download()
- cibuildwheel/resources/build-platforms.toml: add sha256 fields
- cibuildwheel/resources/virtualenv.toml: add sha256 field
- cibuildwheel/resources/python-build-standalone-releases.json: add sha256
- bin/update_pythons.py: compute/store sha256 per source strategy
- bin/update_virtualenv.py: compute sha256 by streaming download
- bin/update_python_build_standalone.py: parse SHA256SUMS file
Closes #908
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: copilot-cli:claude-sonnet-4.6
* fix: populate sha256 in resource files and fix Windows PythonConfiguration
- Add sha256 field to Windows PythonConfiguration (PyPy/GraalPy have
direct download URLs on Windows too)
- Pass sha256 to install_pypy() and install_graalpy() in windows.py
- Fix update_pythons.py: handle empty sha256 from CPython API (older
versions) by streaming download to compute it; fix condition to
check 'not sha256' rather than 'not in dict'
- Fix update_virtualenv.py: compute sha256 even when version unchanged
but sha256 is empty (first-time population)
- Fix update_python_build_standalone.py: resolve file path relative to
the script itself (not the installed package) so writes go to source
checkout, not the uv cache
- Populate actual sha256 values by running all three update scripts
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: copilot-cli:claude-sonnet-4.6
* fix: also include pyodide
Assisted-by: CopilotCLI:gpt-5.3-codex
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
* fix: PR review comments for cache verification and docs wording
Co-authored-by: henryiii <4616906+henryiii@users.noreply.github.com>
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
* fix: require sha256 for download configs
Require sha256 for URL-backed Python and virtualenv download configs. Update the GraalPy updater to refresh macOS x86_64 entries by selecting the latest release that still has a matching asset, and fill the two missing GraalPy checksums in build-platforms.toml.
Assisted-by: CopilotCLI:gpt-5.4
* ci: remove unit test for bin item
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
* refactor: combine sha256 unit tests into test_sha256.py
Merge pyodide_test.py and python_build_standalone_test.py into a
single unit_test/test_sha256.py since both test sha256-related
behaviour.
Assisted-by: opencode:glm-5
---------
Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: henryiii <4616906+henryiii@users.noreply.github.com>
114 lines
6.3 KiB
Markdown
114 lines
6.3 KiB
Markdown
# Maintaining Pyodide support
|
||
|
||
Last updated: May 2026
|
||
|
||
This page describes how to update cibuildwheel's Pyodide platform code when either:
|
||
|
||
- a new Pyodide alpha release arrives with support for a new [PyEmscripten Platform](https://pyodide.org/en/latest/development/abi.html) (which is tied to updates in Emscripten and CPython versions, compiler/linker flags, and so on), or
|
||
- when that alpha release graduates to a stable one.
|
||
|
||
## Background
|
||
|
||
Pyodide has three types of releases that matter to cibuildwheel:
|
||
|
||
- **Stable** – the most recent full Pyodide release (e.g., `0.29.x` / cp313). This is enabled by default with no special `CIBW_ENABLE` flag needed.
|
||
- **Prerelease** – an alpha/beta/rc Pyodide release that uses the _next_ CPython version (e.g., `314.0.0a1` / cp314). Users must opt in with `CIBW_ENABLE: pyodide-prerelease` to build against this version. This may or may not be available at any given time, depending on the Pyodide release cycle.
|
||
- **End-of-life (EoL)** – older Pyodide stable releases that are no longer the current stable. These are kept available behind `CIBW_ENABLE: pyodide-eol` so that users who still need to build for older Pyodide versions can do so.
|
||
|
||
The guards in `cibuildwheel/selector.py` enforce this distinction. The constraints files under `cibuildwheel/resources/` pin the exact tool versions that go with each build.
|
||
|
||
---
|
||
|
||
## When a new Pyodide prerelease becomes available
|
||
|
||
For example, consider a scenario when Pyodide ships a new `315.0.0a1` with cp315 support.
|
||
|
||
### 1. Add the new Python configuration
|
||
|
||
In `cibuildwheel/resources/build-platforms.toml`, add an entry under `[pyodide]`:
|
||
|
||
```toml
|
||
{ identifier = "cp315-pyodide_wasm32", version = "3.15", default_pyodide_version = "315.0.0a1", node_version = "v24", sha256 = "SHA256" },
|
||
```
|
||
|
||
`version` is the CPython version string, `default_pyodide_version` is the Pyodide release to use when the user does not pin one explicitly (use the latest available alpha/beta for a prerelease entry), and `node_version` is the minimum Node.js major required by that Pyodide release — check the [pyodide-build FAQ](https://pyodide-build.readthedocs.io/en/latest/faq.html#what-node-js-version-do-i-need) for a rudimentary idea of what the correct value is. `sha256` is the checksum of the Pyodide xbuildenv tarball.
|
||
|
||
### 2. Update the prerelease guards in the selector
|
||
|
||
In `cibuildwheel/selector.py`, update the patterns in the `PyodidePrerelease` guards to match the new identifier:
|
||
|
||
```python
|
||
if EnableGroup.PyodidePrerelease not in self.enable and fnmatch(
|
||
build_id, "cp315-pyodide_*"
|
||
):
|
||
return False
|
||
```
|
||
|
||
### 3. Generate and pin a constraints file
|
||
|
||
Run the `update_constraints` `nox` session, which reads `build-platforms.toml` and regenerates all Pyodide constraints files automatically:
|
||
|
||
```bash
|
||
nox -s update_constraints
|
||
```
|
||
|
||
### 4. Update tests
|
||
|
||
Update the unit tests so the new identifier is accepted by the selector with `PyodidePrerelease` enabled and rejected without it. Pyodide-specific integration tests may also need their hardcoded expected-wheel lists extended.
|
||
|
||
## When a Pyodide prerelease becomes stable
|
||
|
||
Pyodide uses a versioning scheme where the stable release for a given CPython version is named `[PythonMajorMinor].0.0`, so the first stable release shipping cp314 will be **`314.0.0`**. See [pyodide/pyodide#6084](https://github.com/pyodide/pyodide/issues/6084) for a rationale of this versioning scheme.
|
||
|
||
### 1. Update the stable entries, and remove (or replace) the prerelease entry
|
||
|
||
In `build-platforms.toml`, update the former prerelease entry's `default_pyodide_version` to the new stable release (e.g. `314.0.0`) and remove the prerelease marker from the identifier if present. Remove previous prerelease entries if they are now obsolete, or update them to the next prerelease if one is available. Move the previous stable version behind `pyodide-eol` rather than dropping it outright (see below).
|
||
|
||
### 2. Disable or update the prerelease guards
|
||
|
||
In `selector.py`:
|
||
|
||
- **If a new prerelease is available**: update the `fnmatch` pattern to the next identifier (e.g. `cp315-pyodide_*`) as described above.
|
||
- **If there is no new prerelease**: comment out the `PyodidePrerelease` logic.
|
||
|
||
### 3. Update the constraints file
|
||
|
||
Run `nox -s update_constraints` to regenerate the constraints file for the newly stable version. If the entry was already in `build-platforms.toml` as a prerelease, its constraints file already exists, and this step just refreshes it against the stable Pyodide release and updates the dependencies' versions.
|
||
|
||
### 4. Update tests
|
||
|
||
Update the unit tests so the newly stable identifier is accepted by the selector without needing `PyodidePrerelease` in the enable set. Pyodide-specific integration tests may also need their hardcoded expected-wheel lists extended.
|
||
|
||
## When an old Pyodide version is to be moved to end-of-life
|
||
|
||
When a Pyodide version is superseded by a new stable release, move it behind the `pyodide-eol` enable flag. We want to allow users who still build for older Pyodide ABIs time to upgrade.
|
||
|
||
### 1. Add the `pyodide-eol` guard in the selector
|
||
|
||
In `cibuildwheel/selector.py`, add (or update) the `PyodideEoL` guard to include the old identifier:
|
||
|
||
```python
|
||
if EnableGroup.PyodideEoL not in self.enable and fnmatch(build_id, "cp312-pyodide_*"):
|
||
return False
|
||
```
|
||
|
||
### 2. Update tests
|
||
|
||
Update the unit tests so the EoL identifier requires `PyodideEoL` to be included in the enable set. The default (no `CIBW_ENABLE`) should exclude it.
|
||
|
||
## When an old Pyodide version is to be fully retired
|
||
|
||
Retirement is not expected to happen on a routine basis. It is only warranted when the Pyodide ecosystem itself has evolved to the point where an older ABI version is considered obsolete – for example, if the surrounding toolchain, packaging standards, or runtime infrastructure have moved on so substantially that building for the older version no longer makes practical sense. Any retirement is to be discussed and agreed upon by Pyodide maintainers before proceeding.
|
||
|
||
### 1. Remove the Python configuration
|
||
|
||
Delete the entry from `build-platforms.toml` and remove the `PyodideEoL` guard for that identifier in `selector.py`.
|
||
|
||
### 2. Delete the constraints file
|
||
|
||
Remove `cibuildwheel/resources/constraints-pyodideXYZ.txt`.
|
||
|
||
### 3. Update tests
|
||
|
||
Remove references to the old identifier from the unit tests, integration tests, and drop any expected-wheel entries for it from the test helper.
|