Hongru/cibuildwheel
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #1191 from jack1142/add_cirrus_ci_support

Browse Source 
Add support for Cirrus CI
This commit is contained in:
Joe Rickerby
2022-08-30 22:08:46 +01:00
committed by GitHub
parent 35ffadc277 93986db68c
commit 0117165b02
19 changed files with 217 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.cirrus.yml
+57
View File
@@ -0,0 +1,57 @@
run_tests: &RUN_TESTS
install_cibuildwheel_script:
- python -m pip install -e ".[dev]" pytest-custom-exit-code
run_cibuildwheel_tests_script:
- python ./bin/run_tests.py
linux_x86_task:
compute_engine_instance:
image_project: cirrus-images
image: family/docker-builder
platform: linux
cpu: 8
memory: 8G
install_pre_requirements_script:
- apt install -y python3-venv python-is-python3
<<: *RUN_TESTS
linux_aarch64_task:
compute_engine_instance:
image_project: cirrus-images
image: family/docker-builder-arm64
architecture: arm64
platform: linux
cpu: 4
memory: 4G
install_pre_requirements_script:
- apt install -y python3-venv python-is-python3
<<: *RUN_TESTS
windows_x86_task:
# The task takes ~55 minutes while the timeout happens
# after 60 minutes by default, let's allow some wiggle room.
timeout_in: 90m
windows_container:
image: cirrusci/windowsservercore:visualstudio2022
cpu: 8
memory: 8G
install_pre_requirements_script:
- choco install -y --no-progress python3 --version 3.10.6
- refreshenv
- echo PATH=%PATH% >> "%CIRRUS_ENV%"
<<: *RUN_TESTS
macos_arm64_task:
macos_instance:
image: ghcr.io/cirruslabs/macos-monterey-xcode
env:
PATH: /opt/homebrew/opt/python@3.10/bin:$PATH
install_pre_requirements_script:
- brew install python@3.10
- ln -s python3 /opt/homebrew/opt/python@3.10/bin/python
<<: *RUN_TESTS
CI.md
+5 -5
View File
@@ -1,10 +1,10 @@
This is a summary of the Python versions and platforms covered by the different CI platforms:
| | 3.7 | 3.8 | 3.9 | 3.10 |
|----------|-----------------------|---------------------------|----------|----------------|
| Linux | AppVeyor¹ / Travis CI | Azure Pipelines / GitLab | CircleCI | GitHub Actions |
| macOS | AppVeyor¹ / Travis CI | Azure Pipelines | CircleCI | GitHub Actions |
| Windows | AppVeyor¹ / Travis CI | Azure Pipelines | | GitHub Actions |
| | 3.7 | 3.8 | 3.9 | 3.10 |
|----------|-----------------------|---------------------------|----------|---------------------------|
| Linux | AppVeyor¹ / Travis CI | Azure Pipelines / GitLab | CircleCI | GitHub Actions, Cirrus CI |
| macOS | AppVeyor¹ / Travis CI | Azure Pipelines | CircleCI | GitHub Actions, Cirrus CI |
| Windows | AppVeyor¹ / Travis CI | Azure Pipelines | | GitHub Actions, Cirrus CI |
> ¹ AppVeyor only runs the "basic" test to reduce load.
README.md
+14 -9
View File
@@ -8,6 +8,7 @@ cibuildwheel
[![Appveyor status](https://ci.appveyor.com/api/projects/status/gt3vwl88yt0y3hur/branch/main?svg=true)](https://ci.appveyor.com/project/joerick/cibuildwheel/branch/main)
[![CircleCI Status](https://img.shields.io/circleci/build/gh/pypa/cibuildwheel/main?logo=circleci)](https://circleci.com/gh/pypa/cibuildwheel)
[![Azure Status](https://dev.azure.com/joerick0429/cibuildwheel/_apis/build/status/pypa.cibuildwheel?branchName=main)](https://dev.azure.com/joerick0429/cibuildwheel/_build/latest?definitionId=4&branchName=main)
[![Cirrus CI Status](https://img.shields.io/cirrus/github/pypa/cibuildwheel/main?logo=cirrusci)](https://cirrus-ci.com/github/pypa/cibuildwheel)
[Documentation](https://cibuildwheel.readthedocs.org)
@@ -39,7 +40,7 @@ What does it do?
<sup>³ Alpine 3.14 and very briefly 3.15's default python3 [was not able to load](https://github.com/pypa/cibuildwheel/issues/934) musllinux wheels. This has been fixed; please upgrade the python package if using Alpine from before the fix.</sup><br>
- Builds manylinux, musllinux, macOS 10.9+, and Windows wheels for CPython and PyPy
- Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, and GitLab CI
- Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, GitLab CI, and Cirrus CI
- Bundles shared library dependencies on Linux and macOS through [auditwheel](https://github.com/pypa/auditwheel) and [delocate](https://github.com/matthew-brett/delocate)
- Runs your library's tests against the wheel-installed version of your library
@@ -50,16 +51,19 @@ Usage
`cibuildwheel` runs inside a CI service. Supported platforms depend on which service you're using:
| | Linux | macOS | Windows | Linux ARM |
|-----------------|-------|-------|---------|--------------|
| GitHub Actions | ✅ | ✅ | ✅ | ✅¹ |
| Azure Pipelines | ✅ | ✅ | ✅ | |
| Travis CI | ✅ | | ✅ | ✅ |
| AppVeyor | ✅ | ✅ | ✅ | |
| CircleCI | ✅ | ✅ | | |
| Gitlab CI | ✅ | | | |
| | Linux | macOS | Windows | Linux ARM | macOS ARM |
|-----------------|-------|-------|---------|-----------|-----------|
| GitHub Actions | ✅ | ✅ | ✅ | ✅¹ | ✅² |
| Azure Pipelines | ✅ | ✅ | ✅ | | ✅² |
| Travis CI | ✅ | | ✅ | ✅ | ✅² |
| AppVeyor | ✅ | ✅ | ✅ | | ✅² |
| CircleCI | ✅ | ✅ | | | ✅² |
| Gitlab CI | ✅ | | | | ✅² |
| Cirrus CI | ✅ | ✅³ | ✅ | ✅ | ✅ |
<sup>¹ [Requires emulation](https://cibuildwheel.readthedocs.io/en/stable/faq/#emulation), distributed separately. Other services may also support Linux ARM through emulation or third-party build hosts, but these are not tested in our CI.</sup><br>
<sup>² [Uses cross-compilation](https://cibuildwheel.readthedocs.io/en/stable/faq/#universal2). It is not possible to test `arm64` and the `arm64` part of a `universal2` wheel on this CI platform.</sup><br>
<sup>³ [Uses cross-compilation](https://cibuildwheel.readthedocs.io/en/stable/faq/#universal2). Thanks to Rosetta 2 emulation, it is possible to test `x86_64` and both parts of a `universal2` wheel on this CI platform.</sup><br>
<!--intro-end-->
@@ -179,6 +183,7 @@ Here are some repos that use cibuildwheel.
[circleci icon]: docs/data/readme_icons/circleci.svg
[gitlab icon]: docs/data/readme_icons/gitlab.svg
[travisci icon]: docs/data/readme_icons/travisci.svg
[cirrusci icon]: docs/data/readme_icons/cirrusci.svg
[windows icon]: docs/data/readme_icons/windows.svg
[apple icon]: docs/data/readme_icons/apple.svg
[linux icon]: docs/data/readme_icons/linux.svg
bin/projects.py
+1
View File
@@ -32,6 +32,7 @@ ICONS = (
"circleci",
"gitlab",
"travisci",
"cirrusci",
"windows",
"apple",
"linux",
bin/run_example_ci_configs.py
+5
View File
@@ -66,6 +66,11 @@ services = [
dst_config_path=".gitlab-ci.yml",
badge_md="[![Gitlab](https://gitlab.com/pypa/cibuildwheel/badges/{branch}/pipeline.svg)](https://gitlab.com/pypa/cibuildwheel/-/commits/{branch})",
),
CIService(
name="cirrus-ci",
dst_config_path=".cirrus.yml",
badge_md="[![Cirrus CI](https://api.cirrus-ci.com/github/pypa/cibuildwheel.svg?branch={branch})](https://cirrus-ci.com/github/pypa/cibuildwheel/{branch})",
),
]
cibuildwheel/__main__.py
+3 -3
View File
@@ -160,9 +160,9 @@ def build_in_directory(args: CommandLineArguments) -> None:
textwrap.dedent(
"""
cibuildwheel: Unable to detect platform. cibuildwheel should run on your CI server;
Travis CI, AppVeyor, Azure Pipelines, GitHub Actions, CircleCI, and Gitlab are
supported. You can run on your development machine or other CI providers using the
--platform argument. Check --help output for more information.
Travis CI, AppVeyor, Azure Pipelines, GitHub Actions, CircleCI, Gitlab, and Cirrus CI
are supported. You can run on your development machine or other CI providers
using the --platform argument. Check --help output for more information.
"""
),
file=sys.stderr,
cibuildwheel/linux.py
+1
View File
@@ -349,6 +349,7 @@ def build(options: Options, tmp_path: Path) -> None: # pylint: disable=unused-a
If you're building on Travis CI, add `services: [docker]` to
your .travis.yml. If you're building on Circle CI in Linux,
add a `setup_remote_docker` step to your .circleci/config.yml.
If you're building on Cirrus CI, use `docker_builder` task.
"""
),
file=sys.stderr,
cibuildwheel/util.py
+3
View File
@@ -393,6 +393,7 @@ class CIProvider(Enum):
azure_pipelines = "azure_pipelines"
github_actions = "github_actions"
gitlab = "gitlab"
cirrus_ci = "cirrus_ci"
other = "other"
@@ -409,6 +410,8 @@ def detect_ci_provider() -> CIProvider | None:
return CIProvider.github_actions
elif "GITLAB_CI" in os.environ:
return CIProvider.gitlab
elif "CIRRUS_CI" in os.environ:
return CIProvider.cirrus_ci
elif strtobool(os.environ.get("CI", "false")):
return CIProvider.other
else:
docs/data/projects.yml
+1 -1
View File
@@ -4,7 +4,7 @@
# stars: GitHub repo (optional, if different from package, such as for Twisted)
# pypi: The pypi name, if different from the GitHub package name
# os: Operating system list, [windows, apple, linux] (optional)
# ci: [appveyor, github, azurepipelines, circleci, gitlab, travisci] (optional)
# ci: [appveyor, github, azurepipelines, circleci, gitlab, travisci, cirrusci] (optional)
# notes: (text, optional)
- name: abess
docs/data/readme_icons/cirrusci.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" role="img" viewBox="0 0 24 24" width="16px" fill="#606060"><title>Cirrus CI icon</title><path d="M19.422.453a1.113 1.113 0 0 0-1.113 1.115 1.113 1.113 0 0 0 1.112 1.114c1.31 0 2.35 1.042 2.35 2.363 0 1.32-1.04 2.363-2.35 2.363H1.112A1.113 1.113 0 0 0 0 8.52a1.113 1.113 0 0 0 1.113 1.117h18.31c1.308 0 2.35 1.042 2.35 2.363 0 1.32-1.042 2.363-2.35 2.363H1.112A1.113 1.113 0 0 0 0 15.48a1.113 1.113 0 0 0 1.113 1.112h18.31c1.308 0 2.35 1.042 2.35 2.363 0 1.32-1.042 2.363-2.35 2.363H1.112A1.113 1.113 0 0 0 0 22.432a1.113 1.113 0 0 0 1.113 1.115h18.31a1.113 1.113 0 0 0 .206-.022c2.42-.112 4.37-2.12 4.37-4.57 0-1.393-.642-2.634-1.63-3.478C23.356 14.632 24 13.393 24 12c0-1.393-.643-2.632-1.63-3.477C23.357 7.68 24 6.438 24 5.045c0-2.52-2.06-4.592-4.578-4.592z"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 808 B

docs/faq.md
+8
View File
@@ -83,6 +83,14 @@ Here's an example GitHub Actions workflow with a job that builds for Apple Silic
{% include "../examples/github-apple-silicon.yml" %}
```
Here's an example Cirrus CI workflow with a job that builds for macOS Intel through Rosetta 2 emulation and for Apple Silicon natively:
> .cirrus.yml
```yml
{% include "../examples/cirrus-ci-intel-mac.yml" %}
```
### Building non-native architectures using emulation {: #emulation}
cibuildwheel supports building non-native architectures on Linux, via
docs/options.md
+10
View File
@@ -75,6 +75,16 @@ cibuildwheel to run tests, add the following YAML to your CI config file:
CIBW_TEST_COMMAND: "pytest {project}/tests"
```
!!! tab "Cirrus CI"
> .cirrus.yml ([docs](https://cirrus-ci.org/guide/writing-tasks/#environment-variables))
```yaml
env:
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_COMMAND: "pytest {project}/tests"
```
### Configuration file {: #configuration-file}
You can configure cibuildwheel with a config file, such as `pyproject.toml`.
docs/setup.md
+14
View File
@@ -314,6 +314,20 @@ Commit this file, and push to Gitlab. The pipeline should start automatically.
Gitlab will store the built wheels for you - you can access them from the Pipelines view. Check out the Gitlab [docs](https://docs.gitlab.com/ee/ci/yaml/) for more info on this config file.
## Cirrus CI [linux/mac/windows] {: #cirrus-ci}
To build Linux, Mac, and Windows wheels on Cirrus CI, create a `.cirrus.yml` file in your repo,
> .cirrus.yml
```yaml
{% include "../examples/cirrus-ci-minimal.yml" %}
```
Commit this file, enable building of your repo on Cirrus CI, and push.
Cirrus CI will store the built wheels for you - you can access them from the individual task view. Check out the Cirrus CI [docs](https://cirrus-ci.org/guide/writing-tasks/) for more info on this config file.
> ⚠️ Got an error? Check the [FAQ](faq.md).
## Next steps
examples/cirrus-ci-intel-mac.yml
+21
View File
@@ -0,0 +1,21 @@
build_and_store_wheels: &BUILD_AND_STORE_WHEELS
install_cibuildwheel_script:
- python -m pip install cibuildwheel==2.9.0
run_cibuildwheel_script:
- cibuildwheel
wheels_artifacts:
path: "wheelhouse/*"
macos_task:
name: Build macOS x86_64 and arm64 wheels.
macos_instance:
image: ghcr.io/cirruslabs/macos-monterey-xcode
env:
PATH: /opt/homebrew/opt/python@3.10/bin:$PATH
CIBW_ARCHS_MACOS: x86_64 arm64
install_pre_requirements_script:
- brew install python@3.10
- ln -s python3 /opt/homebrew/opt/python@3.10/bin/python
<<: *BUILD_AND_STORE_WHEELS
examples/cirrus-ci-minimal.yml
+60
View File
@@ -0,0 +1,60 @@
build_and_store_wheels: &BUILD_AND_STORE_WHEELS
install_cibuildwheel_script:
- python -m pip install cibuildwheel==2.9.0
run_cibuildwheel_script:
- cibuildwheel
wheels_artifacts:
path: "wheelhouse/*"
linux_x86_task:
name: Build Linux x86 wheels.
compute_engine_instance:
image_project: cirrus-images
image: family/docker-builder
platform: linux
cpu: 4
memory: 4G
install_pre_requirements_script:
- apt install -y python3-venv python-is-python3
<<: *BUILD_AND_STORE_WHEELS
linux_aarch64_task:
name: Build Linux aarch64 wheels.
compute_engine_instance:
image_project: cirrus-images
image: family/docker-builder-arm64
architecture: arm64
platform: linux
cpu: 4
memory: 4G
install_pre_requirements_script:
- apt install -y python3-venv python-is-python3
<<: *BUILD_AND_STORE_WHEELS
windows_x86_task:
name: Build Windows x86 wheels.
windows_container:
image: cirrusci/windowsservercore:visualstudio2022
cpu: 4
memory: 4G
install_pre_requirements_script:
- choco install -y --no-progress python3 --version 3.10.6
- refreshenv
- echo PATH=%PATH% >> "%CIRRUS_ENV%"
<<: *BUILD_AND_STORE_WHEELS
macos_arm64_task:
name: Build macOS arm64 wheels.
macos_instance:
image: ghcr.io/cirruslabs/macos-monterey-xcode
env:
PATH: /opt/homebrew/opt/python@3.10/bin:$PATH
install_pre_requirements_script:
- brew install python@3.10
- ln -s python3 /opt/homebrew/opt/python@3.10/bin/python
<<: *BUILD_AND_STORE_WHEELS
test/test_dependency_versions.py
+4 -1
View File
@@ -1,5 +1,6 @@
from __future__ import annotations
import platform
import re
import textwrap
@@ -60,6 +61,8 @@ def test_pinned_versions(tmp_path, python_version, build_frontend_env):
build_environment = {}
if python_version == "3.6":
if utils.platform == "macos" and platform.machine() == "arm64":
pytest.skip("macOS arm64 does not support Python 3.6")
constraint_filename = "constraints-python36.txt"
build_pattern = "[cp]p36-*"
elif python_version == "3.7":
@@ -120,7 +123,7 @@ def test_dependency_constraints_file(tmp_path, build_frontend_env):
tool_versions = {
"pip": "20.0.2",
"setuptools": "53.0.0",
"wheel": "0.34.2",
"wheel": "0.36.2",
"virtualenv": "20.11.2",
}
test/test_subdir_package.py
+3 -3
View File
@@ -51,13 +51,13 @@ def test(capfd, tmp_path):
add_env={
"CIBW_BEFORE_BUILD": "python {project}/bin/before_build.py",
"CIBW_TEST_COMMAND": "python {package}/test/run_tests.py",
# this shouldn't depend on the version of python, so build only CPython 3.6
"CIBW_BUILD": "cp36-*",
# this shouldn't depend on the version of python, so build only CPython 3.10
"CIBW_BUILD": "cp310-*",
},
)
# check that the expected wheels are produced
expected_wheels = [w for w in utils.expected_wheels("spam", "0.1.0") if "cp36" in w]
expected_wheels = [w for w in utils.expected_wheels("spam", "0.1.0") if "cp310" in w]
assert set(actual_wheels) == set(expected_wheels)
captured = capfd.readouterr()
test/test_testing.py
+5
View File
@@ -142,6 +142,11 @@ def test_failing_test(tmp_path):
# problems with this, so let's check that.
"CIBW_MANYLINUX_I686_IMAGE": "manylinux1",
"CIBW_MANYLINUX_X86_64_IMAGE": "manylinux1",
# CPython 3.8 when running on macOS arm64 is unusual. The build
# always runs in x86_64, so the arm64 tests are not run. See
# #1169 for reasons why. That means the build succeeds, which
# we don't want. So we skip that build.
"CIBW_SKIP": "cp38-macosx_arm64",
},
)
unit_test/main_tests/main_platform_test.py
+1
View File
@@ -19,6 +19,7 @@ def test_unknown_platform_non_ci(monkeypatch, capsys):
monkeypatch.delenv("GITHUB_ACTIONS", raising=False)
monkeypatch.delenv("GITLAB_CI", raising=False)
monkeypatch.delenv("CIRCLECI", raising=False)
monkeypatch.delenv("CIRRUS_CI", raising=False)
monkeypatch.delenv("CIBW_PLATFORM", raising=False)
with pytest.raises(SystemExit) as exit: