Merge pull request #1191 from jack1142/add_cirrus_ci_support

Add support for Cirrus CI

.cirrus.yml

@@ -0,0 +1,57 @@
+run_tests: &RUN_TESTS
+  install_cibuildwheel_script:
+    - python -m pip install -e ".[dev]" pytest-custom-exit-code
+  run_cibuildwheel_tests_script:
+    - python ./bin/run_tests.py
+
+
+linux_x86_task:
+  compute_engine_instance:
+    image_project: cirrus-images
+    image: family/docker-builder
+    platform: linux
+    cpu: 8
+    memory: 8G
+
+  install_pre_requirements_script:
+    - apt install -y python3-venv python-is-python3
+  <<: *RUN_TESTS
+
+linux_aarch64_task:
+  compute_engine_instance:
+    image_project: cirrus-images
+    image: family/docker-builder-arm64
+    architecture: arm64
+    platform: linux
+    cpu: 4
+    memory: 4G
+
+  install_pre_requirements_script:
+    - apt install -y python3-venv python-is-python3
+  <<: *RUN_TESTS
+
+windows_x86_task:
+  # The task takes ~55 minutes while the timeout happens
+  # after 60 minutes by default, let's allow some wiggle room.
+  timeout_in: 90m
+  windows_container:
+    image: cirrusci/windowsservercore:visualstudio2022
+    cpu: 8
+    memory: 8G
+
+  install_pre_requirements_script:
+    - choco install -y --no-progress python3 --version 3.10.6
+    - refreshenv
+    - echo PATH=%PATH% >> "%CIRRUS_ENV%"
+  <<: *RUN_TESTS
+
+macos_arm64_task:
+  macos_instance:
+    image: ghcr.io/cirruslabs/macos-monterey-xcode
+
+  env:
+    PATH: /opt/homebrew/opt/python@3.10/bin:$PATH
+  install_pre_requirements_script:
+    - brew install python@3.10
+    - ln -s python3 /opt/homebrew/opt/python@3.10/bin/python
+  <<: *RUN_TESTS

CI.md

@@ -1,10 +1,10 @@
 This is a summary of the Python versions and platforms covered by the different CI platforms:
 
-|          | 3.7                   | 3.8                       | 3.9      | 3.10           |
+|          | 3.7                   | 3.8                       | 3.9      | 3.10                      |
-|----------|-----------------------|---------------------------|----------|----------------|
+|----------|-----------------------|---------------------------|----------|---------------------------|
-| Linux    | AppVeyor¹ / Travis CI | Azure Pipelines / GitLab  | CircleCI | GitHub Actions |
+| Linux    | AppVeyor¹ / Travis CI | Azure Pipelines / GitLab  | CircleCI | GitHub Actions, Cirrus CI |
-| macOS    | AppVeyor¹ / Travis CI | Azure Pipelines           | CircleCI | GitHub Actions |
+| macOS    | AppVeyor¹ / Travis CI | Azure Pipelines           | CircleCI | GitHub Actions, Cirrus CI |
-| Windows  | AppVeyor¹ / Travis CI | Azure Pipelines           |          | GitHub Actions |
+| Windows  | AppVeyor¹ / Travis CI | Azure Pipelines           |          | GitHub Actions, Cirrus CI |
 
 > ¹ AppVeyor only runs the "basic" test to reduce load.
 

README.md

@@ -8,6 +8,7 @@ cibuildwheel
 [![Appveyor status](https://ci.appveyor.com/api/projects/status/gt3vwl88yt0y3hur/branch/main?svg=true)](https://ci.appveyor.com/project/joerick/cibuildwheel/branch/main)
 [![CircleCI Status](https://img.shields.io/circleci/build/gh/pypa/cibuildwheel/main?logo=circleci)](https://circleci.com/gh/pypa/cibuildwheel)
 [![Azure Status](https://dev.azure.com/joerick0429/cibuildwheel/_apis/build/status/pypa.cibuildwheel?branchName=main)](https://dev.azure.com/joerick0429/cibuildwheel/_build/latest?definitionId=4&branchName=main)
+[![Cirrus CI Status](https://img.shields.io/cirrus/github/pypa/cibuildwheel/main?logo=cirrusci)](https://cirrus-ci.com/github/pypa/cibuildwheel)
 
 
 [Documentation](https://cibuildwheel.readthedocs.org)
@@ -39,7 +40,7 @@ What does it do?
 <sup>³ Alpine 3.14 and very briefly 3.15's default python3 [was not able to load](https://github.com/pypa/cibuildwheel/issues/934) musllinux wheels. This has been fixed; please upgrade the python package if using Alpine from before the fix.</sup><br>
 
 - Builds manylinux, musllinux, macOS 10.9+, and Windows wheels for CPython and PyPy
-- Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, and GitLab CI
+- Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, GitLab CI, and Cirrus CI
 - Bundles shared library dependencies on Linux and macOS through [auditwheel](https://github.com/pypa/auditwheel) and [delocate](https://github.com/matthew-brett/delocate)
 - Runs your library's tests against the wheel-installed version of your library
 
@@ -50,16 +51,19 @@ Usage
 
 `cibuildwheel` runs inside a CI service. Supported platforms depend on which service you're using:
 
-|                 | Linux | macOS | Windows | Linux ARM |
+|                 | Linux | macOS | Windows | Linux ARM | macOS ARM |
-|-----------------|-------|-------|---------|--------------|
+|-----------------|-------|-------|---------|-----------|-----------|
-| GitHub Actions  | ✅    | ✅    | ✅      | ✅¹          |
+| GitHub Actions  | ✅    | ✅    | ✅      | ✅¹       | ✅²       |
-| Azure Pipelines | ✅    | ✅    | ✅      |              |
+| Azure Pipelines | ✅    | ✅    | ✅      |           | ✅²       |
-| Travis CI       | ✅    |       | ✅      | ✅           |
+| Travis CI       | ✅    |       | ✅      | ✅        | ✅²       |
-| AppVeyor        | ✅    | ✅    | ✅      |              |
+| AppVeyor        | ✅    | ✅    | ✅      |           | ✅²       |
-| CircleCI        | ✅    | ✅    |         |              |
+| CircleCI        | ✅    | ✅    |         |           | ✅²       |
-| Gitlab CI       | ✅    |       |         |              |
+| Gitlab CI       | ✅    |       |         |           | ✅²       |
+| Cirrus CI       | ✅    | ✅³   | ✅      | ✅        | ✅        |
 
 <sup>¹ [Requires emulation](https://cibuildwheel.readthedocs.io/en/stable/faq/#emulation), distributed separately. Other services may also support Linux ARM through emulation or third-party build hosts, but these are not tested in our CI.</sup><br>
+<sup>² [Uses cross-compilation](https://cibuildwheel.readthedocs.io/en/stable/faq/#universal2). It is not possible to test `arm64` and the `arm64` part of a `universal2` wheel on this CI platform.</sup><br>
+<sup>³ [Uses cross-compilation](https://cibuildwheel.readthedocs.io/en/stable/faq/#universal2). Thanks to Rosetta 2 emulation, it is possible to test `x86_64` and both parts of a `universal2` wheel on this CI platform.</sup><br>
 
 <!--intro-end-->
 
@@ -179,6 +183,7 @@ Here are some repos that use cibuildwheel.
 [circleci icon]: docs/data/readme_icons/circleci.svg
 [gitlab icon]: docs/data/readme_icons/gitlab.svg
 [travisci icon]: docs/data/readme_icons/travisci.svg
+[cirrusci icon]: docs/data/readme_icons/cirrusci.svg
 [windows icon]: docs/data/readme_icons/windows.svg
 [apple icon]: docs/data/readme_icons/apple.svg
 [linux icon]: docs/data/readme_icons/linux.svg

bin/projects.py

@@ -32,6 +32,7 @@ ICONS = (
     "circleci",
     "gitlab",
     "travisci",
+    "cirrusci",
     "windows",
     "apple",
     "linux",

bin/run_example_ci_configs.py

@@ -66,6 +66,11 @@ services = [
         dst_config_path=".gitlab-ci.yml",
         badge_md="[![Gitlab](https://gitlab.com/pypa/cibuildwheel/badges/{branch}/pipeline.svg)](https://gitlab.com/pypa/cibuildwheel/-/commits/{branch})",
     ),
+    CIService(
+        name="cirrus-ci",
+        dst_config_path=".cirrus.yml",
+        badge_md="[![Cirrus CI](https://api.cirrus-ci.com/github/pypa/cibuildwheel.svg?branch={branch})](https://cirrus-ci.com/github/pypa/cibuildwheel/{branch})",
+    ),
 ]
 
 

cibuildwheel/__main__.py

@@ -160,9 +160,9 @@ def build_in_directory(args: CommandLineArguments) -> None:
                 textwrap.dedent(
                     """
                     cibuildwheel: Unable to detect platform. cibuildwheel should run on your CI server;
-                    Travis CI, AppVeyor, Azure Pipelines, GitHub Actions, CircleCI, and Gitlab are
+                    Travis CI, AppVeyor, Azure Pipelines, GitHub Actions, CircleCI, Gitlab, and Cirrus CI
-                    supported. You can run on your development machine or other CI providers using the
+                    are supported. You can run on your development machine or other CI providers
-                    --platform argument. Check --help output for more information.
+                    using the --platform argument. Check --help output for more information.
                     """
                 ),
                 file=sys.stderr,

cibuildwheel/linux.py

@@ -349,6 +349,7 @@ def build(options: Options, tmp_path: Path) -> None:  # pylint: disable=unused-a
                 If you're building on Travis CI, add `services: [docker]` to
                 your .travis.yml. If you're building on Circle CI in Linux,
                 add a `setup_remote_docker` step to your .circleci/config.yml.
+                If you're building on Cirrus CI, use `docker_builder` task.
                 """
             ),
             file=sys.stderr,

cibuildwheel/util.py

@@ -393,6 +393,7 @@ class CIProvider(Enum):
     azure_pipelines = "azure_pipelines"
     github_actions = "github_actions"
     gitlab = "gitlab"
+    cirrus_ci = "cirrus_ci"
     other = "other"
 
 
@@ -409,6 +410,8 @@ def detect_ci_provider() -> CIProvider | None:
         return CIProvider.github_actions
     elif "GITLAB_CI" in os.environ:
         return CIProvider.gitlab
+    elif "CIRRUS_CI" in os.environ:
+        return CIProvider.cirrus_ci
     elif strtobool(os.environ.get("CI", "false")):
         return CIProvider.other
     else:

docs/data/projects.yml

@@ -4,7 +4,7 @@
 # stars: GitHub repo (optional, if different from package, such as for Twisted)
 # pypi: The pypi name, if different from the GitHub package name
 # os: Operating system list, [windows, apple, linux] (optional)
-# ci: [appveyor, github, azurepipelines, circleci, gitlab, travisci] (optional)
+# ci: [appveyor, github, azurepipelines, circleci, gitlab, travisci, cirrusci] (optional)
 # notes: (text, optional)
 
 - name: abess

docs/data/readme_icons/cirrusci.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" role="img" viewBox="0 0 24 24" width="16px" fill="#606060"><title>Cirrus CI icon</title><path d="M19.422.453a1.113 1.113 0 0 0-1.113 1.115 1.113 1.113 0 0 0 1.112 1.114c1.31 0 2.35 1.042 2.35 2.363 0 1.32-1.04 2.363-2.35 2.363H1.112A1.113 1.113 0 0 0 0 8.52a1.113 1.113 0 0 0 1.113 1.117h18.31c1.308 0 2.35 1.042 2.35 2.363 0 1.32-1.042 2.363-2.35 2.363H1.112A1.113 1.113 0 0 0 0 15.48a1.113 1.113 0 0 0 1.113 1.112h18.31c1.308 0 2.35 1.042 2.35 2.363 0 1.32-1.042 2.363-2.35 2.363H1.112A1.113 1.113 0 0 0 0 22.432a1.113 1.113 0 0 0 1.113 1.115h18.31a1.113 1.113 0 0 0 .206-.022c2.42-.112 4.37-2.12 4.37-4.57 0-1.393-.642-2.634-1.63-3.478C23.356 14.632 24 13.393 24 12c0-1.393-.643-2.632-1.63-3.477C23.357 7.68 24 6.438 24 5.045c0-2.52-2.06-4.592-4.578-4.592z"/></svg>

docs/faq.md

@@ -83,6 +83,14 @@ Here's an example GitHub Actions workflow with a job that builds for Apple Silic
 {% include "../examples/github-apple-silicon.yml" %}
 ```
 
+Here's an example Cirrus CI workflow with a job that builds for macOS Intel through Rosetta 2 emulation and for Apple Silicon natively:
+
+> .cirrus.yml
+
+```yml
+{% include "../examples/cirrus-ci-intel-mac.yml" %}
+```
+
 ### Building non-native architectures using emulation  {: #emulation}
 
 cibuildwheel supports building non-native architectures on Linux, via

docs/options.md

@@ -75,6 +75,16 @@ cibuildwheel to run tests, add the following YAML to your CI config file:
         CIBW_TEST_COMMAND: "pytest {project}/tests"
     ```
 
+!!! tab "Cirrus CI"
+
+    > .cirrus.yml ([docs](https://cirrus-ci.org/guide/writing-tasks/#environment-variables))
+
+    ```yaml
+    env:
+      CIBW_TEST_REQUIRES: pytest
+      CIBW_TEST_COMMAND: "pytest {project}/tests"
+    ```
+
 ### Configuration file {: #configuration-file}
 
 You can configure cibuildwheel with a config file, such as `pyproject.toml`.

docs/setup.md

@@ -314,6 +314,20 @@ Commit this file, and push to Gitlab. The pipeline should start automatically.
 
 Gitlab will store the built wheels for you - you can access them from the Pipelines view. Check out the Gitlab [docs](https://docs.gitlab.com/ee/ci/yaml/) for more info on this config file.
 
+## Cirrus CI [linux/mac/windows] {: #cirrus-ci}
+
+To build Linux, Mac, and Windows wheels on Cirrus CI, create a `.cirrus.yml` file in your repo,
+
+> .cirrus.yml
+
+```yaml
+{% include "../examples/cirrus-ci-minimal.yml" %}
+```
+
+Commit this file, enable building of your repo on Cirrus CI, and push.
+
+Cirrus CI will store the built wheels for you - you can access them from the individual task view. Check out the Cirrus CI [docs](https://cirrus-ci.org/guide/writing-tasks/) for more info on this config file.
+
 > ⚠️ Got an error? Check the [FAQ](faq.md).
 
 ## Next steps

examples/cirrus-ci-intel-mac.yml

@@ -0,0 +1,21 @@
+build_and_store_wheels: &BUILD_AND_STORE_WHEELS
+  install_cibuildwheel_script:
+    - python -m pip install cibuildwheel==2.9.0
+  run_cibuildwheel_script:
+    - cibuildwheel
+  wheels_artifacts:
+    path: "wheelhouse/*"
+
+
+macos_task:
+  name: Build macOS x86_64 and arm64 wheels.
+  macos_instance:
+    image: ghcr.io/cirruslabs/macos-monterey-xcode
+
+  env:
+    PATH: /opt/homebrew/opt/python@3.10/bin:$PATH
+    CIBW_ARCHS_MACOS: x86_64 arm64
+  install_pre_requirements_script:
+    - brew install python@3.10
+    - ln -s python3 /opt/homebrew/opt/python@3.10/bin/python
+  <<: *BUILD_AND_STORE_WHEELS

examples/cirrus-ci-minimal.yml

@@ -0,0 +1,60 @@
+build_and_store_wheels: &BUILD_AND_STORE_WHEELS
+  install_cibuildwheel_script:
+    - python -m pip install cibuildwheel==2.9.0
+  run_cibuildwheel_script:
+    - cibuildwheel
+  wheels_artifacts:
+    path: "wheelhouse/*"
+
+
+linux_x86_task:
+  name: Build Linux x86 wheels.
+  compute_engine_instance:
+    image_project: cirrus-images
+    image: family/docker-builder
+    platform: linux
+    cpu: 4
+    memory: 4G
+
+  install_pre_requirements_script:
+    - apt install -y python3-venv python-is-python3
+  <<: *BUILD_AND_STORE_WHEELS
+
+linux_aarch64_task:
+  name: Build Linux aarch64 wheels.
+  compute_engine_instance:
+    image_project: cirrus-images
+    image: family/docker-builder-arm64
+    architecture: arm64
+    platform: linux
+    cpu: 4
+    memory: 4G
+
+  install_pre_requirements_script:
+    - apt install -y python3-venv python-is-python3
+  <<: *BUILD_AND_STORE_WHEELS
+
+windows_x86_task:
+  name: Build Windows x86 wheels.
+  windows_container:
+    image: cirrusci/windowsservercore:visualstudio2022
+    cpu: 4
+    memory: 4G
+
+  install_pre_requirements_script:
+    - choco install -y --no-progress python3 --version 3.10.6
+    - refreshenv
+    - echo PATH=%PATH% >> "%CIRRUS_ENV%"
+  <<: *BUILD_AND_STORE_WHEELS
+
+macos_arm64_task:
+  name: Build macOS arm64 wheels.
+  macos_instance:
+    image: ghcr.io/cirruslabs/macos-monterey-xcode
+
+  env:
+    PATH: /opt/homebrew/opt/python@3.10/bin:$PATH
+  install_pre_requirements_script:
+    - brew install python@3.10
+    - ln -s python3 /opt/homebrew/opt/python@3.10/bin/python
+  <<: *BUILD_AND_STORE_WHEELS

test/test_dependency_versions.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import platform
 import re
 import textwrap
 
@@ -60,6 +61,8 @@ def test_pinned_versions(tmp_path, python_version, build_frontend_env):
     build_environment = {}
 
     if python_version == "3.6":
+        if utils.platform == "macos" and platform.machine() == "arm64":
+            pytest.skip("macOS arm64 does not support Python 3.6")
         constraint_filename = "constraints-python36.txt"
         build_pattern = "[cp]p36-*"
     elif python_version == "3.7":
@@ -120,7 +123,7 @@ def test_dependency_constraints_file(tmp_path, build_frontend_env):
     tool_versions = {
         "pip": "20.0.2",
         "setuptools": "53.0.0",
-        "wheel": "0.34.2",
+        "wheel": "0.36.2",
         "virtualenv": "20.11.2",
     }
 

test/test_subdir_package.py

@@ -51,13 +51,13 @@ def test(capfd, tmp_path):
         add_env={
             "CIBW_BEFORE_BUILD": "python {project}/bin/before_build.py",
             "CIBW_TEST_COMMAND": "python {package}/test/run_tests.py",
-            # this shouldn't depend on the version of python, so build only CPython 3.6
+            # this shouldn't depend on the version of python, so build only CPython 3.10
-            "CIBW_BUILD": "cp36-*",
+            "CIBW_BUILD": "cp310-*",
         },
     )
 
     # check that the expected wheels are produced
-    expected_wheels = [w for w in utils.expected_wheels("spam", "0.1.0") if "cp36" in w]
+    expected_wheels = [w for w in utils.expected_wheels("spam", "0.1.0") if "cp310" in w]
     assert set(actual_wheels) == set(expected_wheels)
 
     captured = capfd.readouterr()

test/test_testing.py

@@ -142,6 +142,11 @@ def test_failing_test(tmp_path):
                 # problems with this, so let's check that.
                 "CIBW_MANYLINUX_I686_IMAGE": "manylinux1",
                 "CIBW_MANYLINUX_X86_64_IMAGE": "manylinux1",
+                # CPython 3.8 when running on macOS arm64 is unusual. The build
+                # always runs in x86_64, so the arm64 tests are not run. See
+                # #1169 for reasons why. That means the build succeeds, which
+                # we don't want. So we skip that build.
+                "CIBW_SKIP": "cp38-macosx_arm64",
             },
         )
 

unit_test/main_tests/main_platform_test.py

@@ -19,6 +19,7 @@ def test_unknown_platform_non_ci(monkeypatch, capsys):
     monkeypatch.delenv("GITHUB_ACTIONS", raising=False)
     monkeypatch.delenv("GITLAB_CI", raising=False)
     monkeypatch.delenv("CIRCLECI", raising=False)
+    monkeypatch.delenv("CIRRUS_CI", raising=False)
     monkeypatch.delenv("CIBW_PLATFORM", raising=False)
 
     with pytest.raises(SystemExit) as exit: