ci: rework tests to break out ios/android tests (#2519)

* ci: rework tests to break out ios/android tests

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestions from code review

* ci: fix check for wheels

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Update .github/workflows/test.yml

* tests: pass via flag instead of envvar

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* tests: apply to serial/parallel

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* tests: join together marks

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* tests: join together marks

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* ci: break up azure job

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* ci: use matrix for azure pipelines

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Try installing Java on macos as well

* Revert "Try installing Java on macos as well"

This reverts commit aedf0f5106d2467938705b85180a0b249d49b99d.

* Try to get the most basic android test running first

* Rename platform to test_select, move headers in run_tests

Maybe it's a personal thing, but the header location was really making
me have to think more than I needed to look at that file!

* Only run docker unit tests on machines that are testing linux

* Update the flag name in azure-pipelines too

* Update azure-pipelines.yml

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Matthieu Darbois <mayeut@users.noreply.github.com>
Co-authored-by: Joe Rickerby <joerick@mac.com>

test/test_android.py

@@ -90,8 +90,17 @@ def test_android_home(tmp_path, capfd):
     assert "ANDROID_HOME environment variable is not set" in capfd.readouterr().err
 
 
-# Can fail to setup
+# the first build can fail to setup - mark as flaky, and serial to make sure it runs first
+@pytest.mark.serial
 @pytest.mark.flaky(reruns=2)
+def test_expected_wheels(tmp_path):
+    new_c_project().generate(tmp_path)
+    wheels = cibuildwheel_run(tmp_path, add_env={"CIBW_PLATFORM": "android"})
+    assert wheels == expected_wheels(
+        "spam", "0.1.0", platform="android", machine_arch=native_arch.android_abi
+    )
+
+
 def test_frontend_good(tmp_path):
     new_c_project().generate(tmp_path)
     wheels = cibuildwheel_run(
@@ -112,14 +121,6 @@ def test_frontend_bad(frontend, tmp_path, capfd):
     assert "Android requires the build frontend to be 'build'" in capfd.readouterr().err
 
 
-def test_expected_wheels(tmp_path):
-    new_c_project().generate(tmp_path)
-    wheels = cibuildwheel_run(tmp_path, add_env={"CIBW_PLATFORM": "android"})
-    assert wheels == expected_wheels(
-        "spam", "0.1.0", platform="android", machine_arch=native_arch.android_abi
-    )
-
-
 @needs_emulator
 def test_archs(tmp_path, capfd):
     new_c_project().generate(tmp_path)