feat: Android improvements needed for building NumPy and related packages (#2695)

* Use auditwheel on Android

* Add auditwheel command to defaults

* pkgconfig fixes

* Pre-import ctypes before monkey-patching in _cross_venv

* Set PKG_CONFIG and PKG_CONFIG_RELOCATE_PATHS variables

* Initial attempt at using mzakharo/android-gfortran

* Switch to using termux/ndk-toolchain-clang-with-flang

* Move PKG_CONFIG variables from build_env to android_env

* Simplify flang installation

* Add cross build files for NumPy

* Add ldpaths entry for libomp

* Add `--rm` to docker command line

* Make Rust and Fortran shims consistent

* Update documentation

* Default to API level 24 on all Python versions

* Clarify comments

* Cleanups

* Fix tests:
* Set up Android env after installing pkgconf
* Add tests for successfully using an older API level
* Previous commit's auditwheel failure is fixed in the auditwheel PR

* Update how-it-works diagram

* Add tests for repair errors

* Add more repair tests

* Add test for Meson and Fortran

* Add test for cross build files

* Improve test_api_level error message

* Add xbuild-files option

* use pypa/auditwheel@main

* Remove dependencies which are no longer needed

* Fix README

* Update to auditwheel 6.7.0

* Fix compatibility with pkgconf 2.5.1.post2

* Documentation clarifications

Co-authored-by: Joe Rickerby <joerick@mac.com>

* Fortran shim improvements

* Add Jinja variables to new_meson_project

* Update run_example_ci_configs for changed new_meson_project signature

* Add missing dependency to run_example_ci_configs

---------

Co-authored-by: mayeut <mayeut@users.noreply.github.com>
Co-authored-by: Joe Rickerby <joerick@mac.com>

docs/configuration.md

@@ -58,8 +58,8 @@ cibuildwheel to run tests, add the following YAML to your CI config file:
 You can configure cibuildwheel with a config file, such as `pyproject.toml`.
 Options have the same names as the environment variable overrides, but are
 placed in `[tool.cibuildwheel]` and are lower case, with dashes, following
-common [TOML](https://toml.io) practice. Anything placed in subsections `linux`, `windows`,
-`macos`, or `pyodide` will only affect those platforms. Lists can be used
+common [TOML](https://toml.io) practice. Anything placed in subsections
+named after a platform will only affect those platforms. Lists can be used
 instead of strings for items that are naturally a list. Multiline strings also
 work just like in the environment variables. Environment variables will take
 precedence if defined.

docs/diagram.html

@@ -179,14 +179,14 @@
           },
         },
         {
-          label: 'Repair wheel',
+          label: 'Repair using auditwheel',
           href: 'options/#repair-wheel-command',
           platforms: ['android'],
           style: 'block',
           tooltip: {
             title: 'CIBW_REPAIR_WHEEL_COMMAND',
             tag: 'Customisable step',
-            description: 'By default, bundles libc++ into the wheel if any shared library links against it, using patchelf. Can be overridden with a custom repair command.'
+            description: 'Bundle shared libraries by running auditwheel on each built wheel.'
           },
         },
         {

docs/options.md

@@ -916,6 +916,35 @@ Platform-specific environment variables are also available on platforms that use
     ```
 
 
+### `xbuild-files` {: #xbuild-files env-var toml}
+> Platform-specific files in the build environment
+
+When cross-compiling a package for Android, any dependencies in its [`build-system.requires`](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/) are installed for the build platform. However, some dependencies contain platform-specific files such as headers and static libraries, which must correspond to the target platform.
+
+This option maps a [normalized](https://packaging.python.org/en/latest/specifications/name-normalization/#name-normalization) package name to a list of paths within that package. If the package is present in the build environment, then a matching version will be downloaded for the target platform, and used to overwrite the given paths within the build environment.
+
+The default value of this option includes [paths from popular packages](configuration.md#configuration-file).
+
+Platform-specific environment variables are also available:<br/>
+ `CIBW_XBUILD_FILES_ANDROID`
+
+#### Examples
+
+!!! tab examples "pyproject.toml"
+
+    ```toml
+    [tool.cibuildwheel.xbuild-files]
+    package1 = ["some/header.h", "some/library.a"]
+    package2 = ["other/header.h"]
+    ```
+
+!!! tab examples "Environment variables"
+
+    ```yaml
+    CIBW_XBUILD_FILES: "package1: some/header.h some/library.a; package2: other/header.h"
+    ```
+
+
 ### `repair-wheel-command` {: #repair-wheel-command env-var toml}
 > Execute a shell command to repair each built wheel
 
@@ -923,8 +952,7 @@ Default:
 
 - on Linux: `'auditwheel repair -w {dest_dir} {wheel}'`
 - on macOS: `'delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}'`
-- on Android: There is no default command, but cibuildwheel will add `libc++` to the
-  wheel if anything links against it. Setting a command will replace this behavior.
+- on Android: `'auditwheel repair --ldpaths {ldpaths} -w {dest_dir} {wheel}'`
 - on Pyodide: You can use `pyodide auditwheel repair --libdir /path/to/libraries --output-dir {dest_dir} {wheel}` command to repair the wheel.
   Unlike other platforms, this command is not set by default as you need to explicitly
   specify the library directory. You might not want to use the libraries in the system
@@ -939,6 +967,7 @@ The following placeholders must be used inside the command and will be replaced
 - `{wheel}` for the absolute path to the built wheel
 - `{dest_dir}` for the absolute path of the directory where to create the repaired wheel
 - `{delocate_archs}` (macOS only) comma-separated list of architectures in the wheel.
+- `{ldpaths}` (Android only) colon-separated list of directories to search for external libraries, set by cibuildwheel to include any necessary locations in the NDK. You can add more directories by appending them with a colon separator after the placeholder, or by setting the `AUDITWHEEL_LD_LIBRARY_PATH` environment variable.
 
 You can use the `{package}` or `{project}` placeholders in your `repair-wheel-command` to refer to the package being built or the project root, respectively.
 

docs/platforms.md

@@ -219,13 +219,9 @@ It also requires the following commands to be on the `PATH`:
 
 Android builds will honor the `ANDROID_API_LEVEL` environment variable to set the
 minimum supported [API level](https://developer.android.com/tools/releases/platforms)
-for generated wheels. This will default to the minimum API level of the selected Python
-version.
-
-If the [`repair-wheel-command`](options.md#repair-wheel-command) adds any libraries to
-the wheel, then `ANDROID_API_LEVEL` must be at least 24. This is already the default
-when building for Python 3.14 and later, but you may need to set it when building for
-Python 3.13.
+for generated wheels. This defaults to 24, which is supported by [99% of active
+devices](https://dl.google.com/android/studio/metadata/distributions.json), and is the first
+version to support RUNPATH, which auditwheel needs in order to graft external libraries.
 
 ### Build frontend support