feat: add Pyodide support (#1456)

* feat: add Pyodide support Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com> Co-authored-by: Matthieu Darbois <mayeut@users.noreply.github.com> tests: fix two merge issues Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> fix: include schema Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * Try to fix xbuildenv path [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Try to install pyodide-build from main branch Try again Try again Update constraints file Try again Remove unused variable Drop constraints Remove --download option Fix xbuildenv install Try again Try again [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: remove pinning on pyodide Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * Update for Pyodide 0.26.0a5 * Install pyodide-build from pypi * Update docs/options.md Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com> * Unxfail things that look like they were just a version mismatch * refactor: add constraints for pyodide Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * chore: minor cleanup Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * Apply suggestions from code review * Apply suggestion from code review * refactor: minor touchup Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * ci: xfail the pyodide test Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * review: use a pinned version of node * fix tests * review: error out on Windows * test: check node & test on macos arm64 * chore: minor cleanup * Add reference to emscripten libc issue * Apply suggestion from code review * review: use a pinned pip in test virtual environment * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * chore: rework test virtual environment seed packages * chore: workaround direct invocation of pytest This allows to still test direct invocation of `pytest` on most platforms (including pyodide on Linux) but falls back to `python -m pytest` when running pyodide on macOS. * Use release version of pyodide * fix: tests for 0.26.0 & parallel initialization of xbuildenv * Debug CI * fix: test/test_build_frontend_args.py * Revert "Debug CI" This reverts commit 917646cffc96dbfc619c47d1c03a828f447bcdb7. --------- Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com> Co-authored-by: Matthieu Darbois <mayeut@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-05-28 08:31:36 -04:00
parent 687406f664
commit 9f2a3cb67c
32 changed files with 1154 additions and 87 deletions
@@ -15,6 +15,7 @@ PRETTY_NAMES: Final[dict[PlatformName, str]] = {
    "linux": "Linux",
    "macos": "macOS",
    "windows": "Windows",
+    "pyodide": "Pyodide",
 }

 ARCH_SYNONYMS: Final[list[dict[PlatformName, str | None]]] = [
@@ -46,6 +47,9 @@ class Architecture(Enum):
    AMD64 = "AMD64"
    ARM64 = "ARM64"

+    # WebAssembly
+    wasm32 = "wasm32"
+
    # Allow this to be sorted
    def __lt__(self, other: Architecture) -> bool:
        return self.value < other.value
@@ -72,8 +76,9 @@ class Architecture(Enum):
        return result

    @staticmethod
-    def auto_archs(platform: PlatformName) -> set[Architecture]:
-        native_machine = platform_module.machine()
+    def native_arch(platform: PlatformName) -> Architecture | None:
+        if platform == "pyodide":
+            return Architecture.wasm32

        # Cross-platform support. Used for --print-build-identifiers or docker builds.
        host_platform: PlatformName = (
@@ -82,6 +87,7 @@ class Architecture(Enum):
            else ("macos" if sys.platform.startswith("darwin") else "linux")
        )

+        native_machine = platform_module.machine()
        native_architecture = Architecture(native_machine)

        # we might need to rename the native arch to the machine we're running
@@ -93,11 +99,18 @@ class Architecture(Enum):

                    if synonym is None:
                        # can't build anything on this platform
-                        return set()
+                        return None

                    native_architecture = Architecture(synonym)

-        result = {native_architecture}
+        return native_architecture
+
+    @staticmethod
+    def auto_archs(platform: PlatformName) -> set[Architecture]:
+        native_arch = Architecture.native_arch(platform)
+        if native_arch is None:
+            return set()  # can't build anything on this platform
+        result = {native_arch}

        if platform == "linux" and Architecture.x86_64 in result:
            # x86_64 machines can run i686 containers
@@ -120,6 +133,7 @@ class Architecture(Enum):
            },
            "macos": {Architecture.x86_64, Architecture.arm64, Architecture.universal2},
            "windows": {Architecture.x86, Architecture.AMD64, Architecture.ARM64},
+            "pyodide": {Architecture.wasm32},
        }
        return all_archs_map[platform]