From 83c5f2a58a81f5bf7dd742a37c415cfb20546d68 Mon Sep 17 00:00:00 2001 From: Anthony Shoumikhin Date: Sun, 30 Aug 2026 14:22:34 -0700 Subject: [PATCH 1/3] Report the MLX backend as unavailable on the iOS simulator Loading a model that delegates to MLX crashes the process on an iOS simulator instead of failing cleanly. The demo app in the examples repository crashes on the MLX menu entry, and the llama and image classifier tests crash with it. The simulator's Metal device is real enough to build and dispatch a compute pipeline, but two of the things MLX needs unconditionally are missing from it: `architecture` reports nothing, and a shared storage heap is refused by an assertion inside Metal. MLX reads the first without a null check, so it faults during device construction, and if that is bypassed it aborts on the second. Neither is reachable as an error, so nothing downstream can report it. `is_available()` is the interface's own answer to whether a backend can run here, and the runtime already asks before loading a delegate, so returning false is enough to turn the crash into `Error::NotFound`. MLX's own `is_available()` returns a constant and checks nothing. Test plan: The macro decides everything here, so it is checked on all three targets it affects: simulator TARGET_OS_SIMULATOR 1, is_available() false device TARGET_OS_SIMULATOR 0, is_available() defers to MLX macOS TARGET_OS_SIMULATOR 0, is_available() defers to MLX so a real GPU is unaffected. Compiling the same function with the guard disabled returns true on the simulator again, which is the crashing path, so the guard is doing the work. Confirmed separately that a delegate whose backend reports unavailable makes `load_method` return `Error::NotFound` rather than crash. The include is explicit because an undefined macro is zero in `#if`, which would make the guard silently do nothing. This does not make MLX work on the simulator. It makes the answer honest, so a caller gets an error it can handle. --- backends/mlx/runtime/MLXBackend.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/backends/mlx/runtime/MLXBackend.cpp b/backends/mlx/runtime/MLXBackend.cpp index d6dfc463b76..76727ca1e47 100644 --- a/backends/mlx/runtime/MLXBackend.cpp +++ b/backends/mlx/runtime/MLXBackend.cpp @@ -31,6 +31,8 @@ #include #include +#include + namespace executorch { namespace backends { namespace mlx { @@ -217,7 +219,14 @@ class MLXBackend final : public ::executorch::runtime::BackendInterface { ~MLXBackend() override = default; bool is_available() const override { +#if TARGET_OS_SIMULATOR + // The simulator's Metal device reports no architecture and refuses a + // shared storage heap, and MLX requires both without checking, so it + // faults on its first device access rather than returning an error. + return false; +#else return ::mlx::core::metal::is_available(); +#endif } Result init( From 5a982f40894bc7db54fb0101cc9daad41bd28ad6 Mon Sep 17 00:00:00 2001 From: Anthony Shoumikhin Date: Mon, 31 Aug 2026 10:17:49 -0700 Subject: [PATCH 2/3] Correct the reason in the comment, and say so in the docs Review found one clause of the new comment is wrong about MLX. MLX does not require the shared storage heap: its allocator skips it on a paravirtual device and every use tests for it first, so a heap it never got would not sink it. The real sequence is that Metal traps inside the heap request, so MLX never receives a value it could fall back from. Only the architecture read genuinely has no null check. Left as it was, the next reader goes looking for a missing check in MLX and finds one that is already there. The comment also now says this is a build switch rather than a probe, and what would let it go. Two documentation pages said the opposite of the code. The iOS page lists the prebuilt frameworks as working on devices and simulators and listed the MLX backend with no exception, and the MLX overview gave only Mac under target requirements without mentioning iOS at all. Both now say MLX runs on real devices and Mac but not the simulator, so the first sign of this is not a load error. Test plan: Unchanged from the previous revision, since no behaviour changed here. The guard still compiles clean with -Wall -Wextra for macOS, the iOS simulator and iOS device, and the macro still resolves to 1 only for the simulator. --- backends/mlx/runtime/MLXBackend.cpp | 8 +++++--- docs/source/backends/mlx/mlx-overview.md | 2 ++ docs/source/using-executorch-ios.md | 2 +- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/backends/mlx/runtime/MLXBackend.cpp b/backends/mlx/runtime/MLXBackend.cpp index 76727ca1e47..615c4f1c6ce 100644 --- a/backends/mlx/runtime/MLXBackend.cpp +++ b/backends/mlx/runtime/MLXBackend.cpp @@ -220,9 +220,11 @@ class MLXBackend final : public ::executorch::runtime::BackendInterface { bool is_available() const override { #if TARGET_OS_SIMULATOR - // The simulator's Metal device reports no architecture and refuses a - // shared storage heap, and MLX requires both without checking, so it - // faults on its first device access rather than returning an error. + // The simulator's Metal device reports no architecture, which MLX reads + // without a null check while constructing its device. Past that, requesting + // a shared storage heap traps inside Metal itself, so MLX never gets a + // value it could fall back from. This is a build switch rather than a + // probe: it can go once the simulator has a usable Metal device. return false; #else return ::mlx::core::metal::is_available(); diff --git a/docs/source/backends/mlx/mlx-overview.md b/docs/source/backends/mlx/mlx-overview.md index c22e4d7b67d..930d7d5637f 100644 --- a/docs/source/backends/mlx/mlx-overview.md +++ b/docs/source/backends/mlx/mlx-overview.md @@ -18,6 +18,8 @@ The MLX delegate is experimental and under active development. - Apple Silicon Mac (M1 or later) - [macOS](https://developer.apple.com/macos) >= 14.0 +- iOS and iPadOS on a real device. The iOS simulator has no Metal device MLX can use, so the + backend reports itself unavailable there and a model delegated to it will not load. ## Development Requirements diff --git a/docs/source/using-executorch-ios.md b/docs/source/using-executorch-ios.md index e319ff0a60c..03dffe4019c 100644 --- a/docs/source/using-executorch-ios.md +++ b/docs/source/using-executorch-ios.md @@ -10,7 +10,7 @@ The ExecuTorch Runtime for iOS and macOS (ARM64) is distributed as a collection * `executorch_dump` - ETDump profiling * `executorch_llm` - LLM-specific runtime components * `backend_coreml` - Core ML backend -* `backend_mlx` - MLX backend +* `backend_mlx` - MLX backend, on real devices and Mac only, not the iOS simulator * `backend_xnnpack` - XNNPACK backend * `kernels_llm` - Custom kernels for LLMs * `kernels_optimized` - Accelerated generic CPU kernels From 9ce78e1d81a08a7b58877461929c2d9fbd2a5b6f Mon Sep 17 00:00:00 2001 From: Anthony Shoumikhin Date: Mon, 31 Aug 2026 14:54:00 -0700 Subject: [PATCH 3/3] Pin the unavailable path with a test, and make the four pages agree Nothing would have noticed if this guard were deleted. The only job that compiles this file for a simulator compiles it with tests off, and it passes on the base commit too, so the guard could go and every check would stay green. The executor's stub backend already has a hook to install a custom availability answer, unused until now. Installing false and loading a delegated method asserts `Error::NotFound` rather than the backend being initialized anyway. That runs on every pull request, on Linux, with no Apple hardware, and it fails if the check in `Method::load` goes. On the documentation, my last change left the iOS page contradicting itself. It said the MLX framework is for real devices and Mac only, while the same page maps a simulator platform name, force loads the MLX library for that slice, and asks the reader to ship the simulator kernel file. The framework does still build, ship and link there; only the runtime answer changed. So the list entry goes back to plain, and the behaviour is a note beside the existing ones, or a reader starts pulling the simulator slice out of their build. Two more pages said macOS only, the backend table and the C++ component table, which are where most people look first. Both now read iOS and macOS, matching the Core ML row above. The iOS requirement also carries its version now. The build stops with a fatal error below iOS 17.0 and the Swift package declares iOS 17, so the number belongs on the line. The requirements list reads as alternatives rather than a set that must all hold. Test plan: unavailable backend load_method returns Error::NotFound check in Method::load removed the test fails Guard behaviour unchanged from the previous revision: the macro is 1 only for the simulator, and the same function with the guard disabled returns true there, which is the crashing path. --- docs/source/backends-overview.md | 2 +- docs/source/backends/mlx/mlx-overview.md | 12 ++++++---- docs/source/using-executorch-ios.md | 4 +++- .../test/backend_integration_test.cpp | 22 +++++++++++++++++++ 4 files changed, 34 insertions(+), 6 deletions(-) diff --git a/docs/source/backends-overview.md b/docs/source/backends-overview.md index 049b7d443dc..ab74fe8edd8 100644 --- a/docs/source/backends-overview.md +++ b/docs/source/backends-overview.md @@ -23,7 +23,7 @@ Backends are the bridge between your exported model and the hardware it runs on. | [XNNPACK](backends/xnnpack/xnnpack-overview.md) | All | CPU | General-purpose, fallback | | [CUDA](backends/cuda/cuda-overview.md) | Linux/Windows | GPU | NVIDIA GPU acceleration | | [Core ML](backends/coreml/coreml-overview.md) | iOS, macOS | NPU/GPU/CPU | Apple devices, high performance | -| [MLX](/backends/mlx/mlx-overview.md) | macOS | GPU | Apple Silicon GPU (MLX) | +| [MLX](/backends/mlx/mlx-overview.md) | iOS (experimental), macOS | GPU | Apple Silicon GPU (MLX) | | [Vulkan](backends/vulkan/vulkan-overview.md) | Android, Linux, Windows | GPU | Android devices (mature); Desktops (experimental) | | [WebGPU](backends/webgpu/webgpu-overview.md) | Browser, Linux, macOS | GPU | Cross-platform and browser GPU execution (experimental) | | [Qualcomm](backends-qualcomm) | Android | NPU | Qualcomm SoCs | diff --git a/docs/source/backends/mlx/mlx-overview.md b/docs/source/backends/mlx/mlx-overview.md index 930d7d5637f..ccd73c42b03 100644 --- a/docs/source/backends/mlx/mlx-overview.md +++ b/docs/source/backends/mlx/mlx-overview.md @@ -16,10 +16,14 @@ The MLX delegate is experimental and under active development. ## Target Requirements -- Apple Silicon Mac (M1 or later) -- [macOS](https://developer.apple.com/macos) >= 14.0 -- iOS and iPadOS on a real device. The iOS simulator has no Metal device MLX can use, so the - backend reports itself unavailable there and a model delegated to it will not load. +One of: + +- [macOS](https://developer.apple.com/macos) >= 14.0 on an Apple Silicon Mac (M1 or later) +- [iOS](https://developer.apple.com/ios) or [iPadOS](https://developer.apple.com/ipados) >= 17.0 + on a real device (experimental). The backend is built and shipped for iOS, but + running a model on a physical device is not yet covered by CI. The iOS simulator + has no Metal device MLX can use, so the backend reports itself unavailable there + and a model delegated to it will not load. ## Development Requirements diff --git a/docs/source/using-executorch-ios.md b/docs/source/using-executorch-ios.md index 03dffe4019c..7053c28fd76 100644 --- a/docs/source/using-executorch-ios.md +++ b/docs/source/using-executorch-ios.md @@ -10,7 +10,7 @@ The ExecuTorch Runtime for iOS and macOS (ARM64) is distributed as a collection * `executorch_dump` - ETDump profiling * `executorch_llm` - LLM-specific runtime components * `backend_coreml` - Core ML backend -* `backend_mlx` - MLX backend, on real devices and Mac only, not the iOS simulator +* `backend_mlx` - MLX backend * `backend_xnnpack` - XNNPACK backend * `kernels_llm` - Custom kernels for LLMs * `kernels_optimized` - Accelerated generic CPU kernels @@ -23,6 +23,8 @@ Link your binary with the ExecuTorch runtime and any backends or kernels used by **Note:** To access logs, link against the Debug build of the ExecuTorch runtime, i.e., the `executorch_debug` framework. For optimal performance, always link against the Release version of the deliverables (those without the `_debug` suffix), which have all logging overhead removed. See the [Logging](#Logging) section for more details. +**Note:** The MLX backend links and registers on the iOS simulator, so an app builds for both destinations, but it reports itself unavailable there because the simulator has no Metal device it can use. A model delegated to MLX will not load on the simulator. Use a real device or a Mac to run one. + ### Swift Package Manager The prebuilt ExecuTorch runtime, backend, and kernels are available as a [Swift PM](https://www.swift.org/documentation/package-manager/) package. diff --git a/runtime/executor/test/backend_integration_test.cpp b/runtime/executor/test/backend_integration_test.cpp index 90b1b80c768..89f3b530416 100644 --- a/runtime/executor/test/backend_integration_test.cpp +++ b/runtime/executor/test/backend_integration_test.cpp @@ -347,6 +347,28 @@ TEST_P(BackendIntegrationTest, BasicInitSucceeds) { EXPECT_EQ(method_res.error(), Error::Ok); } +TEST_P(BackendIntegrationTest, UnavailableBackendFailsToLoad) { + // A backend that reports itself unavailable must make load_method return + // NotFound rather than being initialized anyway. Backends that cannot run on + // the current platform rely on this to fail instead of faulting. + StubBackend::singleton().install_is_available([]() { return false; }); + + Result loader = FileDataLoader::from(program_path()); + ASSERT_EQ(loader.error(), Error::Ok); + Result program = Program::load(&loader.get()); + ASSERT_EQ(program.error(), Error::Ok); + + // The gate only means something if this method actually delegates to the + // stub, so confirm that before asserting the load fails on availability. + EXPECT_TRUE( + program->method_meta("forward")->uses_backend(StubBackend::kName)); + + ManagedMemoryManager mmm(kDefaultNonConstMemBytes, kDefaultRuntimeMemBytes); + + Result method_res = program->load_method("forward", &mmm.get()); + EXPECT_EQ(method_res.error(), Error::NotFound); +} + TEST_P(BackendIntegrationTest, GetBackendNamesSuccess) { // Load the program from file. Result loader = FileDataLoader::from(program_path());