Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions backends/mlx/runtime/MLXBackend.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
#include <memory>
#include <mutex>

#include <TargetConditionals.h>

namespace executorch {
namespace backends {
namespace mlx {
Expand Down Expand Up @@ -217,7 +219,16 @@ class MLXBackend final : public ::executorch::runtime::BackendInterface {
~MLXBackend() override = default;

bool is_available() const override {
#if TARGET_OS_SIMULATOR
// The simulator's Metal device reports no architecture, which MLX reads
// without a null check while constructing its device. Past that, requesting
// a shared storage heap traps inside Metal itself, so MLX never gets a
// value it could fall back from. This is a build switch rather than a
// probe: it can go once the simulator has a usable Metal device.
return false;
#else
return ::mlx::core::metal::is_available();
#endif
}

Result<DelegateHandle*> init(
Expand Down
2 changes: 1 addition & 1 deletion docs/source/backends-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Backends are the bridge between your exported model and the hardware it runs on.
| [XNNPACK](backends/xnnpack/xnnpack-overview.md) | All | CPU | General-purpose, fallback |
| [CUDA](backends/cuda/cuda-overview.md) | Linux/Windows | GPU | NVIDIA GPU acceleration |
| [Core ML](backends/coreml/coreml-overview.md) | iOS, macOS | NPU/GPU/CPU | Apple devices, high performance |
| [MLX](/backends/mlx/mlx-overview.md) | macOS | GPU | Apple Silicon GPU (MLX) |
| [MLX](/backends/mlx/mlx-overview.md) | iOS (experimental), macOS | GPU | Apple Silicon GPU (MLX) |
| [Vulkan](backends/vulkan/vulkan-overview.md) | Android, Linux, Windows | GPU | Android devices (mature); Desktops (experimental) |
| [WebGPU](backends/webgpu/webgpu-overview.md) | Browser, Linux, macOS | GPU | Cross-platform and browser GPU execution (experimental) |
| [Qualcomm](backends-qualcomm) | Android | NPU | Qualcomm SoCs |
Expand Down
10 changes: 8 additions & 2 deletions docs/source/backends/mlx/mlx-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,14 @@ The MLX delegate is experimental and under active development.

## Target Requirements

- Apple Silicon Mac (M1 or later)
- [macOS](https://developer.apple.com/macos) >= 14.0
One of:

- [macOS](https://developer.apple.com/macos) >= 14.0 on an Apple Silicon Mac (M1 or later)
- [iOS](https://developer.apple.com/ios) or [iPadOS](https://developer.apple.com/ipados) >= 17.0
on a real device (experimental). The backend is built and shipped for iOS, but
running a model on a physical device is not yet covered by CI. The iOS simulator
has no Metal device MLX can use, so the backend reports itself unavailable there
and a model delegated to it will not load.

## Development Requirements

Expand Down
2 changes: 2 additions & 0 deletions docs/source/using-executorch-ios.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ Link your binary with the ExecuTorch runtime and any backends or kernels used by

**Note:** To access logs, link against the Debug build of the ExecuTorch runtime, i.e., the `executorch_debug` framework. For optimal performance, always link against the Release version of the deliverables (those without the `_debug` suffix), which have all logging overhead removed. See the [Logging](#Logging) section for more details.

**Note:** The MLX backend links and registers on the iOS simulator, so an app builds for both destinations, but it reports itself unavailable there because the simulator has no Metal device it can use. A model delegated to MLX will not load on the simulator. Use a real device or a Mac to run one.

### Swift Package Manager

The prebuilt ExecuTorch runtime, backend, and kernels are available as a [Swift PM](https://www.swift.org/documentation/package-manager/) package.
Expand Down
22 changes: 22 additions & 0 deletions runtime/executor/test/backend_integration_test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,28 @@ TEST_P(BackendIntegrationTest, BasicInitSucceeds) {
EXPECT_EQ(method_res.error(), Error::Ok);
}

TEST_P(BackendIntegrationTest, UnavailableBackendFailsToLoad) {
// A backend that reports itself unavailable must make load_method return
// NotFound rather than being initialized anyway. Backends that cannot run on
// the current platform rely on this to fail instead of faulting.
StubBackend::singleton().install_is_available([]() { return false; });

Result<FileDataLoader> loader = FileDataLoader::from(program_path());
ASSERT_EQ(loader.error(), Error::Ok);
Result<Program> program = Program::load(&loader.get());
ASSERT_EQ(program.error(), Error::Ok);

// The gate only means something if this method actually delegates to the
// stub, so confirm that before asserting the load fails on availability.
EXPECT_TRUE(
program->method_meta("forward")->uses_backend(StubBackend::kName));

ManagedMemoryManager mmm(kDefaultNonConstMemBytes, kDefaultRuntimeMemBytes);

Result<Method> method_res = program->load_method("forward", &mmm.get());
EXPECT_EQ(method_res.error(), Error::NotFound);
}

TEST_P(BackendIntegrationTest, GetBackendNamesSuccess) {
// Load the program from file.
Result<FileDataLoader> loader = FileDataLoader::from(program_path());
Expand Down
Loading