Apple runs Muse Spark Open at 36 tok/s on four Mac Studios with MLX: the numbers, and how the RDMA cluster is set up

Apple’s Machine Learning Research team published benchmarks for distributed LLM inference on Mac Studio on October 15, 2026, alongside a new MLX release. The headline: Meta’s Muse Spark Open (816B total, 42B active) runs at over 36 tokens per second on a cluster of four Mac Studios with M5 Ultra.

Our Muse Spark Open hardware guide covered single-Mac 2–3-bit quants at about 18–21 tok/s. Apple’s post answers the next question: what do you gain by clustering Macs, and how do you wire it up?

Why one Mac isn’t enough

Apple says Muse Spark Open would need more than 1.6 TB of RAM in BF16 for the weights alone. In NVFP4 the footprint drops to about 493 GB. Add the memory needed during inference and it no longer fits on a single 512 GB Mac Studio. So Apple splits it across machines.

The measured numbers (8,192-token context)

All figures were produced with mlx_lm.benchmark and 128 generated tokens.

Model	M3 Ultra	M5 Ultra	2× M5 pipeline	2× M5 tensor	4× M5 tensor
Muse Spark Open (NVFP4), tok/s	—	—	15.74	24.99	36.73
Mistral Medium 3.5 (FP8), tok/s	4.95	7.70	7.70	13.69	23.37
Qwen 3.8 27B (BF16), tok/s	12.46	19.69	19.44	33.27	50.89

Time to first token for Muse Spark Open was 8.4 s (two Macs, pipeline), 8.2 s (two, tensor) and 4.8 s (four, tensor). On a single machine, Apple measured M5 Ultra up to 4.6× faster than M3 Ultra on time to first token and 1.6× faster on decode. It credits the GPU Neural Accelerators and 1.2 TB/s of memory bandwidth versus 819 GB/s. Peak power was about 300 W per M5 Ultra Mac Studio.

Two findings matter when you plan a cluster:

• Pipeline parallelism (splitting layers by depth) barely changes decode speed, but it can speed up long-context prefill by up to about 1.8×.
• Tensor parallelism (splitting each layer) is what raises decode speed, and it depends on very low-latency links. At a 65,536-token context on four M5 Ultras, using two Thunderbolt cables per link instead of one cut Muse Spark Open’s time to first token from about 50.5 s to 40.6 s.

How the cluster is set up

Per Apple and the MLX distributed docs:

• macOS 26.2 or later provides RDMA over Thunderbolt 5. MLX v0.30.1 and later uses it through the JACCL backend, which the docs say cuts latency by an order of magnitude compared with the TCP ring backend.
• RDMA has to be enabled in macOS Recovery: open Terminal, run rdma_ctl enable, and reboot. You can’t do it remotely. ibv_devices confirms it worked.
• JACCL supports only fully connected meshes, so every pair of Macs needs a direct Thunderbolt cable. You describe the mesh in a JSON hostfile with SSH hostnames, an IP for rank 0, and the RDMA device for each peer.

Apple’s launch command for the four-Mac tensor-parallel benchmark:

mlx.launch --hostfile hostfile.json --backend jaccl --env MLX_METAL_FAST_SYNCH=1 -- \
  mlx_lm.benchmark --model meta-models/Muse-Spark-Open-816B-A42B-NVFP4 -p 8192 -g 128 -b 1

To serve it to an agent, swap in mlx_lm.server --model meta-models/Muse-Spark-Open-816B-A42B-NVFP4 --port 8080. Apple then points OpenCode at http://127.0.0.1:8080/v1 through an OpenAI-compatible provider. Apple calls MLX_METAL_FAST_SYNCH=1 experimental.

Bottom line

A single 512 GB Mac Studio comfortably runs Qwen 3.8 27B and Mistral Medium 3.5 128B (Apple measured up to 68 GB and 158 GB at a 65,536-token context). Muse Spark Open needs at least two in NVFP4, and four in a full Thunderbolt mesh to pass 36 tok/s. If that’s out of budget, Muse Glimmer 30B runs on a single Mac, and other MLX models like MiniMax H3 are worth a look.

Sources: Apple Machine Learning Research · MLX distributed docs · MLX LM on GitHub