TL;DR
PyTorch 2.15 adds FP8 tensors and new Metal-based optimizations to its MPS backend for Apple silicon Macs. Hugging Face reports much faster decoding for tested mixture-of-experts models and a large reduction in peak memory for Wan2.2 video VAE decoding; results depend on the model and hardware.
PyTorch 2.15 adds FP8 tensor support and performance changes to MPS, the framework’s backend for using Apple silicon GPUs. Hugging Face’s PyTorch team reports large decoding gains for several tested language models and sharply lower peak memory in a Wan2.2 video-decoding workload, changes that could make more local AI workloads practical on Macs.
The update introduces torch.float8_e4m3fn tensors on MPS and support for scaled matrix multiplication through F.scaled_mm, including tensorwise and rowwise scaling. Hugging Face says FP8 values use one byte each, half the storage of BF16 values. That is a storage comparison, not a guarantee that every model will use half as much total memory: runtime memory also depends on activations, intermediate data and other model components.
For language-model decoding, PyTorch 2.15 routes eligible single-row F.linear operations through specialized matrix-vector kernels, with some bias additions fused. Hugging Face reports a 1.23× geometric-mean speedup across 96 BF16 layer shapes drawn from real models. The team also reports that vocabulary softmax is about eight times faster in its tests. These figures describe measured workloads and should not be read as a universal speedup for all models or Macs.
The largest reported gains involve mixture-of-experts (MoE) models, which route tokens to different expert networks. On an M3 Ultra with torch.compile, Hugging Face measured Qwen3.6-35B-A3B increasing from 9.4 to 61.3 tokens per second, and LFM2.5-8B-A1B from 23.6 to 123.6 tokens per second, compared with PyTorch 2.14. The report credits a native grouped matrix multiplication path, a dedicated decoding path, fewer GPU-CPU synchronizations and faster top-k routing.
More Local Models on Apple Silicon
The changes target practical limits for people running language, image and video models locally through tools built on PyTorch, including Transformers, Diffusers and ComfyUI. Faster token generation can make interactive use more responsive, while reduced memory demand can let some workloads run on machines that previously exceeded available resources. The impact will vary: the benchmark results come from specific models and an M3 Ultra, not from a broad survey of Mac configurations.
The memory result is especially striking in the Hugging Face report: decoding Wan2.2’s tiled VAE peaked at about 8 GiB in PyTorch 2.15, compared with a reported 270 GiB previously. The team also says a padding change corrected a Qwen-Image-2.1 VAE encoder issue that had silently corrupted images on MPS. Its encode-decode test rose from 10.06 dB to 29.95 dB PSNR and matched the CPU result in that test. These are workload-specific findings, but they show that backend updates can affect output correctness as well as speed and memory use.
Apple Silicon Mac compatible AI model training
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
From MPSGraph to Metal Kernels
MPS is PyTorch’s backend for computing on Apple’s Metal GPU technology. Hugging Face says its team tests real models from the Hugging Face Hub across language, image and video tasks, then works with PyTorch maintainers and community contributors to address operations that are unsupported, slow or memory-intensive. A recurring approach in this work is replacing general MPSGraph execution with dedicated Metal kernels for targeted operations.
The 2.15 changes extend beyond language models. The report lists work on complex LU and triangular solves, small-matrix inverses and QR decomposition, as well as convolutions, padding, dropout, interpolation backward and reductions. It also adds public MPS streams and changes intended to make event timing more reliable. The release has updated platform requirements: an Apple silicon Mac with macOS 15 or later and Python 3.11 or later. Hugging Face says MPS availability checks now provide a reason when the operating system is too old or no Metal GPU is available.
“The goal is to make your Mac a first-class place to run and train PyTorch models.”
— Hugging Face PyTorch team
Results Depend on Workload
The source is a report from the team contributing to MPS, and its performance numbers are team-reported measurements. It does not provide a broad comparison across Mac models, system configurations or all supported software stacks. The stated tokens-per-second figures are tied to named models, an M3 Ultra and, for the language-model tests, torch.compile; results on other hardware or with different settings may differ.
It is also unclear from the report how widely the Wan2.2 memory reduction and Qwen-Image-2.1 correction generalize to other models or pipelines. FP8 support is specifically described for float8_e4m3fn tensors and scaled matrix multiplication, not as a blanket claim that every operation or model can run in FP8 on MPS. Users should check compatibility and validate quality and performance in their own workflows.
Testing 2.15 and Watching 2.16
Mac users can test the changes after meeting the macOS 15 and Python 3.11 minimums and installing a compatible PyTorch 2.15 build. For production workflows, the report’s results make local benchmarking important: check generation speed, peak memory and output quality on the exact model and Mac in use, especially when changing precision or using image and video pipelines.
Hugging Face’s report previews further MPS work for PyTorch 2.16, including attention running fully on Metal, additional migrations from MPSGraph to Metal kernels and faster LLM decoding. Those items are a forward-looking preview, not features confirmed as delivered in 2.15; the report does not give a release date or final performance figures for them.
Key Questions
What changed for MPS in PyTorch 2.15?
The update adds float8_e4m3fn tensor support and scaled matrix multiplication on MPS, alongside faster paths for language-model decoding, softmax and several linear algebra, audio, video and training operations.
Does PyTorch 2.15 make every Mac language model six times faster?
No. The 5–6.5× range is reported for specific MoE decoding tests compared with PyTorch 2.14. The report’s results used named models and an M3 Ultra; performance varies by model, hardware and configuration.
How much memory does FP8 save?
Hugging Face says each supported FP8 value takes half the storage of BF16. That does not mean a complete model or workload will use half the total memory.
What are the minimum requirements for MPS in PyTorch 2.15?
The report lists an Apple silicon Mac running macOS 15 or later and Python 3.11 or later. Users should check the compatibility information for the specific PyTorch build they install.
What is planned for MPS in PyTorch 2.16?
Hugging Face previews attention running fully on Metal, more MPSGraph-to-Metal kernel migrations and faster LLM decoding. These are planned or previewed directions, not confirmed PyTorch 2.15 features.
Source: rss
