- Experimental ExecuTorch Native format: a portable .ptn program lowered at load time by Vulkan or ggml engines
- Experimental executorch.runtime.cuda module for choosing the CUDA stream from Python
- Experimental diffusion extension for Stable Diffusion 1.5 and SDXL
- Memory-planned delegate scratch buffers and skip_h2d_for_constants
- Linkable C++ libraries, CUDA runtime, and OpenVINO delegate in the Windows wheels
- pkg-config file in the Linux and macOS wheels
- CUDA 13.4 wheels
- FP8 and BF16 support in the Arm VGF runtime
- Dynamic-range W8A8 quantization for linear layers on Arm
- Experimental delegate-owned streaming state for Ethos-U
- VGF quantization sensitivity and quality analysis tools
- SDPA decomposition and SAME-padding fusion for Cortex-M
- pow, isinf, isnan, TopK, softplus, roll, MaxPool1D, and negative padding operators on Arm
- Flexible Context Binary export for Qualcomm, producing one artifact for multiple Snapdragon SoCs
- On-device OpTrace and host-only Hextimate HTP profiling for Qualcomm
- AdamW optimizer for on-device training on Qualcomm
- Autotuned INT4/5/6/8 quantized GEMM kernels on CUDA and ROCm for decode and small-batch prefill
- Off-graph KV caches with a batching executor, prefix-cache reuse, on-device sampling, and DFlash drafting in CUDA graphs
- Batched MLX KV cache with DFlash support
- Core AI palletization
- Fused SwiGLU in the Vulkan backend
- Recipe-based lowering for the NXP Neutron backend supporting post-training quantization and QAT workflows
- Portable 3-D convolution, antialiased bicubic resize, bucketize, and _fft_r2c operators
- KleidiAI BF16 GEMM and SDPA dispatch
- HiFi5 support in the Cadence backend
- Browser execution and an experimental Playground for the WebGPU backend
- NXP Neutron backend promoted to production-ready
- Python runtime imports no longer pull in exir, Inductor, or Triton
- Default Linux thread pool limited to the CPUs the process may use
- Supported Python versions are now 3.11 through 3.15; Python 3.10 dropped
- CUDA 12.6 and 13.0 wheels dropped; Jetson Orin now requires a source build
- Test-only packages removed from the executorch dependencies
- Arm Model Converter and ML Emulation Layer updated to 0.11 and Ethos-U core driver updated to 26.08
- Dynamic allocation removed from the Ethos-U backend and Vela binary parsing hardened
- Deprecated Arm tools removed, including examples/arm/aot_arm_compiler.py
- QNN SDK updated to 2.51
- Added SC8480XP and Snapdragon 7+ Gen 3 support on Qualcomm
- 8-bit KV-cache I/O for Hugging Face LLMs and W4 embedding and LM-head quantization for LLM QAT on Qualcomm
- Operator autotuning now times candidates in CUDA graphs so CPU overhead does not affect the choice; users can set benchmarking example inputs and the tuner can choose between implementations
- Long-context attention sped up with TMA causal prefill, in-kernel causal masks, and hardware-aware split-K decode on CUDA
- Reduced MLX peak memory with dynamic GroupNorm and upsampling support
- Fully delegated dynamic transformer blocks in the Vulkan backend
- Faster softmax and reductions and kernels tuned for Mali and Adreno in Vulkan
- Updated eIQ Neutron SDK to 3.2.3
- --xnnpack-extended-ops is now on by default in LLM export
- Unsupported BF16 and high-rank ops in XNNPACK stay on the CPU instead of failing
- OpenVINO CPU inference defaults to float32; the new inference_precision compile spec changes it
- Half and BFloat16 matmul and reductions now accumulate in float
- Floating-point floor_divide now matches ATen
- Silently wrong XNNPACK outputs with shared BatchNorms and quantized ops left outside a delegate
- Wrong outputs from Triton SDPA with wide attention heads and from OpenVINO partitions that return a copy of an input
- Export of zero-stride constants and mixed-dtype cat
- Python logging deadlocks between threads
- Arm log-softmax underflow and round behavior
- QNN shared buffers on OE Linux
From ExecuTorch
Highlights
- NXP Neutron is production-ready.
- TensorRT backend in beta.
- Robotics: LeRobot policies such as SmolVLA and pi0 now run correctly on CUDA, MLX, OpenVINO, and XNNPACK.
- Torch-free Python runtime: run
.ptefiles from NumPy, buffers, or DLPack without PyTorch. - Faster GPU LLMs: batched serving, prefix caching, and DFlash speculative decoding on CUDA and MLX (fully integrated into Muse Glimmer).
- Python 3.15 and PyTorch 2.15 support.
Runtime and Core
- Added experimental ExecuTorch Native: a portable
.ptnprogram lowered at load time by Vulkan or ggml engines. - Added the experimental
executorch.runtime.cudamodule for choosing the CUDA stream from Python. - Python runtime imports no longer pull in
exir, Inductor, or Triton. - Added an experimental diffusion extension for Stable Diffusion 1.5 and SDXL.
- Added memory-planned delegate scratch buffers and
skip_h2d_for_constants. - Limited the default Linux thread pool to the CPUs the process may use.
Packaging and Build
- Dropped Python 3.10; supported versions are now 3.11 through 3.15.
- Added CUDA 13.4 wheels and dropped CUDA 12.6 and 13.0 wheels. Jetson Orin now needs a source build.
- Added linkable C++ libraries, the CUDA runtime, and the OpenVINO delegate to the Windows wheels.
- Added a
pkg-configfile to the Linux and macOS wheels. - Removed test-only packages from the
executorchdependencies.
Backends
Arm
- Added FP8 and BF16 to the VGF runtime and updated the Model Converter and ML Emulation Layer to 0.11.
- Added dynamic-range W8A8 quantization for linear layers.
- Added experimental delegate-owned streaming state for Ethos-U and updated the Ethos-U core driver to 26.08.
- Removed dynamic allocation from the Ethos-U backend and hardened Vela binary parsing.
- Added VGF quantization sensitivity and quality analysis tools.
- Added SDPA decomposition and SAME-padding fusion for Cortex-M.
- Added
pow,isinf,isnan, TopK, softplus,roll, MaxPool1D, and negative padding. - Added a person detection example.
- Removed deprecated tools, including
examples/arm/aot_arm_compiler.py.
Qualcomm
- Updated the QNN SDK to 2.51.
- Added Flexible Context Binary export: one artifact for multiple Snapdragon SoCs.
- Added on-device OpTrace and host-only Hextimate HTP profiling.
- Added an HTP graph-splitting option for faster context finalization.
- Added SC8480XP and Snapdragon 7+ Gen 3 support.
- Added 8-bit KV-cache I/O for Hugging Face LLMs and W4 embedding and LM-head quantization for LLM QAT.
- Added an example for supporting Custom Ops on LPAI backend
- Added AdamW optimizer for on-device training.
- Added STFT,
empty_strided, and per-channel-group quantization.
CUDA and ROCm
- Added autotuned INT4/5/6/8 quantized GEMM kernels on CUDA and ROCm, covering decode and small-batch prefill (M ≤ 64).
- Made operator autotuning more accurate: candidates are now timed in CUDA graphs, so CPU overhead no longer affects the choice. Users can also set the example inputs used for benchmarking, and the tuner can choose between implementations (e.g. SDPA with or without split-K).
- Added off-graph KV caches with a batching executor and prefix-cache reuse, on-device sampling, and DFlash drafting in CUDA graphs.
- Sped up long-context attention with TMA causal prefill, in-kernel causal masks, and hardware-aware split-K decode.
- Added lowering for complex-number RoPE, fused attention variants, and boolean mask indexing.
Apple (Core ML, Core AI, Metal, MLX)
- Added a batched MLX KV cache with DFlash support.
- Reduced MLX peak memory and added dynamic GroupNorm and upsampling.
- Added Core AI palletization; removed sidecar asset delivery.
Vulkan
- Fully delegated dynamic transformer blocks.
- Added fused SwiGLU and faster softmax and reductions.
- Tuned kernels for Mali and Adreno.
NXP
- Promoted the Neutron backend to production-ready.
- Updated the eIQ Neutron SDK to 3.2.3.
- Added recipe-based lowering for Neutron backend, Neutron recipe-based lowering supports both Post-training quantization and QAT workflows.
- ExecuTorch test harness model suite enabled for Neutron backend.
XNNPACK
--xnnpack-extended-opsis now on by default in LLM export.- Kept unsupported BF16 and high-rank ops on the CPU instead of failing.
Other Backends
- Cadence: added HiFi5 support.
- Samsung: added Llama 3.2 1B, Gemma 3 1B, and YOLO26.
- OpenVINO: CPU inference now defaults to float32; use the new
inference_precisioncompile spec to change it. - WebGPU: added browser execution and an experimental Playground.
- MediaTek: temporarily unsupported pending a Python 3.11+ NeuroPilot converter.
Operators and Kernels
- Added portable 3-D convolution, antialiased bicubic resize,
bucketize, and_fft_r2c. - Added KleidiAI BF16 GEMM and SDPA dispatch.
- Half and BFloat16 matmul and reductions now accumulate in float.
- Floating-point
floor_dividenow matches ATen.
Notable Fixes
- Fixed silently wrong XNNPACK outputs with shared BatchNorms and quantized ops left outside a delegate.
- Fixed wrong outputs from Triton SDPA with wide attention heads and from OpenVINO partitions that return a copy of an input.
- Fixed export of zero-stride constants and mixed-dtype
cat. - Fixed Python logging deadlocks between threads.
- Fixed Arm log-softmax underflow and
roundbehavior. - Fixed QNN shared buffers on OE Linux.
Contributors
We welcome 35 first-time contributors to ExecuTorch in this release: @shoemoney, @fatih-uzlmz, @peygar, @Air-crypto, @bharqav, @Gallinator, @sqliangx, @Ivan-Dimitrov, @milesmanxp, @Dev-next-gen, @quic-xiyushi, @David-Wu1119, @Nicoshev, @amacharla15, @estervojko, @dongjiang1989, @chinnadhurai, @elbourne12345, @pcwu0329, @u7k4rs6, @TejasweeSulekh, @paul1106, @HiroForYou, @senergin-meta, @alpharomercoma, @kuangyuan-cmu, @JPL11, @usmanmateen, @chongzhou96, @Shakhtar-Sankur, @raedle, @mark-inderhees, @borys3kk, @pigei, @WillZZZy
Full Changelog
https://github.com/pytorch/executorch/compare/v1.5.0...v1.6.0