ExecuTorch

AI

On-device inference for PyTorch models.

Latest v1.6.0 · by PyTorchWritten in PythonWebsitepytorch/executorchRSS

Release activity

Release activity — 9 releases across 9 days in the last year. Each cell is one day; darker means more releases that day. Older weeks are hidden at this screen width.
AugSepOctNov
SundayNo releases on Aug 2, 2026No releases on Aug 9, 2026No releases on Aug 16, 2026No releases on Aug 23, 2026No releases on Aug 30, 2026No releases on Sep 6, 2026No releases on Sep 13, 2026No releases on Sep 20, 2026No releases on Sep 27, 2026No releases on Oct 4, 2026No releases on Oct 11, 2026No releases on Oct 18, 2026No releases on Oct 25, 2026No releases on Nov 1, 2026No releases on Nov 8, 2026No releases on Nov 15, 2026
MondayNo releases on Aug 3, 2026No releases on Aug 10, 2026No releases on Aug 17, 2026No releases on Aug 24, 2026No releases on Aug 31, 2026No releases on Sep 7, 2026No releases on Sep 14, 2026No releases on Sep 21, 2026No releases on Sep 28, 2026No releases on Oct 5, 2026No releases on Oct 12, 2026No releases on Oct 19, 2026No releases on Oct 26, 2026No releases on Nov 2, 2026No releases on Nov 9, 2026No releases on Nov 16, 2026
TuesdayNo releases on Aug 4, 2026No releases on Aug 11, 2026No releases on Aug 18, 2026No releases on Aug 25, 2026No releases on Sep 1, 2026No releases on Sep 8, 2026No releases on Sep 15, 2026No releases on Sep 22, 2026No releases on Sep 29, 2026No releases on Oct 6, 2026No releases on Oct 13, 2026No releases on Oct 20, 2026No releases on Oct 27, 2026No releases on Nov 3, 2026No releases on Nov 10, 2026No releases on Nov 17, 2026
WednesdayNo releases on Aug 5, 2026No releases on Aug 12, 2026No releases on Aug 19, 2026No releases on Aug 26, 2026No releases on Sep 2, 2026No releases on Sep 9, 20261 release on Sep 16, 20261 release on Sep 23, 2026No releases on Sep 30, 2026No releases on Oct 7, 2026No releases on Oct 14, 2026No releases on Oct 21, 2026No releases on Oct 28, 2026No releases on Nov 4, 2026No releases on Nov 11, 2026No releases on Nov 18, 2026
ThursdayNo releases on Aug 6, 2026No releases on Aug 13, 2026No releases on Aug 20, 2026No releases on Aug 27, 2026No releases on Sep 3, 2026No releases on Sep 10, 2026No releases on Sep 17, 2026No releases on Sep 24, 2026No releases on Oct 1, 2026No releases on Oct 8, 2026No releases on Oct 15, 2026No releases on Oct 22, 2026No releases on Oct 29, 2026No releases on Nov 5, 20261 release on Nov 12, 2026
Friday1 release on Aug 7, 20261 release on Aug 14, 2026No releases on Aug 21, 2026No releases on Aug 28, 2026No releases on Sep 4, 2026No releases on Sep 11, 2026No releases on Sep 18, 2026No releases on Sep 25, 2026No releases on Oct 2, 2026No releases on Oct 9, 2026No releases on Oct 16, 2026No releases on Oct 23, 2026No releases on Oct 30, 2026No releases on Nov 6, 2026No releases on Nov 13, 2026
SaturdayNo releases on Aug 8, 2026No releases on Aug 15, 2026No releases on Aug 22, 2026No releases on Aug 29, 2026No releases on Sep 5, 2026No releases on Sep 12, 2026No releases on Sep 19, 2026No releases on Sep 26, 2026No releases on Oct 3, 2026No releases on Oct 10, 2026No releases on Oct 17, 2026No releases on Oct 24, 2026No releases on Oct 31, 2026No releases on Nov 7, 2026No releases on Nov 14, 2026

9 releases in the last year

Changelog

What changed from 0 to 1

v1.6.0

Latest
Added 26
  • Experimental ExecuTorch Native format: a portable .ptn program lowered at load time by Vulkan or ggml engines
  • Experimental executorch.runtime.cuda module for choosing the CUDA stream from Python
  • Experimental diffusion extension for Stable Diffusion 1.5 and SDXL
  • Memory-planned delegate scratch buffers and skip_h2d_for_constants
  • Linkable C++ libraries, CUDA runtime, and OpenVINO delegate in the Windows wheels
  • pkg-config file in the Linux and macOS wheels
Changed 23
  • NXP Neutron backend promoted to production-ready
  • Python runtime imports no longer pull in exir, Inductor, or Triton
  • Default Linux thread pool limited to the CPUs the process may use
  • Supported Python versions are now 3.11 through 3.15; Python 3.10 dropped
  • CUDA 12.6 and 13.0 wheels dropped; Jetson Orin now requires a source build
  • Test-only packages removed from the executorch dependencies
Fixed 6
  • Silently wrong XNNPACK outputs with shared BatchNorms and quantized ops left outside a delegate
  • Wrong outputs from Triton SDPA with wide attention heads and from OpenVINO partitions that return a copy of an input
  • Export of zero-stride constants and mixed-dtype cat
  • Python logging deadlocks between threads
  • Arm log-softmax underflow and round behavior
  • QNN shared buffers on OE Linux

From ExecuTorch

Highlights


  • NXP Neutron is production-ready.
  • TensorRT backend in beta.
  • Robotics: LeRobot policies such as SmolVLA and pi0 now run correctly on CUDA, MLX, OpenVINO, and XNNPACK.
  • Torch-free Python runtime: run .pte files from NumPy, buffers, or DLPack without PyTorch.
  • Faster GPU LLMs: batched serving, prefix caching, and DFlash speculative decoding on CUDA and MLX (fully integrated into Muse Glimmer).
  • Python 3.15 and PyTorch 2.15 support.
Runtime and Core

  • Added experimental ExecuTorch Native: a portable .ptn program lowered at load time by Vulkan or ggml engines.
  • Added the experimental executorch.runtime.cuda module for choosing the CUDA stream from Python.
  • Python runtime imports no longer pull in exir, Inductor, or Triton.
  • Added an experimental diffusion extension for Stable Diffusion 1.5 and SDXL.
  • Added memory-planned delegate scratch buffers and skip_h2d_for_constants.
  • Limited the default Linux thread pool to the CPUs the process may use.
Packaging and Build

  • Dropped Python 3.10; supported versions are now 3.11 through 3.15.
  • Added CUDA 13.4 wheels and dropped CUDA 12.6 and 13.0 wheels. Jetson Orin now needs a source build.
  • Added linkable C++ libraries, the CUDA runtime, and the OpenVINO delegate to the Windows wheels.
  • Added a pkg-config file to the Linux and macOS wheels.
  • Removed test-only packages from the executorch dependencies.
Backends

Arm

  • Added FP8 and BF16 to the VGF runtime and updated the Model Converter and ML Emulation Layer to 0.11.
  • Added dynamic-range W8A8 quantization for linear layers.
  • Added experimental delegate-owned streaming state for Ethos-U and updated the Ethos-U core driver to 26.08.
  • Removed dynamic allocation from the Ethos-U backend and hardened Vela binary parsing.
  • Added VGF quantization sensitivity and quality analysis tools.
  • Added SDPA decomposition and SAME-padding fusion for Cortex-M.
  • Added pow, isinf, isnan, TopK, softplus, roll, MaxPool1D, and negative padding.
  • Added a person detection example.
  • Removed deprecated tools, including examples/arm/aot_arm_compiler.py.

Qualcomm

  • Updated the QNN SDK to 2.51.
  • Added Flexible Context Binary export: one artifact for multiple Snapdragon SoCs.
  • Added on-device OpTrace and host-only Hextimate HTP profiling.
  • Added an HTP graph-splitting option for faster context finalization.
  • Added SC8480XP and Snapdragon 7+ Gen 3 support.
  • Added 8-bit KV-cache I/O for Hugging Face LLMs and W4 embedding and LM-head quantization for LLM QAT.
  • Added an example for supporting Custom Ops on LPAI backend
  • Added AdamW optimizer for on-device training.
  • Added STFT, empty_strided, and per-channel-group quantization.

CUDA and ROCm

  • Added autotuned INT4/5/6/8 quantized GEMM kernels on CUDA and ROCm, covering decode and small-batch prefill (M ≤ 64).
  • Made operator autotuning more accurate: candidates are now timed in CUDA graphs, so CPU overhead no longer affects the choice. Users can also set the example inputs used for benchmarking, and the tuner can choose between implementations (e.g. SDPA with or without split-K).
  • Added off-graph KV caches with a batching executor and prefix-cache reuse, on-device sampling, and DFlash drafting in CUDA graphs.
  • Sped up long-context attention with TMA causal prefill, in-kernel causal masks, and hardware-aware split-K decode.
  • Added lowering for complex-number RoPE, fused attention variants, and boolean mask indexing.

Apple (Core ML, Core AI, Metal, MLX)

  • Added a batched MLX KV cache with DFlash support.
  • Reduced MLX peak memory and added dynamic GroupNorm and upsampling.
  • Added Core AI palletization; removed sidecar asset delivery.

Vulkan

  • Fully delegated dynamic transformer blocks.
  • Added fused SwiGLU and faster softmax and reductions.
  • Tuned kernels for Mali and Adreno.

NXP

  • Promoted the Neutron backend to production-ready.
  • Updated the eIQ Neutron SDK to 3.2.3.
  • Added recipe-based lowering for Neutron backend, Neutron recipe-based lowering supports both Post-training quantization and QAT workflows.
  • ExecuTorch test harness model suite enabled for Neutron backend.

XNNPACK

  • --xnnpack-extended-ops is now on by default in LLM export.
  • Kept unsupported BF16 and high-rank ops on the CPU instead of failing.

Other Backends

  • Cadence: added HiFi5 support.
  • Samsung: added Llama 3.2 1B, Gemma 3 1B, and YOLO26.
  • OpenVINO: CPU inference now defaults to float32; use the new inference_precision compile spec to change it.
  • WebGPU: added browser execution and an experimental Playground.
  • MediaTek: temporarily unsupported pending a Python 3.11+ NeuroPilot converter.
Operators and Kernels

  • Added portable 3-D convolution, antialiased bicubic resize, bucketize, and _fft_r2c.
  • Added KleidiAI BF16 GEMM and SDPA dispatch.
  • Half and BFloat16 matmul and reductions now accumulate in float.
  • Floating-point floor_divide now matches ATen.
Notable Fixes

  • Fixed silently wrong XNNPACK outputs with shared BatchNorms and quantized ops left outside a delegate.
  • Fixed wrong outputs from Triton SDPA with wide attention heads and from OpenVINO partitions that return a copy of an input.
  • Fixed export of zero-stride constants and mixed-dtype cat.
  • Fixed Python logging deadlocks between threads.
  • Fixed Arm log-softmax underflow and round behavior.
  • Fixed QNN shared buffers on OE Linux.
Contributors

We welcome 35 first-time contributors to ExecuTorch in this release: @shoemoney, @fatih-uzlmz, @peygar, @Air-crypto, @bharqav, @Gallinator, @sqliangx, @Ivan-Dimitrov, @milesmanxp, @Dev-next-gen, @quic-xiyushi, @David-Wu1119, @Nicoshev, @amacharla15, @estervojko, @dongjiang1989, @chinnadhurai, @elbourne12345, @pcwu0329, @u7k4rs6, @TejasweeSulekh, @paul1106, @HiroForYou, @senergin-meta, @alpharomercoma, @kuangyuan-cmu, @JPL11, @usmanmateen, @chongzhou96, @Shakhtar-Sankur, @raedle, @mark-inderhees, @borys3kk, @pigei, @WillZZZy

Full Changelog

https://github.com/pytorch/executorch/compare/v1.5.0...v1.6.0

View originalPermalink
How v1.6.0 went

v1.5.1

Added 24
  • Added the public get_thread_count() API
  • Expanded Arm support for symbolic shapes and dynamic convolution and pooling
  • Added multi-method model support to the export pipeline and per-backend load options
  • Added capacity-aware memory planning over target memory maps and support for shared allocations with offsets
  • Added schema-version checks for PTE and PTD files and clearer failures for invalid device-planned copies and missing delegate data
  • Added shared pre-decomposition and graph-transform hooks, including common view/permute propagation and reinplace passes
Changed 5
  • Updated CMSIS-NN to v8.0.0 and aligned Arm runner dependencies
  • Folded redundant Q/DQ pairs in the shared EXIR to_edge lowering path
  • Improved constant propagation, symbolic-shape handling, and device placement for memory-planned buffers
  • Reduced wheel size and improved portability by stripping local macOS symbols, removing unreachable vendored sources, and dropping unusable Linux MKL search paths
  • Improved CUDA delegate integration with per-thread streams, a warm memory pool, cross-method weight sharing, and clearer missing-weight errors
Fixed 2
  • Lowered the supported CMake floor so CMake 3.26 through 3.28 builds work again
  • Fixed XNNPACK pooling with default strides and single-element kernels, convolution dimensionality inference, and shared-observer handling for slices
View originalRead the notes
How v1.5.1 went

v1.5.0

Added 18
  • Expanded the Python wheels into a C++ SDK with linkable libraries for the CUDA, Core ML, MLX, OpenVINO, and Qualcomm delegates and packaged TorchAO kernels
  • Added multi-method model support to the export pipeline and per-backend load options
  • Added capacity-aware memory planning over target memory maps and support for shared allocations with offsets
  • Added schema-version checks for PTE and PTD files and clearer failures for invalid device-planned copies and missing delegate data
  • Added shared pre-decomposition and graph-transform hooks, including common view/permute propagation and reinplace passes
  • Added a C++ SDK to the Python wheel and split the runtime from the Python bindings extension
Changed 3
  • Lowered the supported CMake floor so CMake 3.26 through 3.28 builds work again
  • Reduced wheel size and improved portability by stripping local macOS symbols, removing unreachable vendored sources, and dropping unusable Linux MKL search paths
  • Optimized BF16 SDPA and fixed NaNs in CUDA split-K attention for Voxtral
Fixed 1
  • Fixed pooling with default strides and single-element kernels, convolution dimensionality inference, and shared-observer handling for slices in XNNPACK
View originalRead the notes
How v1.5.0 went

v1.4.1

Fixed 2
  • Fixed an issue with incorrect baked in paths within the wheels.
  • Fixed an issue with XNNPack's PreLU in quantized graphs.
View originalRead the notes
How v1.4.1 went
v1.4.0

v1.4

Added 23
  • Expanded Cortex-M support across additional CPU variants, model tests, scratch-buffer planning, CMSIS-NN integration, and CMSIS Pack builds
  • Added Arm backend coverage for TOSA dialect lowering, dynamic shapes, FP8/MXFP/BFloat16 paths, VGF profiling, and LLM export workflows
  • Added Vulkan support for Linux and Windows desktop GPUs with opt-in wheel builds, automatic build enablement, and a functional backend availability check
  • Expanded Qualcomm QNN support with 2-bit 16a2w quantization, faster VLM/LLM PTQ flows, multi-batch quantization and evaluation, LPAI tooling, profiling, and additional operators
  • Added explicit dim-order legalization for channels-last portable fallbacks
  • Added builds for more Cortex-M variants on Corstone-300, including M0+
Changed 11
  • NXP Neutron Converter flow moved to an MLIR-based flow with all operators using it, and the Neutron backend now supports 48 ATen operators
  • Migrated Android Module, TrainingModule, SGD, and LLM extension APIs from Java to Kotlin
  • Hardened tensor-layout validation and segment loading against null metadata
  • Improved symbolic input metadata preservation during exported-program pass replay
  • Improved direct bundled-program serialization for embedded flows
  • Decoupled calibration, quantization, compilation, and evaluation stages for faster Qualcomm LLM and VLM workflows
Fixed 6
  • Fixed portable anti-aliased bilinear upsampling correctness
  • Fixed channels-last handling for portable fallbacks and Cortex-M DS-CNN
  • Fixed a QNN runtime null dereference when op-package options are absent
  • Fixed quantized Conv1d same padding for even kernels in XNNPACK
  • Fixed several Arm partitioning, dynamic-shape, quantization, and runner integration issues
  • Fixed multiple NXP Neutron partition, format, and conversion issues
Removed 1
  • Removed the legacy _use_edge_ops toggles from the runtime/export path
View originalRead the notes
How v1.4.0 went

v1.3.1

Added 13
  • Qwen3.5 MoE export and runner support across CUDA and Metal, including fused MoE kernels, INT4 paths, CUDA graph support, and runtime stats
  • Gemma 4 31B export and runner support with CUDA and MLX backend support
  • LFM2.5 export and runner support for MLX
  • Voxtral Realtime and Voxtral TTS improvements, including streaming fixes, CUDA backend support, and documentation updates
  • Android LLM runner documentation and HuggingFace-oriented usage docs
  • Portable adaptive average pool 2D kernel
Changed 3
  • Safer ETRecord and export deserialization
  • Refactored dim-order and permute handling in the Arm backend, removing the legacy tosa_dim_order path and reducing unnecessary transposes in lowered graphs
  • eIQ Neutron SDK updated to version 3.1.1 in the NXP backend
Fixed 4
  • Fixes for torch.split alias annotations and partition grouping in EXIR export
  • Quantized max pool and quantized conv1d fixes
  • Integer overflow and heap-buffer fixes in the Vulkan backend
  • Android lifecycle and error handling fixes in the runtime
View originalRead the notes
How v1.3.1 went

v1.2.0

Added 19
  • Voxtral Realtime streaming speech model with true streaming inference on XNNPACK, Metal, and CUDA, including int4 quantization and fp32/bf16 support
  • Qwen3.5 export support for 0.8B, 2B, and 4B variants, plus Qwen2.5-Coder
  • NVIDIA NeMo Parakeet support on XNNPACK by default, with int4 quantization, Metal int4 via HQQ, bfloat16, and CUDA
  • NVIDIA NeMo Sortformer speaker diarization model on XNNPACK and CUDA
  • Silero VAD voice activity detection on XNNPACK for speech endpoint detection
  • Flatbuffer Program serialization in EXIR for faster export and model loading
Changed 11
  • Default memory-mapped load mode is now MmapUseMlockIgnoreErrors for more robust loading
  • Pybindings pybindings.pyi type stubs fixed and completed for better IDE support
  • Ethos-U SDK updated to 26.02 with Vela 5.0.0
  • NXP backend migrated to eIQ Neutron SDK with selective kernel registration and weight prefetching from external memory to SRAM
  • Compile-time optimizations reduce core runtime code size by about 10 to 15kb depending on toolchain
  • Suppress .eh_frame generation when EXECUTORCH_OPTIMIZE_SIZE is ON
Fixed 3
  • MemoryFormatOpsPass now preserves input dim_order for clone and to_copy with no memory_format kwarg
  • Integer overflow in Tensor.numel() is fixed
  • Uninitialized memory in the portable batch norm kernel is fixed
Deprecated 1
  • MPS backend is deprecated and will be removed in a future release (v1.4.0)
View originalRead the notes
How v1.2.0 went

v1.1.0

Added 17
  • CUDA backend for NVIDIA GPUs as an experimental backend with AOTInductor compilation, Triton SDPA kernels, INT4 weight quantization, and async memory allocation, with Windows support
  • Metal backend for Apple GPUs as an experimental backend using Metal Performance Shaders with a custom SDPA kernel
  • Vulkan static int8 quantization infrastructure for statically quantized CNNs with packed int8 tensors, specialized memory layouts for conv/matmul, and fully quantized conv2d variants
  • Cadence DSP optimized Vision module kernel library with quantized implementations, plus RoPE, group-quantized embedding, and batched matmul operator support for LLMs
  • A8W4 quantization (4-bit weights, 8-bit activations) for Arm Ethos-U NPU across conv2d, conv3d, depthwise conv, and linear operators with automatic runtime unpacking
  • Qualcomm backend infrastructure for vision-language models including SmolVLM-500M and InternVL3-1B with vision encoder quantization, multimodal AOT compilation, and Adreno GPU support
Changed 9
  • deserialize_pte_binary() now returns a PTEFile object, with the program accessed via the .program attribute
  • make_tensor_ptr() supports custom sizes, dim order, and strides for squeeze/unsqueeze operations, and views keep the source tensor alive via a custom deleter
  • execute() returns InvalidState if called during step() and auto-resets on error
  • FreeableBuffer supports uint64_t data pointers for cross-core memory and models larger than 2GB
  • op_choose_qparams is parallelized and op_quantize uses ARM NEON SIMD and multithreading, accelerating quantized KV-cache prefill in LLMs
  • slice_copy uses multithreading for large workloads, reducing RoPE computation overhead in LLM prefills
Fixed 2
  • Constants sharing data but having different FQNs now have both FQNs saved
  • mmap on Windows uses _fstat64 and uint64_t offsets, enabling files larger than 2GB
Removed 1
  • The deprecated qnnpack backend has been deleted; users should migrate to XNNPACK
Security 3
  • Integer overflow check added in HierarchicalAllocator
  • Stack buffer overflow in resize_tensor fixed with a bounds check on the dimension limit
  • Overflow-safe arithmetic added to nbytes() and numel() using c10::mul_overflows()
View originalRead the notes
How v1.1.0 went

v1.0.1

Added 1
  • Add pytorch-tokenizers as a Python dependency for ExecuTorch, removing the need to install it from source
Changed 3
  • Update coremltools dependency to version 9.0
  • Improve support for bool and int tensors in the Vulkan backend
  • Display a clear error message when building from source on Windows and git symlinks are not enabled
Fixed 2
  • Fixed start position updates in the text LLM prefill logic
  • Fixed handling of negative indices in the portable index operator
Security 1
  • Hardened several portable operator implementations against out-of-bounds memory access
View originalRead the notes
How v1.0.1 went

v1.0.0

Added 34
  • Expanded platform support for ARM64 Linux and experimental native x86 Windows for Python model preparation and runtime builds
  • New APIs for running multimodal models on Android, iOS, and desktop, with supported models including Voxtral and Gemma3
  • LoRA inference capability allowing multiple LoRA .pte files to share a single set of foundation weights
  • 4-bit HQQ quantization support for improved model accuracy
  • Python and Maven (Android) package variants for Vulkan and QNN backends
  • Experimental JavaScript support for the ExecuTorch runtime via WASM
Changed 13
  • Removed buck as a dependency for CMake builds
  • Vulkan and QNN backends are linked in Python runtime builds when enabled
  • Consolidated and cleaned up CMake options and build structure
  • Ethos-U backend reached production quality with stable operator coverage, CI validation, and verified embedded integration
  • Partitioner and compile interfaces for the Arm backend simplified and refactored for easier extension and maintainability
  • Arm backend ahead-of-time path extended to support the full set of Arm backend operators, exceeding 80% Edge dialect coverage
Fixed 3
  • Correct handling of rank-0 tensors and improved input/output handling in the portable executor runner for the Arm VGF backend
  • Crashes for several operators with mixed dtypes in the Core ML backend
  • Support for models and partitions with no user inputs in the Core ML backend
View originalRead the notes
How v1.0.0 went

v0.7.0

Added 16
  • Unified export_llm CLI and API to export supported LLM models, replacing the old export_llama script
  • Generic text LLM runner supporting all text-only decoder-only LLM models, fully integrated with Android and iOS demo apps
  • Experimental program-data separation APIs to export program and data files separately, introducing the .ptd file extension for data-only files, with support for portable operators and the XNNPACK backend
  • Runtime API for PAL overrides and log routing
  • Support for Qwen3, SmolLM3, and Gemma3 LLM models
  • Model API for data type selective build in CMake
Changed 11
  • KleidiAI is now enabled by default in the XNNPACK backend, providing optimized low-bit matrix multiplication kernels that boost Llama prefill and decode performance
  • Minimum CMake requirement bumped to 3.29
  • XNNPACK enabled by default for pip wheel builds
  • Arm backend updated to support the TOSA 1.0 specification with improved operator coverage
  • Arm backend quantization improved with enhanced per-channel quantization support and QAT improvements
  • Arm backend pooling, broadcasting, and memory allocation strategies improved
Fixed 1
  • MPS example fixed for non-LLM models
Deprecated 1
  • Constant buffer will be deprecated in the runtime in coming releases; PTE files generated before ExecuTorch 0.4 should be re-exported
View originalRead the notes
How v0.7.0 went
View all

Discussion

If you publish ExecuTorch, you can claim this product by proving you administer its repository.