ONNX Runtime

ONNX Runtime 1.30.0 expands generative AI inference, improves CPU and GPU performance, adds Go bindings, and strengthens runtime reliability. These notes cover changes since ONNX Runtime 1.29.1.

Highlights

• Expanded CUDA inference support with variable-length causal convolution for continuous batching, speculative decoding in paged XQA, and INT4 paged KV caches with per-channel scales (#32168, #32340, #32515).
• Improved WebGPU PagedAttention, added GPT-OSS support and INT8 KV-cache block quantization, and extended convolution optimizations (#31727, #32277, #32284, #32420).
• Added fused CPU LinearAttention kernels for AVX-512, Arm64 NEON, and SVE, plus AVX2 LayerNorm/RMSNorm acceleration (#31674, #31973, #32178, #32356).
• Added Go bindings for the ONNX Runtime C API and DeepSeek Engram contrib operators (#29615, #32268).

Announcements & Compatibility

• FP4 QMoE kernels are now enabled by default in CUDA builds, with Windows build support added in this release. Source builds can opt out with -Donnxruntime_USE_FP4_QMOE=OFF (#32096, #32163).
• CUDA fpA-intB builds now default to a compact kernel set for FP16 activations, INT4/INT8 weights, scale-only quantization, and block_size=32. Set -Donnxruntime_USE_FPA_INTB_GEMM_FULL=ON when building from source to retain the full kernel set, including BF16, zero-point, bias, larger-block-size, and native Hopper variants (#32324).
• CPU FP16 Gemm and MatMul execution is gated on hardware acceleration. CPU-assigned FP16 nodes without a matching kernel now fall back to FP32 (#32301, #32197).
• WebGPU plugin EP packaging now supports Linux AArch64. Plugin versions were advanced to WebGPU 0.4.0 and CUDA 0.2 (#32287, #31960, #31970).

Security & Reliability

Model Loading, Memory, and Input Validation

• Limited nested model-graph depth and canonicalized external-data locations to harden model loading (#32344, #32135).
• Added checked rounding for BFC arena allocations and fixed prepacked-weight reference lifetimes (#32010, #32040).
• Strengthened shape, rank, and parameter validation for Split, Scan, GatherND, ScatterND, SpaceToDepth/DepthToSpace, Crop, Conv, Normalizer, and pooling (#29461, #31668, #32034, #32039, #32076, #32157, #32160, #32161, #32345, #32349).
• Hardened generation and attention input handling, including attention-attribute narrowing, BifurcationDetector inputs, generation subgraph shapes, and QEmbed segment inputs. BeamSearch buffer expansion now uses dynamic shape storage (#31648, #31701, #32009, #32078, #32144).
• Validated TreeEnsemble node references and bounded subtree comparison, rejected non-finite CPU RoiAlign coordinates, and required ImageScaler bias to match the channel count (#32031, #32043, #32011, #32002).
• Added an allowlist of safe LoRA adapter parameter data types, validated MatMulFpQ4 shape inputs, and checked MLAS blockwise quantization/dequantization index ranges (#31682, #32032, #32007).

GPU Bounds and Resource Lifetimes

• Hardened CUDA indexing and buffer-size arithmetic in MatMulNBits, RemovePadding, RotaryEmbedding, SparseAttention, Whisper beam search, NMS, QDQ, and GatherElements (#31643, #31994, #31995, #31996, #31998, #32014, #32029, #32030).
• Fixed overflow in CUDA reduction scans and Softmax offset arithmetic, and handled zero-sized outputs in CUDA random-generator kernels (#32137, #32330, #31997).
• Fixed CUDA MultiHeadAttention shared-cache scratch lifetimes and kept CudaAsyncBuffer staging storage alive across CUDA graph replay (#31968, #32121).
• Fixed WebGPU out-of-bounds subgroup-matrix loads for partial tiles, zero-initialized writable device-allocator buffers, and rejected foreign GPU handles in built-in data transfers (#32364, #32063, #32317).

Dependencies and Tooling

• Upgraded Protobuf to 33.6 and refreshed Python documentation dependencies, including an ONNX security-related update (#29906, #32190, #32424).
• Updated JavaScript dependencies including js-yaml, joi, fast-uri, and the Next.js end-to-end fixture (#32397, #32486, #32488, #32505, #32508).
• Pinned GitHub Actions to full-length commit SHAs and strengthened packaging infrastructure with authenticated package feeds and NPM network isolation (#32176, #32005, #32440).

New Features

Core APIs & Runtime

• Added Go bindings for the ONNX Runtime C API (#29615).
• Extended memory importing with host-pointer support and added access to preallocated outputs through KernelContext::GetPreallocatedOutput (#29726, #32089).
• Added packed-attention workspace recipes and estimates, and made workspace input-shape handling aware of optional inputs (#32283, #32321, #32312).
• Added DeepSeek Engram contrib operators, EngramGate and NGramHashMapping, and expanded kernel coverage for Qwen-3.5 operators (#32268, #32106).

Plugin Execution Providers

• Added LoRA adapter support with plugin EP allocators, preserved custom allocators during EP registration, and reset plugin stream chunks before release (#32221, #32272, #31983).
• Fixed CUDA plugin device discovery on WSL and improved Windows ARM64 packaging (#32517, #32355).

Execution Provider Updates

CUDA EP

Attention and Decoding

• Added INT4 paged KV caches with per-channel scales and an is_causal attribute to PagedAttention (#32515, #32225).
• Extended paged XQA with speculative decoding, query-to-KV head group size 6, head size 256 including FP16 caches, and native block tables for 128-token pages (#32340, #32108, #32229, #32263, #32127).
• Enabled split-KV for paged FlashAttention decode and improved PagedAttention dispatch diagnostics (#32102, #32099).
• Added VarlenCausalConvWithState for continuous batching and compact variable-length causal-convolution state updates (#32168, #32290).
• Added a compact GatedDeltaNet operator and BFloat16 support for CUDA GatedDeltaNet (#32282, #32307).

MoE and Quantized Matrix Multiplication

• Added an opt-in FP8 DeepGEMM MoE decode path for supported fixed-shape QMoE workloads on Hopper GPUs (ORT_QMOE_FP4_DEEPGEMM=1, default off). This path is disabled on Windows (#32122, #32485).
• Bounded QMoE workspace with configurable row tiling and FP8 weight-dequantization scratch memory by tiling over output columns (#32097, #32129).
• Vectorized NVFP4 weight dequantization for prefill, tuned NVFP4 GEMV tiling for Qwen multi-token prediction, and extended speculative-decode GEMVs to 64 rows (#32128, #32140, #32289).
• Tuned FP4 and FP8 GEMV scheduling for 48-SM SM121 GPUs, including FP8 KSplit32 scheduling, and improved FP8 GEMV residency for grids just beyond two blocks per SM (#32408, #32409, #32433).
• Added an opt-in split-K GEMV path for small-N FP16 MatMul shapes and refined fpA-intB GEMV support checks (#31478, #32338).

Operators

• Improved TopK, ArgMax, and ArgMin performance for wide last axes, and accelerated low-lane INT64 CumSum (#32404, #32092, #32238).
• Added a single-memcpy Slice fast path for contiguous subregions and removed pinned-buffer use from Split and Concat fast paths (#28902, #32410).
• Registered BF16 ReduceMean kernels and fixed ScatterElements reduction dispatch by element type and signed-zero handling in Abs (#32326, #29879, #31477).

WebGPU EP

• Improved PagedAttention and added PagedAttention metadata, GPT-OSS support, and INT8 KV-cache block quantization (#31727, #32277, #32284).
• Added INT64 Gather support and optimized MatMulNBits wide tiles with subgroup shuffle (#31714, #31703).
• Extended convolution fusion with eight additional activations, fused activation handling in the im2col path, and uniform-based activation parameters for Conv/MatMul (#32117, #32185, #32116).
• Reused subgroup-matrix MatMul for pointwise convolution and added convolution-weight prepacking for the im2col-matmul path (#32304, #32420).
• Enabled GELU and BiasGELU fusion, added transpose-optimizer handling for Elu and contrib GELU variants, and enabled LayerNorm fusion to fix FP16 inference correctness (#32053, #32118, #32294).
• Added subgroup-size control, selected subgroup size 32 for subgroup-matrix MatMul/Gemm, and enabled the subgroup-matrix path in WASM builds (#32056, #32306, #32269).
• Scaled Dawn pipeline-compilation workers with CPU count, vectorized Split when all output segments are vec4-aligned, and selected pooling paths based on occupancy (#29820, #32251, #32313).
• Exposed safe graph-capture I/O in Python, added a robustness provider option, and enabled forwarding of onnxruntime_perf_test -i options to WebGPU (#32074, #31971, #32316).
• Fixed synchronization for user-provided GPUDevice instances, corrected MatMul pipeline-cache keys and the 1D-dispatch shader fast path, and changed copy_tensors misuse to report errors instead of terminating the process (#32259, #32048, #32343, #32315).

WebNN EP

• Added SkipLayerNormalization support and corrected output-rank validation and fallback data-type support checks (#32377, #31708, #32067, #32293).

TensorRT & DirectML

• Fixed TensorRT shape-value handling (#32415).
• Validated kernel_shape and output_padding lengths during DirectML kernel setup (#31999).

CPU & Core Optimizations

MLAS and CPU Kernels

• Added fused LinearAttention kernels for AVX-512, Arm64 NEON, and SVE (#31674, #32178, #32356).
• Added AVX2 LayerNorm/RMSNorm kernels and registered BFloat16 LayerNorm/RMSNorm kernels on the CPU EP (#31973, #31974).
• Added Arm SVE i8mm INT8 QGEMM kernels and enabled the SBGemm fast-math path on Darwin Arm64 (#31146, #32152).
• Improved NCHWc convolution thread utilization, added HardSwish fusion for MobileNetV3 models, and introduced an AVX-512 16-wide Erf kernel and NCHWc reorder improvements for MobileCLIP-S0 (#31660, #31957, #31958).
• Optimized INT4 weight prepacking on CPU, rejected KleidiAI Q4 prepacking with dynamic scales, and fixed Arm64 SymmQgemm INT16 overflow (#31690, #32068, #32057).
• Fixed FP16 QuantizeLinear rounding, prevented CPU TensorScatter index overflow, serialized ScatterND string updates, and widened Compress loop counters (#32452, #32012, #32033, #32008).
• Improved edge-case handling for empty LpNormalization inputs, zero-element BiasGelu/FastGelu, extreme Trilu diagonals, and empty reduction axes (#32020, #31698, #32013, #32156).

Graph, Optimizer, and Runtime

• Hardened graph optimizers against model-supplied indices and invalid fusion inputs, including MatMulNBits dequantization block sizes, GQA projection shapes, and Slice starts rank (#31670, #31678, #32018, #32044).
• Prevented overlapping MatMulIntegerToFloat fusions, skipped overridable-initializer fusion, fixed FuseInitializersTransformer consumer-input lookup, and avoided fusing identity transposes into Gemm (#32038, #32143, #32426, #32435).
• Fixed optional zero-point input handling in QDQ, empty-initializer axis scaling, and NodeAttrHelper string-default lifetimes (#32051, #32138, #32019).
• Released external-data loaders after graph initialization, added portable random-access file reads to Env, and clarified how external-initializer paths interact with EP context paths (#32502, #32503, #32442).
• Avoided redundant ONNX schema registration when static registration is enabled, updated cpuinfo with thread-safe deinitialization, and prevented telemetry crashes in shell-less containers (#32353, #32300, #32226).
• Reduced the active-session mutex scope around ETW callback registration and filtered out display adapters using the Microsoft Basic Render Driver (#32000, #32006).

Language Bindings & Web

• Retained Python asynchronous-run resources and pinned C# RunAsync arguments until completion (#32041, #32015).
• Fixed Java provider-option value cleanup, validated Rust tensor element types, and made Rust string-tensor output extraction safe (#31828, #32035, #32045).
• Avoided duplicate N-API cleanup-hook registration in Node.js and added FP16 support to the JavaScript WebGPU Conv3DNaive shader (#32469, #32357).
• Fixed WinML image-dimension overflow (#32046).

Build, Packaging & CI

• Upgraded CUTLASS to 4.7 and cuDNN Frontend to 1.27, and enabled compact fpA-intB kernel builds by default (#32111, #32324).
• Fixed Windows CUDA 12.9 SM120 compilation, CUDA 13 CCCL include paths in plugin builds, Windows DeepGEMM build errors, and PagedAttention builds without FlashAttention (#32114, #32392, #32485, #32327).
• Added D3D12 Agility SDK support for Dawn WebGPU builds while excluding WebGPU plugin EP builds from Agility SDK use (#32253, #32380).
• Added BTI support to MLAS AArch64 assembly, restored runtime vector-extension checks on RISC-V, and disabled POWER MLAS kernels when VSX is unavailable (#32070, #32406, #32389).
• Improved CUDA plugin ARM64 build parallelism and packaging tests, and based plugin development versions on commit timestamps (#32165, #32399, #32072, #32095).
• Separated provider compilation from archive creation, corrected Python-binding delay-load linkage, and fixed Linux minimal-build dependencies and spurious Windows Spectre-mitigation warnings (#32162, #31637, #32491, #32516).
• Expanded PagedAttention CUDA test coverage and head-size-256 paged XQA coverage, corrected GatedDeltaNet test verification, and removed a Hugging Face download dependency from a dynamic-quantization test (#31687, #32262, #32329, #32443).
• Added WGSL template tests to CI and enabled WebGPU CI on WebGPU plugin EP release branches (#32214, #32090).

Contributors

Thanks to our 57 human contributors for this release!

@4n4ny4, @apsonawane, @arnej27959, @baijumeswani, @bmehta001, @chilo-ms, @crvineeth97, @daijh, @danfiedler-msft, @danielsongmicrosoft, @dannyota, @DKAIN-py, @edgchen1, @ericcraw, @eserscor, @fanchenkong1, @hanbitmyths, @hariharans29, @hdharpure9922, @Honry, @jambayk, @javier-intel, @jchen10, @jiafatom, @jnagi-intel, @justinchuby, @kadyrbekovhamit-cyber, @kunal-vaishnavi, @Lapis0x0, @LOGO127, @Manogna-Sree, @martin-klacer-arm, @mei1127, @miaobin, @mirounga, @miyanyan, @MohamedElashri, @mustjab, @Nikhi00718, @Noperi0r, @Novestars, @pkubaj, @preetha-intel, @qjia7, @rvandermeulen, @sanaa-hamel-microsoft, @skottmckay, @sushraja-msft, @swetha097, @sylvesterkaczmarek, @tianleiwu, @titaiwangms, @toothache, @xadupre, @xhcao, @xiaofeihan1, @Zestion

Full Changelog: rel-1.29.1...rel-1.30.0

Release highlights were prepared with AI assistance.

This is a patch release on top of v1.29.0, containing GroupQueryAttention capability and KV-cache layout improvements, plugin Execution Provider performance tooling updates, and targeted graph and optimizer fixes.

GroupQueryAttention

• Added bidirectional GroupQueryAttention support on CPU and CUDA through a backward-compatible causal attribute, with explicit handling for unsupported execution paths (#31704)
• Added a session option and Execution Provider metadata contract for using the BNHS Value KV-cache layout, with graph transformations that preserve compatibility with the existing BNSH operator schema (#32139)
• Added CPU support for attention_bias with a sliding-window KV cache, including explicit position IDs and post-eviction bias indexing (#32302)

Runtime and Performance Tools

• Fixed Compile API model serialization when output-model and custom initializer-location callbacks are used together, preventing duplicate graph fields in emitted models (#32303)
• Updated onnxruntime_perf_test to use plugin Execution Provider device allocators for generated inputs, loaded test data, and pre-allocated outputs, avoiding unnecessary per-run host/device copies (#32244)

Bug Fixes and Documentation

• Hardened FastGelu fusion to skip malformed Mul and Pow patterns (#32016)
• Added validation for in-memory external initializer references, rejecting unregistered or mismatched data before graph transformation (#32042)
• Restored the C API documentation workflow by switching the pinned Doxygen download to the official GitHub release asset (#32210)

Contributors

Thanks to our 7 contributors for this release!

@adrastogi, @apsonawane, @edgchen1, @javier-intel, @jnagi-intel, @tianleiwu, @Wayne-Ch

Release highlights were drafted with AI assistance and are subject to release-team review.

Full Changelog: v1.29.0...v1.29.1

This is a patch release on top of v1.28.1, containing a targeted fix for Compile API model serialization.

Highlights

Bug Fixes

• Fixed Compile API callback serialization to prevent duplicate graph nodes, inputs, outputs, and value information in emitted optimized models, including models with embedded or external initializers (#32303)

Contributors

Thanks to our contributor for this release!

@adrastogi

Full Changelog: v1.28.1...v1.28.2

These release notes were drafted with assistance from GitHub Copilot.

ONNX Runtime WebGPU Plugin EP 0.3.0 expands model and data-type coverage, improves generative-model performance, and strengthens configuration, reliability, and release tooling.

These release notes were drafted with AI assistance.

Highlights

Model and operator coverage

• Added initial PagedAttention support, MRotaryEmbedding, GRU, DFT, PRelu, HardSwish, Trilu, Max and Min, and MatMulBnb4. (#31611, #31976, #29840, #29454, #30512, #29828, #29845, #29833, #29587)
• Expanded integer support across common operators, including int64 for Add, Cast, Clip, Concat, Equal, Gather, Min, Max, ReduceSum, Reshape, Sub, Tile, and Where; uint8 for Cast, Expand, Gather, and Reshape; and int32/uint32 for CumSum and Tile. (#28804, #29392, #29830, #29834, #29839, #29844, #29847, #29854, #29861, #29897, #31049, #31702, #31709, #31714)
• Added 2-bit GatherBlockQuantized support and integrated ONNX 1.22 with opset 27. (#29054, #28754)

Generative models

• Added quantized KV cache support and expanded GQA with sliding-window cache, batched right-padded prompts, and FlashAttention graph capture for Gemma 4. (#28059, #29236, #29247, #29904)
• Fixed GQA and quantized KV cache correctness for rotary batched prefill, split-reduce decode, and per-batch sequence lengths. (#29002, #29593, #29752)

Performance

• Added deferred dispatch to parallelize shader compilation during cold start. (#29557)
• Added Intel subgroup-matrix MatMul and FP16 Gemm kernels, then extended them with batched-B and odd-N weight support and additional tiling optimizations. (#29271, #29592, #29749, #29813, #29893, #30514)
• Switched Softmax and non-flash Attention to an online softmax algorithm and tuned FlashAttention decode. (#29694, #29724, #29586)

Reliability and configuration

• Added a robustness provider option, made the pending-dispatch limit configurable, and exposed WebGPU buffer-cache options through the JavaScript API. (#31971, #28894, #29017)
• Improved validation and bounds handling in GridSample, LSTM, DynamicQuantizedLSTM, Slice, and GatherBlockQuantized, and fixed WebGPU data-transfer callbacks on Windows x86. (#28704, #29030, #29255, #29595, #31568)
• Corrected profiling timestamp alignment with the ONNX Runtime profiler. (#29021)

Build and release tooling

• Added device-free compile-only sessions for offline graph transformation and removed the dynamic Duktape/Node WGSL-generation path in favor of generated shader sources. (#29681, #28355, #29141)
• Upgraded Dawn, fixed parallel-build races, refreshed release and packaging documentation, and enabled WebGPU CI on plugin release branches. (#29389, #29040, #29777, #29059, #32090)

Contributors

Thank you to everyone who contributed to this release:

@AngelGalindo7, @daijh, @danielsongmicrosoft, @edgchen1, @fanchenkong1, @feich-ms, @guschmue, @haoxli, @hariharans29, @Honry, @huningxin, @jchen10, @Jiawei-Shao, @miaobin, @mingmingtasd, @mirounga, @mustjab, @nicholascelestin, @prathikr, @qjia7, @Reranko05, @Shivani767, @skottmckay, @ssam18, @sushraja-msft, @tairenpiao, @tianleiwu, @titaiwangms, @wuisabel-gif, @xhcao, and @xiaofeihan1.

Scope: commits affecting ONNX Runtime WebGPU Plugin EP code, tests, build integration, and packaging since plugin-ep-webgpu/v0.2.1.

This is a patch release on top of v1.28.0, containing support for device-free WebGPU compilation, improved compatibility with sandboxed Windows processes, and targeted graph-validation fixes.

WebGPU EP

• Added support for device-free compile-only sessions, enabling offline graph transformation and optimized-model serialization without access to GPU hardware (#29681)

Bug Fixes

• Prevented an access violation in Windows processes under Win32k lockdown by skipping DXGI device discovery (#29755)
• Allowed zero-input EPContext nodes, aligning their schema with support for compiling zero-input models (#29799)
• Hardened FastGelu fusion to skip malformed Mul and Pow patterns (#32016)
• Added validation for in-memory external initializer references, rejecting unregistered or mismatched data before graph transformation (#32042)

Contributors

Thanks to our 4 contributors for this release!

@apsonawane, @shiyi9801, @adrastogi, @mingmingtasd

Full Changelog: v1.28.0...v1.28.1

This is the first release of ONNX Runtime CUDA Plugin EP, providing CUDA execution as a separately packaged plugin execution provider.

These notes cover commits affecting CUDA Plugin EP core code, CMake integration, and its primary build and package pipeline.

Please refer to QUICK_START.md for the usage.

Highlights

Plugin Runtime

• Introduces the CUDA Plugin EP core and makes it the default CUDA provider implementation (#27816, #29544).
• Adds arena allocation, resource accounting, available-resource reporting, and IOBinding synchronization (#27931, #28028, #28103, #27919).
• Adds provider options for user compute streams, copy behavior, EP-level unified streams, and external allocators, with compatibility back to ONNX Runtime 1.24.4 through version-gated callbacks (#28603, #28824).

CUDA Graphs and Profiling

• Adds CUDA Graph capture and replay, including user compute stream support, synchronization controls, and a kernel sync stream for scratch allocation (#27958, #28002, #28686, #29221, #29244).
• Adds plugin profiling APIs and memory statistics in profiling output (#28216, #29058).

Operators and Dependencies

• Expands model coverage with NHWC hardening, cuDNN SDPA for MHA/GQA, quantized MoE kernels, block-quantized FP4 MatMul, and native NVFP4 QMoE prefill on SM120 (#28612, #28849, #28467, #29818, #29824).
• Adds cuDNN-free ArgMax, ArgMin, and ReduceSum and fixes LogSoftmax for plugin deployments (#29620).
• Reduces runtime dependencies by making cuDNN and cuFFT optional and removing the nvrtc link requirement (#29252, #29808, #29705).

Build and Packaging

• Adds dedicated build and packaging pipelines, the onnxruntime-ep-cuda Python wheel, and NuGet packages (#27959, #28152, #28299, #28378).
• Adds Windows ARM64 packages, package-size controls, and aligned package metadata and artifact naming (#28896, #31635).
• Improves CUDA build coverage and reliability across CUDA 12.8/13, SM90, and SM120 while reducing CI build time with per-architecture object libraries and quick-build support (#29711, #28736, #29776, #31608, #31609, #28645).

Contributors

Thank all contributors that contributed to the CUDA execution provider:

@abudup, @aciddelgado, @AdamLouly, @adamreeve, @adrianlizarraga, @Alexey-Kamenev, @amancini-N, @amathews-amd, @AndreyOrb, @apsonawane, @ashbhandare, @askhade, @baijumeswani, @BowenBao, @chenfucn, @chengchen666, @chilo-ms, @cloudhan, @cocotdf, @codemzs, @cschreib-ibex, @dakenf, @decahedron1, @derdeljan-msft, @duli2012, @e-tao, @edgchen1, @elwhyjay, @eserscor, @fajin-corp, @feihugis, @frank-dong-ms, @fs-eire, @fthielke, @gedoensmax, @georgthegreat, @gh-yewang, @gineshidalgo99, @GopalakrishnanN, @gramalingam, @guoyu-wang, @guoyuhong, @hadiFute, @hanbitmyths, @hariharans29, @harrysummer, @harshithapv, @HectorSVC, @hubertlu-tw, @jagadish-amd, @jambayk, @jchen351, @jeffdaily, @jessebenson, @jiafatom, @jiangzhuo, @jignparm, @jingyanwangms, @jixiongdeng, @johannes-rehm-snkeos, @jslhcl, @JTischbein, @justinchuby, @jywu-mysoft, @kailums, @ke1337, @khspear, @kit1980, @KnightYao, @kpkbandi, @krahenbuhl, @kunal-vaishnavi, @Lafi7e, @linkerzhang, @liqunfu, @liuziyue, @lukas-folle-snkeos, @mastryukov1990, @mc-nv, @micmelesse, @mika-fischer, @mindest, @mityada, @mszhanyi, @mtavenrath, @mwootton, @namgyu-youn, @naomiOvad, @natke, @neginraoof, @nenad1002, @oliviajain, @PatriceVignola, @PeixuanZuo, @pengwa, @petermcaughan, @poweiw, @pranavsharma, @praneshgo, @prathikr, @qti-jkilpatrick, @qti-kromero, @RandySheriffH, @RandyShuai, @raviskolli, @Rishi-Dave, @RyanUnderhill, @satyajandhyala, @selenayang888, @seungtaek94, @shaahji, @shahasad, @sheetalarkadam, @SherlockNoMad, @ShirasawaSama, @skottmckay, @skyline75489, @smk2007, @snnn, @sohaibiftikhar, @souptc, @stevenlix, @Stonesjtu, @suffiank, @thevishalagarwal, @thiagocrepaldi, @tiagoshibata, @tianleiwu, @titaiwangms, @Tixxx, @tpboudreau, @tracysh, @umangb-09, @viboga, @vraspar, @weixingzhang, @wejoncy, @wschin, @xadupre, @xiaofeihan1, @xiaomsft, @xinyazhang, @XiyinOSS, @xzhu1900, @yangchen-MS, @ybrnathan, @yf711, @yihonglyu, @yinli-systems, @ytaous, @yuanbyu, @yufenglee, @YUNQIUGUO, @yuslepukhin, @zhanghuanrong, @zhijxu-MS, @Zyrin

Announcements & Breaking Changes

• onnxruntime-web has announced the deprecation of WebGL and JSEP. The native WebGPU EP is the recommended path going forward. See the deprecation and migration plans for details (#29716, #31683).
• POSIX telemetry is now available on Linux, macOS, Android, and iOS when ONNX Runtime is built with telemetry enabled. It does not change the public ABI, WebAssembly remains telemetry-free, and setting ORT_DISABLE_TELEMETRY=1 before initialization disables non-Windows telemetry for the process (#27379, #29872).
• The unused internal onnxruntime/python/tools/tensorrt dashboard tooling was removed. This does not affect the TensorRT Execution Provider APIs (#29395).

Security Fixes

Path, bounds, and input validation

• Fixed a path traversal vulnerability in TensorRT and NvTensorRTRTX engine refitting by making external-data path validation unconditional (#29396).
• Validated the CPU MoE k attribute against the number of experts and fixed a CPU TensorScatter security issue (#29907, #29916).
• Added missing rank, shape, and parameter validation for pooling, LSTM and DynamicQuantizeLSTM, Sampling, FeatureVectorizer, SkipLayerNorm, QLinearConv, Whisper decoding, RNN activations, GridSample, contrib Range, and CropAndResize (#29254, #29255, #29265, #29579, #29595, #29605, #29871, #31636, #31671, #31675, #31676, #31684).
• Hardened CUDA indexing and buffer handling in GridSample, transpose, GatherBlockQuantized, InstanceNormalization, LayerNorm/RMSNorm, BeamSearch, DeformConv, AveragePool, and MaxPool (#29581, #29631, #29638, #31640, #31642, #31644, #31645, #31647, #31650).
• Fixed packed sub-byte tensor over-copying in OrtApi::GetValue and validated DML constant tensor byte sizes (#29157, #31665).

Supply chain and tooling

• Updated npm lockfiles, refreshed the Next.js end-to-end fixture lockfile for security advisories, and upgraded adm-zip for onnxruntime-node (#29827, #29926, #31192).

New Features

Core APIs & Runtime

• Default intra-op and inter-op thread-pool sizes can now be set with ORT_INTRA_OP_NUM_THREADS and ORT_INTER_OP_NUM_THREADS. Explicit thread settings still take precedence, and 0 preserves machine-sized defaults (#29688).
• Added weightless-model support for all initializer types, allowed zero-input EpContext nodes, and wired maximum-shape inference into workspace estimation (#29607, #29799, #31613).
• Added ONNX-domain support for rotary embedding and a fused MRotaryEmbedding contrib operator for Qwen mRoPE variants (#29261, #31728).
• Added multi-shape profiling to onnxruntime_perf_test through --data_shape, plus verbose graph-transformer tracing and broader inference-session error-path coverage (#29555, #29558, #29569, #29571).

Execution Provider ABI & Plugin EPs

• WebGPU now supports device-free compile-only sessions for offline graph transformation (#29681).
• Expanded CUDA plugin EP packaging and testing, including Windows ARM64 package and size options, updated package outputs, and aligned architecture selections across Python, C API, TensorRT, Node.js, and plugin packages (#31635, #31722, #31992).
• Improved plugin lifecycle handling by unloading failed EP library loads and fixing allocator-deleter lifetime (#29634, #29770).

Execution Provider Updates

NVIDIA CUDA EP

Attention and decoding

• Added PagedAttention with quantized KV cache, XQA decode, MLA, QK-Norm, and head-sink support (#29912).
• Extended quantized KV-cache support with attention sinks, independent and per-channel scales, sliding-window cache support, and a fused K/V dequantization launch (#29900, #29904, #31480).
• Added a cuDNN SDPA decode tier to the standard ONNX Attention CUDA kernel and enabled cuDNN SDPA for contrib Attention (#29715, #29717).
• Added attention_bias support to the GroupQueryAttention unfused path and state_window support to LinearAttention and CausalConvWithState for MTP (#29525, #31157).
• Fixed LinearAttention on GPUs with limited shared memory (#31982).

MoE and quantized GEMM

• Added NVFP4 QMoE, including native FP4xFP4 prefill on SM120, faster decode GEMV, fused routing/finalization paths, and reduced activation and weight-dequantization overhead (#29697, #29824, #29887, #29919, #31156, #31159, #31349, #31479).
• Added MatMulBlockQuantizedFp4Weight and MatMulBlockQuantizedFp8Weight, plus block-scaled tensor-core/GEMV decode paths, packed FP4 decode, M-tiling, and folded W8A8 activation QDQ (#29818, #29850, #29896, #31155, #31481).
• Improved MatMulNBits and QMoE robustness and efficiency by optimizing 8-bit dequantization, releasing raw MXFP4 initializers after prepack, and fixing subgraph prepacking and mixed FP8/FP4 build failures (#29852, #31141, #31154, #31350).

Operators and collectives

• Added LinearAttentionGate, GatedRMSNorm, and GatedAdd contrib operators (#31158, #31835).
• Added bfloat16 support to AllReduce, AllGather, and AllToAll (#31571).
• Fixed the default zero point in CUDA GatherBlockQuantized (#31693).

WebGPU EP

• Added DFT, HardSwish, Max/Min, Trilu, GRU, PRelu, MatMulBnb4, and MRotaryEmbedding support (#29454, #29587, #29828, #29833, #29840, #29845, #30512, #31976).
• Expanded integer support across Clip, Reshape, Cast, Add, Tile, Concat, Expand, Gather, CumSum, Max, and Min (#29830, #29834, #29835, #29839, #29844, #29847, #29854, #29861, #29897, #29918, #31049, #31702, #31709).
• Added the initial WebGPU PagedAttention implementation and moved Softmax and non-flash Attention to online algorithms (#29694, #29724, #31611).
• Improved MatMulNBits wide-tile accumulation precision (#29611).
• Added and extended Intel subgroup-matrix MatMul/Gemm kernels, including f16, batched-B, and odd-N support (#29592, #29749, #29813, #29893).
• Reduced cold-start and upload overhead with deferred dispatch and staging-buffer improvements; tuned FlashAttention, subgroup Gemm/MatMul, Split-K on Panther Lake, and Xe im2col-matmul (#29271, #29505, #29557, #29586, #29846, #30514).
• Upgraded Dawn and improved reliability by avoiding exceptions in Dawn callbacks, fixing a Linux adapter-failure self-deadlock, correcting Windows x86 transfer callbacks, and fixing TurboQuant batched sequence lengths (#29389, #29591, #29625, #29752, #31568).

WebNN EP

• Added uint8-packed 4-bit GatherBlockQuantized and LpNormalization, reused the shared WASM loader for Blob-backed external data, and fixed per-axis QDQ and MatMulNBits edge cases (#29475, #29801, #31151, #31152, #31197).

OpenVINO / QNN / DML / XNNPACK / TensorRT

• OpenVINO fixed float16 constant-output corruption and output-name routing, added dot-separated KV-cache names to the stateful transform, and corrected raw-data-backed float initializer handling (#29729, #29882, #29895, #31138).
• QNN added a reshape handler for split-axis reshapes (#29660).
• DML fixed wide-string handling and made fused graph kernels own their model paths (#31656, #31664).
• XNNPACK now reads dynamic Gemm M from the input tensor at compute time (#31189).
• TensorRT deduplicated context-path handling and added a build option for fused-attention cubins (#29640, #31632).

CPU & Core Optimizations

MLAS

• Added Arm64 half-precision GEMM and convolution support through KleidiAI, including FP16 MatMul/Gemm/Conv paths and asymmetric Q4 and SME2 MatMulNBits kernels (#28786, #29654, #29709, #29898).
• Added a RISC-V RVV QNBitGemm backend, an Arm64 NEON fp32 RoPE kernel, portable SVE elementwise kernels with FEXPA exp, and Arm64 UDOT routing for S8U8 QGEMM (#29537, #29787, #29836, #31145).
• Added AVX2/VNNI 2-bit weight kernels and vectorized 2-bit dequantization, and improved fp16 MatMulNBits paths by avoiding fp32 temporaries and writing fp16 output directly across 2-, 4-, and 8-bit paths (#29619, #29766, #29791, #29842, #29864, #29901).

CPU Attention & Kernels

• Improved masked Attention performance, enabled CPU FlashAttention on Linux Arm64 through L2-cache detection, and added FP16 GQA with quantized KV cache (#29621, #29719, #29825).
• Added double support to CPU Cos and int32 support to CPU Trilu, and fixed int8 QLinearSoftmax saturation and AvgPool ceil_mode/count_include_pad behavior (#28975, #29476, #29629, #29728).
• Fixed TfIdfVectorizer weight indexing and skipped MinLength logits-processor construction when eos_token_id is negative (#29604, #31649).
• Tightened K/V and cache-indirection shape contracts in CPU Attention and MultiHeadAttention, and fixed LinearAttention output shape inference for grouped-query attention (#29892, #31190, #31634).

Graph, Optimizer, and Runtime

• Extended reshape fusion, fixed double recursion in subgraph type/shape inference, and made constant-folding output deterministic (#29027, #29617, #29789).
• Fixed in-memory external initializer loading, memory-pattern allocation stream selection, and a leak in GetOverridableInitializerNames() (#29349, #29589, #29616).
• Reduced small MatMul batch allocations and redundant LUT initialization (#29085, #29690).
• Fixed static-initialization-order crashes when importing ONNX Runtime and reduced eager runtime initialization (#29880, #31964).
• Negative CPU Split axes now produce an error instead of being accepted (#31149).

Web & JavaScript

• Added on-demand loading of Blob-backed external data in JSPI builds (#29477).
• Fixed JSEP pooling output shape for ceil_mode, allowed DFT to ignore excess input data, and fixed a webpack/Terser release-build crash (#29627, #29680, #31652).

Build, Packaging & CI

• CUDA package architecture selections are now aligned across plugin EP, Python, C API, TensorRT, and Node.js pipelines. Windows arm64 is only available in CUDA plugin EP (#31992):

  OS	CUDA	CUDA architectures (all in -real form)
  Linux x64	12.8	60;70;75;80;86;89;90a;120a
  Linux x64	13.x	75;80;86;89;90a;120a
  Linux aarch64	13.x	89;90a;120a;121a
  Windows x64	12.8	61;75;86;89;120a
  Windows x64	13.x	75;80;86;89;120a
  Windows arm64	13.x	120a;121a

• Reduced CUDA compilation time and memory usage by splitting generated SM80 MoE, fpA_intB, and MatMulNBits translation units and adding two-level workspace estimation (#29614, #29699, #29811, #31834, #31837).

• Fixed CUDA 13 plugin and packaging builds on Windows, Linux, and Windows ARM64, including MSVC/TMA compatibility and CI memory limits (#31608, #31609, #31615, #31616, #31617, #31622, #31729, #31748).

• Fixed MLAS AVX2 builds on toolchains without AVX-VNNI assembler support, GCC 15 -Werror builds, and an MSVC C1001 issue in the W2 AVX-512-VNNI dispatch path (#28767, #29679, #29885).

• Improved Windows compatibility by skipping DXGI discovery when Win32k system calls are unavailable and delay-loading shell32 (#29755, #30889).

• Fixed Dawn parallel-build races, and GPU discovery in build/test environments (#29858, #29866).

Contributors

Thanks to our 63 contributors for this release!

@adrastogi, @ahsan-ca, @AngelGalindo7, @ankitm3k, @apsonawane, @blazingphoenix7, @bmehta001, @chilo-ms, @claude, @daijh, @ducviet00, @edgchen1, @elwhyjay, @eserscor, @GopalakrishnanN, @guptaishaan, @hariharans29, @Honry, @huningxin, @jchen10, @jiafatom, @jiangzhuo, @Jiawei-Shao, @JonathanC-ARM, @justinchuby, @kjg0724, @kunal-vaishnavi, @kylo5aby, @Laan33, @martin-klacer-arm, @mastryukov1990, @mcollinswisc, @miaobin, @mingmingtasd, @mirounga, @mustjab, @n1harika, @namgyu-youn, @neilmsft, @nenad1002, @nicholascelestin, @OscarFree, @prathikr, @qjia7, @quic-muchhsu, @Sammy-Dabbas, @sanaa-hamel-microsoft, @shiyi9801, @skottmckay, @tairenpiao, @TedThemistokleous, @the0cp, @tianleiwu, @titaiwangms, @velonica0, @wangw-1991, @wuisabel-gif, @xadupre, @xhcao, @xiaofeihan1, @xiaoyu-work, @yen-shi, @zlma7001

Full Changelog: v1.28.0...v1.29.0

Highlights

• Major performance work for attention-heavy LLMs.

  • FlashAttention decode kernels were fused and extended for any sequence length (#28389).
  • FlashAttention prefill shared-memory path was generalized (#28520).
  • Dynamic max_k_step was enabled for NVIDIA (#28511).
  • QKV bias support was added for FlashAttention in MultiHeadAttention (#28380).
  • M4 Max-specific FlashAttention optimization landed (#27780).

• Qwen3 and Gemma 4 model-path improvements.

  • QKV and MLP fusions for Qwen3-style models (#28280).
  • Q/K RMSNorm fusion into GroupQueryAttention for Qwen3-style models (#28484).
  • Opset 24 and KV-shared decoder layer support for Gemma 4 (#28501).
  • GroupQueryAttention now supports optional present-key/value outputs (#28242).

• LinearAttention and quantized-path optimizations.

  • LinearAttention subgroup optimizations and larger tile_v with subgroup support (#28412, #28519).
  • GatherBlockQuantized gained 2-bit support (#28530).

• Reliability and hardening fixes.

  • Fixes for out-of-bounds read risks in GatherBlockQuantized and Pad (#28718, #28721).
  • Fix for past_state == present_state buffer handling (#28753).
  • Fixes in QMoE numerical stability and SkipSimplifiedLayerNormalization bias behavior (#28434, #28427).
  • ConvTranspose weight shape validation improvement (#28524).

• Graph-capture and buffer-management improvements.

  • Per-graph buffer manager and session-level buffer pool updates for graph capture reuse (#28260, #28761).

Note: This section was AI-generated. It may have inaccuracies.

Contributors

Thanks to everyone who contributed to the WebGPU EP (human contributors, alphabetical):

@apsonawane, @daijh, @edgchen1, @feich-ms, @GopalakrishnanN, @guschmue, @hariharans29, @HectorSVC, @jchen10, @qjia7, @tianleiwu, @xiaofeihan1, @xenova, @yuslepukhin.

Note: This list was compiled on a best-effort basis from PRs that touched WebGPU EP-specific paths and intentionally includes human contributors only, so it may not capture every contribution. If yours was missed, the omission is unintentional. Your work is no less appreciated.

Announcements & Breaking Changes

• Upgraded to ONNX 1.22.0 and protobuf 6.33.5 (#28754, #29606, #28967). Graph optimizer opset version checks were updated accordingly (#28966).
• cuDNN and cuFFT are now optional at runtime for the CUDA EP, and nvrtc is no longer linked, which significantly reduces the required CUDA redistributable footprint (#29252, #29808, #29705, #29620).
• An experimental C/C++ API surface was introduced. OrtModelPackageApi now lives in the experimental C API and may change in future releases (#28746, #29142, #28990).
• Deprecated / removed:
  • SkipLayerNorm strict mode is deprecated (#29388).
  • The TensorRT fused causal attention kernels were removed from the CUDA EP (#29143).
  • The dynamic WGSL generator (duktape/Node) path was removed in favor of the Python wgsl-gen implementation (#29141, #28355).
  • CUDA_QUANT_PREPROCESS is off by default (#29687).
• NPM packages are now published from the CUDA 13 pipeline (#28773).
• The CUDA 12.8 package architecture list was refreshed for this release (#29711).

Security Fixes

Memory safety & input validation

• Hardened the ORT FlatBuffer model loader against malformed buffers, and removed now-redundant table offset validation (#28186, #29068)
• Fixed type confusion in raw-pointer bind_input causing an out-of-bounds write (#28839)
• Fixed out-of-bounds pointer in TensorAt for sub-byte packed types (#28973)
• Fixed arbitrary memory read, out-of-bounds dereference, and other OOB accesses in kernels (#28991, #29011, #29012, #29014)
• Validated Col2Im inputs to prevent heap over-read (#28706)
• Hardened CropAndResize against malformed crop_size tensors (#28766)
• Validated BeamSearch vocab_size against logits width (#28774)
• Fixed bounds in WhisperDecoderSubgraph::CreateInitialFeeds (#29239)
• Validated SparseAttention CSR indices/key lengths and rejected zero-dimension block_row_indices (#29015, #29242)
• Clamped derived sequence lengths and KV-cache index in CUDA GroupQueryAttention, and fixed a CPU GQA out-of-bounds read in the past-KV buffer (#29240, #29447)
• Clamped 1D attention mask_index to valid bounds (#29449)
• Validated MaxpoolWithMask kernel rank against input spatial rank (#29253)
• Rejected CUDA BERT EmbedLayerNorm/SkipLayerNorm shapes exceeding 32-bit output indexing (#29264)
• Fixed the optional-output guard in DecoderAttention/MultiHeadAttention shape inference and negative-axis handling in ExpandDims shape inference (#29268, #29448)
• Fixed TreeEnsemble target id validation and added input validation to LinearClassifier (#29293, #29060)
• Fixed DynamicQuantizeLSTM zero-point/scale validation typos (#29462)
• Handled non-trivially-copyable types in Loop/Scan output concatenation (#29397)
• Normalized bool tensor raw_data to {0, 1} on unpack (#29238)
• Addressed hardening gaps in Resize, PadFusion, and LoRA handling (#28779, #28780, #28801)
• Fixed unbounded lifetime on WithOutputTensor in the Rust bindings (#29251)

Integer overflow & allocation size

• Guarded MlasConvPrepare working-buffer products and ConvTranspose pad computation with SafeInt (#29444, #29446)
• Fixed signed-int overflow in SamplingState::Init that could cause a heap buffer overflow (#29443)
• Hardened QMoE against integer overflow and partial K tiles (#29067)
• Validated B/scales/zero-points shape in MatMulNBits::PrePack (#29445)
• Pre-checked ConstantOfShape output size against the input initializer before constant folding (#28751)
• Fixed integer overflow in RKNPU implicit bias allocation (#29249)
• Fixed WebGPU out-of-bounds reads in Pad (int64/int32 truncation), Slice, and GatherBlockQuantized (#28721, #28704, #28718)

Supply chain & tooling

• Updated protobuf to mitigate CVE-2026-0994 and bumped ONNX/protobuf to fix additional CVEs (#28967, #29606)
• Avoided shell injection in the training helper and switched Triton compile helpers to subprocess (#28776, #28775)
• Validated archive extraction paths in the transformers tooling (#28777)
• Validated and inlined external data in node tensor attributes during session initialization (#29250)
• Enabled Spectre-mitigated MSVC libraries for BinSkim builds (#29624)
• Bumped npm dependencies: shell-quote, esbuild, tmp, ws, protobufjs, js-yaml, tar, markdown-it, @babel/core (#29022, #29044, #29055, #29057, #29061, #29062, #29063, #29079, #29090, #29156)

New Features

Execution Provider ABI & Plugin EPs

• Model Package support Phase 2, plus authoring tools, schema versioning, and folding external_data into session options (#28271, #28989, #29501)
• Added an API to select the best compiled-model compatibility info from candidate strings (#28387)
• Added crypto support: applications can supply I/O callbacks to an EP, with callback and fallback helpers (#28624)
• Implemented name-based partitioning with accompanying documentation (#28903)
• Added Linux NPU discovery through sysfs accel devices (#28703)
• Relaxed CompileModel validation to accept zero-input OrtModel graphs (#28771)
• CUDA plugin EP: user compute stream with CUDA graph, kernel sync stream exposed for scratch allocation, and Windows ARM64 packages (#29221, #29244, #28896, #28789)
• WebGPU plugin EP version bumped to 0.3.0 (#29056)

Core APIs & Runtime

• Added OrtErrorCode documentation, single-sourced the values so StatusCode stays in sync, and added OrtErrorCode::ORT_DEVICE_RESET (#29018, #29065, #29748)
• Added memory statistics to profiling output (#29058)
• Added EP version logging on inference failure, in the EpDeviceUsage event, and ORT version logging (#28794)
• User-supplied external initializers are now used in place when already on the planned device (#29013)
• model_external_initializers_file_folder_path is now honored for file-path model loads (#29459)
• Added a Python API for HOST_ACCESSIBLE OrtValue allocation (#28038)

Quantization Tooling

• Added CudaQuantizer to onnxruntime.quantization (#29509)
• Registered Flatten as a Direct8Bit op in the Python QDQ static quantizer (#28340)
• Skipped MaxPool during FP8 static quantization and fixed the FP8 (FLOAT8E4M3FN) scale reference distribution (#28488, #29350)
• Added Float16/BFloat16/Float8 support in the TensorArray custom op (#28335)
• Clarified CPU parameter recommendations in the quantization docs (#28415)

Execution Provider Updates

NVIDIA CUDA EP

Attention & LLM decode

• Enabled XQA by default for FP16/BF16 GroupQueryAttention, and extended XQA decode with attention sink, sliding window, and QK-Norm support (#29046, #29162, #29177, #29186)
• Upgraded cudnn_frontend to 1.24 and enabled cuDNN SDPA for MHA/GQA (#28849)
• Added decode-optimized LinearAttention (GatedDeltaNet) kernels (#28985)
• Optimized FlashDecode split planning for local-window GQA and fixed Flash/Lean attention split heuristics (#29161, #29554)
• Updated the GroupQueryAttention contrib op documentation (#29173)

MoE & quantized GEMM

• Prepacked int4/int8 QMoE expert weights in the PrePack hook, symmetric with MatMulNBits, and fixed the prepack to always use the SM80 layout (#28749, #28978, #28965)
• Optimized the QMoE SoftmaxTopK router for small-batch decode, added a Split-K2 SwiGLU GEMV kernel, defaulted to fp16 accumulation for fp16 activations, and fused the MoE router bias into the MatMulNBits GEMV (#28980, #29167, #29166, #29170)
• Added a batched small-M GEMV path for 4-bit/8-bit MatMulNBits (#29451)
• Enabled fpA_intB GEMM in CUDA builds with configurable options, added prepacked fpA_intB weight support, and enabled native SM90, block_size=32, and fused bias (#29622, #29499, #29585)
• Fixed a QMoE profiler cross-stream race and made it CUDA-graph-capture safe (#29584)

Coverage, dependencies & fixes

• Added cuDNN-free ArgMax/ArgMin/ReduceSum and fixed LogSoftmax on the plugin EP (#29620)
• Filled the CUDA opset gap for Softplus and Softsign up to opset 22 (#28982)
• Fixed the XQA GroupQueryAttention cudaErrorInvalidValue on Blackwell (sm_120) (#29706)
• Initialized large-head FMHA shared memory per kernel variant (#29140)
• Fixed CUDA/cuDNN DLL preload paths for the CUDA 13 consolidated wheel layout, and fixed the libcudart.so.13 hard dependency that broke import on CPU-only Linux (#29202, #29590)
• Fixed a null allocator passed to plugin EP kernel PrePack, and plugin EP allocator deleter lifetime (#29658, #29663)
• Removed unused code in moe_kernels.cu (#29295)

WebGPU EP

• Added KV cache quantization for ORT WebGPU (#28059)
• Fused FlashAttention decode kernels and extended them to any sequence length (#28389)
• Fused Q/K RMSNorm into GroupQueryAttention for Qwen3-style models (#28484)
• Gemma 4 enablement: indirect dispatch for FlashAttention graph capture, and INT64 support for Equal/Sub/Where/ReduceSum under the enable_int64 flag (#29236, #29392)
• Added a session-level buffer pool for graph capture reuse and made max_num_pending_dispatches configurable (#28761, #28894)
• Added 2-bit quantization support to GatherBlockQuantized (#29054)
• Enabled Cast to int64 by default and switched to naive reduction (#28804, #28174)
• Default non-debug validation mode is now disabled (#28679)
• Tuned parameters for the gemm-subgroup kernel (#28760)
• Enabled FlashAttention for batched GQA with right-padded prompts and fixed batched right-padded prefill with do_rotary (#29247, #29002)
• Fixed a GQA decode split-reduce head_size out-of-bounds race, the past_state == present_state buffer aliasing case, and GatherBlockQuantized dispatch failure for empty indices (#29593, #28753, #29030)
• Fixed nearest half-tie handling for round_prefer_ceil/round_prefer_floor (#28757)
• Skipped MatMulNBitsMlpFusion when the kernel is unavailable (#29089)
• Fixed profiling timestamp alignment with the ORT profiler (#29021)

CoreML EP

• Added Where and And builders (#28597)
• Fixed extended minimal builds with the Core ML EP (#29387)
• Fixed the "model_path" must not be empty error (#29394)
• Fixed CoreML-enabled static builds on macOS (#27960)

QNN EP

• Fixed a segfault in PadNodeGroupSelector::Check when dq_nodes is empty (#28733)
• Added a Gather handler for the transpose optimizer (#28755)

WebNN EP

• Added support for 8-bit MatMulNBits (#29025)
• Removed unnecessary casts around normalization ops (#29024)

OpenVINO / VitisAI / DML / XNNPACK / VSINPU

• OpenVINO EP development updates (#28954)
• VitisAI: recognize native-endian in-memory address tag in process_ext_address (#29248)
• DML: fixed batched readback size (#28680)
• XNNPACK: fixed Clip/Relu fusion when the producer output has other consumers (#28814)
• VSINPU: fixed EP build error (#29440)

CPU & Core Optimizations

MLAS

• Added AVX512 (+VNNI) 2-bit weight CPU kernels (#29064)
• Added ARM64 2-bit weight CPU kernels (#29466)
• Extended the NHWC-sans-transposes path to support depthwise convolutions, and fixed NHWC conv support gating (#28565, #29127)
• Added RISC-V RVV INT8 GEMM/GEMV, M=1 routing, and activation kernels (#28308)
• Fixed a fastmath corner case in sbgemm_neon_kernel and fixed failing KleidiAI NHWC unit tests (#28394, #29010)
• Hinted GCC >= 12 on aarch64 for bf16/fp16 capability probes (#28785)

CPU Attention & Kernels

• Added flash attention for non-quantized CPU GroupQueryAttention, plus a FP32 GEMV decode kernel (#28962, #29216)
• Fixed packed-QKV and broadcast-head bias strides in quantized GQA flash attention (#28963)
• Enabled pre-packed weight sharing for CPU MatMulNBits (#29163)
• Added int8/uint8 CPU support for SpaceToDepth and int8 for DepthToSpace (#29154)
• Fixed CPU Attention causal mask alignment and is_causal bottom-right alignment for external KV cache (#29050, #28958)
• Fixed CPU GQA NaN output for right-padded batched prompts with rotary embeddings (#29069)
• Fixed ReverseSequence returning zeros for zero-length sequences (#28759)
• Fixed a regression in RandomForestClassifier binary classification predictions (#28685)

Graph, Optimizer, and Fusion

• Fixed NHWC second-pass EP assignment cleanup (#27858)
• Rejected QDQ Gemm → QGemm fusion when alpha != 1 with bias, and validated DQ scale/zero-point shapes before QGemm fusion (#28131, #28714)
• Fixed incorrect output for QDQ Resize with non-nearest interpolation modes under ORT_ENABLE_ALL (#28454)
• Fixed INT64 initializer data propagation and the Shape → Gather → TopK rank-1 regression (#28778, #29084)
• Fixed a TransposeOptimizer type error on zero-point-less DequantizeLinear (#29192)
• Fixed SimplifiedLayerNorm fusion with a node-produced Pow exponent (#29196)
• Fixed DropQDQ not dropping Q/DQ around opset-22 MaxPool (#29201)
• Recovered Conv/ConvTranspose rank from weights when the input shape is unknown (#29149)
• Fixed STFT complex input frame offsets (#28961)

Web & JavaScript

• Forwarded WebGPU EP buffer cache mode options from JS (#29017)
• Added a Python implementation for wgsl-gen and removed the dynamic WGSL generator path (#28355, #29141)
• Switched NPM publishing to consume from the CUDA 13 pipeline, and fixed the Node.js packaging stage to use CFS (#28773, #29006)
• Fixed a Dawn parallel build race condition in the wasm_Release / build-wasm CI (#29040)

Language Bindings

Python

• Added an API for HOST_ACCESSIBLE OrtValue allocation (#28038)
• Made the TF converter tolerate a missing requests package (#28825)

Java

• Released outputHandlesArr in the OrtTrainingSession.evalStep JNI binding (#29576)

Rust

• Fixed unbounded lifetime on WithOutputTensor (#29251)

Build, Packaging & CI

• Updated the cpuinfo dependency and integrated cpuinfo_deinitialize() (#28245)
• Built with abseil 20250814 under NVCC (#28586)
• Fixed the Linux CUDA 13.3 build (abseil + CCCL parse errors) and the Windows CUDA 13.3 build for VS 2026 (#29042, #29266)
• Fixed the Windows build for SM90 and later, and CUDA packaging build errors (#29776, #29580)
• Fixed the wrong ORT version in Windows DLL VersionInfo for pipeline builds (#29545)
• Fixed Python and CUDA packaging pipeline failures (#29200, #29241)
• Installed Java in the Windows NuGet packaging job and added Java setup to the Windows x64 QNN CI pipeline (#29774, #29754)
• Sharded Windows ASan test runs and fixed ASan OOM in QDQ Gemm transformer tests (#29503, #28797)
• Propagated and fixed PIP_INDEX_URL piping for container build stages (#29803, #29823)
• Removed deprecated docfx in favor of a newer docs validation method (#29812)
• Fixed a build failure from a direct ONNX schema include in layernorm optimizer tests (#29073)
• Test stability: fixed a data race in CUDA graph tests, a non-deterministic CUDA custom-op test, a thread-pool parameter self-assignment, duplicate schema registration, and incorrect string tensor creation (#29398, #29082, #28799, #29219, #29218)
• Encapsulated the global random seed in a function and improved LinearRegressor test coverage (#28979, #29083)

Contributors

Thanks to our 103 contributors for this release!

@adrastogi, @ankitm3k, @apsonawane, @ArsalanShakil, @bachelor-dou, @bopeng1234, @cbourjau, @chilo-ms, @chunghow-qti, @chwarr, @claude, @Craigacp, @crvineeth97, @dabhattimsft, @daijh, @danielsongmicrosoft, @derdeljan-msft, @edgchen1, @elwhyjay, @ericcraw, @eserscor, @fanchenkong1, @feich-ms, @fs-eire, @FuZoe, @gaugarg-nv, @gblong1, @GopalakrishnanN, @grybouilli, @guschmue, @haoxli, @hariharans29, @hnsyprst, @Honry, @intbf, @ishwar-raut1, @jambayk, @Jaswanth51, @jatinwadhwa921, @javier-intel, @jiafatom, @Jiawei-Shao, @JonathanC-ARM, @justinchuby, @jwludzik, @Kotomi-Du, @kpkbandi, @lucka-me, @maoger, @martin-klacer-arm, @maxwbuckley, @MayureshV1, @mc-nv, @mcollinswisc, @mdvoretc-intel, @mingyueliuh, @mklimenk, @n1harika, @namgyu-youn, @neilmsft, @orlmon01, @Osamaali313, @prathikr, @preetha-intel, @psakhamoori, @qiurui144, @qjia7, @qti-hungjuiw, @qti-yuduo, @quic-muchhsu, @raagrawal, @RajeevSekar, @Reranko05, @Rishi-Dave, @RyanMetcalfeInt8, @Sammy-Dabbas, @sanaa-hamel-microsoft, @sayanshaw24, @selenayang888, @sfatimar, @sgbihu, @Shivani767, @ssam18, @susbhere, @sushraja-msft, @tairenpiao, @the0cp, @tianleiwu, @titaiwangms, @umireon, @vthaniel, @wangw-1991, @wenqinI, @xadupre, @xenova, @xhcao, @xiaofeihan1, @xieofxie, @xuke537, @yen-shi, @yinli-systems, @yuslepukhin, @ZackyLake

Full Changelog: v1.27.1...v1.28.0

This is a patch release on top of v1.27.0, containing targeted bug fixes, a CUDA QMoE decode-path optimization, and CI/build infrastructure fixes.

Bug Fixes

• [MLAS] Fixed an igemm regression in the KleidiAI path (#28571)
• Fixed a QMoE CPU livelock by eliminating nested intra-op parallelism (#29081)
• Fixed a regression in graph-capture session initialization that rejected an empty graph (#29457)
• Fixed CustomOp forward compatibility by capping the version instead of rejecting it (#29574)

Performance

NVIDIA CUDA EP

• Added a QMoE GEMV fast path for batch-1 decode (#29038)

CI & Build Infrastructure

• Fixed an incorrect identity for azcopy (#29274)
• Fixed a brew install applesimutils failure by trusting the wix/brew tap (#29450)
• Upgraded to Xcode 26 (#29468)
• Stopped echoing the command when setting a VSO variable in mac-cpu-packing-jobs.yml (#29575)
• Fixed the web e2e (npm/vite) and Python DML CI pipelines (#29609)

Contributors

Thanks to our 8 contributors for this release!

@tianleiwu, @chilo-ms, @edgchen1, @adrastogi, @damdoo01-arm, @JonathanC-ARM, @martin-klacer-arm, @sanaa-hamel-microsoft

Full Changelog: v1.27.0...v1.27.1

n.b. This release is targeting ONNX 1.21. ONNX 1.22 will be supported in ORT 1.28. n.b. This changelog was generated via LLM. Only the contributor list has been verified. As always, only trust the commit history.

Announcements & Breaking Changes

• CUDA 12 package files are now explicitly named as such.
• CUDA 12 packages are deprecated, please move to CUDA 13 ASAP.

---

Security Fixes

• Fixed out-of-bounds read in SoftmaxCrossEntropyLoss via label bounds validation (#28004)
• Hardened OneHot input validation and output-size computation (#28014)
• Added SafeInt overflow protection in Expand and capped constant-folding output sizes (#28055)
• Bounded total output allocation size in Tile kernel (#28070)
• Added mask/input shape consistency checks in MaxpoolWithMask::Compute (#28223)
• Fixed BitShift UB for shift amounts greater than or equal to bit width (#28272)
• Validated sequence bounds in GQA (seqlens_k vs cos_cache) (#28277)
• Validated conv bias shape in WordConvEmbedding to prevent OOB reads (#28279)
• Fixed int32 overflow in CUDA Cast and UnaryElementWise kernels for very large tensors (#28386)
• Fixed out-of-bounds read in CropBase scale handling (#28399)
• Fixed rank-underflow bug in Inverse kernel trailing-dimension indexing (#28400)
• Added sparse tensor external file path validation and additional external-path hardening (#28408, #28709, #28725)
• Switched remaining torch.load() calls to weights_only=True (#28421)
• Added CPU cache-indirection beam-index validation (#28486)
• Added additional overflow/bounds checks and test coverage in runtime buffers (#28713, #28747)

---

New Features

Execution Provider Plugin API

• Added zero-copy I/O for plugin EPs with HOST_ACCESSIBLE memory (#28037)
• Added OrtEp::OnSessionInitializationEnd() callback (#28319)
• Added plugin EP session-options getters (#28377)
• Added CUDA Plugin EP provider options for streams and external allocators (#28603)

Core APIs & Runtime

• Added support for ONNX overloaded functions (IR v10+) (#28275)
• Added FLOAT8E8M0 datatype support in ONNX Runtime (#28381)
• Added CPU Cast support for FLOAT8E8M0 (#28435)
• Added kOrtEpDevice_EpMetadataKey_OSDriverVersion example and docs (#28282)

Quantization & Training Tooling

• Added calibration cache support to quantize_static (#28221)
• Added ActivationRestrictedAsymmetric quantization option (#28237)
• Added opset-21 block_size attribute support to QDQ quantization (#28522)
• Added CPU fallback for FusedAdam optimizer in ORT Training (#28233)

---

Execution Provider Updates

NVIDIA CUDA EP

• Added ConvTranspose-22 support (#27710)
• Filled CUDA opset gaps for LSTM, RNN, Reshape, Cast, Round/Equal, ReduceMax/ReduceMin, Sin/Cos, and Random* ops (#27737, #27743, #27742, #27744, #27754, #27755, #27756, #27759)
• Added LpNormalization support for CUDA EP (#28724)
• Added chunked dequant+GEMM for MatMulNBits to reduce peak GPU memory (#28712)
• Added QMoE tests for standard swiglu and improved decode-path routing/softmax kernels (#28741, #29026)
• Fixed CUDA Attention dispatch mismatch for GQA head-size cases (#28358)
• Fixed CUTLASS FMHA bias-loader alignment on unaligned kernel path (#28369)

WebGPU EP

• Added LSTM support on WebGPU (#27881)
• Added per-graph buffer manager for multi-graph capture (#28260)
• Added QKV and MLP layer fusions for Qwen3-style models (#28280)
• Added QKV bias support in FlashAttention for MultiHeadAttention (#28380)
• Added shader dump-to-file environment variable and nightly validation checks (#28674)
• Added opset-24 + KV-shared decoder support (Gemma 4) (#28501)
• Performance improvements: FlashAttention M4 Max optimization and subgroup-based LinearAttention tuning (#27780, #28412, #28519, #28520)
• Fixed numerical and correctness issues in QMoE, LayerNorm/SkipLayerNorm, and MatMul bias indexing (#28427, #28434, #28475)

CoreML EP

• Added support for pre-opset-13 Split (split attribute path) and scalar Gather indices (#28270, #28278)
• Added FusedConv, Identity, Ceil, Tile, Cast(bool), Sin, Cos, and GatherND support (#28289, #28293, #28595, #28596, #28598)

TensorRT / DML / QNN

• Improved TensorRT RTX compatibility (multi-GPU tests, API guards, and subgraph fixes) (#27837, #28361, #28611, #28503)
• Added diagnostics for DML failure paths (#28495)
• Updated QNN ETW log level rule handling (#27593)

---

Web & JavaScript

JavaScript / Node.js

• Updated JavaScript dependencies (next, postcss, tmp, qs, body-parser, and other npm packages) (#27705, #27894, #28304, #28547, #28644, #28683, [#28694]

---

CPU & Core Optimizations

MLAS / Quantization / Attention

• Added NHWC convolution path in MLAS to reduce transpose overhead (#26834)
• Added CPU QMoE 2-bit support and LUT GEMM fast path (#28185)
• Added quantized KV-cache support for CPU GroupQueryAttention with SIMD optimizations and tiled compute (#28576, #28578, #28606, #28695)
• Added RVV-optimized NCHWc convolution/pooling and LLM operators for RISC-V (#28411, #28518)
• Parallelized CPU ScatterElements and optimized MatMulNBits 2-bit float-zero-point path (#28588, #28589)
• Added optional bias support in MatMulNBits CPU LUT GEMM path (#28742)

Graph, Optimizer, and Fusion

• Added DiT attention fusion for F5-TTS and diffusion transformer models (#27999)
• Improved ONNX Attention dispatch path and removed legacy unfused MHA path (#27992)
• Added support for optional present_key/present_value outputs in GQA and Gemma4 support (#28242)
• Added implicit-input handling for partitioning/fusion around control-flow nodes (#28608, #28690)

---

Language Bindings

Python

• Fixed runtime-unresolvable type annotations in Session and InferenceSession (#27802)
• Made sympy an optional runtime dependency (#28141)
• Added PEP 561 py.typed marker to the onnxruntime package (#28438)

C#

• Added EP tests for CUDA Plugin EP (#28375)

---

Bug Fixes

Critical & Correctness Fixes

• Fixed session use-after-free when UserLoggingFunction is used (#28314)
• Prevented double-free in OrtModelEditorApi ownership transfer (#28123)
• Fixed plugin EP provider-library load refcount leak and added regression test (#28396, #28430)
• Fixed dangling pointer from temporary return value (#28419)
• Fixed PRelu returning NaN for infinite inputs on CPU EP (#28750)
• Fixed CPU Attention softcap/attn_mask ordering and added CUDA spec coverage consolidation (#28379)
• Fixed Reshape allowzero=1 handling for chained zero-size tensors (#28455)
• Fixed CPU QLinearConv per-channel weight zero-point handling for distinct values (#28456)
• Fixed Unicode-path handling issues on Windows and AppContainer path canonicalization (#28390, #28509)

Portability Fixes

• Build and portability fixes across AIX, GCC-15, S390x, libc++/Clang, FreeBSD, and cross-compilation flows (#26704, #27191, #28016, #28049, #28074, #28362, #28567, #28507)
• Fixed CPUIDInfo bounds handling for unknown ARM vendors (#28344)
• Fixed Resize nearest-mode rounding bug for negative halfway values (#28345)

---

Contributors

Thanks to our 106 contributors for this release!

@adrastogi, @adrianlizarraga, @AIFrameworksIntegration, @AlekseiNikiforovIBM, @angelser, @angelserMS, @ankitm3k, @anzzraju1997-glitch, @apsonawane, @arajendra, @ayappanec, @bachelor-dou, @badranX, @baijumeswani, @BoarQing, @bopeng1234, @bsosnader, @cbourjau, @chilo-ms, @chunghow-qti, @chwarr, @Craigacp, @daijh, @derdeljan-msft, @dparikh79, @edgchen1, @elwhyjay, @ericcraw, @eserscor, @feich-ms, @fs-eire, @gaugarg-nv, @gblong1, @GopalakrishnanN, @gramalingam, @guschmue, @hariharans29, @HectorSVC, @intbf, @ishwar-raut1, @Jaswanth51, @jatinwadhwa921, @javier-intel, @jchen10, @jiafatom, @Jiawei-Shao, @jnagi-intel, @jocelyn-stericker, @JonathanC-ARM, @justinchuby, @jwludzik, @kevinch-nv, @Kotomi-Du, @kpkbandi, @KV2773, @Laan33, @lhrios, @maxwbuckley, @MayureshV1, @mdvoretc-intel, @mingyueliuh, @mklimenk, @mustjab, @n1harika, @nazanin-beheshti, @orlmon01, @prathikr, @preetha-intel, @psakhamoori, @qiurui144, @qjia7, @qti-ashwshan, @qti-hungjuiw, @qti-yuduo, @rajatmonga, @RajeevSekar, @Rishi-Dave, @rvandermeulen, @RyanMetcalfeInt8, @SamuelLess, @sanaa-hamel-microsoft, @sfatimar, @sgbihu, @shiyi9801, @simonbyrne, @skottmckay, @susbhere, @sushraja-msft, @tairenpiao, @TejalKhade28, @theHamsta, @tianleiwu, @titaiwangms, @umangb-09, @velonica0, @vraspar, @vthaniel, @wenqinI, @xadupre, @xenova, @xhan65, @xiaofeihan1, @xieofxie, @yuslepukhin, @ZackyLake, @zejianzhang1982, @zz002

Full Changelog: https://github.com/microsoft/onnxruntime/compare/v0.1.4...v1.27.0

We're excited to ship the first release of the WebGPU Execution Provider as a plugin EP for ONNX Runtime. Instead of being baked into the core onnxruntime binary, the WebGPU EP is now distributed as a standalone artifact that registers with an existing ONNX Runtime installation at runtime.

Highlights

• Broad operator coverage on WebGPU. Native WebGPU kernels for the operators needed by common transformer, vision, and generative workloads — including Conv variants, MatMul/Gemm, normalizations, attention (Attention, MultiHeadAttention, GroupQueryAttention), rotary embeddings, quantized matmul, quantized Mixture-of-Experts (QMoE), and more. See the Operator coverage section below for a summary.
• Quantized & accelerated kernels. DP4A and subgroup-matrix MatMulNBits, a FlashAttention kernel, and vendor-optimized Intel MatMul/Gemm paths. See the Performance features section below.
• Plugin EP packaging. WebGPU support now ships as a separate, independently versioned library (onnxruntime_providers_webgpu) that plugs into a compatible ONNX Runtime (1.24.4 or newer) at runtime. Users can adopt WebGPU acceleration without switching their core ORT package, and the EP can iterate on its own cadence.
• Cross-platform native binaries for Windows x64/arm64 (bundled with dxil.dll / dxcompiler.dll), Linux x64, and macOS arm64.
• Language packages.
  • Python: onnxruntime-ep-webgpu wheel, installed alongside the onnxruntime package, registered via onnxruntime.register_execution_provider_library(...). See package page for details on installation and usage.
  • .NET: Microsoft.ML.OnnxRuntime.EP.WebGpu NuGet package, referenced alongside Microsoft.ML.OnnxRuntime, registered via OrtEnv.RegisterExecutionProviderLibrary(...). See package page for details on installation and usage.

Operator coverage

The WebGPU EP registers kernels for the majority of ONNX standard-domain operators used by mainstream model architectures, plus a curated set of com.microsoft contrib operators. Highlights by category:

• Math, normalization & reduction: MatMul, Gemm, Softmax, LayerNormalization, RMSNormalization, InstanceNormalization, BatchNormalization, LpNormalization, unary/binary elementwise ops, all standard reductions (ReduceMean, ReduceSum, ReduceMax, ...), CumSum, Einsum, TopK, ArgMax/ArgMin.
• Neural network: Conv, ConvTranspose, MaxPool/AveragePool (and Global* variants), plus a FusedConv contrib op.
• Tensor manipulation: Transpose, Reshape, Slice, Concat, Split, Gather/GatherElements/GatherND, ScatterElements/ScatterND, Pad, Tile, Cast, Resize, GridSample, Where, Flatten, Squeeze, Identity, Shape, and more.
• Transformer / LLM contrib ops: Attention, MultiHeadAttention, GroupQueryAttention, RotaryEmbedding, SkipLayerNormalization, SkipSimplifiedLayerNormalization, SimplifiedLayerNormalization, BiasAdd, BiasGelu, BiasSplitGelu, FastGelu, Gelu, QuickGelu, CausalConvWithState, LinearAttention.
• Quantization: DequantizeLinear, MatMulNBits (with DP4A and subgroup-matrix paths), GatherBlockQuantized, QMoE.

For the authoritative list, see the kernel registrations in webgpu_execution_provider.cc and webgpu_contrib_kernels.cc.

Performance features

• DP4A and subgroup-matrix MatMulNBits paths for accelerated quantized matmul on supported hardware.
• FlashAttention kernel for attention-heavy workloads.
• Intel-optimized MatMul/Gemm code paths for improved performance on Intel GPUs.
• Program caching to amortize shader compilation costs across runs.
• Optional PIX frame capture and WebGPU profiler integration for performance investigation.

Known limitations

• Platform support in this release is limited to the platforms listed above (no mobile, no Linux arm64, no macOS x64).

Acknowledgments

This initial release is the result of contributions from engineers at Microsoft, Intel, and the broader community. Thank you to everyone who built, reviewed, and tested the WebGPU plugin EP — including (in alphabetical order):

@aciddelgado, @adrastogi, @adrianlizarraga, @chilo-ms, @daijh, @derdeljan-msft, @edgchen1, @eserscor, @feich-ms, @fs-eire, @guschmue, @HectorSVC, @ingyukoh, @jchen10, @jiangzhaoming, @Jiawei-Shao, @jing-bao, @justinchuby, @kunal-vaishnavi, @mindest, @prathikr, @qjia7, @satyajandhyala, @shaoboyan091, @sheetalarkadam, @skottmckay, @snnn, @sushraja-msft, @tianleiwu, @titaiwangms, @TomCrypto, @vraspar, @wenqinI, @xenova, @xhcao, @xiaofeihan1, @yuslepukhin.

Special thanks to the Intel team for the vendor-optimized MatMul/Gemm kernels.

Note: This list was compiled on a best-effort basis from PRs that touched WebGPU EP-specific paths, so it may not capture every contribution. If yours was missed, the omission is unintentional — your work is no less appreciated.

n.b. The following was generated via LLM from Git history. Only the contributor list has been verified.

Announcement - Breaking Changes

• Support for CUDA 12 will be removed in 1.27.0.
  • CUDA 13 will continue to be published as onnxruntime-<os>-<arch>-gpu_cuda13-<version>.<ext>
• CUDA runtime will be moving soon to a dedicated Execution Provider (EP) instead of a published package from ORT core.

Highlights

• Added optional memory mapping for .ort model loads (#28164).
• Added RISC-V Vector (RVV) support for CPU EP (#28261).
• OpenVINO EP upgraded for 1.26.0 development release (#28297).
• WebGPU gained GridSample support (#28264) and Split-K improvements (#28151).
• CUDA plugin EP gained graph support (#28002), profiling API (#28216).

Security and Reliability Hardening

• Replaced unrestricted Python setattr configuration with an allowlist (#28083).
• Hardened multiple OOB and overflow scenarios across ML and core ops:
  • Attention mask index OOB write (#27789).
  • MaxPoolGrad indices bounds validation (#27903).
  • SVM and TreeEnsemble bounds/security fixes (#27950, #27951, #27952, #27989).
  • RNN sequence_lens OOB read and integer overflow handling (#28052, #28003).
  • GroupQueryAttention seqlens_k bounds validation and compatibility follow-up (#28031, #28259).
  • MatMulBnb4 and ML coefficient SafeInt checks (#27995, #28001).
  • CUDA Gather int32 overflow fix (#28108).
  • GridSample float->int64 cast hardening for NaN/Inf/out-of-range coords (#28302).
• Fixed session logger use-after-free during EP teardown under verbose logging (#28274).

CUDA, Attention, and MLAS

• Filled CUDA opset/operator gaps and extended support:
  • Transpose opset 23 -> 25 (#27740).
  • QuantizeLinear/DequantizeLinear opset 25 (#28046).
  • CUDA TopK INT8/INT16/UINT8 support (#27862).
  • LabelEncoder CUDA support for numeric types (#28045).
• Attention/GQA improvements:
  • Fixed ONNX Attention min-bias alignment crash on SM<80 and masked-batch NaN behavior (#27831).
  • Added FP32 QK accumulation path for unfused GQA attention (#28198).
  • Added CUDART_VERSION reduction compatibility in GQA attention (#28296).
  • Fixed CUDA 13 build error in GQA unfused attention (#28309).
  • PagedAttention fallback for SM<80 fp16 (#28200).
• MLAS updates:
  • FP16 Gelu enablement (#26815).
  • Arm64 BF16 fast-math conv kernels for NCHW/NCHWc paths (#27878).

WebGPU, WebNN, and JavaScript

• WebGPU feature and correctness updates:
  • Added GridSample (#28264).
  • Split-K support for batch size > 1 (#28151).
  • MatMulNBits refactor and batching improvements (#28109, #28197).
  • MHA correctness fix when present outputs are not requested (#28027).
  • Buffer upload overflow fix (#27948).
  • Position ID bounds validation in WebGPU/JS RotaryEmbedding (#28214).
• WebNN change:
  • Renamed pool2d property roundingType -> outputShapeRounding (#28172).
• JavaScript ecosystem maintenance:
  • Multiple dependency bumps.

Plugin EP and EP Ecosystem

• CUDA plugin EP:
  • Graph capture/replay support ported and expanded (#27958, #28002).
  • Sync support for IOBinding (#27919).
  • Profiling API implementation (#28216).
  • Resource accounting integration (#28028).
• WebGPU plugin EP:
  • Pipeline updates and API init error handling fixes (#28121, #28211).
• Other EP updates:
  • CoreML: HardSigmoid and QuickGelu support; Pad reflect support/fixes (#28182, #28184, #28073, #28062).
  • NvTensorRTRTX compatibility and diagnostics updates (#28263, #27577).
  • QNN file-mapping guard improvements (#27871).

Contributors

@tianleiwu, @yuslepukhin, @edgchen1, @vraspar, @hariharans29, @skottmckay, @eserscor, @xadupre, @sanaa-hamel-microsoft, @elwhyjay, @Rishi-Dave, @titaiwangms, @adrianlizarraga, @jatinwadhwa921, @jchen10, @Jiawei-Shao, @maxwbuckley, @preetha-intel, @qjia7, @qti-hungjuiw, @RajeevSekar, @umangb-09, @adrastogi, @akote123, @amd-genmingz, @ankitm3k, @apsonawane, @bachelor-dou, @baijumeswani, @bopeng1234, @chilo-ms, @chwarr, @Craigacp, @dccarmo, @derdeljan-msft, @ericcraw, @fdwr, @fs-eire, @gaugarg-nv, @gblong1, @GopalakrishnanN, @Honry, @intbf, @ishwar-raut1, @Jaswanth51, @javier-intel, @JonathanC-ARM, @julia-thorn, @justinchuby, @jwludzik, @Kevin-Taha, @Kotomi-Du, @MayureshV1, @mdvoretc-intel, @miaobin, @milpuz01, @mingyueliuh, @mklimenk, @n1harika, @prathikr, @psakhamoori, @qti-yuduo, @quic-calvnguy, @RyanMetcalfeInt8, @sfatimar, @sgbihu, @ShirasawaSama, @ssam18, @susbhere, @sushraja-msft, @TejalKhade28, @theHamsta, @TomCrypto, @TsofnatMaman, @velonica0, @vthaniel, @wenqinI, @xhan65, @xhcao

n.b. This changelog is LLM generated. Only the contributor listing has been verified.

ONNX Runtime Release 1.25.1

📢 Announcements & Breaking Changes

ONNX Op Updates

• Enhanced ONNX operator support with new opset versions: Reshape (opset 25), Transpose (opset 24) (#27752)

---

✨ New Features

📊 New ONNX Ops & Model Support

• LinearAttention and CausalConvState operators for Qwen3.5 model support (#27907)
• RotaryEmbedding (RotEMB) and RMSNorm operators added (#27752)
• Linear Attention signature support (#27842)

---

🌐 Web & JavaScript

WebGPU EP

• Qwen3.5 model support on WebGPU execution provider (#27996)
• QMoE 1-token decode path optimization — fused operations to reduce GPU dispatches for improved performance (#27998)

---

🐛 Bug Fixes

Core Runtime Fixes

• Improved filesystem error messages during Linux device discovery for better debugging experience (#27289)
• Fixed missing include for SetRawDataInTensorProto in NVIDIA TensorRT RTX tests (#28065)

---

🙏 Contributors

Thanks to our 7 contributors for this release: @guschmue, @sanaa-hamel-microsoft, @apsonawane, @eserscor, @ishwar-raut1, @qjia7, @theHamsta

Full Changelog: https://github.com/microsoft/onnxruntime/compare/v1.25.0...v1.25.1

📢 Announcements & Breaking Changes

Build & Platform

• C++20 is now required to build ONNX Runtime from source. Minimum toolchains: MSVC 19.29+, GCC 10+, Clang 10+. Users of prebuilt packages are unaffected. (#27178)
• CUDA minimum version raised to 12.0 — CUDA 11.x is no longer supported. Users pinned to CUDA 11.x should stay on ORT 1.24.x or upgrade their CUDA toolkit/driver. (#27570)
• ONNX upgraded to 1.21.0 (#27601)
• sympy is now an optional dependency for Python builds. (#27200)

Execution Provider Changes

• ArmNN EP has been removed. Users should remove any --use_armnn build flags and migrate to the MLAS/KleidiAI-backed CPU EP or QNN EP for Qualcomm hardware. (#27447)

API Version

• ORT_API_VERSION updated to 25. (#27280)

---

🔒 Security Fixes

• Fixed potential integer truncation leading to heap out-of-bounds read/write (#27544)
• Addressed Pad Reflect vulnerability (#27652)
• Security fix for transpose optimizer (#27555)
• Upgraded minimatch 3.1.2 → 3.1.4 for CVE-2026-27904 (#27667)
• Hardened shell command handling for constant strings (#27840)
• Added validation of onnx::TensorProto data size before allocation (#27547)
• Cleaned up external data path validation (#27539)
• Fixed misaligned address reads for tensor attributes from raw data buffers (#27312)
• Fixed CPU Attention overflow issue (#27822)
• Fixed CPU LRN integer overflow issues (#27886)
• Additional input validation hardening:
  • Tile kernel dim overflow (#27566)
  • Out-of-bounds read in cross entropy (#27568)
  • TreeEnsembleClassifier attributes (#27571)
  • AffineGrid (#27572)
  • EmbedLayerNorm position_ids (#27573)
  • RotaryEmbedding position_ids (#27597)
  • RoiAlign batch_indices (#27603)
  • MaxUnpool indices (#27432)
  • QMoECPU swiglu OOB (#27748)
  • SVMClassifier initializer (#27699)
  • Col2Im SafeInt (#27625)

---

✨ New Features

🔌 Execution Provider Plugin API & CUDA Plugin EP

ORT 1.25.0 introduces the CUDA Plugin EP — the first core implementation that enables third-party CUDA-backed EPs to be delivered as dynamically loaded plugins without rebuilding ORT.

• CUDA Plugin EP: Core implementation (#27816)
• CUDA Plugin EP: BFC-style arena and CUDA mempool allocators for stream-aware memory management (#27931)
• Plugin EP Sync API for synchronous execution (#27538)
• Plugin EP event profiling APIs (#27649)
• Plugin EP APIs to retrieve ONNX operator schemas (#27713)
• Annotation-based graph partitioning with resource accounting (#27595, #27972)
• EP API adapter improvements: header-only adapter, OpKernelInfo::GetConfigOptions, LoggingManager::HasDefaultLogger() (#26879, #26919, #27540, #27541, #27587)
• WebGPU EP made compatible with EP API (#26907)

🔧 Core APIs

• Per-session thread pool work callbacks API (#27253)
• enable_profiling in RunOptions (#26846)
• KernelInfo string-array attribute APIs for C and C++ (#27599)
• OrtModel input support for Compile API (#27332)
• Session config to create weightless EPContext models during compilation (#27197)
• Compiled model compatibility APIs in example plugin EP (#27088)
• Model Package support (preview): Initial infrastructure for automatically selecting compiled EPContext model variants from a packaged collection based on EP, device, and hardware constraints. The directory structure is not yet finalized. (#27786)

📊 New ONNX Ops & Opset Coverage

• Attention opset 23 on CUDA with GQA, boolean masks, softcap, and softmax precision (#26466, #27030, #27082, #27428, #27714)
• Attention opset 24 on CUDA, disjoint from contrib op (#27542); nonpad KV seqlen on CPU (#27384)
• TensorScatter-24 for CPU and CUDA (#27389, #27446)
• DeformConv for CPU/CUDA (#27393)
• LpNormalization-22 (#27164)
• CUDA opset gap fills:
  • Control flow & misc: Flatten, Identity, If, Loop, Scan, ConstantOfShape, Size (opset 21/23) (#27728)
  • Pooling: GlobalAveragePool/GlobalMaxPool (→22) (#27733)
  • Shape ops: Shape (→25), Squeeze/Unsqueeze (→25) (#27734, #27739)
  • TopK (→24, BF16) (#27735), GRU (→22) (#27738)
  • Pad (→25, wrap mode) (#27774), Resize v19 (#27415), RoiAlign v16/v22 (#27646)

---

🖥️ Execution Provider Updates

NVIDIA CUDA EP

• GQA with XQA and quantized KV cache, including FP8 (E4M3) KV cache support (#27246, #27321)
• CUDA graph capture compatibility for LLM ops and pre-compiled paths (#27484, #27477)
• Volumetric (3-D) GridSample support (#27201)
• Optimized 3D nearest resize kernel for 5D tensors (#27578)
• Optional router_weights input to QMoE (#27687)

NVIDIA TensorRT RTX EP

• D3D12 external resource import support (#26948)

Qualcomm QNN EP

• Disabled file mapping for embedded cache (#27627)
• Fixed use-after-free of logger object (#27804)
• Fixed wheel build issues on WSL and Linux SDK version propagation (#27730, #27800)

Other EPs

• VitisAI EP: Added PE version info to provider DLL (#27626)
• DML EP: Fixed overflow in DmlGraphFusionHelper::ProcessInputData (#27815), fixed new-delete mismatch in QuantizeLinear (#27823)

---

🌐 Web & JavaScript

WebGPU EP — Performance

• Gemm/MatMul optimization using subgroup features (#26433)
• MatMulNBits: 2-bit zero-point support (#27285, #27325), higher K-parallelism (#27834), DP4A SmallM tiling (#27910)
• Flash Attention: head_sink support (#27410), configurable multi rotary cache concat offset (#27434)
• Optimized 4D Transpose (#26942), string stream optimization (#27223)

WebGPU EP — New Op Support

• Added TopK (#27560), Softplus (#27457), Identity (#27067)
• Added Conv3D support (#27917), LpNorm support (#27876)
• int64/bool support for Range, Expand, Flatten, Gather, Unsqueeze (#26673, #27478, #27561)
• DequantizeLinear fixes (#27706), Einsum 5D tensor fixes (#27779)

WebGPU EP — Stability

• Fixed device destroyed on session release breaking recreation (#27634)
• Fixed static destruction crash on exit (#27470, #27569)
• Backward compat: Legacy WebGPU/WebNN memory info names are now accepted again (#27637)
• Deterministic Split-K handling (#27086), buffer segment alignment fix (#27853)
• Binary size reduction for WebAssembly builds (#27370, #27371)

WebNN EP

• Broader GQA support and improved MultiHeadAttention (#27234, #27494)
• Added DepthToSpace support (#27508)

Node.js & React Native

• Fixed float16 tensor support in Node.js and React Native (#27327, #27549)
• Added 16KB page size alignment for Android (required for Android 15+) (#27523)

---

🧠 CPU & Core Optimizations

MLAS / KleidiAI / Quantization

• KleidiAI BF16 SME2 kernel integration (#26773), asymmetric 4-bit MatMulNBits on ARM64 (#27751)
• Fused Silu and Gelu kernels for AVX512 (#27690)
• Depthwise conv kernel for NCHW on AVX512 (#27874)
• ARM64 NCHWc NEON asm kernels (#27099, #27788), BF16 KAI SBGemm on NCHWc ARM (#27703)
• POWER10 Sgemm PackA optimization (#27575)
• Improved pre-packing for 2-bit LUT kernels (#27131)
• Backend kernel selector config in MLAS, allowing users to opt out of KleidiAI kernels on ARM platforms (#27136)

DQ→MatMulNBits Fusion

Extended to cover significantly more quantized LLM inference scenarios on CPU:

• 2-bit and 8-bit weights with Cast(fp16→fp32) patterns (#27614)
• FP16 models on CPU EP (#27640), fp16 8-bit on ARM64 (#27692)
• Gemm + per-tensor/per-channel quantization (#27769)
• FP16 quantized weight compatibility: models with HQNBIT quantized weights now route through the FP32 MLAS path for broader CPU compatibility (#27820)

Model Optimizer & Fusions

• Qwen3 model type support and RotaryEmbedding fusion for Qwen3 RoPE patterns (#27556, #27590)
• MobileClip attention fusion for both attention block patterns (#27883)
• Nemotron speech conformer encoder MHA fusion (#27764)
• Fixed GPT-2 no-past attention fusion for transformers ≥ 4.27 (#27449)
• Fixed BART attention fusion for SDPA pattern from transformers ≥ 4.49 (#27458)
• Pre-layer normalization support in attention fusion (#27418)
• SkipLayerNorm fusion with bias Add (#27765), broadcasting skip shapes (#27489)
• SpaceToDepth fusion pattern (#27747)
• NCHWc transformer: more patterns and ONNX-domain Gelu/HardSigmoid activations (#27691, #27821)
• Optimized qMoE code path for single-token execution (#27383)
• ONNX Attention KV cache optimization with ConcatNewToPast (#27613)

---

🔌 Language Bindings

Python

• Exposed OrtDeviceVendorId enum for vendor-aware OrtDevice aliases (#27594)
• Added bindings for GetCompatibilityInfoFromModel / GetCompatibilityInfoFromModelBytes (#27565)
• Fixed OrtValue.from_dlpack rejecting zero-size tensors as non-contiguous (#27451)

C#

• Added bindings for GetCompatibilityInfoFromModel / GetCompatibilityInfoFromModelBytes (#27565)

Java

• Avoid provider resource extraction when library already exists in onnxruntime.native.path (#27668)

---

🐛 Bug Fixes

Critical Fixes

• Fixed CPU Attention overflow issue (#27822)
• Fixed CPU LRN integer overflow issues (#27886)
• Fixed incorrect pad indices in AveragePool count_include_pad computation — silent correctness issue (#27375)
• Fixed integer division/modulo by zero in CPU EP Div and Mod operators (#27693, #27833)
• Fixed non-ASCII Unicode model path crash (#27724)
• Fixed arithmetic overflow in Det operator (#27070)
• Fixed narrow-to-wide string conversion bugs in DLL load error reporting (#27777)

Operator & Graph Fixes

• Fixed 3D attention mask broadcasting in MHA (#27464)
• Fixed GQA shape inference for present outputs (#27250)
• Fixed Einsum bugs for reduction and empty input cases (#27225, #27226)
• Prevented cross-EP Cast fusion in RemoveDuplicateCastTransformer (#27363)
• Fixed ConvTranspose bias input validation on CPU/CUDA (#27209)
• Fixed Cast node naming collisions in float16 conversion (#27469)
• Fixed concat/slice elimination and unsqueeze elimination against optional attrs and invalid models (#27638)
• Improved EPContext error message when node is not assigned to an EP (#27474)

EP-Specific Fixes

• Fixed MiGraphX EP double allocation (#27551)
• Fixed MLAS qgemm dispatch and kernel regressions in quantized conv tests (#27671)
• Fixed run-level profiling for subgraph operators (#27870)
• Fixed --build_wasm_static_lib implicitly enabling --build_wasm (#27342)

---

🙏 Contributors

Thanks to our 72 contributors for this release!

@tianleiwu, @fs-eire, @edgchen1, @titaiwangms, @hariharans29, @eserscor, @Rishi-Dave, @guschmue, @adrianlizarraga, @jambayk, @qjia7, @skottmckay, @adrastogi, @sanaa-hamel-microsoft, @yuslepukhin, @ingyukoh, @Jiawei-Shao, @vraspar, @xhcao, @chilo-ms, @Honry, @JonathanC-ARM, @kunal-vaishnavi, @ShirasawaSama, @chaya2350, @derdeljan-msft, @gedoensmax, @HectorSVC, @milpuz01, @quic-calvnguy, @xenova, @akholodnamdcom, @AlekseiNikiforovIBM, @amd-genmingz, @ashrit-ms, @bachelor-dou, @BODAPATIMAHESH, @Colm-in-Arm, @daijh, @dodokw, @fanchenkong1, @ivarusic-amd, @JanSellner, @jchen10, @jiafatom, @jnagi-intel, @johannes-rehm-snkeos, @justinchuby, @keshavv27, @Kevin-Taha, @kevinlam92, @kpkbandi, @Laan33, @melkap01-Arm, @miaobin, @n-v-k, @nico-martin, @patryk-kaiser-ARM, @praneshgo, @prathikr, @qc-tbhardwa, @sagarbhure-msft, @sdotpeng, @the0cp, @TsofnatMaman, @umangb-09, @walidbr, @wenqinI, @xadupre, @xhan65, @xiaofeihan1

---

Full Changelog: v1.24.4...v1.25.0

This is a patch release for ONNX Runtime 1.24, containing bug fixes and execution provider updates.

Bug Fixes

• Core: Added PCI bus fallback for Linux GPU device discovery in containerized environments (e.g., AKS/Kubernetes) where nvidia-drm is not loaded but GPU PCI devices are still exposed via sysfs. (#27591)
• Plugin EP: Fixed null pointer dereference when iterating output spans in GetOutputIndex. (#27644)
• Plugin EP: Fixed bug that incorrectly assigned duplicate MetaDef IDs to fused nodes in different GraphViews (e.g., then/else branches of an If node), causing session creation to fail with a conflicting kernel error. (#27666)

Execution Provider Updates

• QNN EP: Enabled offline x64 compilation with memhandle IO type by deferring rpcmem library loading to inference time. (#27479)
• QNN EP: Reverted QNN SDK logging verbosity changes that caused segmentation faults on backend destruction. (#27650)

Build and Infrastructure

• Python: Updated python_requires from >=3.10 to >=3.11 to reflect dropped Python 3.10 support. (#27354)
• Build: Replaced __builtin_ia32_tpause with the compiler-portable _tpause intrinsic to fix cross-compiler portability issues between GCC and LLVM. (#27607)

Full Changelog: v1.24.3...v1.24.4

Contributors

@derdeljan-msft, @adrianlizarraga, @apwojcik, @baijumeswani, @edgchen1, @mocknen, @tianleiwu, @XXXXRT666

This is a patch release for ONNX Runtime 1.24, containing bug fixes, security improvements, performance enhancements, and execution provider updates.

Security Fixes

• Core: Fixed GatherCopyData integer truncation leading to heap out-of-bounds read/write. (#27444)
• Core: Fixed RoiAlign heap out-of-bounds read via unchecked batch_indices. (#27543)
• Core: Prevent heap OOB from maliciously crafted Lora Adapters. (#27518)
• Core: Fixed out-of-bounds access for Resize operation. (#27419)

Bug Fixes

• Core: Fixed GatherND division by zero when batch dimensions mismatch. (#27090)
• Core: Fixed validation for external data paths for models loaded from bytes. (#27430)
• Core: Fixed SkipLayerNorm fusion incorrectly applied when gamma/beta are not 1D. (#27459)
• Core: Fixed double-free in TRT EP custom op domain Release functions. (#27471)
• Core: Fixed QMoE CPU Operator. (#27360)
• Core: Fixed MatmulNBits prepacking scales. (#27412)
• Python: Fixed refcount bug in map input conversion that caused shutdown segfault. (#27413)
• NuGet: Fixed DllImportResolver. (#27397)
• NuGet: Added OrtEnv.DisableDllImportResolver to prevent fatal error on resolver conflict. (#27535)

Performance Improvements

• Core: QMoE CPU performance update (up to 4x on 4-bit). (#27364)
• Core: Fixed O(n²) model load time for TreeEnsemble with categorical feature chains. (#27391)

Execution Provider Updates

• NvTensorRtRtx EP:
  • Avoid repetitive creation of fp4/fp8 native-custom-op domains. (#27192)
  • Added missing override specifiers to suppress warnings. (#27288)
  • DQ→MatMulNBits fusion transformer. (#27466)
• WebGPU:
  • Used embedded WASM module in Blob URL workers when wasmBinary is provided. (#27318)
  • Fixed usage of wasmBinary together with a blob URL for .mjs. (#27411)
  • Removed the unhelpful "Unknown CPU vendor" warning. (#27399)
  • Allows new memory info name for WebGPU. (#27475)
• MLAS:
  • Added DynamicQGemm function pointers and ukernel interface. (#27403)
  • Fixed error where bytes is not assigned for dynamic qgemm pack b size. (#27421)
• VitisAI EP: Removed s_kernel_registry_vitisaiep.reset() in deinitialize_vitisai_ep(). (#27295)
• Plugin EPs: Added "library_path" metadata entry to OrtEpDevice instances for plugin and provider bridge EPs. (#27522)

Build and Infrastructure

• Pipelines:
  • Build Windows ARM64X binaries as part of packaging pipeline. (#27316)
  • Moved JAR testing pipelines to canonical pipeline template. (#27480)
• Python: Enabled Python 3.14 CI and upgraded dependencies. (#27401)
• Build: Suppressed spurious Array Out of Bounds warnings produced by GCC 14.2 compiler on Linux builds. (#27454)
• Build: Fixed -Warray-bounds build error in MLAS on clang 17+. (#27499)
• Telemetry: Added/Updated telemetry events. (#27356)
• Config: Increased kMaxValueLength to 8192. (#27521)

Full Changelog: v1.24.2...v1.24.3

Contributors

@tianleiwu, @fs-eire, @adrianlizarraga, @yuslepukhin, @0-don, @anujj, @chaya2350, @chilo-ms, @dabhattimsft, @edgchen1, @eserscor, @hariharans29, @JonathanC-ARM, @lukas-folle-snkeos, @patryk-kaiser-ARM, @praneshgo, @skottmckay, @theHamsta, @vektah, @vishalpandya1990, @vthaniel, @xieofxie, @zz002